Paper deep dive
Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Amirmohammad Farzaneh, Osvaldo Simeone
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/4/2026, 11:16:07 AM
Summary
The paper introduces Think Short, Defer Smart (TSDS), a framework for edge LLM agents that jointly calibrates thought convergence and uncertainty-aware deferral to optimize reasoning budget and reliability. TSDS uses a lightweight convergence probe to halt on-device reasoning when actions stabilize and a perplexity-based rule to defer uncertain actions to a cloud model. Calibrated via a multi-objective Learn-Then-Test procedure, TSDS reduces per-episode thinking compute by 43%-73% compared to baselines while maintaining certified reward and cloud-call rate guarantees across benchmarks like GSM8K, HotpotQA, MBPP, and household robot planning.
Entities (10)
Relation Signals (9)
TSDS → calibratedby → Learn-Then-Test
confidence 95% · Both mechanisms are jointly calibrated... via a multi-objective Learn-Then-Test (LTT) procedure
TSDS → evaluatedon → HotPotQA
confidence 95% · We evaluate TSDS on four ReAct benchmarks spanning... multi-hop question answering (HotpotQA)
TSDS → evaluatedon → MBPP
confidence 95% · We evaluate TSDS on four ReAct benchmarks spanning... code generation (MBPP)
TSDS → evaluatedon → GSM8K
confidence 95% · We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K)
TSDS → uses → Convergence Probe
confidence 95% · TSDS... integrates a lightweight convergence probe... with a perplexity-based deferral rule
TSDS → uses → Perplexity
confidence 95% · TSDS... integrates... a perplexity-based deferral rule
TSDS → reducescomputeby → 43%-73%
confidence 90% · TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines
DeepSeek-R1-Distill-Qwen-7B → usedas → Edge Model
confidence 90% · The edge model DeepSeek-R1-Distill-Qwen-7B... are adopted throughout.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.
Tags
Links
- Source: https://arxiv.org/abs/2607.26865v1
- Canonical: https://arxiv.org/abs/2607.26865v1
Trouble viewing inline? Open PDF directly →
Full Text
67,813 characters extracted from source content.
Expand or collapse full text
Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents Amirmohammad Farzaneh Osvaldo Simeone Institute for Intelligent Networked Systems (INSI) Northeastern University London London, UK a.farzaneh,o.simeone@nulondon.ac.uk Abstract LLM agents following the ReAct paradigm (Yao et al., 2023) are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%43\%–73%73\% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees. 1 Introduction (a) (b) (c) Figure 1: TSDS pipeline as a high-level illustration (a) and detailed pipeline (b): An edge agent πedgeπ^edge interacts with the environment across a number of time steps t=1,2,…t=1,2,… At each step t, an action is taken after a reasoning phase, obtaining an observation from the environment. In the proposed TSDS, a convergence probe φ monitors the hidden reasoning state and halts generation once the intended action has stabilized as gauged via a threshold λL _L. Then, an uncertainty measure on the chosen action, such as the PPL, is calculated: if it is higher than a threshold λD _D, the decision on the action is deferred to a larger cloud model πcloudπ^cloud. The chosen action, namely AtedgeA_t^edge or AtcloudA_t^cloud, is executed in the environment, and an observation OtO_t is collected. A final reward r is produced at the end of an episode. (c) Illustration of the interplay between thought convergence and uncertainty-based deferral rule for a setting in which the agent selects a suboptimal action: The probe score φ (top) crosses the thought convergence threshold λL _L at τ∗τ^* tokens, showing that the edge agent’s action has already converged. At this point, the uncertainty of the chosen action (bottom) is larger than the deferral threshold λD _D, correctly triggering deferral to the cloud agent. If reasoning were allowed to continue past token τ∗τ^*, the edge agent would grow increasingly overconfident, suppressing the deferral mechanism and letting a suboptimal action through unchallenged (see Appendix H for empirical evidence). Autonomous agents powered by large language models (LLMs) are increasingly deployed in resource-constrained settings, from robotic manipulation and autonomous navigation (Ahn et al., 2022; Shridhar et al., 2021) to edge network management and industrial process control (Xu and others, 2024; Chevalier-Boisvert et al., 2023). In these settings, agents must operate on local hardware with limited compute, memory, and power, while remaining reliable. ReAct (Yao et al., 2023) agents, which alternate free-text thoughts with executable actions, serve as the backbone of such agentic systems (Shinn et al., 2023; Zhou et al., 2024; Sun et al., 2023). Thought generation consumes the majority of inference compute per step (Wu et al., 2025; Guo et al., 2025). However, even a well-reasoned action from an edge-side model can be wrong, and errors compound over the episode, making deferral to a cloud-based larger model necessary (Piatrashyn and others, 2026). In settings where cloud queries carry latency and bandwidth costs, an agent must therefore maintain reliability, while minimizing wasted reasoning compute and cloud-based calls. As outlined in Fig. 1, our work addresses this problem by building on the following observations. First, in reasoning models, thinking may continue well beyond the point where a decision on intended action has converged. Esakkiraja and others (2026) show that the committed action is encoded by hidden representations well before the end of the reasoning trace, implying that many generated tokens are pure post-convergence waste. To make matters worse, as exemplified in Fig. 1, excessive thinking may lead the model to become increasingly overconfident in the selection of a suboptimal action. This complicates cloud deferral mechanisms, which are often based on the edge model’s internal uncertainty signals (Shridhar et al., 2021; Chevalier-Boisvert et al., 2023). Overall, a joint calibration of reasoning and deferral mechanisms has the potential to reduce resource consumption in terms of number of edge tokens, while also enhancing the efficiency of cloud deferral. Our contribution: We propose Think Short, Defer Smart (TSDS), a novel reasoning-based agentic framework that integrates and jointly calibrates the two synergetic mechanisms illustrated in Fig. 1: 1. Thought calibration: At each interaction step with the environment, following the ReAct framework (Yao et al., 2023), the edge agent reasons to determine an optimized next action. In order to avoid overthinking, a lightweight convergence probe monitors the hidden representations of the edge agent’s model during reasoning. The probe is trained to detect the convergence of the reasoning trace towards a given decision (Wu et al., 2025). 2. Uncertainty-Aware Deferral: Once reasoning has converged and the edge action has been extracted, an uncertainty score over the resulting action token distribution is computed, and deferral to a cloud agent is triggered selectively by comparing the uncertainty score against a calibrated threshold λD _D (Piatrashyn and others, 2026; Chen et al., 2023). Deciding when to stop thinking and when to defer entails a trade-off between the agent’s performance in the environment, which is measured by the task-specific reward signal r, and resource consumption, which encompasses number of thinking steps, deferral rate, and number of environment interactions. Furthermore, as illustrated in Fig. 1(c), thought calibration and cloud deferral are strongly intertwined, as excessive reasoning can produce over-confident uncertainty signals, causing the edge agent to select suboptimal actions, instead of correctly deferring to the cloud agent. Conversely, early thought truncation may produce an uncertainty signal higher than warranted by the actual confidence of the edge agent, unduly increasing deferral frequency. Joint calibration of reasoning and cloud deferral is carried out via Learn-Then-Test (LTT) (Angelopoulos et al., 2021; Farzaneh and Simeone, 2026), which formulates hyperparameter selection as a multiple hypothesis testing problem, providing distribution-free finite-sample guarantees. 2 Related Work LLM agents: ReAct (Yao et al., 2023) established a by-now canonical pattern of alternating free-text reasoning with tool calls. Subsequent work added verbal self-reflection (Shinn et al., 2023), adaptive planning (Sun et al., 2023), and lifelong skill libraries (Wang et al., 2023). A fundamental limitation shared by all these systems is that each thought trace runs until a positional stop string, and hence inference cost scales with the trace length budget rather than with the actual difficulty of the reasoning step. Uncertainty quantification and deferral: Information-theoretic scores such as perplexity (PPL) provide uncertainty estimates that correlate with actual model errors (Malinin and Gales, 2021; Fomicheva et al., 2020). However, translating these raw scores into reliable deployment-level guarantees requires explicit calibration (Angelopoulos et al., 2021; Frankel and others, 2025). Model cascades (Chen et al., 2023; Yue et al., 2023; Ong et al., 2024; Hou et al., 2026) route single-turn queries across models of different capacities, focusing on independent requests without multi-step dependencies. ReDAct (Piatrashyn and others, 2026) applies PPL-guided deferral inside an agentic loop, but the deferral threshold is chosen heuristically. Test-time compute and thought calibration: Thought calibration (Wu et al., 2025) trains a convergence probe on offline single-turn trajectories and uses LTT (Angelopoulos et al., 2021) to certify a stopping threshold. Early-exit methods (Wang and others, 2025) and conformal prediction wrappers (Frankel and others, 2025; Chen et al., 2024) also aim at an adaptive use of inference resources. 3 Problem Definition Edge agent and environment: As shown in Fig. 1, we consider an edge agent πedgeπ^edge that interacts with a physical or virtual environment over discrete time steps t=1,…,Tt=1,…,T to carry out a task described by a natural language prompt X. At each step t, the agent maintains a history Ht=(X,(O1,T1,A1),…,(Ot−1,Tt−1,At−1),Ot),H_t= (X,\;(O_1,T_1,A_1),\;…,\;(O_t-1,T_t-1,A_t-1),\;O_t ), where (Tt,At)(T_t,A_t) are the thought trace and action at step t, and OtO_t is the observation of the environment upon taking action At−1A_t-1. Actions may include tool calls (e.g., Search, Lookup), code execution, free-form text, or motor commands in a physical environment (Ahn et al., 2022; Shridhar et al., 2021; Chevalier-Boisvert et al., 2023). At episode termination, i.e., after time t=Tt=T, the environment issues a scalar reward r(X,A1,…,AT)∈[0,1]r(X,A_1,…,A_T)∈[0,1] that depends on the task prompt X, which specifies both the goal and the evaluation criterion, and on the trajectory of actions taken. Examples include exact-match for question answering (Yang et al., 2018; Cobbe et al., 2021) and binary pass/fail for goal-oriented tasks (Austin et al., 2021; Shridhar et al., 2021). At each time step t, given history HtH_t, the edge agent generates a thought trace Tt=(τt,1,…,τt,Lt)T_t=( _t,1,…, _t,L_t). Each token τt,i∼πedge(⋅∣Ht,τt,1,…,τt,i−1) _t,i π^edge(· H_t, _t,1,…, _t,i-1) is drawn autoregressively using the edge agent’s model πedgeπ^edge. For future reference, we denote as ht,i(ℓ)∈ℝdh_t,i^( ) ^d the latent activation of model πedgeπ^edge for token i at layer ℓ . After LtL_t reasoning steps, the edge agent may take an action AtedgeA_t^edge or defer to the cloud. The action AtedgeA_t^edge is produced by model πedgeπ^edge as Atedge∼πedge(⋅∣Ht,τt,1,…,τt,i−1)A_t^edge π^edge(· H_t, _t,1,…, _t,i-1). Cloud agent and cascading: A more capable cloud model πcloudπ^cloud is available to the edge agent as a fallback. If a cloud deferral decision is made at time t, the full history HtH_t is transferred to the cloud, and the model πcloudπ^cloud generates an alternative action Atcloud∼πcloud(⋅∣Ht)A_t^cloud π^cloud(· H_t), replacing the edge agent’s action AtedgeA_t^edge for that step. We write as Dt∈0,1D_t∈\0,1\ the indicator for whether the cloud model is called (Dt=1D_t=1) or not (Dt=0D_t=0). Thinking convergence and deferral: The edge agent’s operation is governed by two threshold-parameterized mechanisms. A thinking convergence rule decides at each thought position i whether to halt reasoning and extract an action AtedgeA_t^edge and an uncertainty score UtU_t. To this end, the rule applies a threshold λL _L to a convergence signal Ci,tC_i,t extracted from the latent activations ht,i(ℓ)h_t,i^( ) of the edge model as: stop if Ct,i>λLC_t,i> _L and continue otherwise. A deferral rule decides whether to execute the edge action or hand the step to πcloudπ^cloud using a threshold λD _D as: defer if Ut>λDU_t> _D and not defer otherwise. Accordingly, the executed action AtA_t is At=AtedgeifUt≤λDAtcloudifUt>λD,A_t= casesA_t^edge \;U_t≤ _D\\ A_t^cloud \;U_t> _D, cases (1) with action AtedgeA_t^edge produced at thinking step Lt=minargminii:Ct,i>λL,Lmax,L_t= \ _i\i:C_t,i> _L\,L_max\, (2) where LmaxL_max is the maximum allowed number of thinking steps. Design objectives: We define the expected episode reward R()=[r()]R( λ)=E[r( λ)], where the expectation is over task prompts and environment stochasticity, and we have highlighted the dependence on the vector λ=(λL,λD)λ=( _L, _D) of hyperparameters. The deferral cost, i.e., the expected number of cloud calls per episode, and the thinking cost, i.e., number of reasoning tokens per episode, are given by CD()=[∑t=1TDt]andCL(λL)=[∑t=1TLt],C_D( λ)=E\! [ _t=1^TD_t ]\;\;and\;\;C_L( _L)=E\! [ _t=1^TL_t ], (3) respectively. Another important metric is the episode-length cost CS()=[T]C_S( λ)=E[T], where T is the (random) number of agent steps until episode termination. The overall design objective is to minimize thinking cost CL(λL)C_L( _L), while keeping average reward R(λ)R(λ) above a floor RminR and cloud deferral rate CD(λ)C_D(λ) within a budget CDmaxC_D , i.e., minCL(λL)subject toR()≥RminandCD()≤CDmax. _ λ\;C_L( _L) to R( λ)≥ R \;\;and\;\;C_D( λ)≤ C_D . (4) 4 Think Short, Defer Smart This section introduces TSDS, a framework that addresses problem (4) by proposing concrete instantiations of the thinking convergence rule (Sec. 4.1) and the deferral rule (Sec. 4.2), as well as by incorporating the calibration of their joint threshold vector λ via multi-objective LTT (Sec. 4.3). 4.1 Convergence Probe and Thought Calibration The goal of the thinking convergence rule is to detect, from the edge agent πedgeπ^edge’s internal representations, the moment at which its reasoning has committed to a stable action. We follow the general approach of (Wu et al., 2025) of training a lightweight probe on offline trajectories to certify a stopping threshold, tailoring the methodology to agents that reason to act in a multi-step interactive loop. As illustrated in Fig. 1, define the convergence signal as Ct,i=φθ(ht,i(ℓ))∈ℝC_t,i= _θ(h_t,i^( )) , thus mapping the hidden state ht,i(ℓ)h_t,i^( ) at reasoning step i and layer ℓ to the scalar convergence score Ct,iC_t,i. The layer ℓ is treated as a hyperparameter. The probe φθ _θ is trained offline on a dataset tr=(ht,i(ℓ),(n),yt,i(n))n=1NtrD_tr=\(h_t,i^( ),(n),\,y_t,i^(n))\_n=1^N_tr, where index n=1,…,Ntrn=1,…,N_tr runs over training episodes. The dataset trD_tr is collected by running the model πedgeπ^edge with full thought generation (no early stopping, no deferral) and by recording for each position i in a set ℐI the latent state ht,i(ℓ)h_t,i^( ). Position i is labelled positive, i.e., yt,i(n)=1y_t,i^(n)=1, if the early-stopped action at every subsequent probe position i′≥i ≥ i within the same step agrees with the full-thought action, i.e., yt,i(n)=∏i′∈ℐ:i′≥i[Ai′,t(n)=At(n)]∈0,1,y_t,i^(n)\;=\; _i :i ≥ i1 [A_i ,t^(n)=A_t^(n) ]∈\0,1\, (5) where At(n)A_t^(n) is the full-thought reference action and Ai,t(n)A_i,t^(n) is the early-stopped action obtained by truncating the trace at position i and injecting token </think>. 4.2 Cloud Deferral Once the edge action AtedgeA_t^edge is extracted, the uncertainty score UtU_t is obtained as the PPL of the edge model for action AtedgeA_t^edge, following (Piatrashyn and others, 2026). Alternative scores are defined in Appendix B and can be substituted without modifying the calibration procedure. 4.3 Multi-Objective Calibration In this section, we address the multi-objective optimization problem (4) via LTT (Angelopoulos et al., 2021; Farzaneh and Simeone, 2026). For each candidate reasoning threshold λL _L, we run the TSDS system with thought calibration only, obtaining the held-out calibration dataset as cal(λL)=(r(n)(λL),Ut(n)(λL)t=1T(n))n=1Ncal,D_cal( _L)= \ (r^(n)( _L),\;\U_t^(n)( _L)\_t=1^T^(n) ) \_n=1^N_cal, (6) where r(n)(λL)r^(n)( _L) is the terminal reward of episode n, and Ut(n)(λL)U_t^(n)( _L) is the uncertainty score at step t. Given the scores Ut(n)(λL)U_t^(n)( _L), the per-step deferral indicator Dt(n)(λD)=[Ut(n)(λL)>λD]D_t^(n)( _D)=1[U_t^(n)( _L)> _D] can be evaluated post-hoc for any candidate threshold λD _D. Following LTT, we construct a finite grid Λ=(λL(i),λD(j)) =\( _L^(i), _D^(j))\ of |Λ|| | candidate pairs and follow the LTT procedure summarized in Appendix C. The procedure returns a hyperparameter vector λ^∈Λ λ∈ with the following guarantee. Proposition 1 (Joint risk control guarantee (Angelopoulos et al., 2021)). Assume that calibration data calD_cal is drawn i.i.d. from the same distribution underlying the generation of the test episode. Then, the hyperparameter λ λ returned by LTT satisfies the constraints in problem (4) with probability larger than 1−δ1-δ, i.e., P[R()≥Rmin and CD()≤CDmax]≥1−δ,P\! [R( λ)≥ R \; and \;C_D( λ)≤ C_D ]≥ 1-δ, (7) where δ is a user-defined probability. Algorithm 1 Think Short, Defer Smart (step t) 1:Certified pair =(λL,λD)∈Λ λ=( _L, _D)∈ ; probe φθ _θ; πedgeπ^edge; πcloudπ^cloud; history HtH_t. 2:Generate first token τt,1 _t,1 with πedgeπ^edge; i←1i← 1 3:while i<Lmaxi<L_ do 4: Retrieve hidden state ht,ih_t,i 5: if φθ(ht,i)≥λL _θ(h_t,i)≥ _L then break ⊳ Thought converged 6: else generate next token τt,i+1 _t,i+1; i←i+1i← i+1 7: end if 8:end while 9:Inject </think>; generate AtedgeA_t^edge ⊳ Action extraction 10:ut←U(Atedge∣Ht,τt,1:i,</think>)u_t← U(A_t^edge H_t, _t,1:i, </think>) 11:if ut≤λDu_t≤ _D then 12: At←AtedgeA_t← A_t^edge ⊳ Confident: accept 13:else 14: Atcloud∼πcloud(⋅∣Ht)A_t^cloud π^cloud(· H_t) ⊳ Uncertain: defer 15: At←AtcloudA_t← A_t^cloud 16:end if 17:Execute AtA_t; observe Ot+1O_t+1; update Ht+1H_t+1 5 Experiments We evaluate TSDS on three benchmarks: code generation (MBPP, Sec. 5.2), multi-step question answering (HotpotQA, Sec. 5.3), and simulated household-robot planning (Sec. 5.4). An additional experiment on single-step arithmetic reasoning (GSM8K) can be found in Appendix G.2. The edge model DeepSeek-R1-Distill-Qwen-7B and the Exponential Moving Average (EMA) convergence probe are adopted throughout. All experiments implement LTT with one-sided binomial p-values with Bonferroni correction at level δ=0.10δ=0.10 (see Appendix C). The target deferral rate is CDmax=0.70C_D =0.70. The candidate grid Λ for LTT is formed by considering a uniform sweep of the unit interval with 55 levels for threshold λL _L and 44 uniformly spaced quantiles of the per-episode maximum uncertainty on dataset trD_tr for threshold λD _D. Benchmark-specific settings and parameters are given in each subsection. All reported metrics are means over 5050 independent calibration–test splits. 5.1 Baselines We compare TSDS against the following policies, which isolate the contribution of different components. • E-ReAct (Edge ReAct): As in the original ReAct baseline (Yao et al., 2023), the model πedgeπ^edge runs with full thought traces (λL=∞ _L=∞), and there is no deferral (λD=∞ _D=∞). • Cloud ReAct: The cloud model πcloudπ^cloud is used for every step, providing a performance ceiling at maximum cloud deferral cost. • E-ReAct-TC (Edge ReAct with Thought Calibration): Thought calibration is active, but deferral is disabled (λD=+∞ _D=+∞). This isolates the contribution of thought truncation alone, without the safety net of the deferral mechanism. • ReDAct (Piatrashyn and others, 2026): The edge model πedgeπ^edge generates full thought traces (λL=∞ _L=∞), deferring whenever the PPL exceeds a fixed threshold λ^D λ_D, set to the median PPL on trD_tr. • ReDAct-CD (ReDAct with Calibrated Deferral): The edge model πedgeπ^edge generates full thought traces (λL=0 _L=0), setting Lt=LmaxL_t=L_ in (2), and the deferral threshold λD _D is selected via single-objective LTT on dataset calD_cal to satisfy the reward constraint R()≥RminR( λ)≥ R with probability 1−δ1-δ. This isolates the contribution of calibrated deferral without thought calibration. 5.2 Code Generation: MBPP We evaluate TSDS on MBPP (Austin et al., 2021), a benchmark of mostly basic Python programming problems. Each problem specifies a Python function in natural language with unit tests, and reward is 11 if all unit tests pass and 0 otherwise. Actions are complete Python functions, with a mean of 11661166 thought tokens per problem. This is a single-step setting, i.e., T=1T=1. Setup: The cloud model is DeepSeek-R1-Distill-Qwen-32B, and the thought caps for both the edge and cloud models is set to Lmax=2048L_ =2048. The probe φθ _θ is trained at layer ℓ=5 =5 on 100100 MBPP train-split episodes. We evaluate on 257257 test problems with 5050 random 60/4060/40 splits. The reasoning-threshold grid is λL∈0.1,0.2,…,0.9 _L∈\0.1,0.2,…,0.9\, and the deferral grid λD∈0.084,0.092,0.106,0.117,0.141 _D∈\0.084,0.092,0.106,0.117,0.141\ [nat] is drawn from PPL percentiles on trD_tr, giving |Λ|=45| |=45 candidate pairs. The minimum reward floor is Rmin=0.69R =0.69. Figure 2: Test results for MBPP, shown for five policies and averaged over 5050 random 60/4060/40 calibration–test splits (ntest=103n_test=103 per split). Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin=0.69R =0.69. Center: Average deferral rate to the cloud model. The dashed line marks the maximum allowed deferral fraction CDmax=0.70C_D =0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies. Fig. 2 reports average reward, deferral rate, and thinking cost for all policies, averaged over 5050 calibration–test splits. E-ReAct runs full thought traces without deferral, establishing the uncalibrated edge baseline. Cloud ReAct achieves higher reward but routes every problem to the cloud model, exhausting the deferral budget. E-ReAct-TC is not able to certify any hyperparameters to guarantee reward above the floor RminR due to not having access to the cloud model. ReDAct-CD improves reward through selective cloud escalation, and satisfies the cloud deferral budget on average, but at the cost of requiring full thought traces for the edge agent. TSDS clears the reward floor and respects the deferral budget while using 64%64\% fewer thought tokens than E-ReAct and ReDAct-CD and calling the cloud model on less than half the rate of ReDAct-CD. 5.3 Multi-Step Setting: HotpotQA We evaluate TSDS on HotpotQA (Yang et al., 2018), a multi-hop QA benchmark where the agent takes up to Tmax=7T_ =7 steps, with action space including the tools [⋅] Search[·], [⋅] Lookup[·], and [⋅] Finish[·], to answer questions requiring two supporting passages. Reward is a binary terminal exact match indicator. Figure 3: Reward vs. thinking cost on HotpotQA across five λL _L grid values. The blue dashed curve shows thought calibration only, and the solid green curve shows TSDS at the certified deferral threshold λD=0.318 _D=0.318. All values are means over the full pool of 250250 validation episodes. Setup: The cloud model is DeepSeek-R1-Distill-Qwen-14B. Edge and cloud thought caps are set to Lmax=384L_ =384 and 10241024, respectively. The probe φθ _θ is trained at layer ℓ=5 =5 on 5050 train-split episodes. We evaluate on 250250 validation problems with 5050 random 75/2575/25 splits. The reasoning-threshold grid is λL∈0.1,0.3,0.5,0.7,0.9 _L∈\0.1,0.3,0.5,0.7,0.9\, and the deferral grid λD∈0.258,0.318,0.355,0.392 _D∈\0.258,0.318,0.355,0.392\ [nat], corresponds to percentiles 40,60,75,85\40,60,75,85\ of the per-episode maximum uncertainty on dataset trD_tr, giving |Λ|=20| |=20 candidate hyperparameters. The minimum reward floor is Rmin=0.22R =0.22. Fig. 3 shows the average reward as a function of the average thinking cost, as obtained by varying the threshold λL _L for a fixed deferral threshold λD=0.318 _D=0.318. Thought calibration is seen to significantly reduce the thinking cost relative to E-ReAct, but the average reward is negatively affected. In contrast, TSDS uniformly outperforms E-ReAct, illustrating the benefit of combining thought calibration and cloud deferral. Additionally, ReDAct-CD only marginally improves over E-ReAct, confirming that untruncated reasoning induces overconfidence in the edge model, resulting in fewer actions being deferred to the cloud model. Figure 4: Test results for HotpotQA, shown for all five policies and averaged over 5050 random 75/2575/25 calibration–test splits. Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin=0.22R =0.22. Center: Average deferral rate to the cloud model. The dashed line marks the maximum allowed deferral fraction CDmax=0.70C_D =0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies. Fig. 4 reports average reward, deferral rate, and thinking cost for all policies, averaged over 5050 calibration–test splits. ReDAct-CD improves reward through selective cloud escalation, though at the cost of the full reasoning budget. The pattern mirrors Fig. 2. ReDAct-CD and TSDS are the only two policies clearing the reward floor and respecting the deferral budget, while TSDS does so with substantially fewer thought tokens than ReDAct-CD. In the multi-step setting of this experiment this advantage is further reinforced, as per-step thought calibration concentrates cloud deferral on precisely the steps where the edge model is most uncertain. 5.4 Simulated Household-Robot Planning To evaluate TSDS in a physical AI setting, we introduce a text-based simulated household-manipulation benchmark for evaluating multi-step physical-AI agents without requiring vision or physical hardware. Inspired by SayCan (Ahn et al., 2022) and ALFWorld (Shridhar et al., 2021), the environment places an agent in a four-room setting (bedroom, kitchen, bathroom, living room) and asks it to complete tasks drawn from four families, namely fetch, place, clean, and heat, by issuing five action primitives: go to <room>, pick up <object>, put down <object>, heat <object>, and clean <object>, within at most Tmax=6T_ =6 steps. Reward is binary and is equal to 11 if the task is completed by episode termination, and to 0 otherwise. The environment layout is illustrated in Appendix G.5 (Fig. 12). Setup: The same 7B/14B model pair as HotpotQA is used with Lmax=512L_ =512 for the edge agent and 10241024 for the cloud agent. The probe is retrained at layer ℓ=10 =10 on 5050 household episodes (see Appendix G.5 for an ablation study). We evaluate on 200200 held-out episodes with 5050 random 75/2575/25 splits. The reasoning-threshold grid is λL∈0.1,0.3,0.5,0.7,0.9 _L∈\0.1,0.3,0.5,0.7,0.9\, and the deferral grid λD∈0.287,0.313,0.343,0.384 _D∈\0.287,0.313,0.343,0.384\ [nat] corresponds to percentiles 40,55,70,85\40,55,70,85\ of the per-episode maximum uncertainty on trD_tr, giving |Λ|=20| |=20. The minimum reward floor is Rmin=0.52R =0.52. Figure 5: Test results for the household robot task, shown for five policies and averaged over 5050 random 75/2575/25 calibration–test splits. Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin=0.52R =0.52. Center: Average deferral rate to the cloud model. The dashed line marks the maximum allowed deferral fraction CDmax=0.70C_D =0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies. Fig. 5 reports average reward, deferral rate, and thinking cost for all policies, averaged over 5050 calibration–test splits. The overall pattern follows Figs. 2 and 4, with ReDAct-CD and TSDS being the only two policies that satisfy both constraints, while TSDS does so at 64%64\% lower thinking cost than ReDAct-CD and E-ReAct. One notable difference is that ReDAct-CD achieves lower reward than TSDS here, unlike in Figs. 2 and 4. This can be further evidence that full thought traces allow the edge model to grow overconfident in multi-step plans, suppressing the perplexity signal at the steps where cloud escalation is most needed, while TC-induced truncation preserves uncertainty and directs deferral to the genuinely hard steps. 6 Conclusion We introduced Think Short, Defer Smart (TSDS), which composes thought calibration and uncertainty-aware deferral into a jointly calibrated pipeline for LLM agents. A convergence probe halts on-device reasoning once the intended action has stabilized, and a perplexity-based deferral rule escalates uncertain actions to a cloud model. To ensure reliability, a multi-objective Learn-Then-Test procedure certifies that reward and cloud-call rate remain within user-specified limits, selecting the operating point that minimizes thinking compute. Across all four benchmarks, TSDS simultaneously satisfies the certified reward floor and the cloud-call budget, while consuming substantially fewer thought tokens than deferral-only baselines. This outcome highlights two complementary insights. First, thought calibration and deferral to a cloud model are synergistic, as truncated reasoning leads to detecting uncertainty signals at their most meaningful state. Second, the necessity of deferral varies sharply with task complexity. On simpler single-step tasks, truncation alone nearly suffices and deferral provides a targeted safety net, while on multi-step tasks such as the household robot task, thought calibration without deferral collapses performance severely, and cloud escalation is essential for recovery. Future directions: Immediate extensions include richer uncertainty signals, e.g., semantic entropy, to narrow the deferral gap toward the oracle; multi-tier cascades of agents with multi-constraint LTT certification; and extending thought calibration to models without an explicit reasoning delimiter by inferring action convergence directly from hidden-state trajectories (Esakkiraja and others, 2026). Acknowledgments This work was supported by the European Research Council (ERC) under the European Union’s Horizon Europe Programme (grant agreement No. 101198347). The work of O. Simeone was also supported by an EPSRC Open Fellowship (EP/W024101/1) and by the EPSRC project (EP/X011852/1). References M. Ahn, A. Brohan, N. Brown, et al. (2022) Do as I can, not as I say: grounding language in robotic affordances. arXiv preprint arXiv:2204.01691. Cited by: §1, §3, §5.4. A. N. Angelopoulos, S. Bates, E. J. Candès, M. I. Jordan, and L. Lei (2021) Learn then test: calibrating predictive algorithms to achieve risk control. arXiv preprint arXiv:2110.01052. Cited by: Appendix C, Appendix F, §1, §2, §2, §4.3, Proposition 1. J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. J. Cai, M. Terry, Q. V. Le, and C. Sutton (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: §3, §5.2. J. Chen, S. Park, and O. Simeone (2024) Knowing when to stop: delay-adaptive spiking neural network classifiers with reliability guarantees. IEEE Journal of Selected Topics in Signal Processing 19 (1), p. 88–102. Cited by: §2. L. Chen, M. Zaharia, and J. Zou (2023) FrugalGPT: how to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176. Cited by: item 2, §2. M. Chevalier-Boisvert, B. Dai, M. Towers, R. de Lazcano, L. Willems, S. Lahlou, S. Pal, P. S. Castro, and J. Terry (2023) Minigrid & miniworld: modular & customizable reinforcement learning environments for goal-oriented tasks. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §1, §1, §3. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021) Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: §G.2, §3. O. J. Dunn (1961) Multiple comparisons among means. Journal of the American Statistical Association 56 (293), p. 52–64. Cited by: Appendix C, Appendix C. N. Esakkiraja et al. (2026) Therefore: actions are encoded in hidden representations before text generation. arXiv preprint arXiv:2604.01202. Cited by: §1, §6. A. Farzaneh and O. Simeone (2026) Statistically valid hyperparameter selection: from tuning to guarantees. arXiv preprint arXiv:2606.25601. Cited by: §1, §4.3. M. Fomicheva, S. Sun, L. Yankovskaya, F. Blain, F. Guzmán, M. Fishel, N. Aletras, V. Chaudhary, and L. Specia (2020) Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics 8, p. 539–555. Cited by: §2. M. Frankel et al. (2025) Conformal prediction for large language model inference. arXiv preprint arXiv:2502.09535. Cited by: §2, §2. D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. (2025) DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948. Cited by: §1. Q. Hou, S. Park, M. Zecchin, Y. Cai, G. Yu, O. Simeone, and T. Melodia (2026) Reliable llm-based edge-cloud-expert cascades for telecom knowledge systems. IEEE Transactions on Communications. Cited by: §2. S. Kantamneni (2025) Attention probes. Note: Cited via Kramár et al. (2026) Cited by: 5th item. J. Kramár, J. Engels, S. Wang, B. Chughtai, R. Shah, N. Nanda, and A. Conmy (2026) Building production-ready probes for Gemini. arXiv preprint arXiv:2601.11516. Cited by: 3rd item, 4th item, 6th item, 7th item, S. Kantamneni (2025). A. Malinin and M. Gales (2021) Uncertainty estimation in autoregressive structured prediction. arXiv preprint arXiv:2002.07650. Cited by: Appendix B, Appendix B, §2. I. Ong, A. Almahairi, V. Wu, W. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica (2024) RouteLLM: learning to route LLMs with preference data. arXiv preprint arXiv:2406.18665. Cited by: §2. A. Piatrashyn et al. (2026) ReDAct: reasoning-driven action selection under uncertainty for language model agents. arXiv preprint arXiv:2604.07036. Cited by: Appendix B, item 2, §1, §2, §4.2, 4th item. N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §1, §2. M. Shridhar, X. Yuan, M. Côté, Y. Bisk, A. Trischler, and M. Hausknecht (2021) ALFWorld: aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, Cited by: §1, §1, §3, §5.4. H. Sun, Y. Zhuang, L. Kong, B. Dai, and C. Zhang (2023) AdaPlanner: adaptive planning from feedback with language models. In Advances in Neural Information Processing Systems, Vol. 36. Cited by: §1, §2. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar (2023) Voyager: an open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. Cited by: §2. H. Wang et al. (2025) EAT: efficient adaptive thinking for LLMs via early exit. arXiv preprint arXiv:2503.05061. Cited by: §2. R. Wu, W. Shi, and L. Zettlemoyer (2025) Thought calibration. arXiv preprint arXiv:2505.18404. Cited by: item 1, §1, §2, §4.1. J. Xu et al. (2024) On-device language models: a comprehensive review. arXiv preprint arXiv:2409.00088. Cited by: §1. Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning (2018) HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Proceedings of EMNLP, Cited by: §3, §5.3. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, Cited by: item 1, §1, §2, 1st item. M. Yue, J. Zhao, M. Zhang, L. Du, and Z. Yao (2023) Large language model cascades with mixture of thought representations for cost-efficient reasoning. arXiv preprint arXiv:2310.03094. Cited by: §2. A. Zhou, K. Yan, M. Shlapentokh-Rothman, H. Wang, and Y. Wang (2024) Language agent tree search unifies reasoning, acting, and planning in language models. arXiv preprint arXiv:2310.04406. Cited by: §1. Appendix A Code Availability The full implementation of TSDS, including rollout collection, convergence probe training, LTT calibration, and all experiment scripts, is available at: https://github.com/amirfar76/think-short-defer-smart The Household Robot Task benchmark environment introduced in Sec. 5.4 is released as a standalone package at: https://github.com/amirfar76/household-robot-bench Appendix B Uncertainty Measures The uncertainty score utu_t in the deferral rule (Sec. 3) can be any information-theoretic quantity derived from the token-level probabilities of the generated action sequence y=(τ1,…,τL)y=( _1,…, _L) given context x, with joint probability p(y∣x,θ)=∏i=1Lp(τi∣x,τ<i,θ).p(y x,θ)= _i=1^Lp( _i x, _<i,θ). (8) All three measures below are available as free by-products of the autoregressive decoding pass, requiring no additional forward pass (Malinin and Gales, 2021). Sequence Probability (SP): USP(y∣x)=−∑i=1Llogp(τi∣x,τ<i,θ).U_SP(y x)=- _i=1^L p( _i x, _<i,θ). (9) The negative log-likelihood of the entire sequence. SP is length-sensitive: longer sequences accumulate higher values even at the same per-token confidence. Perplexity (PPL): UPPL(y∣x)=−1L∑i=1Llogp(τi∣x,τ<i,θ).U_PPL(y x)=- 1L _i=1^L p( _i x, _<i,θ). (10) SP normalized by sequence length, measuring per-token average surprise and removing the length bias. High perplexity signals that the model was repeatedly uncertain about its own tokens. PPL is the primary score recommended by (Piatrashyn and others, 2026) for deferral and is used in all our experiments. Mean Token Entropy (MTE): UMTE(y∣x)=1L∑i=1LH(τi∣x,τ<i,θ),U_MTE(y x)= 1L _i=1^LH( _i x, _<i,θ), (11) where H(τi∣⋅)H( _i ·) is the Shannon entropy of the token distribution at position i. While SP and PPL examine the probability of the generated token, MTE measures the spread of the full distribution, capturing uncertainty that SP and PPL may miss when the model happens to generate a high-probability token despite wide disagreement across the vocabulary (Malinin and Gales, 2021). Appendix C Learn-Then-Test Calibration Procedure For each candidate k∈Λ λ_k∈ , we define two null hypotheses corresponding to constraint violations: the reward null ℋkR:R(k)≤RminH_k^R:R( λ_k)≤ R , i.e., that the reward constraint is violated, and the cost null ℋkC:CD(k)≥CDmaxH_k^C:C_D( λ_k)≥ C_D , i.e., that the deferral budget is exceeded. With φθ _θ and Λ fixed before calD_cal is observed, we compute one-sided p-values pkRp_k^R and pkCp_k^C for the null hypotheses ℋkRH_k^R and ℋkCH_k^C, respectively. For each candidate k λ_k, the p-values are computed from the pre-collected calibration data for that specific λL(i) _L^(i) value (Sec. 3), ensuring independence between the data and the threshold under test. When rewards are binary, the exact one-sided binomial p-value pkR=Pr[Binomial(N,Rmin)≤⌊Nr¯k⌋],p_k^R= \! [Binomial(N,\,R )≤ N r_k ], (12) where r¯k r_k is the sample mean reward, is the probability that N i.i.d. Bernoulli trials with success probability RminR produce at most as many successes as observed. An analogous one-sided test gives pkCp_k^C from the sample mean deferral rate c¯k c_k. Since the statistics pkRp_k^R and pkCp_k^C are valid p-values for their respective nulls (Angelopoulos et al., 2021), the maximum pk=max(pkR,pkC)p_k= (p_k^R,p_k^C) is a valid p-value for the union null ℋkR∪ℋkCH_k^R _k^C (Angelopoulos et al., 2021). The certified set is then Λ^=(pk,δ), =A\! (\p_k\,\,δ ), (13) where A is any FWER-controlling procedure (Dunn, 1961). We select the final hyperparameter λ from the certified set Λ via lexicographic minimization over Λ . We first minimize the thinking cost CL(λL)C_L( _L) (3), and in multi-step settings, break ties by minimizing the episode-length cost CS()C_S( λ). Any remaining ties are broken by minimizing CD()C_D( λ). We adopt Bonferroni correction (Dunn, 1961) as the default choice for A as it requires no assumptions on the structure of Λ , and provides unconditional FWER control in any setting. Appendix D Thought Calibration: Stopping Rule Illustration Fig. 6 illustrates the stopping rule from Section 4.1 on a single interaction step. The edge model generates thought tokens τt,1,τt,2,… _t,1, _t,2,… sequentially; at each strided position i∈ti _t, the convergence probe φθ _θ evaluates the hidden state ht,i(ℓ)h_t,i^( ) and returns a scalar score. The score rises as the model’s intermediate action stabilises: early in the thought, the model is still exploring reasoning paths and its committed action fluctuates (score near zero); as reasoning consolidates, the score climbs toward one. Stopping fires the first time the probe score crosses the certified threshold λL _L. In the example shown, this occurs at position i=5i=5: tokens τt,6 _t,6 and τt,7 _t,7 are never generated, saving two of the seven allocated positions. The delimiter </think> is injected immediately after τt,1:5 _t,1:5, and the edge model generates AtedgeA_t^edge from the truncated trace. The action is then passed to the deferral mechanism (Section 4.2): its PPL score is compared to λD _D, and if the action is insufficiently confident the cloud model is invoked instead. Figure 6: Illustration of the thought calibration stopping rule. The probe signal σ(φθ(ht,i))σ( _θ(h_t,i)) rises as the intended action stabilizes. Generation halts at the first position crossing the certified threshold λL _L (here i=5i=5), leaving τt,6,τt,7 _t,6, _t,7 ungenerated. The delimiter </think> is injected and the model generates AtedgeA_t^edge from the partial trace. Appendix E Probe Architecture Details We compare seven candidate instantiations of the convergence probe φθ _θ, all trained by minimizing the binary cross-entropy loss with soft targets using AdamW (learning rate 10−310^-3, weight decay 10−310^-3, full-batch, early stopping on validation loss with patience 2020 out of a maximum 200200 epochs). Window probes use the W=16W=16 most recent strided probe positions left-padded with zeros and a binary mask. • Linear: logistic regression φ(h)=w⊤h+b (h)=w h+b on the single hidden vector ht,p(ℓ)h_t,p^( ). • MLP: two-layer MLP with hidden width 100100 and ReLU activations on the single vector ht,p(ℓ)h_t,p^( ). • Mean-linear: per-position linear score uj=w⊤xj+bu_j=w x_j+b averaged over unmasked window slots (Kramár et al., 2026, §3.1.1). • EMA: trained identically to Mean-linear; at inference, per-position scores are passed through an EMA (ej=αuj+(1−α)ej−1e_j=α u_j+(1-α)e_j-1, α=0.5α=0.5) and the probe output is maxjej _je_j (Kramár et al., 2026, §3.1.2). • Attention probe: per-position MLP transform followed by softmax-weighted aggregation across K=4K=4 heads, aggregated by a linear projection (Kantamneni, 2025). • MultiMax probe: same per-position MLP transform but with hard max over the window for K=4K=4 heads, aggregated by a linear projection (Kramár et al., 2026, §3.2.1). • Rolling-mean attention probe: per-position MLP transform, attention-weighted mean over each sub-window of width w′=4w =4 strided positions, maximum over K=10K=10 sub-window scores. The architecture identified as strongest across a broad range of probing tasks in (Kramár et al., 2026, §3.2.2). Appendix F Proof of Proposition 1 The p-values pkRp_k^R (12) and pkCp_k^C are valid one-sided p-values for ℋkRH_k^R and ℋkCH_k^C respectively (Angelopoulos et al., 2021); hence pk=max(pkR,pkC)p_k= (p_k^R,p_k^C) is a valid p-value for the joint null ℋkR∪ℋkCH_k^R _k^C. Because A controls the FWER at level δ, the probability that any true joint null is rejected is at most δ, which is equivalent to (7). Appendix G Additional Experimental Details G.1 Convergence Probe Ablation We compare seven probe architectures on Nep=100N_ep=100 GSM8K train-split problems with DeepSeek-R1-Distill-Qwen-7B (2828 layers, d=3584d=3584, greedy, bfloat16), yielding 10761076 probe samples at stride 1616 (cap Lmax=384L_ =384). The probe target is the stable-run label, yt,p=∏p′∈t,p′≥p[Ap′,t=At],y_t,p\;=\; _p _t,\,p ≥ p1 [A_p ,t=A_t ], (14) which is 11 iff the early-stopped action agrees with the full-thought action at position p and at every subsequent probe position in step t, directly aligned with the irrevocability of the stopping rule (Sec. 4.1). To isolate hidden-state signal from the global trend in positive rate across the thought trace, we stratify by relative position p/Ltp/L_t and report per-quartile AUROC AUROCQq=AUROC((s^i,yi):pi/Lti∈[(q−1)/4,q/4)),q∈1,2,3,4.AUROC_Q_q=AUROC\! (\( s_i,y_i):p_i/L_t_i∈[(q-1)/4,\,q/4)\ ), q∈\1,2,3,4\. (15) Windowed probes outperform single-state probes: the EMA probe leads with pooled AUROC 0.8150.815 at ℓ⋆=5 =5 (vs. ≤ 0.673≤\,0.673 for single-state), generalizing consistently across all four quartiles. Best probing layers are shallow (ℓ≈5 ≈ 5 for windowed probes; ℓ=16 =16 for single-state), suggesting action-relevant information concentrates early in the transformer stack and benefits from temporal smoothing across the window. The non-monotone quartile pattern (Q2 highest at 0.8570.857, Q3 dip at 0.7480.748, Q4 recovery to 0.8120.812 for EMA) reflects class-balance shifts: Q1 has a low positive rate (stability requires all subsequent probe positions to agree, rare for p≪Ltp L_t); Q2 shows peak discrimination because the model encodes the action before committing irrevocably; Q4 recovers as the model converges toward its natural stopping string. Table 1: Probe ablation on GSM8K with DeepSeek-R1-Distill-Qwen-7B (Nep=100N_ep=100 problems, 10761076 probe samples, 33 seeds) under the stable-run label (14). For each probe architecture we report the best layer ℓ⋆ and its seed-averaged pooled AUROC, plus per-quartile AUROC AUROCQ1,…,AUROCQ4AUROC_Q_1,…,AUROC_Q_4 (defined in (15)). Bold marks the best entry in each column. Probe ℓ⋆ pooled AUROC Q1Q_1 Q2Q_2 Q3Q_3 Q4Q_4 Linear 1616 0.6370.637 0.4940.494 0.6880.688 0.6060.606 0.6030.603 MLP 1616 0.6730.673 0.4960.496 0.7270.727 0.6060.606 0.7130.713 Mean-linear 55 0.6860.686 0.6080.608 0.6870.687 0.6630.663 0.7580.758 Attention 55 0.7330.733 0.6720.672 0.7270.727 0.7080.708 0.8230.823 MultiMax 2626 0.6260.626 0.5560.556 0.7000.700 0.7250.725 0.5540.554 Rolling-mean attn. 55 0.7620.762 0.6720.672 0.7700.770 0.7330.733 0.8740.874 EMA 55 0.8150.815 0.8170.817 0.8570.857 0.7480.748 0.8120.812 Reasoning model and data collection: We use DeepSeek-R1-Distill-Qwen-7B with 2828 transformer layers and hidden size d=3584d=3584. For each of the 100100 GSM8K train-split problems, we decode the thought trace greedily until </think> or Lmax=384L_ =384 tokens. A single forward pass with hidden-state output provides ht,p(ℓ)h_t,p^( ) at every thought position and every layer at zero marginal cost. For each strided position p∈tp _t, we splice </think> after τt,1:p _t,1:p and decode to obtain Ap,tA_p,t. The convergence label yt,py_t,p is then computed via (14). Ablation grid and metrics: We train each of the seven architectures at every layer ℓ∈1,…,28 ∈\1,…,28\ from three random initializations with the 60/20/20 problem-level split fixed, giving 7×28×3=5887× 28× 3=588 configurations. The formal AUROC is the U-statistic AUROC=Pr(s^I>s^J|yI=1,yJ=0),I,J∼Uniform1,…,nindependently,AUROC= \! ( s_I> s_J\; |\;y_I=1,\;y_J=0 ), I,J \1,…,n\\ independently, (16) equal to 0.50.5 for a probe whose scores are independent of the label. Figure 7: Test AUROC of φθ(ht,p(ℓ)) _θ(h_t,p^( )) vs. probing layer ℓ on GSM8K under the stable-run label (14). Solid lines: seed-averaged test AUROC; shaded bands: ±1± 1 standard deviation. Dashed line: chance (0.50.5). The EMA and rolling-mean attention probes dominate at ℓ≈5 ≈ 5. Figure 8: Per-quartile AUROC at the best layer ℓ⋆ for each probe architecture on GSM8K under the stable-run similarity score (14). The EMA probe leads on Q1Q_1, Q2Q_2, and Q3Q_3, while the rolling-mean attention probe attains the highest late-thought AUROC (AUROCQ4=0.874AUROC_Q_4=0.874). The dashed grey line marks chance performance (AUROC=0.5AUROC=0.5). Position-only floor: A logistic regression on the relative thought position p/Ltp/L_t alone achieves AUROC=0.530AUROC=0.530. This position-only floor quantifies the trivial signal available from late-thought samples having higher positive rates. Only 22%22\% of problems are stable throughout their thought; the median first-stable position falls at 91%91\% of thought length, and the mean flip count is 2.82.8 per step. G.2 GSM8K End-to-End TSDS We evaluate TSDS on GSM8K (Cobbe et al., 2021), a benchmark of grade-school arithmetic word problems. The task is single-step (T=1T=1): the agent generates a complete reasoning trace followed by a final numerical answer in a boxed format, and reward is a binary exact-match indicator. Setup: We use DeepSeek-R1-Distill-Qwen-7B as the edge model πedgeπ^edge (Lmax=384L_ =384, mean thought length 159159 tokens) and DeepSeek-R1-Distill-Qwen-14B as the cloud agent πcloudπ^cloud (Lmax=1024L_ =1024, mean thought length 741741 tokens). The EMA probe is set at layer ℓ=5 =5 and trained on 100100 GSM8K train-split problems. We evaluate on 13001300 held-out problems averaged over 5050 random 85/1585/15 calibration–test splits (Ntest=195N_test=195 per split); full-thought accuracies are 84.5%84.5\% for πedgeπ^edge and 90.2%90.2\% for πcloudπ^cloud. Calibration: The reasoning-threshold grid is λL∈0.1,0.2,…,0.9 _L∈\0.1,0.2,…,0.9\. The deferral grid λD∈0.053,0.063,0.073,0.088,0.127 _D∈\0.053,0.063,0.073,0.088,0.127\ [nat] is set to quantiles 3030–9595 of the PPL distribution on trD_tr, following the general procedure described in Sec. 5.3. This gives |Λ|=45| |=45 candidates. We use exact one-sided binomial p-values with Bonferroni correction at δ=0.10δ=0.10, Rmin=0.855R =0.855, and CDmax=0.70C_D =0.70. Results: Table 2: GSM8K test-set results (testD_test), means over 5050 random splits; each policy evaluated at its own LTT-selected λ. R R: empirical reward; C^D C_D: deferral rate; C^L C_L: mean thinking tokens. Hatched entries fail the joint LTT constraints. Bold: lowest C^L C_L among policies satisfying the joint constraints. Policy R R C^D C_D C^L C_L (tok) GSM8K (ntest=195n_test=195); ^=(0.10, 0.088) λ=(0.10,\,0.088), Rmin=0.855R =0.855, CDmax=0.70C_D =0.70 E-ReAct 0.8520.852 0.0000.000 173173 Cloud ReAct 0.9060.906 1.0001.000 — E-ReAct-TC (λL=0.10 _L=0.10) 0.8520.852 0.0000.000 18 0018 ReDAct-CD (λL=0 _L=0) 0.8640.864 0.0820.082 173173 TSDS @ λ λ 0.8900.890 0.2550.255 0018 Figure 9: GSM8K test-set results (5050-seed averages ± std, ntest=195n_test=195 per split, δ=0.10δ=0.10). Across 5050 independent 85/1585/15 splits of 13001300 held-out problems, each method’s operating point λ is selected by LTT on the calibration portion; all metrics are measured on the held-out test portion. Left: test-set reward; dashed line marks Rmin=0.855R =0.855. A bar above this line reflects empirical test performance, not the LTT guarantee. Center: test-set deferral rate, CDmax=0.70C_D =0.70 dashed. Right: mean thinking tokens. E-ReAct-TC fails the LTT reward constraint in all 5050 splits (hatched, shown for reference); TSDS and ReDAct-CD satisfy the joint constraints in their respective splits. TSDS reaches R^test=0.890 R_test=0.890, a 0.0260.026 improvement over ReDAct-CD, while using just 1818 thought tokens per problem, one-tenth of ReDAct-CD’s cost. ReDAct-CD retains the full thought trace (173173 tokens) and defers on only 8%8\% of problems, achieving R^test=0.864 R_test=0.864. E-ReAct-TC does not certify: without a deferral safety net, truncation-induced errors have no recovery path and the reward lower bound fails to clear the floor in all 5050 splits. The elevated deferral rate of TSDS (26%26\%) relative to ReDAct-CD (8%8\%) reflects the thought-calibration–deferral synergy: aggressively truncated thoughts produce higher PPL, directing more problems to πcloudπ^cloud, with that deferral concentrated on the problems where truncated reasoning is most fragile. Cloud ReAct achieves R^test=0.906 R_test=0.906 but defers unconditionally (CD=1.0C_D=1.0), violating CDmax=0.70C_D =0.70. Rollout quality: On 13001300 test-split problems, full-thought accuracy is 84.5%84.5\% for πedgeπ^edge and 90.2%90.2\% for πcloudπ^cloud. The two models agree on 10261026 problems, are both wrong on 5454, only πedgeπ^edge is right on 7373, and only πcloudπ^cloud is right on 147147. An oracle deferral catching exactly the 147147 “only-large-right” problems would lift accuracy from 84.5%84.5\% to 95.8%95.8\%. Baseline details: • E-ReAct (λL=λD=+∞ _L= _D=+∞): full thought, no deferral. • Cloud ReAct (λD=−∞ _D=-∞): always defer; no Thought Calibration. • E-ReAct-TC (λD=+∞ _D=+∞, λL=0.10 _L=0.10): truncate but never defer; excluded from Λ because R^cal=0.839<Rmin R_cal=0.839<R . • ReDAct-CD (λL=0 _L=0, λD=0.088 _D=0.088): full thought; λD _D selected via single-objective LTT; certifies in the 50-split analysis. G.3 HotpotQA: Bar Chart and Pareto Frontier Figure 10: HotpotQA results under two constraint regimes, averaged over 5050 random splits. Each panel shows test-set means and ±1± 1 std across 5050 independent 75/2575/25 splits of 250250 episodes; the RminR and CDmaxC_D lines mark the bounds derived from calD_cal, not thresholds on test-set values. Upper (Rmin=0.22R =0.22): E-ReAct-TC (R^cal=0.202 R_cal=0.202) and E-ReAct (R^cal=0.218 R_cal=0.218) fall below the reward floor; Cloud ReAct exceeds CDmaxC_D ; TSDS and ReDAct-CD satisfy the joint constraints. Lower (Rmin=0.12R =0.12): E-ReAct-TC also satisfies the reward constraint; TSDS selects a more aggressive λL=0.10 _L=0.10, further reducing thinking cost with the deferral safety net. Columns: R R, C^D C_D, C^L C_L. Hatched bars fail the joint constraints. Pareto frontier: Figure 11: Pareto frontiers in (C^L,C^S)( C_L, C_S) space on HotpotQA for four deferral budgets CDmaxC_D (Rmin≈0.10R ≈ 0.10 throughout), averaged over 200200 random 75/25 calibration/test splits. For each split and each certified λL _L, we record (C^L,minλD∈Λ^(λL)C^S)( C_L,\, _ _D∈ ( _L) C_S); plotted points are means across splits; shaded bands are ±1± 1 std of C^S C_S. Tightening CDmaxC_D shifts the frontier upward and rightward: fewer allowed deferrals force more environment steps and exclude aggressive truncation from the certified set. Within each Pareto curve the frontier slopes downward: more aggressive truncation (lower C^L C_L) produces more uncertain actions and therefore more deferral to πcloudπ^cloud, which solves problems in fewer environment steps (3.113.11 vs. 4.704.70 for πedgeπ^edge). At CDmax=0.90C_D =0.90 the budget is so permissive that all λL _L values converge on the same aggressive-deferral optimum, collapsing the frontier to a single point. G.4 Code Generation: MBPP Additional details for Sec. 5.2. Table 3 reports per-policy means. Table 3: MBPP test-set results averaged over 5050 random 60/4060/40 calibration–test splits (ntest=103n_test=103 per split). R R is the empirical test reward, C^D C_D is the fraction of problems deferred to the cloud model, and C^L C_L is the mean number of thinking tokens per problem. Policies that violate either LTT constraint are shown with hatching; the bold entry is the LTT-selected policy with the fewest thinking tokens. Policy R R C^D C_D C^L C_L (tok) E-ReAct 0.7190.719 0.0000.000 11831183 Cloud ReAct 0.8100.810 1.0001.000 — E-ReAct-TC (λL=0.80 _L=0.80) 0.6200.620 0.0000.000 422 0422 ReDAct-CD (λL=0 _L=0) 0.7790.779 0.7860.786 12061206 TSDS @ λ λ 0.7140.714 0.3460.346 0422 Multi-seed evaluation methodology: Because LTT controls the average over random calibration/test partitions, we report results averaged over Nseeds=50N_seeds=50 independent random 60/4060/40 splits of the 257257 MBPP test episodes. For each split we rerun the LTT procedure (fixing all other hyperparameters) and record whether TSDS certifies and its operating-point metrics. Reported means and standard deviations are across splits. TSDS certifies in 22/5022/50 splits; across those 2222 splits, mean R^test=0.714±0.026 R_test=0.714± 0.026 and mean C^L=422±118 C_L=422± 118 tokens. Non-certified splits contribute to the baseline averages reported in Fig. 2. Rollout details: Both πedgeπ^edge and πcloudπ^cloud are run with max_thought=2048=2048. The 7B model achieves mean thought length 11661166 tokens; the 32B model produces substantially longer traces. The probe is EMA at ℓ=5 =5, retrained on 930930 probe samples from 100100 MBPP train-split episodes. Uncertainty scores (PPL) are computed from the token probabilities of the generated Python function body. Baseline details: • E-ReAct: 119/154119/154 wins on the seed-0 partition, binomial p-value 0.013≫δ/|Λ|=0.002220.013 δ/| |=0.00222. • E-ReAct-TC (λL=0.80 _L=0.80, λD=+∞ _D=+∞): 111/154111/154 wins, p-value 0.210.21. • ReDAct-CD (λL=0 _L=0, λD=0.084 _D=0.084): certifies on reward but C^D,cal=0.786>CDmax=0.70 C_D,cal=0.786>C_D =0.70. • Cloud ReAct: C^D,cal=1.000>CDmax C_D,cal=1.000>C_D . • TSDS (^=(0.80,0.084) λ=(0.80,0.084)): 123/154123/154 wins, p-value 0.0018<0.002220.0018<0.00222, satisfying both constraints and receiving LTT certification. G.5 Household Robot: Environment Layout and Diagnostics Probe layer: On the household robot task, an ablation over ℓ∈5,8,10,14,20 ∈\5,8,10,14,20\ on trD_tr shows the EMA probe reaches its best AUROC at ℓ=10 =10 (0.8630.863) versus 0.8300.830 at ℓ=5 =5. This differs from the other three benchmarks where ℓ=5 =5 is optimal, and confirms that the best probing layer is task-dependent rather than a fixed architectural default. Fig. 12 shows the floor plan of the simulated household environment used in Section 5.4. The environment consists of four rooms (living room, kitchen, bedroom, bathroom), each containing a fixed set of surfaces and appliances on which objects are placed at the start of each episode. The agent navigates between rooms via go to <room> and interacts with objects via pick up, put down, heat, and clean actions, aiming to complete one of four task families (fetch, place, clean, heat) within at most six steps. Figure 12: Floor plan of the simulated household environment. Each room contains a set of surfaces and appliances (see legend); objects are placed on surfaces at episode initialisation. All four rooms are mutually accessible via the go to <room> action in a single step, irrespective of the agent’s current location. Appendix H Thought Calibration–Deferral Interplay: Probe Signal and PPL Across the Thought Trace Figure 13: Thought calibration–deferral interplay on the 2222 slow-converging GSM8K probe-training episodes (those where the EMA probe score at the first strided position, p=16p=16 tokens, is below 0.100.10), using the EMA probe at ℓ=5 =5. Top: mean EMA probe score (±1±1 s.e.) as a function of absolute thought-token count. The score rises from near zero at 1616 tokens, crosses λL=0.50 _L=0.50 (dashed orange) at ≈90≈90 tokens (grey dotted vertical line), and reaches ≈0.87≈0.87 at 192192 tokens. Bottom: mean PPL of the early-stopped action Ap,tA_p,t (±1±1 s.e.) with a three-point moving-average overlay. At the crossing (≈90≈90 tokens), mean PPL ≈0.076≈0.076 [nat] is above the deferral threshold λD=0.073 _D=0.073 (dashed red line): Thought Calibration early-stopping would trigger deferral at this point. By 192192 tokens, mean PPL has fallen to ≈0.064≈0.064 [nat], below λD _D, showing that had reasoning continued, the small model’s action would have been confident enough to avoid deferral entirely. The dotted vertical line marks this crossing: early-stopped thoughts that look fragile to the deferral mechanism would have been fine if allowed to run to completion. Fig. 13 offers empirical evidence of the thought-calibration–deferral interplay introduced in Section 1 and depicted schematically in Fig. 1(c), drawn from the trD_tr episodes of the GSM8K experiment (Appendix G.2). We focus on the 2222 episodes whose probe score at the first strided position is below 0.100.10, the “slow-converging” subset in which Thought Calibration and deferral interact most visibly, because both the convergence signal and the action uncertainty evolve gradually across the thought trace rather than snapping to their final values within the first few tokens. The top panel shows the EMA probe score rising from near zero to ≈0.87≈0.87 over 192192 thought tokens. Had Thought Calibration been applied with threshold λL=0.50 _L=0.50, it would have halted reasoning at ≈90≈90 tokens. The bottom panel reveals the consequence: at that crossing, mean action PPL is still ≈0.076≈0.076 [nat], above λD=0.073 _D=0.073, so the deferral rule would fire and the cloud model would be invoked. Allowing reasoning to continue to 192192 tokens reduces mean PPL to ≈0.064≈0.064 [nat], below λD _D, at which point the edge model’s action is confident enough to execute without deferral. This is precisely the tension that motivates joint calibration of λ (Section 4.3): reasoning thresholds that are too aggressive shorten thoughts but inflate the deferral rate, while overly conservative thresholds waste reasoning compute on problems the edge model could already handle confidently. The LTT procedure selects λ to minimise thinking cost subject to both the reward floor and the deferral budget simultaneously, automatically trading off these competing pressures.