Paper deep dive
Optimal Stopping of Self-Refining Foundation Models
Kim Hammar, Tansu Alpcan, Emil C. Lupu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/16/2026, 3:13:54 AM
Summary
This paper formalizes the self-refinement process of foundation models as an optimal stopping problem to determine the most cost-efficient number of refinement iterations. By modeling the score evolution as a Markov process and using stochastic approximation, the authors derive optimal threshold-based stopping policies. Empirical evaluation on a coding benchmark (effibench) with models like Gemini, Haiku, and GPT Codex demonstrates that these policies significantly outperform heuristic approaches in cost-efficiency while maintaining or improving output quality.
Entities (13)
Relation Signals (9)
Optimal Stopping → appliedto → Self-refinement
confidence 98% · we formalize this process as an optimal stopping problem
Self-refinement → usedby → Foundation Models
confidence 97% · Foundation models can improve their outputs through a self-refinement process
Haiku 4.5 → evaluatedon → Effibench
confidence 95% · We apply three frontier foundation models to this benchmark: haiku 4.5
Gemini Flash Lite 3.1 → evaluatedon → Effibench
confidence 95% · We apply three frontier foundation models to this benchmark: ... gemini flash-lite 3.1
GPT Codex Mini 5.1 → evaluatedon → Effibench
confidence 95% · We apply three frontier foundation models to this benchmark: ... and gpt codex mini 5.1
Kim Hammar → affiliatedwith → Imperial College London
confidence 90% · K. Hammar and E.C. Lupu are with Imperial College London
Emil C. Lupu → affiliatedwith → Imperial College London
confidence 90% · K. Hammar and E.C. Lupu are with Imperial College London
Tansu Alpcan → affiliatedwith → University of Melbourne
confidence 90% · T. Alpcan is with the University of Melbourne, Australia.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Foundation models can improve their outputs through a self-refinement process driven by external feedback. In this process, the model is embedded in an iterative loop where it generates outputs, receives feedback from verifiers, and refines its responses through in-context learning. Following a novel approach, we formalize this process as an optimal stopping problem where the number of refinement iterations is decided based on expected improvement relative to cost. We derive optimal stopping policies and show that they can be efficiently computed through stochastic approximation. To evaluate our approach experimentally, we apply it to a coding benchmark for foundation models. The empirical results show that our stopping policies are significantly more cost-efficient than stopping policies proposed in prior work.
Tags
Links
- Source: https://arxiv.org/abs/2608.10729v1
- Canonical: https://arxiv.org/abs/2608.10729v1
Trouble viewing inline? Open PDF directly →
Full Text
51,156 characters extracted from source content.
Expand or collapse full text
Optimal Stopping of Self-Refining Foundation Models Kim Hammar, Tansu Alpcan, and Emil C. Lupu K. Hammar is supported by the Swedish Research Council under contract 2024-06436. T. Alpcan is with the University of Melbourne, Australia. tansu.alpcan@unimelb.edu.au. K. Hammar and E.C. Lupu are with Imperial College London, United Kingdom. k.hammar,e.c.lupu@imperial.ac.uk. Abstract Foundation models can improve their outputs through a self-refinement process driven by external feedback. In this process, the model is embedded in an iterative loop where it generates outputs, receives feedback from verifiers, and refines its responses through in-context learning. Following a novel approach, we formalize this process as an optimal stopping problem where the number of refinement iterations is decided based on expected improvement relative to cost. We derive optimal stopping policies and show that they can be efficiently computed through stochastic approximation. To evaluate our approach experimentally, we apply it to a coding benchmark for foundation models. The empirical results show that our stopping policies are significantly more cost-efficient than stopping policies proposed in prior work. I Introduction Foundation models are becoming an important component of decision-making systems across many domains, including software development [16], scientific research [21], content generation [23], and systems engineering [10]. They consist of large neural networks that are trained on vast datasets (e.g., web-scale text corpora), which enables them to generalize across tasks and modalities. Prominent examples include large language models (e.g., Gemini [9]), time-series models (e.g., Chronos [3]), and visual models (e.g., Flamingo [1]). These types of models generate outputs autoregressively by sampling from a conditional distribution pθ(z∣v)p_θ(z v), where θ denotes the model parameters, v is an input (e.g., a task description), and z is the generated output (e.g., code or text). Since the model adapts to the input v, the same model can be applied to different tasks without updating the parameters θ, a property known as in-context learning [6]. This flexibility also enables the model to refine its outputs based on feedback. In particular, given an initial output z and feedback x (e.g., from automated verification procedures), the model can generate a revised output by sampling from pθ(z′∣v,z,x)p_θ(z v,z,x), as initially shown by Madaan et al. [22]. The process of generating and revising outputs based on feedback typically continues for a fixed number of iterations or until a predetermined stopping criterion is met. While prior work has demonstrated that this approach improves outputs across various application domains, such as coding (see e.g., [17]) and logical reasoning (see e.g., [41, 40]), the literature is focused on empirical evaluation and lacks a formal analysis. In particular, there is little understanding of how to optimally decide when to stop refining. In this paper, we address this research gap by presenting a decision-theoretic model of the self-refinement process. Specifically, we formulate self-refinement as an optimal stopping problem in which the number of refinement iterations is decided based on the expected improvement relative to the cost of invoking the foundation model. For example, if the foundation model is hosted locally, each invocation incurs computational expenditure. Similarly, if the model is accessed via an external provider, it incurs a monetary cost. Figure 1: Illustration of a self-refining foundation model with stopping. The model is integrated into an iterative verification and refinement loop, in which it is used to generate outputs that are verified against constraints to generate a score. The score is then input to a stopping policy μ that decides whether to stop and accept the output or to continue revising it. Leveraging this mathematical formulation, we establish conditions for optimal threshold-based stopping policies and validate them on a coding benchmark across three frontier models. Empirical evaluations show that our stopping policies are cost-efficient and consistently improve performance when compared to other approaches proposed in the research literature, which rely on heuristic stopping policies. Our contributions can be summarized as follows: • We present a novel formulation of self-refinement in foundation models as an optimal stopping problem. • We derive optimal stopping policies and validate them on a coding benchmark. The results show that they are more cost-efficient than stopping policies proposed in prior work. Our implementation is available at [15]. I Related Work Optimal stopping is a classical problem with a well-developed theory; see e.g., Wald [39], Shiryaev [32], and Chow et al. [7]. Example use cases for this theory include asset selling [5], intrusion detection [37], network management [14], machine replacement [19], hypothesis testing [39], gambling [7], industrial control [28], change detection [38, 30, 26, 24, 20], and flow control [12, 13, 11]. To our knowledge, we are the first to apply this theory to analyze the self-refinement process of a foundation model. Many variants of the stopping problem have been studied. Examples include discrete-time and continuous-time problems, finite-horizon and infinite-horizon problems, problems with fully observed and partially observed state spaces, problems with finite and infinite state spaces, Markovian and non-Markovian problems, and single-stop and multi-stop problems. Consequently, different solution approaches have been developed. The most common are the martingale approach (see e.g., Snell [33]) and the Markovian approach (see e.g., Bather [4]). In this paper, we investigate a stopping problem with a finite time horizon, discrete-time progression, a continuous state space, and the Markov property. I The Self-Refinement Use Case We consider a scenario where a foundation model is used to solve a task specified by an instruction in natural language. As an example, the task could be to solve a programming problem and the model could be a large language model, such as Gemini [9]. To evaluate the solution produced by the model, we associate it with a score x∈[0,1]x∈[0,1], with x=1x=1 being the optimal score. For instance, in the programming example, the solution could be a piece of code and the score x could capture its correctness and computational efficiency. We assume that the model is embedded in an iterative loop where it refines its output based on feedback, as illustrated in Fig. 1. Each iteration invokes the model with the current score x∈[0,1]x∈[0,1] as feedback, which leads to a new output that receives an updated score x′x and incurs a cost c>0c>0. This cost can be either computational (if the model is run locally) or monetary (if the model is provided externally). After each refinement iteration, a stopping policy μ uses the current score to decide whether to stop and accept the output or to continue refining, where the maximum number of iterations is N. When designing this policy, the goal is to optimally balance the refinement costs against the potential improvement in the score x, as formally defined below. IV Optimal Stopping Formulation We formalize the self-refinement use case described above as a discrete-time dynamical system where the score evolves as a Markov process (xk)k=0N(x_k)_k=0^N. At each stage k∈0,1,…,N−1k∈\0,1,…,N-1\, two controls are available: ( S)top and ( C)ontinue. Control u=u= S in state x yields a payoff g(x)≥0g(x)≥ 0 that quantifies the quality of the output and terminates the process. Conversely, control u=u= C incurs a cost c>0c>0 and transitions the system to the next state according to xk+1=f(xk,wk),k=0,1,…,N−1,x_k+1=f(x_k,w_k), k=0,1,…,N-1, (1) where the initial state x0x_0 is given, (wk)k=0N−1(w_k)_k=0^N-1 are i.i.d. random variables with distribution PwP_w that capture the stochasticity of the generation process, and f is the system function. Let (Ω,ℱ,P)( ,F,P) be the underlying probability space and let (ℱk)k=0N(F_k)^N_k=0 denote the filtration generated by the quality scores, i.e., each ℱkF_k is the sigma-algebra generated by the random variables x0,…,xkx_0,…,x_k. Within this space, we define a stopping policy μ=(μ0,μ1,…,μN)μ=( _0, _1,…, _N) as a sequence of measurable functions μk:[0,1]→, _k:[0,1]→\ S, C\ that map the state to a control. Given a policy μ, we define the stopping time as τμ=mink∈0,1,…,N∣μk(xk)=, _μ= \k∈\0,1,…,N\ _k(x_k)= S\, (2) with the constraint μN(xN)= _N(x_N)= S, which ensures τμ≤N _μ≤ N. Proposition 1. The random variable τμ _μ defined in (2) is a stopping time with respect to the filtration (ℱk)k=0N(F_k)^N_k=0. Proof. It suffices to show that the event τμ=k\ _μ=k\ belongs to the sigma-algebra ℱkF_k for all k∈0,1,…,Nk∈\0,1,…,N\. For k<Nk<N, τμ=k\ _μ=k\ is the event that the stopping policy continues at all preceding iterations and stops at iteration k, i.e., τμ=k=(⋂j=0k−1μj(xj)=)∩μk(xk)=, \ _μ=k\= ( _j=0^k-1\ _j(x_j)= C\ )∩\ _k(x_k)= S\, where the intersection is Ω when k=0k=0. Since each policy μj _j is a measurable function of xjx_j and xjx_j is ℱjF_j-measurable (and hence ℱkF_k-measurable for j≤kj≤ k), each set in the intersection belongs to ℱkF_k. As a consequence, we have τμ=k∈ℱk\ _μ=k\ _k. For k=Nk=N, the event τμ=N=⋂j=0N−1μj(xj)=\ _μ=N\= _j=0^N-1\ _j(x_j)= C\ belongs to ℱN−1⊆ℱNF_N-1 _N. Thus τμ _μ is a stopping time with respect to (ℱk)k=0N(F_k)^N_k=0. ∎ Given the preceding definition of a stopping policy μ, we define the stopping and continuation sets as kμ S_k^μ =x∣x∈[0,1],μk(x)=, =\x x∈[0,1], _k(x)= S\, (Stopping set) (Stopping set) kμ C_k^μ =x∣x∈[0,1],μk(x)=. =\x x∈[0,1], _k(x)= C\. (Continuation set) (Continuation set) When designing the stopping policy μ, the objective is to maximize the expected payoff Eg(xτμ)E\g(x_ _μ)\ while minimizing refinement costs. In particular, each refinement iteration incurs a cost c>0c>0 (e.g., per-token charges) and the objective is to find an optimal stopping policy μ⋆μ that satisfies μ⋆∈argmaxμEg(xτμ)−cτμ subject to (1),μ ∈ *arg\,max_μE \g(x_ _μ)-c _μ \ subject to eq:dynamics, (3) where the expectation is taken over the randomness in the dynamics (1) and the stopping time τμ _μ is defined in (2). We denote the optimal stopping time, stopping set, and continuation set by τ⋆,k⋆τ , S_k , and k⋆ C_k , respectively. Remark 1. We model the cost c as a constant. In principle, c could vary with the stage k, e.g., if token consumption changes across iterations. However, in our experimental evaluation (see §V), token consumption per iteration is approximately constant, which justifies this assumption. Characterizing the Optimal Stopping Policy To characterize the optimal stopping policy for problem (3), we use a dynamic programming formulation. Following this approach, we define the stage-dependent stopping time τμ,k=minj∈k,k+1,…,N∣μj(xj)=, _μ,k= \j∈\k,k+1,…,N\ _j(x_j)= S\, so that τμ=τμ,0 _μ= _μ,0; cf. (2). We then define the value function Vkμ:[0,1]→ℜV^μ_k:[0,1]→ to represent the expected payoff minus refinement costs from state x at stage k under policy μ, i.e., Vkμ(x) V^μ_k(x) =Eg(xτμ,k)−c(τμ,k−k)∣xk=x, =E\g(x_ _μ,k)-c( _μ,k-k) x_k=x\, for all stages k=0,1,…,N−1k=0,1,…,N-1 and states x∈[0,1]x∈[0,1], with the terminal condition VNμ(x)=g(x)V^μ_N(x)=g(x) for all x∈[0,1]x∈[0,1]. The optimal value function Vk⋆V_k , which is derived by optimizing over all policies μ, satisfies the Bellman equation Vk⋆(x)=maxg(x),−c+EVk+1⋆(f(x,wk)),V _k(x)= \g(x),\;-c+E \V _k+1(f(x,w_k)) \ \, (4) for all stages k=0,1,…,N−1k=0,1,…,N-1 and states x∈[0,1]x∈[0,1]. The first term in the maximization (4) corresponds to the decision to stop (u=u= S), which yields the payoff g(x)g(x). The second term corresponds to the decision to continue (u=u= C), which incurs the cost c and leads to a new state. We denote the value of continuing under an optimal policy by Qk⋆(x)=−c+EVk+1⋆(f(x,wk)). Q _k(x)=-c+E\V _k+1(f(x,w_k))\. Given this notation, the optimal policy can be expressed as μk⋆(x) _k (x) =,if g(x)≥Qk⋆(x),,otherwise, = dcases S,&if g(x)≥ Q _k(x),\\ C,&otherwise, dcases k=0,1,…,N−1. k=0,1,…,N-1. The above structure implies that the sequence (Vk⋆)k=0N(V _k)_k=0^N is the smallest superharmonic majorant of g.111A sequence (hk)k=0N(h_k)_k=0^N is superharmonic with respect to the system f(x,w)f(x,w) if hk(x)≥−c+Ehk+1(f(x,w))h_k(x)≥-c+E\h_k+1(f(x,w))\ for all x and k<Nk<N. It is a superharmonic majorant of g if additionally hk(x)≥g(x)h_k(x)≥ g(x) everywhere. In other words, the stopping set is the set of states where Vk⋆V _k equals g, while the continuation set is the set where Vk⋆V _k exceeds g. This characterization admits a geometric interpretation of the optimal stopping policy, as illustrated in Fig. 2. Figure 2: Geometric characterization of the optimal stopping policy. Remark 2. When c=0c=0, the Bellman equation (4) reduces to Vk⋆(x)=maxg(x),EVk+1⋆(f(x,wk))V _k(x)= \g(x),E\V _k+1(f(x,w_k))\\. In this case, the process (Vk⋆(xk))k=0N(V _k(x_k))_k=0^N is the Snell envelope of the payoff process (g(xk))k=0N(g(x_k))_k=0^N [33] and the problem corresponds to the classical optimal stopping formulation of Dynkin [8]. Figure 3: Empirical measurements of the self-refinement process across three frontier models on effibench [18]. The top row shows performance metrics (execution time and memory usage) as functions of the refinement iteration. The middle row shows cumulative token consumption (total, input, and output). The bottom row shows the refinement time (per iteration and cumulative) and cumulative monetary cost. Curves show the mean values and shaded regions indicate one standard deviation. The cost is computed from per-token charges as listed by each model provider on March 17, 2026. V Identification of the Stopping Problem The optimal stopping problem formulated in the preceding section is defined by three components: the transition dynamics f, the continuation cost c, and the payoff function g; cf. (3). In this section, we identify these components based on empirical observations of three foundation models. V-A Example Use Case: Code Optimization The optimal stopping formulation presented in (3) is general and applies to any task that a foundation model can be used to solve, provided that the output of the model can be associated with a quality score. However, for the purpose of analysis and experimental validation, we instantiate the stopping problem for a specific code optimization use case. In this use case, the input to the foundation model is a programming task and the output is code for solving the task. At each refinement iteration, the generated code is executed against a test suite that measures its correctness, execution time, and memory usage. These measurements are aggregated into a candidate score x~k∈[0,1] x_k∈[0,1] defined as x~k x_k =0,if tests fail,clip(Tmax−TkTmax−Tmin, 0, 1),otherwise, = dcases0,&if tests fail,\\ clip\! ( T_ -T_kT_ -T_ ,\;0,\;1 ),&otherwise, dcases (5a) Tk T_k =1Tref∫0kmk(t)dt, = 1T_ref _0^T_km_k(t)\,dt, (5b) where mk(t)m_k(t) is the memory footprint at time t when executing the code generated at iteration k, kT_k is the total execution time, clip(⋅,0,1)clip(·,0,1) restricts the input to the unit interval, and TrefT_ref is the memory-over-time integral of a given reference code, as defined in [18]. The ratio TkT_k thus captures both execution time and memory consumption in a single metric. The constants TminT_ and TmaxT_ are normalization bounds chosen so that code that passes the tests with Tk≤TminT_k≤ T_ receives the maximal score x~k=1 x_k=1, while failing code and code that passes the tests with Tk≥TmaxT_k≥ T_ receive x~k=0 x_k=0. After computing the candidate score x~k x_k through (5), we update the state of the stopping problem, which is the score of the best code retained so far, i.e., xk=maxxk−1,x~kx_k= \x_k-1, x_k\ with initial state x0=x~0x_0= x_0. Next, we feed the updated state xkx_k to the stopping policy μ, which decides whether to stop or continue the refinement process by generating new code. V-B Data Collection We collect data using effibench [18], which is a coding benchmark for large language models. This benchmark consists of programming tasks for which solutions are assessed based on the candidate score defined in (5). We apply three frontier foundation models to this benchmark: haiku 4.5 [2], gemini flash-lite 3.1 [9], and gpt codex mini 5.1 [25]. For each model, we run the self-refinement loop illustrated in Fig. 1 for N=10N=10 iterations across 5050 tasks. Figure 3 (on the next page) summarizes the collected data. The top row shows that all three models improve code efficiency through self-refinement, but with diminishing returns. Most of the improvement occurs within the first 3−43-4 iterations. Moreover, the middle and bottom rows in Fig. 3 show that token consumption and monetary cost grow approximately linearly with the number of refinement iterations. Together, these trends illustrate the trade-off between the quality of the generated output and the cost of refinement. V-C System Identification We use the collected data to estimate the parameters of the stopping problem as follows. First, we define the continuation cost c [cf. (3)] to be the average monetary cost of performing a self-refinement iteration. This cost depends on the choice of foundation model and is defined in Table I. Foundation model Continuation cost c Explanation haiku 4.5 0.01 Avg. cost ($) per iteration; see Fig. 3. gemini flash-lite 3.1 0.0025 Avg. cost ($) per iteration; see Fig. 3. gpt codex mini 5.1 0.005 Avg. cost ($) per iteration; see Fig. 3. TABLE I: The continuation cost for the three foundation models. (Computed based on per-token charges as of March 17, 2026.) Second, we define the payoff function g as g(x)=βxg(x)=β x, where β>0β>0 is a weighting factor that allows controlling the relative importance of output quality compared to refinement cost. This parameter is not estimated from data; rather, it is a configuration parameter that can be adapted to the use case. Lastly, we estimate the system function f [cf. (1)] from empirical observations of the refinement process. In particular, we model the transition dynamics as a regression problem where the candidate score x~k+1 x_k+1 [cf. (5)] is predicted from the current state xkx_k; the state transition then follows from xk+1=maxxk,x~k+1x_k+1= \x_k, x_k+1\. To capture both the nonlinear relationship and the inherent uncertainty in the refinement process, we estimate f using a Gaussian process (GP) model. Formally, we assume that f can be represented as xk+1 x_k+1 =f(xk,wk)=min1,maxxk,q(xk)+wk, =f(x_k,w_k)= \1, \x_k,q(x_k)+w_k\\, (6) where k=0,1,…,N−1k=0,1,…,N-1 is the stage of the refinement process, q is an unknown score function, and wkw_k is a zero-mean Gaussian noise variable with variance σ2σ^2. The max operator in (6) reflects the fact that the previous code is retained if the newly generated code achieves a lower score. Similarly, the min operator is used to ensure that x∈[0,1]x∈[0,1]. To estimate the unknown function q, we use a Gaussian process prior, which allows us to obtain a posterior distribution over functions conditioned on the observed data. Specifically, we define the prior as q∼(m,κ)q (m,κ), where m is the mean function and κ is the covariance function of the GP. In this paper, we define these functions as m(x) m(x) =x, =x, (7a) κ(x,x′) κ(x,x ) =(1+5r+5r23)exp(−5r), = (1+ []5r+ 5r^23 ) (- []5r ), (7b) for all states x,x′∈[0,1]x,x ∈[0,1], where r=|x−x′|r=|x-x |. This covariance function encodes the assumption that q varies smoothly over the input space. Similarly, the mean function encodes that refinement preserves the current quality score. Given this prior, we update it using the empirical observations from evaluations on effibench [18] (see Fig. 3) and Bayes’ rule. (We set σ2σ^2 equal to the variance estimated during the GP fit.)222See Rasmussen and Williams for detailed formulas of the Bayesian updates [29, Def. 2.1]. Due to clipping [cf. (5)], a candidate score x~k+1∈0,1 x_k+1∈\0,1\ may correspond to a value of q(xk)+wkq(x_k)+w_k outside [0,1][0,1]. We nevertheless treat all candidate scores as direct observations of q(xk)+wkq(x_k)+w_k, which yields a Gaussian approximation of the posterior. We denote the mean of the resulting posterior as q~(x)=Eq(x)∣observed data q(x)=E\q(x) data\. Figure 4 shows the posteriors for the three foundation models. We observe that the model q~(x) q(x) exhibits diminishing returns as the state x increases. In particular, for lower-quality scores (states), the gain from refinement is relatively large, while for higher-quality scores, the improvement becomes smaller. Figure 4: Estimation of the score function q [cf. (6)] when applying the foundation models to effibench [18]. Curves show the mean values of the Gaussian processes; shaded regions indicate one, two, and three standard deviations (darker to lighter shades). The dashed lines show q~(x)=x q(x)=x. From the identified models in Fig. 4, we extract the following two structural observations of the system dynamics. Observation 1 (Monotonicity). The identified model q~ q [cf. (6)] is nondecreasing on [0,1][0,1]; see Fig. 4. Since the pointwise minimum and maximum of two nondecreasing functions are also nondecreasing, it follows that f(x,w)=min1,maxx,q~(x)+wf(x,w)= \1, \x,\, q(x)+w\\ is nondecreasing in x. Observation 2 (Diminishing returns). The difference q~(x)−x q(x)-x [cf. (6)] is nonincreasing on [0,1][0,1]; see Fig. 4. Since the operations min1−x,… \1-x,…\ and max0,… \0,…\ preserve this nonincreasing structure, it follows that f(x,w)−x=min1−x,max0,q~(x)−x+wf(x,w)-x= \1-x, \0, q(x)-x+w\\ is nonincreasing in x. These observations will be exploited in the next section to derive theoretical properties of an optimal stopping policy. VI Structure of an Optimal Stopping Policy Leveraging the identified system dynamics (see Fig. 4), we now derive structural properties of an optimal stopping policy. In the following analysis, we treat Observations 1–2 as exact characteristics of the system function f [cf. (1)] and assume the linear payoff structure defined in §V-C. That is, we model the GP posterior mean q~ q (which is continuous) as the score function q in (6) and define the payoff function g as g(x)=βxg(x)=β x, where β>0β>0 is a weighting factor. Lemma 1. For each stage k∈0,1,…,Nk∈\0,1,…,N\, 1. The value function Vk⋆(x)V_k (x) is nondecreasing in x. 2. The difference Vk⋆(x)−g(x)V_k (x)-g(x) is nonincreasing in x. 3. The functions Vk⋆V_k and Qk⋆Q _k (for k<Nk<N) are continuous. Proof. We proceed by induction on k. Base case: VN⋆(x)=g(x)=βxV_N (x)=g(x)=β x is nondecreasing and continuous, and VN⋆(x)−g(x)=0V_N (x)-g(x)=0 is nonincreasing. Inductive step: Assume that Vk+1⋆V_k+1 is nondecreasing and continuous, and that Vk+1⋆(y)−g(y)V_k+1 (y)-g(y) is nonincreasing in y. Since f(x,w)f(x,w) is nondecreasing in x by Observation 1 and Vk+1⋆V_k+1 is nondecreasing by the inductive hypothesis, Qk⋆(x)=−c+EVk+1⋆(f(x,wk))Q _k(x)=-c+E\V_k+1 (f(x,w_k))\ is nondecreasing in x. Since g(x)g(x) is also nondecreasing, Vk⋆(x)=maxg(x),Qk⋆(x)V_k (x)= \g(x),\,Q _k(x)\ is nondecreasing. This completes the proof of the first property. For the second property, we start by expanding Vk⋆(x)−g(x)V_k (x)-g(x) as Vk⋆(x)−g(x)=max0,Qk⋆(x)−g(x)V_k (x)-g(x)= \0,\,Q _k(x)-g(x)\, which implies that it suffices to show that Qk⋆(x)−g(x)Q _k(x)-g(x) is nonincreasing. We expand this expression as Qk⋆(x)−g(x) Q _k(x)-g(x) =−c+EVk+1⋆(f(x,wk))−g(f(x,wk)) =-c+E \V_k+1 (f(x,w_k))-g(f(x,w_k)) \ +βEf(x,wk)−x. +β\,E \f(x,w_k)-x \. By the inductive hypothesis, Vk+1⋆(y)−g(y)V_k+1 (y)-g(y) is nonincreasing in y. Since f(x,w)f(x,w) is nondecreasing in x by Observation 1, the first expectation is nonincreasing in x. By Observation 2, f(x,wk)−xf(x,w_k)-x is nonincreasing in x, and so is the second expectation. As the sum of nonincreasing functions is nonincreasing, Qk⋆(x)−g(x)Q _k(x)-g(x) is nonincreasing in x. For the third property, f(x,w)=min1,maxx,q(x)+wf(x,w)= \1, \x,\,q(x)+w\\ is continuous in x for each w since q is continuous. Hence, since Vk+1⋆V_k+1 is continuous by the inductive hypothesis, x↦Vk+1⋆(f(x,w))x V_k+1 (f(x,w)) is continuous and bounded for each w. As a consequence, Qk⋆(x)=−c+EVk+1⋆(f(x,wk))Q _k(x)=-c+E\V_k+1 (f(x,w_k))\ is continuous by the dominated convergence theorem. Therefore, Vk⋆=maxg,Qk⋆V_k = \g,\,Q _k\ is the maximum of two continuous functions, which implies that Vk⋆V_k is continuous. ∎ Given the above lemma, there exists an optimal policy with threshold structure, as formally stated below. Proposition 2. There exist an optimal policy μ⋆μ and thresholds α0,α1,…,αN−1∈[0,1] _0, _1,…, _N-1∈[0,1] such that μk⋆(x)=if x≥αk,if x<αk,k=0,1,…,N−1. _k (x)= dcases S&if x≥ _k,\\ C&if x< _k, dcases k=0,1,…,N-1. (8) Proof. The optimal policy stops at stage k if and only if g(x)≥Qk⋆(x)g(x)≥ Q _k(x). By Lemma 1, Qk⋆(x)−g(x)Q _k(x)-g(x) is nonincreasing in x. Therefore, if g(x′)≥Qk⋆(x′)g(x )≥ Q _k(x ) for some x′∈[0,1]x ∈[0,1], then g(x)≥Qk⋆(x)g(x)≥ Q _k(x) for all x≥x′x≥ x . This means that the stopping set k⋆=x∣x∈[0,1],g(x)≥Qk⋆(x) S_k =\x x∈[0,1],g(x)≥ Q _k(x)\ is an upper set, i.e., if x∈k⋆x∈ S_k then x′∈k⋆x ∈ S_k for all x′∈[x,1]x ∈[x,1]. It remains to show that k⋆ S_k is nonempty and closed. For nonemptiness, note that f(1,w)=1f(1,w)=1 for all w [cf. (6)]. Hence, any policy that continues j≥0j≥ 0 times from state 11 receives g(1)−jcg(1)-jc, which is maximized at j=0j=0. Therefore Vk+1⋆(1)=g(1)V _k+1(1)=g(1), which gives Qk⋆(1)=g(1)−c<g(1)Q _k(1)=g(1)-c<g(1) and thus 1∈k⋆1∈ S_k . Further, k⋆ S_k is closed since g−Qk⋆g-Q _k is continuous by Lemma 1. A nonempty closed upper subset of [0,1][0,1] has the form [αk,1][ _k,1] with αk=mink⋆ _k= S_k , which yields (8). ∎ Proposition 3. The optimal thresholds [cf. (8)] are nonincreasing in the stage k, i.e., α0≥α1≥⋯≥αN−1 _0≥ _1≥·s≥ _N-1. Proof. We start by showing that Vk⋆(x)≥Vk+1⋆(x)V_k (x)≥ V_k+1 (x) for all stages k∈0,1,…,N−1k∈\0,1,…,N-1\ and states x∈[0,1]x∈[0,1]. Base case (k=N−1k=N-1): We have VN−1⋆(x)=maxg(x),−c+Eg(f(x,w))≥g(x)=VN⋆(x)V_N-1 (x)= \g(x),\,-c+E\g(f(x,w))\\≥ g(x)=V_N (x). Inductive step: Assume Vk+1⋆(x)≥Vk+2⋆(x)V_k+1 (x)≥ V_k+2 (x) for all states x∈[0,1]x∈[0,1]. Then Qk⋆(x) Q _k(x) =−c+EVk+1⋆(f(x,w)) =-c+E\V_k+1 (f(x,w))\ ≥−c+EVk+2⋆(f(x,w)) ≥-c+E\V_k+2 (f(x,w))\ =Qk+1⋆(x). =Q _k+1(x). Consequently, Vk⋆(x) V_k (x) =maxg(x),Qk⋆(x) = \g(x),\,Q _k(x)\ ≥maxg(x),Qk+1⋆(x) ≥ \g(x),\,Q _k+1(x)\ =Vk+1⋆(x). =V_k+1 (x). Since Qk⋆(x)≥Qk+1⋆(x)Q _k(x)≥ Q _k+1(x), we have k⋆ S_k =x∣x∈[0,1],g(x)≥Qk⋆(x) =\x x∈[0,1],g(x)≥ Q _k(x)\ ⊆x∈[0,1]∣g(x)≥Qk+1⋆(x)=k+1⋆. \x∈[0,1] g(x)≥ Q _k+1(x)\= S_k+1 . Taking minima of k⋆ S_k yields αk≥αk+1 _k≥ _k+1. ∎ The preceding propositions imply that each stopping set k⋆ S_k has the form [αk,1][ _k,1] for some threshold αk∈[0,1] _k∈[0,1]. We now show that these sets are all equal. Figure 5: Convergence curves of three simulation-based optimization methods (SPSA [34], Cross-entropy method [31], and Differential evolution [35]) for computing the optimal threshold α⋆α [cf. Prop. 4] across the three foundation models and four values of the payoff weight β (defined in §V-C). Curves show mean values and shaded regions indicate one standard deviation over 3 runs with different random seeds. The y-axes indicate the expected value Eg(xτμα)−cτμαE\g(x_ _ _α)-c _ _α\ (i.e., the expected payoff minus the refinement costs) and the x-axes indicate the computation times in seconds. Proposition 4. The optimal stopping sets satisfy 0⋆=1⋆=⋯=N−1⋆ S_0 = S_1 =·s= S_N-1 . Equivalently, the optimal thresholds are stage-independent, i.e., α0=α1=⋯=αN−1=α⋆ _0= _1=·s= _N-1=α . Proof. By Prop. 3, we have k⋆⊆k+1⋆ S_k S_k+1 for all stages k∈0,1,…,N−2k∈\0,1,…,N-2\. It remains to show the reverse inclusion. Fix k∈0,1,…,N−2k∈\0,1,…,N-2\ and let x∈k+1⋆x∈ S_k+1 , so that x≥αk+1x≥ _k+1 and g(x)≥Qk+1⋆(x)g(x)≥ Q _k+1(x). Since f(x,w)=min1,maxx,q(x)+w≥x≥αk+1≥αk+2f(x,w)= \1, \x,\,q(x)+w\\≥ x≥ _k+1≥ _k+2 (where αN=0 _N=0 since stopping is enforced at stage N), we have f(x,w)∈k+2⋆f(x,w)∈ S_k+2 for all realizations of w, and therefore Vk+2⋆(f(x,w))=g(f(x,w))V_k+2 (f(x,w))=g(f(x,w)). Similarly, f(x,w)∈k+1⋆f(x,w)∈ S_k+1 , so Vk+1⋆(f(x,w))=g(f(x,w))V_k+1 (f(x,w))=g(f(x,w)). It follows that Qk⋆(x) Q _k(x) =−c+EVk+1⋆(f(x,w))=−c+Eg(f(x,w)) =-c+E\V_k+1 (f(x,w))\=-c+E\g(f(x,w))\ =−c+EVk+2⋆(f(x,w))=Qk+1⋆(x). =-c+E\V_k+2 (f(x,w))\=Q _k+1(x). Since g(x)≥Qk+1⋆(x)=Qk⋆(x)g(x)≥ Q _k+1(x)=Q _k(x), we have x∈k⋆x∈ S_k . Thus k+1⋆⊆k⋆ S_k+1 S_k , which together with k⋆⊆k+1⋆ S_k S_k+1 gives k⋆=k+1⋆ S_k = S_k+1 for every k∈0,1,…,N−2k∈\0,1,…,N-2\. ∎ Propositions 2–4 establish that, given the identified dynamics in Fig. 4 and assuming that Observations 1–2 are exact properties of the system function, an optimal stopping policy is characterized by a single threshold α⋆∈[0,1]α ∈[0,1]. This structure implies a simple method for computing an optimal policy: parameterize the stopping policy μ by a threshold α [cf. (8)] and estimate the optimal threshold α⋆α by maximizing the expected value Eg(xτμα)−cτμαE\g(x_ _ _α)-c _ _α\, where μα _α denotes the parameterized policy. This optimization can be efficiently performed using simulation-based stochastic approximation methods, as detailed in the following section. VII Computing an Optimal Stopping Policy We now exploit the structural properties established in the preceding section to compute an optimal stopping policy. By Props. 2–4, this computation reduces to optimizing the stopping threshold α, which we use to parameterize a threshold-based stopping policy μα _α; cf. (8). We implement this optimization using three methods: simultaneous perturbation stochastic approximation (SPSA) [34], the cross-entropy method [31], and differential evolution [35]. For each method, the value Eg(xτμα)−cτμαE \g(x_ _ _α)-c _ _α \ is estimated via simulation of the Gaussian process model identified in §V-C. We run all simulations and optimizations on an M4 Pro Chip with macOS Sequoia 15.6.1 and Python 3.11. Figure 5 shows the convergence curves for varying values of the payoff weight β, as defined in §V-C. All methods converge to similar expected values, which suggests that they identify optimal or near-optimal thresholds. In all cases, convergence is achieved within seconds, which demonstrates the computational efficiency afforded by the threshold structure. Figure 6 shows the optimized thresholds for each model and payoff weight β. We observe that the thresholds increase with β, which is expected. As the relative importance of payoff grows, the policy becomes more selective and demands higher scores before stopping. For small β (e.g., β=0.01β=0.01), the thresholds are low, indicating that the cost of refinement dominates and early stopping is preferred. Across the three foundation models, haiku 4.5 admits lower thresholds than the other models for small β, which is consistent with its higher continuation cost; cf. Table I. For large β, however, haiku 4.5 exhibits higher thresholds than the other models. Figure 6: The optimized threshold α across the three foundation models and four values of the payoff weight β (defined in §V-C). Figure 7: Evaluation results on effibench [18]. Each group of bars shows the expected value Eg(xτ)−cτE\g(x_τ)-cτ\ for the optimized threshold policy μα _α, the UCB baseline, and fixed-iteration stopping policies with 11 to 66 iterations. Rows correspond to the three foundation models. Columns indicate the payoff weight β (defined in §V-C). Bar heights indicate the mean over evaluations with three different random seeds; error bars indicate one standard deviation. VIII Experimental Evaluation on Effibench In this section, we evaluate the optimized stopping policy described in the preceding section on effibench [18] and compare it against stopping policies proposed in prior work. Baseline stopping policies. We consider two categories of baselines. The first category consists of fixed-iteration policies, which stop after a predetermined number of iterations. This is the most common approach in the self-refinement literature; see e.g., [17, 22, 42, 43, 27]. The second category is the upper confidence bound (UCB) policy proposed by Sun et al. [36], which adaptively decides when to stop based on confidence bounds on the values of a finite set of stopping thresholds. We instantiate this policy by defining the set of thresholds as α∈0,0.1,0.2,…,1α∈\0,0.1,0.2,…,1\ and selecting the threshold with the highest upper confidence bound. Experiment setup. We evaluate all stopping policies on 5050 problems from effibench [18]. (These problems are disjoint from the problems used for system identification in §V-C.) Each stopping policy is applied to the three foundation models with a maximum of N=10N=10 refinement iterations. We run each method three times with different random seeds and report the average performance. Evaluation results. Figure 7 shows the expected value Eg(xτμ)−cτμE\g(x_ _μ)-c _μ\ for each stopping policy μ across the three foundation models and four values of the payoff weight β. We observe that the optimized threshold policy μα _α achieves the highest expected value in all configurations. For the fixed-iteration policies, we find that the best policy varies with β. In particular, we observe that fewer iterations perform better when β is small, while more iterations are preferred when β is large. Finally, our experimental results show that the UCB policy automatically adapts to β but consistently underperforms the optimized threshold policy. IX Discussion of the Experimental Results The main takeaways from the theoretical and experimental results can be summarized as follows. 1. Self-refinement exhibits diminishing returns. The empirical measurements in Fig. 3 show that all three foundation models improve output quality through self-refinement, but with progressively smaller gains per refinement iteration. Meanwhile, token consumption and monetary cost grow approximately linearly. 2. Optimal threshold-based stopping policies exist. Propositions 2–4 establish that, under the conditions in Observations 1–2, an optimal stopping policy is characterized by a single, stage-independent threshold α⋆α . This reduces the policy search from a sequence of functions μ0,…,μN−1 _0,…, _N-1 to a scalar optimization problem, which can be efficiently solved; see Fig. 5. 3. Optimal stopping improves over the state-of-the-art. The most common approach in the self-refinement literature is to stop after a fixed number of iterations. As shown in Fig. 7, the optimized threshold policy consistently outperforms this approach. X Conclusion Foundation models can improve their outputs through a self-refinement process where they iteratively critique and refine their own outputs. We show that this process can be formulated as an optimal stopping problem in which the decision to continue refining or to stop is made sequentially based on the expected improvement relative to the refinement cost. We establish conditions for optimal threshold-based stopping policies and validate them on a coding benchmark across three frontier models. Empirical results demonstrate that our stopping policy significantly improves cost-efficiency compared to the state-of-the-art. These results show the value of leveraging decision theory for efficient control of foundation models. As these models are increasingly deployed in large-scale automation pipelines with thousands of daily invocations, we believe that our optimal stopping approach can translate into substantial monetary savings. Future work. A natural direction for future work is to evaluate our approach on benchmark types other than coding benchmarks (e.g., mathematical reasoning benchmarks) and to instantiate it with foundation models other than large language models (e.g., time-series models or multi-modal models). From a theoretical perspective, our structural results (Props. 2–4) rely on Observations 1–2 holding exactly. Since the system function is estimated via a Gaussian process, a promising direction for further analysis is to relax these assumptions by exploiting the posterior uncertainty of the GP to derive probabilistic guarantees on the threshold structure. References [1] J. Alayrac et al. (2022) Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 23716–23736. Cited by: §I. [2] Anthropic (2024) The Claude 3 model family. Technical report Cited by: §V-B. [3] S. P. Arango, P. Mercado, S. Kapoor, A. F. Ansari, L. Stella, H. Shen, H. H. J. Senetaire, A. C. Turkmen, O. Shchur, D. C. Maddix, M. Bohlke-Schneider, B. Wang, and S. S. Rangapuram (2025-03–05 May) ChronosX: adapting pretrained time series models with exogenous variables. In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, Y. Li, S. Mandt, S. Agrawal, and E. Khan (Eds.), Proceedings of Machine Learning Research, Vol. 258, p. 2242–2250. Cited by: §I. [4] J. Bather (2000) Decision theory: an introduction to dynamic programming and sequential decisions. John Wiley and Sons, Inc., USA. External Links: ISBN 0471976490 Cited by: §I. [5] D. P. Bertsekas (2005) Dynamic programming and optimal control. 3rd edition, Vol. I, Athena Scientific, Belmont, MA, USA. Cited by: §I. [6] T. Brown et al. (2020) Language models are few-shot learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33, p. 1877–1901. Cited by: §I. [7] Y. Chow, H. Robbins, and D. Siegmund (1971) Great expectations: the theory of optimal stopping. Houghton Mifflin. External Links: ISBN 0395053145 Cited by: §I. [8] E. B. Dynkin (1963) The optimum choice of the instant for stopping a Markov process. Sov. Math., Dokl. 4, p. 627–629. Note: Translation from Dokl. Akad. Nauk SSSR 150, 238–240 (1963) Cited by: Remark 2. [9] Gemini Team, Google et al. (2024) Gemini: a family of highly capable multimodal models. External Links: 2312.11805 Cited by: §I, §I, §V-B. [10] K. Hammar, T. Alpcan, and E. C. Lupu (2026) Incident response planning using a lightweight large language model with reduced hallucination. In 33rd Annual Network and Distributed System Security Symposium, NDSS 2026, San Diego, California, USA, February 23-27, 2026, Cited by: §I. [11] K. Hammar and T. Alpcan (2026-Mar.) Scalable solutions to zero-sum partially observable stochastic games through belief aggregation with approximation guarantees. Proceedings of the AAAI Conference on Artificial Intelligence 40 (20), p. 17023–17031. External Links: Document Cited by: §I. [12] K. Hammar, T. Li, R. Stadler, and Q. Zhu (2025) Adaptive security response strategies through conjectural online learning. IEEE Transactions on Information Forensics and Security 20 (), p. 4055–4070. External Links: Document Cited by: §I. [13] K. Hammar, Y. Li, T. Alpcan, E. C. Lupu, and D. P. Bertsekas (2026) Adaptive network security policies via belief aggregation and rollout. IEEE Transactions on Information Forensics and Security 21 (), p. 6544–6559. External Links: Document Cited by: §I. [14] K. Hammar and R. Stadler (2022) Intrusion prevention through optimal stopping. IEEE Transactions on Network and Service Management 19 (3), p. 2333–2348. External Links: Document Cited by: §I. [15] K. Hammar (2026) Supplementary material of the paper ”Optimal Stopping of Self-Refining Foundation Models”. Note: Code: https://github.com/Kim-Hammar/optimal_stopping_foundation_models Cited by: 2nd item. [16] J. He, C. Treude, and D. Lo (2025) LLM-based multi-agent systems for software engineering: literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology 34 (5), p. 1–30. Cited by: §I. [17] D. Huang, J. Dai, H. Weng, P. Wu, Y. Qing, H. Cui, Z. Guo, and J. M. Zhang (2024) EffiLearner: enhancing efficiency of generated code via self-optimization. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 84482–84522. External Links: Document Cited by: §I, §VIII. [18] D. Huang, Y. Qing, W. Shang, H. Cui, and J. M. Zhang (2024) EffiBench: benchmarking the efficiency of automatically generated code. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, Vol. 37, p. 11506–11544. Cited by: Figure 3, Figure 4, §V-A, §V-B, §V-C, Figure 7, §VIII, §VIII. [19] V. Krishnamurthy (2016) Partially observed markov decision processes: from filtering to controlled sensing. Cambridge University Press. External Links: Document Cited by: §I. [20] H. Liu, Y. Li, J. Mårtensson, L. Xie, and K. H. Johansson (2020) Reinforcement learning based approach for flip attack detection. In 2020 59th IEEE Conference on Decision and Control (CDC), p. 3212–3217. External Links: Document Cited by: §I. [21] C. Lu, C. Lu, R. T. Lange, Y. Yamada, S. Hu, J. Foerster, D. Ha, and J. Clune (2026-03-01) Towards end-to-end automation of AI research. Nature 651 (8107), p. 914–919. External Links: ISSN 1476-4687, Document Cited by: §I. [22] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark (2023) Self-Refine: iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, Vol. 36, p. 46534–46594. Cited by: §I, §VIII. [23] M. F. Maleki and R. Zhao (2024) Procedural content generation in games: a survey with insights on emerging LLM integration. In Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Vol. 20, p. 167–178. Cited by: §I. [24] J. Milošević, D. Umsonst, H. Sandberg, and K. H. Johansson (2018) Quantifying the impact of cyber-attack strategies for control systems equipped with an anomaly detector. In 2018 European Control Conference (ECC), Vol. , p. 331–337. External Links: Document Cited by: §I. [25] OpenAI, J. Achiam, S. Adler, et al. (2024) GPT-4 technical report. External Links: 2303.08774 Cited by: §V-B. [26] G. Peng and Q. Zhu (2020) Sequential hypothesis testing game. In 2020 54th Annual Conference on Information Sciences and Systems (CISS), Vol. , p. 1–6. External Links: Document Cited by: §I. [27] A. Piché, A. Milios, D. Bahdanau, and C. Pal (2025) LLMs can learn self-restraint through iterative self-reflection. Transactions on Machine Learning Research. Note: External Links: ISSN 2835-8856 Cited by: §VIII. [28] M. Rabi and K. H. Johansson (2008) Event-triggered strategies for industrial control over wireless networks. In Proceedings of the 4th Annual International Conference on Wireless Internet, WICON ’08. External Links: ISBN 9789639799363 Cited by: §I. [29] C. E. Rasmussen and C. K. I. Williams (2006) Gaussian processes for machine learning. The MIT Press. Cited by: footnote 2. [30] X. Ren, K. H. Johansson, and L. Shi (2017) Quickest change detection with observation scheduling. IEEE Transactions on Automatic Control 62 (6), p. 2635–2647. External Links: Document Cited by: §I. [31] R. Rubinstein (1999) The cross-entropy method for combinatorial and continuous optimization. Methodology and Computing in Applied Probability 1 (2), p. 127–190. Cited by: Figure 5, §VII. [32] A. N. Shirayev (2007) Optimal stopping rules. Springer-Verlag Berlin. Note: Reprint of Russian edition from 1969 Cited by: §I. [33] J. L. Snell (1952) Applications of martingale system theorems. Transactions of the American Mathematical Society 73. Cited by: §I, Remark 2. [34] J. C. Spall (1992) Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37 (3), p. 332–341. Cited by: Figure 5, §VII. [35] R. Storn and K. Price (1997-12-01) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11 (4), p. 341–359. External Links: ISSN 1573-2916, Document Cited by: Figure 5, §VII. [36] R. Sun, W. Cheng, D. Li, H. Chen, and W. Wang (2026) Stop when enough: adaptive early-stopping for chain-of-thought reasoning. In Proc. 64th Annual Meeting of the Association for Computational Linguistics (ACL), San Diego, CA, USA, p. 27250–27268. Cited by: §VIII. [37] A. G. Tartakovsky, B. L. Rozovskii, R. B. Blažek, and H. Kim (2006) Detection of intrusions in information systems by sequential change-point methods. Statistical Methodology. External Links: ISSN 1572-3127, Document Cited by: §I. [38] V. V. Veeravalli, G. Fellouris, and G. V. Moustakides (2024) Quickest change detection with controlled sensing. IEEE Journal on Selected Areas in Information Theory 5 (), p. 1–11. External Links: Document Cited by: §I. [39] A. Wald (1947) Sequential analysis. Wiley and Sons, New York. Cited by: §I. [40] Y. Wang, Y. Wu, Z. Wei, S. Jegelka, and Y. Wang (2024) A theoretical understanding of self-correction through in-context alignment. In Advances in Neural Information Processing Systems, Vol. 37. Cited by: §I. [41] Y. Weng, M. Zhu, F. Xia, B. Li, S. He, S. Liu, B. Sun, K. Liu, and J. Zhao (2023-12) Large language models are better reasoners with self-verification. In Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali (Eds.), p. 2550–2575. External Links: Document Cited by: §I. [42] E. Zelikman, E. Lorch, L. Mackey, and A. T. Kalai (2024) Self-taught optimizer (STOP): recursively self-improving code generation. In First Conference on Language Modeling (COLM), Cited by: §VIII. [43] Z. Zhou, J. Song, K. Yao, Z. Shu, and L. Ma (2024) ISR-LLM: iterative self-refined large language model for long-horizon sequential task planning. In 2024 IEEE International Conference on Robotics and Automation (ICRA), Vol. , p. 2081–2088. External Links: Document Cited by: §VIII.