Paper deep dive
Contracting for LLM Delegation: Moral Hazard in Technology and Effort Choice
Nanda Kishore Sreenivas, Kate Larson
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 8/20/2026, 3:43:48 AM
Summary
This paper extends the Principal-Agent framework to address moral hazard in Large Language Model (LLM) delegation, where agents choose both a technology (model) and an effort level (token budget). The authors model output quality as a concave, saturating function of effort and derive an optimal linear contract. They demonstrate that an agent's best response involves switching between models based on a threshold reward share. The theoretical model is calibrated using open-weight LLM pairings on MATH and MMLUPro benchmarks, showing that bandit algorithms converge to strategies aligning with the theoretical equilibrium, suggesting simple linear contracts effectively incentivize technology-aware delegation.
Entities (9)
Relation Signals (7)
Nanda Kishore Sreenivas → affiliatedwith → University of Waterloo
confidence 99% · Nanda Kishore Sreenivas ... Affiliation: University of Waterloo
Kate Larson → affiliatedwith → University of Waterloo
confidence 99% · Kate Larson ... Affiliation: University of Waterloo
Principal-Agent Framework → extendedby → LLM Delegation
confidence 95% · We extend the standard Principal-Agent framework to scenarios where the Agent selects from a suite of technologies
Linear Contract → usedtoincentivize → Technology-Aware Delegation
confidence 93% · simple linear contracts can effectively incentivize complex, technology-aware delegation in agentic workflows
MMLUPro Benchmark → usedforcalibration → Principal-Agent Model
confidence 92% · calibrate our model using open-weight LLM pairings across the MATH and MMLUPro benchmarks
MATH Benchmark → usedforcalibration → Principal-Agent Model
confidence 92% · calibrate our model using open-weight LLM pairings across the MATH and MMLUPro benchmarks
Bandit Algorithms → convergeto → Theoretical Equilibrium
confidence 90% · both Principal and Agent, when employing bandit algorithms to navigate this environment, converge to strategies that closely align with our theoretical equilibrium
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:We extend the standard Principal-Agent framework to scenarios where the Agent selects from a suite of technologies, each characterized by a distinct cost-capability profile. This framework is increasingly critical in the era of Large Language Models (LLMs), where Agents choose both a model and an associated effort level (e.g., token budget). We model the relationship between output quality and effort as a concave, saturating function, which depends on the Agent's hidden two-dimensional action choice balancing technology selection and effort allocation. We derive the optimal linear contract for the Principal, demonstrating that the Agent's best response is characterized by a threshold reward share that triggers technology switching. Finally, we calibrate our model using open-weight LLM pairings across the MATH and MMLUPro benchmarks. We show that both Principal and Agent, when employing bandit algorithms to navigate this environment, converge to strategies that closely align with our theoretical equilibrium. These results suggest that simple linear contracts can effectively incentivize complex, technology-aware delegation in agentic workflows.
Tags
Links
- Source: https://arxiv.org/abs/2608.18232v1
- Canonical: https://arxiv.org/abs/2608.18232v1
Trouble viewing inline? Open PDF directly →
Full Text
75,263 characters extracted from source content.
Expand or collapse full text
Contracting for LLM Delegation: Moral Hazard in Technology and Effort Choice Nanda Kishore Sreenivas Kate Larson Affiliation: University of Waterloo Affiliation: nksreenivas, kate.larson@uwaterloo.ca Abstract We extend the standard Principal-Agent framework to scenarios where the Agent selects from a suite of technologies, each characterized by a distinct cost-capability profile. This framework is increasingly critical in the era of Large Language Models (LLMs), where Agents choose both a model and an associated effort level (e.g., token budget). We model the relationship between output quality and effort as a concave, saturating function, which depends on the Agent’s hidden two-dimensional action choice balancing technology selection and effort allocation. We derive the optimal linear contract for the Principal, demonstrating that the Agent’s best response is characterized by a threshold reward share that triggers technology switching. Finally, we calibrate our model using open-weight LLM pairings across the MATH and MMLUPro benchmarks. We show that both Principal and Agent, when employing bandit algorithms to navigate this environment, converge to strategies that closely align with our theoretical equilibrium. These results suggest that simple linear contracts can effectively incentivize complex, technology-aware delegation in agentic workflows. 1 Introduction The rapid capability improvements in large language models (LLMs) in recent years have fundamentally changed how tasks are executed: from reactive, single-prompt chat interfaces toward delegation to autonomous agents. Principals, whether individual users or firms, now routinely hand off complex, open-ended tasks to specialized AI Agents. This is evident in the rise of both generalized and domain-specific tools, e.g., legal firms delegate due diligence to systems like Harvey [13], software teams assign bug fixes to autonomous coding agents like Devin [8], etc. In these interactions, the Agent operates as a black box; the Principal provides the objective and receives the final output, completely blind to the internal reasoning process and choices. As this ecosystem matures, delegation will increasingly occur not just from humans to AI, but from AI to AI across open ‘agentic markets’. Orchestrator algorithms will dynamically delegate specialized sub-tasks to agents to minimize computation costs or leverage domain expertise [12, 25]. In these agent-oriented markets, the two transacting parties possess distinct economic identities and potentially misaligned utilities [22]. This economic reality exposes an issue with the current standard of “pay-per-token” or pay-for-compute API pricing. When an autonomous Agent is billed based on its computational effort, it creates a severe moral hazard. As Bu and Ma note [7], token-based pricing under-incentivizes hidden effort and misaligns the Agent’s objectives with those of the Principal. An Agent paid per token is incentivized to maximize verbosity, unnecessarily “overthink” simple problems, or covertly utilize cheaper, lower-capability underlying models to save on its own costs [26, 24]. Consequently, delegating a task to an autonomous Agent increasingly means delegating a choice of which tool to use, not just how hard to work. This is a moral hazard problem, but not one the classical Principal-Agent literature typically addresses. We formalize this with a linear contract (parametrized by α∈[0,1]α∈[0,1]) between a Principal and an Agent who selects a model m and a token budget x, where output quality follows a saturating, diminishing-returns production function in x which is a natural fit for LLM inference. Under this model, we characterize the Agent’s best response as a threshold in the linear contract at which the Agent’s optimal model choice switches between models, and we also derive the Principal’s optimal linear contract. We calibrate the production function on six pairings of open-weight models spanning three model families, across two task domains (MATH and MMLUPro), and show that a Principal and Agent using simple online learning algorithms converge to contracts and best responses close to our theoretical predictions. 2 Related Work Our work sits at the intersection of several areas: classical and algorithmic contract theory, anytime algorithms, inference-time compute allocation, and LLM mechanism design. The classical principal-agent model [16] studies how a Principal can incentivize an Agent whose action or effort is hidden. Algorithmic contract theory [10] extends this to discrete, combinatorial action spaces and multiple outcomes, but treats the technology matrix (the action-outcome probabilities) as exogenous and fixed. A related thread studies online learning of contracts focusing on questions of learnability and regret, typically over discrete action and outcome spaces [32, 2, 3]. Existing work, including the multitask model of Holmstrom and Milgrom [15], decomposes a single technology into multiple effort or task dimensions; none lets the Agent choose which technology governs the effort-to-outcome mapping. The anytime algorithm literature studies systems that can be interrupted at any point, returning an output whose quality improves with computation time [33, 18]. This relationship is formalized via performance profiles, concave and monotonically increasing functions mapping allocated compute to expected output quality. We borrow this object to model the production function of an LLM. This literature typically treats resource allocation as a single-agent deliberation [17, 6]. Since our model uses common performance profiles from this literature, our results apply here too. Recent LLM research investigates how LLMs can dynamically scale their inference-time compute based on task difficulty. AnytimeReasoner [21] trains a single model via RL to produce a usable answer at any token budget; related approaches similarly train models to regulate their own reasoning length [29, 1], and document diminishing and eventually saturating returns to additional tokens. We rely on these findings to motivate the shape of our production curves. There are also recent frameworks for model routing with the objective of minimizing inference costs from a single-agent perspective [9, 19, 30]. Our work also relates to the emerging application of mechanism design to LLMs. Much of this literature focuses on the seller’s side, analyzing how an LLM provider screens buyers who have different task requirements [4, 5]. Dutting et al. [11] explore token-level auctions to influence the output of LLMs for applications like ad generation. Contemporary work explores contracts over LLM generation, focusing on how and when the Principal should verify the Agent’s work when verification is costly [24]. Closest in spirit to our application is the work of Saig et al. [23], who motivate ‘pay-for-performance’ contracts and design threshold and monotone contracts for an Agent choosing among a discrete menu of LLMs, robust to unknown Agent costs, over a discrete space of quality levels. We build on this premise of contracting over AI generation but alter the fundamental mechanics. The Agent is a task/domain specialist using publicly available models and, therefore, costs are known in our theoretical model. We also introduce an explicit continuous, unobservable token budget nested within the discrete model choice. 3 Model Figure 1: Principal-Agent Model with hidden model choice and effort. A Principal P delegates a task to an Agent A. The Agent is an anytime reasoner: it selects a language model m from a finite set of models and allocates token budget x≥0x≥ 0 to inference. Output quality q is a function of m and x, and only the quality is observable by P; not the Agent’s model choice or token budget. We assume that the Agent specializes in a specific task domain, and therefore, all contracted tasks are reasonably similar. The overall setup is shown in Figure 1. The Principal offers a linear contract11 1 We explore fixed payments in Appendix C and show that, under limited liability, it is equivalent to this reward sharing model. with reward share α∈[0,1]α∈[0,1]. The Agent receives α⋅qα· q and the Principal retains (1−α)⋅q(1-α)· q. The contract is over output q only; model choice and token count are not directly observable, and therefore, not contractible. Let ℳM be the set of available models, and models are arranged in increasing order of capability in the task domain. We generalize to multiple models in Appendix H. For clarity, we present the two-model scenario with ℳ=L,HM=\L,H\, where L represents the cheaper model with lower capability and H denotes a more capable and more expensive model. For example, L could be a smaller model that is optimized for edge applications, while H could be an expensive, slower frontier model. Alternatively, L could be a simple, instruction-tuned chat model while H could have other capabilities such as reasoning, tool calls, etc. Each model m∈L,Hm∈\L,H\ is characterized by a set of performance and cost parameters. We assume linear cost per token, and the output q is measured in terms of accuracy (as a percent, q∈[0,100]q∈[0,100])22 2 Measuring quality of models is outside the scope of this paper.. The Agent’s two decisions are: • Model choice: m∈L,Hm∈\L,H\. • Effort: token count x≥0x≥ 0. The utilities of the Agent and Principal are given by: U¯=α⋅v⋅qm(x)−c¯mx,V¯=(1−α)⋅v⋅qm(x). U=α· v· q_m(x)- c_mx, V=(1-α)· v· q_m(x). where m is the model choice and c¯m c_m is the true cost per token. We assume that the monetary return to the Principal scales linearly with accuracy by a factor v. We normalize utilities by dividing by the Principal’s valuation parameter v throughout33 3 This assumes that the Principal’s task valuation v is public, which may not always be true. The information asymmetry over the Principal’s ‘type’ forces the Agent to ‘screen’ the Principal. We leave this for future work. to get: U(α,m,x) U(α,m,x) =α⋅qm(x)−cmx, =α· q_m(x)-c_mx, V(α,m,x) V(α,m,x) =(1−α)⋅qm(x). =(1-α)· q_m(x). (1) where cmc_m denotes the normalized cost per token for model m, i.e., cm=c¯m/vc_m= c_m/v. We will use these normalized utilities for the Agent and Principal in the rest of this paper. 3.1 Optimal Effort To further analyse the Agent’s optimal model choice and effort level, we model the output function qm(x)q_m(x) as a saturation function, where MmM_m is the capability ceiling (in terms of accuracy on the task) and kmk_m is the saturation rate of model m. qm(x)=Mm(1−e−kmx) q_m(x)=M_m\! (1-e^-k_mx ) (2) Note that the function is increasing, concave, and has an asymptotic upper bound MmM_m, and this shape is consistent with empirical studies of LLM inference. Several papers report that accuracy stagnates and the marginal returns diminish at higher token budgets [28, 31, 21]. Beyond LLMs, the anytime algorithm literature has also used similar curves to model performance profiles [33, 6]. We assume that the model H is more capable (in the limit) and also costs more per token. MH>MLcH>cLM_H>M_L\, c_H>c_L For a given model m and reward share α, the Agent solves: maxx≥0U(α,m,x)⟹maxx≥0αMm(1−e−kmx)−cmx. _x≥ 0\;U(α,m,x) _x≥ 0\;α M_m\! (1-e^-k_mx )-c_mx. The FOC gives: αMmkme−kmx∗=cm.α M_mk_me^-k_mx^*=c_m. Solving for the optimal effort for model m, xm∗x^*_m: xm∗=1kmln(αMmkmcm). x^*_m= 1k_m \! ( α M_mk_mc_m ). (3) This is valid (i.e. xm∗≥0x^*_m≥ 0) only when αMmkmcm≥1 α M_mk_mc_m≥ 1. Activation threshold τm _m is defined as the minimum contract share α at which it is viable for the Agent to use model m to start producing tokens. In other words, for any reward share less than τm _m, the marginal cost exceeds marginal benefit for model m. It is given by: τm=cmMmkm. _m= c_mM_mk_m. • If α≤τmα≤ _m: the Agent will not spend any tokens on reasoning, i.e., xm∗=0x^*_m=0 and UA=0U_A=0. • If α>τmα> _m: xm∗=1kmln(ατm)x^*_m= 1k_m \! ( α _m ). Substituting xm∗x^*_m into Eq. 1, and for simplicity, we denote U∗(α,m,xm∗)U^*(α,m,x^*_m) as Um(α)U_m(α) which is given by: Um(α)=Mm[α−τm−τmln(ατm)],α>τm. U_m(α)=M_m\! [α- _m- _m \! ( α _m ) ], α> _m. (4) Note that Um(τm)=Mm[τm−τm−τm⋅0]=0U_m( _m)=M_m[ _m- _m- _m· 0]=0. So utility is continuous at the threshold. The first derivative is: Um′(α)=Mm[1−τmα]U _m(α)=M_m\! [1- _mα ] It is positive when α>τmα> _m, and the second derivative is also positive, which implies Um(α)U_m(α) is convex and increasing. We have now derived the optimal token budget (xm∗x^*_m) and resulting utility Um(α)U_m(α) for the Agent using any model m given a linear contract α proposed by the Principal. 3.2 Switching Threshold θ Next, we characterize the optimal model choice of the Agent given the linear contract α. Specifically, at what value of α=θα=θ the Agent’s optimal model choice switches from one model to another. We consider two scenarios depending on which model activates first, i.e., which model requires the least reward share α to start producing tokens. (a) Standard Order τL<τH _L< _H (b) Flipped Order τL>τH _L> _H Figure 2: Illustration of model utilities for Agent Um(α)U_m(α) under both scenarios. Standard Order τL<τH _L< _H Model L activates earlier than H, and therefore, at τH _H, UL(τH)>0U_L( _H)>0, whereas model H is just activated, i.e., UH(τH)U_H( _H) = 0. We define the utility differential as: D(α)≡UH(α)−UL(α)D(α)≡ U_H(α)-U_L(α) (5) Differentiating D(α)D(α) yields: D′(α)=MH−ML−MHτH−MLτLαD (α)=M_H-M_L- M_H _H-M_L _Lα D′(α)=MHτH−MLτLα2D (α)= M_H _H-M_L _Lα^2 Under standard order (τH>τL _H> _L) and the fact that MH>MLM_H>M_L, the second derivative D′D is positive throughout. Therefore, the utility differential D(α)D(α) is convex. At the threshold, D(τH)=−UL(τH)<0D( _H)=-U_L( _H)<0. D′D is also negative at this point. Due to convexity and the negative slope, as we increase α beyond τH _H, D decreases further until D′=0D =0 and then begins to increase. Therefore, there is exactly one solution θ in [τH,∞)[ _H,∞) where D(θ)=0D(θ)=0 (see Fig. 2(a)). The switching threshold θ exists within the permissible range [0,1][0,1] only if D(1)D(1) is positive. Otherwise, model L will dominate throughout the domain. If D(1)>0D(1)>0, the domain of α is partitioned into 4 regions (shown in Fig. 2(a) in Appendix A). Neither model is viable in the first interval (until τL _L) and only L is viable in the second. Both models are viable in the third region where α>τHα> _H, but L is still more rewarding for the Agent. Beyond θ, the quality ‘premium’ of H is worth its higher cost, and the Agent’s model choice jumps from L to H. Note that the optimal token budget is discontinuous at this point (Eq. (3)). Flipped Order τL>τH _L> _H In this case, model L activates at a higher contract share than the higher model H. Because τH<τL _H< _L, it follows that τHα<τLα _Hα< _Lα, which implies: (1−τHα)>(1−τLα) (1- _Hα )> (1- _Lα ) Coupled with the fact that MH>MLM_H>M_L, this implies UH′(α)>UL′(α)U _H(α)>U _L(α) ∀α>τL∀α> _L. Since Model H activates earlier and climbs strictly faster at every point, model H dominates L throughout the domain, and there is no switching in this case (see Fig. 2(b) for illustration). We have thus derived the optimal token budget (Eq. (3)) for each model m given a contract from the Principal, parametrized by α. Using that, we then derived the optimal model choice for the Agent, characterized by the switching threshold θ (its existence and meaning determined by the two scenarios outlined above). 3.3 Principal’s Optimization Using the two-dimensional best response of the Agent for any given contract α, we now derive the Principal’s optimal linear contract. The Principal maximises V(α,m,x)=(1−α)⋅qm(xm∗(α))V(α,m,x)=(1-α)· q_m(x^*_m(α)) over α, but m here is not a free choice for the Principal; rather it is determined by the Agent’s best response. For a given model m, Agent’s optimal token budget xm∗x^*_m is given by (3). The corresponding quality is qm(xm∗)=Mm(1−τm/α)q_m(x^*_m)=M_m(1- _m/α) and therefore, the Principal’s utility for a given model is: Vm(α)=(1−α)qm(xm∗)=Mm[1+τm−α−τmα]V_m(α)=(1-α)\,q_m(x^*_m)=M_m\! [1+ _m-α- _mα ] This is the payoff the Principal would get if the Agent’s optimal model choice would be m at share α. Differentiating, dVdα=Mm(τmα2−1)=0⟹αm∗=τm, dVdα=M_m\! ( _mα^2-1 )=0\; \;α^*_m= _m, d2Vdα2=−2Mmτmα3<0 d^2Vdα^2=- 2M_m _mα^3<0 So αm∗α^*_m is a maximum of VmV_m over α>0α>0, and VmV_m is strictly concave, single-peaked at τm _m. We assume τm<1 _m<1 for both models, so that τm∈(τm,1) _m∈( _m,1) is a feasible share. Standard order τL<τH _L< _H. If the switching threshold θ exists within [0,1][0,1], then the Agent picks L on [τL,θ)[ _L,θ) and H on [θ,1][θ,1]. Vm(α)V_m(α) is achievable only when restricted to the interval where m is actually the Agent’s choice, and because each Vm(α)V_m(α) is single-peaked, the constrained optimum on each interval is either the peak (if it falls inside the interval) or the boundary (if the peak falls outside), i.e., clip αm∗α^*_m such that m is chosen by the Agent. αL†=min(τL,θ),αH†=max(τH,θ). _L = \! ( _L,\,θ ), _H = \! ( _H,\,θ ). (6) If τL≥θ _L≥θ, the function VL(α)V_L(α) is increasing in [τL,θ)[ _L,θ) with the peak not yet attained. So, the maximum value is at the boundary, but note that the Agent switches to model H at θ. Therefore, the maximum value of VLV_L is the left-hand limit of VL(θ)V_L(θ). Symmetrically, if τH≤θ _H≤θ, VH(α)V_H(α) is decreasing on [θ,1][θ,1] and its maximum is attained at the left endpoint θ. The Principal’s optimal contract is then α∗=argmaxα∈αL†,αH†V(α,m(α))α^*= _α∈\ _L ,\ _H \V(α,m(α)) (7) Note that if D(1)≤0D(1)≤ 0, i.e., the switching θ does not exist in the domain, L dominates throughout [0,1][0,1] and the Principal’s problem reduces to α∗=τLα^*= _L. Flipped order τL>τH _L> _H. Here H weakly dominates L everywhere it is active, so the Agent never chooses L and the Principal’s problem reduces to the single unconstrained optimization α∗=τHα^*= _H. Thus, we have derived the optimal linear contract for the Principal and characterized the Agent’s best response. Further derivation of the first-best benchmark, total surplus, and agency costs are in Appendix A. 3.4 Burn-in Tokens for Reasoning Models The modeling choice of saturation function for the LLM quality is appropriate for simpler tasks with instruction-tuned models. For complex tasks, especially with reasoning models, there is a burn-in period, where accuracy is zero despite spending tokens. We denote x~ x to be the raw tokens spent, and bmb_m denotes the number of burn-in tokens for model m. Then, qm(x~)=0,x~<bmMm(1−e−km(x~−bm)),x~≥bm q_m( x)= cases0,& x<b_m\\ M_m\! (1-e^-k_m( x-b_m) ),& x≥ b_m cases (8) Rewriting x=x~−bmx= x-b_m as the effective tokens spent by the Agent, we get the basic saturation function from earlier (2). However, the associated cost cmbmc_mb_m was not considered. The true utility of the Agent is: (α,m,x)=α⋅qm(x)−cmx−cmbmU(α,m,x)=α· q_m(x)-c_mx-c_mb_m However, this extra fixed cost is a constant in terms of x, and therefore the optimal effective token xm∗x^*_m does not change from (3). Note that the optimal true token x~m∗ x^*_m is shifted by bmb_m. The Agent utility under a given model m now becomes: m(α)=Mm[α−τm−τmln(ατm)]−cmbmU_m(α)=M_m\! [α- _m- _m \! ( α _m ) ]-c_mb_m Note that the burn-in utility is simply the same Um(α)U_m(α) as before with an additional negative term corresponding to the fixed cost due to the burn-in tokens. While the activation threshold τm _m ensured the Agent’s participation in the base case, now the Agent’s utility is in fact negative at τm _m due to the (fixed) burn-in cost. The participation threshold τmPτ^P_m defines the lowest value of α such that the Agent breaks even using model m, i.e., m(τmP)=0U_m(τ^P_m)=0. Mm[τmP−τm−τmln(τmPτm)]=cmbmM_m [τ^P_m- _m- _m ( τ^P_m _m ) ]=c_mb_m Simplifying, we get (τmPτm)−1−ln(τmPτm)=kmbm ( τ^P_m _m )-1- ( τ^P_m _m )=k_mb_m Rewriting the improper fraction as z, consider f(z)=z−1−lnzf(z)=z-1- z evaluated over the active domain z>1z>1. Computing its derivatives: f′(z)=1−1z>0,f′(z)=1z2>0f (z)=1- 1z>0, f (z)= 1z^2>0 Thus, f(z)f(z) is strictly increasing and strictly convex. Its inverse function f−1(x)f^-1(x) is strictly increasing and strictly concave. This allows an exact closed-form expression of the participation threshold: τmP=τm⋅f−1(kmbm)τ^P_m= _m· f^-1(k_mb_m) (9) Switching Threshold θ0 _0 We rewrite the burn-in utility as m(α)=Um(α)−cmbmU_m(α)=U_m(α)-c_mb_m, and reuse the baseline differential D(α)≡UH(α)−UL(α)D(α)≡ U_H(α)-U_L(α) from (5). The burn-in switching threshold θ0 _0 satisfies H(θ0)=L(θ0)U_H( _0)=U_L( _0), i.e. UH(θ0)−cHbH=UL(θ0)−cLbLU_H( _0)-c_Hb_H=U_L( _0)-c_Lb_L. Rearranging: D(θ0)=cHbH−cLbLD( _0)=c_Hb_H-c_Lb_L (10) This is not closed-form in general: D is a transcendental function of α. In practice, θ0 _0 is found numerically as the root of (10), via bracketed root-finding like Brent’s method. To obtain sharper qualitative results, we restrict to a special case with common k and b for both models. Under this assumption, θ0 _0 is a strict rightward shift of the base-model threshold θ (i.e., burn-in delays switching) in the Standard Order case (τL<τH _L< _H). In the Flipped Order case (τH<τL _H< _L), H continues to dominate everywhere, exactly as in the base model. See Appendix B for further details. 4 Experimental Setup We evaluate our proposed framework empirically across two distinct domains: advanced mathematical reasoning and multi-discipline question answering. Specifically, we evaluate model pairings on the MATH dataset (difficulty levels 3 and 4) and the MMLU-Pro dataset [14, 27]. Our model suite consists of instruction-tuned models (e.g., Llama-3.2), distilled reasoning models (e.g. DeepSeek-R1-Distill-Qwen), and recent edge reasoning models (e.g. Gemma4). We consider models of different sizes and consider pairings of same and different types. In this section, we describe how we calibrate the production function and then describe the learning processes. 4.1 Calibration of Production Functions To bridge our analytical model with real-world LLM performance, we first empirically calibrate the production parameters (Mm,km,bm)(M_m,k_m,b_m) for each model on both evaluation datasets. For each dataset and model, we let the model answer every question under a maximum token budget of xmaxx_ tokens at temperature 00. Let xix_i denote the number of tokens used to answer question i. For a dense grid of budgets from 00 to xmaxx_ , accuracy at budget x is the percentage of questions answered correctly using at most x tokens, i.e. with xi≤x_i≤ x and a correct answer. This produces an empirical production curve for each model-dataset pair, to which we fit the saturating function with burn-in tokens (Eq. (8)) via nonlinear least squares (using curve_fit method from scipy). Table 1 reports the fitted parameters, along with R2R^2, for both tasks. Figure 3 shows the accuracy at each budget level along with the fitted curves for the Llama models in MATH (refer Fig. 7 in Appendix E for all other models and tasks). Figure 3: Accuracy vs. budget for Llama models on MATH. Model MmM_m kmk_m bmb_m R2R^2 RMSE MATH task domain Llama-3.2-1B-Instruct 50 0.00529 134 0.998 0.772 Llama-3.2-3B-Instruct 82 0.00473 177 0.994 2.153 DS-R1-Distill-1.5B 75 0.00081 982 0.998 0.979 DS-R1-Distill-7B 92 0.00073 1009 0.997 1.373 Gemma-4-E2B-it 84 0.00159 430 0.999 0.952 Gemma-4-E4B-it 85 0.00190 382 0.997 1.411 MMLUPro task domain Llama-3.2-1B-Instruct 22 0.00366 162 0.983 0.928 Llama-3.2-3B-Instruct 40 0.00474 164 0.986 1.589 DS-R1-Distill-1.5B 24 0.00212 479 0.996 0.536 DS-R1-Distill-7B 41 0.00141 440 0.999 0.414 Gemma-4-E2B-it 59 0.00202 422 0.997 1.029 Gemma-4-E4B-it 68 0.00235 413 0.998 1.084 Table 1: Model calibration and fitted params for both tasks. We anchor the token cost of the lower-tier model to cL=0.005c_L=0.005 monetary units per token in every pairing. This fixes a common scale and the variation between various pairings is through the capability parameters (Mm,km,bm)(M_m,k_m,b_m) and the cost ratio cH/cLc_H/c_L. The H model’s cost cHc_H is chosen per pairing: for most pairings we select cHc_H to fall within the range that yields Standard Order with a genuine switching threshold θ0∈(0,1] _0∈(0,1], giving representative cases of model switching; for a small number of pairings we deliberately choose cHc_H outside this range to illustrate the Flipped Order and always dominant L scenarios discussed in the Model Section. 4.2 Learning Processes of Agent and Principal To evaluate how efficiently the Agent can learn the optimal mechanism without prior knowledge of the calibration parameters, we frame the model choice and budget selection problem as a Contextual Multi-Armed Bandit. At each sequential round t, the Agent receives a contract stake αt _t drawn uniformly at random from the domain αt∈[0,1] _t∈[0,1], which acts as the context. The task at each round t is a randomly sampled set of 1616 questions from the dataset. See Appendix E for further experimental details and parameters. The action space A is structured as a joint choice space =ℳ×A=M×X, where ℳ=L,HM=\L,H\ represents model choices and =x(1),x(2),…,x(N)X=\x^(1),x^(2),…,x^(N)\ is a discretized set of token budgets spanning from xminx_min up to xmax_max tokens. For a given context αt _t, the net reward observed by choosing arm a=(m,x)a=(m,x) is given by Rt(a)=αt⋅yt,m(x)−cm⋅xR_t(a)= _t· y_t,m(x)-c_m· x where yt,m(x)y_t,m(x) is the empirical accuracy achieved by model m under budget x (questions answered correctly by the LLM). We deploy the LinUCB algorithm [20] to model the expected reward of each arm. Crucially, the optimal utility of the Agent Um(α)U_m(α) maps exactly to a linear combination of α and ln(α) (α) due to the structure of the Agent’s FOC; see Eq. (4). To ensure faster convergence44 4 We also explored using standard UCB with discretized α; similar trends (Fig. 11 in App. E) but takes longer to converge., we construct a handcrafted context feature vector: [1αtln(αt)] bmatrix1& _t& ( _t) bmatrix. The Principal’s learning is modeled as classic (non-contextual) UCB, with arms representing a discretized grid of contract shares α∈[0,1]α∈[0,1]. Unlike the Agent, the Principal has no natural context to condition on as it chooses α rather than responding to it. Each round, the Principal selects an arm (contract share), the frozen, previously-trained Agent best-responds with its model and token-budget choice, and the Principal observes its realized payoff; (1−αt)⋅yt,m(x)(1- _t)· y_t,m(x). This reward is used to update the estimate for the pulled arm. 5 Results In this section, we compare the learned policies against our theoretical model with fitted parameters across different model pairings and two task domains. 5.1 MATH domain First, we evaluate our model on the MATH domain [14]. The LLM prompts for question-answering are included in App. G. The theoretical value of θ0 _0 is calculated using the calibrated parameters in Table 1 and solving for the root of (10) using brentq method in scipy package. We train the LinUCB controller for 2,0002,000 episodes, the learned policy is used to greedily choose model and token budget for 50 equally interspersed values of α∈[0.001,1]α∈[0.001,1] to obtain the learned value of θ0 _0. These numbers are reported in Table 2. We observe that the learned values are generally close to the estimated values (typically within about 10−15%10-15\%); the difference is an artifact of mapping the continuous budget x onto a discrete token budget space (N=21N=21 bins) compounded by noisy LLM inference. Fig. 4 shows the learned token budget along with the learned switching threshold for the Llama 1B vs. 3B pairing. Similar figures for all other pairings are in App. E. Model Pairing cHc_H θ0 _0 (est.) θ0 _0 (learn) Intra-Family Pairs Llama 3.2: 1B vs. 3B 0.0200 0.367 0.409 DeepSeek R1: 1.5B vs. 7B 0.0075 0.731 0.817 Gemma 4: E2B vs. E4B 0.0060 0.255 0.286 Inter-Family Pairs Llama 1B vs. Gemma 4B 0.0100 0.419 0.388 Llama 1B vs. DS 1.5B 0.0075 1.331 L dom. DS 1.5B vs. Gemma 4B 0.0075 N/A H dom. Table 2: MATH Domain: Learned model choices (θ0)( _0) for the LinUCB controller. We fix cL=0.005c_L=0.005 for all configurations. While values of cHc_H were mostly chosen to induce standard order model switching, we also included some other scenarios. Llama 1B vs. DS 1.5B is one such case where the switching threshold θ0 _0 lies outside the domain [0,1][0,1]. Intuitively, H, which is the Deepseek model here, is significantly more capable than Llama 1B (about 50%; see Table 1). However, Deepseek has higher burn-in tokens and that combined with the higher cost makes it unattractive for any α within the domain. The LinUCB Agent learns to always pick L in this scenario. The cost cHc_H for the pairing DS 1.5B vs. Gemma 4B was chosen to exhibit yet another interesting scenario. Here, τL=0.0833 _L=0.0833 and τH=0.0464 _H=0.0464. That is, this specific configuration belongs to the flipped order scenario, where H activates earlier and dominates L everywhere. There is no switching threshold θ0 _0 in this scenario and the LinUCB Agent matches this exactly, where H, the Gemma 4B model dominates everywhere. This is explained by the fact that Gemma models have higher capability ceiling than Deepseek R1 1.5B on this task and they have lower burn-in tokens (see calibration Table 1). So, despite the slightly higher cost, it is economically rational to always use the expensive model. Figure 4: MATH: Llama 1B vs. Llama 3B Instead of the LinUCB bandit controller, we also explored prompting a much stronger LLM to choose the model and token budget in Appendix F; results are broadly consistent with optimal model choice but not the optimal budget. 5.2 MMLUPro Domain We also evaluate our results on the MMLUPro domain [27], and the results are shown in Table 3. Similar to the results from the MATH domain, the bandit algorithms learned the switching thresholds that are generally close to the estimated theoretical value of θ0 _0. Figure 5 shows the learned token budget and model choice for the DeepSeek 1.5B vs. Gemma 4B pairing (other pairings are in Fig. 9 in App. E). Figure 5: MMLUPro: DeepSeek 1.5B vs. Gemma 4B Model Pairing cHc_H θ0 _0 (est.) θ0 _0 (learn) Intra-Family Pairs Llama 3.2: 1B vs. 3B 0.0200 0.589 0.633 DeepSeek R1: 1.5B vs. 7B 0.0075 0.309 0.286 Gemma 4: E2B vs. E4B 0.0060 0.339 0.286 Inter-Family Pairs Llama 1B vs. Gemma 4B 0.0200 0.520 0.592 Llama 1B vs. DS 1.5B 0.0075 6.931 L dom. DS 1.5B vs. Gemma 4B 0.015 0.410 0.409 DS 1.5B vs. Gemma 2B 0.015 0.280 0.307 DS 1.5B vs. Llama 3B 0.006 N/A H dom. Table 3: MMLUPro Domain: Learned model choices (determined by θ0 _0). We fix cL=0.005c_L=0.005 for all pairings. 5.3 Principal’s Learning In this section, we show that the Principal can learn the appropriate linear contract when interacting with a trained best-responding Agent from previous sections. The Principal’s learning process is outlined in the previous section. Figure 6: Principal’s Learning for Llama 3.2: 1B vs. 3B We use 2121 arms for the Principal and train it for 3,0003,000 episodes against a frozen, trained Agent from the previous section. We test this in the MATH domain with different model pairings. For the Llama 3.2: 1B vs. 3B pairing, Fig. 6 shows the theoretically expected utility values of the Principal over the domain (in black) and the learned value at each arm (orange circles). The constrained αm†α _m (see Eq. (6)) for each model are calculated and shown based on the calibrated parameters. Finally, the arm (contract share) chosen by a greedy Principal post training is shown as the purple diamond, which is close to theoretical expectations. Similar results are seen for the other pairing; refer to Fig. 10 in App. E. 6 Conclusion In this paper, we extended the classical Principal-Agent framework to address the emerging moral hazard in autonomous LLM delegation, a setting where an Agent dynamically selects both a reasoning technology (model choice) and a continuous effort level (token budget). By modelling the LLM inference process as a saturating production function, we derived the Agent’s optimal token allocation and characterized the switching threshold at which it becomes economically rational to adopt a more capable, yet more expensive, model. Furthermore, we determined the Principal’s optimal linear contract that maximizes expected utility despite the Agent’s hidden actions. Our empirical calibrations on the MATH and MMLUPro benchmarks, paired with simulations using contextual bandit algorithms, demonstrated that both the Principal and the Agent converge toward strategies consistent with our theoretical equilibrium. As LLM inference-time scaling and budget-aware reasoning continue to advance, the economic implications of optimizing joint model and budget choices will only grow in relevance. While our current framework assumes verification is costless, we have established the foundations for constant-cost verification in Appendix D, and it would be worthwhile to explore other, more complex verification cost families in the future. Additionally, extending this framework to non-ground truth settings presents a compelling direction for future research. In such environments, both the evaluation of correctness and the valuation of the task depend entirely on the Principal’s specific type, introducing severe information asymmetry. Without knowing the exact type of the Principal, the Agent’s best response would require reasoning over the type distribution, bridging mechanism design with Bayesian delegation in complex agentic markets. References [1] M. A. Alomrani, Y. Zhang, D. Li, Q. Sun, S. Pal, Z. Zhang, Y. Hu, R. D. Ajwani, A. Valkanas, R. Karimi, P. Cheng, Y. Wang, P. Liao, H. Huang, B. Wang, J. Hao, and M. Coates (2025) Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs. arXiv. Note: http://arxiv.org/abs/2507.02076 External Links: Document, 2507.02076 Cited by: §2. [2] F. Bacchiocchi, M. Castiglioni, A. Marchesi, and N. Gatti (2024) Learning Optimal Contracts: How to Exploit Small Action Spaces. International Conference on Learning Representations 2024, p. 11944–11970 (en). External Links: Link Cited by: §2. [3] F. Bacchiocchi, M. Castiglioni, A. Marchesi, and N. Gatti (2025) Regret Minimization for Piecewise Linear Rewards: Contracts, Auctions, and Beyond. In Proceedings of the 26th ACM Conference on Economics and Computation, EC ’25, New York, NY, USA, p. 1020. External Links: ISBN 979-8-4007-1943-1, Link, Document Cited by: §2. [4] D. Bergemann, A. Bonatti, and A. Smolin (2025) The Economics of Large Language Models: Token Allocation, Fine-Tuning, and Optimal Pricing. In Proceedings of the 26th ACM Conference on Economics and Computation, EC ’25, New York, NY, USA, p. 786. External Links: ISBN 979-8-4007-1943-1, Link, Document Cited by: §2. [5] D. Bergemann, A. Bonatti, and A. Smolin (2026) Menu Pricing of Large Language Models. arXiv. Note: http://arxiv.org/abs/2502.07736 External Links: Document, 2502.07736, Link Cited by: §2. [6] M. Boddy and T. Dean (1989) Solving time-dependent planning problems. In Proceedings of the 11th International Joint Conference on Artificial Intelligence - Volume 2, IJCAI’89, San Francisco, CA, USA, p. 979–984. Cited by: §2, §3.1. [7] Y. Bu and Y. Ma (2026) Position: ai-agent pricing should become more outcome-dependent: an economic perspective. Note: Preprint available at https://buyuheng.github.io/publications.html Cited by: §1. [8] Devin (2026) Devin — the ai software engineer. Note: https://devin.ai/Accessed: 2026-07-20 Cited by: §1. [9] D. Ding, A. Mallick, S. Zhang, C. Wang, D. Madrigal, M. D. C. H. Garcia, M. Xia, L. V. S. Lakshmanan, Q. Wu, and V. Rühle (2025) BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute. In ICML 2025, (en). External Links: Link Cited by: §2. [10] P. Dütting, M. Feldman, and I. Talgam-Cohen (2025) Algorithmic contract theory: a survey. Found. Trends Theor. Comput. Sci. 16 (3–4), p. 211–412. External Links: ISSN 1551-305X, Link, Document Cited by: §2. [11] P. Dütting, V. Mirrokni, R. Paes Leme, H. Xu, and S. Zuo (2024) Mechanism Design for Large Language Models. In Proceedings of the ACM Web Conference 2024, W ’24, New York, NY, USA, p. 144–155. External Links: ISBN 979-8-4007-0171-9, Link, Document Cited by: §2. [12] G. K. Hadfield and A. Koh (2025) An Economy of AI Agents. arXiv. Note: arXiv:2509.01063 [econ.GN] External Links: Link, Document Cited by: §1. [13] Harvey AI (2026) Harvey — ai software for legal and professional services. Note: https://w.harvey.ai/Accessed: 2026-07-20 Cited by: §1. [14] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) Measuring mathematical problem solving with the math dataset. NeurIPS. Cited by: Appendix E, §4, §5.1. [15] B. Holmstrom and P. Milgrom (1991) Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design. The Journal of Law, Economics, and Organization 7 (special_issue), p. 24–52. External Links: ISSN 8756-6222, Link, Document Cited by: §2. [16] B. Holmström (1979) Moral Hazard and Observability. The Bell Journal of Economics 10 (1), p. 74–91. External Links: ISSN 0361-915X, Link, Document Cited by: §2. [17] E. Horvitz and J. Breese (1990) Ideal partition of resources for metareasoning. Technical report Technical Report KSL-90-26, Stanford University. External Links: Link Cited by: §2. [18] E. J. Horvitz (1987) Reasoning about beliefs and actions under computational resource constraints. In Proceedings of the Third Conference on Uncertainty in Artificial Intelligence, UAI’87, Arlington, Virginia, USA, p. 429–447. External Links: ISBN 0444874178 Cited by: §2. [19] W. Jitkrittum, H. Narasimhan, A. S. Rawat, J. Juneja, C. Wang, Z. Wang, A. Go, C. Lee, P. Shenoy, R. Panigrahy, A. K. Menon, and S. Kumar (2025) Universal Model Routing for Efficient LLM Inference. In ICML 2026, (en). External Links: Link Cited by: §2. [20] L. Li, W. Chu, J. Langford, and R. E. Schapire (2010) A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web, W ’10, New York, NY, USA, p. 661–670. External Links: ISBN 978-1-60558-799-8, Link, Document Cited by: Appendix E, §4.2. [21] P. Qi, Z. Liu, T. Pang, C. Du, W. S. Lee, and M. Lin (2025) Optimizing Anytime Reasoning via Budget Relative Policy Optimization. In Advances in Neural Information Processing Systems, Vol. 38, p. 23429–23451. External Links: Link Cited by: §2, §3.1. [22] P. Rauba, S. Cepenas, and M. van der Schaar (2026) Multi-agent systems should be treated as principal-agent problems. Note: https://arxiv.org/abs/2601.23211 External Links: 2601.23211 Cited by: §1. [23] E. Saig, O. Einav, and I. Talgam-Cohen (2024) Incentivizing quality text generation via statistical contracts. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Vol. 37, Red Hook, NY, USA, p. 51196–51222. External Links: ISBN 979-8-3313-1438-5 Cited by: §2. [24] E. Saig, T. Garbuz, A. D. Procaccia, I. Talgam-Cohen, and J. Tucker-Foltz (2026) Adaptive Contracts for Cost-Effective AI Delegation. In ICML 2026, Note: http://arxiv.org/abs/2603.17212 External Links: Document Cited by: §1, §2. [25] N. Tomašev, M. Franklin, and S. Osindero (2026) Intelligent AI Delegation. arXiv (en). Note: arXiv:2602.11865 [cs.AI] External Links: Link, Document Cited by: §1. [26] A. A. Velasco, S. Tsirtsis, and M. G. Rodriguez (2026) Auditing pay-per-token in large language models. In The 29th International Conference on Artificial Intelligence and Statistics, External Links: Link Cited by: §1. [27] Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, et al. (2024) Mmlu-pro: a more robust and challenging multi-task language understanding benchmark. arXiv preprint arXiv:2406.01574. Cited by: Appendix E, §4, §5.2. [28] H. Wen, Y. Su, F. Zhang, Y. Liu, Y. Liu, Y. Zhang, and Y. Li (2025) ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time Compute. arXiv. Note: http://arxiv.org/abs/2509.04475 External Links: Document Cited by: §3.1. [29] H. Wen, X. Wu, Y. Sun, F. Zhang, L. Chen, J. Wang, Y. Liu, Y. Liu, Y. Zhang, and Y. Li (2025) BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens. arXiv. Note: http://arxiv.org/abs/2508.17196 External Links: Document Cited by: §2. [30] J. Yang, B. Hou, W. Wei, Y. Bao, and S. Chang (2026) Ares: Adaptive Reasoning Effort Selection for Efficient LLM Agents. arXiv (en). Note: http://arxiv.org/abs/2603.07915 External Links: Document Cited by: §2. [31] S. Zhou, R. Ling, J. Chen, X. Wang, T. Fan, and H. Wang (2026) When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling. arXiv. Note: http://arxiv.org/abs/2604.10739 External Links: Document, 2604.10739, Link Cited by: §3.1. [32] B. Zhu, S. Bates, Z. Yang, Y. Wang, J. Jiao, and M. I. Jordan (2023) The Sample Complexity of Online Contract Design. In Proceedings of the 24th ACM Conference on Economics and Computation, EC ’23, New York, NY, USA, p. 1188. External Links: ISBN 979-8-4007-0104-7, Link, Document Cited by: §2. [33] S. Zilberstein (1996) Using Anytime Algorithms in Intelligent Systems. AI Magazine 17 (3), p. 73–83 (en). External Links: ISSN 2371-9621, Link, Document Cited by: §2, §3.1. Appendix Appendix A Surplus and Agency Costs We now consider the first-best benchmark for the base model. In the first-best scenario, a single party holds the model, reaps the benefit and spends compute cost, i.e., single party receives the total social surplus, with no moral hazard. This benchmark is the limiting case α=1α=1 of the contract space above. The optimal token budget is the same as optimal effort for model m at α=1α=1 from Eq. (3): xm∗,FB=1kln(1τm)=xm∗(1)x^*,FB_m= 1k\, \! ( 1 _m )=x^*_m(1) At α=1α=1 the Agent’s payoff Um(1)=qm(xm∗(1))−cmxm∗(1)U_m(1)=q_m(x_m^*(1))-c_mx_m^*(1) equals total surplus under model m at efficient effort, so D(1)=UH(1)−UL(1)D(1)=U_H(1)-U_L(1) is exactly the comparison of first-best surplus across the two models. Recall that, in the standard order, D(1)>0D(1)>0 is the condition under which a switching threshold θ exists in [0,1][0,1]. The interpretation: the Agent eventually adopts H under some contract if and only if H is the first-best efficient model. We write mFB=argmaxmUm(1)m^FB= _mU_m(1) for the first-best model, and SmFB≡Um(1)=Mmh(τm),h(z)≡1−z+zlnz,S_m^FB\;≡\;U_m(1)\;=\;M_m\,h( _m), h(z)≡ 1-z+z z, for the first-best surplus achievable under model m. Agency Cost At the Principal’s unconstrained optimum αm∗=τm _m^*= _m, total surplus is Sm(τm)=Um(τm)+Vm(τm)S_m( _m)=U_m( _m)+V_m( _m), strictly below SmFBS_m^FB. The loss or cost of inducing Agent to select each model m is given by: SmFB−Sm(τm)=Mmτmh(τm),S_m^FB-S_m( _m)\;=\;M_m _m\,h( _m), and normalizing by first-best surplus gives a loss ratio that depends only on the activation threshold τm _m, ℓm≡SmFB−Sm(τm)SmFB=τmh(τm)h(τm). _m\;≡\; S_m^FB-S_m( _m)S_m^FB\;=\; _m\,h( _m)h( _m). Further, re-arranging terms gives us: Um(τm)=SmFB−Sm(τm).U_m( _m)\;=\;S_m^FB-S_m( _m). The Agent’s utility at the Principal’s optimal contract is exactly equal to the Agency cost, a consequence of the specific saturating-exponential functional form we assume; not a general property of moral hazard models. Appendix B Burn-in Switching Threshold This appendix derives the qualitative behavior of the burn-in switching threshold θ0 _0 (Eq. 10) for an illustrative special case. Motivated by calibration tables where k and b are roughly similar (not exactly equal) for models of the same family for a given task, we consider a special case with identical token saturation rates (kL=kH=k_L=k_H=k) and identical burn-in token requirements (bL=bH=b_L=b_H=b). Under this assumption, the participation thresholds (Eq. 9) become constant scalar multiples of the corresponding activation thresholds, with a common scale factor ρ=f−1(kb)≥1ρ=f^-1(kb)≥ 1: τLP=ρτL,τHP=ρτH.τ^P_L=ρ\, _L, τ^P_H=ρ\, _H. B.1 Flipped Order (τH<τL _H< _L) When the high-capability model activates first, the baseline derivative condition remains intact, since the fixed burn-in costs vanish under differentiation: H′(α)=MH(1−τHα)>ML(1−τLα)=L′(α).U _H(α)=M_H (1- _Hα )>M_L (1- _Lα )=U _L(α). Coupled with τHP<τLPτ^P_H<τ^P_L (which follows immediately from τH<τL _H< _L and the common scale factor ρ), model H both breaks even earlier and climbs strictly faster than model L at every point in the active domain. Consequently, model H globally dominates, and no switching point exists exactly as in the base model without burn-in. Remark. Without the same-family assumption, it is possible to have τH<τL _H< _L but τHP>τLPτ^P_H>τ^P_L (i.e., different km,bmk_m,b_m break the common scale factor ρ). This would open a brief window in which L is the only economically viable model, before H eventually catches up and dominates. B.2 Standard Order (τL<τH _L< _H) When model L activates first, the uniform scaling yields τLP<τHPτ^P_L<τ^P_H. We reuse the baseline utility differential D(α)D(α) from Eq. (5), which is strictly convex under this ordering (since MHτH>MLτLM_H _H>M_L _L term-wise) with a unique zero θ on its increasing branch. Rewriting Eq. (10) with common b: D(θ0)=(cH−cL)b.D( _0)=(c_H-c_L)\,b. (11) Under the standing assumption cH>cLc_H>c_L (the more capable model carries a higher per token cost) and b>0b>0, the right-hand side is strictly positive, so D(θ0)>D(θ)=0.D( _0)>D(θ)=0. Since D is increasing on (θ,∞)(θ,∞) (the increasing branch of the convex function), D(θ0)>D(θ)D( _0)>D(θ) directly implies θ0>θ. _0>θ. The burn-in cost, being more expensive in absolute terms for the costlier model (cHb>cLbc_Hb>c_Lb), strictly delays the switch. As in the base model, θ0 _0 exists in [0,1][0,1] only if D(1)≥(cH−cL)bD(1)≥(c_H-c_L)b; otherwise L dominates throughout the domain. Appendix C Linear Contracts with Fixed Payments We now consider a contract ⟨α,β⟩ α,β , where α∈[0,1]α∈[0,1] remains the performance-based revenue share and β∈ℝβ is a flat transfer payment independent of output q. The updated normalized utilities for the Agent and Principal are: U(α,β,m,x) U(α,β,m,x) =α⋅qm(x)−cmx+β, =α· q_m(x)-c_mx+β, V(α,β,m,x) V(α,β,m,x) =(1−α)⋅qm(x)−β. =(1-α)· q_m(x)-β. Optimal Effort and Model Choice The introduction of a fixed payment β does not alter the marginal incentives for effort. Taking the first-order derivative of U(α,β,m,x)U(α,β,m,x) with respect to x eliminates β, leaving the first-order condition identical to the pure revenue-sharing case. Consequently, the optimal effort xm∗x^*_m and the activation threshold τm _m remain exactly as derived in Equations (3) and (4). Substituting optimal effort back into the Agent’s utility gives: Um(α,β) U_m(α,β) =Um(α)+β =U_m(α)+β =Mm[α−τm−τmln(ατm)]+β,α>τm. =M_m\! [α- _m- _m \! ( α _m ) ]+β, α> _m. (12) Similarly, because β shifts the utility curves of all models m∈ℳm by the same constant amount, it cancels out during the Agent’s model selection phase. The condition UH(α,β)>UL(α,β)U_H(α,β)>U_L(α,β) is mathematically equivalent to UH(α)>UL(α)U_H(α)>U_L(α). Therefore, the switching threshold θ and the two scenarios (Standard vs. Flipped) remain entirely unchanged from the previous section. Principal’s Optimization The presence of the fixed transfer fundamentally alters the Principal’s optimization strategy. The Principal seeks to maximize Vm(α,β)V_m(α,β) subject to the Agent’s Individual Rationality (IR) constraint, assuming an outside option utility of zero: Um(α,β)≥0⟹Um(α)+β≥0U_m(α,β)≥ 0 U_m(α)+β≥ 0 To maximize its own utility, the Principal will extract all surplus from the Agent by setting the fixed payment such that the IR constraint binds exactly: β∗=−Um(α)β^*=-U_m(α) If Um(α)>0U_m(α)>0, this requires β∗<0β^*<0, functioning as a fee paid by the Agent to the Principal for the right to perform the task. Substituting β∗β^* into the Principal’s objective function aligns the Principal’s utility with the total social surplus of the system Sm(α)S_m(α): Vm(α,β∗) V_m(α,β^*) =(1−α)⋅qm(xm∗)−(−Um(α)) =(1-α)· q_m(x^*_m)-(-U_m(α)) =(1−α)⋅qm(xm∗)+α⋅qm(xm∗)−cmxm∗ =(1-α)· q_m(x^*_m)+α· q_m(x^*_m)-c_mx^*_m =qm(xm∗)−cmxm∗ =q_m(x^*_m)-c_mx^*_m =Sm(α) =S_m(α) The Principal effectively designs the contract to maximize total surplus, which they then fully extract via β∗β^*. Using qm(xm∗)=Mm(1−τm/α)q_m(x^*_m)=M_m(1- _m/α) and xm∗=1kmln(ατm)x^*_m= 1k_m ( α _m ), the total surplus function is: Sm(α)=Mm(1−τmα)−cmkmln(ατm)S_m(α)=M_m\! (1- _mα )- c_mk_m \! ( α _m ) Differentiating with respect to α and setting to zero: dSmdα=Mmτmα2−cmkm1α=0 dS_mdα=M_m _mα^2- c_mk_m 1α=0 Recall from the activation threshold definition that cmkm=Mmτm c_mk_m=M_m _m. Substituting this identity yields: Mmτm(1α2−1α)=0.M_m _m ( 1α^2- 1α )=0. Since Mm>0M_m>0 and τm>0 _m>0, the only strictly positive solution is α=1α=1; also dSm/dα>0dS_m/dα>0 throughout (0,1)(0,1). This is the classic selling the firm result: by setting α=1α=1, the Principal eliminates the misalignment between the Agent’s private return and total surplus, since the Agent now keeps the full marginal return and bears the full marginal cost of every token spent. The Agent’s model choice at α=1α=1 is governed by the switching threshold θ from previous section , independent of β. Since SmS_m is increasing throughout (0,1)(0,1) for both models, α=1α=1 is surplus-maximizing and the sign of the utility differential D(1)D(1) (see Eq. (5)) identifies which model the Agent selects: α∗=1,m∗=Hif D(1)>0Lif D(1)≤0.α^*=1, m^*= casesH&if D(1)>0\\ L&if D(1)≤ 0. cases The Principal sets the fixed transfer to bind the Agent’s IR constraint exactly, extracting the realized surplus in full: β∗=−Um∗(1)=−Mm∗[1−τm∗−τm∗ln(1τm∗)].β^*=-U_m^*(1)=-M_m^*\! [1- _m^*- _m^* \! ( 1 _m^* ) ]. Remark: This implements the first-best outcome. At α=1α=1, the Agent’s optimal effort (see Eq. (3)) coincides with the surplus-maximizing FOC a Principal with direct control over tokens would solve, so xm∗(1)x^*_m(1) is the efficient token budget. The fixed-payment contract therefore induces the efficient model and efficient effort simultaneously, with zero loss due to moral hazard (albeit trivially by selling the firm, which is not always realistic). Limited Liability Constraint (β≥0β≥ 0) If we assume limited liability, which is common in most real-world settings, it imposes the added constraint β≥0β≥ 0. The Principal solves: maxα,β(1−α)⋅qm(xm∗(α))−βs.t.Um(α)+β≥0,β≥0 _α,β\;(1-α)· q_m(x^*_m(α))-β .t. U_m(α)+β≥ 0, β≥ 0 Because any β>0β>0 directly reduces the Principal’s payoff without modifying the Agent’s marginal incentives for effort, the limited liability constraint gives: βLL∗=0β^*_L=0 Consequently, the problem collapses entirely back to the pure revenue-sharing model from previous section. Appendix D Costly Verification We extend the base model to allow the Principal to incur a fixed cost cv≥0c_v≥ 0 per round to observe (verify) the realized output quality q. The constant cost assumption is justified in cases such as math verification or unit tests for code outputs. In the base model this cost is implicitly zero; here we make it explicit. Note that we consider the case where the Principal verifies in every round. An interesting extension could be probabilistic verification, but we leave that for future work. The verification cost cvc_v is borne entirely by the Principal and, therefore, does not appear in the Agent’s utility. Consequently, the Agent’s optimal budget for model m and the Agent’s model choice through switching threshold θ remain unchanged. The Principal’s utility from inducing model m at share α becomes Vmv(α)=(1−α)qm(xm∗(α))−cv=Vm(α)−cv,V_m^v(α)\;=\;(1-α)\,q_m(x_m^*(α))-c_v\;=\;V_m(α)-c_v, i.e. the verified payoff is the unverified payoff Vm(α)V_m(α) shifted down by the constant fixed cost cvc_v. For any cv≥0c_v≥ 0 constant in α, the FOC remains the same because the constant term gets differentiated out to zero. This implies the unconstrained optima τm _m remains unchanged. We have shown that the model choice from Agent’s side, determined by θ also does not change. Consequently the constrained optima αL† _L and αH† _H , which are τm _m clipped against the switching threshold θ, are also unchanged. So a constant verification cost changes none of the base model’s structural results: the Principal’s optimal contract share is exactly as derived in the base model. Verification cost only ever shows up as a constant shift in the Principal’s realized payoff. However, this matters for whether the Principal wants to contract at all. In the base model, the Principal’s utility was always non-negative, but the fixed cost introduces the need to check participation constraint. D.1 Participation The Principal’s payoff at the (unconstrained) optimum is Vm(τm)=Mm(1+τm−τm−τm/τm)=Mm(1−τm)2,V_m( _m)\;=\;M_m (1+ _m- _m- _m/ _m )\;=\;M_m (1- _m )^2, The Principal participates i.e., offers a contract to induce model m only if Vmv(αm†)≥0V_m^v( _m )≥ 0, cv≤Vm(αm†).c_v\;≤\;V_m( _m ). At the unconstrained optimum this becomes the clean threshold cv≤Mm(1−τm)2c_v≤ M_m(1- _m)^2: a small verification cost relative to this never changes the Principal’s optimal share, but a sufficiently large cvc_v can make model m non-viable for the Principal. D.2 Alternative verification costs Treating cvc_v as a fixed constant, while realistic in some scenarios, makes the verification problem fairly trivial as it gets differentiated out in all first-order conditions. Other interesting choices for verification cost are as follows. • cvc_v depending on q. If verification cost scales with the quality being verified (e.g., low quality or buggy code will not even compile or error out quickly with the unit tests), cv=cv(q)c_v=c_v(q) enters Vmv(α)V_m^v(α) as a function of α through qm(xm∗(α))q_m(x_m^*(α)), and the first-order condition picks up a cv′(q)⋅qm′(xm∗)⋅dxm∗dαc_v (q)· q_m (x_m^*)· dx_m^*dα term and αm∗ _m^* would shift in general. • cvc_v depending on m. Even holding cvc_v constant in α, allowing cv=cv,mc_v=c_v,m to differ by model (e.g. verifying a more elaborate H-model’s output costs more than a short L-model output) does not change any of the structural results, but it does mean the participation thresholds in the previous subsection differ across models for a second reason (beyond Mm,τmM_m, _m already varying), which could be a source of an additional, verification-driven bias toward the cheaper-to-verify model. • Endogenous/probabilistic verification. The Principal could choose to verify only with some probability p<1p<1, trading off expected verification cost p⋅cvp· c_v against reduced ability to enforce the contract (an Agent who anticipates low verification probability may deviate). This introduces strategic aspects to the interaction between verification and the Agent’s incentives. Appendix E Experiment Details and Additional Results Datasets and Filtering. For the MATH dataset [14], which contains questions across difficulty levels 1 through 5, we filter exclusively for questions in levels 3 and 4 to maintain consistent task difficulty for calibrating production curves. We filter out questions containing graphic components by matching raw markup tags such as [asy], resulting in 2,9622,962 questions for evaluation. The final JSON file is included in the code package. For MMLU-Pro [27], we evaluate across all 12,03212,032 available questions in the test partition. Models, Serving, and Infrastructure. All base models are loaded directly using their official Hugging Face repository identifiers, as listed in Table 4. All bandit runs for different pairings are executed on a single NVIDIA H100 GPU (80GB). Open-weights LLMs are served locally via vLLM using an OpenAI-compatible HTTP API server without additional model quantization. To ensure deterministic generation, all inference requests are made with zero sampling temperature (temperature=0.0temperature=0.0). Answer Parsing and Evaluation. • MATH Dataset: Evaluated using the math_verify library to verify symbolic and numerical equivalence against the ground truth answer inside outputs. • MMLU-Pro Dataset: Using regex pattern matching to locate response strings matching "answer is (X)" or "answer: (X)". In both benchmarks, we first strip reasoning scratchpads (e.g., extracting text following </think> tags for DeepSeek models). Also, if a model generation is truncated due to budget constraints or fails answer extraction, it is assigned a accuracy score of 00 (00 reward). Action Space and Discretization. Across both tasks, the action space for the bandit is discretized into N=21N=21 linearly spaced token bins spanning dataset- and model-specific minimum (xminx_ ) and maximum (xmaxx_ ) token limits. (xmin,xmaxx_ ,x_ ) Model Family Hugging Face Repository ID MATH MMLU-Pro Llama 3.2 meta-llama/Llama-3.2-1B-Instruct [100,1000][100,1000] [100,1000][100,1000] meta-llama/Llama-3.2-3B-Instruct [100,1000][100,1000] [100,1000][100,1000] DeepSeek R1 deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B [1000,5000][1000,5000] [400,5000][400,5000] deepseek-ai/DeepSeek-R1-Distill-Qwen-7B [1000,5000][1000,5000] [400,5000][400,5000] Gemma 4 google/gemma-4-E2B-it [400,3000][400,3000] [400,3000][400,3000] google/gemma-4-E4B-it [400,3000][400,3000] [400,3000][400,3000] Table 4: Model identifiers and corresponding discretized token budget bounds (xmin,xmaxx_ ,x_ ) Hyperparameters and Reproducibility. All experiments use a single fixed random seed of 4242 (seed=42seed=42). The decision agent runs for 2,0002,000 episodes on MATH and MMLU-Pro, using a batch size parameter of k=16k=16, i.e., in each round, the reward is the accuracy over a set of 16 questions. All code is attached as a zip. A more organized version will be made publicly available upon acceptance. LinUCB is a contextual bandit algorithm [20]: it assumes each arm’s expected reward is a linear function of the current context, and maintains a running estimate of the weights (using ridge-regression) from observed rewards. Similar to UCB, at each round, it adds an exploration bonus which shrinks as more data accumulates for that arm. We use a smaller exploration weight at the model-choice level (γ=1γ=1) and a larger one at the token-budget level (γ=2γ=2): since the model-choice level’s reward estimate depends on the token-budget policy already being close to its optimum, we use a larger exploration weight at the token-budget level, which has a larger action space (21 arms vs. 2), to ensure its reward estimates are reliable before the model-choice level’s decision settles. Figure 7: Accuracy vs budget for all 3 model families and 2 task domains, along with the fitted curves. (a) DeepSeek R1 1.5B vs. 7B (b) Gemma E2B vs. E4B (c) Llama 1B vs. 3B (d) Llama 1B vs. DeepSeek 1.5B (e) Llama 1B vs. Gemma 4B (f) DeepSeek 1.5B vs. Gemma 4B Figure 8: Learned policies of various model pairings in the MATH domain. (a) DeepSeek R1 1.5B vs. 7B (b) Gemma E2B vs. E4B (c) Llama 1B vs. 3B (d) Llama 1B vs. DeepSeek 1.5B (e) Llama 1B vs. Gemma 4B (f) DeepSeek 1.5B vs. Gemma 2B (g) DeepSeek 1.5B vs. Gemma 4B (h) DeepSeek 1.5B vs. Llama 3B Figure 9: Learned policies for the MMLU Pro domain. (a) DeepSeek R1 1.5B vs. 7B (b) Gemma E2B vs. E4B (c) Llama 1B vs. 3B (d) Llama 1B vs. Gemma 4B Figure 10: Principal’s Learning for different model pairings on the MATH task (a) Llama 1B vs. 3B (b) Gemma E2B vs. E4B Figure 11: Agent using simple UCB in MATH task for two different pairings. The context α is discretized into 10 bins, and we use a standard UCB instead of LinUCB with handcrafted features. The Agent considers each bin to be independent, and therefore takes a longer time to converge, and these were trained for 5,0005,000 episodes. Appendix F Appendix F: LLM as Controller Instead of the LinUCB learning Agent, we replace it with a LLM Controller, where we prompt a LLM with details about the general task domain, the 2 LLM models it has access to, their respective costs, and the profit objective it is supposed to maximize. Note that the prompt does not include any of our theory or calibrated parameters. We instead include a sliding window of the last 2020 decisions including details about the contract offered, the model chosen, token budget allocated, the resulting accuracy and reward. Finally, we append the contract offered in the current round, and prompt it to return a JSON object with model choice, token budget, and a justification. The exact prompt used is in Appendix G. Figure 12: LLM as Controller We experiment with a few different controller models; significantly more advanced and larger models compared to the model choices. We test it out for 220 rounds, with the first 20 rounds not being used in analysis as the sliding window of history is not yet full. We consider Gemma 4 31B, Qwen 3.6 35B, Deepseek R1 70B with zero temperature. We test this on the MATH domain and the Llama 1B vs. 3B pairing, and the observed results are shown in Figure 12. Recall that the controller receives a contract stake αt _t drawn uniformly at random from the domain αt∈[0,1] _t∈[0,1]. So, we discretize the contracts into 20 bins and average the model choices within each bin; the standard errors are also reported. All 3 controllers pick up the qualitative direction of model choice, with P(H)P(H) rising with α and saturating near the theoretical threshold. The sharpness of this switch is not the same as the bandit controller and varies significantly within these models; the Qwen and Deepseek controllers pick model H at higher rates than optimal below the threshold. On token budgets, all controllers spend less than optimal, with the token budget plateauing past the threshold. We take it as preliminary evidence that finding the Agent’s two-dimensional best response is not trivial, and leave a systematic study of LLM controllers for future work. More importantly, this also does not account for the token cost of prompting the controller itself each round, on top of the Agent’s own token spend for the actual task; a cost the bandit-based approach does not incur. Appendix G Appendix G: Prompts MATH Prompt Template [System Instructions] Please reason step by step, and put your final answer within . [User Payload] Question: [problem_text] MMLU-Pro Prompt Template [System Instructions] The following is a multiple-choice question (with answer) about [category]. Think step by step and then finish your answer with "The answer is (X)" where X is the correct letter choice. [User Payload] Question: [question_text] Options: (A) [Option 1] (B) [Option 2] ... (N) [Option N] Let’s think step by step. LLM Controller Prompt Template [System Instructions] You are an adaptive, economically optimal routing agent. Context: In each round, you manage a batch of 16 mathematical problems sampled uniformly from a fixed distribution (MATH dataset, difficulty levels 3-4). Objective: Maximize net profit per question = (alpha * accuracy_pct) - (cost_per_token * token_budget) Configuration Space: - Option A: ’[Model Low Name]’ (cost: [Cost Low]) - Option B: ’[Model High Name]’ (cost: [Cost High]) - Allowed Budget Range per question: [Token Min] to [Token Max] --- START RECENT PERFORMANCE LOG --- [History Window Log String, e.g., ‘Round i: Alpha=... | Model=... | Budget=...‘] --- END RECENT PERFORMANCE LOG --- Constraint: Respond ONLY with a valid JSON object matching this schema. Choose any integer value between [Token Min] and [Token Max] for token_budget: "justification": "reasoning based on the objective, history and current alpha", "chosen_model": "model_name", "token_budget": <integer value here> [User Payload] Current Contract: alpha = <sampled alpha> Appendix H Appendix H: Extension to N Models We consider ℳ=1,…,NM=\1,…,N\ in place of L,H\L,H\. The per-model quantities τm _m, xm∗(α)x_m^*(α), and Um(α)U_m(α) are unchanged, since they follow from a single-model optimization and do not depend on the size of ℳM. What changes is the Agent’s model choice, which is now m∗(α)=argmaxm∈ℳUm(α),m^*(α)= _m U_m(α), i.e., the Agent’s optimal model choice is based on the upper envelope of N curves of the form (4), rather than a single crossing between two. We characterize this envelope in two steps: first, which models can be removed from consideration entirely; second, whether the remaining models are visited in capability order as α increases. Capability Order Model i is dominated by model j if Mj≥MiM_j≥ M_i and τj≤τi _j≤ _i, with at least one inequality strict. We claim a dominated model is never the Agent’s best response. Suppose τj≤τi _j≤ _i. For α>τiα> _i, both models are active and τj/α≤τi/α _j/α≤ _i/α, so (1−τjα)≥(1−τiα). (1- _jα )≥ (1- _iα ). Combined with Mj≥MiM_j≥ M_i, this gives Uj′(α)≥Ui′(α)U_j (α)≥ U_i (α) for all α>τiα> _i, by the same argument used for the Flipped Order case above. At α=τiα= _i, Ui(τi)=0U_i( _i)=0 while Uj(τi)≥0U_j( _i)≥ 0 (model j is already active, having τj≤τi _j≤ _i). Since UjU_j starts weakly ahead of UiU_i at α=τiα= _i and climbs at least as fast for every α beyond it, Uj(α)≥Ui(α)U_j(α)≥ U_i(α) for all α≥τiα≥ _i; for α<τiα< _i, Ui(α)=0≤Uj(α)U_i(α)=0≤ U_j(α) trivially. So Uj(α)≥Ui(α)U_j(α)≥ U_i(α) on all of [0,1][0,1], and model i never wins the envelope. Removing dominated models, the remaining set 1,…,K⊆ℳ\1,…,K\ has no pair related this way. Sorting by capability, M(1)<⋯<M(K)M_(1)<·s<M_(K), we must also have τ(1)<⋯<τ(K), _(1)<·s< _(K), since a violation would mean some pair is still dominated. We refer to this as capability order: within the surviving set, more capable models are also more costly to activate. Capability order alone does not imply the Agent moves through models 1,…,K1,…,K one at a time as α increases. For i<ji<j define Dij(α)=Uj(α)−Ui(α)D_ij(α)=U_j(α)-U_i(α); as before, DijD_ij is convex with at most one root θij _ij. For three models i<j<ki<j<k, Dik(α)=Dij(α)+Djk(α).D_ik(α)=D_ij(α)+D_jk(α). If θij≤θjk _ij≤ _jk, this forces θik∈[θij,θjk] _ik∈[ _ij, _jk] and model j is optimal on that interval, as expected. But capability order does not guarantee θij≤θjk _ij≤ _jk; if instead θij>θjk _ij> _jk, model j is never optimal on [0,1][0,1] even though it is undominated and correctly placed in capability order, since the Agent prefers switching directly from i to k. For example, consider (M1,τ1)=(10,0.1),(M2,τ2)=(11,0.5),(M3,τ3)=(50,0.6),(M_1, _1)=(10,0.1), (M_2, _2)=(11,0.5), (M_3, _3)=(50,0.6), which are in capability order and pairwise undominated. Direct computation gives D12(1)≈−5.01<0D_12(1)≈-5.01<0, so θ12>1 _12>1; and D23(1)≈2.99>0D_23(1)≈ 2.99>0, so θ23∈(0,1) _23∈(0,1). Since θ12>θ23 _12> _23, model 2 is never the Agent’s choice for any α∈[0,1]α∈[0,1]: it would only become optimal at a contract share past the allowed range. Staircase structure Proposition. Let models 1,…,K1,…,K be undominated and in capability order. If the adjacent switching points satisfy θ1,2<θ2,3<⋯<θK−1,K, _1,2< _2,3<·s< _K-1,K, then m∗(α)=im^*(α)=i for α∈(θi−1,i,θi,i+1)α∈( _i-1,i, _i,i+1), and the K−1K-1 adjacent thresholds fully determine the envelope; the remaining pairwise comparisons are unnecessary. The case K=2K=2 is the base model. For the inductive step, suppose the ordering holds up to model i−1i-1. Since Di−1,iD_i-1,i is convex with a single root at θi−1,i _i-1,i, once model i overtakes model i−1i-1 it remains ahead for all larger α. Any earlier model j<i−1j<i-1 is, by the induction hypothesis, already behind model i−1i-1 once α exceeds θi−2,i−1<θi−1,i _i-2,i-1< _i-1,i, and hence remains behind model i as well by the same convexity argument applied to Dj,iD_j,i. So checking neighbors suffices. When the ordering condition fails, as in the example above, the envelope must be computed from all pairwise thresholds rather than adjacent ones alone; this is the standard problem of finding the upper envelope of pairwise-crossing curves. Principal’s Problem Given the ordering condition of the Proposition, the Principal’s problem extends directly. On each interval (θi−1,i,θi,i+1)( _i-1,i, _i,i+1) where model i is the Agent’s choice, Vi(α)V_i(α) remains single-peaked at τi _i exactly as in (6)-(7), so αi†=clip(τi,θi−1,i,θi,i+1),α∗=argmaxiVi(αi†). _i =clip\! ( _i,\ _i-1,i,\ _i,i+1 ), α^*= _iV_i( _i ). No new derivation is required beyond the base model once the interval structure is known.