Paper deep dive
Outperformance Inverse Optimization: Learning Objective Functions that Outperform Agent Decisions
Akira Kitaoka
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 10/11/2026, 3:26:50 AM
Summary
This paper introduces 'Outperformance Inverse Optimization' (Outperformance IO), a framework for learning objective function weights that induce optimal solutions outperforming observed agent actions, rather than merely reproducing them as optimal. Unlike existing methods for Mixed-Integer Linear Programs (MILPs) that learn compromise weights for suboptimal observations, this approach uses a novel 'Outperformance Suboptimality (OS) loss' evaluable via forward-problem oracles. The authors propose gradient-based (Projected Subgradient, Online Newton Step, MetaGrad) and DC (Difference of Convex) optimization algorithms to minimize this loss. Theoretical analysis provides a tight bound on the generalization error, inversely proportional to the number of observations. Experiments on synthetic and real data demonstrate that the proposed methods improve the prediction of outperforming solutions compared to existing inverse optimization techniques.
Entities (8)
Relation Signals (6)
Outperformance Inverse Optimization → proposedby → Akira Kitaoka
confidence 99% · Akira Kitaoka Address: NEC Corporation... We propose outperformance inverse optimization
Outperformance Inverse Optimization → appliesto → Mixed-Integer Linear Programs
confidence 98% · is thus applicable to MILPs
Outperformance Inverse Optimization → uses → OS Loss
confidence 97% · as its surrogate loss we introduce the outperformance suboptimality (OS) loss
Generalization Error → boundedby → Number of Observations
confidence 96% · generalization error) is bounded by a quantity inversely proportional to the number of observations
Outperformance Inverse Optimization → optimizedby → DC Optimization
confidence 95% · together with gradient-based and DC optimization algorithms for minimizing it
Outperformance Inverse Optimization → optimizedby → Projected Subgradient Method
confidence 93% · we adopt the projected subgradient method (PSG)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Inverse optimization estimates the weights of an objective function that explain observed decisions as optimal solutions, and is used in a variety of fields. For mixed-integer linear programs (MILPs), existing methods aim to reproduce the observations as optimal solutions, and thus learn compromise weights when the observations are suboptimal. We propose outperformance inverse optimization, which instead seeks weights that induce, at each state, an optimal solution outperforming the observed action in every component. We give a loss function that can be evaluated with forward-problem oracles alone and is thus applicable to MILPs, together with gradient-based and DC optimization algorithms for minimizing it. For weights inducing a unique outperforming optimal solution at all observations, we prove that the probability of failing to induce such a solution at a new state (the generalization error) is bounded by a quantity inversely proportional to the number of observations, and that this bound is tight in the number of observations up to logarithmic factors. In experiments on synthetic and real data, the proposed methods improve the prediction of solutions outperforming the actions over existing methods.
Tags
Links
- Source: https://arxiv.org/abs/2610.09890v1
- Canonical: https://arxiv.org/abs/2610.09890v1
Trouble viewing inline? Open PDF directly →
Full Text
325,090 characters extracted from source content.
Expand or collapse full text
Outperformance Inverse Optimization: Learning Objective Functions that Outperform Agent Decisions Akira Kitaoka Address: NEC Corporation, 1753 Shimonumabe, Nakahara-ku, Kawasaki, Kanagawa, Japan Email address: akira-kitaoka@nec.com Abstract. Inverse optimization estimates the weights of an objective function that explain observed decisions as optimal solutions, and is used in a variety of fields. For mixed-integer linear programs (MILPs), existing methods aim to reproduce the observations as optimal solutions, and thus learn compromise weights when the observations are suboptimal. We propose outperformance inverse optimization, which instead seeks weights that induce, at each state, an optimal solution outperforming the observed action in every component. We give a loss function that can be evaluated with forward-problem oracles alone and is thus applicable to MILPs, together with gradient-based and DC optimization algorithms for minimizing it. For weights inducing a unique outperforming optimal solution at all observations, we prove that the probability of failing to induce such a solution at a new state (the generalization error) is bounded by a quantity inversely proportional to the number of observations, and that this bound is tight in the number of observations up to logarithmic factors. In experiments on synthetic and real data, the proposed methods improve the prediction of solutions outperforming the actions over existing methods. Key words and phrases: inverse optimization, outperformance, suboptimality loss, mixed-integer linear programming, DC optimization, generalization error 1. Introduction Many processes and systems, from human decision making to natural phenomena, can often be described by mathematical optimization. In practice, however, the true objective function of such a model is in many cases unknown in advance. Therefore, the inverse optimization problem (Ahuja and Orlin, 2001; Heuberger, 2004; Chan et al., 2019; Chan et al., 2025), which estimates the objective function from observed data, is of great practical importance. Inverse optimization problems have been studied extensively (Ahuja and Orlin, 2001; Heuberger, 2004; Chan et al., 2019; Chan et al., 2025) and applied in a variety of fields, including geophysics (Tarantola, 2005; Burton and Toint, 1992), transportation (Bertsimas et al., 2015), power systems (Birge et al., 2017), scheduling of television advertisements (Suzuki et al., 2019), and healthcare (Chan et al., 2022). They have also been developed as a foundation of machine learning methods such as inverse reinforcement learning (Ng and Russell, 2000) and contrastive learning (Shi et al., 2023). A challenge in inverse optimization is that the observed actions are not necessarily optimal solutions of the mathematical optimization under consideration (due to, e.g., measurement noise and the bounded rationality of the decision maker; Aswani et al., 2018). For such observations, tractable methods that relax the KKT conditions, the variational inequality, or the strong duality constraint (Keshavarz et al., 2011; Bertsimas et al., 2015; Chan et al., 2014; Chan et al., 2019) have been proposed, but all of them presuppose the convexity of the forward problem and cannot be extended to mixed-integer linear programs (MILPs) (Bärmann et al., 2018). On the other hand, methods (Bärmann et al., 2017; Bärmann et al., 2018; Kitaoka, 2024; Lin et al., 2024; Sakaue et al., 2025a; Sakaue et al., 2025b; Kitaoka, 2026) that use losses evaluable only with an optimization oracle for the forward problem, such as the suboptimality loss (Mohajerin Esfahani et al., 2018), are applicable also to MILPs, but since they search for weights that explain the actions as optimal solutions, for suboptimal actions they learn compromise weights. Moreover, in designing mathematical models, an objective function that reproduces the observed actions may be required, but there are also situations that require one inducing solutions that outperform the observed actions in every indicator. In this paper, we therefore propose a framework that learns weights inducing solutions that outperform the agent’s actions instead of reproducing them. The contributions of this paper are the following four. Formulation and the OS loss We formulate the problem of finding weights that give, at each state, an optimal solution outperforming the agent’s action in every component as outperformance inverse optimization, and as its surrogate loss we introduce the outperformance suboptimality (OS) loss, which can be evaluated using only optimization oracles for the forward problem and its dominance-constrained version (Sections 3 and 4). Minimization methods As methods for minimizing the OS loss, we propose gradient-based optimization methods, including the projected subgradient method, and a DC (difference of convex) optimization method that terminates after finitely many iterations if the inner convex subproblems are solved exactly (Section 5). Moreover, when the feasible region of each state is an M♮-convex set (including M-convex sets) and the weight space is the probability simplex, we show that the uniform weight makes the OS loss 00 at all observations and thus attains its global minimum (Proposition F.24, Corollary F.32). Generalization error For weights inducing a unique outperforming optimal solution at all observations, we show that the probability of failing to induce such a solution at a new state (the generalization error) is bounded by a quantity inversely proportional to the number of observations N, and via a lower bound that this upper bound is tight in N up to log factors (Section 6). Experiments In five experiments with synthetic and real data, we show that, with the optimization method fixed, replacing the suboptimality loss of existing inverse optimization with the OS loss improves the prediction of solutions outperforming the actions (smaller componentwise shortfalls): indeed, the OS-loss variant wins in 14 of the 15 (optimization method, experiment) pairs (Section 7). 2. Related work Details, including classical inverse optimization, are given in Appendix A. Inverse optimization for noisy data As data-driven inverse optimization that handles the case where the observed actions are not optimal solutions, methods have been proposed that use as the loss the distance between the observations and the optimal solutions (Aswani et al., 2018) (NP-hard even when the forward problem is convex), the residual of the KKT conditions (Keshavarz et al., 2011), the violation of the variational inequality (Bertsimas et al., 2015), the relaxation of the strong duality constraint (Chan et al., 2014; Chan et al., 2019), and distributionally robust optimization of the suboptimality loss (Mohajerin Esfahani et al., 2018). However, since these tractable reformulations use the KKT conditions or duality, they presuppose the convexity of the forward problem and cannot be extended to the integer case (Bertsimas et al., 2015; Bärmann et al., 2018). Inverse optimization for MILPs Methods that minimize the suboptimality loss using only the forward-problem oracle do not require the convexity of the feasible region and are therefore applicable to MILPs (Bärmann et al., 2017; Bärmann et al., 2018; Kitaoka, 2024; Lin et al., 2024; Sakaue et al., 2025a; Sakaue et al., 2025b; Kitaoka, 2026). However, since these seek weights that explain the agent’s actions as optimal solutions, when the actions are suboptimal the suboptimality loss is positive even at the true weight (Sakaue et al., 2025a; Sakaue et al., 2025b). Generalization error analysis of inverse optimization Kitaoka (2025) bounded the expected suboptimality loss by O(d/N)O( d/N) for inverse optimization of MILPs that learns both the objective function and the constraints (d is the dimension of the weights and N is the number of observations). Fatemi et al. (2026) bounded, for noiseless inverse optimization in which the action is the unique optimal solution under the true weight, the probability that the true optimal solution ceases to be optimal under the learned weight by O(d/N)O(d/N), and showed that this order is tight. Both presuppose that the agent’s actions are optimal solutions. Position of this paper Rather than explaining the agent’s actions as optimal solutions, this paper seeks weights that induce optimal solutions outperforming the actions in every component. Since the OS loss, like the suboptimality loss, can be evaluated using only the forward-problem oracle, it is applicable to nonconvex forward problems including MILPs, and it can learn weights inducing solutions that outperform the actions even when the actions are suboptimal. In addition, the generalization error analysis (Section 6) evaluates, in a setting where the actions may be suboptimal, the probability that a unique optimal solution outperforming the action is not obtained at a new state. 3. Problem setting We consider an offline learning setting with two players, a learner and an agent.11 1 The “agent” is sometimes called the “expert,” but we do not use this term to avoid confusion with experts in online learning. Let d be a positive integer, and let ℝdR^d be the space on which the forward optimization is defined. We call a nonempty set S the state set, and for each state s∈s we denote the feasible region by (s)(⊆ℝd)X(s)( ^d). For a weight θ∈ℝdθ ^d and a state s∈s , we write the forward problem (linear optimization) and its optimal solution as (3.1) x∗(θ,s):∈argmaxx∈(s)⟨θ,x⟩.x^*(θ,s):∈ _x (s) θ,x . Let Θ⊆ℝ≥0d(:=[0,∞)d) ^d_≥ 0(:=[0,∞)^d) be the weight space. For n=1,…,Nn=1,…,N, given a state sn∈s^n , the agent chooses xn∈(sn)x^n (s^n) as its action. From the observations (sn,xn)n=1N\(s^n,x^n)\_n=1^N, we wish to find a weight θ∗θ^* satisfying, at each state sns^n, (3.2) x∗(θ∗,sn)≥xn,x^*(θ^*,s^n)≥ x^n, where ≥ means that ≥ holds componentwise. Hereafter, when x≥x′x≥ x , we say that x dominates (outperforms) x′x . We call solving Equation 3.2 outperformance inverse optimization (outperformance IO). Each component of x is oriented so that larger is better (e.g., times enter with reversed sign), and observation noise is assumed to keep xnx^n feasible. Note that the set (s)X(s) is not necessarily convex. If (s)X(s) is the feasible region of a MILP, the solution of any MILP solver, e.g., Gurobi (Gurobi Optimization, LLC, 2026) or SCIP via OR-Tools (Perron and Furnon, 2026), can serve as the oracle x∗(θ,s)x^*(θ,s). 4. Performance metrics For s∈s , we define an optimal solution that dominates x′∈(s)x (s) as follows (we call this problem the dominance-constrained forward problem): (4.1) x∗(θ,x′,s):∈argmaxx∈≥x′(s)⟨θ,x⟩,≥x′(s):=x∈(s)|x≥x′.x^*(θ;x ,s):∈ _x _≥ x (s) θ,x , _≥ x (s):=\x (s)\,|\,x≥ x \. As a metric useful for outperformance inverse optimization, we define the outperformance suboptimality loss (OS loss) ℓOS ^OS for any s∈s and data x′∈(s)x (s) by (4.2) ℓOS(θ,x′,s):=maxx∈(s)⟨θ,x⟩−maxx∈≥x′(s)⟨θ,x⟩. ^OS(θ;x ,s):= _x (s) θ,x - _x _≥ x (s) θ,x . By the inclusion (s)⊇≥x′(s)X(s) _≥ x (s), the OS loss ℓOS ^OS is nonnegative. The OS loss ℓOS(θ,x′,s) ^OS(θ;x ,s) is 00 if and only if there exists an optimal solution x∈argmaxx∈(s)⟨θ,x⟩x∈ _x (s) θ,x satisfying x≥x′x≥ x . In particular, if no point of ≥x′(s)X_≥ x (s) is optimal under any θ∈Θθ∈ (e.g., an efficient but unsupported action; Appendix B), no weight attains Equation 3.2. However, when the optimal solution is not unique, the optimal solution x∗(θ,s)x^*(θ,s) returned by the oracle does not necessarily dominate x′x . To solve Equation 3.2, we introduce the empirical mean of the OS loss (4.3) ℓ¯OS(θ):=1N∑n=1NℓOS(θ,xn,sn). ^OS(θ):= 1N _n=1^N ^OS(θ;x^n,s^n). The empirical mean of the OS loss ℓ¯OS(θ) ^OS(θ) is 00 if and only if, for each n, there exists an optimal solution x∈argmaxx∈(sn)⟨θ,x⟩x∈ _x (s^n) θ,x that dominates xnx^n. In particular, if θ satisfies Equation 3.2, then ℓ¯OS(θ)=0 ^OS(θ)=0 since x∗(θ,sn)∈≥xn(sn)x^*(θ,s^n) _≥ x^n(s^n). Conversely, even if ℓ¯OS(θ)=0 ^OS(θ)=0, when the optimal solution is not unique, x∗(θ,sn)x^*(θ,s^n) returned by the oracle does not necessarily dominate xnx^n, and attaining Equation 3.2 is not guaranteed. Accordingly, since a weight satisfying Equation 3.2 attains the minimum value 00 of the empirical mean of the OS loss ℓ¯OS ^OS, we treat Equation 3.2 as the following problem using the empirical mean of the OS loss ℓ¯OS ^OS: (4.4) minimizeℓ¯OS(θ) such that θ∈Θ.minimize\, ^OS(θ) such that θ∈ . Remark 4.1. The suboptimality loss is defined by ℓsub(θ,x′,s):=maxx∈(s)⟨θ,x⟩−⟨θ,x′⟩ ^sub(θ;x ,s):= _x (s) θ,x - θ,x (Mohajerin Esfahani et al., 2018). To solve outperformance inverse optimization, we introduce the following assumption. Assumption 4.2. (1): The weight space Θ⊂ℝ≥0d ^d_≥ 0 is a nonempty bounded polyhedron Θ=θ∈ℝ≥0d∣AΘθ≤bΘ =\θ ^d_≥ 0 A_ θ≤ b_ \ with AΘ∈ℝmΘ×dA_ ^m_ × d and bΘ∈ℝmΘb_ ^m_ , and ∉Θ0∉ (in the implementation, we take the probability simplex Δd−1:=θ∈ℝ≥0d∣∑i=1dθi=1 ^d-1:=\θ ^d_≥ 0 _i=1^d _i=1\). (2): For each state s∈s , (s)⊂ℝdX(s) ^d is a finite union of nonempty bounded polyhedra (we call each bounded polyhedron constituting the union a piece). Moreover, (s)X(s) is uniformly bounded in s, that is, sups∈supx∈(s)‖x‖<∞ _s _x (s)\|x\|<∞. (3): (Extreme-point oracle) For any θ∈ℝdθ ^d, the forward-problem oracle x∗(⋅,s)x^*(·,s) and the dominance-constrained forward-problem oracle x∗(⋅,x′,s)x^*(·\,;x ,s) return a vertex of a bounded polyhedral piece attaining the maximum. We denote the sets of all values that the forward-problem oracle x∗(⋅,s)x^*(·,s) and the dominance-constrained forward-problem oracle x∗(⋅,x′,s)x^*(·\,;x ,s) can return by (s)Y(s) and ≥x′(s)Y_≥ x (s), respectively. Remark 4.3 (On Assumption 4.2). By (2), (s)X(s) is nonempty and compact, and for x′∈(s)x (s) the dominance-constrained feasible region ≥x′(s):=x∈(s)∣x≥x′X_≥ x (s):=\x (s) x≥ x \ (Equation 4.1) is also a finite union of nonempty bounded polyhedra. (3) is a natural assumption as the standard behavior of LP and MILP solvers based on the simplex method and branch-and-bound. Moreover, (s)Y(s) and ≥x′(s)Y_≥ x (s) are finite subsets of (s)X(s), since they consist of vertices of the bounded polyhedral pieces. 5. Proposed methods 5.1. Gradient-based optimization methods Proposition 5.1. Assume Assumption 4.2(2). Let s∈s and x′∈(s)x (s). Then g(θ,x′,s):=x∗(θ,s)−x∗(θ,x′,s)g(θ;x ,s):=x^*(θ,s)-x^*(θ;x ,s) is a gradient of ℓOS(θ,x′,s) ^OS(θ;x ,s) with respect to θ in the sense of weak derivatives (derivatives in the sense of Schwartz distributions; Kunzinger, 2019, cf.).22 2 For a continuously differentiable function, the weak derivative coincides with the ordinary gradient. Moreover, by Rademacher’s theorem (cf. Villani, 2009, Theorem 10.8 (i)), ℓOS ^OS is differentiable almost everywhere, and g=∇θℓOSg= _θ ^OS holds at such points (Appendix C). For the proof, see Appendix C. By Proposition 5.1, a gradient of the empirical mean of the OS loss ℓ¯OS ^OS in the sense of weak derivatives is given by 1N∑n=1Ng(θ,xn,sn) 1N _n=1^Ng(θ;x^n,s^n). Therefore, gradient-based optimization methods can be applied to solve Equation 4.4, although they may stall at a local minimum or a stationary point without reaching the minimum value. In particular, in its simplest form, the projected subgradient method for the OS loss is the update θk+1←ΠΘ(θk−αkN∑n=1N(x∗(θk,sn)−x∗(θk,xn,sn)))θ^k+1← _ (θ^k- _kN _n=1^N(x^*(θ^k,s^n)-x^*(θ^k;x^n,s^n)) ) (ΠΘ _ is the Euclidean projection onto Θ and αk>0 _k>0 is the learning rate at iteration k), which differs from that for the suboptimality loss (Kitaoka, 2024) only in that the observed action xnx^n is replaced by x∗(θk,xn,sn)x^*(θ^k;x^n,s^n) (Algorithm 8). In this paper, as gradient-based optimization methods, we adopt the projected subgradient method (PSG) (Boyd et al., 2003, cf.), the Online Newton Step (ONS) (Hazan et al., 2007), and MetaGrad (van Erven and Koolen, 2016). For the Online Newton Step and MetaGrad, we adapt the implementation that Sakaue et al. (2025b) used for online inverse linear optimization. Although these are online methods, we update them with the average of the gradients over all states at each iteration (full-batch). Implementation details are given in Appendix H. 5.2. DC optimization We call a tuple of dominating feasible solutions y′=(y′n)n=1Ny =(y n)_n=1^N (y′n∈≥xn(sn)y n _≥ x^n(s^n)) a branch (not to be confused with branching in branch-and-bound). Fixing a branch replaces the second term of Equation 4.2 (concave in θ) with the linear function ⟨θ,y′n⟩ θ,y n , which yields the convex upper bound on the empirical mean of the OS loss (5.1) ℓ¯sub(θ,y′):=1N∑n=1N[maxx∈(sn)⟨θ,x⟩−⟨θ,y′n⟩]≥ℓ¯OS(θ). ^sub(θ;y ):= 1N _n=1^N [ _x (s^n) θ,x - θ,y n ]\;≥\; ^OS(θ). We call minθ∈Θℓ¯sub(θ,y′) _θ∈ ^sub(θ;y ) the convex subproblem of the branch y′y . We denote the set of all branches by ≥all:=∏n=1N≥xn(sn)X^all_≥:= _n=1^NX_≥ x^n(s^n) and the set of those consisting of values that the dominance-constrained forward-problem oracle can return by ≥all:=∏n=1N≥xn(sn)Y^all_≥:= _n=1^NY_≥ x^n(s^n), and we let msub(y′):=minθ∈Θℓ¯sub(θ,y′)m_sub(y ):= _θ∈ ^sub(θ;y ) be the minimum value of the convex subproblem of a branch y′∈≥ally ^all_≥ (the minimum is attained since Θ is compact and ℓ¯sub(⋅,y′) ^sub(·\,;y ) is continuous). Proposition 5.2 (Decomposition into branches). Under Assumption 4.2, for any θ∈ℝdθ ^d, we have (5.2) ℓ¯OS(θ)=miny′∈≥allℓ¯sub(θ,y′)=miny′∈≥allℓ¯sub(θ,y′). ^OS(θ)= _y ^all_≥ ^sub(θ;y )= _y ^all_≥ ^sub(θ;y ). Therefore, for the finite set ≥allY^all_≥, we have (5.3) minθ∈Θℓ¯OS(θ)=miny′∈≥allmsub(y′). _θ∈ ^OS(θ)= _y ^all_≥m_sub(y ). The proof is given in Section E.2. Proposition 5.2 specializes to our problem the result of Sections 4.1–4.2 of Pham Dinh and Le Thi (1997) that a global solution of a polyhedral DC program reduces to finitely many convex programs (the DC structure of the empirical mean of the OS loss is described in Remark E.1, and the correspondence in Section E.5). Our DC optimization uses this structure directly. By Equation 5.2, Equation 4.4 is equivalent to the joint minimization of ℓ¯sub(θ,y′) ^sub(θ;y ) over (θ,y′)∈Θ×≥all(θ,y )∈ ×Y^all_≥, and Algorithm 6 in the appendix (Appendix E) performs this joint minimization alternately in y′y and θ. The minimization in y′y amounts to updating the branch by calling the dominance-constrained forward-problem oracle at each state, and the minimization in θ amounts to solving the convex subproblem of the branch. This alternating minimization is nothing but the simplified DCA (a simplified version of the DC Algorithm) of Pham Dinh and Le Thi (1997) (Remarks E.1 and E.5). Moreover, if we take the agent’s actions themselves as the branch, ℓ¯sub(θ,(xn)n=1N) ^sub(θ;(x^n)_n=1^N) is the empirical mean of the suboptimality loss (Mohajerin Esfahani et al., 2018). Hence the first equality of Equation 5.2 means that the empirical mean of the OS loss is the pointwise minimum of the empirical means of the suboptimality loss obtained by replacing the agent’s actions with feasible solutions dominating them. In this paper, we solve Equation 4.4 by a double-loop method with this simplified DCA as the outer loop and a solver such as the cutting-plane method (Kelley, 1960) (the inner solver) as the inner loop. Details such as the pseudocode are given in Appendix E. If the inner convex subproblems are solved exactly, this double-loop method terminates after finitely many iterations (Proposition E.4). If the optimal solutions of the dominance-constrained forward problems are unique at a point where Algorithm 6 stops because the branch is no longer updated (a fixed point of DCA), that point is a local minimizer of the empirical mean of the OS loss (Proposition E.5; the set of weights at which the optimal solution is not unique has measure 00, Proposition E.8). However, reaching the global minimum is not guaranteed. Since the OS loss is nonnegative, if the loss value becomes 00, it can be confirmed that the point is a global minimizer (Proposition E.9). In particular, when the feasible region of each state is an M♮-convex set (including M-convex sets) and Θ=Δd−1 = ^d-1, the uniform weight makes the empirical mean of the OS loss 00 and attains the global minimum (Proposition F.24, Corollary F.32). 6. Generalization error analysis In this section, we evaluate whether weights learned from the observations induce a unique optimal solution outperforming the agent’s action also at states that were not observed, that is, the generalization of Equation 3.2. We show that the generalization error (Definition 6.2) of weights that give such optimal solutions at all observations is uniformly bounded by order d/Nd/N up to log factors for fixed κ (Assumption 6.1) (in general, the bound contains a term dlogκ/Nd κ/N), and that this upper bound is tight up to log factors. The proofs are given in Appendix G. Suppose that the pair (s,x)(s,x) of a state s and the agent’s action x at that state follows a distribution P, that the observations (sn,xn)n=1N\(s^n,x^n)\_n=1^N are i.i.d. samples from P, and we denote the probability with respect to the distribution of the observations by ℙP (see Section G.1 for details of the setting). We make the following assumption. Assumption 6.1. (1): κ:=esssups|(s)|κ:=ess\,sup_s|Y(s)| satisfies 2≤κ<∞2≤κ<∞. (2): (Realizability) There exists θ∗∈Θθ^*∈ such that, a.s. for (s,x)∼(s,x) , the optimal solution of the forward problem Equation 3.1 at the state s is unique and that optimal solution x∗(θ∗,s)x^*(θ^*,s) dominates the agent’s action x (x∗(θ∗,s)≥x^*(θ^*,s)≥ x). We define the generalization error as follows. Definition 6.2. For a weight θ∈Θθ∈ , a state s∈s , and data x′∈ℝdx ^d, we define the outperformance error by ℓ01(θ,x′,s):=0if argmaxy∈(s)⟨θ,y⟩=v for some v≥x′,1otherwise, ^01(θ;x ,s):= cases0&if _y (s) θ,y =\v\ for some v≥ x ,\\ 1&otherwise, cases and define the generalization error as the probability GE(θ):=(ℓ01(θ,x,s)=1)GE(θ):=P ( ^01(θ;x,s)=1 ) with respect to (s,x)∼(s,x) . We say that θ is consistent with the observations if ℓ01(θ,xn,sn)=0 ^01(θ;x^n,s^n)=0 for all n. The upper bound is as follows. Theorem 6.3. Assume Assumption 4.2(2)(3) and Assumption 6.1(1), and let N≥dN≥ d. For any β∈(0,1)β∈(0,1), with probability at least 1−β1-β, any weight θ^N∈Θ θ^N∈ consistent with the observations (it may depend on the observations) satisfies GE(θ^N)≤2N(dlog22eNκ2d+log22β),GE( θ^N)≤ 2N (d _2 2eNκ^2d+ _2 2β ), where ee is Napier’s constant. Example 6.4 (Integer programming). In addition to the assumptions of Theorem 6.3, suppose that there exist positive integers M1,…,MdM_1,…,M_d such that (s)⊆∏i=1d0,1,…,MiX(s) _i=1^d\0,1,…,M_i\ holds a.s. Since κ≤∏i=1d(Mi+1)κ≤ _i=1^d(M_i+1) by (s)⊆(s)Y(s) (s) (Remark 4.3), Theorem 6.3 implies that, for any β∈(0,1)β∈(0,1), with probability at least 1−β1-β, any weight θ^N∈Θ θ^N∈ consistent with the observations satisfies GE(θ^N)≤2N(dlog22eNd+2d∑i=1dlog2(Mi+1)+log22β).GE( θ^N)≤ 2N (d _2 2eNd+2d _i=1^d _2(M_i+1)+ _2 2β ). In particular, for 0-1 integer programming (Mi=1M_i=1), the second term in the parentheses on the right-hand side is 2d22d^2. As in Example 6.4, κ can depend exponentially on d, in which case the upper bound of Theorem 6.3 contains a term of order d2/Nd^2/N. Since Theorem 6.3 does not depend on a specific estimator, it applies directly to the weight reached by the methods of Section 5 whenever that weight has OS loss 00 and a unique optimal solution at all observations. The lower bound is as follows. It is obtained by embedding the instance that Fatemi et al. (2026) constructed to show that their upper bound cannot be improved (Theorem 3.1 of that paper; the case where the agent’s action is the unique optimal solution under the true weight) into our setting, in which the weights are restricted to the probability simplex. Theorem 6.5. For any integer d0≥2d_0≥ 2, with d=2d0+1d=2d_0+1, there exists a tuple of a state set S, a distribution P, feasible regions (s)X(s), and the weight space Θ=Δd−1 = ^d-1 that satisfies Assumption 4.2 and Assumption 6.1 with κ=3κ=3 such that, for any β∈(0,1/2]β∈(0,1/2] and N≥maxd0,ln(1/β)N≥ \d_0, (1/β)\, there exists a weight θ^N∈Θ θ^N∈ consistent with the observations (depending on the observations) that satisfies, with probability at least β, GE(θ^N)>d0+ln(1/β)4N=d−1+2ln(1/β)8N.GE( θ^N)> d_0+ (1/β)4N= d-1+2 (1/β)8N. Since Theorem 6.3 is an upper bound holding with probability at least 1−β1-β and Theorem 6.5 is a lower bound holding with probability at least β, we compare them by applying Theorem 6.3 with β/2β/2. Then, for β∈(0,1/2]β∈(0,1/2] and N≥maxd,ln(1/β)N≥ \d, (1/β)\, both events hold simultaneously with probability at least β/2β/2, so the right-hand side of Theorem 6.3 with β replaced by β/2β/2, for κ=3κ=3 and d=2d0+1d=2d_0+1, is larger than that of Theorem 6.5. This replacement only changes the constant inside the logarithm, so tightness up to log factors is preserved. That is, Theorem 6.5 guarantees the tightness of Theorem 6.3. 7. Numerical experiments In this section, we compare outperformance inverse optimization (minimization of the empirical mean of the OS loss Equation 4.3) with existing inverse optimization (minimization of the suboptimality loss) in terms of the degree to which Equation 3.2 is attained. The five experiments are the knapsack problem and the single-machine scheduling problem with release times using synthetic data, the movie selection problem with viewing costs using MovieLens100K (Harper and Konstan, 2015), and the weighted multi-cover problem (selection of reading lists and reference lists) using Cora (Sen et al., 2008) and ogbn-arxiv (Hu et al., 2020). The results of all methods, implementation details, hyperparameters, and the experimental environment are given in Appendix H. 7.1. Experimental setup Compared methods In the main text, we compare seven methods. Sub- and OS- indicate minimizing the empirical mean of the suboptimality loss ℓ¯sub(θ,(xn)n=1N) ^sub(θ;(x^n)_n=1^N) (Equation 5.1) and the empirical mean of the OS loss, respectively, and the following name indicates the optimization method (PSG: projected subgradient method (Boyd et al., 2003, cf.), ONS (Hazan et al., 2007), MetaGrad (van Erven and Koolen, 2016), (FB): the full-batch version using the average of the subgradients (weak derivatives for the OS loss) over all states at each iteration). The comparison of the three pairs consisting of the existing methods Sub-PSG (Kitaoka, 2024), Sub-ONS(FB), and Sub-MetaGrad(FB) (Sakaue et al., 2025b) and the proposed methods OS-PSG, OS-ONS(FB), and OS-MetaGrad(FB), each obtained by replacing only the loss function, represents the effect of the difference in the loss function. We call the OS- method of each pair the OS-loss variant and the Sub- method the suboptimality-loss variant. The remaining method, OS-DC, is the proposed method based on the DC optimization of Section 5.2. As the inner solver we use the proximal bundle method with cutting-plane finishing (Section D.4), and after DC optimization stops, we apply the projected subgradient method with a small step size to the empirical mean of the OS loss, starting from the obtained weight (subgradient finishing; OS-DC[PB+CPF]+SGF in Table 3). For all methods, the initial point is the uniform weight θ=/dθ=1/d and the weight space is the probability simplex Θ=Δd−1 = ^d-1. The list of all 13 methods including these (Table 3), and the detailed settings and hyperparameters, are given in Appendix H. Evaluation metrics The main evaluation metric is the outperformance prediction loss ℓOP(θ):=1N∑n=1N∑i=1dmaxxin−xi∗(θ,sn),0 ^OP(θ):= 1N _n=1^N _i=1^d \x_i^n-x_i^*(θ,s^n),0\ (Equation E.6). That is, ℓOP ^OP is the average over states of the sum of the componentwise shortfalls of the oracle’s solution relative to the agent’s action. We have ℓOP(θ)=0 ^OP(θ)=0 if and only if θ is a solution of outperformance inverse optimization Equation 3.2. In addition, we record the attainment rate of ℓOP=0 ^OP=0 (the fraction of episodes—the unit in which one set of observations (sn,xn)n=1N\(s^n,x^n)\_n=1^N is generated or extracted and all methods are run on it—that reached ℓOP=0 ^OP=0) and the empirical mean of the OS loss ℓ¯OS ^OS. Measure of computational cost We measure the computational cost by the number of calls to the forward-problem oracles. Calls to x∗(⋅,s)x^*(·,s) and to x∗(⋅,x′,s)x^*(·\,;x ,s) are both counted as one, and calls to x∗(⋅,x′,s)x^*(·\,;x ,s), which the existing methods do not solve, are counted only on the side of the proposed methods. Letting T be the number of iterations of OS-PSG (Sub-PSG, which makes half as many oracle calls per iteration, runs 2T2T iterations) and N the number of states, we impose a common oracle-call budget B:=2TNB:=2TN on all methods, evaluate each method by the weight with the smallest ℓOP ^OP among the weights visited so far (best-so-far), and compare the values at the point where the number of calls reaches B. Details of how the calls are counted and of the evaluation procedure are given in Section H.5. 7.2. Overview of the experiments In the main text, among the five experiments, we present the results of the knapsack problem with synthetic data and the weighted multi-cover problem with real data (Cora (Sen et al., 2008), ogbn-arxiv (Hu et al., 2020)). The results of the single-machine scheduling problem and the movie selection problem with viewing costs are given in Section H.8, and the mathematical models and instances of all experiments in Section H.1. The knapsack problem uses the bounded integer knapsack problem as the forward problem, and the agent’s action is obtained by decreasing each component of the optimal solution under the true weight θ∗θ^* by 11 with probability 0.20.2. Hence a weight θ∗θ^* satisfying Equation 3.2 exists. The weighted multi-cover problem models, as a weighted multi-cover of words, the situation in which a researcher selects a limited number of papers, and the agent’s action is the depth to which the set of papers actually in citation relations covers each word (the depth to which a word is covered is the number of selected papers containing that word, capped at an upper bound r). Except for the single-machine scheduling problem (N=10N=10, d∈4,6,8d∈\4,6,8\, B=2000B=2000), we set N=20N=20, d∈20,40,60,80,100d∈\20,40,60,80,100\, and T=100T=100 (B=4000B=4000); in each episode, all methods use the same instance. 7.3. Experimental results Table 1. Comparison of the three pairs differing only in the loss function (OS-PSG vs. Sub-PSG, OS-ONS(FB) vs. Sub-ONS(FB), OS-MetaGrad(FB) vs. Sub-MetaGrad(FB)). The numbers of episodes on the same instances in which ℓOP ^OP at the point where the number of calls reaches B is smaller/equal/larger for the OS-loss variant (wins/ties/losses). The larger of wins and losses is shown in bold. Problem PSG ONS(FB) MetaGrad(FB) Episodes Knapsack 241/9/0 75/27/148 230/4/16 250 Single machine 24/51/0 30/42/3 28/47/0 75 Movie (MovieLens) 97/2/1 45/23/32 90/9/1 100 Multi-cover (Cora) 92/2/6 59/19/22 72/11/17 100 Multi-cover (ogbn-arxiv) 81/15/4 74/22/4 63/30/7 100 Table 1 shows, for all five experiments, the wins and losses of the three pairs differing only in the loss function; Table 2 shows the attainment rates for the knapsack problem; and Figure 1 shows the evolution of ℓOP ^OP for the weighted multi-cover problem (that for the knapsack problem is shown in Figure 2 in the appendix). Overall, when the optimization method is fixed and only the suboptimality loss is replaced with the OS loss, the prediction of solutions outperforming the agent’s actions improves. Indeed, the OS-loss variant won in 14 of the 15 pairs. A similar tendency is observed in the comparison replacing only the loss function also for the single-machine scheduling problem and the movie selection problem with viewing costs in the appendix (Section H.8). Table 2. Knapsack problem: the fraction [%] of episodes that attained ℓOP=0 ^OP=0 at the point where the number of calls reaches B=4000B=4000. 50 episodes for each d. For each optimization method, the OS loss (OS) and the suboptimality loss (Sub) are shown side by side, and the higher one in each pair is shown in bold. The DC optimization uses only the OS loss. The median of ℓOP ^OP for each dimension is shown in Table 4. PSG ONS(FB) MetaGrad(FB) OS-DC d OS Sub OS Sub OS Sub 20 98 16 34 0 86 6 90 40 96 0 2 0 52 0 88 60 92 0 0 0 46 0 86 80 90 0 0 0 8 0 82 100 76 0 0 0 12 0 84 (a) Weighted multi-cover problem (Cora) (b) Weighted multi-cover problem (ogbn-arxiv) Figure 1. Relation between the worst 10% (the 90th percentile across episodes) of ℓOP ^OP (best-so-far) and the number of oracle calls. The vertical axis is ℓOP+0.01 ^OP+0.01 on a logarithmic scale, and 0.010.01 represents ℓOP=0 ^OP=0. Since the actions are integer-valued, ℓOP ^OP takes discrete values in steps of 1/N1/N, and the percentiles are chosen from the observed values without interpolation. The colors indicate the optimization methods (PSG, MetaGrad(FB), ONS(FB), DC optimization); solid lines indicate the OS loss (proposed) and dashed lines the suboptimality loss (existing). Knapsack problem In this problem, where the agent’s actions are degraded versions of optimal solutions, the improvement is the most pronounced. OS-PSG and OS-DC attained Equation 3.2 in at least 76% of the episodes in all dimensions, whereas the attainment rate of the methods with the suboptimality loss is at most 16% (Table 2; the attainment rates of Sub-ONS and Sub-MetaGrad, which update state by state (Table 3), are also at most 9% when aggregated over all dimensions; Table 8). Existing inverse optimization searches for weights that make the degraded actions optimal and therefore moves toward compromise weights that fit the degradation, whereas the OS loss measures the gap to the best feasible solution dominating the action and is thus presumably less affected by it. However, only in the ONS(FB) pair did the suboptimality-loss variant win (75 wins and 148 losses; Table 1). In this pair, the attainment rate is 00% for both losses for d≥60d≥ 60 (Table 2), presumably because the full-batch ONS does not converge within the oracle-call budget B. Weighted multi-cover problem In this problem, which uses the actions of real data, on both Cora and ogbn-arxiv the OS-loss variant won in all three pairs (Table 1). In particular, in the PSG pair, the OS-loss variant won in 92 of 100 episodes on Cora and 81 of 100 on ogbn-arxiv. 8. Conclusion In this paper, we proposed outperformance inverse optimization. Instead of explaining the actions as optimal solutions, it seeks weights that induce, at each state, an optimal solution outperforming the action in every component. Its surrogate, the OS loss, requires only oracles for the forward problem and its dominance-constrained version, and thus applies to nonconvex forward problems including MILPs. Its empirical mean vanishes if and only if optimal solutions dominating the actions exist. To minimize the OS loss, we gave gradient-based optimization methods and a DC optimization that terminates finitely if the inner convex subproblems are solved exactly. When each feasible region is M♮-convex (including M-convex) and Θ=Δd−1 = ^d-1, the uniform weight attains the global minimum 00 of the OS loss. We also showed that the upper bound on the generalization error is tight up to log factors for fixed κ. In the experiments, replacing only the loss function made the OS-loss variant win in 14 of 15 pairs, with particularly large gains when the actions were degraded optimal solutions. We list two directions for future work. The first is to clarify when the DC optimization for the OS loss reaches a global minimizer and how it can be made to reach one. The second is to clarify whether the dependence of the upper bound on the generalization error on κ (which yields a term of order d2/Nd^2/N for integer programming; Example 6.4) is essential, and whether an upper bound without log factors can be obtained for a specific estimator. References Ahuja and Orlin (2001) R. K. Ahuja and J. B. Orlin Inverse optimization. Operations Research 49 (5), p. 771–783. External Links: Document Cited by: Appendix A, §1, §1. Aswani et al. (2018) A. Aswani, Z. Shen, and A. Siddiq Inverse optimization with noisy data. Operations Research 66 (3), p. 870–892. External Links: Document Cited by: Appendix A, §1, §2. Bärmann et al. (2018) A. Bärmann, A. Martin, S. Pokutta, and O. Schneider An online-learning approach to inverse optimization. Note: Available at arXiv:1810.12997v2 Cited by: Appendix A, §1, §2, §2. Bärmann et al. (2017) A. Bärmann, S. Pokutta, and O. Schneider Emulating the expert: inverse optimization through online learning. In The 34th International Conference on Machine Learning, Vol. 70, p. 400–410. Cited by: Appendix A, §E.2, §E.5, §1, §2. Bertsimas et al. (2015) D. Bertsimas, V. Gupta, and I. C. Paschalidis Data-driven estimation in equilibrium using inverse optimization. Mathematical Programming 153 (2), p. 595–633. External Links: Document Cited by: Appendix A, §1, §1, §2. Besbes et al. (2025) O. Besbes, Y. Fonseca, and I. Lobel Contextual inverse optimization: offline and online learning. Operations Research 73 (1), p. 424–443. External Links: Document Cited by: Appendix A. Birge et al. (2017) J. R. Birge, A. Hortaçsu, and J. M. Pavlin Inverse optimization for the recovery of market structure from market outcomes: an application to the MISO electricity market. Operations Research 65 (4), p. 837–855. External Links: Document Cited by: §1. Boyd et al. (2003) S. Boyd, L. Xiao, and A. Mutapcic Subgradient methods. Note: Notes for E392o, Stanford University, Autumn, 2003 Cited by: §5.1, §7.1. Bulut and Ralphs (2021) A. Bulut and T. K. Ralphs On the complexity of inverse mixed integer linear optimization. SIAM Journal on Optimization 31 (4), p. 3014–3043. External Links: Document Cited by: Appendix A. Burton and Toint (1992) D. Burton and P. L. Toint On an instance of the inverse shortest paths problem. Mathematical Programming 53, p. 45–61. External Links: Document Cited by: §1. Campi and Garatti (2008) M. C. Campi and S. Garatti The exact feasibility of randomized solutions of uncertain convex programs. SIAM Journal on Optimization 19 (3), p. 1211–1230. External Links: Document Cited by: Appendix A, §G.3, Proposition G.7. Chan et al. (2014) T. C. Y. Chan, T. Craig, T. Lee, and M. B. Sharpe Generalized inverse multiobjective optimization with application to cancer therapy. Operations Research 62 (3), p. 680–695. External Links: Document Cited by: Appendix A, §1, §2. Chan et al. (2022) T. C. Y. Chan, M. Eberg, K. Forster, C. Holloway, L. Ieraci, Y. Shalaby, and N. Yousefi An inverse optimization approach to measuring clinical pathway concordance. Management Science 68 (3), p. 1882–1903. External Links: Document Cited by: §1. Chan and Kaw (2020) T. C. Y. Chan and N. Kaw Inverse optimization for the recovery of constraint parameters. European Journal of Operational Research 282 (2), p. 415–427. External Links: ISSN 0377-2217, Document Cited by: Appendix A. Chan et al. (2019) T. C. Y. Chan, T. Lee, and D. Terekhov Inverse optimization: closed-form solutions, geometry, and goodness of fit. Management Science 65 (3), p. 1115–1135. External Links: Document Cited by: Appendix A, §1, §1, §1, §2. Chan and Lee (2018) T. C. Y. Chan and T. Lee Trade-off preservation in inverse multi-objective convex optimization. European Journal of Operational Research 270 (1), p. 25–39. External Links: Document Cited by: Appendix A. Chan et al. (2025) T. C. Y. Chan, R. Mahmood, and I. Y. Zhu Inverse optimization: theory and applications. Operations Research 73 (2), p. 1046–1074. External Links: Document Cited by: Appendix A, §1, §1. De Loera et al. (2012) J. A. De Loera, R. Hemmecke, and M. Köppe Algebraic and geometric ideas in the theory of discrete optimization. MOS-SIAM Series on Optimization, Vol. 14, SIAM. External Links: Document Cited by: Definition F.19. de Oliveira (2019) W. de Oliveira Proximal bundle methods for nonsmooth DC programming. Journal of Global Optimization 75, p. 523–563. External Links: Document Cited by: §E.4, §E.5, Appendix E. Fatemi et al. (2026) P. Fatemi, H. Maskan, S. Sra, and P. Mohajerin Esfahani Tight generalization bounds for noiseless inverse optimization. Note: Available at arXiv:2605.08866v1 Cited by: Appendix A, §G.3, §G.3, §G.3, §G.3, Proposition G.7, Proposition G.8, Remark G.9, §2, §6. Fey and Lenssen (2019) M. Fey and J. E. Lenssen Fast graph representation learning with PyTorch Geometric. In ICLR Workshop on Representation Learning on Graphs and Manifolds, Cited by: §H.2. Gaudioso et al. (2023) M. Gaudioso, S. Taheri, A. M. Bagirov, and N. Karmitsa Bundle enrichment method for nonsmooth difference of convex programming problems. Algorithms 16 (8), p. 394. External Links: Document Cited by: §E.5. Ghobadi and Mahmoudzadeh (2021) K. Ghobadi and H. Mahmoudzadeh Inferring linear feasible regions using inverse optimization. European Journal of Operational Research 290 (3), p. 829–843. External Links: ISSN 0377-2217, Document, Link Cited by: Appendix A. Gurobi Optimization, LLC (2026) Gurobi Optimization, LLC Gurobi Optimizer Reference Manual. External Links: Link Cited by: §3. Harper and Konstan (2015) F. M. Harper and J. A. Konstan The MovieLens datasets: history and context. ACM Transactions on Interactive Intelligent Systems 5 (4), p. 1–19. External Links: Document Cited by: §H.1, §H.2, §7. Hazan et al. (2007) E. Hazan, A. Agarwal, and S. Kale Logarithmic regret algorithms for online convex optimization. Machine Learning 69, p. 169–192. External Links: Document Cited by: §H.3, Table 3, §5.1, §7.1. Heuberger (2004) C. Heuberger Inverse combinatorial optimization: a survey on problems, methods, and results. Journal of Combinatorial Optimization 8 (3), p. 329–361. External Links: Document Cited by: §1, §1. Hu et al. (2020) W. Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec Open Graph Benchmark: datasets for machine learning on graphs. In Advances in Neural Information Processing Systems, Vol. 33, p. 22118–22133. Cited by: §H.1, §H.2, §7.2, §7. Joki et al. (2018) K. Joki, A. M. Bagirov, N. Karmitsa, M. M. Mäkelä, and S. Taheri Double bundle method for finding Clarke stationary points in nonsmooth DC programming. SIAM Journal on Optimization 28 (2), p. 1892–1919. External Links: Document Cited by: §E.5. Kelley (1960) J. E. Kelley The cutting-plane method for solving convex programs. Journal of the Society for Industrial and Applied Mathematics 8 (4), p. 703–712. External Links: Document Cited by: §D.1, Appendix D, §E.3, §E.5, Appendix E, §5.2. Keshavarz et al. (2011) A. Keshavarz, Y. Wang, and S. Boyd Imputing a convex objective function. In 2011 IEEE International Symposium on Intelligent Control, p. 613–619. External Links: Document Cited by: Appendix A, §1, §2. Kitaoka (2024) A. Kitaoka Exact solution to data-driven inverse optimization of MILPs in finite time via gradient-based methods. Note: Available at arXiv:2405.14273v8 Cited by: Appendix A, §E.6, Proposition E.8, §F.3, Remark F.8, Appendix F, §1, §2, §5.1, §7.1. Kitaoka (2025) A. Kitaoka Inverse mixed-integer programming: learning constraints then objective functions. Note: Available at arXiv:2510.04455v3 Cited by: Appendix A, Appendix A, §2. Kitaoka (2026) A. Kitaoka Explicit Iteration Complexity of Exact Data-Driven Inverse Optimization for Integer Linear Programs. Note: Available at arXiv:2607.22263v1 Cited by: Appendix A, §F.3, §F.4, Corollary F.10, Remark F.11, Remark F.13, Remark F.14, Appendix F, §1, §2. Kiwiel (1990) K. C. Kiwiel Proximity control in bundle methods for convex nondifferentiable minimization. Mathematical Programming 46, p. 105–122. External Links: Document Cited by: §D.2, §D.2, Appendix D, §E.3, §E.5, 5. Kunzinger (2019) M. Kunzinger Theory of distributions. Note: Lecture notes, Universität Wien Cited by: Proposition C.1, Proposition 5.1. Le Thi and Pham Dinh (2018) H. A. Le Thi and T. Pham Dinh DC programming and DCA: thirty years of developments. Mathematical Programming 169, p. 5–68. External Links: Document Cited by: §E.5. Lemaréchal et al. (1995) C. Lemaréchal, A. Nemirovskii, and Y. Nesterov New variants of bundle methods. Mathematical Programming 69, p. 111–147. External Links: Document Cited by: §D.3, Appendix D, §E.3, §E.5, Remark F.4. Lin et al. (2024) B. Lin, E. Delage, and T. C. Y. Chan Conformal inverse optimization. In Advances in Neural Information Processing Systems, Vol. 37, p. 63534–63564. External Links: Document Cited by: Appendix A, §1, §2. Mohajerin Esfahani et al. (2018) P. Mohajerin Esfahani, S. Shafieezadeh-Abadeh, G. A. Hanasusanto, and D. Kuhn Data-driven inverse optimization with imperfect information. Mathematical Programming 167, p. 191–234. External Links: Document Cited by: Appendix A, §E.1, §E.5, §1, §2, Remark 4.1, §5.2. Murota (2003) K. Murota Discrete convex analysis. Society for Industrial and Applied Mathematics, Philadelphia. External Links: Document Cited by: §F.4, §F.4, Proposition F.21, Proposition F.22. Ng and Russell (2000) A. Y. Ng and S. J. Russell Algorithms for inverse reinforcement learning. In The 17th International Conference on Machine Learning, p. 663–670. Cited by: §1. Onn (2010) S. Onn Nonlinear discrete optimization: an algorithmic theory. Zurich Lectures in Advanced Mathematics, European Mathematical Society. External Links: Document Cited by: §F.4, Proposition F.15, Proposition F.17, Definition F.19. Perron and Furnon (2026) L. Perron and V. Furnon OR-Tools. Note: Version 9.15, Google, https://developers.google.com/optimization/ Cited by: §H.2, §3. Pham Dinh and Le Thi (1997) T. Pham Dinh and H. A. Le Thi Convex analysis approach to d.c. programming: Theory, Algorithm and Applications. Acta Mathematica Vietnamica 22 (1), p. 289–355. Cited by: §E.5, Remark E.6, Appendix E, §5.2. Ren et al. (2025) K. Ren, P. Mohajerin Esfahani, and A. Georghiou Inverse optimization via learning feasible regions. In The 42nd International Conference on Machine Learning, Vol. 267, p. 51471–51488. Cited by: Appendix A. Sakaue et al. (2025a) S. Sakaue, H. Bao, and T. Tsuchiya Revisiting online learning approach to inverse linear optimization: a Fenchel–Young loss perspective and gap-dependent regret analysis. In The 28th International Conference on Artificial Intelligence and Statistics, Vol. 258, p. 46–54. Cited by: Appendix A, §1, §2. Sakaue et al. (2025b) S. Sakaue, T. Tsuchiya, H. Bao, and T. Oki Online inverse linear optimization: efficient logarithmic-regret algorithm, robustness to suboptimality, and lower bound. In Advances in Neural Information Processing Systems, Vol. 38, p. 94876–94904. External Links: Document Cited by: Appendix A, Table 3, Table 3, §1, §2, §5.1, §7.1. Sen et al. (2008) P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Gallagher, and T. Eliassi-Rad Collective classification in network data. AI Magazine 29 (3), p. 93–106. External Links: Document Cited by: §H.1, §7.2, §7. Shi et al. (2023) L. Shi, G. Zhang, H. Zhen, J. Fan, and J. Yan Understanding and generalizing contrastive learning from the inverse optimal transport perspective. In The 40th International Conference on Machine Learning, Vol. 202, p. 31408–31421. Cited by: §1. Suzuki et al. (2019) Y. Suzuki, W. M. Wee, and I. Nishioka TV advertisement scheduling by learning expert intentions. In The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p. 3071–3081. External Links: Document Cited by: §1. Tarantola (2005) A. Tarantola Inverse problem theory and methods for model parameter estimation. edition, Society for Industrial and Applied Mathematics, . External Links: Document, Link, https://epubs.siam.org/doi/pdf/10.1137/1.9780898717921 Cited by: §1. van Erven and Koolen (2016) T. van Erven and W. M. Koolen MetaGrad: multiple learning rates in online learning. In Advances in Neural Information Processing Systems, Vol. 29, p. 3666–3674. Cited by: §H.3, Table 3, §5.1, §7.1. Van Rossum and Drake (2009) G. Van Rossum and F. L. Drake Python 3 reference manual. CreateSpace, Scotts Valley, CA. External Links: ISBN 1441412697 Cited by: §H.2. Villani (2009) C. Villani Optimal transport: old and new. Grundlehren der mathematischen Wissenschaften, Vol. 338, Springer, Berlin. External Links: Document Cited by: Appendix C, footnote 2. Wang et al. (2020) K. Wang, Z. Shen, C. Huang, C. Wu, Y. Dong, and A. Kanakia Microsoft Academic Graph: when experts are not enough. Quantitative Science Studies 1 (1), p. 396–413. External Links: Document Cited by: §H.1, §H.2. Wang (2009) L. Wang Cutting plane algorithms for the inverse mixed integer linear programming problem. Operations Research Letters 37 (2), p. 114–116. External Links: Document Cited by: Appendix A. Yang et al. (2016) Z. Yang, W. W. Cohen, and R. Salakhutdinov Revisiting semi-supervised learning with graph embeddings. In The 33rd International Conference on Machine Learning, Vol. 48, p. 40–48. Cited by: §H.2. Zattoni Scroccaro et al. (2025) P. Zattoni Scroccaro, B. Atasoy, and P. Mohajerin Esfahani Learning in inverse optimization: incenter cost, augmented suboptimality loss, and algorithms. Operations Research 73 (5), p. 2661–2679. External Links: Document Cited by: Appendix A. Appendix A Related work Classical inverse optimization Classical inverse optimization is the problem of finding the objective weight closest to a prior estimate such that a given solution becomes optimal; when the forward problem is an LP, it can be solved via LP duality (Ahuja and Orlin, 2001). As extensions to MILP, Wang (2009) proposed a cutting-plane method that uses the forward problem as an oracle, and Bulut and Ralphs (2021) studied the computational complexity of inverse MILP. All of these assume that the given solution is optimal under some weight. When this assumption fails, classical inverse optimization returns trivial solutions such as the zero vector in the linear case (Chan and Lee, 2018). For an overview of inverse optimization, see the survey by Chan et al. (2025). Details of inverse optimization for noisy data The method that minimizes the distance between the observations and the optimal solutions (the prediction loss) (Aswani et al., 2018) is statistically consistent, but it leads to a bilevel program that is NP-hard even when the forward problem is convex, and it has to rely on enumerating the parameters. The tractable reformulations based on the residual of the KKT conditions (Keshavarz et al., 2011), the violation of variational inequalities (Bertsimas et al., 2015), the relaxation of the strong duality constraint (Chan et al., 2014; Chan et al., 2019), and distributionally robust optimization of the suboptimality loss (Mohajerin Esfahani et al., 2018) all assume convexity (conic representability) of the forward problem. Methods that estimate the constraints of the feasible region (Chan and Kaw, 2020; Ghobadi and Mahmoudzadeh, 2021; Ren et al., 2025) also mainly target LPs. Details of inverse optimization for MILP Among the methods that use only the forward-problem oracle, in the offline setting, Lin et al. (2024) proposed point estimation by a cutting-plane method, and Kitaoka (2024); Kitaoka (2026) obtained exact solutions in finitely many iterations by gradient methods. The augmented suboptimality loss (Zattoni Scroccaro et al., 2025) is evaluated by solving an augmented forward problem with a distance term, and its convex reformulation over mixed-integer sets enumerates the integer points; except for binary decision variables, for which the augmented forward problem reduces to the forward problem with modified weights, it cannot be evaluated with the forward-problem oracle alone. Moreover, like the suboptimality loss, it aims to reproduce the observations as optimal solutions. We therefore do not compare with it in our experiments. As a method that learns the constraints in addition to the objective, Kitaoka (2025) proposed two-stage learning of the constraints and the objective. In the online setting, Bärmann et al. (2017); Bärmann et al. (2018) and Sakaue et al. (2025a); Sakaue et al. (2025b) give regret upper bounds, and also give guarantees in the offline setting via online-to-batch conversion (a conversion that outputs, e.g., the average of the iterates of online learning and derives an upper bound on the offline loss from a regret upper bound). Among these, Sakaue et al. (2025b) give a regret upper bound that adapts to the cumulative suboptimality of the agent’s actions. Moreover, although it does not minimize the suboptimality loss, Besbes et al. (2025) proposed a method that handles offline and online learning in a setting that does not assume the structure of the forward problem. The theoretical guarantees of Kitaoka (2024); Kitaoka (2026); Kitaoka (2025) are limited to the setting where the agent’s actions are optimal. Details of generalization error analysis for inverse optimization Kitaoka (2025) observed that the “distance” between parameters that naturally arises in the generalization error analysis of inverse optimization is in general not a metric but a pseudometric, and extended the Dudley-type bound via covering numbers of sub-Gaussian processes to pseudometric spaces for losses that are Lipschitz with respect to a pseudometric. Applying this to inverse optimization for MILP that learns the objective and the constraints, they bounded the expectation of the suboptimality loss at the learned weight by O(d/N)O( d/N), where d is the dimension of the features. Fatemi et al. (2026) treated noiseless inverse optimization, in which the agent’s action is the unique optimal solution under the true weight, regarded the consistency constraint that each observation imposes on the weight as a constraint of a scenario program, and applied the result of Campi and Garatti (2008). For the estimator that minimizes a convex function under the consistency constraints, they bounded the probability that the true optimal solution is not contained in the set of optimal solutions under the learned weight (the set-level mismatch) by 2(d+log(1/β))/N2(d+ (1/β))/N with probability at least 1−β1-β (Fatemi et al., 2026, Proposition 2.1), and, under an assumption of covariance diversity or for the incenter estimator, bounded the probability that the action selected under the learned weight by a fixed measurable rule that chooses one optimal solution when there are several does not coincide with the true optimal solution (the action-level mismatch) at the same order (Fatemi et al., 2026, Theorems 2.2 and 2.3). Furthermore, for the class of estimators that minimize a linear objective, they constructed an instance showing that these orders are tight (Fatemi et al., 2026, Theorem 3.1). Fatemi et al. (2026, Appendix A.4) also point out that, for an arbitrary consistent estimator, only a bound with a log factor based on the VC dimension is available. Theorem 6.3 of this paper gives a uniform bound of this type for outperformance inverse optimization, in which the agent’s actions may be suboptimal, and Theorem 6.5 is obtained by embedding the instance of Theorem 3.1 of Fatemi et al. (2026) into the setting of this paper. Appendix B An action that cannot be outperformed We give a concrete example in which no weight makes the OS loss 00. A point x′∈(s)x (s) is called efficient if there is no x∈(s)x (s) with x≥x′x≥ x and x≠x′x≠ x . An efficient point is called supported if it is an optimal solution of maxx∈(s)⟨θ,x⟩ _x (s) θ,x for some θ∈ℝ≥0d∖0θ ^d_≥ 0 \0\, and unsupported otherwise. If x′x is efficient, then ≥x′(s)=x′X_≥ x (s)=\x \, so ℓOS(θ,x′,s)=0 ^OS(θ;x ,s)=0 if and only if x′x is an optimal solution under θ. Hence, if x′x is unsupported and 0∉Θ0∉ , we have ℓOS(θ,x′,s)>0 ^OS(θ;x ,s)>0 for every θ∈Θθ∈ . Let d=2d=2, Θ=Δ1 = ^1, (s)=(3,0),(0,3),(1,1)X(s)=\(3,0),(0,3),(1,1)\, and x′=(1,1)x =(1,1). Since (s)X(s) is a union of three bounded polyhedra each consisting of a single point, it satisfies Assumption 4.2(2). Neither (3,0)(3,0) nor (0,3)(0,3) dominates x′x , so x′x is efficient and ≥x′(s)=x′X_≥ x (s)=\x \. For θ=(θ1,θ2)∈Δ1θ=( _1, _2)∈ ^1, since maxθ1,θ2≥1/2 \ _1, _2\≥ 1/2 and θ1+θ2=1 _1+ _2=1, we have ℓOS(θ,x′,s)=max3θ1,3θ2,θ1+θ2−(θ1+θ2)=3maxθ1,θ2−1≥12, ^OS(θ;x ,s)= \3 _1,3 _2, _1+ _2\-( _1+ _2)=3 \ _1, _2\-1≥ 12, with equality if and only if θ=(1/2,1/2)θ=(1/2,1/2). Hence x′x is unsupported, and the OS loss at this state is at least 1/21/2 for every θ∈Θθ∈ . In particular, if (s,x′)(s,x ) is one of N observations, the empirical mean of the OS loss is at least 1/(2N)1/(2N) for every θ∈Θθ∈ , and Equation 3.2 cannot be attained. Thus, even an action that is not dominated by any feasible solution can be impossible to outperform by an optimal solution under a single linear objective. Appendix C Weak derivative of the OS loss Proposition C.1 (Weak derivative in the sense of Schwartz distributions). Let W⊂ℝdW ^d be an open set and let ⊂ℝdZ ^d be nonempty and compact. Define F(θ):=maxx∈⟨θ,x⟩.F(θ):= _x θ,x . Moreover, for each θ∈Wθ∈ W, take x∗(θ)∈argmaxx∈⟨θ,x⟩.x^*(θ)∈ _x θ,x . Then, for each i=1,…,di=1,…,d, in the sense of Schwartz distributions ′(W)D (W) (Kunzinger, 2019, cf.), we have ∂iF=xi∗in ′(W). _iF=x_i^* D (W). That is, for any test function ψ∈Cc∞(W)ψ∈ C_c^∞(W), we have ⟨∂iF,ψ⟩:=−∫WF(θ)∂iψ(θ)dθ=∫Wxi∗(θ)ψ(θ)dθ. _iF,ψ :=- _WF(θ)\, _iψ(θ)\,dθ= _Wx_i^*(θ)\,ψ(θ)\,dθ. Proof. In what follows, let LF:=maxx∈‖x‖L_F:= _x \|x\| (since Z is nonempty and compact, 0≤LF<∞0≤ L_F<∞). Step 0. F is a finite-valued convex function on ℝdR^d with Lipschitz constant LFL_F. Since Z is nonempty and compact and x↦⟨θ,x⟩x θ,x is continuous, the maximum is attained at each θ∈ℝdθ ^d, and F is well defined as a finite-valued function on ℝdR^d. Since F is the supremum of the family of linear functions ⟨⋅,x⟩x∈\ ·,x \_x , it is convex. Moreover, for any θ,θ′∈ℝdθ,θ ^d and x∈x , we have ⟨θ′,x⟩=⟨θ,x⟩+⟨θ′−θ,x⟩≤F(θ)+LF‖θ′−θ‖, θ ,x = θ,x + θ -θ,x ≤ F(θ)+L_F\|θ -θ\|, so taking the maximum over x∈x yields F(θ′)≤F(θ)+LF‖θ′−θ‖F(θ )≤ F(θ)+L_F\|θ -θ\|. Exchanging the roles of θ and θ′θ , (C.1) |F(θ′)−F(θ)|≤LF‖θ′−θ‖(∀θ,θ′∈ℝd)|F(θ )-F(θ)|≤ L_F\|θ -θ\| (∀θ,θ ^d) follows. In particular, F is continuous and F∈Lloc1(W)F∈ L^1_loc(W). Step 1. F is differentiable a.e., and ∇F(θ)=x∗(θ)∇ F(θ)=x^*(θ) at such points. By Equation C.1 and Rademacher’s theorem (cf. Villani, 2009, Theorem 10.8 (i)), F is differentiable in the classical sense almost everywhere on W. Take a point of differentiability θ∈Wθ∈ W. Since x∗(θ)∈argmaxx∈⟨θ,x⟩x^*(θ)∈ _x θ,x , for any θ′∈ℝdθ ^d, F(θ′)≥⟨θ′,x∗(θ)⟩=⟨θ,x∗(θ)⟩+⟨θ′−θ,x∗(θ)⟩=F(θ)+⟨θ′−θ,x∗(θ)⟩,F(θ )≥ θ ,x^*(θ) = θ,x^*(θ) + θ -θ,x^*(θ) =F(θ)+ θ -θ,x^*(θ) , that is, x∗(θ)x^*(θ) is a subgradient of F at θ: (C.2) F(θ′)−F(θ)≥⟨θ′−θ,x∗(θ)⟩(∀θ′∈ℝd).F(θ )-F(θ)≥ θ -θ,x^*(θ) (∀θ ^d). Take any h∈ℝdh ^d. Since W is open, θ+th∈Wθ+th∈ W for sufficiently small t>0t>0; setting θ′=θ+thθ =θ+th in Equation C.2 and dividing by t>0t>0 gives F(θ+th)−F(θ)t≥⟨h,x∗(θ)⟩. F(θ+th)-F(θ)t≥ h,x^*(θ) . By differentiability at θ, letting t↓0t 0 yields ⟨h,∇F(θ)⟩≥⟨h,x∗(θ)⟩. h,∇ F(θ) ≥ h,x^*(θ) . Replacing h with −h-h gives the reverse inequality, so ⟨h,∇F(θ)⟩=⟨h,x∗(θ)⟩(∀h∈ℝd), h,∇ F(θ) = h,x^*(θ) (∀ h ^d), that is, ∇F(θ)=x∗(θ)∇ F(θ)=x^*(θ). Hence we have (C.3) ∇F(θ)=x∗(θ)a.e. θ∈W.∇ F(θ)=x^*(θ) .e. θ∈ W. Note that θ↦∇F(θ)θ ∇ F(θ) is Lebesgue measurable (as the pointwise limit of difference quotients on the set of points of differentiability), and by the completeness of the Lebesgue measure and Equation C.3, x∗x^* is also Lebesgue measurable. Moreover, since ‖x∗(θ)‖≤LF\|x^*(θ)\|≤ L_F, we have x∗∈L∞(W,ℝd)⊂Lloc1(W,ℝd)x^*∈ L^∞(W;R^d)⊂ L^1_loc(W;R^d), and the integral on the right-hand side of the claim converges absolutely. Step 2. Identification of the distributional derivative via convergence of difference quotients. Take any i∈1,…,di∈\1,…,d\ and ψ∈Cc∞(W)ψ∈ C_c^∞(W), and let :=suppψK:=suppψ (a compact subset of W). Since W is open, δ:=dist(,ℝd∖W)>0δ:=dist(K,R^d W)>0 (when W=ℝdW=R^d, read δ:=+∞δ:=+∞). Let δ:=θ∈ℝd|dist(θ,)≤δ/2K_δ:=\θ ^d\,|\,dist(θ,K)≤δ/2\. Then δK_δ is a compact subset of W, and if 0<|t|<δ/20<|t|<δ/2, then +tei⊂δ⊂WK+te_i _δ⊂ W (where eie_i is the iith standard basis vector). First, for 0<|t|<δ/20<|t|<δ/2, since the support of ψ is contained in K, the change of variables θ↦θ−teiθ θ-te_i gives ∫WF(θ+tei)ψ(θ)θ=∫+teiF(θ)ψ(θ−tei)θ=∫WF(θ)ψ(θ−tei)θ, _WF(θ+te_i)\,ψ(θ)\,dθ= _K+te_iF(θ)\,ψ(θ-te_i)\,dθ= _WF(θ)\,ψ(θ-te_i)\,dθ, and therefore we have (C.4) ∫WF(θ+tei)−F(θ)tψ(θ)θ=∫WF(θ)ψ(θ−tei)−ψ(θ)tθ. _W F(θ+te_i)-F(θ)t\,ψ(θ)\,dθ= _WF(θ)\, ψ(θ-te_i)-ψ(θ)t\,dθ. For the right-hand side of Equation C.4, since ψ∈Cc∞(W)ψ∈ C_c^∞(W), the convergence (ψ(⋅−tei)−ψ(⋅))/t→−∂iψ (ψ(·-te_i)-ψ(·) )/t→- _iψ (t→0t→ 0) is uniform, and for 0<|t|<δ/20<|t|<δ/2 the support of the integrand is contained in δK_δ. Since F is continuous (Step 0), it is bounded on the compact set δK_δ, and therefore limt→0∫WF(θ)ψ(θ−tei)−ψ(θ)tdθ=−∫WF(θ)∂iψ(θ)dθ. _t→ 0 _WF(θ)\, ψ(θ-te_i)-ψ(θ)t\,dθ=- _WF(θ)\, _iψ(θ)\,dθ. For the left-hand side of Equation C.4, by Step 1, for almost every θ∈Wθ∈ W, limt→0F(θ+tei)−F(θ)t=∂iF(θ)=xi∗(θ), _t→ 0 F(θ+te_i)-F(θ)t= _iF(θ)=x_i^*(θ), and by Equation C.1, |F(θ+tei)−F(θ)tψ(θ)|≤LF|ψ(θ)|∈L1(W), | F(θ+te_i)-F(θ)t\,ψ(θ) |≤ L_F\,|ψ(θ)|∈ L^1(W), so the integrand is dominated by an integrable function independent of t. Hence, by Lebesgue’s dominated convergence theorem, limt→0∫WF(θ+tei)−F(θ)tψ(θ)θ=∫Wxi∗(θ)ψ(θ)θ. _t→ 0 _W F(θ+te_i)-F(θ)t\,ψ(θ)\,dθ= _Wx_i^*(θ)\,ψ(θ)\,dθ. Combining these by letting t→0t→ 0 in Equation C.4, for any ψ∈Cc∞(W)ψ∈ C_c^∞(W), we have ⟨∂iF,ψ⟩=−∫WF(θ)∂iψ(θ)dθ=∫Wxi∗(θ)ψ(θ)dθ. _iF,ψ =- _WF(θ)\, _iψ(θ)\,dθ= _Wx_i^*(θ)\,ψ(θ)\,dθ. That is, ∂iF=xi∗ _iF=x_i^* in ′(W)D (W). ∎ Proof of Proposition 5.1. Apply Proposition C.1 with W=ℝdW=R^d and =(s)Z=X(s), and with W=ℝdW=R^d and =≥x′(s)Z=X_≥ x (s) (both are nonempty and compact), and take the difference of the two terms in Equation 4.2. ∎ Appendix D Pseudocode of the rules for selecting evaluation points in the inner solver: Step 12 of Algorithm 5 In this section, as concrete forms of Step 12 of Algorithm 5, we give the pseudocode of the cutting-plane method (Kelley, 1960), the proximal bundle method (Kiwiel, 1990), and the level method (Lemaréchal et al., 1995). Furthermore, in Section D.4, we give the pseudocode of the cutting-plane finishing of Theorem F.3, which starts with one of these and switches to the cutting-plane method for finishing. Below, UBUB denotes the minimum of the values of the convex subproblem evaluated so far in Algorithm 5 (an upper bound), LBLB denotes the minimum value of the model of cuts (a lower bound), and ϵin _in denotes the inner tolerance. The cut gjg_j, the cut set G, the solution pool V, the maximum number of iterations J, the point θ θ with the smallest evaluated value, and convergedconverged are defined in Algorithm 5 (Section E.3), and inherited cuts and cut inheritance are defined in Section E.4. The selection step refers to Step 12 of Algorithm 5. Each is written as a procedure that is called at iteration j of Algorithm 5 immediately after the cut gjg_j is added to G and UB,LBUB,LB are updated (after passing the stopping test), and that returns θj+1θ^j+1 (only the proximal bundle method also returns the quantities carried over to the next call). We fix the notation. Let jG_j be the cut set at the selection step of iteration j (including the inherited cuts and g0,…,gjg_0,…,g_j), and define the cutting-plane model (D.1) ℓ^j(θ):=maxg∈j⟨θ,g⟩. _j(θ):= _g _j θ,g . By Proposition E.3, ℓ^j≤ℓ¯sub(⋅,y′) _j≤ ^sub(·\,;y ), and at an evaluated point θiθ^i (i≤ji≤ j), since gi∈jg_i _j, we have ℓ^j(θi)=ℓ¯sub(θi,y′)=⟨θi,gi⟩ _j(θ^i)= ^sub(θ^i;y )= θ^i,g_i . The LBLB of Algorithm 5 equals minθ∈Θℓ^j(θ) _θ∈ _j(θ). Let ∥⋅∥\|·\| denote the Euclidean norm. The subproblems of the three rules differ only in the objective over the common polyhedron (D.2) Pj:=(θ,τ)∈ℝd×ℝ|θ∈Θ,⟨θ,g⟩≤τ(∀g∈j)P_j:= \(θ,τ) ^d×R\ |\ θ∈ ,\ θ,g ≤τ\ (∀ g _j) \ (the epigraph of the model ℓ^j _j over Θ ); since Θ is a polyhedron by Assumption 4.2(1), they are an LP for the cutting-plane method and a convex quadratic program (QP) for the proximal bundle method and the level method. D.1. Cutting-plane method The cutting-plane method (Kelley, 1960) takes a minimizer of the model ℓ^j _j over Θ as the next evaluation point (Algorithm 1). Since this minimization is exactly the LP that Algorithm 5 solves to compute LBLB, no additional solve is needed, and the θ component of the optimal solution of that LP can be reused. When the LP has multiple optimal solutions, any of them may be chosen (the proof of Lemma E.10 uses only optimality). Finite exactness and an upper bound on the number of iterations under this rule are given in Lemma E.10, and its use as the rule after the switch in cutting-plane finishing (Section D.4) is given in Theorem F.3. Algorithm 1 Rule for selecting evaluation points: cutting-plane method 0: Cut set jG_j 1: (θLP,τLP)←(θ^LP,τ^LP)← optimal solution and optimal value of the LP minτ∣(θ,τ)∈Pj \\,τ (θ,τ)∈ P_j\,\ 2: θj+1←θLPθ^j+1←θ^LP 3: return θj+1θ^j+1 D.2. Proximal bundle method The proximal bundle method (Kiwiel, 1990) maintains a stability center θc∈Θθ^c∈ and takes as the candidate point the optimal solution of (D.3) minτ+u2‖θ−θc‖2|(θ,τ)∈Pj, \\,τ+ u2\|θ-θ^c\|^2\ |\ (θ,τ)∈ P_j\, \, which adds the proximal term u2‖θ−θc‖2 u2\|θ-θ^c\|^2 (u>0u>0 is the proximal weight) to the model (Algorithm 2). Since the value ℓ¯sub(θj+1,y′) ^sub(θ^j+1;y ) of the candidate point θj+1θ^j+1 is evaluated at the next iteration of Algorithm 5, the test of whether to update the center (the descent test) is performed at the beginning of the selection step of the next iteration. The predicted decrease ν:=fc−ℓ^j(θj+1)ν:=f^c- _j(θ^j+1) (where fc:=ℓ¯sub(θc,y′)f^c:= ^sub(θ^c;y ) is the value at the center) is carried over to the next call together with the candidate point; there, if the actual decrease fc−ℓ¯sub(θj+1,y′)f^c- ^sub(θ^j+1;y ) is at least ρνρ\,ν (ρ∈(0,1)ρ∈(0,1) is the descent parameter), the center is moved to the candidate point (serious step). Otherwise, the center is kept and only the cut of the candidate point is added (null step). Since (θc,τ)=(θc,fc)∈Pj(θ^c,τ)=(θ^c,f^c)∈ P_j is a feasible solution of Equation D.3 with objective value fcf^c, we have ℓ^j(θj+1)≤fc _j(θ^j+1)≤ f^c, that is, always ν≥0ν≥ 0. We make two remarks specific to this framework. First, the θ θ of Algorithm 5 (the point with the smallest evaluated value) and the center θcθ^c differ in general (a null-step point may update UBUB). Always UB≤fcUB≤ f^c. Second, the stopping test specific to the proximal bundle method (ν≤ϵν≤ε, where ϵ>0ε>0 is a tolerance) is replaced by the stopping test UB−LB≤ϵinUB-LB≤ _in of Algorithm 5. Indeed, when the selection step is reached, UB−LB>ϵin≥0UB-LB> _in≥ 0, and in particular LB<UB≤fc=ℓ^j(θc)LB<UB≤ f^c= _j(θ^c), so θcθ^c is not a minimizer of the model ℓ^j _j over Θ . Since θcθ^c being a minimizer of the model with the proximal term Equation D.3 is equivalent to its being a minimizer of ℓ^j _j (because the gradient of the proximal term vanishes at θcθ^c), we have θj+1≠θcθ^j+1≠θ^c, and since the optimal value of Equation D.3 is at most fcf^c, ℓ^j(θj+1)<fc _j(θ^j+1)<f^c, that is, ν>0ν>0 holds. Conversely, in the situation where ν=0ν=0 (θcθ^c is a minimizer of the model), LB=fc≥UBLB=f^c , so Algorithm 5 has already stopped at the stopping test of the same iteration. Moreover, since cuts are never discarded owing to cut inheritance, the bundle compression (aggregate subgradient) of Kiwiel (1990) is not used. The proximal weight u is updated according to the proximity control rule of Kiwiel (1990) (u is not decreased at null steps, and the lower bound umin>0u_ >0 is maintained; keeping u constant is also allowed). Algorithm 2 Rule for selecting evaluation points: proximal bundle method 0: Cut set jG_j, current evaluation point θjθ^j and its value ℓ¯sub(θj,y′)=⟨θj,gj⟩ ^sub(θ^j;y )= θ^j,g_j , descent parameter ρ∈(0,1)ρ∈(0,1), initial proximal weight u0u_0 and its lower bound uminu_ (u0≥umin>0u_0≥ u_ >0) 0: Quantities carried over from the call at iteration j−1j-1 (undefined for j=0j=0): stability center θc∈Θθ^c∈ , its value fc=ℓ¯sub(θc,y′)f^c= ^sub(θ^c;y ), proximal weight u≥uminu≥ u_ , predicted decrease ν=fc−ℓ^j−1(θj)>0ν=f^c- _j-1(θ^j)>0 1: if j=0j=0 then 2: θc←θ0θ^c←θ^0, fc←⟨θ0,g0⟩f^c← θ^0,g_0 , u←u0u← u_0 (initialization of the center) 3: else if ⟨θj,gj⟩≤fc−ρν θ^j,g_j ≤ f^c-ρ\,ν then 4: θc←θjθ^c←θ^j, fc←⟨θj,gj⟩f^c← θ^j,g_j (serious step: update of the center) 5: u←u← update by the proximity control rule (Kiwiel, 1990) 6: else 7: u←u← update by the proximity control rule (without decreasing u) (null step: the center is kept) 8: end if 9: (θj+1,τj+1)←(θ^j+1, _j+1)← optimal solution of the QP Equation D.3 (minimization of the model with the proximal term; τj+1=ℓ^j(θj+1) _j+1= _j(θ^j+1)) 10: ν←fc−τj+1ν← f^c- _j+1 (predicted decrease; ν>0ν>0) 11: return θj+1θ^j+1, and θc,fc,u,νθ^c,f^c,u,ν carried over to the call at iteration j+1j+1 D.3. Level method The level method (Lemaréchal et al., 1995) sets a level between the lower bound LBLB and the upper bound UBUB, (D.4) levj:=LB+λ(UB−LB)(λ∈(0,1) is the level parameter),lev_j:=LB+λ\,(UB-LB) (λ∈(0,1) is the level parameter), and takes the projection of the current evaluation point θjθ^j onto the level set θ∈Θ∣ℓ^j(θ)≤levj\θ∈ _j(θ) _j\ of the model as the next evaluation point (Algorithm 3). The level set is nonempty because the optimal solution θLPθ^LP of the LP attaining LBLB satisfies ℓ^j(θLP)=LB≤levj _j(θ^LP)=LB _j, and the projection is the unique optimal solution of (D.5) min12∥θ−θj∥2|θ∈Θ,⟨θ,g⟩≤levj(∀g∈j) \\, 12\|θ-θ^j\|^2\ |\ θ∈ ,\ θ,g _j\ (∀ g _j)\, \ (the feasible region is the slice of PjP_j at τ=levjτ=lev_j). Since UB,LBUB,LB are the quantities maintained by Algorithm 5, this rule carries no quantities over across calls. The number of iterations required to reach a certified gap UB−LB≤ϵUB-LB≤ε (where ϵ>0ε>0 is a target accuracy) is discussed in Remark F.4 (see also the same place for a remark on using inherited cuts). Algorithm 3 Rule for selecting evaluation points: level method 0: Cut set jG_j, current evaluation point θjθ^j, upper bound UBUB, lower bound LBLB, level parameter λ∈(0,1)λ∈(0,1) 1: levj←LB+λ(UB−LB)lev_j +λ\,(UB-LB) (setting the level; Equation D.4) 2: θj+1←θ^j+1← optimal solution of the QP Equation D.5 (projection of θjθ^j onto the level set θ∈Θ∣ℓ^j(θ)≤levj\θ∈ _j(θ) _j\; nonempty) 3: return θj+1θ^j+1 D.4. Cutting-plane finishing The cutting-plane finishing of Theorem F.3 can also be written as a selection rule of Step 12 (Algorithm 4). Under the guarantee of Theorem F.3, Algorithm 5 is run with tolerance 00 and J=∞J=∞, and the switching threshold ϵsw>0 _sw>0 is held by the rule (since ϵsw _sw is a different quantity from the tolerance ϵin _in of Algorithm 5, we use a separate symbol). The rule distinguishes cases by comparing UB−LBUB-LB with ϵsw _sw: if UB−LB>ϵswUB-LB> _sw, it applies a selection rule R (e.g., Algorithm 1, 2, 3); otherwise, it applies the cutting-plane method (Algorithm 1). We call the iterations at which the former condition holds Phase 1, and those at which the latter holds Phase 2 (they correspond to the two phases of Theorem F.3). This case distinction gives a two-phase procedure because UB−LBUB-LB is nonincreasing in the iteration j. Indeed, by Steps 5–7 of Algorithm 5, UBUB is nonincreasing as the minimum of the evaluated values, and since the cut set G only grows in Step 4, LB=minθ∈Θℓ^j(θ)LB= _θ∈ _j(θ) in Step 8 is nondecreasing. Therefore, once UB−LB≤ϵswUB-LB≤ _sw holds, it continues to hold thereafter, and the switch of phases occurs only once and never goes back. The guarantee is as stated in Theorem F.3. If we take ϵsw<δ(y′) _sw<δ(y ) (δ(y′)δ(y ) is the submodel gap, Equation F.3), then at the time of the switch the lower bound is determined exactly as LB=msub(y′)LB=m_sub(y ) (msub(y′)m_sub(y ) is the minimum value of the convex subproblem of the branch y′y , Equation E.4) (Lemma F.2), and since the cuts only increase thereafter, this value is maintained. In Phase 2, the evaluation point is a minimizer of the model, so UB=LB=msub(y′)UB=LB=m_sub(y ) is reached in finitely many iterations, the stopping test with tolerance 00 is satisfied, and an exact minimizer is returned. Note that since the cut set G and the solution pool V are carried over unchanged within Algorithm 5 across the phases, the switch requires no additional calls to the forward-problem oracle. When the proximal bundle method is used as the rule R, the quantities carried over by Algorithm 2 may be discarded at the switch. Algorithm 4 Rule for selecting evaluation points: cutting-plane finishing (Theorem F.3) 0: Cut set jG_j, current evaluation point θjθ^j, upper bound UBUB, lower bound LBLB, switching threshold ϵsw>0 _sw>0, termination tolerance ϵterm≥0 _term≥ 0, selection rule R (e.g., Algorithm 1, 2, 3) 1: if UB−LB>ϵswUB-LB> _sw then 2: θj+1←θ^j+1← the point obtained by applying the rule R 3: else 4: (finishing by the cutting-plane method) θj+1←θ^j+1← the point obtained by applying Algorithm 1 5: if ‖θj+1−θi‖2≤ϵterm\|θ^j+1-θ^i\|_2≤ _term for some evaluated point θiθ^i (i≤ji≤ j) then 6: stop Algorithm 5 with converged=TRUEconverged=TRUE without evaluating θj+1θ^j+1 7: end if 8: end if 9: return θj+1θ^j+1 In Phase 2, if the minimizer of the LP coincides with an evaluated point within ϵterm _term, the inner loop is stopped with converged=TRUEconverged=TRUE without evaluating that point (in this case UB−LB=0UB-LB=0 does not necessarily hold). This is because re-evaluating the same point yields no new cut, and what remains in UB−LBUB-LB is a difference of the order of the rounding error of the loss values, so we avoid continuing the iterations due to rounding errors. When ϵterm=0 _term=0 and the LPs are solved exactly, this test is never triggered: if θj+1=θiθ^j+1=θ^i for an evaluated point θiθ^i (i≤ji≤ j), then the cut gig_i obtained at θiθ^i belongs to G, so LB=maxg∈⟨θj+1,g⟩≥⟨θi,gi⟩=ℓ¯sub(θi,y′)≥UBLB= _g θ^j+1,g ≥ θ^i,g_i = ^sub(θ^i;y ) , and Algorithm 5 has already stopped by UB−LB≤ϵinUB-LB≤ _in. Theorem F.3 guarantees an exact minimizer under ϵterm=0 _term=0 (the positive ϵterm _term used in the experiments is a safeguard against rounding errors; Section H.3). When Algorithm 6 is run with Algorithm 4 as the rule of Step 12, as long as we take ϵsw<δin:=miny′∈≥allδ(y′) _sw< _in:= _y ^all_≥δ(y ) and ϵterm=0 _term=0, and as long as Phase 1 ends after finitely many iterations under the rule R (which always holds if R is the cutting-plane method), the inner loop is solved exactly at every outer iteration, so the assumption of Proposition E.4 is satisfied (Remark F.8). Appendix E DC optimization This section describes the DC optimization method that solves Equation 4.4 on the basis of the decomposition into branches in Proposition 5.2 and the DC structure in Remark E.1. Its skeleton is a double loop: the outer loop is simplified DCA (Pham Dinh and Le Thi, 1997), which linearizes the concave part and reduces the problem to convex subproblems, and the inner loop solves the convex subproblems by a cutting-plane-type solver, typified by the cutting-plane method (Kelley, 1960). In addition, the method has a mechanism that accumulates the feasible solutions returned by the forward-problem oracle as global lower-bounding cuts and reuses them across iterations and runs (cut inheritance). Cut inheritance instantiates the idea of the proximal bundle method for DC programming of de Oliveira (2019) in accordance with the structure of our problem. Below, Section E.1 gives the notation and assumptions, Section E.2 the relation between branches and DCA, Section E.3 the pseudocode, Section E.4 cut inheritance, Section E.5 the underlying literature, Section E.6 finite convergence and local optimality, and Section E.7 the theoretical analysis. E.1. Notation and assumptions Throughout this section, we assume Assumption 4.2. In the appendices below, a point u of a convex set C is an extreme point of C if u=λv+(1−λ)wu=λ v+(1-λ)w with v,w∈Cv,w∈ C and λ∈(0,1)λ∈(0,1) implies v=w=uv=w=u. For a state s∈s and x′∈(s)x (s), we set (E.1) F1(θ,s):=maxx∈(s)⟨θ,x⟩,F2(θ,x′,s):=maxx∈≥x′(s)⟨θ,x⟩.F_1(θ;s):= _x (s) θ,x , F_2(θ;x ,s):= _x _≥ x (s) θ,x . Since x′∈≥x′(s)x _≥ x (s), the set ≥x′(s)X_≥ x (s) is nonempty, and it is compact as the intersection of the compact set (s)X(s) and the closed set x∈ℝd∣x≥x′\x ^d x≥ x \; hence both maxima in Equation E.1 are attained. By definition (Equation 4.2), ℓOS(θ,x′,s)=F1(θ,s)−F2(θ,x′,s). ^OS(θ;x ,s)=F_1(θ;s)-F_2(θ;x ,s). By the extreme-point oracle assumption (Assumption 4.2(3)), (sn)⊆(sn)Y(s^n) (s^n) and ≥xn(sn)⊆≥xn(sn)Y_≥ x^n(s^n) _≥ x^n(s^n) are finite sets. Furthermore, since the value returned by the oracle attains the maximum for every θ, we have (E.2) F1(θ,sn)=maxy∈(sn)⟨θ,y⟩,F2(θ,xn,sn)=maxy∈≥xn(sn)⟨θ,y⟩.F_1(θ;s^n)= _y (s^n) θ,y , F_2(θ;x^n,s^n)= _y _≥ x^n(s^n) θ,y . That is, F1F_1 and F2F_2 are maxima of finitely many linear functions (convex, piecewise linear, and positively homogeneous of degree 11). Among the branches introduced in the main text (Section 5.2), we deal with the tuples y′=(y′n)n=1N∈≥all=∏n=1N≥xn(sn)y =(y n)_n=1^N ^all_≥= _n=1^NY_≥ x^n(s^n) consisting of values that the oracle can return. For y∈ℝdy ^d and a branch y′∈≥ally ^all_≥, we set (E.3) ℓsub(θ,y,s) ^sub(θ;y,s) :=F1(θ,s)−⟨θ,y⟩, :=F_1(θ;s)- θ,y , (E.4) ℓ¯sub(θ,y′) ^sub(θ;y ) :=1N∑n=1Nℓsub(θ,y′n,sn),msub(y′):=minθ∈Θℓ¯sub(θ,y′). := 1N _n=1^N ^sub(θ;y n,s^n), m_sub(y ):= _θ∈ ^sub(θ;y ). We call msub(y′)m_sub(y ) the branch minimum of the branch y′y . The function ℓsub ^sub has the same form as the suboptimality loss (Mohajerin Esfahani et al., 2018). By Equation E.2, each ℓ¯sub(⋅,y′) ^sub(·\,;y ) is convex and piecewise linear, and by the compactness of Θ , the minimum in Equation E.4 is attained. E.2. Relation between branches and DCA Remark E.1 (Relation to the DC structure). The empirical mean of the OS loss ℓ¯OS ^OS is a DC function of θ∈Θθ∈ , that is, a function that can be written in the form convex function −- convex function. Indeed, by definition, ℓ¯OS(θ)=1N∑n=1Nmaxx∈(sn)⟨θ,x⟩−1N∑n=1Nmaxx∈≥xn(sn)⟨θ,x⟩, ^OS(θ)= 1N _n=1^N _x (s^n) θ,x - 1N _n=1^N _x _≥ x^n(s^n) θ,x , and both terms on the right-hand side are convex functions of θ. By this observation, Equation 4.4 can be regarded as a DC optimization problem and connected to the literature on DC programming. The operation of linearizing the concave part −1N∑n=1Nmaxx∈≥xn(sn)⟨θ,x⟩- 1N _n=1^N _x _≥ x^n(s^n) θ,x at the dominating vertex (the solution returned by the dominance-constrained forward-problem oracle) x∗(θ,xn,sn)x^*(θ;x^n,s^n) is nothing but the minimization with respect to y′y in Proposition 5.2. Proof of Proposition 5.2. By Assumption 4.2(2), ≥xn(sn)X_≥ x^n(s^n) is nonempty and compact for each n, so the maximum in the second term of Equation 4.2 is attained, and −maxx∈≥xn(sn)⟨θ,x⟩=miny∈≥xn(sn)(−⟨θ,y⟩)- _x _≥ x^n(s^n) θ,x = _y _≥ x^n(s^n) (- θ,y ). Since the minimizations for different n can be carried out independently, ℓ¯OS(θ)=1N∑n=1N[maxx∈(sn)⟨θ,x⟩+miny∈≥xn(sn)(−⟨θ,y⟩)]=miny′∈≥allℓ¯sub(θ,y′), ^OS(θ)= 1N _n=1^N [ _x (s^n) θ,x + _y _≥ x^n(s^n) (- θ,y ) ]= _y ^all_≥ ^sub(θ;y ), which gives the first equality in Equation 5.2. By Assumption 4.2(3), for every θ the value returned by the oracle x∗(θ,xn,sn)∈≥xn(sn)⊆≥xn(sn)x^*(θ;x^n,s^n) _≥ x^n(s^n) _≥ x^n(s^n) attains the maximum, so ≥xn(sn)X_≥ x^n(s^n) may be replaced by ≥xn(sn)Y_≥ x^n(s^n) in the argument above, which gives the second equality. Since ≥allY^all_≥ is a finite set (Remark 4.3) and each msub(y′)m_sub(y ) is attained, Equation 5.3 follows by exchanging the min over θ and over y′y . ∎ The decomposition into branches was given in Proposition 5.2 of the main text. Here we supplement the relation between DCA and branches. DCA can be regarded as an iterative method that selects branches adaptively without enumerating them explicitly. If the concave part −1N∑n=1NF2(⋅;xn,sn)- 1N _n=1^NF_2(\,·\,;x^n,s^n) is linearized at the current point θkθ^k by a subgradient, namely the dominating vertices y(k),n:=x∗(θk,xn,sn)y^(k),n:=x^*(θ^k;x^n,s^n), then the resulting convex subproblem is exactly minθ∈Θℓ¯sub(θ,y(k)) _θ∈ ^sub(θ;y^(k)), that is, the minimization of the empirical mean of the suboptimality loss with the branch y(k)=(y(k),n)ny^(k)=(y^(k),n)_n fixed. A subgradient oracle for the convex subproblem is obtained immediately from the forward-problem oracle as ∂θℓ¯sub(θ,y(k))∋1N∑n=1N[x∗(θ,sn)−y(k),n] _θ ^sub(θ;y^(k)) 1N _n=1^N [x^*(θ,s^n)-y^(k),n ] (as in Bärmann et al., 2017). Evaluated at θ=θkθ=θ^k, it coincides with the gradient in the sense of weak derivatives 1N∑n=1Ng(θk,xn,sn) 1N _n=1^Ng(θ^k;x^n,s^n) of Proposition 5.1. That is, whereas the gradient-based optimization methods of Section 5.1 update the branch at every step, DCA fixes the branch, solves the convex subproblem to completion, and then updates the branch. E.3. Algorithm The pseudocode is given in Algorithms 5 and 6. Algorithm 5 Inner solver based on a cutting-plane model for the convex subproblem (with cut inheritance) 0: Branch y′=(y′n)n=1Ny =(y n)_n=1^N, initial point θ0∈Θθ^0∈ , solution pool 0V_0, tolerance ϵin≥0 _in≥ 0, maximum number of iterations J 1: ←1N∑n=1N(vn−y′n)∣v∈0G←\\, 1N _n=1^N(v^n-y n) v _0\,\, UB←+∞UB←+∞ (initialization with inherited cuts) 2: for j=0,1,…,J−1j=0,1,…,J-1 do 3: Compute vjn←x∗(θj,sn)v_j^n← x^*(θ^j,s^n) for each n, and set j+1←j∪(vjn)nV_j+1 _j∪\(v_j^n)_n\ (update of the pool) 4: gj←1N∑n=1N(vjn−y′n)g_j← 1N _n=1^N(v_j^n-y n), ←∪gjG ∪\g_j\ (ℓ¯sub(θj,y′)=⟨θj,gj⟩ ^sub(θ^j;y )= θ^j,g_j ) 5: if ⟨θj,gj⟩<UB θ^j,g_j <UB then 6: UB←⟨θj,gj⟩UB← θ^j,g_j , θ^←θj θ←θ^j 7: end if 8: LB←minθ∈Θmaxg∈⟨θ,g⟩LB← _θ∈ _g θ,g 9: if UB−LB≤ϵinUB-LB≤ _in then 10: return (θ^,converged=TRUE,j+1)( θ,\ converged=TRUE,\ V_j+1) 11: end if 12: Choose the next evaluation point θj+1∈Θθ^j+1∈ according to the selection rule of the inner solver (see Appendix D for details). 13: end for 14: return (θ^,converged=FALSE,J)( θ,\ converged=FALSE,\ V_J) Algorithm 6 DC optimization of the empirical mean of the OS loss (simplified DCA ++ cut inheritance) 0: Data (sn,xn)n=1N\(s^n,x^n)\_n=1^N, oracles x∗(⋅,sn)x^*(·,s^n) and x∗(⋅,xn,sn)x^*(·\,;x^n,s^n), initial point θ0∈Θθ^0∈ , thresholds ϵgap,ϵtie≥0 _gap, _tie≥ 0, maximum number of iterations K 1: 0←∅V^0← , and y′n←x∗(θ0,xn,sn)y n← x^*(θ^0;x^n,s^n) for each n (initialization of the branch) 2: for k=0,1,…,K−1k=0,1,…,K-1 do 3: Apply Algorithm 5 to (y′,θk,k)(y ,θ^k,V^k) and obtain (θk+1,converged,~k)(θ^k+1,converged, V^k) 4: For each n, vn←x∗(θk+1,sn)v^n← x^*(θ^k+1,s^n) and y′n←x∗(θk+1;xn,sn)y n← x^*(θ^k+1;x^n,s^n); k+1←~k∪(vn)nV^k+1← V^k∪\(v^n)_n\ (update of the pool) 5: ℓk←1N∑n=1N⟨θk+1,vn−y′n⟩ _k← 1N _n=1^N θ^k+1,v^n-y n (=ℓ¯OS(θk+1)= ^OS(θ^k+1)) 6: Δk←1N∑n=1N⟨θk+1,y′n−y′n⟩ _k← 1N _n=1^N θ^k+1,y n-y n 7: if ℓk≤ϵgap _k≤ _gap then 8: break (certificate stopping; certificate of global optimality; Proposition E.9) 9: end if 10: if Δk≤ϵtie _k≤ _tie and converged=TRUEconverged=TRUE then 11: break (tie stopping; fixed point of DCA) 12: end if 13: y′n←y′ny n← y n (∀n∀ n) (update of the branch) 14: end for 15: return In the case of target-attainment stopping (see Equation E.6), that point; otherwise, among the points θk+1θ^k+1 of the outer iterations so far, the one with the smallest ℓk _k (if several points attain the minimum, the one from the earliest iteration) Remark E.2 (Reasons for adopting the inner solver and cut inheritance). There are two reasons for using the cutting-plane method in the inner loop. First, the unit of computational cost in our problem is a call to the forward-problem oracle (solving a MILP for each state), and the value and a subgradient of the objective function of the convex subproblem are obtained simultaneously from a single oracle evaluation (without additional solves). The cutting-plane method keeps all the information obtained in this way as the cut set G, and thus makes full use of the information per oracle evaluation. Moreover, since the convex subproblem is convex and piecewise linear (Equation 5.1), minimizing the model requires only an LP, and an exact minimizer is reached in finitely many iterations (Lemma E.10). Furthermore, with cutting-plane finishing (Section D.4), which switches the selection rule for evaluation points only once during the run, the number of iterations until reaching it can be bounded in terms of the number T1T_1 of iterations of Phase 1 per call (Proposition F.6), M (M:=∏n=1N|(sn)|M:= _n=1^N|Y(s^n)|; Lemma E.10), and |≥all||Y^all_≥| (Theorem F.3, Proposition F.6, and Remark F.8). Moreover, in the integer case with the level method as the rule R for selecting evaluation points before the switch in cutting-plane finishing (Algorithm 4), this number is given as an explicit function of the problem size, M, and |≥all||Y^all_≥| whenever the iteration bound of Remark F.4 holds (Corollary F.12). Second, the cutting-plane method returns the gap between the upper and lower bounds, UB−LBUB-LB, as a certificate of convergence. Tie stopping in Algorithm 6 may stop erroneously at an accidental tie (a spurious fixed point) when the inner loop returns an unconverged approximate solution, and it is therefore adopted only when the convergence of the inner loop has been confirmed. This decision requires a certificate. When a gradient-based method without a lower-bounding model is used as the inner solver, convergence is decided from the value of the convex subproblem and the subgradient at the weight (e.g., for ONS and MetaGrad, when the value is at most ϵin _in or the subgradient is 00; Algorithm 11). The reason for accumulating and reusing cuts (cut inheritance) is as follows. Cuts are obtained for free as by-products of oracle evaluations, and their validity depends only on the feasibility of the solutions, not on the choice of the branch (Proposition E.3). Hence they can be safely reused across branch updates and outer iterations. The effect can be quantified: the total number of inner iterations executed in K outer iterations decreases from K(M+1)K(M+1) without sharing the pool to M+2K−1M+2K-1 (Lemma E.10; M is an upper bound on the total number of tuples of solutions that the oracle can return). Note that Algorithm 5 does not fix the selection rule for the next evaluation point θj+1θ^j+1. Since UBUB is the minimum of the values actually evaluated and LBLB is a lower bound obtained from the model of cuts, the validity of the stopping criterion UB−LB≤ϵinUB-LB≤ _in (the ϵin _in-optimality of θ θ) is preserved however θj+1θ^j+1 is chosen. The pseudocode of each rule is given in Appendix D. The outer loop (Algorithm 6) is simplified DCA, which repeats fixing the branch by the dominating vertices (linearization), solving the convex subproblem in the inner loop, and updating the branch. The inner loop (Algorithm 5) is a method based on a cutting-plane model for the convex subproblem minθ∈Θℓ¯sub(θ,y′) _θ∈ ^sub(θ;y ) (if the selection rule for evaluation points is taken to be the minimizer of the model, it becomes the cutting-plane method (Kelley, 1960)). The linearization of ℓ¯sub(⋅,y′) ^sub(·\,;y ) at a point θjθ^j is, with vjn:=x∗(θj,sn)v_j^n:=x^*(θ^j,s^n) and gj:=1N∑n=1N(vjn−y′n)g_j:= 1N _n=1^N(v_j^n-y n), (E.5) ℓ¯sub(θ,y′)≥ℓ¯sub(θj,y′)+⟨gj,θ−θj⟩=⟨θ,gj⟩, ^sub(θ;y )\ ≥\ ^sub(θ^j;y )+ g_j,θ-θ^j \ =\ θ,g_j , which is a homogeneous linear lower bound (cut) with intercept 00 (by ℓ¯sub(θj,y′)=⟨θj,gj⟩ ^sub(θ^j;y )= θ^j,g_j ). The problem of minimizing the model of cuts (the master problem) is the minimization over Θ of the maximum of the cuts, that is, a linear program (LP). There are three stopping criteria. First, certificate stopping (stopping by ℓk≤ϵgap _k≤ _gap in Algorithm 6): since the OS loss is nonnegative (Section 4), its value itself is an upper bound on the gap to the global optimal value, and if ℓk≤ϵgap _k≤ _gap, then θk+1θ^k+1 is ϵgap _gap-globally optimal (Proposition E.9). This stopping does not depend on the convergence of the inner solver and is always valid. Second, tie stopping (stopping by Δk≤ϵtie _k≤ _tie): Δk=0 _k=0 means that the branch y′y used for the linearization is also optimal as a linearization of the concave part at θk+1θ^k+1 (a fixed point of DCA has been reached). However, a θk+1θ^k+1 returned while the inner loop has not converged is an approximate solution of the convex subproblem, and accidental ties (spurious fixed points) can occur for combinatorial forward problems; hence, in the implementation, tie stopping is adopted only when the inner solver reports convergence. Third, target-attainment stopping: the outperformance prediction loss (E.6) ℓOP(θ):=1N∑n=1N∑i=1dmaxxin−xi∗(θ,sn), 0 ^OP(θ):= 1N _n=1^N _i=1^d \x_i^n-x_i^*(θ,s^n),\,0\ is nonnegative, and ℓOP(θ)=0 ^OP(θ)=0 is equivalent to x∗(θ,sn)≥xnx^*(θ,s^n)≥ x^n (∀n∀ n), that is, to the attainment of Equation 3.2. The implementation computes ℓOP ^OP every time it evaluates x∗(⋅,sn)x^*(·,s^n), including at the inner evaluation points, and as soon as it falls below a threshold (default 00), it outputs that θ and immediately stops the whole procedure. We describe the main points of the implementation. Besides the cutting-plane method (the default), the inner solver can be replaced by the projected subgradient method, the proximal bundle method (Kiwiel, 1990), the level method (Lemaréchal et al., 1995), the ellipsoid method, the analytic center cutting-plane method, the center of gravity method, the randomized center of gravity method, ONS, or MetaGrad (Algorithm 11), and every solver that has a lower-bounding model (all except the projected subgradient method, ONS, and MetaGrad) can use inherited cuts (corresponding to the initialization of G in Algorithm 5). The initial point of the inner loop is θ0=θkθ^0=θ^k (warm start) (when ONS or MetaGrad is used as the inner solver, it resumes from the state of the learner at the end of the previous outer iteration), and the return value is projected onto the probability simplex to clean up numerical errors. The default values of the implementation are J=20J=20 (the inner loop is an inexact solve that allows truncation), K=100K=100, ϵtie=10−8 _tie=10^-8, and ϵgap=10−4 _gap=10^-4 (the values used in the experiments are given in Section H.3). Computational cost is accounted for by the number of calls to the forward-problem oracle rather than by the number of iterations (calls to the forward-problem oracle occur only in computing vjv_j in Algorithm 5 and v,y′v,y and the initialization of the branch in Algorithm 6; the additional calls when ONS or MetaGrad is used as the inner solver are shown in Algorithm 11). E.4. Cut inheritance In DC programming, the idea of keeping the cuts (cutting-plane model) of the convex component across iterations was proposed by de Oliveira (2019) as a proximal bundle method for DC programming. That paper takes as its issue that DCA and the proximal linearized method (PLM) must solve a nonsmooth convex subproblem exactly at each iteration, and replaces the convex component f1f_1 in the subproblem by a cutting-plane model f˘1k f_1^k constructed from an information bundle ℬ1kB_1^k that accumulates the linearizations obtained in past iterations. Whereas the linearization of the concave component −f2-f_2 is updated only when the stability center is updated, the model of f1f_1 is kept across iterations, so the convex subproblem need not be solved exactly. Cut inheritance, described in this subsection, instantiates this idea in accordance with the structure of our problem that the convex component F1F_1 is a support function. In the initialization of G in Algorithm 5, tuples of solutions of the forward-problem oracle obtained in the past, v=(vn)nv=(v^n)_n (the pool V), are converted into cuts for the current branch y′y and used as initial cuts. We call the cuts made in this way from the tuples in the pool V inherited cuts, and this mechanism cut inheritance. Their validity follows from the next proposition. Proposition E.3 (Validity of cuts). Let v=(vn)n=1Nv=(v^n)_n=1^N be any tuple satisfying vn∈(sn)v^n (s^n) (n=1,…,Nn=1,…,N). Then, for any branch y′∈≥ally ^all_≥, we have ℓ¯sub(θ,y′)≥⟨θ,1N∑n=1N(vn−y′n)⟩(∀θ∈ℝd). ^sub(θ;y )\ ≥\ θ,\, 1N _n=1^N(v^n-y n) (∀θ ^d). Furthermore, if each vnv^n attains F1(θ0,sn)F_1(θ^0;s^n) at some θ0θ^0, then equality holds at θ=θ0θ=θ^0. Proof. Since vn∈(sn)v^n (s^n), we have F1(θ,sn)≥⟨θ,vn⟩F_1(θ;s^n)≥ θ,v^n for every θ. Hence ℓ¯sub(θ,y′) ^sub(θ;y ) =1N∑n=1N[F1(θ,sn)−⟨θ,y′n⟩]≥1N∑n=1N⟨θ,vn−y′n⟩ = 1N _n=1^N [F_1(θ;s^n)- θ,y n ]≥ 1N _n=1^N θ,v^n-y n =⟨θ,1N∑n=1N(vn−y′n)⟩. = θ, 1N _n=1^N(v^n-y n) . The equality condition follows by substituting F1(θ0,sn)=⟨θ0,vn⟩F_1(θ^0;s^n)= θ^0,v^n (∀n∀ n). ∎ As Proposition E.3 shows, these cuts have the following properties. First, the validity of a cut depends only on the feasibility of v (vn∈(sn)v^n (s^n)), and not on at which θ, under which branch, or by which solver v was obtained. Hence the same pool V can be shared across branch updates, DC iterations, types of inner solvers, and different runs of DC optimization (as long as the sequence of states sn\s^n\ is the same). Second, since v need not even be optimal (it is valid as a lower bound as long as it is feasible), the cuts are robust to truncation and approximate solution of the forward-problem solver (although the tightness of the cuts depends on optimality). Third, v has already been obtained as a by-product of evaluating F1F_1, and generating the cuts requires no additional forward-problem solves. These are consequences of F1F_1 being a support function (positively homogeneous of degree 11), in contrast to cuts of a general convex function, which have nonzero intercepts depending on the linearization point. E.5. Underlying literature Every component of the skeleton of our method is a known method. We make the correspondence explicit. The outer loop is, among the variants of DCA formulated by Pham Dinh and Le Thi (1997) for DC programming, simplified DCA, which linearizes the concave part by a subgradient (here, the dominating vertices). Our setting fits directly into the polyhedral DC programs of Section 4 of that paper. Indeed, in (P~):infϕ(θ)−h~(θ)∣θ∈ℝd, ( P) \ \φ(θ)- h(θ) θ ^d\, h~(θ)=max⟨ai,θ⟩−αi∣i∈I(I is a finite set) h(θ)= \ a^i,θ -α^i i∈ I\ (I is a finite set) of that paper, it suffices to set ϕ(θ)=1N∑n=1NF1(θ,sn)+χΘ(θ),h~(θ)=1N∑n=1NF2(θ,xn,sn)φ(θ)= 1N _n=1^NF_1(θ;s^n)+ _ (θ), h(θ)= 1N _n=1^NF_2(θ;x^n,s^n) (where χΘ _ is the indicator function of Θ ). By Equation E.2, h~(θ)=maxy′∈≥all⟨θ,1N∑n=1Ny′n⟩, h(θ)= _y ^all_≥ θ, 1N _n=1^Ny n , so h~ h is a polyhedral convex function with ay′=1N∑n=1Ny′na^y = 1N _n=1^Ny n, αy′=0α^y =0, and I=≥allI=Y^all_≥ (and ϕφ is polyhedral convex as well), and the convex subproblem (Pi)(P_i) (infθϕ(θ)−(⟨ai,θ⟩−αi) _θ\φ(θ)-( a^i,θ -α^i)\) of that paper coincides exactly with the convex subproblem of the branch y′y msub(y′)=minθ∈Θℓ¯sub(θ,y′)m_sub(y )= _θ∈ ^sub(θ;y ) of this section. Hence Equation 5.3 of Proposition 5.2 is an instantiation, for our problem, of the equality (where α is the optimal value of the DC program (P~)( P): infθϕ(θ)−h~(θ) _θ\φ(θ)- h(θ)\ of that paper) α=infi∈Iinfθϕ(θ)−(⟨ai,θ⟩−αi)α= _i∈ I _θ\φ(θ)-( a^i,θ -α^i)\ in Section 4.1 of that paper and of the remark at the beginning of Section 4.2 that “solving the polyhedral DC program (P~)( P) globally reduces to solving |I||I| convex programs (Pi)(P_i).” The monotone decrease of the outer loop (Proposition E.11) corresponds to Theorem 5(i) of that paper (although Proposition E.11 differs in that it allows inexact solution of the inner problem). The relation between finite convergence (Proposition E.4) and Theorem 6 of that paper was discussed in Remark E.6. For later developments of DCA, see Le Thi and Pham Dinh (2018). The default inner loop is the cutting-plane method (Kelley, 1960); the facts that the convex subproblem reduces to the minimization of the suboptimality loss (Mohajerin Esfahani et al., 2018) and that its subgradient is obtained from the forward-problem oracle share the same structure as the subgradient computation of Bärmann et al. (2017). Among the interchangeable inner solvers, the proximal bundle method is due to Kiwiel (1990) and the level method to Lemaréchal et al. (1995). The idea of keeping the cuts (model) of the convex part across updates of the linearization of the concave part was proposed by de Oliveira (2019), as described in Section E.4. Related prior work includes the double bundle method (Joki et al., 2018), which has a bundle for each of the two DC components, and BEM-DC (Gaudioso et al., 2023), which assumes polyhedral DC and distributes cuts to both components; the latter is the closest to our method. The differences from these are that (i) the cuts are made not from a subgradient oracle but from the feasible solutions themselves returned by the forward-problem oracle, and, by the support-function structure, they are global lower bounds with intercept 00 that do not depend on the branch (Proposition E.3); (i) since their validity depends only on feasibility, they are robust to truncation; (i) they are kept not only within a single run but across types of inner solvers and runs (in the experiments of this paper, the pool was shared within a single run of DC optimization); and (iv) the method is applied to outperformance inverse optimization (OS loss minimization). E.6. Finite convergence of DCA and local optimality This subsection describes the finite convergence of the DC optimization method of Section 5.2 and local optimality. We use the notation of Section 5.2 (in particular, the finite set of branches ≥allY^all_≥ and the minimum value of the convex subproblem msub(y′)m_sub(y )). We also write the branch at iteration k of Algorithm 6 as y(k):=(x∗(θk,xn,sn))n=1Ny^(k):=(x^*(θ^k;x^n,s^n))_n=1^N. Then, if the inner convex subproblems are solved exactly, DC optimization stops in finitely many iterations. Proposition E.4 (Finite convergence of exact DCA). Under Assumption 4.2, suppose that in line 3 of Algorithm 6 (the application of Algorithm 5), the inner convex subproblem is solved exactly, that is, θk+1∈argminθ∈Θℓ¯sub(θ,y(k))θ^k+1∈ _θ∈ ^sub(θ;y^(k)) (by Lemma E.10, this is possible with finitely many oracle calls, in which case we may set converged=TRUEconverged=TRUE). With ϵtie=0 _tie=0 and K=∞K=∞, Algorithm 6 stops within at most |≥all||Y^all_≥| outer iterations (by certificate stopping or tie stopping). In the case of tie stopping, the point θk+1θ^k+1 of the iteration at which it stopped satisfies (E.7) ℓ¯OS(θk+1)=msub(y(k))=minθ∈Θℓ¯sub(θ,y(k)). ^OS(θ^k+1)=m_sub(y^(k))= _θ∈ ^sub(θ;y^(k)). That is, θk+1θ^k+1 is an exact minimizer of the convex subproblem for its own branch (a fixed point of DCA). Since the output of Algorithm 6 is the point with the smallest ℓk _k, its empirical mean of the OS loss is at most msub(y(k))m_sub(y^(k)). The proof is given in Section E.7. Proposition E.5 (Local optimality of tie-stopping points). In the setting of Proposition E.4, suppose that Algorithm 6 stops by tie stopping at iteration k. If argmaxx∈≥xn(sn)⟨θk+1,x⟩ _x _≥ x^n(s^n) θ^k+1,x is a singleton for every n, then θk+1θ^k+1 is a local minimizer of ℓ¯OS ^OS over Θ . Proof. Let y′:=y(k)y :=y^(k). Since y′n:=x∗(θk+1;xn,sn)y n:=x^*(θ^k+1;x^n,s^n) is an optimal solution of maxx∈≥xn(sn)⟨θk+1,x⟩ _x _≥ x^n(s^n) θ^k+1,x and y′n∈≥xn(sn)y n _≥ x^n(s^n), each term ⟨θk+1,y′n−y′n⟩ θ^k+1,y n-y n of Δk _k in Algorithm 6 is nonnegative. Under tie stopping, Δk≤ϵtie=0 _k≤ _tie=0, so each term is 00, and y′ny n is also an optimal solution at θk+1θ^k+1. By assumption, this optimal solution is unique, so argmaxx∈≥xn(sn)⟨θk+1,x⟩=y′n _x _≥ x^n(s^n) θ^k+1,x =\y n\. Let VnV_n be the set of extreme points of the pieces of ≥xn(sn)X_≥ x^n(s^n) (the nonempty intersections P∩x∈ℝd∣x≥xnP∩\x ^d x≥ x^n\ with the pieces P of (sn)X(s^n)). Then VnV_n is finite, y′n∈Vny n∈ V_n by Assumption 4.2(3), and maxx∈≥xn(sn)⟨θ,x⟩=maxv∈Vn⟨θ,v⟩ _x _≥ x^n(s^n) θ,x = _v∈ V_n θ,v for every θ∈ℝdθ ^d. By the uniqueness of the optimal solution, ⟨θk+1,y′n⟩>⟨θk+1,v⟩ θ^k+1,y n > θ^k+1,v for every v∈Vn∖y′nv∈ V_n \y n\. Since VnV_n and the number of n are finite, there exists a neighborhood U of θk+1θ^k+1 on which ⟨θ,y′n⟩>⟨θ,v⟩ θ,y n > θ,v for every n and every v∈Vn∖y′nv∈ V_n \y n\. Hence maxx∈≥xn(sn)⟨θ,x⟩=⟨θ,y′n⟩ _x _≥ x^n(s^n) θ,x = θ,y n for θ∈Uθ∈ U, and by Equations 4.2 and 5.1, ℓ¯OS(θ)=ℓ¯sub(θ,y′) ^OS(θ)= ^sub(θ;y ) on U∩ΘU∩ . By Proposition E.4, θk+1θ^k+1 is a minimizer of ℓ¯sub(⋅,y′) ^sub(·\,;y ) over Θ , and by Equation E.7, ℓ¯OS(θk+1)=ℓ¯sub(θk+1,y′) ^OS(θ^k+1)= ^sub(θ^k+1;y ). Therefore, for every θ∈U∩Θθ∈ U∩ , ℓ¯OS(θ)=ℓ¯sub(θ,y′)≥ℓ¯sub(θk+1,y′)=ℓ¯OS(θk+1) ^OS(θ)= ^sub(θ;y )≥ ^sub(θ^k+1;y )= ^OS(θ^k+1). ∎ Remark E.6 (Relation to a known finite convergence theorem). Pham Dinh and Le Thi (1997) proved the finite convergence of simplified DCA (Theorem 6 of that paper) for DC programs infθϕ(θ)−h~(θ) _θ\φ(θ)- h(θ)\ (ϕφ is a convex function) whose concave part is explicitly written as a polyhedral convex function h~(θ)=maxi∈I(⟨ai,θ⟩−αi)(I is a finite index set, ai∈ℝd,αi∈ℝ). h(θ)= _i∈ I ( a^i,θ -α^i ) (I is a finite index set, a^i ^d,\ α^i ). The empirical mean of the OS loss can also be written in this form ϕ−h~φ- h, and the index i of the linear pieces corresponds to the branch y′∈≥ally ^all_≥ (Section E.5). However, that theorem assumes the natural choice of subgradients, that is, a choice in the relative interior of ∂h~(θ)∂ h(θ) that puts positive weights on all active linear pieces (Lemma 3 of that paper), and implementing it requires enumerating all linear pieces active at each point θ, that is, all dominating vertices that are optimal solutions of maxx∈≥xn(sn)⟨θ,x⟩ _x _≥ x^n(s^n) θ,x for each state sns^n. Since the forward-problem oracle returns only one optimal solution, this enumeration is difficult in general, and applying that theorem directly to our setting is difficult. Therefore, in this paper, we prove finiteness directly using the tie stopping rule in the setting where the oracle returns only one optimal solution (Proposition E.4). Remark E.7 (Guarantee of global optimality). The point obtained by tie stopping is a minimizer for the branch y(k)y^(k). Global optimality is guaranteed by certificate stopping (ℓk≤ϵgap _k≤ _gap; Proposition E.9); in particular, in settings where Equation 3.2 is realizable (the global minimum value is 00), reaching the global minimum can be confirmed by a loss value of 00. Note that, when Θ=Δd−1 = ^d-1, the set of weights at which the optimal solution is not unique has measure 00 on Δd−1 ^d-1 (Proposition E.8). Proposition E.8 (Kitaoka, 2024, Lemma 3.3). Assume Assumption 4.2(2) and let Θ=Δd−1 = ^d-1. Then, for any states s1,…,sN∈s^1,…,s^N and xn∈(sn)x^n (s^n) (n=1,…,Nn=1,…,N), for almost every θ∈Δd−1θ∈ ^d-1 (with respect to the measure on Δd−1 ^d-1 induced by the Lebesgue measure), both argmaxx∈(sn)⟨θ,x⟩ _x (s^n) θ,x and argmaxx∈≥xn(sn)⟨θ,x⟩ _x _≥ x^n(s^n) θ,x are singletons for every n. Kitaoka (2024, Lemma 3.3) shows that, when the feasible region of each state is a finite union of bounded closed convex polytopes, for almost every θ∈Δd−1θ∈ ^d-1, the feature of the optimal solution (in that paper, the image f(x,s)f(x,s) of a feasible solution x under the feature map f is called the feature; our setting corresponds to f(x,s)=xf(x,s)=x) is uniquely determined for every n. Its proof shows that the maximizer of ⟨θ,⋅⟩ θ,· over the finite set of extreme points of the pieces is unique. If the maximizer among the extreme points is unique, the optimal face of each piece is a single point; hence, applying this with the identity feature map to (sn)X(s^n) and ≥xn(sn)X_≥ x^n(s^n), both of which are finite unions of nonempty bounded polyhedra by Remark 4.3, yields Proposition E.8. Although that lemma is stated under the assumption that the observations are optimal solutions for a true weight, its proof does not use this assumption. E.7. Theoretical analysis Complete proofs are given for all propositions in this subsection. We first state the validity of certificate stopping. Proposition E.9 (Certificate of global optimality). For any θ∈Θθ∈ and ϵ≥0ε≥ 0, if ℓ¯OS(θ)≤ϵ ^OS(θ)≤ε, then we have ℓ¯OS(θ)≤minθ′∈Θℓ¯OS(θ′)+ϵ. ^OS(θ)≤ _θ ∈ ^OS(θ )+ε. In particular, if ϵ=0ε=0, then θ is a global minimizer of Equation 4.4. Proof. By the inclusion ≥xn(sn)⊆(sn)X_≥ x^n(s^n) (s^n), each term satisfies ℓOS(θ′,xn,sn)≥0 ^OS(θ ;x^n,s^n)≥ 0, and hence minθ′∈Θℓ¯OS(θ′)≥0 _θ ∈ ^OS(θ )≥ 0. Therefore ℓ¯OS(θ)≤ϵ≤minθ′ℓ¯OS(θ′)+ϵ ^OS(θ)≤ε≤ _θ ^OS(θ )+ε. ∎ Next, we show that the inner cutting-plane method reaches an exact minimizer in finitely many iterations in the ideal setting (no truncation and tolerance 00). Lemma E.10 (Finite exactness of the inner loop). Under Assumption 4.2, let ϵin=0 _in=0 and J=∞J=∞, suppose that the LP in Algorithm 5 is solved exactly, and let the selection rule for evaluation points be the cutting-plane method (θj+1θ^j+1 is a minimizer attaining LBLB). Set M:=∏n=1N|(sn)|M:= _n=1^N|Y(s^n)| (an upper bound on the total number of values that a tuple of oracle solutions (vn)n(v^n)_n can take). (1): When Algorithm 6 calls Algorithm 5 once, the call stops within at most M+1M+1 inner iterations (iterations of the loop over j in Algorithm 5), and the returned θ θ is an exact minimizer of ℓ¯sub(⋅,y′) ^sub(·\,;y ) over Θ . This bound does not depend on the contents of the pool V at the time of the call. (2): When Algorithm 6 is run for K outer iterations (iterations of the loop over k), the total number of inner iterations executed during the run is at most M+2K−1M+2K-1. Proof. Let jG_j be the cut set at the end of iteration j, and let LBj+1=minθ∈Θmaxg∈j⟨θ,g⟩LB_j+1= _θ∈ _g _j θ,g be the optimal value of the LP. By Proposition E.3, all cuts in jG_j (including the inherited cuts) are linear lower bounds of ℓ¯sub(⋅,y′) ^sub(·\,;y ), so LBj+1≤msub(y′)LB_j+1≤ m_sub(y ), and since jG_j is monotonically increasing, LBj+1LB_j+1 is nondecreasing in j. Moreover, since UBUB is the minimum of the values actually evaluated, ℓ¯sub(θi,y′)=⟨θi,gi⟩ ^sub(θ^i;y )= θ^i,g_i , we always have UB≥msub(y′)UB≥ m_sub(y ). Therefore, at stopping (UB≤LBUB ), UB=msub(y′)UB=m_sub(y ), and θ θ is a minimizer. We show finiteness. Let 0V_0 be the pool at the time of the call. By the initialization of G in Algorithm 5, the cut of each tuple in 0V_0 (for the current branch y′y ) is already contained in G at the first inner iteration. The tuple of oracle solutions vjv_j takes values in the finite set ∏n(sn) _nY(s^n). Suppose that at some inner iteration j≥1j≥ 1, vjv_j coincides with an element of 0∪v0,…,vj−1V_0∪\v_0,…,v_j-1\. Then its cut equals gjg_j and belongs to j−1G_j-1. Since θjθ^j (j≥1j≥ 1) is an optimal solution of the LP over j−1G_j-1, ℓ¯sub(θj,y′)=⟨θj,gj⟩≤maxg∈j−1⟨θj,g⟩=LBj. ^sub(θ^j;y )= θ^j,g_j ≤ _g _j-1 θ^j,g =LB_j. Hence, after the update at inner iteration j, UB≤LBj≤LBj+1UB _j _j+1, and the call stops at this iteration. By contraposition, at an inner iteration j≥1j≥ 1 that does not stop, vjv_j is always a tuple newly added to V. At j=0j=0, the initial point θ0=θkθ^0=θ^k is a warm start and is not necessarily an optimal solution of the LP, so v0v_0 may coincide with an element of 0V_0 without stopping; this can happen only if |0|≥1|V_0|≥ 1. Since ||≤M|V|≤ M, the inner iterations that do not stop occur at most (M−|0|)+min1,|0|≤M(M-|V_0|)+ \1,|V_0|\≤ M times, and adding the stopping iteration, the number of inner iterations is at most M+1M+1. This bound does not depend on 0V_0. This proves (1). We show (2). In Algorithm 6, the pool V increases monotonically across outer iterations and is never reinitialized. By the argument of (1), every inner iteration that does not stop either adds one new tuple to V or is an iteration j=0j=0 whose tuple v0v_0 already belongs to the pool. Throughout the whole run, the former occur at most M times. The latter occur at most once per call, and not in the first call, since 0=∅V^0= ; hence they occur at most K−1K-1 times. In addition, each call has one stopping inner iteration, and the number of calls equals the number K of outer iterations; hence the total number of inner iterations is at most M+(K−1)+K=M+2K−1M+(K-1)+K=M+2K-1. ∎ Lemma E.10(2) expresses the quantitative effect of cut inheritance (Proposition E.3). Without sharing the pool, the bound M+1M+1 of (1) is needed independently for each call, and the bound on the total is only K(M+1)K(M+1). With a shared pool, a tuple obtained once is reused as a cut in subsequent calls, and an inner iteration j≥1j≥ 1 that returns a duplicate tuple causes stopping on the spot, so the total drops to M+2K−1M+2K-1. For the outer loop, we first show, as a property that holds even if the inner loop is inexact, the monotone decrease of the OS loss. Proposition E.11 (Descent property). Write the branch at iteration k of Algorithm 6 (the y′y when Algorithm 5 is applied) as y(k):=(x∗(θk,xn,sn))n=1Ny^(k):=(x^*(θ^k;x^n,s^n))_n=1^N. If the inner solver returns θk+1θ^k+1 satisfying (E.8) ℓ¯sub(θk+1,y(k))≤ℓ¯sub(θk,y(k)) ^sub(θ^k+1;y^(k))≤ ^sub(θ^k;y^(k)) (since Algorithm 5 first evaluates the initial point θ0=θkθ^0=θ^k and returns the best of the evaluated points, it satisfies Equation E.8 even when truncated), then we have 0≤ℓ¯OS(θk+1)≤ℓ¯OS(θk).0≤ ^OS(θ^k+1)≤ ^OS(θ^k). Proof. Since y(k),ny^(k),n attains F2(θk,xn,sn)F_2(θ^k;x^n,s^n), we have ℓ¯sub(θk,y(k))=ℓ¯OS(θk) ^sub(θ^k;y^(k))= ^OS(θ^k). On the other hand, since y(k),n∈≥xn(sn)y^(k),n _≥ x^n(s^n), we have F2(θk+1,xn,sn)≥⟨θk+1,y(k),n⟩F_2(θ^k+1;x^n,s^n)≥ θ^k+1,y^(k),n , so ℓ¯OS(θk+1)=1N∑n=1N[F1(θk+1,sn)−F2(θk+1,xn,sn)]≤ℓ¯sub(θk+1,y(k)). ^OS(θ^k+1)= 1N _n=1^N [F_1(θ^k+1;s^n)-F_2(θ^k+1;x^n,s^n) ]≤ ^sub(θ^k+1;y^(k)). Combined with Equation E.8, this gives the claim (nonnegativity holds as in the proof of Proposition E.9). ∎ Finally, we prove the finite convergence of the outer loop when the inner convex subproblems are solved exactly (Proposition E.4). Proof of Proposition E.4. At iteration k, y′n=y(k+1),ny n=y^(k+1),n. First, since y(k),n∈≥xn(sn)y^(k),n _≥ x^n(s^n) and y(k+1),ny^(k+1),n attains F2(θk+1,xn,sn)F_2(θ^k+1;x^n,s^n), each term of Δk _k satisfies ⟨θk+1,y(k+1),n−y(k),n⟩=F2(θk+1,xn,sn)−⟨θk+1,y(k),n⟩≥0; θ^k+1,y^(k+1),n-y^(k),n =F_2(θ^k+1;x^n,s^n)- θ^k+1,y^(k),n ≥ 0; hence Δk≥0 _k≥ 0, and moreover we have (E.9) ℓ¯OS(θk+1)=ℓ¯sub(θk+1,y(k))−Δk=msub(y(k))−Δk. ^OS(θ^k+1)= ^sub(θ^k+1;y^(k))- _k=m_sub(y^(k))- _k. Here, the second equality follows from the exactness of the inner loop. Also, since ℓ¯sub(θk+1,y(k+1))=ℓ¯OS(θk+1) ^sub(θ^k+1;y^(k+1))= ^OS(θ^k+1), (E.10) msub(y(k+1))≤ℓ¯sub(θk+1,y(k+1))=msub(y(k))−Δk.m_sub(y^(k+1))≤ ^sub(θ^k+1;y^(k+1))=m_sub(y^(k))- _k. If neither stopping occurs at iteration k, then Δk>0 _k>0, and by Equation E.10, msub(y(k+1))<msub(y(k))m_sub(y^(k+1))<m_sub(y^(k)), i.e., the branch minimum strictly decreases. Since msubm_sub takes values in the finite set msub(y′)∣y′∈≥all\m_sub(y ) y ^all_≥\, strict decreases can occur at most |≥all|−1|Y^all_≥|-1 times, and one of the stoppings occurs within at most |≥all||Y^all_≥| iterations. At tie stopping, Δk=0 _k=0, so Equation E.9 gives Equation E.7. ∎ Appendix F Exact solution of the inner convex subproblems based on gaps In this section, we extend the finite exactness of Lemma E.10 (the guarantee when the rule for selecting evaluation points is the cutting-plane method) to stabilized selection rules such as the level method and the proximal bundle method. The key is to transplant to the inner convex subproblems the idea used in Kitaoka (2024); Kitaoka (2026): strengthen an approximation guarantee to exact attainment through a positive gap possessed by a piecewise-linear structure, and make the number of iterations explicit. Throughout this section, we assume Assumption 4.2 and fix a branch y′∈≥ally ^all_≥. F.1. Submodel gap Let all:=∏n=1N(sn)Y^all:= _n=1^NY(s^n) be the set of all tuples of solutions that the oracle can return (a finite set; |all|=M|Y^all|=M is the M of Lemma E.10), and write the cut of a tuple v∈allv ^all as gv:=1N∑n=1N(vn−y′n)g_v:= 1N _n=1^N(v^n-y n) (gvg_v depends on the fixed branch y′y , which we suppress in the notation). We call gvg_v a vertex cut. By Equation E.2, we have (F.1) ℓ¯sub(θ,y′)=maxv∈all⟨θ,gv⟩(∀θ∈ℝd). ^sub(θ;y )= _v ^all θ,g_v (∀θ ^d). For a nonempty W⊆allW ^all, we define the minimum value of the submodel by (F.2) mW(y′):=minθ∈Θmaxv∈W⟨θ,gv⟩.m_W(y ):= _θ∈ _v∈ W θ,g_v . If W⊆W′W W , then mW(y′)≤mW′(y′)m_W(y )≤ m_W (y ), and by Equation F.1, mall(y′)=msub(y′)m_Y^all(y )=m_sub(y ). We define the submodel gap by (F.3) δ(y′):=msub(y′)−maxmW(y′)|∅≠W⊆all,mW(y′)<msub(y′)δ(y ):=m_sub(y )- \\,m_W(y )\ |\ ≠ W ^all,\ m_W(y )<m_sub(y )\, \ (when the set over which the max on the right-hand side is taken is empty, we read δ(y′):=+∞δ(y ):=+∞). δ(y′)δ(y ) is not an assumption but a complexity constant that is automatically positive by finiteness. Lemma F.1 (Positivity of the gap and discreteness of the lower bound). (i) δ(y′)>0δ(y )>0. (i) When Algorithm 5 is run with the branch y′y and a pool 0V_0 consisting only of tuples of solutions returned by the oracle, the value LBLB computed at each iteration equals mW(y′)m_W(y ) for some nonempty W⊆allW ^all. In particular, the values that LBLB can take belong to the finite set mW(y′)∣∅≠W⊆all\m_W(y ) ≠ W ^all\, and we always have LB≤msub(y′)LB≤ m_sub(y ). Proof. (i) Since allY^all is finite, it has finitely many subsets, so the largest of the submodel minimum values with mW(y′)<msub(y′)m_W(y )<m_sub(y ) is attained and is strictly smaller than msub(y′)m_sub(y ). (i) The cut set G of Algorithm 5 is always the set of all cuts gv∣v∈W\g_v v∈ W\ of the set W consisting of “the tuples of 0V_0” together with “the tuples evaluated in that call”. Since every tuple added to the pool in Algorithms 5 and 6 is a return value of the oracle, W⊆allW ^all, and LB=minθ∈Θmaxv∈W⟨θ,gv⟩=mW(y′)LB= _θ∈ _v∈ W θ,g_v =m_W(y ). The inequality LB≤msub(y′)LB≤ m_sub(y ) follows from monotonicity with respect to W⊆allW ^all. ∎ F.2. Exact agreement of the lower bound and finite exactness of cutting-plane finishing Lemma F.2 (Exact agreement of the lower bound). In the setting of Lemma F.1(i), let 0≤ϵin<δ(y′)0≤ _in<δ(y ). When Algorithm 5 stops with UB−LB≤ϵinUB-LB≤ _in (converged=TRUEconverged=TRUE), regardless of the rule for selecting evaluation points, we have LB=msub(y′),ℓ¯sub(θ^,y′)=UB≤msub(y′)+ϵin.LB=m_sub(y ), ^sub( θ;y )=UB≤ m_sub(y )+ _in. That is, the minimum value of the convex subproblem itself is identified exactly. Proof. Since UBUB is the minimum of the true values of ℓ¯sub(⋅,y′) ^sub(·\,;y ) at the evaluation points, UB≥msub(y′)UB≥ m_sub(y ). At stopping, LB≥UB−ϵin≥msub(y′)−ϵin>msub(y′)−δ(y′)LB - _in≥ m_sub(y )- _in>m_sub(y )-δ(y ). On the other hand, by Lemma F.1(i), LB=mW(y′)≤msub(y′)LB=m_W(y )≤ m_sub(y ), and by Equation F.3, every value with mW(y′)<msub(y′)m_W(y )<m_sub(y ) is at most msub(y′)−δ(y′)m_sub(y )-δ(y ); hence LB=msub(y′)LB=m_sub(y ). ∎ What becomes exact in Lemma F.2 is the lower bound (the identification of the minimum value), and the returned point θ θ is ϵin _in-optimal. To obtain an exact minimizer, we use the following cutting-plane finishing: in Phase 1, run with an arbitrary rule for selecting evaluation points (e.g., the level method) until UB−LB≤ϵswUB-LB≤ _sw (where ϵsw>0 _sw>0 is the switching threshold); in the subsequent Phase 2, switch the selection rule to the cutting-plane method (the minimizer of the model) and continue with tolerance 00. Theorem F.3 (Finite exactness of cutting-plane finishing). In the setting of Lemma F.1(i), let the tolerance of Algorithm 5 be 00, 0<ϵsw<δ(y′)0< _sw<δ(y ), ϵterm=0 _term=0 in Algorithm 4, and J=∞J=∞, and assume that the LPs are solved exactly. If Phase 1 reaches UB−LB≤ϵswUB-LB≤ _sw in finitely many iterations (which holds for the cutting-plane method by Lemma E.10 and, for the level method, whenever the iteration bound of Remark F.4 holds, e.g., for the standard level method without inherited cuts), then cutting-plane finishing stops in finitely many iterations, and the returned θ θ is an exact minimizer satisfying ℓ¯sub(θ^,y′)=msub(y′)=minθ∈Θℓ¯sub(θ,y′). ^sub( θ;y )=m_sub(y )= _θ∈ ^sub(θ;y ). Proof. At the end of Phase 1, since UB−LB≤ϵsw<δ(y′)UB-LB≤ _sw<δ(y ), applying the proof of Lemma F.2 with ϵin _in replaced by ϵsw _sw gives LB=msub(y′)LB=m_sub(y ). In Phase 2, cuts are only added, so LBLB is nondecreasing, and by Lemma F.1(i) it is always at most msub(y′)m_sub(y ); hence LB=msub(y′)LB=m_sub(y ) is maintained. Each iteration of Phase 2 evaluates at the model minimizer θjθ^j (a point attaining LBLB). If the tuple vjv_j obtained by the evaluation already belongs to the set W corresponding to G, then ℓ¯sub(θj,y′)=⟨θj,gvj⟩≤maxg∈⟨θj,g⟩=LB=msub(y′), ^sub(θ^j;y )= θ^j,g_v_j ≤ _g θ^j,g =LB=m_sub(y ), and since ℓ¯sub(⋅,y′)≥msub(y′) ^sub(·\,;y )≥ m_sub(y ) always holds (on Θ ), ℓ¯sub(θj,y′)=msub(y′) ^sub(θ^j;y )=m_sub(y ). At this point the algorithm stops with UB=msub(y′)=LBUB=m_sub(y )=LB, and θ^=θj θ=θ^j is an exact minimizer (it likewise stops if ℓ¯sub(θj,y′)=msub(y′) ^sub(θ^j;y )=m_sub(y ) occurs with a new tuple). In every iteration that does not stop, a new tuple is added to the pool, and since allY^all is finite, Phase 2 stops in finitely many iterations. ∎ Remark F.4 (Explicit number of iterations with the level method). For a convex Lipschitz function on a compact convex set, the number of iterations the level method needs to reach a certified gap UB−LB≤ϵUB-LB≤ε is bounded by c(λ)(Lg(y′)DΘ/ϵ)2c(λ)\, (L_g(y )\,D_ /ε )^2 (λ∈(0,1)λ∈(0,1) is the level parameter and c(λ)c(λ) is a constant depending only on it; Lemaréchal et al., 1995). Here Lg(y′):=maxv∈all‖gv‖L_g(y ):= _v ^all\|g_v\| is an upper bound on the Lipschitz constant of ℓ¯sub(⋅,y′) ^sub(·\,;y ), and DΘ:=diam(Θ)D_ := ( ). Therefore, choosing ϵsw<δ(y′) _sw<δ(y ), the number of iterations of Phase 1 is of order c(λ)(Lg(y′)DΘ/δ(y′))2c(λ)\, (L_g(y )\,D_ /δ(y ) )^2 and is bounded explicitly, independently of the bound M+1M+1 on the number of tuples in Lemma E.10. Note that this bound is a guarantee for the standard level method whose model is built only from tangents at the evaluation points. Using inherited cuts does not affect the validity of the stopping test (Lemma F.2). Evaluating the number of iterations with inherited cuts is future work. Lemma F.5 (Total number of finishing iterations). Under Assumption 4.2, call Algorithm 5 at line 3 of Algorithm 6 with tolerance 00 and J=∞J=∞, and let the rule of its Step 12 be Algorithm 4. Assume that the LPs of Algorithm 5 are solved exactly, and let M:=∏n=1N|(sn)|M:= _n=1^N|Y(s^n)|. An inner iteration j(≥1)j\ (≥ 1) is called a finishing iteration if its evaluation point θjθ^j is chosen in the cutting-plane branch of Algorithm 4 (the side UB−LB≤ϵswUB-LB≤ _sw). Then the total number of finishing iterations executed while Algorithm 6 runs up to K outer iterations is at most M+KM+K. This bound does not depend on the value of the switching threshold ϵsw _sw. Proof. We use the notation of the proof of Lemma E.10: let jG_j be the cut set at the end of inner iteration j, and LBj+1=minθ∈Θmaxg∈j⟨θ,g⟩LB_j+1= _θ∈ _g _j θ,g . In a finishing iteration j, θjθ^j is the output of Algorithm 1, that is, a minimizer on Θ of the model of j−1G_j-1, so maxg∈j−1⟨θj,g⟩=LBj _g _j-1 θ^j,g =LB_j. If the tuple vjv_j returned by the oracle already belongs to the pool V at that time, then its cut (for the current branch y′y ) equals gjg_j and belongs to j−1G_j-1, so ℓ¯sub(θj,y′)=⟨θj,gj⟩≤maxg∈j−1⟨θj,g⟩=LBj≤LBj+1. ^sub(θ^j;y )= θ^j,g_j ≤ _g _j-1 θ^j,g =LB_j _j+1. After UBUB is updated, UB≤ℓ¯sub(θj,y′)≤LBj+1UB≤ ^sub(θ^j;y ) _j+1, so the stopping test with tolerance 00 is satisfied and the call stops at this iteration. By contraposition, in a finishing iteration that does not stop, vjv_j is necessarily a tuple newly added to the pool. In Algorithm 6, the pool V increases monotonically across outer iterations and is never reinitialized (as in the proof of Lemma E.10(2)). Therefore, throughout the whole run, the number of finishing iterations that do not stop is at most ||≤M|V|≤ M (iterations of Phase 1 also consume the pool, but this only acts to decrease the number of finishing iterations). Each call has at most one stopping iteration, and the number of calls equals the number K of outer iterations, so the total number of finishing iterations is at most M+KM+K. The threshold ϵsw _sw does not appear in the above argument. ∎ Proposition F.6 (Total number of inner iterations over the whole run). Under Assumption 4.2, let the rule of Step 12 of Algorithm 5 be Algorithm 4 (with switching threshold ϵsw _sw, termination tolerance ϵterm=0 _term=0, and Phase-1 rule R), and call Algorithm 5 with tolerance 00 and J=∞J=∞. Assume that the LPs are solved exactly, and let δin:=miny′∈≥allδ(y′)>0,0<ϵsw<δin _in:= _y ^all_≥δ(y )>0, 0< _sw< _in (since ≥allY^all_≥ is finite, δin>0 _in>0). Assume further that for every branch, the rule R reaches UB−LB≤ϵswUB-LB≤ _sw in at most T1T_1 inner iterations. Then the following hold. (1): Algorithm 6 stops in at most |≥all||Y^all_≥| outer iterations, and each call returns an exact minimizer of ℓ¯sub(⋅,y′) ^sub(·\,;y ). (2): The total number of inner iterations executed over the whole run is at most |≥all|T1⏟Phase 1+M+|≥all|⏟finishing |Y^all_≥|\,T_1_Phase 1\;+\; M+|Y^all_≥|_finishing (M:=∏n=1N|(sn)|M:= _n=1^N|Y(s^n)|). (3): The number of calls to the forward-problem oracle is at most N(|≥all|T1+M+|≥all|)+2N|≥all|+N (|Y^all_≥|\,T_1+M+|Y^all_≥| )+2N|Y^all_≥|+N. When the level method (Algorithm 3) is taken as the rule R, T1=c(λ)(LgDΘ/ϵsw)2T_1=c(λ) (L_g\,D_ / _sw )^2 (Lg:=maxy′∈≥allLg(y′)L_g:= _y ^all_≥L_g(y ), DΘ:=diam(Θ)D_ := ( ); Remark F.4. This T1T_1 is the value when the bound of Remark F.4 holds). Proof. We show (1). Since ϵsw<δin≤δ(y′) _sw< _in≤δ(y ) holds for every branch y′∈≥ally ^all_≥, each call satisfies the assumptions of Theorem F.3, returns an exact minimizer, and gives converged=TRUEconverged=TRUE. This is the premise of Proposition E.4 (exact solution of the inner problems), and by that proposition, Algorithm 6 stops in at most |≥all||Y^all_≥| outer iterations. We show (2). Split the inner iterations of one call into those up to the iteration at which UB−LB≤ϵswUB-LB≤ _sw first holds and those after it. Since UB−LBUB-LB is nonincreasing in the iterations (Section D.4), this split yields two intervals, a first half and a second half, and all evaluation points of the second half are chosen in the cutting-plane branch of Algorithm 4. That is, all iterations of the second half are finishing iterations in the sense of Lemma F.5. By assumption, the number of iterations of the first half is at most T1T_1 in each call, and by (1) the number of calls is at most |≥all||Y^all_≥|, so the total of the first halves is at most |≥all|T1|Y^all_≥|T_1. Substituting K≤|≥all|K≤|Y^all_≥| into Lemma F.5, the total of the second halves is at most M+|≥all|M+|Y^all_≥|. Together, we obtain (2). We show (3). Calls to the forward-problem oracle occur N times per inner iteration at Step 3 of Algorithm 5 and 2N2N times per outer iteration at Step 4 of Algorithm 6, and N times at the initialization of the branch in Step 1 of Algorithm 6, and nowhere else. By (1) and (2), the first two are bounded by N(|≥all|T1+M+|≥all|)N(|Y^all_≥|T_1+M+|Y^all_≥|) and 2N|≥all|2N|Y^all_≥|, respectively, and the last is N. ∎ Remark F.7 (Roles of the two terms). The first term of Proposition F.6(2) depends on the numerical fineness of the problem through the gap δin _in, whereas the second term M+|≥all|M+|Y^all_≥| does not depend on the gap and is determined only by two combinatorial quantities: the number M=|all|M=|Y^all| of tuples of solutions that the oracle can return and the number |≥all||Y^all_≥| of branches (all=∏n(sn)Y^all= _nY(s^n) and ≥all=∏n≥xn(sn)Y^all_≥= _nY_≥ x^n(s^n) are different sets). Without cut inheritance, the second term would be |≥all|(M+1)|Y^all_≥|(M+1) (applying Lemma E.10(1) to each call), which differs by a factor of |≥all||Y^all_≥|. Remark F.8 (Connection to the outer loop and future work). First, the number of iterations and the number of oracle calls over the whole run are summarized in Proposition F.6. Second, a lower bound on δin _in is given explicitly for the integer case in Section F.3. Improvements using the structure of integer programs (total unimodularity, Graver bases, M-convexity) are left for future work, as described in Remark F.14. Third, the finite exact attainment of Kitaoka (2024) uses the structure in which the set of minimizers has a relative interior point, but in the inner problem of a general branch, the argmin can be a low-dimensional face; hence, in this paper, we combine the discreteness of the lower bound (Lemma F.2) with finishing by the cutting-plane method (Theorem F.3) rather than relying on the convergence of the values at the evaluation points. F.3. Explicit lower bound on the submodel gap in the integer case With the same motivation as Kitaoka (2026), who gave an explicit lower bound on the constant γ(ℓsub)γ( ^sub) (a positive constant determined by the geometry of the set of minimizers of the suboptimality loss, which bounds by O(1/γ(ℓsub)2)O(1/γ( ^sub)^2) the number of iterations a gradient method needs to reach ℓsub=0 ^sub=0; Kitaoka, 2024), we give an explicit lower bound on δ(y′)δ(y ). The key is that all submodel minimum values mW(y′)m_W(y ) are rational numbers with a common upper bound on their denominators; although there are exponentially many subsets W, enumerating them is unnecessary (only the upper bound on the denominators matters). Proposition F.9 (Explicit lower bound on δ in the integer case). In addition to Assumption 4.2, let N=1N=1, Θ=Δd−1 = ^d-1, and (s1)⊆ℤdX(s^1) ^d, and let R:=maxx∈(s1)‖x‖∞R:= _x (s^1)\|x\|_∞. Then for every branch y′∈≥ally ^all_≥, δ(y′)≥1Q2,Q:=((d+1)1/2max(2R,1))d+1.δ(y )\ ≥\ 1Q^2, Q:= ((d+1)^1/2 (2R,1) )^d+1. Proof. First, we show that for every nonempty W⊆allW ^all, mW(y′)m_W(y ) is a rational number with denominator at most Q. Since N=1N=1, each cut gv=v−y′g_v=v-y (v,y′∈(s1)⊆ℤdv,y (s^1) ^d) is an integer vector with ‖gv‖∞≤2R\|g_v\|_∞≤ 2R. mW(y′)m_W(y ) is the optimal value of the linear program minτ|(θ,τ)∈PW,PW:=(θ,τ)∈ℝd×ℝ|θ≥0,∑i=1dθi=1,⟨gv,θ⟩≤τ(∀v∈W). \\,τ\ |\ (θ,τ)∈ P_W\, \, P_W:= \(θ,τ) ^d×R\ |\ aligned &θ≥ 0,\ _i=1^d _i=1,\\ & g_v,θ ≤τ\ (∀ v∈ W) aligned \. PWP_W is nonempty and contains no line (θ is bounded, and the direction (0,−1)(0,-1) is incompatible with the constraints ⟨gv,θ⟩≤τ g_v,θ ≤τ), and the optimal value is finite by the compactness of Θ , so it is attained at some extreme point (θ,τ)(θ,τ). An extreme point is the unique solution of a nonsingular system of linear equations consisting of (d+1)(d+1) linearly independent tight constraints, and each row of its coefficient matrix A is an integer vector of type (gv,−1)(g_v,-1), (,0)(1,0), or (−ei,0)(-e_i,0). The ℓ2 _2 norms of the rows are bounded by d(2R)2+1≤(d+1)1/2max(2R,1) d(2R)^2+1≤(d+1)^1/2 (2R,1), d d, and 11, respectively, so by Hadamard’s inequality, 1≤|detA|≤((d+1)1/2max(2R,1))d+1=Q1≤| A|≤ ((d+1)^1/2 (2R,1) )^d+1=Q (the lower bound holds because A is a nonsingular integer matrix). By Cramer’s rule, τ is a rational number with denominator |detA|≤Q| A|≤ Q, and we can write mW(y′)=p/qm_W(y )=p/q (p∈ℤp , 1≤q≤Q1≤ q≤ Q). If δ(y′)=+∞δ(y )=+∞, the claim holds. Otherwise, there exists W attaining the max in Equation F.3, and δ(y′)=msub(y′)−mW(y′)>0δ(y )=m_sub(y )-m_W(y )>0. Since msub(y′)=mall(y′)m_sub(y )=m_Y^all(y ) is also of the above form, we can write msub(y′)=p1/q1m_sub(y )=p_1/q_1 and mW(y′)=p2/q2m_W(y )=p_2/q_2 (q1,q2≤Qq_1,q_2≤ Q); since the two are distinct, δ(y′)=|p1q2−p2q1|q1q2≥1q1q2≥1Q2.δ(y )= |p_1q_2-p_2q_1|q_1q_2≥ 1q_1q_2≥ 1Q^2. ∎ Corollary F.10 (Extension to N≥2N≥ 2). In addition to Assumption 4.2, let N≥2N≥ 2, Θ=Δd−1 = ^d-1, and (sn)⊆ℤdX(s^n) ^d for each n, and let R:=maxnmaxx∈(sn)‖x‖∞R:= _n _x (s^n)\|x\|_∞. Then for every branch y′∈≥ally ^all_≥, we have δ(y′)≥1NQN2,QN:=((d+1)1/2max(2NR,1))d+1.δ(y )\ ≥\ 1N\,Q_N^2, Q_N:= ((d+1)^1/2 (2NR,1) )^d+1. As in the lower bound on γ(ℓsub)γ( ^sub) (Kitaoka, 2026), the dependence on N appears through QNQ_N and the factor 1/N1/N. Proof. For the cut gv=1N∑n=1N(vn−y′n)g_v= 1N _n=1^N(v^n-y n), we have Ngv∈ℤdNg_v ^d and ‖Ngv‖∞≤2NR\|Ng_v\|_∞≤ 2NR. Applying the first step of the proof of Proposition F.9 to the family of integer cuts Ngvv\Ng_v\_v, for every nonempty W⊆allW ^all, NmW(y′)=minθ∈Δd−1maxv∈W⟨θ,Ngv⟩N\,m_W(y )= _θ∈ ^d-1 _v∈ W θ,Ng_v is a rational number with denominator at most QNQ_N. Hence the difference of two distinct values of NmW(y′)N\,m_W(y ) is at least 1/QN21/Q_N^2, and the difference of two distinct values of mW(y′)m_W(y ) is at least 1/(NQN2)1/(N\,Q_N^2). The rest follows as in the second step of the proof of Proposition F.9. ∎ Remark F.11 (Fully explicit number of iterations of Phase 1). In the setting of Proposition F.9, choosing ϵsw:=1/(2Q2)<δ(y′) _sw:=1/(2Q^2)<δ(y ) and combining with Remark F.4, the number of iterations of Phase 1 (level method) is bounded by c(λ)(Lg(y′)DΘ⋅2Q2)2≤c(λ)(2Rd⋅2⋅2Q2)2,c(λ) (L_g(y )\,D_ \,· 2Q^2 )^2≤ c(λ)\, (2R d· 2· 2Q^2 )^2, a fully explicit function of the problem size (d, R) (using Lg(y′)≤2RdL_g(y )≤ 2R d and diam(Δd−1)=2 ( ^d-1)= 2). Since Q4=((d+1)max(2R,1)2)2(d+1)Q^4= ((d+1) (2R,1)^2 )^2(d+1), this bound is exponential in d. This is consistent with the fact that the lower bound on γ(ℓsub)γ( ^sub) for general ILPs can be exponentially small in d (and that this is essential) (Kitaoka, 2026). Corollary F.12 (Explicit number of iterations over the whole run in the integer case). In the setting of Proposition F.9 for N=1N=1 and of Corollary F.10 for N≥2N≥ 2 (Θ=Δd−1 = ^d-1, (sn)⊆ℤdX(s^n) ^d for each n, R:=maxnmaxx∈(sn)‖x‖∞R:= _n _x (s^n)\|x\|_∞), use the framework of Proposition F.6 with the rule R being the level method and the switching threshold ϵsw:=12NQN2,QN=((d+1)1/2max(2NR,1))d+1. _sw:= 12NQ_N^2, Q_N= ((d+1)^1/2 (2NR,1) )^d+1. Assume that the iteration bound of Remark F.4 holds for the level method in Phase 1. Then ϵsw<δin _sw< _in, and the total number of inner iterations over the whole run is at most |≥all|c(λ)(2Rd⋅2⋅2NQN2)2+M+|≥all| |Y^all_≥|\,c(λ)\, (2R d· 2·2NQ_N^2 )^2+M+|Y^all_≥| =32c(λ)|≥all|R2dN2QN4+M+|≥all|, =32\,c(λ)\,|Y^all_≥|\,R^2\,d\,N^2\,Q_N^4\;+\;M+|Y^all_≥|, which is a fully explicit function of the problem size (d, R, N), |≥all||Y^all_≥|, and M. Proof. By Corollary F.10, δ(y′)≥1/(NQN2)δ(y )≥ 1/(NQ_N^2) for every branch, so δin≥1/(NQN2)>ϵsw _in≥ 1/(NQ_N^2)> _sw. For the Lipschitz constant, ‖gv‖=‖1N∑n(vn−y′n)‖≤maxn‖vn−y′n‖≤2Rd\|g_v\|= \| 1N _n(v^n-y n) \|≤ _n\|v^n-y n\|≤ 2R d gives Lg≤2RdL_g≤ 2R d, and DΘ=diam(Δd−1)=2D_ = ( ^d-1)= 2. By Remark F.4, T1=c(λ)(LgDΘ/ϵsw)2≤c(λ)(2Rd⋅2⋅2NQN2)2=32c(λ)R2dN2QN4T_1=c(λ)(L_gD_ / _sw)^2≤ c(λ) (2R d· 2· 2NQ_N^2 )^2=32\,c(λ)R^2dN^2Q_N^4, and it suffices to substitute this into Proposition F.6(2). ∎ Remark F.13 (Which term dominates). Since QN4=((d+1)max(2NR,1)2)2(d+1)Q_N^4= ((d+1) (2NR,1)^2 )^2(d+1), the first term of Corollary F.12 is exponential in d. On the other hand, the second term M+|≥all|M+|Y^all_≥| depends neither on δin _in nor on any exponent of R, and is determined only by the number M=|all|M=|Y^all| of tuples of solutions that the oracle can return and the number |≥all||Y^all_≥| of branches. Which term dominates depends on the ratio M/|≥all|=∏n=1N|(sn)|/|≥xn(sn)|M/|Y^all_≥|= _n=1^N|Y(s^n)|/|Y_≥ x^n(s^n)|. If this ratio is at most 32c(λ)R2dN2QN432\,c(λ)R^2dN^2Q_N^4, the first term dominates, and the improvement on the finishing side by Lemma F.5 (|≥all|(M+1)→M+|≥all||Y^all_≥|(M+1)→ M+|Y^all_≥|) does not change the overall order; in this case, the only way to lower the overall order is to improve the lower bound on δin _in. On the other hand, for fixed d, 32c(λ)R2dN2QN432\,c(λ)R^2dN^2Q_N^4 is polynomial in N, whereas the ratio can grow exponentially in N; hence, when N is large compared with d, the second term can dominate, and then the improvement by Lemma F.5 reduces the bound by a factor of up to about |≥all||Y^all_≥|. Given that the lower bound on γ(ℓsub)γ( ^sub) for general ILPs can be exponentially small in d (and that this is essential) (Kitaoka, 2026), improvements using the structure of the forward problem are given for the test-direction-cut inner solver in Sections F.4 and F.5; improving the lower bound on δin _in for Algorithm 5 itself is future work (Remark F.27). Remark F.14 (Improvement by structure is left for future work). Kitaoka (2026) replaced the dependence of the lower bound on γ(ℓsub)γ( ^sub) on the range of the features (which correspond to the feasible solutions in this paper) by structural constants, using structures of the forward problem such as total unimodularity, Graver bases, and M-convexity. A similar improvement is expected for δ(y′)δ(y ), but the minors in Proposition F.9 are those of the cut matrix (gv)v(g_v)_v, and properties such as the total unimodularity of the constraint matrix of the forward problem are not immediately inherited by the cut matrix. This improvement is future work. Restricting attention to the value gap of the branch minimum value msub(y′)m_sub(y ) at 00 (a positive lower bound on msub(y′)m_sub(y ) when msub(y′)>0m_sub(y )>0), an explicit lower bound via Graver bases (test sets) is obtained in Sections F.4 and F.5, and for a variant of the inner solver restricted to test-direction cuts, gap lower bounds independent of R (polynomial in the dimension in the M♮-convex case) are obtained in Propositions F.20 and F.22. F.4. Explicit lower bounds via test sets (single state) The lower bound of Section F.3 is of order R−2(d+1)R^-2(d+1) in the range R of the features and is exponentially small in the dimension d. In this subsection, using the integer structure of the forward problem (Graver bases), we bound from below explicitly, only in terms of the norm of the test set, the value gap of the branch minimum value msub(y′)m_sub(y ) at 00 instead of the submodel gap δ(y′)δ(y ). In the second half, we define a variant of the inner solver whose cut generation is restricted to test directions, and give a lower bound on its gap independent of R (a lower bound polynomial in the dimension in the M♮-convex case). In this subsection, let N=1N=1 (the extension to N≥2N≥ 2 is given in Section F.5) and Θ=Δd−1 = ^d-1, and suppose that the feasible region of the forward problem is given in the form (s1)=x∈ℤd|Ax≤b, 0≤x≤u(A∈ℤnA×d,b∈ℤnA,u∈ℤ≥0d)X(s^1)= \x ^d\ |\ Ax≤ b,\ 0≤ x≤ u \ (A ^n_A× d,\ b ^n_A,\ u _≥ 0^d) (a special case of Assumption 4.2(2)). As in Proposition F.9, let R:=maxx∈(s1)‖x‖∞R:= _x (s^1)\|x\|_∞. Introducing slack variables and setting A~:=[A∣InA]∈ℤnA×(d+nA) A:=[A I_n_A] ^n_A×(d+n_A) and z(x):=(x,b−Ax)∈ℤd+nAz(x):=(x,\,b-Ax) ^d+n_A, x∈(s1)x (s^1) is equivalent to A~z=b Az=b and 0≤z≤u~0≤ z≤ u (u~ u is a finite upper bound determined by the box 0≤x≤u0≤ x≤ u). Let :=Gr(A~)T:=Gr( A) be the Graver basis of A~ A (the set of all nonzero elements of kerℤA~ _Z A that are minimal with respect to the conformal order (for t,w∈ℤd+nAt,w ^d+n_A, t⊑wt w if tlwl≥0t_lw_l≥ 0 and |tl|≤|wl||t_l|≤|w_l| for every component l); Onn, 2010), and let C:=maxt∈‖t‖∞C:= _t \|t\|_∞. We write the x component of t∈t as tx∈ℤdt^x ^d (‖tx‖∞≤C\|t^x\|_∞≤ C). We further set CR:=min(C,R)C_R:= (C,R). Since (s1)⊆[0,R]dX(s^1) [0,R]^d, every test direction t such that both y′y and y′+txy +t^x belong to (s1)X(s^1) satisfies ‖tx‖∞≤CR\|t^x\|_∞≤ C_R. The Graver basis is a test set for linear optimization over (s1)X(s^1); that is, for any θ∈ℝdθ ^d and y∈(s1)y (s^1), y is a maximizer of ⟨θ,⋅⟩ θ,· if and only if there exists no t∈t with 0≤z(y)+t≤u~0≤ z(y)+t≤ u and ⟨θ,tx⟩>0 θ,t^x >0 (Onn, 2010). For a branch y′∈≥ally ^all_≥ (since N=1N=1, y′∈≥x1(s1)⊆(s1)y _≥ x^1(s^1) (s^1)), we define the test-set model by (F.4) h(θ):=max(0,max⟨θ,tx⟩|t∈,z(y′)+tis feasible)h_T(θ):= (0,\ \ θ,t^x \ |\ t ,\ z(y )+t\ is feasible \ ) (feasible means satisfying the equality constraints of A~ A and the box constraints; if there is no such t, we read the inner max as −∞-∞, so that h=0h_T=0). For the sandwich bound, we use the following property of Graver bases. Here, let D:=d+nAD:=d+n_A, and for t,w∈ℤDt,w ^D, t⊑wt w means that tlwl≥0t_lw_l≥ 0 and |tl|≤|wl||t_l|≤|w_l| for every component l (conformality, i.e., sign compatibility). Proposition F.15 (Conformal decomposition by the Graver basis; Onn, 2010). For an integer matrix A~ A and w∈kerℤA~∖0w∈ _Z A \0\, there exist positive integers α1,…,αr _1,…, _r and t1,…,tr∈Gr(A~)t_1,…,t_r ( A) such that w=∑i=1rαitiw= _i=1^r _it_i and ti⊑wt_i w for each i. Lemma F.16 (Sandwich by the test-set model). Let Dz:=max(1,maxx,x′∈(s1)‖z(x′)−z(x)‖1).D_z:= (1,\ _x,x (s^1)\|z(x )-z(x)\|_1 ). For every θ∈ℝdθ ^d, h(θ)≤ℓ¯sub(θ,y′)≤Dzh(θ).h_T(θ)\ ≤\ ^sub(θ;y )\ ≤\ D_z\,h_T(θ). Proof. Lower side: if z(y′)+tz(y )+t is feasible, then its x component y′+txy +t^x belongs to (s1)X(s^1), so ℓ¯sub(θ,y′)=maxv∈(s1)⟨θ,v−y′⟩≥⟨θ,tx⟩ ^sub(θ;y )= _v (s^1) θ,v-y ≥ θ,t^x . Moreover, ℓ¯sub(θ,y′)≥0 ^sub(θ;y )≥ 0 since y′∈(s1)y (s^1). Together, ℓ¯sub(θ,y′)≥h(θ) ^sub(θ;y )≥ h_T(θ). Upper side: let v∈(s1)v (s^1) be a maximizer of ⟨θ,⋅−y′⟩ θ,·-y . Since z(v)−z(y′)∈kerℤA~z(v)-z(y )∈ _Z A, if z(v)≠z(y′)z(v)≠ z(y ), then by Proposition F.15 we can write z(v)−z(y′)=∑iαiti,αi∈ℤ>0,ti∈,ti⊑z(v)−z(y′)z(v)-z(y )= _i _it_i, _i _>0,\ t_i ,\ t_i z(v)-z(y ) (⊑ denotes componentwise conformality). By conformality, z(y′)+tiz(y )+t_i lies componentwise between z(y′)z(y ) and z(v)z(v) and satisfies the box constraints, and since A~(z(y′)+ti)=b A(z(y )+t_i)=b, it is feasible. Hence ⟨θ,tix⟩≤h(θ) θ,t_i^x ≤ h_T(θ) for each i; moreover, by conformality, ∑iαi‖ti‖1=‖z(v)−z(y′)‖1≤Dz _i _i\|t_i\|_1=\|z(v)-z(y )\|_1≤ D_z, and since ‖ti‖1≥1\|t_i\|_1≥ 1 for each i, ∑iαi≤Dz _i _i≤ D_z. Therefore, ℓ¯sub(θ,y′)=⟨θ,∑iαitix⟩=∑iαi⟨θ,tix⟩≤(∑iαi)h(θ)≤Dzh(θ). ^sub(θ;y )= θ, _i _it_i^x = _i _i θ,t_i^x ≤ ( _i _i )\,h_T(θ)≤ D_z\,h_T(θ). ∎ Proposition F.17 (Graver lower bound on the branch value gap). In the above setting, we have msub(y′)∈0∪[1Q,∞),Q:=((d+1)1/2max(CR,1))d+1.m_sub(y )∈\0\∪ [ 1Q_T,\ ∞ ), Q_T:= ((d+1)^1/2 (C_R,1) )^d+1. In particular, since CR≤C_R≤ C, we have Q≤((d+1)1/2max(C,1))d+1Q_T≤((d+1)^1/2 (C,1))^d+1, so this lower bound is bounded independently of R by the norm C of the Graver basis. Moreover, since CR≤RC_R≤ R, Q_T is at most Q of Proposition F.9. For example, if A (and hence A~=[A∣InA] A=[A I_n_A]) is totally unimodular, then C=1C=1 (Onn, 2010) and Q≤(d+1)(d+1)/2Q_T≤(d+1)^(d+1)/2. Proof. If msub(y′)=0m_sub(y )=0, there is nothing to show. Suppose msub(y′)>0m_sub(y )>0. By the upper side of Lemma F.16, for every θ∈Δd−1θ∈ ^d-1, h(θ)≥ℓ¯sub(θ,y′)/Dz≥msub(y′)/Dz>0h_T(θ)≥ ^sub(θ;y )/D_z≥ m_sub(y )/D_z>0, so μ:=minθ∈Δd−1h(θ)>0μ:= _θ∈ ^d-1h_T(θ)>0. μ is the optimal value of the linear program minτ|θ≥0,∑iθi=1,τ≥0,⟨tx,θ⟩≤τ(t∈ such that z(y′)+t is feasible), \τ\ |\ aligned &θ≥ 0,\ _i _i=1,\ τ≥ 0,\\ & t^x,θ ≤τ\ (t such that z(y )+t is feasible) aligned \, and the rows of its coefficient matrix are integer vectors of type (tx,−1)(t^x,-1) (ℓ2 _2 norm ≤dCR2+1≤(d+1)1/2max(CR,1)≤ dC_R^2+1≤(d+1)^1/2 (C_R,1)), (,−1)(0,-1), (,0)(1,0), or (−ei,0)(-e_i,0). By the same extreme-point, Cramer, and Hadamard argument as in the proof of Proposition F.9, μ is a rational number with denominator at most Q_T. Since μ>0μ>0, μ≥1/Qμ≥ 1/Q_T, and by the lower side of Lemma F.16, msub(y′)=minθ∈Δd−1ℓ¯sub(θ,y′)≥μ≥1Q.m_sub(y )= _θ∈ ^d-1 ^sub(θ;y )\ ≥\ μ\ ≥\ 1Q_T. Note that DzD_z is used only to derive μ>0μ>0 and does not appear in the value of the lower bound itself. ∎ The decomposition into test directions in Lemma F.16 and the estimates via extreme points, Cramer’s rule, and Hadamard’s inequality in Propositions F.9 and F.17 are similar to the arguments used for the lower bound on γ(ℓsub)γ( ^sub) in Kitaoka (2026, Lemma 6.9, Theorem 6.24); in this section, we apply them to δ(y′)δ(y ), msub(y′)m_sub(y ), and μ(y′)μ(y ) (defined after Equation F.5). We call a branch y′y with msub(y′)=0m_sub(y )=0 a zero branch. Corollary F.18 (Exact decision of zero branches). In the above setting, if ℓ¯sub(θ¯,y′)<1/Q ^sub( θ;y )<1/Q_T is observed for some θ¯∈Δd−1 θ∈ ^d-1, then msub(y′)=0m_sub(y )=0. In particular, msub(y′)=0m_sub(y )=0 is established as soon as UB<1/QUB<1/Q_T during the execution of Algorithm 5. Furthermore, for N=1N=1, by Equation 5.3, minθ∈Θℓ¯OS(θ)=miny′∈≥allmsub(y′) _θ∈ ^OS(θ)= _y ^all_≥m_sub(y ), so this gives a decision of the realizable case (global minimum value 00) with the explicit tolerance 1/Q1/Q_T (for N≥2N≥ 2, see Corollary F.31). Proof. This follows from msub(y′)≤ℓ¯sub(θ¯,y′)<1/Qm_sub(y )≤ ^sub( θ;y )<1/Q_T and Proposition F.17. The same holds for UBUB, since it is the minimum of the values of ℓ¯sub(⋅,y′) ^sub(·\,;y ) at the evaluation points. ∎ Next, we define a variant of the inner solver whose cut generation is restricted to test directions (Definition F.19), and give a lower bound on its gap independent of R (Proposition F.20), a lower bound polynomial in the dimension in the M-convex and M♮-convex cases (Proposition F.22), and the connection to the decision of zero branches (Corollary F.26). Definition F.19 (Test-direction-cut inner solver). Let y′:=tx∣t∈,z(y′)+tis feasibleT_y :=\t^x t ,\ z(y )+t\ is feasible\. A test-direction oracle is an oracle that, given θ∈ℝdθ ^d, returns the value maxg∈y′⟨θ,g⟩ _g _y θ,g and a cut g∈y′g _y attaining the maximum (a maximizing cut) (its implementability depends on the structure: in the M-convex case it reduces to feasibility checks of at most d(d−1)d(d-1) directions, and of at most d(d+1)d(d+1) directions in the M♮-convex case, whereas for a general Graver basis it can be implemented by computing the Graver basis (Onn, 2010; De Loera et al., 2012) and checking the feasibility of each of its elements). The test-direction-cut inner solver is Algorithm 5 with the objective function replaced by h_T (Equation F.4), cut generation replaced from the forward-problem oracle by the test-direction oracle, and the zero cut g=g=0 (corresponding to h≥0h_T≥ 0) always added as an initial cut. h_T is the maximum of the cuts of the finite family y′∪T_y ∪\0\, and the test-direction oracle returns the value and a maximizing cut at each evaluation point, so the proofs of Lemmas F.1, F.2 and F.3 go through verbatim with the family of cuts gv∣v∈all\g_v v ^all\ replaced by y′∪T_y ∪\0\. We define the corresponding submodel gap by (F.5) δ(y′):=μ(y′)−maxmG(y′)|∅≠G⊆y′∪,mG(y′)<μ(y′). _T(y ):=μ(y )- \\,m^T_G(y )\ |\ ≠ G _y ∪\0\,\ m^T_G(y )<μ(y )\, \. Here mG(y′):=minθ∈Δd−1maxg∈G⟨θ,g⟩m^T_G(y ):= _θ∈ ^d-1 _g∈ G θ,g and μ(y′):=minθ∈Δd−1h(θ)μ(y ):= _θ∈ ^d-1h_T(θ), and as in Equation F.3, if there is no such G, we read δ(y′):=+∞ _T(y ):=+∞. Proposition F.20 (Lower bound on the test-direction gap independent of R). In the setting of Proposition F.17, we have δ(y′)≥1Q2. _T(y )\ ≥\ 1Q_T^2. In particular, since Q≤((d+1)1/2max(C,1))d+1Q_T≤((d+1)^1/2 (C,1))^d+1, this lower bound is bounded by a quantity independent of R. Furthermore, when the test-direction-cut inner solver is run with 0≤ϵin<δ(y′)0≤ _in< _T(y ), analogues of Lemmas F.2 and F.3 hold: at stopping with UB−LB≤ϵinUB-LB≤ _in, LB=μ(y′)LB=μ(y ) is identified exactly, and finishing with the selection rule of the cutting-plane method reaches an exact minimizer of h_T in finitely many iterations. Proof. Each mG(y′)m^T_G(y ) is the optimal value of an LP consisting of integer cuts whose components have absolute values at most CRC_R, so by the same extreme-point, Cramer, and Hadamard argument as in the proof of Proposition F.17, it is a rational number with denominator at most Q_T. The difference of two distinct values is at least 1/Q21/Q_T^2, which gives the first half. The second half holds because the proofs of Lemmas F.1, F.2 and F.3 use only “the family of cuts is finite” and “the oracle returns a maximizing cut at each evaluation point”. ∎ Next, we consider the case where the feasible region of the forward problem is an M-convex set. A set ℳ⊆ℤdM ^d is an M-convex set if ℳM is nonempty and, for any x,y∈ℳx,y and any i with xi>yix_i>y_i, there exists j with xj<yjx_j<y_j such that x−ei+ej∈ℳx-e_i+e_j and y+ei−ej∈ℳy+e_i-e_j (exchange axiom). A set ℳ⊆ℤdM ^d is an M♮-convex set if ℳM is nonempty and, for any x,y∈ℳx,y and any i with xi>yix_i>y_i, either x−ei∈ℳx-e_i and y+ei∈ℳy+e_i , or there exists j with xj<yjx_j<y_j such that x−ei+ej∈ℳx-e_i+e_j and y+ei−ej∈ℳy+e_i-e_j (Murota, 2003). Every M-convex set is M♮-convex. For linear optimization over M-convex sets, the following optimality criterion holds. Proposition F.21 (Optimality criterion for linear optimization over M-convex sets; Murota, 2003). For a bounded M-convex set ℳ⊂ℤdM ^d, θ∈ℝdθ ^d, and y∈ℳy , y is a maximizer of ⟨θ,⋅⟩ θ,· over ℳM if and only if ⟨θ,ej−ei⟩≤0 θ,e_j-e_i ≤ 0 for all i≠ji≠ j with y+ej−ei∈ℳy+e_j-e_i . Proposition F.22 (Lower bound polynomial in the dimension for the M-convex and M♮-convex cases). Consider the case where (s1)⊂ℤdX(s^1) ^d is a bounded M-convex set, let M:=ei−ej∣i≠jT_M:=\e_i-e_j i≠ j\ and M,y′:=g∈M∣y′+g∈(s1)T_M,y :=\g _M y +g (s^1)\, and replace y′T_y in Equation F.4 and Equation F.5 by M,y′T_M,y . Then: (a) MT_M is a test set for linear optimization over (s1)X(s^1). That is, for every θ∈ℝdθ ^d and y∈(s1)y (s^1), y is a maximizer of ⟨θ,⋅⟩ θ,· if and only if there is no g∈Mg _M with y+g∈(s1)y+g (s^1) and ⟨θ,g⟩>0 θ,g >0. (b) For every nonempty G⊆M,y′∪G _M,y ∪\0\, mG(y′)m^T_G(y ) is a rational number with denominator at most d2d^2; hence we have δM(y′)≥1d4, _T_M(y )\ ≥\ 1d^4, and we have μ(y′)>0⟹μ(y′)≥1d2.μ(y )>0\ \ μ(y )\ ≥\ 1d^2. For M♮-convex sets, the same lower bounds hold with =ei−ej∪±eiT=\e_i-e_j\∪\± e_i\ (that T is a test set follows from the optimality criterion for linear optimization over M♮-convex sets; Murota, 2003). Proof. (a) is Proposition F.21 applied to ℳ=(s1)M=X(s^1). We show (b). As in the proof of Proposition F.9, an extreme point of the LP of mG(y′)m^T_G(y ) is determined by a nonsingular integer matrix A consisting of (d+1)(d+1) linearly independent tight constraints, and each row of A is of type (ei−ej,−1)(e_i-e_j,\,-1), (,−1)(0,\,-1), (, 0)(1,\,0), or (−ek, 0)(-e_k,\,0). Since the equality constraint ∑iθi=1 _i _i=1 is tight at every feasible point, we can choose these d+1d+1 tight constraints so that they include the row (, 0)(1,\,0) (extend (,0)\(1,0)\ to a basis of the row space of the tight constraints). By cofactor expansion along the τ column, |detA|| A| is bounded by the sum, over “the rows having a nonzero entry in the τ column (at most d rows, since the τ entry of (,0)(1,0) is 00)”, of the absolute values of the determinants of the d×d× d submatrices M′M formed by the θ parts of the remaining d rows. Each M′M contains exactly one all-ones row, and its other rows are of type ei−eje_i-e_j, of type −ek-e_k, or zero rows. If it contains a zero row, detM′=0 M =0. Otherwise, cofactor expansion along the all-ones row bounds |detM′|| M | by the sum of d determinants of (d−1)×(d−1)(d-1)×(d-1) matrices in which each row has at most one +1+1 and at most one −1-1; their transposes are submatrices of the incidence matrix of a directed graph and are totally unimodular, so |detM′|≤d| M |≤ d. Hence 1≤|detA|≤d21≤| A|≤ d^2, and mG(y′)m^T_G(y ) is a rational number with denominator at most d2d^2. The difference of two distinct values is at least 1/d41/d^4, and positive values are at least 1/d21/d^2. Adding rows of type ±ei± e_i does not change the structure “each row has at most one +1+1 and at most one −1-1”, so the same bound holds in the M♮-convex case. ∎ Remark F.23 (Branch minima in the M-convex case). Let Θ=Δd−1 = ^d-1 and let (s1)X(s^1) be an M-convex set. Then msub(y′)=μ(y′)=0m_sub(y )=μ(y )=0 for every branch y′y . Indeed, by the exchange axiom, an M-convex set lies on a hyperplane x∣∑ixi=const\x _ix_i=const\, so for the uniform weight θ¯:=(1/d,…,1/d)∈Δd−1 θ:=(1/d,…,1/d)∈ ^d-1, we have ⟨θ¯,v−y′⟩=0 θ,v-y =0 for every v∈(s1)v (s^1) and ⟨θ¯,ei−ej⟩=0 θ,e_i-e_j =0 for every i≠ji≠ j. Hence ℓ¯sub(θ¯,y′)=h(θ¯)=0 ^sub( θ;y )=h_T( θ)=0, and together with nonnegativity, msub(y′)=μ(y′)=0m_sub(y )=μ(y )=0. Therefore, in the M-convex case, no branch satisfies the hypothesis μ(y′)>0μ(y )>0 of the latter half of Proposition F.22(b), and Corollary F.26 decides that every branch is a zero branch. The latter half of Proposition F.22(b) and Corollary F.26 are used in the case of M♮-convex sets that are not M-convex to identify branches with a positive branch minimum. Proposition F.24 (Exact minimization of the OS loss in the M♮-convex case). Let Θ=Δd−1 = ^d-1, and for each n, let (sn)⊂ℤdX(s^n) ^d be a bounded M♮-convex set (including M-convex sets) and xn∈(sn)x^n (s^n). Then the uniform weight θ¯:=(1/d,…,1/d)∈Δd−1 θ:=(1/d,…,1/d)∈ ^d-1 satisfies ℓOS(θ¯,xn,sn)=0 ^OS( θ;x^n,s^n)=0 for every n. Hence θ¯ θ is a global minimizer of ℓ¯OS ^OS over Θ . Proof. Fix n and let :=(sn)X:=X(s^n). The set =ei−ej∣i≠j∪±eiT=\e_i-e_j i≠ j\∪\± e_i\ is a test set for linear optimization over M♮-convex sets (Proposition F.22; Murota, 2003). Since ⟨θ¯,ei−ej⟩=0 θ,e_i-e_j =0 and ⟨θ¯,−ei⟩<0 θ,-e_i <0, if x∈x is not a maximizer of ⟨θ¯,⋅⟩ θ,· , then x+ei∈x+e_i for some i. Let z0:=xnz^0:=x^n, and as long as zkz^k is not a maximizer, update zk+1:=zk+ei∈z^k+1:=z^k+e_i . Since ⟨θ¯,zk⟩ θ,z^k strictly increases and X is finite, this update reaches a maximizer z∈argmaxx∈⟨θ¯,x⟩z∈ _x θ,x after finitely many steps. Each update only increases a component, so z≥xnz≥ x^n, that is, z∈≥xn(sn)z _≥ x^n(s^n). Hence maxx∈≥xn(sn)⟨θ¯,x⟩=maxx∈⟨θ¯,x⟩ _x _≥ x^n(s^n) θ,x = _x θ,x , and ℓOS(θ¯,xn,sn)=0 ^OS( θ;x^n,s^n)=0 by Equation 4.2. Since ℓ¯OS≥0 ^OS≥ 0, θ¯ θ is a global minimizer. ∎ Proposition F.24 gives a global minimizer of the OS loss. Attaining Equation 3.2 additionally requires that the optimal solution be unique or that the oracle return an optimal solution dominating the action (Section 4). Proposition F.25 (Number of iterations of one call of the test-direction-cut inner solver). In the setting of Proposition F.17 or Proposition F.22, run the test-direction-cut inner solver (Definition F.19) for a branch y′y with tolerance 00 and J=∞J=∞, and assume that the LPs are solved exactly. (i) If the rule for selecting evaluation points is the cutting-plane method, the solver stops within at most |y′|+2|T_y |+2 iterations, and LB=UB=μ(y′)LB=UB=μ(y ) at stopping; the returned point is an exact minimizer over Δd−1 ^d-1 of the test-set model h_T (Equation F.4), with minimum value μ(y′)μ(y ). In particular, in the M♮-convex case, |y′|≤d(d+1)|T_y |≤ d(d+1), so the number is at most d(d+1)+2d(d+1)+2. (i) In the setting of Proposition F.17, let the rule be cutting-plane finishing (Algorithm 4) with the level method as the rule R, ϵsw:=1/(2Q2) _sw:=1/(2Q_T^2), and ϵterm=0 _term=0. If the iteration bound of Remark F.4 holds for the level method in Phase 1, then the number of iterations of Phase 1 is at most c(λ)(CRd⋅2⋅2Q2)2c(λ) (C_R d· 2· 2Q_T^2 )^2, that of Phase 2 is at most |y′|+1|T_y |+1, and the returned point is an exact minimizer of h_T over Δd−1 ^d-1, with minimum value μ(y′)μ(y ). Proof. Let jG_j be the cut set at the end of iteration j; the zero cut 0 always belongs to jG_j. Let gjg_j be the maximizing cut returned by the test-direction oracle at the evaluation point θjθ^j; then h(θj)=max⟨θj,gj⟩,0h_T(θ^j)= \ θ^j,g_j ,0\. We first show the following: if the evaluation point θjθ^j of iteration j≥1j≥ 1 is a minimizer over Δd−1 ^d-1 of the model maxg∈j−1⟨θ,g⟩ _g _j-1 θ,g and gj∈j−1g_j _j-1, then the solver stops at iteration j. Indeed, then j=j−1G_j=G_j-1, so LB=minθ∈Δd−1maxg∈j−1⟨θ,g⟩=maxg∈j−1⟨θj,g⟩≥h(θj)≥UBLB= _θ∈ ^d-1 _g _j-1 θ,g = _g _j-1 θ^j,g ≥ h_T(θ^j) , and the stopping test UB−LB≤0UB-LB≤ 0 holds. (i) With the cutting-plane method, the evaluation point of each iteration j≥1j≥ 1 is a minimizer of the model, so by the claim above, if the solver does not stop at iteration j≥1j≥ 1, then gj∈y′∖j−1g_j _y _j-1, and a new element of y′T_y is added to G. Since y′T_y is finite, there are at most |y′||T_y | non-stopping iterations j≥1j≥ 1. Adding iteration j=0j=0 and the stopping iteration, the number of iterations is at most |y′|+2|T_y |+2. Since the model is at most h_T, LB≤μ(y′)LB≤μ(y ), and since UBUB is the minimum of the values of h_T at the evaluation points, μ(y′)≤UBμ(y ) . At stopping, UB≤LBUB , so LB=UB=μ(y′)LB=UB=μ(y ), and the returned point θ θ satisfies h(θ^)=UB=μ(y′)h_T( θ)=UB=μ(y ). In the M♮-convex case, y′⊆ei−ej∣i≠j∪±eiT_y \e_i-e_j i≠ j\∪\± e_i\, whose cardinality is d(d−1)+2d=d(d+1)d(d-1)+2d=d(d+1). (i) Each element g of y′∪T_y ∪\0\ satisfies ‖g‖2≤CRd\|g\|_2≤C_R d, so h_T is CRdC_R d-Lipschitz. Since diam(Δd−1)=2 ( ^d-1)= 2, the assumed bound of Remark F.4 implies that the number of iterations of Phase 1 is at most c(λ)(CRd⋅2/ϵsw)2=c(λ)(CRd⋅2⋅2Q2)2c(λ) (C_R d· 2/ _sw )^2=c(λ) (C_R d· 2· 2Q_T^2 )^2. By Proposition F.20, ϵsw<1/Q2≤δ(y′) _sw<1/Q_T^2≤ _T(y ); hence, applying the proof of Lemma F.2 with ϵin _in, δ(y′)δ(y ), and msub(y′)m_sub(y ) replaced by ϵsw _sw, δ(y′) _T(y ), and μ(y′)μ(y ), we have LB=μ(y′)LB=μ(y ) at the end of Phase 1. The evaluation point of each iteration of Phase 2 is a minimizer of the model, so by the claim above, each non-stopping iteration of Phase 2 adds a new element of y′T_y to G. Hence there are at most |y′||T_y | non-stopping iterations in Phase 2, and adding the stopping iteration, at most |y′|+1|T_y |+1. At stopping, UB=μ(y′)UB=μ(y ) as in (i), so the returned point is an exact minimizer of h_T. ∎ Corollary F.26 (Connection to the decision of zero branches). In the setting of Proposition F.17 or Proposition F.22, we have μ(y′)=0⟺msub(y′)=0.μ(y )=0 m_sub(y )=0. Therefore, the test-direction-cut inner solver decides exactly whether a branch is a zero branch through the exact computation of μ(y′)μ(y ). In particular, in the M♮-convex case, the size of the family of cuts is at most d(d+1)+1d(d+1)+1, so the decision is completed with at most d(d+1)+2d(d+1)+2 calls to the test-direction oracle using only the selection rule of the cutting-plane method (Proposition F.25(i)). Moreover, when μ(y′)>0μ(y )>0, msub(y′)≥μ(y′)m_sub(y )≥μ(y ), and in the Graver-basis case, msub(y′)≤Dzμ(y′)m_sub(y )≤ D_z\,μ(y ) (Lemma F.16) also holds. Proof. Suppose μ(y′)=0μ(y )=0. Then h(θ¯)=0h_T( θ)=0 for some θ¯∈Δd−1 θ∈ ^d-1, that is, no feasible test direction from y′y has ⟨θ¯,⋅⟩>0 θ,· >0. In the Graver-basis case, by the upper side of Lemma F.16, ℓ¯sub(θ¯,y′)≤Dzh(θ¯)=0 ^sub( θ;y )≤ D_z\,h_T( θ)=0; in the M-convex case, by Proposition F.22(a), and in the M♮-convex case, by the test-set property for M♮-convex sets stated at the end of that proposition, y′y is a maximizer with respect to θ¯ θ, so ℓ¯sub(θ¯,y′)=0 ^sub( θ;y )=0. Hence msub(y′)=0m_sub(y )=0 (together with nonnegativity). Conversely, suppose msub(y′)=0m_sub(y )=0. Then ℓ¯sub(θ∗,y′)=0 ^sub(θ ;y )=0 for some θ∗θ , and for every feasible test direction t, y′+tx∈(s1)y +t^x (s^1), so ⟨θ∗,tx⟩≤ℓ¯sub(θ∗,y′)=0 θ ,t^x ≤ ^sub(θ ;y )=0; hence h(θ∗)=0h_T(θ )=0 and μ(y′)=0μ(y )=0. The inequality msub(y′)≥μ(y′)m_sub(y )≥μ(y ) follows from ℓ¯sub(⋅,y′)≥h ^sub(·\,;y )≥ h_T (the lower side of Lemma F.16; it uses only the feasibility of single steps, so it also holds in the M-convex case). The bound on the size of the family of cuts and the finite stopping follow from Proposition F.25(i). ∎ Remark F.27 (Scope). First, what Propositions F.20 and F.22 improve is δ(y′) _T(y ) of the test-direction-cut inner solver, not δ(y′)δ(y ) of Algorithm 5 itself, which uses vertex cuts. The denominators of δ(y′)δ(y ) are determined by the cuts v−y′v-y actually generated (with components of magnitude up to 2R2R), and the tie values of two cuts with slopes of order R can approach each other to within order 1/R21/R^2; we therefore conjecture that δ(y′)δ(y ) itself admits no lower bound independent of R. Its proof is future work. Second, what the test-direction-cut inner solver minimizes is h_T, not ℓ¯sub(⋅,y′) ^sub(·\,;y ), so it cannot be used directly for the premise of Proposition E.4 (exact solution of the convex subproblems). Its uses are the exact decision of zero branches (Corollary F.26; for N≥2N≥ 2, Corollaries F.31 and F.32) and the DzD_z-approximation of msub(y′)m_sub(y ) in the Graver-basis case. Remark F.28 (What can be said in the M-convex and Graver-basis cases). We discuss whether Proposition F.6 can be specialized to the structured cases, in contrast with Corollary F.12. The conclusion is that the lower bounds of this subsection give a bound on the number of iterations of one call (per branch) of the test-direction-cut inner solver (Proposition F.25). The reason why the lower bounds of this subsection are for the test-direction-cut inner solver is as described in Remark F.27. Proposition F.6 requires a lower bound on δin=miny′δ(y′) _in= _y δ(y ), whereas Propositions F.20 and F.22 give lower bounds on δ(y′) _T(y ) of the test-direction-cut inner solver. The former is determined by the vertex cuts v−y′v-y (with components of magnitude up to 2R2R), and the latter by the test directions txt^x (with components of magnitude up to CRC_R, and 11 in the M♮-convex case). If the conjecture of Remark F.27 is true, δ(y′)δ(y ) admits no lower bound independent of R, and the dependence on R in Corollary F.12 is essential. The number of iterations of one call (per branch) of the test-direction-cut inner solver is given in Proposition F.25. In the Graver-basis case, the number of iterations of Phase 1 is bounded by c(λ)(CRd⋅2⋅2Q2)2c(λ) (C_R d· 2· 2Q_T^2 )^2 (part (i) of that proposition). Since Q≤((d+1)1/2max(C,1))d+1Q_T≤ ((d+1)^1/2 (C,1) )^d+1, this number has an upper bound independent of R. It is exponential in d. The M♮-convex case is even stronger: the size of the family of cuts is at most d(d+1)+1d(d+1)+1, so the solver stops with at most d(d+1)+2d(d+1)+2 calls to the test-direction oracle using only the selection rule of the cutting-plane method, without going through the level method (Proposition F.25(i)). That is, O(d2)O(d^2) calls per branch, polynomial in the dimension and independent of both R and δ. Therefore, the question of whether the number over the whole run can be made polynomial in the dimension reduces to whether the exact computation of μ(y′)μ(y ) in Corollary F.26 can be lifted to the exact computation of msub(y′)m_sub(y ) (the premise of Proposition E.4). In the M-convex case, the uniform weight θ¯ θ gives a minimizer of ℓ¯sub(⋅,y′) ^sub(·\,;y ) (with minimum value 00) for every branch (Remark F.23), so msub(y′)m_sub(y ) is computed directly. In the M♮-convex case as well, the global minimization of the OS loss itself is attained by the uniform weight (Proposition F.24). What remains is, as a subproblem of DC optimization, recovering the minimization of ℓ¯sub(⋅,y′) ^sub(·\,;y ) at each branch from the minimization of h_T in the M♮-convex case, which is, as in Remark F.14, future work. F.5. Explicit lower bounds via test sets (multiple states) In this subsection, we extend the results of Section F.4 to N≥1N≥ 1. Let Θ=Δd−1 = ^d-1, and suppose that the constraint matrix A∈ℤnA×dA ^n_A× d is common to all states, while the right-hand side and the upper bounds of the box may depend on the state: (sn)=x∈ℤd|Ax≤bn, 0≤x≤un(bn∈ℤnA,un∈ℤ≥0d,n=1,…,N).X(s^n)= \x ^d\ |\ Ax≤ b^n,\ 0≤ x≤ u^n \ (b^n ^n_A,\ u^n _≥ 0^d,\ n=1,…,N). Setting zn(x):=(x,bn−Ax)∈ℤd+nAz_n(x):=(x,\,b^n-Ax) ^d+n_A, x∈(sn)x (s^n) is equivalent to A~zn(x)=bn Az_n(x)=b^n and 0≤zn(x)≤u~n0≤ z_n(x)≤ u^n (u~n u^n is the finite upper bound determined by the box 0≤x≤un0≤ x≤ u^n). The Graver basis =Gr(A~)T=Gr( A) and C=maxt∈‖t‖∞C= _t \|t\|_∞ are determined by A~ A alone and are therefore common to all states. Setting R:=maxnmaxx∈(sn)‖x‖∞R:= _n _x (s^n)\|x\|_∞ and CR:=min(C,R)C_R:= (C,R), every test direction t feasible at a state satisfies ‖tx‖∞≤CR\|t^x\|_∞≤ C_R. For a branch y′=(y′n)n=1Ny =(y n)_n=1^N (y′n∈(sn)y n (s^n)), let y′,n:=tx∣t∈, 0≤zn(y′n)+t≤u~n,h,n(θ):=max(0,maxg∈y′,n⟨θ,g⟩),T_y ,n:=\t^x t ,\ 0≤ z_n(y n)+t≤ u^n\, h_T,n(θ):= (0,\ _g _y ,n θ,g ), hN(θ):=1N∑n=1Nh,n(θ),μN(y′):=minθ∈Δd−1hN(θ)h^N_T(θ):= 1N _n=1^Nh_T,n(θ), _N(y ):= _θ∈ ^d-1h^N_T(θ) (since t∈kerℤA~t∈ _Z A, whether zn(y′n)+tz_n(y n)+t is feasible at state n is determined by the box constraints). We also set Dz,n:=max(1,maxx,x′∈(sn)‖zn(x′)−zn(x)‖1)D_z,n:= (1,\ _x,x (s^n)\|z_n(x )-z_n(x)\|_1 ) and DzN:=maxnDz,nD^N_z:= _nD_z,n. Lemma F.29 (Sandwich by the test-set model with multiple states). In the above setting, for any θ∈ℝdθ ^d, hN(θ)≤ℓ¯sub(θ,y′)≤DzNhN(θ).h^N_T(θ)\ ≤\ ^sub(θ;y )\ ≤\ D^N_z\,h^N_T(θ). Proof. We have ℓ¯sub(θ,y′)=1N∑n=1Nℓsub(θ,y′n,sn) ^sub(θ;y )= 1N _n=1^N ^sub(θ;y n,s^n). The proof of Lemma F.16 uses only the feasible region x∈ℤd∣Ax≤b, 0≤x≤u\x ^d Ax≤ b,\ 0≤ x≤ u\ of a single state and y′∈(s1)y (s^1); applying it to each n with (b,u)(b,u) replaced by (bn,un)(b^n,u^n) gives h,n(θ)≤ℓsub(θ,y′n,sn)≤Dz,nh,n(θ)h_T,n(θ)≤ ^sub(θ;y n,s^n)≤ D_z,n\,h_T,n(θ). Averaging over n with Dz,n≤DzND_z,n≤ D^N_z and h,n≥0h_T,n≥ 0 yields the claim. ∎ Proposition F.30 (Graver lower bound on the branch value gap with multiple states). In the above setting, let Q,N:=(1+dN)N(d1/2max(CR,1))d≤(ed1/2max(C,1))d.Q_T,N:= (1+ dN )^N (d^1/2 (C_R,1) )^d\ ≤\ (e\,d^1/2 (C,1) )^d. Then NμN(y′)∈0∪[1Q,N,∞),msub(y′)∈0∪[1NQ,N,∞).N _N(y )∈\0\∪ [ 1Q_T,N,\ ∞ ), m_sub(y )∈\0\∪ [ 1N\,Q_T,N,\ ∞ ). Moreover, μN(y′)=0 _N(y )=0 if and only if msub(y′)=0m_sub(y )=0. In particular, setting Q¯:=(ed1/2max(C,1))d Q_T:=(e\,d^1/2 (C,1))^d, if msub(y′)>0m_sub(y )>0, then msub(y′)≥1/(NQ¯)m_sub(y )≥ 1/(N\, Q_T); this lower bound depends neither on the right-hand sides bnb^n nor on the upper bounds unu^n of the boxes, and depends on N only through the factor 1/N1/N. Proof. NμN(y′)N _N(y ) is the optimal value of the linear program in the variables (θ,τ)∈ℝd×ℝN(θ,τ) ^d×R^N min∑n=1Nτn|θ≥0,∑iθi=1,τn≥0,τn≥⟨g,θ⟩(g∈y′,n,n=1,…,N). \ _n=1^N _n\ |\ θ≥ 0,\ _i _i=1,\ _n≥ 0,\ _n≥ g,θ \ (g _y ,n,\ n=1,…,N) \. The feasible region is nonempty and contains no line (θ is bounded and τ≥0τ≥ 0), and the objective is nonnegative, so the optimal value is attained at an extreme point. An extreme point is determined by a nonsingular integer matrix A′A consisting of d+Nd+N linearly independent tight constraints and an integer right-hand side, so by Cramer’s rule each component of the extreme point, and hence the objective value ∑nτn _n _n, is a rational number with denominator |detA′|| A |. We bound |detA′|| A |. Each row of A′A is of type (g,−en)(g,-e_n) (g∈y′,ng _y ,n), (,−en)(0,-e_n), (,)(1,0), or (−ei,)(-e_i,0), and in the N columns corresponding to τ, each row has at most one nonzero entry (−1-1). Let knk_n be the number of rows with a nonzero entry in the τn _n column; since A′A is nonsingular, kn≥1k_n≥ 1, and ∑nkn≤d+N _nk_n≤ d+N. In the Laplace expansion along the N columns of τ, the choices of N rows for which the τ part is nonsingular are exactly those choosing one row from each τn _n column, and the absolute value of the determinant of that part is 11. There are ∏nkn _nk_n such choices, and by the inequality of arithmetic and geometric means, ∏nkn≤((d+N)/N)N=(1+d/N)N _nk_n≤ ((d+N)/N )^N=(1+d/N)^N. The θ part of the remaining d rows has rows of norm at most d1/2max(CR,1)d^1/2 (C_R,1), since ‖g‖2≤d1/2CR\|g\|_2≤ d^1/2C_R, ‖2=d1/2\|1\|_2=d^1/2, and ‖ei‖2=1\|e_i\|_2=1; hence by Hadamard’s inequality, the absolute value of its determinant is at most (d1/2max(CR,1))d(d^1/2 (C_R,1))^d. Therefore 1≤|detA′|≤Q,N1≤| A |≤ Q_T,N, and NμN(y′)N _N(y ) is a nonnegative rational number with denominator at most Q,NQ_T,N. Hence, if NμN(y′)>0N _N(y )>0, then NμN(y′)≥1/Q,N _N(y )≥ 1/Q_T,N. Also, (1+d/N)N≤ed(1+d/N)^N ^d. By Lemma F.29, μN(y′)≤msub(y′)≤DzNμN(y′) _N(y )≤ m_sub(y )≤ D^N_z\, _N(y ). Hence μN(y′)=0 _N(y )=0 if and only if msub(y′)=0m_sub(y )=0, and if msub(y′)>0m_sub(y )>0, then msub(y′)≥μN(y′)≥1/(NQ,N)m_sub(y )≥ _N(y )≥ 1/(N\,Q_T,N). ∎ Corollary F.31 (Decision of zero branches and realizability with multiple states). In the above setting, if ℓ¯sub(θ¯,y′)<1/(NQ,N) ^sub( θ;y )<1/(N\,Q_T,N) for some θ¯∈Δd−1 θ∈ ^d-1 and a branch y′∈≥ally ^all_≥, then msub(y′)=0m_sub(y )=0. In particular, if ℓ¯OS(θ¯)<1/(NQ,N) ^OS( θ)<1/(N\,Q_T,N) for some θ¯∈Δd−1 θ∈ ^d-1, then minθ∈Θℓ¯OS(θ)=0 _θ∈ ^OS(θ)=0. Proof. The first half follows from Proposition F.30. For the second half, ℓ¯OS(θ¯)=miny′∈≥allℓ¯sub(θ¯,y′) ^OS( θ)= _y ^all_≥ ^sub( θ;y ) by Equation 5.2; applying the first half to a branch y′y attaining this minimum gives msub(y′)=0m_sub(y )=0, and by Equation 5.3, minθ∈Θℓ¯OS(θ)≤msub(y′)=0 _θ∈ ^OS(θ)≤ m_sub(y )=0. Equality holds since ℓ¯OS≥0 ^OS≥ 0. ∎ The test-direction-cut inner solver (Definition F.19) also extends to N≥1N≥ 1. At each evaluation point θ, let the test-direction oracle of each state return a maximizing cut gng_n of maxg∈y′,n∪⟨θ,g⟩ _g _y ,n∪\0\ θ,g , and use 1N∑ngn 1N _ng_n, which we call the aggregated cut, as the cut. For the family of aggregated cuts ℱy′:=1N∑n=1Ngn|gn∈y′,n∪,F_y := \ 1N _n=1^Ng_n\ |\ g_n _y ,n∪\0\ \, we have hN(θ)=maxf∈ℱy′⟨θ,f⟩h^N_T(θ)= _f _y θ,f , and the aggregated cut is a maximizer of this maximum. Corollary F.32 (Number of iterations of one call of the test-direction-cut inner solver with multiple states). In the above setting, run the test-direction-cut inner solver with aggregated cuts, with the zero cut 0 as the initial cut, the cutting-plane method as the rule for selecting evaluation points, tolerance 00, and J=∞J=∞, and assume that the LPs are solved exactly. Then the solver stops within at most |ℱy′|+1≤∏n=1N(|y′,n|+1)+1|F_y |+1≤ _n=1^N (|T_y ,n|+1 )+1 iterations, and the returned point is an exact minimizer over Δd−1 ^d-1 of hNh^N_T, with minimum value μN(y′) _N(y ). If each (sn)X(s^n) is an M♮-convex set, then with the test set ei−ej∣i≠j∪±ei\e_i-e_j i≠ j\∪\± e_i\, |y′,n|≤d(d+1)|T_y ,n|≤ d(d+1), so the number is at most (d(d+1)+1)N+1(d(d+1)+1)^N+1. Proof. The proof of Proposition F.25(i) uses only that the family of cuts is finite, that the oracle returns a maximizing cut of the model function at the evaluation point, and that the zero cut always belongs to the cut set. Replacing the family y′∪T_y ∪\0\ by ℱy′F_y and h_T by hNh^N_T, the same argument holds. Since the zero cut is an element of ℱy′F_y and belongs to the cut set from the beginning, each non-stopping iteration j≥1j≥ 1 adds a new element of ℱy′∖F_y \0\, so there are at most |ℱy′|−1|F_y |-1 of them. Adding iteration j=0j=0 and the stopping iteration, the number of iterations is at most |ℱy′|+1|F_y |+1. Moreover, |ℱy′|≤∏n(|y′,n|+1)|F_y |≤ _n(|T_y ,n|+1), and the bound in the M♮-convex case follows as in Proposition F.25(i). ∎ Appendix G Proofs for the generalization analysis In this section, we prove the results of Section 6. We follow the notation of Section 6. Following the standard treatment in empirical process theory, we assume that the events appearing below (such as unions over all weights consistent with the observations) are measurable. ℓ01(θ,x′,s)=0 ^01(θ;x ,s)=0 holds if and only if the OS loss ℓOS(θ,x′,s) ^OS(θ;x ,s) is 00 and the optimal solution is unique (Lemma G.2). In this case, the oracle returns the unique optimal solution, which dominates x′x ; hence GE(θ)GE(θ) is an upper bound on the probability that, at a new state, the optimal solution returned by the oracle does not dominate the agent’s action, regardless of which optimal solution the oracle returns when there are several. We do not use the OS loss itself because, for a weight whose optimal solution is not unique, the oracle may return an optimal solution that does not dominate the agent’s action even when the OS loss is 00 (Section 4). Under Assumption 6.1(2), θ∗θ^* is a.s. consistent with the observations. In this section, we use the following notation. For a set Z⊆ℝdZ ^d, we write ConvZ Z for its convex hull, and for a convex set C, we write extC C for the set of its extreme points (Section E.1). For a bounded polyhedron P, extP P is the set of vertices of P. A point u of a convex set C is called an exposed point of C if, for some θ∈ℝdθ ^d, u is the unique maximizer of ⟨θ,⋅⟩ θ,· over C. We write intZ Z for the interior of a set Z. A face of a bounded polyhedron P is a subset that can be written as argmaxy∈P⟨θ,y⟩ _y∈ P θ,y for some θ∈ℝdθ ^d. A finite family A of distinct affine hyperplanes H=x∣gH(x)=0H=\x g_H(x)=0\ in ℝkR^k (where gHg_H is a nonconstant affine function) is called a hyperplane arrangement. For a sign pattern τ∈−,+τ∈\-,+\^A, if x∈ℝk∣signgH(x)=τH(∀H∈)\x ^k _H(x)= _H\ (∀ H )\ is nonempty, it is called a region of A. The regions form a partition of ℝk∖⋃H∈HR^k _H H, and their number does not depend on the choice of signs of the gHg_H. G.1. Preliminaries We supplement the setting of Section 6. We assume that x∈(s)x (s) holds almost surely (a.s.) for (s,x)∼(s,x) . The esssupess\,sup in Assumption 6.1(1) is the essential supremum with respect to the marginal distribution of P in s. When S is finite, κ<∞κ<∞ in Assumption 6.1(1) holds automatically, since each (s)Y(s) is finite (Remark 4.3). Lemma G.1. Assume Assumption 4.2(2)(3), and fix s∈s and θ∈ℝdθ ^d. (i): extConv(s)⊆(s)⊆(s) (s) (s) (s), (s)Y(s) is a finite set, and we have maxx∈(s)⟨θ,x⟩=maxu∈(s)⟨θ,u⟩ _x (s) θ,x = _u (s) θ,u . (i): argmaxy∈(s)⟨θ,y⟩ _y (s) θ,y is a singleton if and only if there is exactly one point u in (s)Y(s) that maximizes ⟨θ,⋅⟩ θ,· , and in this case argmaxy∈(s)⟨θ,y⟩=u _y (s) θ,y =\u\. In particular, if argmaxy∈(s)⟨θ,y⟩=v _y (s) θ,y =\v\, then v∈(s)v (s). Proof. (i) By Assumption 4.2(2), we can write (s)=⋃jQjX(s)= _jQ_j (finitely many nonempty bounded polyhedra QjQ_j). Conv(s)=Conv(⋃jextQj) (s)= ( _j Q_j ) is a bounded polyhedron, and its extreme points are contained in the finite set ⋃jextQj⊆(s) _j Q_j (s). Let u∈extConv(s)u∈ (s). Since the extreme points of a polyhedron are exposed points, there exists θ′θ such that u is the unique maximizer over Conv(s) (s), and argmaxy∈(s)⟨θ′,y⟩=u _y (s) θ ,y =\u\. Since the oracle returns an optimal solution, x∗(θ′,s)=ux^*(θ ,s)=u, and thus u∈(s)u (s). (s)⊆(s)Y(s) (s) and the finiteness of (s)Y(s) are as stated in the remark after Assumption 4.2(3). Since the maximum of a linear function over a compact convex set is attained at an extreme point, we have max(s)⟨θ,⋅⟩≥max(s)⟨θ,⋅⟩≥maxextConv(s)⟨θ,⋅⟩=maxConv(s)⟨θ,⋅⟩≥max(s)⟨θ,⋅⟩ _X(s) θ,· ≥ _Y(s) θ,· ≥ _ (s) θ,· = _ (s) θ,· ≥ _X(s) θ,· . (i) Let c∗c^* be the maximum, and set U:=u∈extConv(s)∣⟨θ,u⟩=c∗U:=\u∈ (s) θ,u =c^*\ and U:=u∈(s)∣⟨θ,u⟩=c∗U_Y:=\u (s) θ,u =c^*\. By (i), ∅≠U⊆U⊆argmaxy∈(s)⟨θ,y⟩ ≠ U U_Y _y (s) θ,y . The set F∗F^* of optimal solutions over Conv(s) (s) is a face of the bounded polyhedron Conv(s) (s), and since a face of a bounded polyhedron is the convex hull of the vertices it contains, F∗=ConvUF^*= U; by (i), argmaxy∈(s)⟨θ,y⟩=(s)∩F∗ _y (s) θ,y =X(s)∩ F^*. If |U|=1|U_Y|=1, then |U|=1|U|=1, so F∗=U⊆(s)F^*=U (s) gives argmaxy∈(s)⟨θ,y⟩=U=U _y (s) θ,y =U=U_Y. If |U|≥2|U_Y|≥ 2, then U⊆argmaxy∈(s)⟨θ,y⟩U_Y _y (s) θ,y shows that argmaxy∈(s)⟨θ,y⟩ _y (s) θ,y is not a singleton. ∎ Lemma G.2. Let θ∈Θθ∈ , s∈s , and x′∈(s)x (s). (i): We have ℓ01(θ,x′,s)=0 ^01(θ;x ,s)=0 if and only if ℓOS(θ,x′,s)=0 ^OS(θ;x ,s)=0 and the set argmaxy∈(s)⟨θ,y⟩ _y (s) θ,y is a singleton. (i): If ℓ01(θ,x′,s)=0 ^01(θ;x ,s)=0, then the optimal solution returned by the oracle satisfies x∗(θ,s)≥x′x^*(θ,s)≥ x . (i): Under Assumption 6.1(2), θ∗θ^* is a.s. consistent with the observations. Proof. (i) As stated in Section 4, ℓOS(θ,x′,s)=0 ^OS(θ;x ,s)=0 is equivalent to argmaxy∈(s)⟨θ,y⟩∩≥x′(s)≠∅ _y (s) θ,y _≥ x (s)≠ . When argmaxy∈(s)⟨θ,y⟩=v _y (s) θ,y =\v\, this is equivalent to v≥x′v≥ x . (i) Since v is the only optimal solution, x∗(θ,s)=v≥x′x^*(θ,s)=v≥ x . (i) This follows from the fact that ℓ01(θ∗,x,s)=0 ^01(θ^*;x,s)=0 holds a.s. ∎ G.2. Proof of Theorem 6.3 Fix finitely many points zi=(si,xi)z_i=(s_i,x_i) (i=1,…,mi=1,…,m, |(si)|≤κ|Y(s_i)|≤κ). For each i and distinct u,u′∈(si)u,u (s_i), consider the hyperplane through the origin θ∣⟨θ,u−u′⟩=0\θ θ,u-u =0\; let A be the family of all such hyperplanes (identical hyperplanes are counted once), and let h be their number. Then h≤mκ(κ−1)/2≤mκ2/2h≤ mκ(κ-1)/2≤ mκ^2/2. Fix a normal vector aHa_H of each hyperplane H∈H . For a sign vector σ∈−,0,+σ∈\-,0,+\^A, if θ∈ℝd∣sign⟨θ,aH⟩=σH(∀H∈)\θ ^d θ,a_H = _H\ (∀ H )\ is nonempty, it is called a cell of A (we use this name to distinguish it from a face of a bounded polyhedron). ℝdR^d is partitioned into the cells of A, and a cell whose signs are all nonzero is a region of A. Lemma G.3. Assume Assumption 4.2(2)(3). The map θ↦(ℓ01(θ,xi,si))i=1mθ ( ^01(θ;x_i,s_i))_i=1^m (θ∈Θθ∈ ) is constant on the intersection of each cell of A with Θ . Proof. On a cell, the sign of ⟨θ,u⟩−⟨θ,u′⟩ θ,u - θ,u is constant for all i and u,u′∈(si)u,u (s_i), so the set of points in (si)Y(s_i) that maximize ⟨θ,⋅⟩ θ,· is constant. By Lemma G.1(i), whether argmaxy∈(si)⟨θ,y⟩ _y (s_i) θ,y is a singleton, and its element, are determined by this set alone. Hence ℓ01(θ,xi,si) ^01(θ;x_i,s_i) is constant. ∎ The following two lemmas bound the number of cells. In what follows, let Φk(h):=∑i=0k(hi) _k(h):= _i=0^k hi (with (hi)=0 hi=0 for i>hi>h). Φk(h) _k(h) is nondecreasing in h and satisfies Φk(0)=Φ0(h)=1 _k(0)= _0(h)=1 and Φk(h)=Φk(h−1)+Φk−1(h−1) _k(h)= _k(h-1)+ _k-1(h-1) (k,h≥1k,h≥ 1). Lemma G.4. The number of regions of an arrangement of at most h hyperplanes in ℝkR^k is at most Φk(h) _k(h). Proof. Let rk(h)r_k(h) be the maximum number of regions of an arrangement of at most h hyperplanes in ℝkR^k; we show rk(h)≤Φk(h)r_k(h)≤ _k(h) by induction on k+hk+h. If h=0h=0, the only region is ℝkR^k, and if k=0k=0, ℝ0R^0 is a single point and has no hyperplanes; hence rk(0)=r0(h)=1r_k(0)=r_0(h)=1. Let k,h≥1k,h≥ 1, let A be an arrangement of h hyperplanes in ℝkR^k, let H∈H , and set ′:=∖HA :=A \H\. For a region R′R of ′A with sign pattern τ′τ , if R′∩H=∅R ∩ H= , then gHg_H has no zeros on the convex set R′R , so its sign is constant, and R′R is a region of A. If R′∩H≠∅R ∩ H≠ , then, since R′R is an open convex set, it contains both points with gH>0g_H>0 and points with gH<0g_H<0, and R′∖HR H splits into the two regions R′∩gH>0R ∩\g_H>0\ and R′∩gH<0R ∩\g_H<0\ of A. Since each region of A is contained in exactly one R′R , the number of regions of A equals the number of regions of ′A plus the number of regions of ′A that intersect H. We bound the number of regions of ′A that intersect H. Identify H with ℝk−1R^k-1 via an affine isomorphism. If H′∈′H is parallel to H (H∩H′=∅H∩ H = ), then gH′g_H has constant sign on H; otherwise, H∩H′H∩ H is a hyperplane of H. Let HA_H be the family of all the latter H∩H′H∩ H (duplicates counted once); then HA_H is an arrangement of at most h−1h-1 hyperplanes in H. If R′∩H≠∅R ∩ H≠ , then R′∩H=x∈H∣signgH′(x)=τH′(∀H′∈′)R ∩ H=\x∈ H _H (x)=τ _H \ (∀ H )\ is a region of HA_H (the conditions for parallel H′H hold identically on H). Since distinct R′R are disjoint, the map R′↦R′∩HR R ∩ H is injective. Hence the number of regions of ′A that intersect H is at most rk−1(h−1)r_k-1(h-1). Therefore rk(h)≤rk(h−1)+rk−1(h−1)r_k(h)≤ r_k(h-1)+r_k-1(h-1), and the induction hypothesis together with Φk(h)=Φk(h−1)+Φk−1(h−1) _k(h)= _k(h-1)+ _k-1(h-1) yields rk(h)≤Φk(h)r_k(h)≤ _k(h). ∎ Lemma G.5. The number of cells of an arrangement of h hyperplanes through the origin in ℝdR^d is at most Φd(2h) _d(2h). Proof. Let H∈H be the hyperplanes of the arrangement and aHa_H their normal vectors, and let Σ⊆−,0,+ \-,0,+\^A be the set of sign vectors that can be written as (sign⟨θ,aH⟩)H∈(sign θ,a_H )_H for some θ∈ℝdθ ^d (that is, the sign vectors corresponding to cells); Σ is a finite set. For each σ∈Σσ∈ , choose a point θσθ^σ realizing the sign vector σ. Choose δ>0δ>0 such that |⟨θσ,aH⟩|>δ| θ^σ,a_H |>δ for all (σ,H)(σ,H) with ⟨θσ,aH⟩≠0 θ^σ,a_H ≠ 0. Let δA_δ be the arrangement of the 2h2h affine hyperplanes θ∣⟨θ,aH⟩=δ\θ θ,a_H =δ\ and θ∣⟨θ,aH⟩=−δ\θ θ,a_H =-δ\ (H∈H ) (these are distinct, since the normal vectors of distinct H are not parallel). By the choice of δ, no θσθ^σ lies on any hyperplane of δA_δ, so each θσθ^σ belongs to some region RσR_σ of δA_δ. RσR_σ determines, for each H, the signs of ⟨θ,aH⟩−δ θ,a_H -δ and ⟨θ,aH⟩+δ θ,a_H +δ, and σH _H is determined from them: −- if both are negative, 00 if they have opposite signs, and ++ if both are positive. Hence σ↦Rσ R_σ is injective, and by Lemma G.4, |Σ|≤Φd(2h)| |≤ _d(2h). ∎ Let G(m)G(m) denote the maximum, over m points (si,xi)(s_i,x_i) (|(si)|≤κ|Y(s_i)|≤κ), of the number of error value patterns (ℓ01(θ,xi,si))i=1m∈0,1m( ^01(θ;x_i,s_i))_i=1^m∈\0,1\^m realized by θ∈Θθ∈ (the growth function). From 2h≤mκ(κ−1)≤mκ22h≤ mκ(κ-1)≤ mκ^2, the monotonicity of Φd _d, and Lemmas G.3 and G.5, we obtain (G.1) G(m)≤Φd(mκ2)G(m)≤ _d(mκ^2) Moreover, if n≥dn≥ d, then we have (G.2) Φd(n)≤(nd)d∑i=0d(ni)(dn)i≤(nd)d(1+dn)n≤(end)d. _d(n)≤ ( nd )^d _i=0^d ni ( dn )^i≤ ( nd )^d (1+ dn )^n≤ ( end )^d. Here, the first inequality holds because n/d≥1n/d≥ 1 implies, for each i=0,…,di=0,…,d, (ni)≤(ni)(nd)d−i=(nd)d(ni)(dn)i ni≤ ni ( nd )^d-i= ( nd )^d ni ( dn )^i. The second inequality follows by adding the nonnegative terms for i=d+1,…,ni=d+1,…,n (recall d≤nd≤ n) and applying the binomial theorem ∑i=0n(ni)(dn)i=(1+dn)n _i=0^n ni ( dn )^i= (1+ dn )^n. The last inequality follows from 1+t≤et1+t ^t (t∈ℝt ) with t=d/nt=d/n, which gives (1+dn)n≤ed (1+ dn )^n ^d. Proof of Theorem 6.3. Write zn:=(sn,xn)z^n:=(s^n,x^n) and Eθ:=(s,x)∣ℓ01(θ,x,s)=1E_θ:=\(s,x) ^01(θ;x,s)=1\, and set ε:=2N(dlog22eNκ2d+log22β). := 2N (d _2 2eNκ^2d+ _2 2β ). Define the event ℰE as “there exists θ∈Θθ∈ that is consistent with the observations and satisfies GE(θ)>εGE(θ)> .” If ℰE does not occur, the generalization error of every weight consistent with the observations is at most ε , so it suffices to show ℙ(ℰ)≤βP(E)≤β. Below, we show ℙ(ℰ)≤2ℙ(ℰ′)P(E)≤ 2P(E ) in Step 2 (where ℰ′E is the event defined in Step 2), ℙ(ℰ′)≤G(2N)2−εN/2P(E )≤ G(2N)2^- N/2 in Step 3, and 2G(2N)2−εN/2≤β2G(2N)2^- N/2≤β in Step 4. Step 1 (checking Nε≥8N ≥ 8): Since N≥dN≥ d and κ≥2κ≥ 2, we have Nκ2/d≥4Nκ^2/d≥ 4, so Nε≥2dlog2(8e)>8N ≥ 2d _2(8e)>8. Step 2 (symmetrization): Independently of the observations, prepare an i.i.d. sample z′1,…,z′N∼z 1,…,z N . Define the event ℰ′E as “there exists θ∈Θθ∈ that is consistent with the observations and for which the number of n with z′n∈Eθz n∈ E_θ is at least εN/2 N/2.” Write ℰω′E _ω for the section of ℰ′E obtained by fixing the observations ω:=(z1,…,zN)ω:=(z^1,…,z^N). Since the observations and z′:=(z′1,…,z′N)z :=(z 1,…,z N) are independent, when integrating the indicator of ℰ′E with respect to the product measure in (ω,z′)(ω,z ), we may first integrate in z′z and then in ω (the Fubini–Tonelli theorem), so ℙ(ℰ′)=ω[ℙz′(ℰω′)]P(E )=E_ω[P_z (E _ω)]. Let ω∈ℰω , and take one θωθ^ω that is consistent with the observations and satisfies p:=GE(θω)>εp:=GE(θ^ω)> . The number Y of n with z′n∈Eθωz n∈ E_θ^ω follows the binomial distribution Bin(N,p)Bin(N,p), so by Chebyshev’s inequality and Step 1, ℙz′(Y<Np2)≤ℙz′(|Y−Np|>Np2)≤4Np(1−p)(Np)2≤4Nε≤12.P_z (Y< Np2 ) _z (|Y-Np|> Np2 )≤ 4Np(1-p)(Np)^2≤ 4N ≤ 12. If Y≥Np/2Y≥ Np/2, then Y≥εN/2Y≥ N/2, and θωθ^ω satisfies the condition of ℰω′E _ω; hence ℙz′(ℰω′)≥1/2P_z (E _ω)≥ 1/2. Therefore ℙ(ℰ′)≥ω[ℰ(ω)/2]=ℙ(ℰ)/2P(E ) _ω[1_E(ω)/2]=P(E)/2. Step 3 (swapping): Let w:=(z1,…,zN,z′1,…,z′N)w:=(z^1,…,z^N,z 1,…,z N). For τ∈0,1Nτ∈\0,1\^N, write SwτwSw_τw for the tuple obtained by swapping znz^n and z′nz n for each n with τn=1 _n=1. Since the 2N2N components of w are i.i.d., SwτwSw_τw and w have the same distribution for each τ, and ℙ(Swτw∈ℰ′)=ℙ(w∈ℰ′)=ℙ(ℰ′)P(Sw_τw )=P(w )=P(E ). Averaging this over the 2N2^N choices of τ∈0,1Nτ∈\0,1\^N and interchanging the finite sum and the expectation, we obtain ℙ(ℰ′)=12N∑τ∈0,1Nw[Swτw∈ℰ′]=w[ℙτ(Swτw∈ℰ′)].P(E )= 12^N _τ∈\0,1\^NE_w [1\Sw_τw \ ]=E_w [P_τ(Sw_τw ) ]. Here, for fixed w, ℙτ(Swτw∈ℰ′):=2−N|τ∈0,1N∣Swτw∈ℰ′|P_τ(Sw_τw ):=2^-N|\τ∈\0,1\^N _τw \| is the probability when τ is drawn uniformly. Fix w, and let Ξ(w)⊆0,12N (w) \0,1\^2N be the set of error value patterns realized by θ∈Θθ∈ on the 2N2N points of w. Since each component of w satisfies |(s)|≤κ|Y(s)|≤κ a.s., we have |Ξ(w)|≤G(2N)| (w)|≤ G(2N) a.s. Since the error value pattern of θ on SwτwSw_τw is obtained by swapping the components of its pattern ξ on w according to τ, if Swτw∈ℰ′Sw_τw , then for some ξ∈Ξ(w)ξ∈ (w) the following event ℰξE_ξ occurs: “after swapping, the first N components are all 00, and at least εN/2 N/2 of the last N components are 11.” Let r be the number of components of ξ equal to 11. If r<εN/2r< N/2, then ℰξE_ξ does not occur. If, for some n, both the nnth and the (N+n)(N+n)th components of ξ are 11, then a 11 remains in the first half regardless of the swap, so ℰξE_ξ does not occur. Otherwise, the r components equal to 11 belong to distinct n, and ℰξE_ξ is equivalent to all of them being placed in the second half, so ℙτ(ℰξ)=2−r≤2−εN/2P_τ(E_ξ)=2^-r≤ 2^- N/2. Therefore ℙτ(Swτw∈ℰ′)≤∑ξ∈Ξ(w)ℙτ(ℰξ)≤G(2N)2−εN/2P_τ(Sw_τw )≤ _ξ∈ (w)P_τ(E_ξ)≤ G(2N)2^- N/2, and we obtain ℙ(ℰ′)≤G(2N)2−εN/2P(E )≤ G(2N)2^- N/2. Step 4: By Steps 2 and 3, ℙ(ℰ)≤2G(2N)2−εN/2P(E)≤ 2G(2N)2^- N/2. Since 2Nκ2≥d2Nκ^2≥ d, Equations G.1 and G.2 give G(2N)≤(2eNκ2/d)dG(2N)≤(2eNκ^2/d)^d, and by the definition of ε , 2G(2N)2−εN/2≤2(2eNκ2d)d2−dlog2(2eNκ2/d)−log2(2/β)=β.2G(2N)2^- N/2≤ 2 ( 2eNκ^2d )^d2^-d _2(2eNκ^2/d)- _2(2/β)=β. Hence ℙ(ℰ)≤βP(E)≤β. That is, with probability at least 1−β1-β, ℰE does not occur, and every weight θ^N θ^N consistent with the observations satisfies GE(θ^N)≤εGE( θ^N)≤ . This is the statement of Theorem 6.3. ∎ G.3. Proof of Theorem 6.5 Consider the following instance. Let d0≥2d_0≥ 2 and d:=2d0+1d:=2d_0+1, and write elements of ℝdR^d as (y0,y+,y−)∈ℝ×ℝd0×ℝd0(y_0,y_+,y_-) ×R^d_0×R^d_0. Let the state set be the unit sphere :=d0−1⊂ℝd0S:=S^d_0-1 ^d_0, and let the state distribution P_S be absolutely continuous with respect to the uniform surface measure on d0−1S^d_0-1. Let the feasible region be (s):= 0,(−1,s,−s),(−3,−s,s)X(s):=\\,0,\ (-1,s,-s),\ (-3,-s,s)\,\ and let the agent’s action be x=0x=0. That is, the distribution P of state–action pairs is given by s∼s _S and x=0x=0. Let the weight space be Θ:=Δd−1 := ^d-1 and θ∗:=(1,0,0)∈Θθ^*:=(1,0,0)∈ . For θ=(θ0,p,q)θ=( _0,p,q), setting u:=p−q∈ℝd0u:=p-q ^d_0, we have (G.3) ⟨θ,0⟩=0,⟨θ,(−1,s,−s)⟩=−θ0+⟨s,u⟩,⟨θ,(−3,−s,s)⟩=−3θ0−⟨s,u⟩. θ,0 =0, θ,(-1,s,-s) =- _0+ s,u , θ,(-3,-s,s) =-3 _0- s,u . This is obtained from the instance in the proof of Theorem 3.1 of Fatemi et al. (2026) (ψ(s,a)=(−2|a|+a,as)ψ(s,a)=(-2|a|+a,as), a∈[−1,1]a∈[-1,1], θ=(1,u)θ=(1,u)) by lifting the features corresponding to a∈−1,0,1a∈\-1,0,1\ so that the positive and negative parts of u are placed in separate coordinates. Lemma G.6. This instance satisfies Assumption 4.2 and Assumption 6.1 with κ=3κ=3. Moreover, for θ=(θ0,p,q)∈Θθ=( _0,p,q)∈ and u=p−qu=p-q, ℓ01(θ,0,s)=0 ^01(θ;0,s)=0 is equivalent to “⟨s,u⟩<θ0 s,u < _0 and −⟨s,u⟩<3θ0- s,u <3 _0.” Proof. Since (s)X(s) consists of three points (a union of bounded polyhedra each consisting of a single point), it satisfies Assumption 4.2(2), and every oracle satisfies Assumption 4.2(3). The three points are distinct and are all extreme points of the convex hull, so by Lemma G.1(i), (s)=(s)Y(s)=X(s) and κ=3κ=3. By Equation G.3, the values under θ∗θ^* are 0,−1,−30,-1,-3, so argmaxy∈(s)⟨θ∗,y⟩=0 _y (s) θ^*,y =\0\ and x∗(θ∗,s)=0≥x^*(θ^*,s)=0≥ x. Since the other two points have a negative first component, ≥0(s)=0X_≥ 0(s)=\0\. argmaxy∈(s)⟨θ,y⟩=v _y (s) θ,y =\v\ with v≥0v≥ 0 holds only when v=0v=0 is the unique optimal solution, so Equation G.3 gives the latter equivalence. ∎ The proof uses the following two propositions. The first is Theorem 1 of Campi and Garatti (2008) on the violation probability of scenario programs. Here we state it in the form used by Fatemi et al. (2026) in Theorem A.1 and in the proof of Theorem 3.1. Proposition G.7 (Theorem 1 of Campi and Garatti (2008)). Let k be a positive integer and c∈ℝkc ^k, and suppose that each point ξ of a probability space (Ξ,PΞ)( ,P_ ) is associated with a closed convex set Cξ⊆ℝkC_ξ ^k. Let ξ1,…,ξNξ^1,…,ξ^N be an i.i.d. sample from PΞP_ , and suppose that the scenario program minu∈ℝk⟨c,u⟩s.t.u∈Cξn(n=1,…,N) _u ^k\ c,u .t. u∈ C_ξ^n (n=1,…,N) has a.s. a unique solution u^N u_N. Set V(u):=PΞ(u∉Cξ)V(u):=P_ (u∉ C_ξ). Then, for any ε∈(0,1] ∈(0,1], we have ℙ(V(u^N)>ε)≤∑i=0k−1(Ni)εi(1−ε)N−i.P (V( u_N)> )≤ _i=0^k-1 Ni ^i(1- )^N-i. For each n, the constraint u∈Cξnu∈ C_ξ^n is called a support constraint if the scenario program with only this constraint removed has an optimal solution different from u^N u_N or has no optimal solution. If the number of support constraints is a.s. exactly k (that is, if the problem is fully supported in the sense of Definition A.5 of Fatemi et al. (2026)), then the above inequality holds with equality. The second is the following property of an LP, shown in the proof of Theorem 3.1 of Fatemi et al. (2026). Fix c′∈ℝd0∖0c ^d_0 \0\, and for s1,…,sN∼s^1,…,s^N _S (i.i.d.) consider (G.4) minu∈ℝd0⟨c′,u⟩s.t.⟨sn,u⟩≤1,−⟨sn,u⟩≤3(n=1,…,N). _u ^d_0\ c ,u .t. s^n,u ≤ 1, - s^n,u ≤ 3 (n=1,…,N). Proposition G.8 (Proof of Theorem 3.1 of Fatemi et al. (2026)). Let N≥d0N≥ d_0. Then Equation G.4 has a.s. a unique solution u^N u_N. Moreover, the number of n such that the LP with both inequalities ⟨sn,u⟩≤1 s^n,u ≤ 1 and −⟨sn,u⟩≤3- s^n,u ≤ 3 corresponding to observation n removed has an optimal solution different from u^N u_N or has no optimal solution is a.s. exactly d0d_0. Equation G.4 is the scenario program in Proposition G.7 with k=d0k=d_0, Ξ= =S, PΞ=P_ =P_S, c=c′c=c , and Cs:=u∣⟨s,u⟩≤1,−⟨s,u⟩≤3C_s:=\u s,u ≤ 1,\ - s,u ≤ 3\. Hence, when N≥d0N≥ d_0, Propositions G.7 and G.8 imply that, for V(u):=(⟨s,u⟩>1 or −⟨s,u⟩>3)V(u):=P_S( s,u >1 or - s,u >3) and any ε∈(0,1] ∈(0,1], (G.5) ℙ(V(u^N)>ε)=∑i=0d0−1(Ni)εi(1−ε)N−i.P (V( u_N)> )= _i=0^d_0-1 Ni ^i(1- )^N-i. We assume d0≥2d_0≥ 2 because, when d0=1d_0=1, the distribution on 0=±1S^0=\± 1\ has atoms, identical constraints are duplicated, and the property in Proposition G.8 fails. Note that Equation G.4 is Eq. (6) of Fatemi et al. (2026) written out for the normalization ΘF:=(1,u)∣u∈ℝd0 _F:=\(1,u) u ^d_0\ and Ψ(θ):=⟨(0,c′),θ⟩ (θ):= (0,c ),θ . This Ψ belongs to the class of objective functions :=θ↦f(⟨c,θ⟩+b)∣f is strictly monotone,b∈ℝ,⟨c,θ⟩ is not constant on ΘFJ:=\θ f( c,θ +b) f is strictly monotone,\ b ,\ c,θ is not constant on _F\ targeted by the lower bound in that paper. Moreover, V(u^N)V( u_N) is the set-level mismatch of θ^=(1,u^N) θ=(1, u_N), that is, the probability that the true optimal solution 00 is not contained in the set of optimal solutions of θ θ (d and T in that paper correspond to d0d_0 and N in this section). Proof of Theorem 6.5. Step 1: For η∈(0,1)η∈(0,1), let uη:=(1−η)u^Nu_η:=(1-η) u_N and θη:=(1,uη+,uη−)/(1+‖uη‖1)∈Θθ^η:=(1,u_η^+,u_η^-)/(1+\|u_η\|_1)∈ (where u±u^± are the componentwise positive and negative parts). Setting Zη:=1+‖uη‖1>0Z_η:=1+\|u_η\|_1>0, the first component of θηθ^η is 1/Zη1/Z_η and p−q=uη/Zηp-q=u_η/Z_η, so by Lemma G.6, ℓ01(θη,0,s)=0 ^01(θ^η;0,s)=0 is equivalent to “⟨s,uη⟩<1 s,u_η <1 and −⟨s,uη⟩<3- s,u_η <3.” For each n, the constraints of Equation G.4 give ⟨sn,uη⟩≤1−η<1 s^n,u_η ≤ 1-η<1 and −⟨sn,uη⟩≤3(1−η)<3- s^n,u_η ≤ 3(1-η)<3, so θηθ^η is consistent with the observations. Step 2: By Lemma G.6, GE(θη)=(Λη),Λη:=⟨s,u^N⟩≥11−η or −⟨s,u^N⟩≥31−η.GE(θ^η)=P_S( _η), _η:= \ s, u_N ≥ 11-η or - s, u_N ≥ 31-η \. As η decreases, Λη _η increases monotonically, and ⋃η∈(0,1)Λη=⟨s,u^N⟩>1 or −⟨s,u^N⟩>3 _η∈(0,1) _η=\ s, u_N >1 or - s, u_N >3\. By continuity of measure from below, limη↓0GE(θη)=V(u^N) _η 0GE(θ^η)=V( u_N). Fix ε∈(0,1] ∈(0,1]; if there exists an integer k≥2k≥ 2 with GE(θ1/k)>εGE(θ^1/k)> , let kNk_N be the smallest such integer, and otherwise let kN:=2k_N:=2; set θ^N:=θ1/kN θ^N:=θ^1/k_N. θ^N θ^N is a function of the observations and, by Step 1, is consistent with the observations. Since GE(θ1/k)→V(u^N)GE(θ^1/k)→ V( u_N) also along η=1/kη=1/k (k→∞k→∞), V(u^N)>εV( u_N)> implies GE(θ^N)>εGE( θ^N)> . Hence, by Equation G.5, (G.6) ℙ(GE(θ^N)>ε)≥∑i=0d0−1(Ni)εi(1−ε)N−i.P (GE( θ^N)> )≥ _i=0^d_0-1 Ni ^i(1- )^N-i. Step 3: Let X∼Bin(N,ε)X (N, ); then the right-hand side of Equation G.6 equals ℙ(X≤d0−1)P(X≤ d_0-1). For ε1:=d0/(2N) _1:=d_0/(2N), Markov’s inequality gives ℙ(X≥d0)≤Nε1/d0=1/2P(X≥ d_0)≤ N _1/d_0=1/2, so the right-hand side is at least 1/2≥β1/2≥β. For ε2:=ln(1/β)/(2N)≤1/2 _2:= (1/β)/(2N)≤ 1/2, since 1−x≥e−2x1-x ^-2x for x∈[0,1/2]x∈[0,1/2], the right-hand side is at least (1−ε2)N≥e−2Nε2=β(1- _2)^N ^-2N _2=β. Taking θ^N θ^N of Step 2 for ε:=maxε1,ε2 := \ _1, _2\, Equation G.6 shows that, with probability at least β, GE(θ^N)>ε≥(ε1+ε2)/2=(d0+ln(1/β))/(4N)GE( θ^N)> ≥( _1+ _2)/2=(d_0+ (1/β))/(4N). ∎ Remark G.9. Remove the nonnegativity constraint on the weights, and let the weight space be ΘF=(1,u)∣u∈ℝd0⊂ℝd0+1 _F=\(1,u) u ^d_0\ ^d_0+1. In this remark only, we define ℓ01 ^01 and GEGE of Definition 6.2 for elements of ΘF _F by the same formulas. In the instance of Theorem 3.1 of Fatemi et al. (2026), (s):=0,(−1,s),(−3,−s)X(s):=\0,(-1,s),(-3,-s)\ and x=0x=0, we have ≥0(s)=0X_≥ 0(s)=\0\, so the OS loss coincides with the suboptimality loss. Hence the estimator θ^:=argminθ∈ΘFΨ(θ) θ:= _θ∈ _F (θ) s.t. ℓOS(θ,xn,sn)≤0 ^OS(θ;x^n,s^n)≤ 0 (∀n∀ n), which is Eq. (6) of that paper written with the OS loss, coincides with the estimator of that paper written with the suboptimality loss (Eq. (6) of that paper, denoted there by θ^Tsub θ^sub_T). Furthermore, since the set-level mismatch 0∉argmaxy∈(s)⟨θ^,y⟩0∉ _y (s) θ,y implies ℓ01(θ^,0,s)=1 ^01( θ;0,s)=1 (by the same argument as in Remark 3.2 of that paper), the lower bound of Theorem 3.1 of that paper, supθ∗,infΨ∈ℙ(GE(θ^)>ε)≥∑i=0d0−1(Ni)εi(1−ε)N−i _θ^*,P_S _ P(GE( θ)> )≥ _i=0^d_0-1 Ni ^i(1- )^N-i, holds as it is. However, since θ θ is a vertex of the LP, the optimal solution is not unique at observations with active constraints, so θ θ is not consistent in the sense of this paper. Theorem 6.5 converts this into a lower bound for consistent weights by shrinking θ θ toward the interior. Appendix H Details of the numerical experiments This section gives the details of the numerical experiments in Section 7. H.1. Problem settings of the experiments This subsection gives the mathematical models of the forward problems of the five experiments and how their instances are constructed. In all four experiments other than the single-machine scheduling problem, the number of states is N=20N=20, the dimension is d∈20,40,60,80,100d∈\20,40,60,80,100\, and T=100T=100, that is, the oracle-call budget is B=4000B=4000. In every experiment, all the methods use the same instance within the same episode. Knapsack problem We consider decisions on how many units of each item to pack under a limited capacity (e.g., loading or inventory planning). The forward problem is the bounded integer knapsack problem maxx∈ℤd⟨θ,x⟩s.t.⟨ws,x⟩≤cs, 0≤x≤us. _x ^d\ θ,x .t. w_s,x ≤ c_s,\ \ 0≤ x≤ u_s. A state s=(ws,cs,us)s=(w_s,c_s,u_s) is generated by drawing the weights ws∈1,…,9dw_s∈\1,…,9\^d, the capacity cs∈1,…,19c_s∈\1,…,19\, and the upper bounds on the numbers of items us∈1,…,4du_s∈\1,…,4\^d uniformly and independently for each component. The true weight θ∗θ^* is drawn from the Dirichlet distribution Dir()Dir(1), and the action xnx^n of the agent is generated by decreasing each component of x∗(θ∗,sn)x^*(θ^*,s^n) by 11 independently with probability 0.20.2 (without going below 00). Hence x∗(θ∗,sn)≥xnx^*(θ^*,s^n)≥ x^n, and θ∗θ^* satisfies Equation 3.2. Solving outperformance inverse optimization in this problem means finding weights of the item values under which an optimal packing contains every item at least as many times as the agent’s packing. The forward problem is solved exactly by dynamic programming. We run 50 episodes for each d (θ∗θ^* and the states are redrawn for each pair of an episode and d). Single-machine scheduling problem We consider deciding the order in which jobs with different release times are processed on a single machine according to their importance (e.g., sequencing tasks on a production line). The forward problem is single-machine scheduling with release times that minimizes the weighted sum of completion times ∑j=1dθjCj _j=1^d _jC_j. A state s consists of the release times rj∈0,…,14r_j∈\0,…,14\ and the processing times pj∈1,…,4p_j∈\1,…,4\ of the jobs j=1,…,dj=1,…,d (each drawn independently and uniformly), and, to match the maximization form, the action is the sign-reversed completion times x=−(C1,…,Cd)x=-(C_1,…,C_d). Hence x≥xnx≥ x^n means that every job is completed no later than by the agent, that is, a schedule earlier than that of the agent. From the optimal schedule under the true weight θ∗∼Dir()θ^* (1), we generate the action xnx^n of the agent by using thresholds τ1n∈8,…,12τ^n_1∈\8,…,12\ and τ2n∈18,…,22τ^n_2∈\18,…,22\ drawn for each state, delaying by 11 every job whose completion time is at least τ1nτ^n_1, and further delaying by another 11 every job whose completion time after this delay is at least τ2nτ^n_2 (the generation procedure is given in Algorithm 14). Hence x∗(θ∗,sn)≥xnx^*(θ^*,s^n)≥ x^n, and θ∗θ^* satisfies Equation 3.2. Solving outperformance inverse optimization in this problem means finding weights of the job importance under which an optimal schedule completes every job no later than the agent’s schedule. We set the number of states to N=10N=10, d∈4,6,8d∈\4,6,8\, 25 episodes for each d, and T=100T=100, that is, the oracle-call budget is B=2000B=2000, and the forward problem is solved by SCIP. Movie selection problem with viewing costs We consider a user selecting movies to watch under a budget on the viewing cost and per-genre upper limits. We use the rating data (u1.base) of MovieLens100K (Harper and Konstan, 2015). The ground set consists of the top d movies in the number of ratings, and Γ∈0,118×d ∈\0,1\^18× d is the genre indicator matrix of the movies (the rows correspond to the genres g=1,…,18g=1,…,18). A state s corresponds to one user, and WsW_s denotes the set of movies in the ground set rated by the user s. The forward problem is the movie selection under per-genre upper limits and a budget on the viewing cost, maxx∈0,1d⟨θ,x⟩s.t.⟨w,x⟩≤bs,Γx≤c¯s. _x∈\0,1\^d\ θ,x .t. w,x ≤ b_s,\ \ x≤ c_s. The cost wiw_i is the number of years elapsed from the release of the movie i at the time of rating (the median over users, rounded to years and set to at least 11), and the budget and the per-genre upper limits are determined from the viewing history as bs=⟨w,Ws⟩+miniwib_s= w,1_W_s + _iw_i (where Ws1_W_s is the indicator vector of WsW_s) and c¯s,g=(ΓWs)g+1 c_s,g=( 1_W_s)_g+1. The action of the agent is the indicator vector xn=Rsx^n=1_R_s of the set Rs⊆WsR_s W_s of movies to which the user s gave a rating of at least 44. The users with |Rs|≥5|R_s|≥ 5 are divided into groups of 20 in the order of user IDs, each group forming one episode, and we run 20 episodes for each d. No randomness is used. Since Rs⊆WsR_s W_s, xnx^n is feasible. In this experiment, we do not specify a true weight and do not assume the existence of a weight satisfying Equation 3.2. Solving outperformance inverse optimization in this problem means finding preference weights on the movies under which an optimal selection contains all the movies to which the user gave a rating of at least 44. The forward problem is solved by SCIP. Weighted multi-cover problem (Cora, ogbn-arxiv) We formulate the situation in which a researcher selects (reads or cites) a limited number of papers as a weighted multi-cover of words. For candidate papers i∈Psi∈ P_s and words j=1,…,dj=1,…,d (the set PsP_s is described below), let Aij∈0,1A_ij∈\0,1\ indicate whether the word j appears in the title or the text of the paper i, and let the forward problem be maxz∈0,1|Ps|,x∈0,…,rd⟨θ,x⟩s.t.xj≤∑i∈PsAijzi,∑i∈Pszi≤ks _z∈\0,1\^|P_s|,\,x∈\0,…,r\^d\ θ,x .t. x_j≤ _i∈ P_sA_ijz_i,\ \ _i∈ P_sz_i≤ k_s (x∈0,…,rdx∈\0,…,r\^d; the oracle returns, as the action, the componentwise min(r,A⊤z) (r,A z) computed from the selected papers z; that is, (s)X(s) is the set of all x satisfying the above constraints for some z). The action xjx_j is “the number of papers covering the word j (up to r)”, and θ is the weight of interest in the words. A state s corresponds to a paper v, the action of the agent is the depth xjn=min(r,|i∈Es:Aij=1|)x^n_j= (r,\,|\i∈ E_s:A_ij=1\|) of the set Es⊆PsE_s P_s of papers in a citation relation with v (described below), and the upper bound on the number of papers is ks=|Es|k_s=|E_s|. Hence x≥xnx≥ x^n means “covering every word at a depth no smaller than that of the agent with the same number of papers as the agent”, and EsE_s itself is a feasible solution. Solving outperformance inverse optimization in this problem means finding weights of interest in the words under which an optimal selection of papers covers every word at a depth no smaller than that of the agent with the same number of papers. The words are the first d words obtained by taking, alternately in decreasing order of the field size, the words ranked by their per-field scores, so that the words of several fields compete for the limited number of papers even when the number of words is small. We impose the upper bound on the depth because, with r=1r=1 (maximum coverage), the OS loss tends to be 00 by construction under the uniform weight, and without the upper bound (r≥ksr≥ k_s) the problem reduces to selecting the top ksk_s papers (Section H.4). Note, however, that this upper bound is not effective in the states with ks≤rk_s≤ r (38% for Cora and 35% for ogbn-arxiv). For Cora (Sen et al., 2008), the words are the 1433-word Bag-of-Words, the scores are the occurrence rates in each of the 7 fields, the states are the papers v of degree 3–10, EsE_s is the set of adjacent papers in the undirected citation graph (the papers cited by v and the papers citing v), PsP_s is the 2-hop neighborhood of v excluding v itself, and r=3r=3. For ogbn-arxiv (Hu et al., 2020), we use the direction of citations and the publication years; the states are the papers v of 2019 that cite 5–8 papers (published in 2019 or earlier), EsE_s is the set of papers actually cited by v, and PsP_s consists of EsE_s and the papers cited by them (all published in 2019 or earlier, excluding v itself). The words are constructed from the titles and abstracts (Wang et al., 2020), the scores are the distinctiveness in each of the 40 fields (the occurrence rate within the field minus the overall occurrence rate), and r=5r=5. For ogbn-arxiv, the states of the episode e are selected from the field with the (e+1)(e+1)-th largest number of papers in 2019 among the fields with at least N papers satisfying the conditions (e=0,…,19e=0,…,19). In both cases, we run 20 episodes for each d, and the forward problem is solved by SCIP. The generation procedures are given in Algorithms 18 and 19. We describe the computing environment in Section H.2, the implementation and the hyperparameters of each method in Section H.3, the details of the instances of the weighted multi-cover problem in Section H.4, the pseudocode of the evaluation protocol, of each method, and of the instance generation in Sections H.5 and H.6, and the results of all the methods in the five experiments in Section H.9. H.2. Computing environment All the experiments were run on a machine with Intel Xeon Gold 6444Y CPUs (16 cores, 2 sockets, 64 threads in total), 1 TB of memory, and Ubuntu 24.04.4 LTS. The implementation is in Python 3.11.15 (Van Rossum and Drake, 2009) with NumPy 2.4.3 and SciPy 1.17.1. As forward problems, the knapsack problem was solved by dynamic programming, and the MovieLens, Cora, ogbn-arxiv, and single-machine scheduling problems were solved by SCIP 10.0.0 (with SoPlex 8.0.0 as the LP solver) via OR-Tools 9.15 (Perron and Furnon, 2026). The time limit of SCIP was 60 seconds per solve for the MovieLens, Cora, and ogbn-arxiv problems, and 300 seconds for the single-machine scheduling problem. For the MovieLens, Cora, ogbn-arxiv, and single-machine scheduling problems, optimality was proved in every solve (the implementation stops with an error when optimality is not proved, and solutions without a proof of optimality are not used). For the single-machine scheduling problem, we observed the following two issues with SCIP, and therefore we first solved with presolve turned off and, when optimality was not proved, rebuilt the model and solved again with presolve turned on. First, with presolve turned on, the process terminates abnormally within presolve for certain inputs. Second, with presolve turned off, SCIP sometimes erroneously declares a feasible problem infeasible when the objective coefficients include extremely small values (of the order of 10−910^-9). In either path, the output of the oracle is a solution whose optimality for the original weight is proved. Among the master problems of the inner solvers, the LPs were solved by HiGHS 1.12.0 in SciPy and the QPs by Clarabel 0.11.1. The metric projections of ONS and MetaGrad were also solved by Clarabel. MovieLens100K and Cora were loaded with PyTorch Geometric 2.8.0.post1 (Fey and Lenssen, 2019) (for Cora, the Planetoid version (Yang et al., 2016)). MovieLens100K was used in accordance with the terms of use of the GroupLens Research Project (use for research purposes only, no redistribution of the data, and citation of Harper and Konstan (2015) in publications). For Cora (the Planetoid version), we used the distributed files at https://github.com/kimiyoung/planetoid (MIT license). For ogbn-arxiv, we directly loaded the files distributed by OGB (Open Graph Benchmark) (Hu et al., 2020) (citations, publication years, fields, and the correspondence to the paper IDs of the Microsoft Academic Graph) and the raw titles and abstracts (Wang et al., 2020) also distributed by OGB. We used ogbn-arxiv as distributed under the ODC-BY (Open Data Commons Attribution License) license. Three CPU cores were allocated to the computation, and at most three processes were run simultaneously. The movie selection problem with viewing costs was run in three simultaneous processes by dividing the episodes into 0–7, 8–15, and 16–19, and the knapsack problem was run in a single process following the end of one of these processes. The single-machine scheduling problem was run in three simultaneous processes by dividing the episodes into 0–8, 9–16, and 17–24. The two experiments on the weighted multi-cover problem were run in three simultaneous processes by dividing the episodes (while three processes run simultaneously, each process uses one core). Each method was run sequentially within a single process, and the computation time was measured in wall-clock time. As the random number generator we used numpy.random.default_rng; for the knapsack problem and the single-machine scheduling problem, a generator with the seed (20260928,e,d)(20260928,e,d) was created for each pair of an episode e and a dimension d to generate the instances. For the weighted multi-cover problem, the seed (20260928,e)(20260928,e) was used to sample the states of the episode e. Since in both cases the seed is determined only by the episode and the dimension, the same instances are obtained even when a run is resumed from the middle. H.3. Implementation and hyperparameters of the methods All 13 methods are listed in Table 3. In addition to the 7 methods compared in the main text (Section 7.1), they include Sub-ONS and Sub-MetaGrad, which are online learning methods updating state by state, and variants of OS-DC with different inner solvers (OS-DC[CP]+SGF, OS-DC[LV+CPF]+SGF, OS-DC[ONS], and OS-DC[MetaGrad]). SGF (subgradient finishing) denotes applying the projected subgradient method with a small step size after DC optimization stops, starting from the obtained weight. It is added because, even at a weight at which the OS loss is 00, the optimal solution of the forward problem is in general not unique, and the solution of the oracle does not necessarily dominate the action. All hyperparameters were fixed in advance and were not tuned on the evaluation instances. Within each pair that differs only in the loss function, the OS side and the Sub side use the same values (the Sub side is given twice as many iterations so that the oracle-call budgets coincide). Table 3. The 13 compared methods. PSG is the projected subgradient method, ONS is the Online Newton Step (Hazan et al., 2007), and MetaGrad is van Erven and Koolen (2016); we also call these optimization methods learners. FB (full-batch) denotes the update that gives the learner the average of the subgradients over all states only once per iteration and does not use the averaged iterate (the average of the iterates so far). The online-to-batch conversion is the conversion that outputs the averaged iterate. The inner solvers CP, LV+CPF, and PB+CPF are the cutting-plane method, the level method with cutting-plane finishing, and the proximal bundle method with cutting-plane finishing, respectively (Appendix D). In the main text, OS-DC[PB+CPF]+SGF is simply written as OS-DC. Method Optimization method Pseudocode Remarks Empirical mean of the suboptimality loss (existing) Sub-PSG PSG Algorithm 8 Sub-ONS ONS Algorithm 9 state-by-state update, online-to-batch conversion (Sakaue et al., 2025b) Sub-MetaGrad MetaGrad Algorithm 9 state-by-state update, online-to-batch conversion (Sakaue et al., 2025b) Sub-ONS(FB) ONS Algorithm 10 FB Sub-MetaGrad(FB) MetaGrad Algorithm 10 FB Empirical mean of the OS loss (proposed) OS-PSG PSG Algorithm 8 OS-ONS(FB) ONS Algorithm 10 FB OS-MetaGrad(FB) MetaGrad Algorithm 10 FB OS-DC[CP]+SGF DC optimization Algorithm 12 inner: CP, with SGF OS-DC[LV+CPF]+SGF DC optimization Algorithm 12 inner: LV+CPF, with SGF OS-DC[PB+CPF]+SGF DC optimization Algorithm 12 inner: PB+CPF, with SGF (OS-DC in the main text) OS-DC[ONS] DC optimization Algorithm 12 inner: ONS (FB; Algorithm 11) OS-DC[MetaGrad] DC optimization Algorithm 12 inner: MetaGrad (FB; Algorithm 11) Implementation of DC optimization The implementation of DC optimization (Algorithms 6 and 5 in the appendix) is a DCA that truncates the inner loop at J=20J=20 iterations by default (inexact DCA). Its validity under truncation is ensured by the following design: (i) the descent property that holds even under truncation (Proposition E.11); (i) tie stopping only when the inner solver reports convergence (converged=TRUEconverged=TRUE) (to avoid stopping at a tie that occurs by chance while the inner problem has not converged); (i) cut inheritance, which improves the accuracy of the inner solver without additional oracle cost (Proposition E.3); and (iv) certificate stopping (stopping when the empirical mean of the OS loss falls below the threshold ϵgap _gap; Proposition E.9) and target-attainment stopping (stopping when ℓOP ^OP falls below the threshold), both of which are valid regardless of the convergence of the inner loop. Projected subgradient method (Sub-PSG, OS-PSG) The update at iteration j is θj+1=ΠΔd−1(θj−η(j+1)−1/2gj‖gj‖2),gj=∑n=1N(x∗(θj,sn)−yjn).θ^j+1= _ ^d-1\! (θ^j-η\,(j+1)^-1/2\, g_j\|g_j\|_2 ), g_j= _n=1^N (x^*(θ^j,s^n)-y_j^n ). Here ΠΔd−1 _ ^d-1 is the Euclidean projection onto the probability simplex (an exact sorting-based method), and yjny_j^n is xnx^n for Sub-PSG and x∗(θj,xn,sn)x^*(θ^j;x^n,s^n) for OS-PSG (gj/Ng_j/N of OS-PSG is the gradient of the empirical mean of the OS loss in the sense of weak derivatives; Proposition 5.1). The step-size coefficient is η=0.1η=0.1. The method stops when gj=0g_j=0. The same coefficient, normalization, and projection are used in all five experiments, including the single-machine scheduling problem. When ⟨θj,x∗(θj,sn)−yjn⟩<0 θ^j,x^*(θ^j,s^n)-y_j^n <0 holds because of solver errors, x∗(θj,sn)x^*(θ^j,s^n) is replaced by yjny_j^n before computing ℓOP ^OP (the solution before the replacement is used to compute gjg_j). The number of iterations is T for OS-PSG and 2T2T for Sub-PSG. Sub-ONS One pass over the states s1,…,sNs^1,…,s^N in order constitutes one epoch, and at each round we update by the Online Newton Step (Hazan et al., 2007) Ht=Ht−1+gtgt⊤,θt+1=ΠΔd−1Ht(θt−γ−1Ht−1gt)H_t=H_t-1+g_tg_t , θ^t+1= ^H_t_ ^d-1\! (θ^t-γ^-1H_t^-1g_t ) using the subgradient gt=x∗(θt,sn)−xng_t=x^*(θ^t,s^n)-x^n of ℓsub(⋅,xn,sn) ^sub(·\,;x^n,s^n). Here ΠΔd−1H ^H_ ^d-1 is the projection onto the probability simplex in the H-norm, H0=ϵregIH_0= _regI, and γ=1γ=1, ϵreg=1 _reg=1. At the end of each epoch, the average of the θtθ^t visited so far (online-to-batch conversion) is evaluated on all states and recorded. The number of oracle calls per epoch is 2N2N in total, N for the updates and N for the evaluation, so that T epochs use up the oracle-call budget B=2TNB=2TN. Sub-MetaGrad This method replaces the learner of Sub-ONS by MetaGrad (van Erven and Koolen, 2016). It runs in parallel an ONS for each learning rate ηi=2−i/(5DG) _i=2^-i/(5DG) (i=0,…,⌈log2(TN+1)/2⌉i=0,…, _2(TN+1)/2 ) and aggregates them by tilted exponential weights. We use the diameter D=2D= 2 of the probability simplex, and for the upper bound G on the norm of the subgradients we use the norm of the first nonzero (per-state) subgradient. The online-to-batch conversion and the counting of oracle calls are the same as for Sub-ONS. Full-batch ONS and MetaGrad (Sub-ONS(FB), Sub-MetaGrad(FB), OS-ONS(FB), OS-MetaGrad(FB)) As in the projected subgradient method, at iteration j all states are solved with the same weight θjθ^j, and the full-batch subgradient gj=1N∑n=1N(x∗(θj,sn)−yjn)g_j= 1N _n=1^N (x^*(θ^j,s^n)-y_j^n ) is given to the learner only once. Here yjny_j^n is xnx^n on the Sub side and x∗(θj,xn,sn)x^*(θ^j;x^n,s^n) on the OS side. ONS updates as Hj=Hj−1+gjgj⊤,θj+1=ΠΔd−1Hj(θj−γ−1Hj−1gj)H_j=H_j-1+g_jg_j , θ^j+1= ^H_j_ ^d-1\! (θ^j-γ^-1H_j^-1g_j ) (with the initial metric ϵregI _regI, γ=1γ=1, ϵreg=1 _reg=1), and MetaGrad applies the same update as Sub-MetaGrad to gjg_j (D=2D= 2, G=‖g0‖2G=\|g_0\|_2). The number of learning rates of MetaGrad is set to ⌈log2(nit+1)/2⌉+1 _2(n_it+1)/2 +1 from the number of iterations nitn_it. The number of iterations is nit=Tn_it=T on the OS side (2N2N oracle calls per iteration) and nit=2Tn_it=2T on the Sub side (N per iteration), and both use up exactly the oracle-call budget B=2TNB=2TN. The averaged iterate is not used; at each iteration, ℓOP ^OP at θjθ^j and the empirical mean of the OS loss are computed directly from the solutions used for the update and recorded (x∗(θj,xn,sn)x^*(θ^j;x^n,s^n) used for recording the OS loss on the Sub side is a solve used only for logging and is not counted as an oracle call). Stopping when gj=0g_j=0 is the same as in the projected subgradient method. The replacement against solver errors is used not only for ℓOP ^OP but also for computing gjg_j. These methods differ from Sub-ONS and Sub-MetaGrad in that each update uses all states and in that the averaged iterate is not used. DC optimization of the OS loss (OS-DC) The maximum number of outer iterations is K=2T/20K=2T/20, the maximum number of inner iterations is J=20J=20, the threshold of tie stopping is ϵtie=10−8 _tie=10^-8, and the threshold of certificate stopping is ϵgap=10−6 _gap=10^-6. As the rules for selecting the evaluation points of the inner solver (Appendix D), we compare the cutting-plane method (CP), the level method with cutting-plane finishing (LV+CPF), and the proximal bundle method with cutting-plane finishing (PB+CPF), and subgradient finishing is added to all of them. In addition, we also compare OS-DC with ONS or MetaGrad as the inner solver (without subgradient finishing) (next paragraph). For CP, the inner tolerance is ϵin=10−6 _in=10^-6; for LV+CPF and PB+CPF, ϵin=0 _in=0, the threshold for switching phases is 10−610^-6, and the termination tolerance is ϵterm=10−9 _term=10^-9 (Algorithm 4). The exactness of Theorem F.3 is guaranteed for ϵterm=0 _term=0; the positive value is a safeguard against rounding errors. In subgradient finishing (SGF), OS-PSG is run for T iterations from the output of DC optimization with the step-size coefficient η=10−3η=10^-3 (1/1001/100 of that of the usual projected subgradient method) in all five experiments. The step size is made small because a large move makes the OS loss positive again. At every evaluation of the forward-problem oracle (including the inner evaluation points), ℓOP ^OP is computed, and the whole procedure stops when it becomes 00. The proximal weight of the proximal bundle method is fixed at u=1u=1, the descent parameter is ρ=0.1ρ=0.1, and the level parameter of the level method is λ=1/(2+2)λ=1/(2+ 2). The output of DC optimization is the attaining point in the case of target-attainment stopping and, otherwise, the point with the smallest ℓk _k among all outer iterations (Algorithm 6). The whole procedure is given in Algorithm 12. The evaluation points computed by the inner solver along the way (the solutions of the LPs and QPs of the master problems) are passed to the oracle without being projected onto the probability simplex, and only the output of the inner solver is projected onto the probability simplex. Hence the intermediate evaluation points may lie outside the probability simplex within the tolerance of the LP and QP solvers. The weights reported as best-so-far may also lie outside the probability simplex within the same tolerance. OS-DC in the main text is OS-DC[PB+CPF]+SGF. OS-DC with ONS or MetaGrad as the inner solver (OS-DC[ONS], OS-DC[MetaGrad]) This is OS-DC that solves the convex subproblem minθ∈Θℓ¯sub(θ,y′) _θ∈ ^sub(θ;y ) (Equation 5.1) of a branch y′y by ONS or MetaGrad. The convex subproblem ℓ¯sub(⋅,y′) ^sub(·\,;y ) is exactly the empirical mean of the suboptimality loss in which the actions of the agent are replaced by the dominating vertices y′ny n. The outer loop (Algorithm 6) and K, ϵtie _tie, and ϵgap _gap are the same as for OS-DC, and only the inner loop is replaced as follows. At each inner iteration t, x∗(θt,sn)x^*(θ^t,s^n) is computed for all states (N oracle calls), and, using the full-batch subgradient gt=1N∑n=1N(x∗(θt,sn)−y′n)g_t= 1N _n=1^N (x^*(θ^t,s^n)-y n ), OS-DC[ONS] performs the same update as Sub-ONS, Ht=Ht−1+gtgt⊤,θt+1=ΠΔd−1Ht(θt−γ−1Ht−1gt)H_t=H_t-1+g_tg_t , θ^t+1= ^H_t_ ^d-1\! (θ^t-γ^-1H_t^-1g_t ) (H0=ϵregIH_0= _regI, γ=1γ=1, ϵreg=1 _reg=1). OS-DC[MetaGrad] replaces this update by the same MetaGrad as Sub-MetaGrad (D=2D= 2, and G is the norm of the first subgradient). The number of learning rates is set to ⌈log2(KJ+1)/2⌉+1 _2(KJ+1)/2 +1 from the total number of rounds KJKJ of the whole DC optimization. Within each outer iteration, the inner loop outputs the point with the smallest ℓ¯sub(⋅,y′) ^sub(·\,;y ) among the visited θtθ^t and their average (online-to-batch conversion). The internal state of the learner is carried over across outer iterations. That is, the first outer iteration starts from θ0=/dθ^0=1/d, and each subsequent one resumes from the state at the end of the previous outer iteration. What is carried over is the metric HtH_t and the last iterate for ONS, and, for MetaGrad, the point and the metric of each learning rate, the aggregation weights, G, and the learning rates. On the other hand, the cumulative sum for the average is reset at each outer iteration, because the guarantee of the online-to-batch conversion holds only for a fixed loss, and the convex subproblem changes when the branch changes. The maximum number of inner iterations is J=20J=20 and the tolerance is ϵin=10−6 _in=10^-6; the inner loop stops when ℓ¯sub(θt,y′)≤ϵin ^sub(θ^t;y )≤ _in. Since ONS and MetaGrad have no cutting-plane model, no lower bound LBLB is available, and cut inheritance is not used. The inner loop reports converged == TRUE only when ℓ¯sub(θt,y′)≤ϵin ^sub(θ^t;y )≤ _in or when the subgradient becomes 00. Neither subgradient finishing nor cutting-plane finishing is used. Even when the branch changes, the metric built from the subgradients of the previous branch is carried over without correction. Hence the whole procedure takes the form of “running a single ONS (MetaGrad) while updating the linearization point along the way”. Remark H.1 (Replacement against solver errors). In the projected subgradient method, subgradient finishing, and the full-batch ONS and MetaGrad, when ⟨θ,x∗(θ,sn)−yn⟩<0 θ,x^*(θ,s^n)-y^n <0, x∗(θ,sn)x^*(θ,s^n) is replaced by yny^n (xnx^n on the Sub side and x∗(θ,xn,sn)x^*(θ;x^n,s^n) on the OS side) before computing ℓOP ^OP (in the projected subgradient method and subgradient finishing, the solution before the replacement is used for the subgradient gjg_j, whereas in the full-batch ONS and MetaGrad the solution after the replacement is used). On the other hand, in Sub-ONS, Sub-MetaGrad, and the main body of DC optimization, ℓOP ^OP is computed from the solution returned by the oracle as it is. However, since every yny^n is a feasible solution in (sn)X(s^n), ⟨θ,x∗(θ,sn)⟩≥⟨θ,yn⟩ θ,x^*(θ,s^n) ≥ θ,y^n always holds if x∗(θ,sn)x^*(θ,s^n) is an exact optimal solution, and this replacement does not occur. In our experiments, the knapsack problem is solved exactly by dynamic programming, and the MovieLens, Cora, ogbn-arxiv, and single-machine scheduling problems stop with an error when optimality is not proved (Section H.2); hence this replacement does not occur. Therefore, the difference in the treatment among the methods does not affect the results. Reuse of the models of the forward problems For the MovieLens, Cora, and ogbn-arxiv problems (for the single-machine scheduling problem the model is rebuilt at every solve), the model of the forward problem for the same state and the same dominance constraint is kept, and only the objective coefficients are replaced for re-solving. Since which optimal solution is returned when the optimal solution is not unique may depend on the preceding solve history, the kept models are reset before running each method, so that the trajectory of a method does not depend on the order in which the other methods are run. H.4. Details of the instances of the weighted multi-cover problem Words For Cora, the 1433-word Bag-of-Words (presence or absence) is used as it is. For ogbn-arxiv, the titles and abstracts are lowercased, and we use the words consisting only of alphabetic characters with at least three letters, excluding common English stop words (no stemming). Words that appear in fewer than 20 papers are discarded. For each field c, the score of a word j is the fraction of the papers of the field c containing the word j (occurrence rate) for Cora, and the occurrence rate minus the occurrence rate over all papers (distinctiveness) for ogbn-arxiv; the ranking of the words is constructed by taking one word at a time alternately from the per-field rankings of the scores, starting from the field with the largest number of papers (skipping words already taken). The words of dimension d are the first d words of this ranking, so that the sets of words are nested as d increases. We use the distinctiveness rather than the occurrence rate for ogbn-arxiv because with the occurrence rate, general words independent of the fields, such as based, paper, show, and model, occupy the top ranks. States The selection of the states does not depend on the dimension, and 20 papers are sampled at random for each episode (since the episode number is included in the seed, the same states are used in the same episode even when the dimension is changed). For Cora, the number of candidates has median 25 (3–210), and the number of papers selected by the agent has median 4 (3–10). For ogbn-arxiv, an edge (u,w)(u,w) represents that u cites w (98.2% of the edges have a citing paper whose publication year is no earlier than that of the cited paper). By the condition on the publication year (2019 or earlier), EsE_s and PsP_s consist only of papers that existed at the time v was written. The number of candidates has median 38 (6–211), and the number of papers selected by the agent has median 6 (5–8). The fields of the episodes 0,1,…,190,1,…,19 are, in decreasing order of the number of papers in 2019, cs.LG, cs.CV, cs.CL, cs.IT, cs.RO, …. Degeneracy at the uniform weight In a state in which the OS loss is 00 at the uniform weight θ0=/dθ^0=1/d, the barycenter of the probability simplex is itself a global minimizer. The upper bound ksk_s on the number of papers and the upper bound r on the depth were chosen so that this degeneracy is reduced. The fraction of the states at which the OS loss is positive at the uniform weight (60 states of the episodes 0–2) is 57% for d=20d=20 and 88% for d=40d=40 for Cora (r=3r=3), and 33% for d=20d=20 and 82% for d=40d=40 for ogbn-arxiv (r=5r=5); for d=100d=100, it was positive at all 20 states of the episode 0 in both experiments. Setting the upper bound on the number of papers to ks=|Es|+1k_s=|E_s|+1 increases the degeneracy (because the extra paper can recover the words of the agent), so we set ks=|Es|k_s=|E_s|. In the measured range, the states at which the OS loss is positive under the uniform weight account for at least 80% for d=40d=40 and for all states for d=100d=100 (episode 0). H.5. Pseudocode of the evaluation protocol and of each method This subsection gives the evaluation protocol as implemented and the pseudocode of each method. In what follows, “evaluating” a weight θ means solving vn←x∗(θ,sn)v^n← x^*(θ,s^n) for all states (N oracle calls) and computing and recording ℓOP(θ)=1N∑n=1N∑i=1dmaxxin−vin,0 ^OP(θ)= 1N _n=1^N _i=1^d \x^n_i-v^n_i,0\ (Equation E.6). In every method, if ℓOP≤0 ^OP≤ 0 at an evaluation, the whole method stops at that point and returns the weight evaluated then (target-attainment stopping). The solves y~n←x∗(θ,xn,sn) y^n← x^*(θ;x^n,s^n) used only for logging (for recording ℓ¯OS ^OS) are not counted as oracle calls. Each method is evaluated at the weight with the smallest ℓOP ^OP among the weights visited so far (for Sub-ONS and Sub-MetaGrad, the averaged iterates at the end of each epoch (one pass over the states); for OS-DC, all evaluation points of the inner loop, the outer loop, and subgradient finishing) (best-so-far; among weights with the same ℓOP ^OP, the one with the smaller empirical mean of the OS loss), and the empirical mean of the OS loss is also measured at the same weight. Letting T be the number of iterations of the projected subgradient method and N the number of states, we impose the oracle-call budget B:=2TNB:=2TN common to all methods and compare the values at the time when the number of oracle calls reaches B. OS-PSG, OS-ONS(FB), and OS-MetaGrad(FB) make 2N2N oracle calls per iteration and are run for T iterations, and Sub-PSG, Sub-ONS(FB), and Sub-MetaGrad(FB) make N oracle calls per iteration and are run for 2T2T iterations. Sub-ONS and Sub-MetaGrad make 2N2N oracle calls per epoch (one pass over the states), N for the updates and N for the evaluation of the averaged iterate, and are run for T epochs. The number of oracle calls of OS-DC differs from one outer iteration to another, and it may be run beyond the oracle-call budget B, but the values at the time when the number of oracle calls reaches B are used for the comparison. All methods stop as target attainment when ℓOP=0 ^OP=0 is reached. Algorithm 7 Common evaluation protocol (oracle-call budget B and best-so-far) 0: method, oracle-call budget B=2TNB=2TN, attainment tolerance ϵatt=10−9 _att=10^-9 1: Run the method and obtain the sequence of records (ci,ℓiOP,ℓ¯iOS)i=0,1,…(c_i, ^OP_i, ^OS_i)_i=0,1,… (cic_i is the cumulative number of oracle calls at the time of recording: the number of calls to x∗(⋅,s)x^*(·,s) for the 5 existing methods, and the sum of the numbers of calls to x∗(⋅,s)x^*(·,s) and x∗(⋅,x′,s)x^*(·\,;x ,s) for the 8 proposed methods) 2: for i=0,1,…i=0,1,… do 3: (Li,Oi)←(L_i,O_i)← the lexicographic minimum of (ℓi′OP,ℓ¯i′OS)i′≤i( ^OP_i , ^OS_i )_i ≤ i (an undefined ℓ¯OS ^OS is regarded as +∞+∞, and ties keep the earlier record) 4: end for 5: iB←maxi∣ci≤Bi_B← \i c_i≤ B\ 6: return LiBL_i_B, OiBO_i_B, attainment [LiB≤ϵatt][L_i_B≤ _att], and computation time (the maximum wall-clock time from the start of the method) Algorithm 8 Projected subgradient method (Sub-PSG, OS-PSG, subgradient finishing) 0: initial point θ0θ^0, step-size coefficient η, number of iterations nitn_it, type ∈Sub,OS∈\Sub,OS\ 1: for j=0,1,…,nit−1j=0,1,…,n_it-1 do 2: For each n, vn←x∗(θj,sn)v^n← x^*(θ^j,s^n) and y~n←x∗(θj,xn,sn) y^n← x^*(θ^j;x^n,s^n) (counted for OS; used only for logging for Sub) 3: For each n, yn←y~ny^n← y^n (OS) or yn←xny^n← x^n (Sub) 4: ℓ¯jOS←1N∑n=1Nmax⟨θj,vn−y~n⟩,0 ^OS_j← 1N _n=1^N \ θ^j,v^n- y^n ,0\ 5: For each n, v^n←yn v^n← y^n if ⟨θj,vn−yn⟩<0 θ^j,v^n-y^n <0, and v^n←vn v^n← v^n otherwise 6: Record ℓjOP←1N∑n=1N∑i=1dmaxxin−v^in,0 ^OP_j← 1N _n=1^N _i=1^d \x^n_i- v^n_i,0\ 7: if ℓjOP≤0 ^OP_j≤ 0 then 8: return θjθ^j 9: end if 10: gj←∑n=1N(vn−yn)g_j← _n=1^N(v^n-y^n) (using vnv^n before the replacement) 11: if gj=0g_j=0 then 12: return θjθ^j 13: end if 14: θj+1←ΠΔd−1(θj−η(j+1)−1/2gj/∥gj∥2)θ^j+1← _ ^d-1 (θ^j-η\,(j+1)^-1/2g_j/\|g_j\|_2 ) 15: end for 16: return θnit−1θ^n_it-1 (the last evaluated point) In Algorithm 8, (θ0,η,nit)(θ^0,η,n_it) is (/d,0.1,2T)(1/d,0.1,2T) for Sub-PSG, (/d,0.1,T)(1/d,0.1,T) for OS-PSG, and (the output of DC optimization,10−3,T)(the output of DC optimization,10^-3,T) for subgradient finishing (type OS). In subgradient finishing, no stopping based on ℓ¯OS ^OS is performed, and the decay (j+1)−1/2(j+1)^-1/2 of the step size is counted again from j=0j=0. These parameters are common to all five experiments, including the single-machine scheduling problem. In what follows, let ℋdH_d be the set of d×d× d symmetric positive definite matrices, and for H∈ℋdH _d and z∈ℝdz ^d, let ΠΔd−1H(z):=argminθ∈Δd−1(θ−z)⊤H(θ−z) ^H_ ^d-1(z):= _θ∈ ^d-1(θ-z) H(θ-z) (the minimizer is unique since the objective function is strictly convex; it is solved as a quadratic program by Clarabel). An online learner is given by a set L of internal states together with an initialization :Δd−1→ I ^d-1 , a prediction :→Δd−1 P → ^d-1, and an update :×(ℝd∖0)→ U ×(R^d \0\) . Definition H.2 (ONS). Let γ=1γ=1 and ϵreg=1 _reg=1. The set of internal states is ONS:=Δd−1×ℋdL_ONS:= ^d-1×H_d, and for ℒ=(θ,H)∈ONSL=(θ,H) _ONS we define (θinit) I(θ^init) :=(θinit,ϵregI),(ℒ):=θ, :=(θ^init, _regI), P(L):=θ, (ℒ,g) U(L,g) :=(ΠΔd−1H′(θ−γ−1(H′)−1g),H′),H′:=H+gg⊤. := ( ^H _ ^d-1(θ-γ^-1(H )^-1g),\,H ), H :=H+g . Definition H.3 (MetaGrad). Let the diameter be D=2D= 2 and the total number of rounds be nrn_r, and put nη:=⌈log2(nr+1)/2⌉+1n_η:= _2(n_r+1)/2 +1. The set of internal states is MG:=(ℝ>0nη∪⊥)×Δnη−1×(Δd−1)nη×(ℋd)nη,L_MG:= (R_>0^n_η∪\ \ )× ^n_η-1×( ^d-1)^n_η×(H_d)^n_η, whose elements we write as ℒ=(η,π,(wi)i=0nη−1,(Hi)i=0nη−1)L=(η,π,(w^i)_i=0^n_η-1,(H^i)_i=0^n_η-1). Here ⊥ is a symbol indicating that the learning rates η are not yet set; they are determined from the norm of the first subgradient at the first update. We define the initialization and the prediction by (θinit):=(⊥,πinit,(θinit)i,(I/D2)i),πiinit:=nη+1nη(i+1)(i+2), I(θ^init):= ( ,π^init,(θ^init)_i,(I/D^2)_i ), π^init_i:= n_η+1n_η(i+1)(i+2), (ℒ):=w0(η=⊥),∑iπiηiwi∑iπiηi(otherwise). P(L):= casesw^0&(η= ),\\ _i _i _iw^i _i _i _i&(otherwise). cases The update (ℒ,g)=(η′,π′,(wi′)i,(Hi′)i) U(L,g)=(η ,π ,(w^i )_i,(H^i )_i) is defined, with θ:=(ℒ)θ:= P(L), by η′ η :=(2−i/(5D‖g‖2))i=0nη−1(η=⊥),η(otherwise), := cases (2^-i/(5D\|g\|_2) )_i=0^n_η-1&(η= ),\\ η&(otherwise), cases ςi _i :=⟨wi−θ,g⟩,Hi′:=Hi+2(ηi′)2g⊤, := w^i-θ,g , H^i :=H^i+2(η _i)^2g , wi′ w^i :=ΠΔd−1Hi′(wi−ηi′(1+2ηi′ςi)(Hi′)−1g), := ^H^i _ ^d-1 (w^i-η _i(1+2η _i _i)(H^i )^-1g ), πi′ π _i :=πiexp(−ηi′ςi−(ηi′ςi)2)∑jπjexp(−ηj′ςj−(ηj′ςj)2). := _i (-η _i _i-(η _i _i)^2) _j _j (-η _j _j-(η _j _j)^2). Since wi=θinitw^i=θ^init for all i in (θinit) I(θ^init), the prediction right after η is set equals w0w^0 also by the general formula, so the case distinction in P causes no discontinuous change of the prediction. Algorithm 9 Sub-ONS and Sub-MetaGrad (state-by-state update and online-to-batch conversion) 0: learner (,,)( I, P, U) (ONS: Definition H.2, MetaGrad: Definition H.3; nr=TNn_r=TN for MetaGrad), θinit=/dθ^init=1/d, number of epochs T 1: ℒ←(θinit)L← I(θ^init), S←0S← 0, t←0t← 0 2: for e=0,1,…,T−1e=0,1,…,T-1 do 3: for n=1,…,Nn=1,…,N do 4: θt←(ℒ)θ^t← P(L), S←S+θtS← S+θ^t, t←t+1t← t+1 5: gt←x∗(θt,sn)−xng_t← x^*(θ^t,s^n)-x^n (one oracle call) 6: if gt≠0g_t≠ 0 then 7: ℒ←(ℒ,gt)L← U(L,g_t) 8: end if 9: end for 10: Evaluate θ¯e←S/t θ^e← S/t (N oracle calls), and record ℓ¯OS ^OS from y~n←x∗(θ¯e,xn,sn) y^n← x^*( θ^e;x^n,s^n) (used only for logging) 11: end for 12: return θ¯T−1 θ^T-1 (the last evaluated point) Algorithm 10 Full-batch ONS and MetaGrad (Sub-ONS(FB), Sub-MetaGrad(FB), OS-ONS(FB), OS-MetaGrad(FB)) 0: learner (,,)( I, P, U) (ONS: Definition H.2, MetaGrad: Definition H.3; nr=nitn_r=n_it for MetaGrad), θinit=/dθ^init=1/d, number of iterations nitn_it (OS: T, Sub: 2T2T) 1: ℒ←(θinit)L← I(θ^init) 2: for j=0,1,…,nit−1j=0,1,…,n_it-1 do 3: θj←(ℒ)θ^j← P(L) 4: For each n, vn←x∗(θj,sn)v^n← x^*(θ^j,s^n) and y~n←x∗(θj,xn,sn) y^n← x^*(θ^j;x^n,s^n) (counted for OS; used only for logging for Sub) 5: For each n, let yn←y~ny^n← y^n (OS) or yn←xny^n← x^n (Sub), and set vn←ynv^n← y^n if ⟨θj,vn−yn⟩<0 θ^j,v^n-y^n <0 6: Record ℓjOP ^OP_j and ℓ¯jOS ^OS_j with vnv^n after the replacement 7: if ℓjOP≤0 ^OP_j≤ 0 then 8: return θjθ^j 9: end if 10: gj←1N∑n=1N(vn−yn)g_j← 1N _n=1^N(v^n-y^n) 11: if gj=0g_j=0 then 12: return θjθ^j 13: end if 14: ℒ←(ℒ,gj)L← U(L,g_j) 15: end for 16: return θnit−1θ^n_it-1 (the last evaluated point) Algorithm 11 Inner loop of OS-DC[ONS] and OS-DC[MetaGrad] (replacing line 3 of Algorithm 6) 0: branch y′y , outer point θkθ^k, learner (,,)( I, P, U) (ONS: Definition H.2, MetaGrad: Definition H.3; D=2D= 2 and nr=KJn_r=KJ for MetaGrad), internal state of the learner ℒL (undefined for k=0k=0), ϵin=10−6 _in=10^-6, J=20J=20, feasibility tolerance ϵfeas=10−9 _feas=10^-9 1: f(θ):=ℓ¯sub(θ,y′)f(θ):= ^sub(θ;y ), g(θ):=1N∑n=1N(x∗(θ,sn)−y′n)g(θ):= 1N _n=1^N(x^*(θ,s^n)-y n) (one evaluation is shared for the same θ) 2: f⋆←+∞f ←+∞, converged←FALSEconverged 3: if ℒL is undefined then 4: if the learner is ONS then 5: ℒ←(θk)L← I(θ^k) 6: else if the learner is MetaGrad then 7: Evaluate θkθ^k (N oracle calls), and set f⋆←f(θk)f ← f(θ^k), θ^←θk θ←θ^k, and G←‖g(θk)‖2G←\|g(θ^k)\|_2 8: if G=0G=0 then 9: return (θ^,TRUE,ℒ)( θ,TRUE,L) (ℒL remains undefined) 10: end if 11: Set ℒ←(θk)L← I(θ^k) and replace the η component of ℒL by (2−i/(5DG))i=0nη−1 (2^-i/(5DG) )_i=0^n_η-1 12: end if 13: else 14: Use the input ℒL as it is (θkθ^k is not used) 15: end if 16: Σ←0 ← 0 17: for t=1,…,Jt=1,…,J do 18: Evaluate θt←(ℒ)θ^t← P(L) (N oracle calls; at t=1t=1 of the first outer iteration of MetaGrad, the prediction is mathematically equal to θkθ^k, but it is evaluated again if the bit patterns do not coincide because of floating-point rounding) 19: If f(θt)<f⋆f(θ^t)<f , then f⋆←f(θt)f ← f(θ^t) and θ^←θt θ←θ^t. Σ←Σ+θt ← +θ^t 20: if f⋆≤ϵinf ≤ _in or g(θt)=0g(θ^t)=0 then 21: converged←TRUEconverged , break 22: end if 23: ℒ←(ℒ,g(θt))L← U(L,g(θ^t)) 24: end for 25: if converged=FALSEconverged=FALSE and θ¯:=Σ/t∈Θ θ:= /t∈ (tolerance ϵfeas _feas) then 26: Evaluate θ¯ θ (N oracle calls), and set θ^←θ¯ θ← θ if f(θ¯)<f⋆f( θ)<f 27: end if 28: return (θ^,converged,ℒ)( θ,converged,L) Algorithm 12 The whole procedure of OS-DC[⋅·] (+SGF) 0: K=⌊2T/20⌋K= 2T/20 , J=20J=20, ϵtie=10−8 _tie=10^-8, ϵgap=10−6 _gap=10^-6, inner solver, whether to perform subgradient finishing 1: θ←ΠΔd−1(/d)θ← _ ^d-1(1/d), y′n←x∗(θ,xn,sn)y n← x^*(θ;x^n,s^n) for each n (N oracle calls), ←∅V← , and let the internal state of the learner ℒL be undefined (ONS, MetaGrad) 2: for k=0,1,…,K−1k=0,1,…,K-1 do 3: Run the inner solver (CP, LV+CPF, PB+CPF: Algorithm 5 (using inherited cuts from V); ONS, MetaGrad: Algorithm 11 (inputs θk:=θ^k:=θ and ℒL; the output ℒL replaces ℒL)) from the initial point θ, and obtain (θ′,converged)(θ ,converged) 4: Evaluate θ←ΠΔd−1(θ′)θ← _ ^d-1(θ ) to obtain vnv^n and add them to V, and set y′n←x∗(θ;xn,sn)y n← x^*(θ;x^n,s^n) for each n (2N2N oracle calls in total) 5: ℓk←1N∑n=1N⟨θ,vn−y′n⟩ _k← 1N _n=1^N θ,v^n-y n , Δk←1N∑n=1N⟨θ,y′n−y′n⟩ _k← 1N _n=1^N θ,y n-y n , θ(k)←θ^(k)←θ 6: if ℓk≤ϵgap _k≤ _gap, or (Δk≤ϵtie _k≤ _tie and converged=TRUEconverged=TRUE) then 7: break 8: end if 9: y′←y′y ← y 10: end for 11: θDC←θ(k⋆)θ^DC←θ^(k ), where k⋆k is the smallest index minimizing ℓk _k (regardless of the reason for stopping) 12: if subgradient finishing is performed then 13: Run Algorithm 8 with type OS and (θ0,η,nit)=(θDC,10−3,T)(θ^0,η,n_it)=(θ^DC,10^-3,T), and let θSGFθ^SGF be its output 14: return θSGFθ^SGF 15: end if 16: return θDCθ^DC In OS-DC, ℓ¯OS ^OS is recorded at all evaluation points of the inner and outer loops by solves used only for logging. The pool V of inherited cuts is shared within a single run of DC optimization. H.6. Pseudocode for instance generation For each experiment, an instance is generated once for each pair of an episode and a dimension d, and the same (sn,xn)n=1N\(s^n,x^n)\_n=1^N is given to all methods. All random numbers in the following pseudocode are drawn in order from a single generator of numpy.random.default_rng. Its seed is (20260928,e,d)(20260928,e,d) for the knapsack problem and the single-machine scheduling problem (Algorithms 13 and 14) and (20260928,e)(20260928,e) for the weighted multi-cover problem (Algorithms 18 and 19), where e is the episode index (Section H.2). Knapsack problem For a state s=(w,c,u)s=(w,c,u), we have x∗(θ,s)∈argmax⟨θ,x⟩∣x∈ℤd,⟨w,x⟩≤c, 0≤x≤ux^*(θ,s)∈ \ θ,x x ^d,\ w,x ≤ c,\ 0≤ x≤ u\ and x∗(θ;xn,s)∈argmax⟨θ,x⟩∣x∈ℤd,⟨w,x⟩≤c,xn≤x≤ux^*(θ;x^n,s)∈ \ θ,x x ^d,\ w,x ≤ c,\ x^n≤ x≤ u\, both of which are solved exactly by dynamic programming for the bounded knapsack problem (for the latter, we solve the problem in x−xnx-x^n with capacity c−⟨w,xn⟩c- w,x^n and upper bound u−xnu-x^n). Algorithm 13 Instance generation for the knapsack problem (one episode and dimension d) 0: episode index e, dimension d, number of states N=20N=20 1: θ∗∼Dir()θ^* (1) 2: for n=1,…,Nn=1,…,N do 3: win∼U1,…,9w^n_i \1,…,9\, uin∼U1,…,4u^n_i \1,…,4\ (i=1,…,di=1,…,d), cn∼U1,…,19c^n \1,…,19\ (all independent), sn←(wn,cn,un)s^n←(w^n,c^n,u^n) 4: end for 5: for n=1,…,Nn=1,…,N do 6: xn←x∗(θ∗,sn)x^n← x^*(θ^*,s^n) 7: for i=1,…,di=1,…,d do 8: with probability 0.20.2, xin←maxxin−1,0x^n_i← \x^n_i-1,0\ 9: end for 10: end for 11: return (sn,xn)n=1N\(s^n,x^n)\_n=1^N Since 0≤xn≤x∗(θ∗,sn)0≤ x^n≤ x^*(θ^*,s^n) and w≥0w≥ 0, xnx^n is feasible, and x∗(θ∗,sn)x^*(θ^*,s^n) is a feasible solution of the dominance-constrained problem. Single-machine scheduling problem For a state s=(r,p)s=(r,p) and an upper bound C¯∈ℤd C ^d on the completion times, we solve minS,C∈ℤd,z∈0,1(j,k):j<k _S,C ^d,\ z∈\0,1\^\(j,k):j<k\\ ∑j=1dθjCj _j=1^d _jC_j s.t. Cj=Sj+pj,rj≤Sj,Cj≤C¯j(j=1,…,d), C_j=S_j+p_j, r_j≤ S_j, C_j≤ C_j (j=1,…,d), Sk≥Sj+pj−Mbig(1−zjk)(1≤j<k≤d), S_k≥ S_j+p_j-M_big(1-z_jk) (1≤ j<k≤ d), Sj≥Sk+pk−Mbigzjk(1≤j<k≤d) S_j≥ S_k+p_k-M_bigz_jk (1≤ j<k≤ d) with SCIP and return the action x=−Cx=-C. For x∗(θ,s)x^*(θ,s) we set C¯j=maxkrk+∑kpk C_j= _kr_k+ _kp_k, for x∗(θ,xn,s)x^*(θ;x^n,s) we set C¯j=−xjn C_j=-x^n_j, and Mbig=2maxj(C¯j+pj)M_big=2 _j( C_j+p_j). Algorithm 14 Instance generation for the single-machine scheduling problem (one episode and dimension d) 0: episode index e, number of jobs d, number of states N=10N=10 1: θ∗∼Dir()θ^* (1) 2: for n=1,…,Nn=1,…,N do 3: rjn∼U0,…,14r^n_j \0,…,14\, pjn∼U1,…,4p^n_j \1,…,4\ (j=1,…,dj=1,…,d, all independent), sn←(rn,pn)s^n←(r^n,p^n) 4: end for 5: for n=1,…,Nn=1,…,N do 6: xn←x∗(θ∗,sn)x^n← x^*(θ^*,s^n) 7: τ1n∼U8,…,12τ^n_1 \8,…,12\, τ2n∼U18,…,22τ^n_2 \18,…,22\ 8: for j=1,…,dj=1,…,d do 9: if xjn≤−τ1nx^n_j≤-τ^n_1, then xjn←xjn−1x^n_j← x^n_j-1 10: if xjn≤−τ2nx^n_j≤-τ^n_2, then xjn←xjn−1x^n_j← x^n_j-1 11: end for 12: end for 13: return (sn,xn)n=1N\(s^n,x^n)\_n=1^N Let Cj∗C^*_j be the completion times of x∗(θ∗,sn)x^*(θ^*,s^n). Then the completion times of the agent are f(Cj∗):=Cj∗+[Cj∗≥τ1n]+[Cj∗≥τ2n−1]f(C^*_j):=C^*_j+[C^*_j≥τ^n_1]+[C^*_j≥τ^n_2-1]. Since f(c+1)−f(c)≥1f(c+1)-f(c)≥ 1, f preserves the processing order and the release-time constraints, so xnx^n is feasible and x∗(θ∗,sn)x^*(θ^*,s^n) is a feasible solution of the dominance-constrained problem. Weighted multi-cover problem The word ranking common to both experiments is given in Algorithm 15, the word scores for Cora and ogbn-arxiv in Algorithms 16 and 17, and the instance generation in Algorithms 18 and 19. Here, argsort↓argsort denotes a stable sort in descending order in which ties are ordered by increasing index. The stop words for ogbn-arxiv are 164 words based on the lists of NLTK and scikit-learn. The forward problem is solved with SCIP with x∈0,…,rdx∈\0,…,r\^d (xn≤x≤rx^n≤ x≤ r in the dominance-constrained version), and returns the componentwise min(r,A⊤z) (r,A z) of the selected papers z. Algorithm 15 Word ranking that alternates between fields 0: word scores Sc(j)S_c(j) for each field (j=0,…,nw−1j=0,…,n_w-1), an order of fields (c1,…,cnf)(c_1,…,c_n_f), upper limit on the number of words L=200L=200 1: For each field c, πc←argsortj↓Sc(j) _c _jS_c(j), pc←0p_c← 0; ℛ←()R←() 2: while |ℛ|<L|R|<L and a word was added to ℛR in the previous round do 3: for l=1,…,nfl=1,…,n_f do 4: c←clc← c_l, and pc←pc+1p_c← p_c+1 while πc(pc)∈ℛ _c(p_c) 5: if pc<nwp_c<n_w and |ℛ|<L|R|<L then 6: append πc(pc) _c(p_c) to the end of ℛR, and pc←pc+1p_c← p_c+1 7: end if 8: end for 9: end while 10: return ℛR (the words for dimension d are the first d words) Algorithm 16 Word scores for Cora 0: Bag-of-Words BBoW∈0,12708×1433B^BoW∈\0,1\^2708× 1433, fields c(i)∈0,…,6c(i)∈\0,…,6\ 1: For each field c, nc←|i:c(i)=c|n_c←|\i:c(i)=c\|, Sc(j)←1nc∑i:c(i)=cBijBoWS_c(j)← 1n_c _i:c(i)=cB^BoW_ij (occurrence rate) 2: Sort the fields in descending order of ncn_c (ties by increasing field index), apply Algorithm 15, and let ℛR be its output 3: return ℛR Algorithm 17 Word scores for ogbn-arxiv 0: titles and abstracts of all papers i=0,…,ntot−1i=0,…,n_tot-1 (ntot=169343n_tot=169343, including papers from 2020), fields c(i)∈0,…,39c(i)∈\0,…,39\, a set of stop words stopW_stop 1: For each paper i, concatenate the title and the abstract and convert them to lowercase; let WiW_i be the set of words, extracted as maximal runs of alphabetic characters, that have length at least 3 and are not in stopW_stop 2: Sort the words that appear in at least 20 papers in lexicographic order, and denote them by w0,…,wnw−1w_0,…,w_n_w-1 3: For each field c, nc←|i:c(i)=c|n_c←|\i:c(i)=c\| (all years), Sc(j)←|i:c(i)=c,wj∈Wi|nc−|i:wj∈Wi|ntotS_c(j)← |\i:c(i)=c,\ w_j∈ W_i\|n_c- |\i:w_j∈ W_i\|n_tot (distinctiveness) 4: Sort the fields in descending order of ncn_c (ties by increasing field index), apply Algorithm 15, and let ℛR be its output 5: return ℛR Algorithm 18 Instance generation for Cora (episode e, dimension d) 0: neighborhoods (⋅)N(·) of the citation graph made undirected with self-loops removed, word ranking ℛR, N=20N=20, r=3r=3 1: Let (C0,…,Cncand−1)(C_0,…,C_n_cand-1) be the sequence of papers v with 3≤|(v)|≤103≤|N(v)|≤ 10, sorted in increasing order of paper index 2: Choose a subset from the uniform distribution over all N-element subsets of 0,…,ncand−1\0,…,n_cand-1\, and denote its elements in increasing order by I1<⋯<INI_1<·s<I_N 3: for n=1,…,Nn=1,…,N do 4: v←CInv← C_I_n, Esn←(v)E_s^n (v), Psn←(Esn∪⋃u∈Esn(u))∖vP_s^n← (E_s^n∪ _u∈ E_s^nN(u) ) \v\ 5: Aij←Bi,ℛjBoWA_ij← B^BoW_i,R_j (i∈Psni∈ P_s^n, j=1,…,dj=1,…,d), ksn←|Esn|k_s^n←|E_s^n|, xjn←min(r,∑i∈EsnAij)x^n_j← (r, _i∈ E_s^nA_ij ) 6: end for 7: return (sn,xn)n=1N\(s^n,x^n)\_n=1^N Algorithm 19 Instance generation for ogbn-arxiv (episode e, dimension d) 0: set of directed edges ℰE ((u,w)∈ℰ(u,w) means that u cites w), publication year t(⋅)t(·), field c(⋅)c(·), word ranking ℛR, N=20N=20, r=5r=5 1: Ref(u):=w:(u,w)∈ℰ,t(w)≤2019,w≠uRef(u):=\w:(u,w) ,\ t(w)≤ 2019,\ w≠ u\ 2: For each field c, let GcG_c be the sequence of papers v with t(v)=2019t(v)=2019, c(v)=c(v)=c, and 5≤|Ref(v)|≤85≤|Ref(v)|≤ 8, sorted in increasing order of paper index, and let mc←|v:t(v)=2019,c(v)=c|m_c←|\v:t(v)=2019,\ c(v)=c\| 3: Sort the fields in descending order of mcm_c (ties by increasing field index), keep only those with |Gc|≥N|G_c|≥ N, let c be the e-th field of this sequence (0-indexed), and set (G0,…,Gncand−1)←Gc(G_0,…,G_n_cand-1)← G_c 4: Choose a subset from the uniform distribution over all N-element subsets of 0,…,ncand−1\0,…,n_cand-1\, and denote its elements in increasing order by I1<⋯<INI_1<·s<I_N 5: for n=1,…,Nn=1,…,N do 6: v←GInv← G_I_n, Esn←Ref(v)E_s^n (v), Psn←(Esn∪⋃w∈EsnRef(w))∖vP_s^n← (E_s^n∪ _w∈ E_s^nRef(w) ) \v\ 7: Aij←[word ℛj appears in paper i]A_ij←[word R_j appears in paper i] (i∈Psni∈ P_s^n, j=1,…,dj=1,…,d), ksn←|Esn|k_s^n←|E_s^n|, xjn←min(r,∑i∈EsnAij)x^n_j← (r, _i∈ E_s^nA_ij ) 8: end for 9: return (sn,xn)n=1N\(s^n,x^n)\_n=1^N H.7. Results for each dimension on the knapsack problem and the weighted multi-cover problem For the knapsack problem and the weighted multi-cover problem, Tables 4 and 5 show, for each dimension, the median of ℓOP ^OP of the seven methods at B oracle calls. Figure 2 shows the evolution of the worst 10% of ℓOP ^OP (best-so-far) on the knapsack problem. Figure 2. Relation between the worst 10% of ℓOP ^OP (best-so-far) and the number of oracle calls on the knapsack problem. The vertical axis and the lines are drawn as in Figure 1. Table 4. Knapsack problem: median of ℓOP ^OP at B=4000B=4000 oracle calls (the attainment rate [%] of ℓOP=0 ^OP=0 in parentheses). 50 episodes for each d. For each optimization method, the pair of the OS loss (proposed) and the suboptimality loss (existing) is shown, and the one with the smaller median in each pair (with the higher attainment rate in case of a tie) is shown in bold. Method d=20d=20 d=40d=40 d=60d=60 d=80d=80 d=100d=100 OS-PSG 0.00 (98) 0.00 (96) 0.00 (92) 0.00 (90) 0.00 (76) Sub-PSG 0.15 (16) 0.20 (0) 0.23 (0) 0.28 (0) 0.35 (0) OS-MetaGrad(FB) 0.00 (86) 0.00 (52) 0.08 (46) 0.35 (8) 0.38 (12) Sub-MetaGrad(FB) 0.15 (6) 0.40 (0) 0.65 (0) 0.80 (0) 0.95 (0) OS-ONS(FB) 0.10 (34) 0.40 (2) 0.70 (0) 0.95 (0) 1.15 (0) Sub-ONS(FB) 0.15 (0) 0.40 (0) 0.62 (0) 0.70 (0) 0.88 (0) OS-DC 0.00 (90) 0.00 (88) 0.00 (86) 0.00 (82) 0.00 (84) Table 5. Weighted multi-cover problem (left: Cora, right: ogbn-arxiv): median of ℓOP ^OP at B=4000B=4000 oracle calls. 20 episodes for each d. For each optimization method, the pair of the OS loss (proposed) and the suboptimality loss (existing) is shown, and the one with the smaller median in each pair is shown in bold. Cora (d) ogbn-arxiv (d) Method 20 40 60 80 100 20 40 60 80 100 OS-PSG 0.78 2.58 4.40 5.72 6.82 0.57 2.40 4.30 6.50 9.30 Sub-PSG 1.25 3.00 4.95 6.25 7.47 1.12 2.95 4.85 7.22 9.40 OS-MetaGrad(FB) 0.97 2.92 5.25 6.75 7.80 0.78 2.73 4.65 7.18 9.55 Sub-MetaGrad(FB) 1.27 3.23 5.18 7.05 8.15 1.15 2.97 4.85 7.20 9.47 OS-ONS(FB) 1.00 3.05 5.22 6.88 8.00 0.82 2.67 4.68 6.92 9.43 Sub-ONS(FB) 1.33 3.23 5.20 6.82 8.38 1.15 2.97 4.85 7.00 9.43 OS-DC 0.82 2.60 4.40 5.83 7.30 0.65 2.52 4.53 7.00 9.35 H.8. Results on the single-machine scheduling problem and the movie selection problem with viewing costs Tables 6 and 7 show, for each dimension, ℓOP ^OP at B oracle calls for the three pairs that differ only in the loss function and for OS-DC, and Figure 3 shows its trajectory on the worst 10% of the episodes. Table 6. Single-machine scheduling problem: median of ℓOP ^OP at B=2000B=2000 oracle calls (the attainment rate [%] of ℓOP=0 ^OP=0 in parentheses). 25 episodes for each d. For each optimization method, the pair of the OS loss (proposed) and the suboptimality loss (existing) is shown, and the one with the smaller median in each pair (with the higher attainment rate in case of a tie) is shown in bold. Method d=4d=4 d=6d=6 d=8d=8 OS-PSG 0.00 (100) 0.00 (100) 0.00 (96) Sub-PSG 0.00 (88) 0.00 (72) 0.10 (44) OS-MetaGrad(FB) 0.00 (100) 0.00 (100) 0.00 (96) Sub-MetaGrad(FB) 0.00 (84) 0.00 (76) 0.30 (28) OS-ONS(FB) 0.00 (100) 0.00 (100) 0.00 (84) Sub-ONS(FB) 0.00 (84) 0.00 (52) 0.30 (40) OS-DC 0.00 (100) 0.00 (96) 0.00 (96) Table 7. Movie selection problem with viewing costs: median of ℓOP ^OP at B=4000B=4000 oracle calls. 20 episodes for each d. For each optimization method, the pair of the OS loss (proposed) and the suboptimality loss (existing) is shown, and the one with the smaller median in each pair is shown in bold. Method d=20d=20 d=40d=40 d=60d=60 d=80d=80 d=100d=100 OS-PSG 0.72 [0.59,1.01] 2.52 [2.02,2.90] 4.05 [3.65,4.22] 5.18 [4.57,5.71] 6.50 [5.65,7.09] Sub-PSG 0.95 [0.70,1.11] 2.85 [2.44,3.17] 4.60 [4.40,5.11] 5.80 [5.39,6.53] 7.57 [6.84,7.98] OS-MetaGrad(FB) 0.95 [0.79,1.10] 2.70 [2.60,3.10] 4.50 [4.09,4.79] 5.65 [5.42,6.19] 7.10 [6.50,7.75] Sub-MetaGrad(FB) 1.30 [1.12,1.52] 3.25 [2.91,3.42] 4.88 [4.67,5.20] 6.18 [5.85,6.53] 7.90 [6.66,8.49] OS-ONS(FB) 1.10 [0.85,1.30] 3.15 [2.91,3.31] 4.95 [4.49,5.16] 6.45 [6.01,6.76] 8.00 [7.08,8.49] Sub-ONS(FB) 1.35 [1.19,1.51] 3.27 [2.95,3.40] 4.95 [4.67,5.15] 6.17 [5.85,6.65] 7.72 [6.96,8.26] OS-DC 0.62 [0.50,0.85] 2.30 [2.05,2.96] 4.20 [3.89,4.36] 5.28 [4.99,5.81] 6.83 [6.12,7.15] (a) Single-machine scheduling problem (b) Movie selection problem with viewing costs Figure 3. Relation between the worst 10% of ℓOP ^OP (best-so-far) and the number of oracle calls on the single-machine scheduling problem and the movie selection problem with viewing costs. The vertical axis and the lines are drawn as in Figure 1. Single-machine scheduling problem In all three pairs and in all dimensions, the OS-loss variant achieved a higher attainment rate (Table 6). The attainment rates of the four methods with the OS loss are 84–100%, whereas those of the suboptimality-loss variant decrease with the dimension; at d=8d=8, Sub-PSG attains 44%, Sub-MetaGrad(FB) 28%, and Sub-ONS(FB) 40% (the OS-loss variants of the same pairs attain 96%, 96%, and 84%, respectively). Aggregated over all dimensions, the attainment rates of the eight methods with the OS loss are 91–99%, which exceed those of the five existing methods including the online versions (23–68%) (Table 9). Comparing on the same instances, among the 75 episodes, the numbers of episodes in which ℓOP ^OP of OS-PSG is smaller than that of Sub-PSG, Sub-ONS, and Sub-MetaGrad are 24, 58, and 36, respectively, and the numbers of episodes in which it is larger are 0, 0, and 0 (for OS-DC, the numbers of episodes with smaller ℓOP ^OP are 24, 58, and 36, and those with larger ℓOP ^OP are 2, 0, and 0). In the three pairs that differ only in the loss function, the OS-loss variant loses in only 3 episodes in total (Table 1). The remaining episodes are ties. As in the knapsack problem, in this setting where the actions of the agent are degraded optimal solutions, the advantage of the proposed method becomes more pronounced in higher dimensions. Movie selection problem with viewing costs In this problem, the actions of the agent are real data, and since we do not assume the existence of a weight satisfying Equation 3.2, we compare the values of ℓOP ^OP (Table 10). In the comparison that replaces only the loss function, the OS-loss variant won in all three pairs: 97 wins and 1 loss in the PSG pair, 90 wins and 1 loss in the MetaGrad(FB) pair, and 45 wins and 32 losses in the ONS(FB) pair (Table 1). Looking at each dimension, in the PSG pair and the MetaGrad(FB) pair, the median of the OS-loss variant is smaller in all dimensions (Table 7). In particular, in the PSG pair the difference widens with the dimension; at d=100d=100, OS-PSG attains 6.506.50 whereas Sub-PSG attains 7.577.57. In the ONS(FB) pair, the median of OS-ONS(FB) is smaller for d≤40d≤ 40, the two are equal at d=60d=60, and the median of Sub-ONS(FB) is smaller for d≥80d≥ 80. Among the seven methods in the table, the smallest median is attained by OS-DC (0.620.62, 2.302.30) for d=20,40d=20,40 and by OS-PSG (4.054.05, 5.185.18, 6.506.50) for d=60,80,100d=60,80,100; that is, it is attained by a method with the OS loss in all dimensions. In other words, under the same optimization method, merely replacing the suboptimality loss with the OS loss improves the prediction of solutions that outperform the agent. Note that the online versions Sub-ONS and Sub-MetaGrad, which update state by state, have medians of 3.5503.550 and 3.4253.425 over all dimensions, respectively, which are smaller than that of OS-PSG (3.9753.975) (Table 10). Since the medians of the full-batch versions of the same learners, Sub-ONS(FB) and Sub-MetaGrad(FB), are 4.9504.950 and 4.8754.875, respectively, this difference is presumably due not to the use of the suboptimality loss but to the optimization method, namely online learning that updates state by state. Among the proposed methods, the median of OS-DC[ONS] over all dimensions (3.5503.550) equals that of Sub-ONS (3.5503.550), and its median computation time is 3131 seconds, about one third of that of Sub-ONS (9191 seconds). Combining the OS loss with methods that minimize it state by state via online learning is left for future work. H.9. Results of all methods on the five experiments Results of the methods not included in the main text On the knapsack problem, comparing on the same instances, among the 250 episodes, the numbers of episodes in which ℓOP ^OP of OS-PSG is smaller than that of Sub-PSG, Sub-ONS, and Sub-MetaGrad are 241, 227, and 243, respectively, and there is no episode in which it is larger (for OS-DC, the numbers of episodes with smaller ℓOP ^OP are 240, 222, and 242, and those with larger ℓOP ^OP are 1, 3, and 0). Here, Sub-ONS and Sub-MetaGrad are the online versions that update state by state, and their attainment rates are also 9% and 3%, respectively, over all dimensions (Table 8). The empirical mean of the OS loss measured at the same weights is also 00 in median for OS-PSG and OS-DC, whereas it remains at 3.9×10−43.9× 10^-4–2.8×10−32.8× 10^-3 for Sub-PSG, Sub-ONS, and Sub-MetaGrad (Table 8). On the weighted multi-cover problem, unlike the movie selection problem with viewing costs (Section H.8), the medians of ℓOP ^OP of the online versions Sub-ONS and Sub-MetaGrad also fall short of the best method with the OS loss. Among all 13 methods, the smallest median of ℓOP ^OP over all dimensions is attained by OS-DC[LV+CPF]+SGF (4.0004.000) on Cora and by OS-DC[ONS] (3.5753.575) on ogbn-arxiv, which are smaller than the best of the existing methods (4.5004.500 of Sub-ONS on Cora and 3.9503.950 of Sub-MetaGrad on ogbn-arxiv) (Tables 11 and 12). Comparing on the same instances the pairs of the proposed method and the existing method with the smallest mean of ℓOP ^OP over all episodes (OS-DC[LV+CPF]+SGF vs. Sub-MetaGrad on Cora, and OS-DC[MetaGrad] vs. Sub-MetaGrad on ogbn-arxiv), the proposed method wins 65 times and loses 30 times on Cora, and wins 85 times and loses 11 times on ogbn-arxiv. The median computation time of OS-PSG is 8.78.7 seconds on Cora (second to 8.28.2 seconds of OS-ONS(FB)) and 7.47.4 seconds on ogbn-arxiv (the shortest among all methods), about one quarter of that of Sub-MetaGrad (35.435.4 seconds and 31.131.1 seconds); note that the time of Sub-MetaGrad includes the solves used only for recording, which OS-PSG does not need (see the definition of “Time” below). Among the three pairs differing only in the loss function, in the two full-batch pairs, although the OS-loss variant won overall, it lost in 17 episodes (MetaGrad(FB)) and 22 episodes (ONS(FB)) on Cora, so the margin is smaller than in the PSG pair (Table 1). Tables 8, 9, 10, 11 and 12 show the results of all 13 methods on the five experiments. In each table, the upper block shows existing inverse optimization (the suboptimality loss), and the lower block shows outperformance inverse optimization (the OS loss). In the method names, the bracket of OS-DC[⋅·] indicates the inner solver, and +SGF indicates that subgradient finishing is added. Each value is the median, over all dimensions and episodes, of the value at B oracle calls; for ℓOP ^OP, the interquartile range (the first and third quartiles) is also shown. ℓ¯OS ^OS is measured at the weight that gives the best-so-far ℓOP ^OP (Section 7.1). “Time” is the computation time until the run stops, that is, until the number of oracle calls reaches B or ℓOP=0 ^OP=0 is attained, whichever comes first (the part after that is not included), and its mean and median are shown. It also includes the time for the solves used only for recording and not counted as oracle calls, that is, the solves of x∗(θ,xn,sn)x^*(θ;x^n,s^n) for recording ℓ¯OS ^OS in the Sub methods and OS-DC. Hence, in comparisons with the OS methods other than OS-DC, which do not need these solves, the times of the Sub methods are overestimated. Table 8. Knapsack problem (oracle-call budget B=4000B=4000). Each value is the median, over the 250 runs (d∈20,40,60,80,100d∈\20,40,60,80,100\, 50 episodes for each d), of the value at B oracle calls, and the attainment rate is the fraction of these 250 runs that attained ℓOP=0 ^OP=0. For the time (in seconds), the mean is also shown. Method ℓOP ^OP median interquartile range attainment rate [%] ℓ¯OS ^OS median time mean time median Sub-PSG 0.250 [0.150,0.300] 3 4.9×10−44.9× 10^-4 1.3 1.3 Sub-ONS 0.150 [0.100,0.250] 9 3.9×10−43.9× 10^-4 2.2 2.2 Sub-MetaGrad 0.350 [0.200,0.550] 3 2.8×10−32.8× 10^-3 8.4 8.3 Sub-ONS(FB) 0.600 [0.300,0.800] 0 6.2×10−36.2× 10^-3 1.4 1.4 Sub-MetaGrad(FB) 0.550 [0.300,0.850] 1 5.5×10−35.5× 10^-3 2.4 2.4 OS-PSG 0.000 [0.000,0.000] 90 00 0.3 0.2 OS-ONS(FB) 0.700 [0.300,1.000] 7 8.9×10−38.9× 10^-3 0.7 0.7 OS-MetaGrad(FB) 0.100 [0.000,0.450] 41 4.8×10−44.8× 10^-4 0.9 0.8 OS-DC[ONS] 0.100 [0.000,0.250] 26 5.2×10−45.2× 10^-4 1.3 1.3 OS-DC[MetaGrad] 0.050 [0.000,0.150] 44 1.1×10−41.1× 10^-4 1.8 1.5 OS-DC[CP]+SGF 0.650 [0.213,1.050] 12 7.2×10−37.2× 10^-3 1.7 1.7 OS-DC[LV+CPF]+SGF 0.000 [0.000,0.000] 85 00 0.9 0.7 OS-DC[PB+CPF]+SGF 0.000 [0.000,0.000] 86 00 0.7 0.6 Table 9. Single-machine scheduling problem (oracle-call budget B=2000B=2000). Each value is the median, over the 75 runs (d∈4,6,8d∈\4,6,8\, 25 episodes for each d), of the value at B oracle calls, and the attainment rate is the fraction of these 75 runs that attained ℓOP=0 ^OP=0. For the time (in seconds), the mean is also shown. Method ℓOP ^OP median interquartile range attainment rate [%] ℓ¯OS ^OS median time mean time median Sub-PSG 0.000 [0.000,0.150] 68 00 17.7 2.8 Sub-ONS 0.500 [0.100,1.350] 23 1.5×10−21.5× 10^-2 32.6 17.3 Sub-MetaGrad 0.000 [0.000,0.300] 52 00 24.9 11.8 Sub-ONS(FB) 0.000 [0.000,0.300] 59 00 24.6 13.7 Sub-MetaGrad(FB) 0.000 [0.000,0.200] 63 00 19.7 1.2 OS-PSG 0.000 [0.000,0.000] 99 00 6.3 1.1 OS-ONS(FB) 0.000 [0.000,0.000] 95 00 10.3 1.2 OS-MetaGrad(FB) 0.000 [0.000,0.000] 99 00 6.3 1.5 OS-DC[ONS] 0.000 [0.000,0.000] 91 00 19.6 1.9 OS-DC[MetaGrad] 0.000 [0.000,0.000] 93 00 18.7 3.3 OS-DC[CP]+SGF 0.000 [0.000,0.000] 97 00 14.4 3.4 OS-DC[LV+CPF]+SGF 0.000 [0.000,0.000] 95 00 9.9 1.7 OS-DC[PB+CPF]+SGF 0.000 [0.000,0.000] 97 00 10.9 2.2 Table 10. Movie selection problem with viewing costs (oracle-call budget B=4000B=4000). Each value is the median, over the 100 runs (d∈20,40,60,80,100d∈\20,40,60,80,100\, 20 episodes for each d), of the value at B oracle calls, and the attainment rate is the fraction of these 100 runs that attained ℓOP=0 ^OP=0. For the time (in seconds), the mean is also shown. Method ℓOP ^OP median interquartile range attainment rate [%] ℓ¯OS ^OS median time mean time median Sub-PSG 4.600 [2.438,6.000] 0 3.1×10−23.1× 10^-2 106.7 95.3 Sub-ONS 3.550 [1.950,4.612] 0 2.0×10−22.0× 10^-2 98.9 91.2 Sub-MetaGrad 3.425 [1.900,4.600] 0 2.4×10−22.4× 10^-2 107.4 97.5 Sub-ONS(FB) 4.950 [2.950,6.312] 0 5.0×10−25.0× 10^-2 39.6 40.4 Sub-MetaGrad(FB) 4.875 [2.913,6.300] 0 5.4×10−25.4× 10^-2 76.1 62.5 OS-PSG 3.975 [2.025,5.362] 0 1.2×10−21.2× 10^-2 75.7 81.1 OS-ONS(FB) 4.950 [2.913,6.487] 0 4.9×10−24.9× 10^-2 7.0 7.1 OS-MetaGrad(FB) 4.500 [2.600,5.850] 0 7.5×10−27.5× 10^-2 18.0 17.4 OS-DC[ONS] 3.550 [2.000,5.013] 0 3.9×10−23.9× 10^-2 21.6 23.0 OS-DC[MetaGrad] 3.900 [2.188,5.312] 0 5.5×10−25.5× 10^-2 23.2 23.6 OS-DC[CP]+SGF 4.750 [2.700,6.475] 0 4.3×10−24.3× 10^-2 16.2 16.0 OS-DC[LV+CPF]+SGF 4.150 [1.938,5.463] 0 3.4×10−33.4× 10^-3 115.0 110.3 OS-DC[PB+CPF]+SGF 4.200 [2.050,5.450] 0 1.0×10−21.0× 10^-2 105.2 102.9 Table 11. Weighted multi-cover problem (Cora) (oracle-call budget B=4000B=4000). Each value is the median, over the 100 runs (d∈20,40,60,80,100d∈\20,40,60,80,100\, 20 episodes for each d), of the value at B oracle calls, and the attainment rate is the fraction of these 100 runs that attained ℓOP=0 ^OP=0. For the time (in seconds), the mean is also shown. Method ℓOP ^OP median interquartile range attainment rate [%] ℓ¯OS ^OS median time mean time median Sub-PSG 4.800 [2.787,6.338] 0 7.0×10−27.0× 10^-2 18.9 16.9 Sub-ONS 4.500 [2.688,6.100] 0 7.9×10−27.9× 10^-2 17.9 16.9 Sub-MetaGrad 4.600 [2.537,6.075] 0 9.1×10−29.1× 10^-2 40.8 35.4 Sub-ONS(FB) 5.150 [2.888,6.812] 0 1.1×10−11.1× 10^-1 17.7 16.2 Sub-MetaGrad(FB) 5.150 [2.975,7.075] 0 1.1×10−11.1× 10^-1 21.8 20.3 OS-PSG 4.375 [2.362,5.900] 0 6.3×10−26.3× 10^-2 8.9 8.7 OS-ONS(FB) 5.150 [2.650,6.963] 0 1.4×10−11.4× 10^-1 8.8 8.2 OS-MetaGrad(FB) 5.125 [2.712,6.975] 0 1.1×10−11.1× 10^-1 10.5 10.1 OS-DC[ONS] 4.350 [2.337,5.663] 0 7.9×10−27.9× 10^-2 17.2 16.4 OS-DC[MetaGrad] 4.100 [2.375,5.713] 0 7.1×10−27.1× 10^-2 20.1 19.4 OS-DC[CP]+SGF 4.775 [2.300,7.075] 0 8.3×10−28.3× 10^-2 13.5 12.3 OS-DC[LV+CPF]+SGF 4.000 [1.550,6.100] 0 4.6×10−24.6× 10^-2 15.3 13.7 OS-DC[PB+CPF]+SGF 4.400 [2.287,6.150] 0 5.1×10−25.1× 10^-2 15.1 13.7 Table 12. Weighted multi-cover problem (ogbn-arxiv) (oracle-call budget B=4000B=4000). Each value is the median, over the 100 runs (d∈20,40,60,80,100d∈\20,40,60,80,100\, 20 episodes for each d), of the value at B oracle calls, and the attainment rate is the fraction of these 100 runs that attained ℓOP=0 ^OP=0. For the time (in seconds), the mean is also shown. Method ℓOP ^OP median interquartile range attainment rate [%] ℓ¯OS ^OS median time mean time median Sub-PSG 4.650 [2.612,7.375] 0 8.4×10−28.4× 10^-2 17.2 14.2 Sub-ONS 4.475 [2.663,6.650] 0 3.4×10−23.4× 10^-2 16.3 14.6 Sub-MetaGrad 3.950 [2.012,6.013] 0 7.2×10−27.2× 10^-2 36.5 31.1 Sub-ONS(FB) 4.650 [2.650,7.312] 0 1.1×10−11.1× 10^-1 18.0 14.1 Sub-MetaGrad(FB) 4.650 [2.650,7.537] 0 1.1×10−11.1× 10^-1 21.4 17.3 OS-PSG 4.200 [1.738,7.088] 0 4.7×10−24.7× 10^-2 8.5 7.4 OS-ONS(FB) 4.625 [2.300,7.188] 0 1.2×10−11.2× 10^-1 9.1 7.5 OS-MetaGrad(FB) 4.625 [2.325,7.263] 0 1.2×10−11.2× 10^-1 10.4 8.6 OS-DC[ONS] 3.575 [1.700,5.812] 0 5.9×10−25.9× 10^-2 17.5 14.4 OS-DC[MetaGrad] 3.600 [1.688,5.562] 0 6.9×10−26.9× 10^-2 19.4 16.8 OS-DC[CP]+SGF 4.550 [2.438,7.500] 0 8.2×10−28.2× 10^-2 10.0 8.6 OS-DC[LV+CPF]+SGF 4.325 [1.887,7.050] 0 1.2×10−41.2× 10^-4 11.7 9.8 OS-DC[PB+CPF]+SGF 4.225 [1.938,7.125] 0 1.4×10−21.4× 10^-2 10.8 9.2 On the knapsack problem, OS-PSG (attainment rate 90%), OS-DC[PB+CPF]+SGF (86%), and OS-DC[LV+CPF]+SGF (85%) attained a median of ℓOP ^OP equal to 00 and outperform the five existing methods (attainment rates 0–9%). The attainment rates of OS-DC[CP]+SGF and OS-ONS(FB) are 12% and 7%, respectively. On the single-machine scheduling problem, the attainment rates of the eight methods with the OS loss are 91–99%, which exceed those of the five existing methods (23–68%). On the movie selection problem with viewing costs, the attainment rates of all methods are 0%; the median of ℓOP ^OP is the smallest for Sub-MetaGrad (3.4253.425), followed by Sub-ONS and OS-DC[ONS] (both 3.5503.550). On this problem, OS-DC[LV+CPF]+SGF, which has the smallest empirical mean of the OS loss (median 3.4×10−33.4× 10^-3), has a median of ℓOP ^OP equal to 4.1504.150, which is larger than that of Sub-ONS (median empirical mean of the OS loss 2.0×10−22.0× 10^-2). This is why we use ℓOP ^OP as the main evaluation metric in addition to the empirical mean of the OS loss (Section 4). On the weighted multi-cover problem, the attainment rates of all methods are 0%, and the smallest median of ℓOP ^OP is attained by OS-DC[LV+CPF]+SGF (4.0004.000) on Cora and by OS-DC[ONS] (3.5753.575) on ogbn-arxiv. On ogbn-arxiv, OS-DC[LV+CPF]+SGF has the smallest median empirical mean of the OS loss among all methods, 1.2×10−41.2× 10^-4, but its median of ℓOP ^OP is 4.3254.325, which is larger than that of OS-DC[ONS].