Paper deep dive
Learnable Randomization as Commitment Against Adaptive Optimizers
Zihan Deng, Chuanzhi Xu, Xiaozhen Zhong, Haoyang Li, Junjie Huang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A pricing page can walk the posted price up to the last amount a buyer still accepts, a recommender can hold back a better item for a barely acceptable promoted one, and a classifier can shift its boundary once applicants change their features. The system predicts the response and then picks the menu that serves its own objective, so the surplus above the user's cutoff is taken. Playing the single best action publishes that cutoff, while noise on actions the user would never take throws away payoff and teaches the platform that a worse menu is still acceptable. We study unpredictable near-optimal policies (UNOP), which mix uniformly on near-best actions that remain individually rational. The mixture is a commitment about the response. On a finite price grid, when the best sure-demand price strictly out-earns the randomized band, a seller who already knows the curve posts below the band, and the purchase that occurs is deterministic. Knowing that curve is not the same as predicting the next draw. The mixture can be learned and the optimizer can match its best response, while the user's payoff stays higher because the mixture changes which action is targeted. In pricing and in policy-aware recommendation this leaves more surplus than greedy play when the platform optimizes against the curve and more than one action is acceptable. The gain goes away under quality ranking, a singleton near-optimal set, a wrong utility estimate, or a short-horizon explorer. That is also where mixing should be turned off if the other side is trying to cooperate.
Tags
Links
- Source: https://arxiv.org/abs/2609.32772v1
- Canonical: https://arxiv.org/abs/2609.32772v1
Trouble viewing inline? Open PDF directly →
Full Text
130,127 characters extracted from source content.
Expand or collapse full text
Learnable Randomization as Commitment Against Adaptive Optimizers Zihan Deng †thanks: Corresponding author: zihandeng@connect.hku.hk. Affiliation: The University of Hong Kong Chuanzhi Xu Affiliation: The University of Sydney Xiaozhen Zhong Affiliation: University of Electronic Science and Technology of China Haoyang Li Affiliation: The University of Sydney Junjie Huang Affiliation: University of Science and Technology of China Code Abstract A pricing page can walk the posted price up to the last amount a buyer still accepts, a recommender can hold back a better item for a barely acceptable promoted one, and a classifier can shift its boundary once applicants change their features. The system predicts the response and then picks the menu that serves its own objective, so the surplus above the user’s cutoff is taken. Playing the single best action publishes that cutoff, while noise on actions the user would never take throws away payoff and teaches the platform that a worse menu is still acceptable. We study unpredictable near-optimal policies (UNOP), which mix uniformly on near-best actions that remain individually rational. The mixture is a commitment about the response. On a finite price grid, when the best sure-demand price strictly out-earns the randomized band, a seller who already knows the curve posts below the band, and the purchase that occurs is deterministic. Knowing that curve is not the same as predicting the next draw. The mixture can be learned and the optimizer can match its best response, while the user’s payoff stays higher because the mixture changes which action is targeted. In pricing and in policy-aware recommendation this leaves more surplus than greedy play when the platform optimizes against the curve and more than one action is acceptable. The gain goes away under quality ranking, a singleton near-optimal set, a wrong utility estimate, or a short-horizon explorer. That is also where mixing should be turned off if the other side is trying to cooperate. 1 Introduction Figure 1: (A) A human and an adaptive optimizer in a closed loop. The optimizer may learn the response distribution and still revise the menu. (B) A greedy threshold publishes the user’s cutoff. (C) Uniform noise refuses good actions and destroys payoff. The protective randomness in pricing is the response the buyer would give on menus the optimizer then declines to offer. A mixed security strategy is required in zero-sum play, while mixing is a coordination tax when interests coincide. A growing fraction of human decisions are taken next to an adaptive algorithm, whether a platform sets a price after watching who buys (Calvano et al., 2020; Braverman et al., 2018), a recommender builds a list after watching who clicks (Dean et al., 2020; Hron et al., 2023; Stray et al., 2024), or a classifier is retrained on the features applicants choose to present (Hardt et al., 2016; Perdomo et al., 2020). In each case the algorithm is an optimizer rather than a fixed environment, because it predicts behavior and then chooses the action that maximizes its own objective given that prediction. Once the prediction is accurate the menu moves with it, so the user can keep accepting the offered price, list, or decision while the surplus above the user’s cutoff is taken by the optimizer. Once the interaction is written as an ordinary learning problem, it is easy to study only how to predict the user or how to extract surplus from a user who has already been predicted. Strategic buyers are usually treated as long-horizon manipulators of a learning seller, or the seller is designed so that the manipulation does not pay (Drutsa, 2018; Drutsa, 2020; Huh and Kandasamy, 2026). A pure best response is a deterministic function of the current menu, so an optimizer that identifies it can post a price just below value, withhold a better organic item in favor of a barely acceptable promoted one, or shift a decision boundary against a known gaming direction. Uniform randomization conceals that function only by collapsing the user’s own payoff, while ε -greedy and Boltzmann exploration put noise on actions that are far from optimal or individually irrational, and an adaptive optimizer can read those loss-making plays as evidence that the current menu is still acceptable. We do not try to hide the response distribution, and the pricing guarantee below assumes the optimizer knows it exactly. Figure 1 shows the closed loop. In pricing, once the seller stays below the randomized band, the action that is actually played is often deterministic, because the randomization only specifies what the buyer would do at prices the seller then chooses not to post. We call this the response-curve commitment effect. The platform can learn the distribution and optimize against the whole curve, yet still be led to a menu that takes less surplus. The question is therefore not how to keep the policy secret. It is which response curve the user can credibly commit to once the platform is free to learn it. The shape of the user’s payoffs decides how much room that commitment has. Let u be the estimated payoff vector and let α(u)=a:u(a)≥maxu−αA_α(u)=\a:u(a)≥ u-α\ be the set of actions that never lose more than α relative to the best one-step action. When the set is a singleton, effectiveness forces a predictable action, whereas a larger set can be mixed so as to maximize entropy or to put less weight on actions the current forecast already expects. We call the resulting family unpredictable near-optimal policies (UNOP). Individual rationality is part of the support, because mixing into a strictly loss-making action, such as buying above value or clicking below reservation, teaches the optimizer that the cutoff can still be moved. UNOP therefore mixes only inside α(u)A_α(u) and above a reservation floor. The same payoff shape produces three different conclusions, which we prove and then measure. In zero-sum games such as matching pennies and rock, paper, scissors, an adaptive best responder turns a greedy pure policy into a security-level loss, so unpredictability is what makes the user effective (Nash, 1950; Osborne and Rubinstein, 1994). In common-interest games such as coordination and stag hunt, the Pareto-optimal pure profile is already effective and fully predictable, so mixing is only a coordination tax and being read is what good play looks like when the other side is a collaborator (Carroll et al., 2019; Hadfield-Menell et al., 2016). In surplus extraction the platform identifies a threshold and sits on the user’s cutoff (Hardt et al., 2022). A greedy user gives that cutoff away and a uniform user refuses both the cutoff and the surplus, while UNOP blurs only the boundary, with sure demand below v−αv-α, sure refusal above v, and mixing in between. Under Proposition 4 this preserves Ω(α) (α)-scale pricing surplus, and the same support restriction forces a margin-seeking list to trade acceptance against user quality. Our contributions can be summarized as follows: • We separate distributional learning error, realized-action predictability, optimizer regret relative to an exact best response, and the surplus that best response extracts. • We characterize constant-band commitments on a finite price grid. UNOP is the maximum-entropy support-safe response. It induces the same posted price as the welfare-maximizing band rate precisely when purchase rate 1/21/2 strictly keeps the seller below the band. • We show that predictors can learn the mixture and match an oracle best response, while user surplus stays above greedy play in pricing and under a policy-aware recommender. Quality ranking reverses that comparison, a singleton α-set on every feasible menu removes it, and large utility noise or a short-horizon explorer makes the protection fail. 2 Related work Learning in Games and Strategizing against Learners. Classical learning in games studies no-regret dynamics and their equilibria (Fudenberg and Levine, 1998; Cesa-Bianchi and Lugosi, 2006). Weighted-majority and exponential-weights bounds make the same point (Littlestone and Warmuth, 1994; Freund and Schapire, 1997). A recent line asks the converse: how should a patient optimizer play against a no-regret learner (Deng et al., 2019; Braverman et al., 2018; Mansour et al., 2022; Camara et al., 2020; Haghtalab et al., 2022; Kolumbus and Nisan, 2022)? The typical conclusion is that the optimizer can secure at least the Stackelberg value, and sometimes more against mean-based learners. Repeated posted-price auctions with a strategic buyer, and seller algorithms designed to remain robust to that buyer, study the other side of the interaction as long-horizon manipulation or as a robust mechanism (Drutsa, 2018; Drutsa, 2020; Huh and Kandasamy, 2026). We study a simple support-constrained commitment and separate policy learning, realization prediction, optimizer regret, and extraction. Proper scoring rules make the same separation at the level of a forecast: log loss equals entropy plus a divergence, so a high log loss need not mean the forecast missed the distribution (Cover and Thomas, 2006). Performative Prediction and Platforms. Performative prediction (Perdomo et al., 2020; Hardt et al., 2022) and strategic classification (Hardt et al., 2016; Brückner et al., 2012; Dong et al., 2018; Milli et al., 2019; Zrnic et al., 2021; Levanon and Rosenfeld, 2021; Chen et al., 2020; Shavit et al., 2020; Kleinberg and Raghavan, 2019) study distribution shift caused by agent responses. The usual goal is a stable or Stackelberg-optimal predictor. Agent welfare, when it appears, is often a constraint on the institution (Milli et al., 2019; Levanon and Rosenfeld, 2021). We focus on the human side. The same literature, with recommender choice-set design (Dean et al., 2020; Ben-Porat and Tennenholtz, 2018; Hron et al., 2023; Ie et al., 2019; Schnabel et al., 2016) and algorithmic pricing (Calvano et al., 2020; Auer et al., 2002), is where surplus extraction lives. Performative power (Hardt et al., 2022) quantifies how much a platform can steer. We treat that steering as sitting on a predicted threshold. Entropy-Regularized Play and Coordination. Maximum-entropy IRL and soft RL (Ziebart et al., 2008; Haarnoja et al., 2018; Jaynes, 1957), quantal response (McKelvey and Palfrey, 1995), and information theory (Cover and Thomas, 2006) all justify Gibbs distributions. We use them as the Shannon-Pareto baseline. For a human facing extraction the natural constraint is worst-case utility loss, and the maximum-entropy solution is uniform on the actions within α of the best payoff. Cooperative IRL (Hadfield-Menell et al., 2016; Carroll et al., 2019; Russell, 2019) instead emphasizes making humans easy to predict. We agree when the other side is a collaborator, and we locate where that advice reverses. 3 Method 3.1 Interaction and objectives We consider a T-round interaction between a human ℋH and an adaptive optimizer O. At round t, a context xtx_t is realized and the history ht−1=(xs,as,os)s<th_t-1=(x_s,a_s,o_s)_s<t is public. The human chooses at∈Aa_t∈ A from a finite action set and the optimizer chooses ot∈Oo_t∈ O. Instantaneous payoffs are uH(at,ot,xt)u_H(a_t,o_t,x_t) and uO(at,ot,xt)u_O(a_t,o_t,x_t). The optimizer maintains a predictive distribution Pt(⋅∣xt,ht−1)P_t(· x_t,h_t-1) on A, plays a best response that may still explore ot∈argmaxo∈Oa∼Pt[uO(a,o,xt)]o_t∈ _o∈ O\ E_a P_t [u_O(a,o,x_t) ] (1) or a bandit version of equation 1, such as UCB on posted prices or unique-best recommendation lists, and updates PtP_t from ata_t. In simultaneous-move games the human does not see the optimizer’s current mix, and forms u^t(a)=∑o∈Oσ^t(o)uH(a,o,xt), u_t(a)\;=\; _o∈ O σ_t(o)\,u_H(a,o,x_t), (2) where σ^t σ_t is the empirical mix of past optimizer actions in simultaneous bimatrix play. In sequential menus, namely pricing and recommendation, the human observes the posted oto_t before acting, so u^t u_t is the displayed utility of the current menu rather than an average over past menus. Then at∼πt(⋅∣u^t,Pt)a_t _t(· u_t,P_t). Definition 1. The human’s effectiveness is VH=1T∑t=1T[uH(at,ot,xt)]V_H= 1T _t=1^TE[u_H(a_t,o_t,x_t)]. In surplus-extraction domains we report the analogous surplus over a reservation utility. Definition 2. Let π be the user’s mixed policy and P a predictive distribution, with probabilities clipped at 10−1210^-12 before logarithms. Distributional learning error is L(π,P)=KL(π∥P)L(π,P)=KL(π\|P), and realization predictability is the Bayes accuracy R(π)=maxaπ(a)R(π)= _aπ(a), which upper-bounds every predictor’s one-step hit rate. Write VO⋆(π)=maxoa∼π[uO(a,o)]V_O (π)= _oE_a π[u_O(a,o)] for the optimizer’s exact best-response value and o(P)o(P) for the action it plays from P. Optimizer regret is the oracle gap G(π,P)=VO⋆(π)−VO(π,o(P))G(π,P)=V_O (π)-V_O(π,o(P)), while extraction is VO⋆(π)V_O (π) itself, or the user’s surplus loss relative to a menu that is not chosen to extract. A zero gap means only that the learned optimizer has reached the oracle best response, not that the policy cannot be exploited. Proposition 1. For any π and P in the simplex, the best one-step hit rate satisfies maxPℙa∼π(argmaxP=a)=maxaπ(a) _PP_a π( P=a)= _aπ(a). The expected log loss decomposes as a∼π[−logP(a)]=H(π)+KL(π∥P)E_a π[- P(a)]=H(π)+KL(π\|P), so the excess over the entropy is zero if and only if P=πP=π on the support of π. Raw log loss and raw top-1 accuracy therefore cannot, by themselves, show that a predictor failed to learn the mixture. These are operational measures: they are what an adaptive AI system computes. We do not equate unpredictability with differential privacy (Dwork, 2006; Duchi et al., 2013). Write u∈ℝ|A|u ^|A| for the current expected-utility vector, u⋆=maxau(a)u = _au(a), and α(u)=a∈A:u(a)≥u⋆−α.A_α(u)\;=\;\a∈ A:u(a)≥ u -α\. (3) Every action in α(u)A_α(u) loses at most α relative to the best one-step action. If |α(u)|=1|A_α(u)|=1, any policy supported on that singleton is a δ-peak with δ=1δ=1, aside from how ties are broken, and effectiveness forces predictability. Appendix D measures robustness when u u is a noisy version of u. 3.2 Three cases and two tradeoffs Proposition 2. In a finite zero-sum game, any pure action whose security level is below the game value v⋆v is exploitable by an exact best responder. In matching pennies, every deterministic policy has value −1-1. More generally, let =π∈Δ(A):minouH(π,o)≥v⋆S=\π∈ (A): _ou_H(π,o)≥ v \ be the security-strategy polytope. There is a game-dependent constant κG<∞ _G<∞ such that every policy with value at least v⋆−εv - satisfies infπ⋆∈TV(π,π⋆)≤κGε. _π TV(π,π )≤ _G . Thus, in games such as matching pennies and rock, paper, scissors, whose security set contains no peaked policy, sufficiently effective play must remain non-peaked. The constant and the conclusion are game dependent. Zero-sum structure alone does not imply that every game has a unique or fully mixed security strategy. The proof is standard security-level reasoning (Osborne and Rubinstein, 1994). We record it in Appendix B. Empirically, greedy obtains mean payoff −0.143-0.143 in our zero-sum suite, while UNOP sits at 0.0000.000 and Aware-UNOP at +0.054+0.054. Proposition 3. Suppose uH=uO=u_H=u_O=u and (a⋆,o⋆)(a ,o ) is the unique global payoff-maximizing pure profile. Let u⋆=u(a⋆,o⋆)u =u(a ,o ) and u¯=maxa≠a⋆,ou(a,o)<u⋆ u= _a≠ a ,ou(a,o)<u . Then the effectiveness-maximizing human policy is a point mass on a⋆a , paired with o⋆o . For any mixture with π(a⋆)=p<1π(a )=p<1, and for any partner action, including a best response, expected common payoff is at most pu⋆+(1−p)u¯pu +(1-p) u, hence it pays at least the linear coordination tax (1−p)(u⋆−u¯)(1-p)(u - u). A Shannon constraint H(π)≥h>0H(π)≥ h>0 therefore forces p≤p⋆(h)<1p≤ p (h)<1. In our common-interest suite, greedy, UNOP with small α, and Aware-UNOP all achieve payoff 2.5982.598, with predictive accuracy 1.01.0. Uniform falls to 1.7621.762. The two natural ways of asking a policy to be random but not too costly are not interchangeable. One constrains every atom of the support. The other constrains only the mean. Theorem 1. Fix u∈ℝ|A|u ^|A| and write Δ(A) (A) for the simplex. Consider (WCα) (WC_α) maxπ∈Δ(A)H(π)s.t.π(a)=0 whenever u(a)<u⋆−α, _π∈ (A)\ H(π) .t. π(a)=0 whenever u(a)<u -α, (4) (SHh) (SH_h) maxπ∈Δ(A)π⋅us.t.H(π)≥h. _π∈ (A)\ π· u .t. H(π)≥ h. (5) 1. Per-action worst-case loss. (WCα)(WC_α) is uniquely solved by π=Unif(α(u))π=Unif(A_α(u)), with value H=log|α(u)|H= |A_α(u)| and π[u]≥u⋆−αE_π[u]≥ u -α. Conversely, a policy places positive mass outside αA_α if and only if it violates the support-wise guarantee maxa∈supp(π)(u⋆−u(a))≤α _a (π)(u -u(a))≤α. This converse concerns the loss of every action that may be played. A policy with very small mass outside αA_α may still have expected loss below α. 2. Shannon constraint. For any h∈(H(Unif(argmaxu)),log|A|]h∈ (H(Unif( u)), |A| ], (SHh)(SH_h) is uniquely solved by a Gibbs distribution πτ(a)∝exp(u(a)/τ) _τ(a) (u(a)/τ) for some τ>0τ>0 (Jaynes, 1957; Cover and Thomas, 2006). Equivalently, πτ _τ maximises π⋅u+τH(π)π· u+τ H(π) over Δ(A) (A). For h≤H(Unif(argmaxu))h≤ H(Unif( u)), every expected-utility maximizer with entropy at least h is supported on argmaxu u. (WCα)(WC_α) maximises a strictly concave objective on a simplex face and is therefore uniquely solved. (SHh)(SH_h) maximises a linear objective over the strictly convex set of policies whose entropy is at least h. Uniqueness of the Gibbs solution on (H(Unif(argmaxu)),log|A|] (H(Unif( u)), |A| ] follows from that shape. KKT stationarity for (WCα)(WC_α) on the relative interior of Δ(α) (A_α) is −logπ(a)−1+λ=0- π(a)-1+λ=0, so π is constant on αA_α. Equivalently, Unif(α)Unif(A_α) is the I-projection of Unif(A)Unif(A) onto π:supp(π)⊆α\π:supp(π) _α\, since KL(π∥Unif(A))=log|A|−H(π)KL(π\|Unif(A))= |A|-H(π). For (SHh)(SH_h), stationarity of π⋅u+τH(π)π· u+τ H(π) is u(a)−τ(logπ(a)+1)+μ=0u(a)-τ( π(a)+1)+μ=0, hence Gibbs. The programs coincide only in special cases, when a single α-level set is already A, or when h=0h=0. For any τ>0τ>0, πτ(a)>0 _τ(a)>0 for every a. If α≠AA_α≠ A, write γ=mina∉α(u⋆−α−u(a))>0γ= _a _α(u -α-u(a))>0. Matching bounds on the normalizer Z=∑beu(b)/τZ= _be^u(b)/τ give 1|A|exp(−(α+γ)/τ)≤πτ(A∖α)≤|A∖α||α|exp(−γ/τ). 1|A| (-(α+γ)/τ )\;≤\; _τ (A _α )\;≤\; |A _α||A_α| (-γ/τ). (6) The leak is acceptable as exploration in RL. Against an extractor it is a teaching signal. Unrestricted Boltzmann and ε -greedy put mass outside αA_α, including individually irrational actions. The pricing comparisons in Section 4.2 use two different baselines: IR-Boltzmann refuses purchases above value but can still leave αA_α, and support-matched Boltzmann is renormalized on αIRA_α^IR. Appendix F.4 compares UNOP with and without the reservation floor. Theorem 2. Let PtP_t be the Dirichlet-multinomial posterior mean with a uniform prior on a finite action set, and let actions be drawn i.i.d. from a fixed π. Then Pt→πP_t→π almost surely, so KL(π∥Pt)→0KL(π\|P_t)→ 0 on the support of π and maxaPt(a)→maxaπ(a) _aP_t(a)→ _aπ(a). If π is a δ-peak with δ>1/2δ>1/2, the posterior mode equals that action after O(log(1/η)/(δ−1/2)2)O ( (1/η)/(δ-1/2)^2 ) observations with probability at least 1−η1-η. If π is uniform on k actions, the same convergence leaves maxaPt(a)→1/k _aP_t(a)→ 1/k. The mixture is identified, and the one-step hit rate remains bounded by the policy. The proof records how the visit counts grow. Proposition 4. Let P be a price grid with spacing Δ and let the buyer use the constant-band response qα,ρ(p)=1p<v−α,ρv−α≤p≤v,0p>v,q_α,ρ(p)= cases1&p<v-α,\\ ρ&v-α≤ p≤ v,\\ 0&p>v, cases (7) with ρ∈(0,1)ρ∈(0,1). Write pα−(v)=maxp∈:p<v−αp^-_α(v)= \p :p<v-α\ and pα+(v)=max(∩[v−α,v])p^+_α(v)= (P∩[v-α,v]), when those sets are nonempty. A seller who knows qα,ρq_α,ρ and breaks revenue ties by lowest buyer surplus posts in the sure-demand region whenever pα−(v)>ρpα+(v).p^-_α(v)>ρ\,p^+_α(v). (8) The posted price is then pα−(v)p^-_α(v) and buyer surplus is v−pα−(v)∈(α,α+Δ]v-p^-_α(v)∈(α,α+ ]. The guarantee uses the response curve itself and does not use prediction error. UNOP is the case ρ=1/2ρ=1/2. Let r⋆=pα−(v)/pα+(v)r =p^-_α(v)/p^+_α(v). Every ρ<r⋆ρ<r induces that same price, so those rates are exactly the welfare-maximizing constant-band commitments. Binary entropy on (0,1)(0,1) is maximized at 1/21/2, which means that ρ=1/2ρ=1/2 is the unique entropy-maximizing rate that keeps the seller below the band when r⋆>1/2r >1/2. When r⋆≤1/2r ≤ 1/2 that set of rates, (0,r⋆)(0,r ), is open, so no entropy-maximizing rate exists and entropy only increases as ρ approaches r⋆r from below. The inequality α≤v/2α≤ v/2 is not enough for this conclusion, because if v=2αv=2α and v∈v , then pα+(v)=vp^+_α(v)=v and pα−(v)<α=v/2p^-_α(v)<α=v/2, so equation 8 fails at ρ=1/2ρ=1/2 and the seller can post inside the band. Proposition 4 is a commitment about prices that are not posted. Once the seller stays below the band, the action at the posted price pα−(v)p^-_α(v) is a sure purchase, and the mixture that moved the seller sits on prices that are not posted. On a uniform grid the strict inequality holds for ρ=1/2ρ=1/2 when α<v/2−Δα<v/2- , and it can fail on the boundary α=v/2α=v/2. 3.3 Mixing only where it is safe Figure 2 summarizes the pipeline. Individual rationality is part of the support: mixing into strictly loss-making actions, such as buying above value or clicking below reservation, teaches the optimizer to move the cutoff. Given u, slack α≥0α≥ 0, and reservation u¯∈−∞∪ℝ u∈\-∞\ , define αIR(u)=α(u)∩a:u(a)≥u¯if that intersection is nonempty,α(u)otherwise.A_α^IR(u)\;=\; casesA_α(u)∩\a:u(a)≥ u\&if that intersection is nonempty,\\ A_α(u)&otherwise. cases (9) Exact arithmetic makes the first branch apply whenever the reservation set is nonempty, because u⋆≥u¯u ≥ u then forces the global maximizer into the intersection. The second branch covers rare numerical cases. The reservation is 00 for skip in posted pricing, τ in recommendation, and −∞-∞ in bimatrix games. Figure 2: UNOP pipeline. From the current menu the human forms u u, builds the α-optimal set, intersects it with the reservation floor, and mixes uniformly on that set. The optimizer observes ata_t, updates P, and revises the menu, and that update can converge to the mixture. The welfare effect comes from the best response to the mixture. In pricing, once the seller stays below the band, the relevant mixture is off the posted price. Unaware UNOP is the unique solution of (WCα)(WC_α) on αIRA_α^IR. When a forecast P is available in the same simplex, Aware-UNOP replaces maximising H(π)H(π) by a strictly concave objective that downweights actions the forecast already expects πaware∈argmaxπ∈Δ(A)supp(π)⊆αIR(H(π)−a∼π[log(P(a)+ε)]),π^aware\;∈\; _ subarraycπ∈ (A)\\ supp(π) _α^IR subarray (H(π)-E_a π [ (P(a)+ ) ] ), (10) with ε>0 >0 a numerical floor. The unique maximizer is πaware(a)∝(P(a)+ε)−1π^aware(a) (P(a)+ )^-1 on αIRA_α^IR, equivalently the I-projection πaware=argminπ∈Δ(αIR)KL(π∥νP),νP(a)∝(P(a)+ε)−1.π^aware\;=\; _π∈ (A_α^IR)\ KL (π\, \|\, _P ), _P(a)\; \; (P(a)+ )^-1. (11) If P is uniform on that set, equation 10 reduces to unaware UNOP. Algorithm A1 is the one-round map used in every experiment, and the surrounding loops are in Appendix C. It is a good-enough mixer rather than a no-regret algorithm, and Hedge can be exploited by a patient optimizer (Deng et al., 2019). Across extraction domains, α on the order of 0.050.05 to 0.150.15 of the utility range is consistently near-best (Appendix Figure A7). 4 Experiments 4.1 Experimental Settings Bimatrix games, posted pricing, and recommendation are the three interaction classes. The new comparisons below use an oracle seller that knows qπq_π, a Thompson seller, a stationary tax game, and a policy-aware slate optimizer. Seeds, horizons, and raw tables for the UCB, MovieLens, and tabular runs are in supplement/legacy. Aware-UNOP sees the true forecast PtP_t. Unaware UNOP does not. Probabilities are clipped at 10−1210^-12 before every logarithm. 4.2 Posted pricing Figure 3: Test-split oracle pricing. (a) Buyer surplus against worst-case utility loss. UNOP is the maximum-entropy support-safe point. The plotted IR-Boltzmann curve still leaves αA_α, while support-matched Boltzmann at the same α has surplus 0.14540.1454, against 0.14530.1453 for UNOP. The oracle band commitment rises above UNOP once ρ=1/2ρ=1/2 fails equation 8. (b) Bayes accuracy at the posted price can be high at the same time, because the mixture that keeps the seller below the band is off that price. We draw 200200 values v∼U[0.25,0.95]v U[0.25,0.95] and hold out 120120 for test. The α we report is 0.150.15, chosen before the test split is used. Mean-surplus selection on the validation split returns 0.200.20, which raises the average and worsens the lower tail, so the test numbers stay at 0.150.15. The static test uses a demand oracle that knows qπq_π and breaks revenue ties by lowest buyer surplus, then by highest price. On the 4141-point test grid the experimental unit is a value, greedy surplus is 0.0120.012, and UNOP surplus is 0.1450.145. The paired difference is 0.1330.133, with a 95%95\% bootstrap CI [0.124,0.141][0.124,0.141] over 10,00010,000 resamples of the 120120 values, and the difference is positive on 89.2%89.2\% of them. A 10,00110,001-point average over values on [0.25,0.95][0.25,0.95], with the 4141-point price grid held fixed, gives 0.0130.013 and 0.1460.146. Bayes accuracy at the posted price is 0.9460.946, because the purchase is sure whenever equation 8 holds. The 10.8%10.8\% of test values on which UNOP does not beat greedy are exactly the values on which ρ=1/2ρ=1/2 fails equation 8 and the seller enters the band. An oracle support-safe commitment, which may lower ρ until equation 8 holds, reaches mean surplus 0.1620.162 on the same test split and 0.1630.163 under the same fine average. It uses the seller’s revenue objective, so we treat it as an upper bound rather than a second rule a user could deploy. Support-restricted Boltzmann at the same α=0.15α=0.15 has surplus 0.14540.1454, against 0.14530.1453 for UNOP, so the gain is carried by the hard support and the induced demand curve, with uniform mixing as the parameter-free maximum-entropy point of that class. Table 1 places this static pricing comparison next to the online demand oracle, the two recommendation objectives, and the tax game. Green cells are the higher user payoff when the counterpart evaluates the response curve. The blue cell is quality ranking, where the list is not chosen against that curve and greedy remains ahead. The subsections below report these comparisons one setting at a time. Table 1: Headline user payoffs used in this section. Green is the higher payoff when the counterpart evaluates the response curve. Blue is quality ranking, where greedy stays ahead. Comparison Greedy UNOP Static oracle pricing 0.0120.012 0.1450.145 Online demand oracle 0.0110.011 0.1510.151 Quality-ranking lists 0.4950.495 0.4590.459 Policy-aware lists 0.0820.082 0.1600.160 Tax-game payoff 0.7500.750 0.8600.860 Figure 4a is a separate online comparison, with four sellers on 6060 random values for 300300 rounds, and those bars are not a ranked strength scale. Table 2 records buyer surplus for greedy and UNOP under each seller. Green marks where UNOP pulls ahead. Under UCB the two policies stay close, because that seller is still walking down a 4141-point grid after 300300 rounds and has not yet evaluated the response curve. Under the demand oracle the per-price KL is 0.00430.0043. Table 2: Buyer surplus against four online sellers, 6060 values and 300300 rounds. Green marks a clear UNOP gain. The UCB row stays close because that seller is still exploring. Seller Greedy UNOP EXP3 0.1560.156 0.1910.191 UCB 0.0040.004 0.0060.006 Thompson sampling 0.0100.010 0.1070.107 Demand oracle 0.0110.011 0.1510.151 Figure 4: (a) Posted pricing under four online sellers, 6060 values and 300300 rounds, not a ranked strength scale. EXP3 mixes exploration with importance-weighted updates, and UCB at this horizon is still exploring. Thompson sampling and the demand oracle are closer to a best response to qπq_π. (b) Recommendation. The experimental unit is a catalog. Quality ranking favors greedy, and an exact policy-aware slate optimizer reverses the comparison. 4.3 Recommendation On catalogs of 1212 items and slates of size 44, every one of the 495495 slates is scored with the true π(⋅∣S)π(· S) over 160160 catalogs, and beam search of width 5050 matches that exact optimum. Quality ranking favors greedy, with paired difference −0.036-0.036 and 95%95\% CI [−0.040,−0.032][-0.040,-0.032]. The policy-aware oracle reverses the sign, so UNOP minus greedy is 0.0780.078, CI [0.064,0.092][0.064,0.092], positive on 74.4%74.4\% of catalogs, with means 0.1600.160 versus 0.0820.082. A separate 4848-item experiment, with slate size 88, beam width 3030, and 8080 catalogs, gives paired difference 0.1020.102, CI [0.090,0.115][0.090,0.115], positive on 98.8%98.8\% of catalogs. A later latent-correlation sweep, using beam width 5050 and 6060 catalogs per cell, asks whether that gap depends on conflict between quality and margin (Appendix Figure A6). It does not trace support size. The policy-aware gap is largest under strong negative latent correlation, where at correlation −0.8-0.8 and reservation quantile 0.20.2 the mean gap is 0.1040.104, 95%95\% CI [0.073,0.139][0.073,0.139]. Under strong positive latent correlation the same gap is indistinguishable from zero (−0.002-0.002, CI [−0.009,0.004][-0.009,0.004]). Several intermediate cells have a positive mean and a zero median, and quality ranking stays negative at every tested latent correlation from −0.8-0.8 to 0.80.8. The sweep is about conflict, and it is not a map over how large the acceptable set is. 4.4 Learning the mixture Figure 5 is a six-action stationary tax game with 100100 seeds. The user commits to a fixed mixture, the optimizer taxes the predicted mode, and the oracle taxes the true mode. For UNOP, Bayes accuracy is 0.2500.250, realized accuracy is 0.2480.248, frequency KL is 0.0120.012, and optimizer regret is 0.00000.0000, while user payoff is 0.8600.860 against 0.7500.750 for greedy. The predictor has reached the oracle best response, so what remains random is only the next draw from the learned mixture, and the user’s payoff stays higher because that mixture changes which action is taxed. Figure 5: Stationary tax game, frequency predictor, 100100 seeds, rolling means. Solid curves in (a) are realized accuracy and dashed curves are Bayes accuracy. Panel (b) is policy KL. In (d), solid curves are the learned optimizer and dashed curves are VO⋆(π)V_O (π), and their gap is optimizer regret rather than a measure of whether the policy can be exploited. The mixture is learned, the next draw stays at accuracy 1/k1/k, and user payoff stays above greedy. 4.5 Discussion The pricing and recommendation gains come from the response curve the optimizer best-responds to. They are not evidence that the mixture stayed hidden. In the tax game the predictor’s regret falls to zero, so the mixture has been identified, and the higher user payoff comes from which action that mixture makes costly to tax. The pricing misses line up with the same object. The held-out values on which uniform mixing does not beat greedy are the values on which a purchase rate of one half fails to keep the seller below the band, so those misses are named by the commitment condition rather than left as residual noise. Recommendation makes the contrast sharper. A ranker that ignores the response prefers the greedy list, while a slate optimizer that scores the true mixture reverses the comparison, and the reversal is largest when high quality and high margin pull apart. The short-horizon dynamic program is the boundary of the claim. Once the seller is still collecting samples instead of evaluating the curve, the optimal buyer collapses to greedy and the mixture can cost surplus. Against a five-price UCB seller, exact dynamic programming at horizons 1010, 1515, 2020, and 2525 gives the optimal buyer and greedy the same value at every tested value, and UNOP is weakly worse on several of those values, which fits a seller that has not yet evaluated the curve. A singleton acceptable set is a different failure. If αIR(S)A_α^IR(S) is a singleton for every feasible menu S, UNOP and greedy induce the same response, while a singleton only on the full candidate set still leaves the platform free to offer another menu. At α=0α=0 the two policies have identical buyer surplus on all 120120 test values, and the restricted MovieLens-100K cohort is a near-collapse, with mean difference −0.0009-0.0009 over 3333 users. Utility noise u^(a)=u(a)+ξ u(a)=u(a)+ξ, ξ∼(0,σ2)ξ (0,σ^2), against the oracle seller at α=0.15α=0.15 leaves surplus 0.1450.145 at σ=0σ=0 and −0.006-0.006 at σ=0.30σ=0.30, with a true loss-above-α rate of 0.1190.119. A practical reading is therefore narrow. Keep the mixture when the platform will best-respond to the whole curve and the menu still leaves more than one acceptable action, and drop it when the estimate of that set is wrong or the other side is a collaborator. 5 Conclusion The menu moves because the platform best-responds to a response curve. UNOP mixes uniformly on near-optimal, individually rational actions, and in pricing that mixture can keep a seller below the randomized band even after the curve is known. The gain appears against an extractive counterpart that evaluates the curve, and it disappears when the acceptable set collapses, the utility estimate is badly wrong, or the other side is still exploring. Ethics statement This work studies how humans can avoid having their preferences precisely identified by adaptive pricing and recommendation systems. The methods can resist targeting that takes surplus and, in principle, useful personalization. We do not provide tools for attacking deployed systems. MovieLens-100K/1M (Harper and Konstan, 2015), UCI Adult (Kohavi, 1996), German Credit (Hofmann, 1994), FICO HELOC, UCI Credit Card, Give Me Some Credit, and UCI Bank Marketing are used as public research files. Any real-world assistant implementing UNOP should make α user-visible and disable mixing in explicitly cooperative modes, as in Proposition 3. Reproducibility statement Code and raw outputs for every reported comparison are at https://github.com/FrankDengAI/unop and in the supplement. Oracle pricing, the stationary tax game, dynamic programming, utility noise, the seller family, policy-aware recommendation, the grid condition, the fine value-grid average, the latent-correlation sweep, and the singleton-support runs are under supplement/sprint, while bimatrix, MovieLens, and tabular outputs are under supplement/legacy. MovieLens-100K u.data is included, and MovieLens-1M together with the OpenML tabular sources are fetched by download_data.py and run_public.py when absent. Values for the oracle suite use seed 2026092620260926, and the test split is the last 60%60\% of 200200 draws. Commands and package versions are in supplement/README.md. No GPU is required. Proofs are in Appendix B. AI use statement Artificial intelligence tools assist in formatting and text polishing. The authors take full responsibility for the final content. References Auer et al. (2002) P. Auer, N. Cesa-Bianchi, and P. Fischer Finite-time analysis of the multiarmed bandit problem. Machine Learning 47, p. 235–256. Cited by: §2. Ben-Porat and Tennenholtz (2018) O. Ben-Porat and M. Tennenholtz A game-theoretic approach to recommendation systems with strategic content providers. In Advances in Neural Information Processing Systems, Vol. 31, p. 1110–1120. Cited by: §2. Braverman et al. (2018) M. Braverman, J. Mao, J. Schneider, and S. M. Weinberg Selling to a no-regret buyer. In Proceedings of the 2018 ACM Conference on Economics and Computation, p. 523–538. Cited by: §1, §2. Brückner et al. (2012) M. Brückner, C. Kanzow, and T. Scheffer Static prediction games for adversarial learning problems. Journal of Machine Learning Research 13 (85), p. 2617–2654. Cited by: §2. Calvano et al. (2020) E. Calvano, G. Calzolari, V. Denicolò, and S. Pastorello Artificial intelligence, algorithmic pricing, and collusion. American Economic Review 110 (10), p. 3267–3297. Cited by: §1, §2. Camara et al. (2020) M. K. Camara, J. D. Hartline, and A. Johnsen Mechanisms for a no-regret agent: beyond the common prior. In 2020 IEEE 61st Annual Symposium on Foundations of Computer Science (FOCS), p. 259–270. Cited by: §2. Carroll et al. (2019) M. Carroll, R. Shah, M. K. Ho, T. L. Griffiths, S. A. Seshia, P. Abbeel, and A. D. Dragan On the utility of learning about humans for human-AI coordination. In Advances in Neural Information Processing Systems, Vol. 32, p. 5174–5185. Cited by: §1, §2. Cesa-Bianchi and Lugosi (2006) N. Cesa-Bianchi and G. Lugosi Prediction, learning, and games. Cambridge University Press. Cited by: §2. Chen et al. (2020) Y. Chen, Y. Liu, and C. Podimata Learning strategy-aware linear classifiers. In Advances in Neural Information Processing Systems, Vol. 33, p. 15265–15276. Cited by: §2. Cover and Thomas (2006) T. M. Cover and J. A. Thomas Elements of information theory. 2 edition, Wiley-Interscience. Cited by: Appendix B, §2, §2, item 2. Dean et al. (2020) S. Dean, S. Rich, and B. Recht Recommendations and user agency: the reachability of collaboratively-filtered information. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, p. 436–445. Cited by: §1, §2. Deng et al. (2019) Y. Deng, J. Schneider, and B. Sivan Strategizing against no-regret learners. In Advances in Neural Information Processing Systems, Vol. 32. Cited by: §D.1, §2, §3.3. Dong et al. (2018) J. Dong, A. Roth, Z. Schutzman, B. Waggoner, and Z. S. Wu Strategic classification from revealed preferences. In Proceedings of the 2018 ACM Conference on Economics and Computation, p. 55–70. Cited by: §2. Drutsa (2018) A. Drutsa Weakly consistent optimal pricing algorithms in repeated posted-price auctions with strategic buyer. In Proceedings of the 35th International Conference on Machine Learning, Cited by: §1, §2. Drutsa (2020) A. Drutsa Optimal non-parametric learning in repeated contextual auctions with strategic buyer. In Proceedings of the 37th International Conference on Machine Learning, Cited by: §1, §2. Duchi et al. (2013) J. C. Duchi, M. I. Jordan, and M. J. Wainwright Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, p. 429–438. Cited by: §3.1. Dwork (2006) C. Dwork Differential privacy. In International Colloquium on Automata, Languages, and Programming, p. 1–12. Cited by: §3.1. Freund and Schapire (1997) Y. Freund and R. E. Schapire A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55 (1), p. 119–139. Cited by: §2. Fudenberg and Levine (1998) D. Fudenberg and D. K. Levine The theory of learning in games. MIT Press. Cited by: §2. Haarnoja et al. (2018) T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proceedings of the 35th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 80, p. 1861–1870. Cited by: §2. Hadfield-Menell et al. (2016) D. Hadfield-Menell, A. D. Dragan, P. Abbeel, and S. Russell Cooperative inverse reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 29, p. 3909–3917. Cited by: §1, §2. Haghtalab et al. (2022) N. Haghtalab, T. Lykouris, S. Nietert, and A. Wei Learning in Stackelberg games with non-myopic agents. In Proceedings of the 23rd ACM Conference on Economics and Computation, p. 917–918. Cited by: §2. Hardt et al. (2022) M. Hardt, M. Jagadeesan, and C. Mendler-Dünner Performative power. In Advances in Neural Information Processing Systems, Vol. 35, p. 22969–22981. Cited by: §1, §2. Hardt et al. (2016) M. Hardt, N. Megiddo, C. Papadimitriou, and M. Wootters Strategic classification. In Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science, p. 111–122. Cited by: §1, §2. Harper and Konstan (2015) F. M. Harper and J. A. Konstan The MovieLens datasets: history and context. ACM Transactions on Interactive Intelligent Systems 5 (4), p. 19:1–19:19. Cited by: §5. Hoffman (1952) A. J. Hoffman On approximate solutions of systems of linear inequalities. Journal of Research of the National Bureau of Standards 49 (4), p. 263–265. Cited by: Appendix B. Hofmann (1994) H. Hofmann Statlog (German credit data). Note: UCI Machine Learning Repository External Links: Document Cited by: §5. Hron et al. (2023) J. Hron, K. Krauth, M. I. Jordan, N. Kilbertus, and S. Dean Modeling content creator incentives on algorithm-curated platforms. In International Conference on Learning Representations, Cited by: §1, §2. Huh and Kandasamy (2026) J. S. Huh and K. Kandasamy Strategy-robust online learning in contextual pricing. In Proceedings of the 37th International Conference on Algorithmic Learning Theory, Cited by: §1, §2. Ie et al. (2019) E. Ie, V. Jain, J. Wang, S. Narvekar, R. Agarwal, R. Wu, H. Cheng, T. Chandra, and C. Boutilier SlateQ: a tractable decomposition for reinforcement learning with recommendation sets. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, p. 2592–2599. Cited by: §2. Jaynes (1957) E. T. Jaynes Information theory and statistical mechanics. Physical Review 106 (4), p. 620–630. Cited by: Appendix B, §2, item 2. Kleinberg and Raghavan (2019) J. Kleinberg and M. Raghavan How do classifiers induce agents to invest effort strategically?. In Proceedings of the 2019 ACM Conference on Economics and Computation, p. 825–844. Cited by: §2. Kohavi (1996) R. Kohavi Scaling up the accuracy of naive-Bayes classifiers: a decision-tree hybrid. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, p. 202–207. Cited by: §5. Kolumbus and Nisan (2022) Y. Kolumbus and N. Nisan Auctions between regret-minimizing agents. In Proceedings of the ACM Web Conference 2022, p. 100–111. Cited by: §2. Levanon and Rosenfeld (2021) S. Levanon and N. Rosenfeld Strategic classification made practical. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 6243–6253. Cited by: §2. Littlestone and Warmuth (1994) N. Littlestone and M. K. Warmuth The weighted majority algorithm. Information and Computation 108 (2), p. 212–261. Cited by: §2. Mansour et al. (2022) Y. Mansour, M. Mohri, J. Schneider, and B. Sivan Strategizing against learners in Bayesian games. In Proceedings of Thirty Fifth Conference on Learning Theory, Proceedings of Machine Learning Research, Vol. 178, p. 5221–5252. Cited by: §2. McKelvey and Palfrey (1995) R. D. McKelvey and T. R. Palfrey Quantal response equilibria for normal form games. Games and Economic Behavior 10 (1), p. 6–38. Cited by: §2. Milli et al. (2019) S. Milli, J. Miller, A. D. Dragan, and M. Hardt The social cost of strategic classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, p. 230–239. Cited by: §2. Nash (1950) J. F. Nash Equilibrium points in n-person games. Proceedings of the National Academy of Sciences 36 (1), p. 48–49. Cited by: §1. Osborne and Rubinstein (1994) M. J. Osborne and A. Rubinstein A course in game theory. MIT Press. Cited by: §1, §3.2. Perdomo et al. (2020) J. Perdomo, T. Zrnic, C. Mendler-Dünner, and M. Hardt Performative prediction. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, p. 7599–7609. Cited by: §1, §2. Russell (2019) S. Russell Human compatible: artificial intelligence and the problem of control. Viking. Cited by: §2. Schnabel et al. (2016) T. Schnabel, A. Swaminathan, A. Singh, N. Chandak, and T. Joachims Recommendations as treatments: debiasing learning and evaluation. In Proceedings of the 33rd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 48, p. 1670–1679. Cited by: §2. Shavit et al. (2020) Y. Shavit, B. Edelman, and B. Axelrod Causal strategic linear regression. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, p. 8676–8686. Cited by: §2. Stray et al. (2024) J. Stray, A. Y. Halevy, P. Assar, D. Hadfield-Menell, C. Boutilier, A. Ashar, C. Bakalar, L. Beattie, M. D. Ekstrand, C. Leibowicz, C. M. Sehat, S. Johansen, L. Kerlin, D. Vickrey, S. Singh, S. Vrijenhoek, A. X. Zhang, M. Andrus, N. Helberger, P. Proutskova, T. Mitra, and N. Vasan Building human values into recommender systems: an interdisciplinary synthesis. ACM Transactions on Recommender Systems 2 (3), p. 20:1–20:57. Cited by: §1. Ziebart et al. (2008) B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey Maximum entropy inverse reinforcement learning. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence, p. 1433–1438. Cited by: §2. Zrnic et al. (2021) T. Zrnic, E. Mazumdar, S. S. Sastry, and M. I. Jordan Who leads and who follows in strategic classification?. In Advances in Neural Information Processing Systems, Vol. 34, p. 15257–15269. Cited by: §2. Supplementary Material Appendix The appendices collect proofs of the claims in Section 3, the interaction loops used in every experiment, and further empirical detail. Algorithm A1 is the human’s one-round mixer. Algorithms A2 to A4 are the environments it is dropped into. They match the repository simulators on the objects that affect reported numbers: support construction, the IR intersection, UCB scores, and unique-best filler items. Algorithm A1 One-round UNOP / Aware-UNOP 1: u∈ℝ|A|u ^|A|, α≥0α≥ 0, reservation u¯ u, optional forecast P, aware flag, ε>0 >0 2: u⋆←maxau(a)u ← _au(a) ←a:u(a)≥u⋆−αA←\a:u(a)≥ u -α\ 3: if ∩a:u(a)≥u¯≠∅A∩\a:u(a)≥ u\≠ then 4: ←∩a:u(a)≥u¯A ∩\a:u(a)≥ u\ ⊳ equation 9 5: end if 6: if aware and P is present then 7: π(a)∝(P(a)+ε)−1π(a) (P(a)+ )^-1 on A ⊳ equation 10 8: else 9: π←Unif()π (A) ⊳ equation 4. Unaware UNOP ignores P 10: end if 11: return at∼πa_t π Appendix A Earlier controlled comparisons A.1 Comparison Results Figure A1 plots, for each bimatrix game, predictive accuracy against human payoff normalized within the game. In zero-sum games the effective policies are the unpredictable ones: greedy and ε -greedy sit at the bottom, Aware-UNOP at the top. In common-interest games the mass concentrates at the top-right: being predicted is what good play looks like. In general-sum games the cloud is interior, and UNOP occupies the attractive region, slightly above greedy in payoff, with lower accuracy. Absolute payoffs (Appendix D) match the picture: zero-sum suite greedy −0.143-0.143 vs. Aware-UNOP +0.054+0.054. Coordination greedy == UNOP =1.95=1.95 vs. uniform 1.001.00. Figure A1: (a) Zero-sum: mixing is mandatory. (b) General-sum: an interior frontier, and UNOP sits on it. (c) Common-interest: the Pareto point is pure. Faint markers are game and predictor pairs, with size proportional to policy entropy. Diamonds and circles are class-wise means ± sd. Table A1 and Figure A2 summarize a UCB monopolist learning a hidden value v∼U[0.25,0.95]v U[0.25,0.95]. Greedy buyers are identified almost perfectly (accuracy 0.9990.999) and retain surplus 0.0180.018, on the order of the price grid. UNOP with α=0.05α=0.05 more than doubles surplus to 0.0420.042 while remaining individually rational. α=0.15α=0.15 keeps 0.0400.040 at accuracy 0.8070.807 rather than 0.9990.999. Aware-UNOP matches this surplus at still lower accuracy (0.7850.785). Boltzmann and uniform buy above value and finish negative. ε -greedy, which spends exploration on the dominated action, is worse than greedy. Table A2 reports the matching log-loss and optimizer-payoff numbers: under UNOP-α=0.15α=0.15, pricing VOV_O falls from 0.4790.479 (greedy) to 0.2410.241, and log loss rises from 0.0260.026 to 0.3120.312. Appendix Figure A11 shows the same runs as time series: greedy posted prices climb toward v. UNOP keeps a lower posted price. The predictor’s hit rate settles near the policy’s Bayes accuracy rather than revealing a failure to learn the mixture (Section 4.2). Table A1: Main extraction results (mean over seeds). Surplus is buyer surplus in pricing and q−τq-τ in recommendation. Synth. strat. utility is approval minus manipulation cost on the synthetic population (Adult/German are in Table A3). Green: best surplus in each extraction column, and the least-predictable effective recsys policy. Blue: extractor lock-in (greedy accuracy) or the weak-dilemma peak (greedy still wins on strategic utility). Policy Pricing Recsys Synth. strat. surplus acc. surplus acc. util. acc. Greedy 0.0180.018 0.9990.999 0.0520.052 0.7040.704 0.5170.517 0.3040.304 ε -greedy 0.0110.011 0.9540.954 0.0510.051 0.4890.489 0.5000.500 0.2850.285 Boltzmann 0.20.2 −0.021-0.021 0.6650.665 0.0240.024 0.1690.169 0.4350.435 0.2370.237 Boltzmann 0.50.5 −0.068-0.068 0.5950.595 −0.043-0.043 0.1340.134 0.4020.402 0.2140.214 UNOP α=0.05α=0.05 0.0420.042 0.9620.962 0.1260.126 0.3150.315 0.4960.496 0.2530.253 UNOP α=0.15α=0.15 0.0400.040 0.8070.807 0.1480.148 0.3400.340 0.4580.458 0.2760.276 UNOP α=0.3α=0.3 0.0220.022 0.7350.735 0.1100.110 0.4350.435 0.4090.409 0.2200.220 Aware-UNOP α=0.15α=0.15 0.0410.041 0.7850.785 0.1240.124 0.2410.241 0.4420.442 0.2400.240 Uniform −0.133-0.133 0.4990.499 −0.094-0.094 0.1070.107 0.3680.368 0.1990.199 Table A2: Same runs as Table A1. Predictor log loss is higher when the policy is less readable, and optimizer payoff VOV_O is lower when there is less extraction in pricing and recommendation. In the synthetic strategic setting, VOV_O is institution accuracy. Policy Pricing Recsys Synth. strat. nll VOV_O nll VOV_O nll VOV_O Greedy 0.0260.026 0.4790.479 0.8470.847 1.1831.183 1.5961.596 0.8820.882 ε -greedy 0.1980.198 0.4460.446 1.4531.453 1.1291.129 1.5991.599 0.8800.880 Boltzmann 0.20.2 0.5930.593 0.2790.279 2.6392.639 0.8710.871 1.5891.589 0.8870.887 Boltzmann 0.50.5 0.6730.673 0.3460.346 2.6572.657 0.9510.951 1.6081.608 0.8880.888 UNOP α=0.05α=0.05 0.0820.082 0.3960.396 1.6941.694 0.9620.962 1.5891.589 0.8950.895 UNOP α=0.15α=0.15 0.3120.312 0.2410.241 2.0572.057 0.6230.623 1.5611.561 0.9020.902 UNOP α=0.3α=0.3 0.4080.408 0.1890.189 1.8731.873 0.5330.533 1.5871.587 0.8910.891 Aware-UNOP α=0.15α=0.15 0.3090.309 0.2380.238 2.0802.080 0.7640.764 1.5981.598 0.8970.897 Uniform 0.7040.704 0.4620.462 2.5962.596 1.0091.009 1.6151.615 0.8870.887 Figure A2: (a) Buyer surplus (seed strip + mean). (b) The same runs in the effectiveness-predictability plane (marker size == buy rate). Dashed path greedy → UNOP-α → uniform. Per-value scatter is in Appendix Figure A10. Boltzmann/uniform go negative because they buy above value. We first study a unique-best withholding extractor, and then a policy-aware slate optimizer that best-responds to each user policy (Section 4.2). In the unique-best design, once acceptance can be predicted, the platform withholds better organic content so that a high-margin item is the unique quality-best offer. Figure A3(a) and Table A1 show that greedy users are exactly the users this attack is designed for: surplus 0.0520.052, promoted-item rate 0.8580.858, accuracy 0.7040.704. UNOP with α=0.15α=0.15 raises surplus to 0.1480.148 (≈2.8×≈ 2.8×), cuts promoted exposure to 0.3300.330, and drops accuracy to 0.3400.340. Aware-UNOP is the least predictable effective policy (accuracy 0.2410.241). Uniform again goes negative. Removing the individual-rationality floor changes the sign of pricing surplus (Appendix Table A8): UNOP-α=0.15α=0.15 becomes negative once the policy may buy above value, because those purchases teach the monopolist that the current price is still acceptable. Figure A3: (a) Synthetic margin-seeking recommender: user surplus (seed strip + mean). (b) Complete MovieLens-100K, per-user greedy vs. UNOP scatter. Promoted-item rate versus surplus is in Appendix Figure A12. A.2 Discussion The experiments do not say that unpredictability is always useful. They separate the cases in which a pure best response gives the optimizer a threshold, the cases in which there is no slack left to mix over, and the cases in which the counterpart is not trying to extract surplus at all. The practical question is where the α-optimal set is large enough to mix over, and whether the other side gains by sitting on that threshold. On MovieLens-100K (Figure A3(b), all 943943 users and 66016601 user-policy jobs), five-star ratings often leave a single top item in the α-optimal set, so UNOP nearly reduces to greedy. Surplus is 0.1890.189 for greedy, 0.1870.187 for UNOP at α=0.1α=0.1, and −0.115-0.115 for uniform. MovieLens-1M (6,0406,040 users, 42,28042,280 jobs) gives the same ordering (0.1870.187, 0.1850.185, and −0.130-0.130). Theorem 1 predicts this boundary. Coarse utilities remove the room to mix. Pricing and finely graded quality keep it. A frequency predictor reaches accuracy 1.001.00 on greedy behavior in 3030 steps. For UNOP with α=0.15α=0.15, realized accuracy stays near 0.300.30 through 350350 steps in 1515 of 1616 seeds (Appendix Figure A15), which is the Bayes accuracy of a mixture on several near-optimal actions, not a failure to identify the mixture (Proposition 1). ε -greedy still concentrates, because 90%90\% of its mass stays on one action. Recommendation surplus peaks near α=0.10α=0.10 and then falls as skipping becomes excessive (Appendix Figure A7). With about 30%30\% UNOP users, a shared margin-seeking optimizer’s accuracy falls from 0.4600.460 on a purely greedy population to 0.2190.219, and greedy users’ surplus rises from 0.1430.143 to about 0.170.17 (Appendix Figure A15, right). The optimizer no longer settles on one threshold, so part of the gain reaches users who do not change policy. When the institution maximizes accuracy, lower predictability usually brings little or no utility gain. Across Adult and the other credit and marketing sets in Table A3, greedy stays ahead on utility. The counterpart’s objective decides whether being readable is costly. The same loop appears when an assistant learns which tasks, queues, or revisions a person will accept. That map helps delegation, and it can also steer later work toward the most readable person. Keeping several acceptable ways to carry out a task, when real slack exists, is the practical form of UNOP. Proposition 3 still applies when the counterpart is cooperative. Appendix B Proofs The arguments below concern support, security level, coordination tax, demand curves, and Dirichlet concentration. Proof of Proposition 1. The hit rate of a predictor that names an action a⋆a is π(a⋆)≤maxaπ(a)π(a )≤ _aπ(a), and the bound is achieved by naming a mode. For the log loss, a∼π[−logP(a)]=−∑aπ(a)logπ(a)+∑aπ(a)logπ(a)P(a)=H(π)+KL(π∥P).E_a π[- P(a)]=- _aπ(a) π(a)+ _aπ(a) π(a)P(a)=H(π)+KL(π\|P). The KL of two distributions is zero if and only if they agree π-almost everywhere. Off the support the value of P does not enter the expectation, which is why the statement is restricted to supp(π)supp(π). Clipping at 10−1210^-12 is only a numerical guard in the experiments. The identity itself is stated for strictly positive probabilities on the support. ∎ Proof of Proposition 2. Write the human payoff matrix as U, with rows indexed by a and columns by pure optimizer actions o. The security set is the nonempty polytope =π:π≥0, 1⊤π=1,U⊤π≥v⋆.S=\π:π≥ 0,\ 1 π=1,\ U π≥ v 1\. If minouH(π,o)≥v⋆−ε _ou_H(π,o)≥ v - , then every violated security inequality has residual at most ε : ‖[(v⋆−U⊤π)+]‖∞≤ε\|[(v 1-U π)_+]\|_∞≤ . Hoffman’s error bound for a fixed feasible system of linear equalities and inequalities (Hoffman, 1952) gives a finite constant CGC_G such that infq∈‖π−q‖1≤CGε _q \|π-q\|_1≤ C_G . Since total variation is one half of the ℓ1 _1 distance, the displayed claim follows with κG=CG/2 _G=C_G/2. This constant depends on the payoff matrix. The proposition does not assert a uniform constant over all games. For a pure action a, an exact best responder realizes minouH(a,o) _ou_H(a,o), which is strictly below v⋆v unless a itself is a security strategy. In matching pennies with payoffs +1,−1\+1,-1\, =(1/2,1/2)S=\(1/2,1/2)\ and a policy (1/2+d,1/2−d)(1/2+d,1/2-d) has security value −2|d|-2|d| and total-variation distance |d||d| from S. Thus value within ε requires |d|≤ε/2|d|≤ /2, ruling out a fixed peak as ε→0 → 0. Rock, paper, scissors has the analogous unique uniform security strategy. This is the precise, game-dependent sense in which effective play in our zero-sum examples must be mixed. ∎ Proof of Proposition 3. Let u⋆=u(a⋆,o⋆)u =u(a ,o ) and u¯=maxa≠a⋆,ou(a,o) u= _a≠ a ,ou(a,o). Uniqueness of the global maximizer implies u¯<u⋆ u<u . The point mass on a⋆a , paired with o⋆o , attains u⋆u . For any π with π(a⋆)=pπ(a )=p and any fixed partner action o, a∼πu(a,o)≤pu⋆+(1−p)u¯.E_a πu(a,o)≤ p\,u +(1-p) u. The inequality therefore also holds after maximizing the left-hand side over the partner’s actions. Relative to u⋆u , every p<1p<1 loses at least (1−p)(u⋆−u¯)(1-p)(u - u). Finally, grouping a⋆a against all other actions gives H(π)≤h2(p)+(1−p)log(|A|−1)H(π) _2(p)+(1-p) (|A|-1). The right-hand side tends to zero as p→1p→ 1, so for each h>0h>0 the constraint H(π)≥hH(π)≥ h implies p≤p⋆(h)<1p≤ p (h)<1. ∎ Proof of Theorem 1. (1) The feasible set of (WCα)(WC_α) is Δ(α(u)) (A_α(u)). Shannon entropy H(π)=−∑aπ(a)logπ(a)H(π)=- _aπ(a) π(a) is strictly concave on this simplex and is uniquely maximised by the uniform distribution (Jensen). Thus Unif(α)Unif(A_α) is the unique maximiser, H=log|α|H= |A_α|, and u(a)≥u⋆−αu(a)≥ u -α on that set implies π[u]≥u⋆−αE_π[u]≥ u -α. Moreover, maxa∈supp(π)(u⋆−u(a))≤α⟺supp(π)⊆α(u). _a (π)(u -u(a))≤α (π) _α(u). Hence any positive mass outside αA_α violates the advertised per-action guarantee. It need not violate an expected-loss constraint when that mass is small. (WCα)(WC_α) deliberately enforces the stronger support-wise condition. (2) The Lagrangian for maxππ⋅u+τH(π) _π· u+τ H(π) over Δ(A) (A) gives the Gibbs distribution (Jaynes, 1957; Cover and Thomas, 2006): π(a)∝eu(a)/τπ(a) e^u(a)/τ. The map τ↦H(πτ)τ H( _τ) is continuous and onto (H(Unif(argmaxu)),log|A|](H(Unif( u)), |A|] for τ∈(0,∞]τ∈(0,∞], so every (SHh)(SH_h) with h in that interval is attained at some τ>0τ>0. Uniqueness follows from strict concavity of entropy. At h=0h=0 the feasible set includes every mass on argmaxu u. The leak bounds equation 6 follow from Z≤|A|eu⋆/τZ≤|A|e^u /τ and the existence of some a∉αa _α with u(a)=u⋆−α−γu(a)=u -α-γ for the lower bound, and from Z≥|α|e(u⋆−α)/τZ≥|A_α|e^(u -α)/τ together with u(a)≤u⋆−α−γu(a)≤ u -α-γ off αA_α for the upper bound. ∎ Figure A4: (a) Two Pareto curves on a six-action utility vector. Gibbs is optimal for a Shannon constraint and immediately puts weight outside the near-best set. UNOP is optimal for a worst-case loss constraint. (b) |α||A_α| jumps when the next action enters, while expected utility steps down. Proof of equation 10. On Δ(αIR) (A_α^IR) the map π↦H(π)−π[log(P+ε)]π H(π)-E_π[ (P+ )] is strictly concave. The Lagrangian ∑aπ(a)log1π(a)(P(a)+ε)+λ(1−∑aπ(a)) _aπ(a) 1π(a)(P(a)+ )+λ (1- _aπ(a) ) has stationary point log1π(a)(P(a)+ε)−1=λ 1π(a)(P(a)+ )-1=λ, hence π(a)∝(P(a)+ε)−1π(a) (P(a)+ )^-1. The same point is the unique I-projection of νP∝(P+ε)−1 _P (P+ )^-1 onto the face, which is equation 11. ∎ Proof of Theorem 2. The posterior mean is Pt(a)=(na+1)/(|A|+t)P_t(a)=(n_a+1)/(|A|+t). The strong law gives na/t→π(a)n_a/t→π(a) almost surely, hence Pt→πP_t→π. On supp(π)supp(π) the map P↦KL(π∥P)P (π\|P) is continuous for P bounded away from zero, which holds for large t on that support, so the KL vanishes and the maximum coordinate converges to maxaπ(a) _aπ(a). For a δ-peak on a⋆a with δ>1/2δ>1/2, Hoeffding gives ℙ(na⋆≤t/2)≤exp(−2t(δ−1/2)2)P(n_a ≤ t/2)≤ (-2t(δ-1/2)^2 ). On that event a⋆a is the unique count maximizer. For uniform-on-k, [maxana]=t/k+Θ((tlogk)/k)E[ _an_a]=t/k+ ( (t k)/k ) for fixed k, so [maxaPt(a)]=tk(|A|+t)+Θ((tlogk)/k|A|+t),E [ _aP_t(a) ]= tk(|A|+t)+ \! ( (t k)/k|A|+t ), which tends to 1/k1/k. ∎ Proof of Proposition 4. For p<v−αp<v-α, buy is the unique α-optimal individually rational action, so q=1q=1. For p∈[v−α,v]p∈[v-α,v], both skip and buy lie in the α-set, and the constant-band rule sets q=ρq=ρ. For p>vp>v, buy is individually irrational, so q=0q=0. Sure-demand revenue is therefore maximized uniquely at pα−(v)p^-_α(v), with value pα−(v)p^-_α(v), whenever that price exists. Every band price earns at most ρpα+(v)ρ\,p^+_α(v). If pα−(v)>ρpα+(v)p^-_α(v)>ρ\,p^+_α(v), every band price, and every price above v, has strictly lower revenue, so the unique revenue maximizer is pα−(v)p^-_α(v). On a grid of spacing Δ , v−α−Δ<pα−(v)<v−αv-α- <p^-_α(v)<v-α, hence buyer surplus v−pα−(v)v-p^-_α(v) lies in (α,α+Δ](α,α+ ]. Any two rates that keep the seller below the band induce this same price and the same surplus. Let r⋆=pα−(v)/pα+(v)r =p^-_α(v)/p^+_α(v). Binary entropy on (0,1)(0,1) increases toward 1/21/2. If r⋆>1/2r >1/2, then ρ=1/2ρ=1/2 lies in that set and uniquely maximizes entropy there. If r⋆≤1/2r ≤ 1/2, every such rate is strictly below 1/21/2, entropy is increasing on (0,r⋆)(0,r ), and the upper value r⋆r is not reached. A deployment that wants a maximizer can impose a revenue margin δ>0δ>0, replacing equation 8 by pα−(v)≥ρpα+(v)+δp^-_α(v)≥ρ\,p^+_α(v)+δ. The resulting set is closed and the entropy maximizer is min(1/2,(pα−(v)−δ)/pα+(v)) (1/2,(p^-_α(v)-δ)/p^+_α(v) ) whenever that quantity is positive. If v=2αv=2α and v∈v , then pα+(v)=vp^+_α(v)=v and band revenue at ρ=1/2ρ=1/2 is v/2=αv/2=α, while every sure-demand price is strictly below v−α=αv-α=α. Condition equation 8 fails. The seller can prefer the band, including p=vp=v, where buyer surplus is zero. Thus α≤v/2α≤ v/2 does not imply that the seller stays below the band under seller-favorable tie-breaking. ∎ Figure A5: Buyer surplus for a constant-band response on an 8181-point grid at v=0.8v=0.8. The white curve is the open boundary ρ=r⋆ρ=r . Seller-favorable tie-breaking does not treat the boundary itself as keeping the seller out. The dashed line is ρ=1/2ρ=1/2. To the right of their intersection, uniform mixing fails equation 8, and the highest entropy of the rates that keep the seller below the band is not reached. Figure A6: Policy-aware beam search of width 5050 on catalogs drawn with a chosen Gaussian correlation, n=48n=48, slate size 88, α=0.15α=0.15, 6060 catalogs per cell. On 450450 matching 1212-item catalogs (3030 per cell, both policies, 900900 policy and catalog rows) the same width matches exact enumeration in every case. Entries are mean user-surplus differences, UNOP minus greedy. The horizontal axis is the correlation used to draw the pair, which is not the sample Pearson correlation of the realized quality and margin. The gain is largest at strong negative correlation. At strong positive correlation the paired interval covers zero. Paired intervals, medians, and positive fractions are in supplement/sprint/results/recsys_phase_ci.json. Quality ranking is negative in every cell. Appendix C Simulator algorithms Algorithm A1 is the human’s one-round map. Everything that surrounds it in the experiments is an adaptive optimizer that (i) posts a menu, (i) observes ata_t, (i) updates a predictor or a bandit count, and (iv) posts a new menu. The three loops below are those optimizers. They match experiments/simulate.py. They specify the individual-rationality floor, the 5%5\% random slip, and the unique-best filler-item construction. Two implementation details that Algorithm A1 compresses. First, under exact arithmetic the IR intersection is nonempty whenever the reservation set is nonempty, so equation 9 reduces to intersecting αA_α with a:u(a)≥u¯\a:u(a)≥ u\. If floating-point noise empties that set, the code falls back to αA_α. Second, Aware-UNOP uses ε=10−6 =10^-6 in the (P+ε)−1(P+ )^-1 weights. Both choices are implementation details. They are what make the released rows match the algorithms below. What is being predicted. In bimatrix games the predictor PtP_t is a distribution on the human’s action set, and the optimizer chooses an exact best response to that prediction. In posted pricing there is no recommendation list: the menu is the posted price, and PtP_t is a Bernoulli on skip,buy\skip,buy\ at that price, maintained as a Beta posterior. In recommendation PtP_t is an empirical distribution on the current recommendation list plus skip. Lock-in experiments use the frequency predictor of Algorithm A2 on a synthetic six-action menu and declare tlockt_lock at the first time a length-3030 rolling accuracy exceeds 0.70.7. Algorithm A2 Bimatrix closed loop (frequency / Markov / softmax predictor) 1: game (UH,UO)(U_H,U_O), human policy π, horizon T, predictor P 2: σ^←Unif(O) σ (O) ⊳ empirical mix of optimizer actions 3: for t=1,…,Tt=1,…,T do 4: u^←UHσ u← U_H σ ⊳ mean-based human, equation 2 5: at∼π(⋅∣u^,Pt)a_t π(· u,P_t) ⊳ Algorithm A1, or a baseline 6: ot←argmaxo(UO⊤Pt)o_t← _o(U_O P_t)_o, with a 5%5\% uniform random slip 7: observe payoffs UH(at,ot)U_H(a_t,o_t), UO(at,ot)U_O(a_t,o_t) 8: update Pt+1P_t+1 from ata_t. σ^← σ← empirical mix of o1:to_1:t 9: end for 10: return post-burn-in means of uHu_H, uOu_O, argmaxPt=at1\ P_t=a_t\, and H(πt)H( _t) The 5%5\% random slip is there so that a pure profile cannot freeze the dynamics by accident. It is also why greedy in matching pennies finishes near −0.14-0.14 rather than at −1-1: identification is fast, not instantaneous, and the opponent occasionally plays off the punishing action. Reported bimatrix means drop the first quarter of the horizon. Algorithm A3 UCB posted-price monopolist 1: price grid pjj=1m\p_j\_j=1^m, hidden value v, buyer policy π, horizon T 2: sj←1s_j← 1, fj←1f_j← 1 for all j ⊳ Beta-Bernoulli prior on ℙ(buy∣pj)P(buy p_j) 3: for t=1,…,Tt=1,…,T do 4: b^j←sj/(sj+fj) b_j← s_j/(s_j+f_j) 5: UCBj←pj⋅clip(b^j+2log(t+2)/(sj+fj), 0,1)UCB_j← p_j·clip ( b_j+ 2 (t+2)/(s_j+f_j),\,0,1 ) 6: post pt←pj⋆p_t← p_j with j⋆=argmaxjUCBjj = _jUCB_j 7: b^j⋆←sj⋆/(sj⋆+fj⋆) b_j ← s_j /(s_j +f_j ). Pt←(1−b^j⋆,b^j⋆)P_t←(1- b_j ,\, b_j ) 8: buyer utilities u←(0,v−pt)u←(0,\,v-p_t). at∼π(⋅∣u,Pt)a_t π(· u,P_t) with reservation 00 9: if at=buya_t=buy then 10: sj⋆←sj⋆+1s_j ← s_j +1 11: else 12: fj⋆←fj⋆+1f_j ← f_j +1 13: end if 14: end for 15: return post-burn-in buyer surplus, posted price, buy rate, predictor accuracy Algorithm A3 instantiates Proposition 4. A greedy buyer produces a step-function demand curve, so the UCB scores concentrate on the last grid cell below v and surplus is about one grid step. An IR-UNOP buyer produces the three-piece demand equation 7, so the same scores cannot treat the coin-flip band as sure demand. Thompson sampling replaces the UCB bonus by a Beta draw (Appendix H.3). The qualitative gap survives. Turning the reservation off is Appendix F.4: the buyer then sometimes buys above value, the success count sjs_j is incremented at p>vp>v, and surplus changes sign. Algorithm A4 Margin-seeking unique-best recommendation list 1: catalog (qi,mi)i=1n(q_i,m_i)_i=1^n, list size k, reservation τ, user policy π, horizon T 2: Ni←1N_i← 1, Ci←1C_i← 1 for all i ⊳ counts when i was the unique-best offer 3: for t=1,…,Tt=1,…,T do 4: p^i←Ci/Ni p_i← C_i/N_i. UCBi←2log(t+2)/NiUCB_i← 2 (t+2)/N_i 5: profiti←mi⋅clip(p^i+0.7⋅UCBi, 0,1.2)profit_i← m_i·clip( p_i+0.7·UCB_i,\,0,1.2) 6: offer i⋆←argmaxiprofitii ← _iprofit_i (uniform among items for the first 3030 rounds, and with an 8%8\% random slip thereafter) 7: fill the remaining k−1k-1 slots with items of quality strictly below qi⋆q_i 8: Pt←P_t← predictive distribution on the displayed actions (simulation: extractor forecast on i⋆i , remaining mass on skip and filler items) 9: user utilities ←(qj1,…,qjk,τ)←(q_j_1,…,q_j_k,τ). at∼π(⋅∣u,Pt)a_t π(· u,P_t) with reservation τ 10: Ni⋆←Ni⋆+1N_i ← N_i +1 11: if the click is i⋆i then 12: Ci⋆←Ci⋆+1C_i ← C_i +1 13: end if 14: surplus is qclicked−τq_clicked-τ on a click and 00 on skip 15: end for The unique-best construction is the extraction. A vanilla ranking UCB that also shows better organic items is a weaker adversary, because a greedy quality maximizer then clicks the organic item and never certifies the high-margin offer. MovieLens uses the same loop with quiq_ui from the official rating files, scaled to [0,1][0,1], and the same quantile reservation. When the five-star grid makes αA_α a singleton at the top star, Algorithm A1 returns greedy, which is the boundary in Theorem 1. Baselines, for the same menus. Greedy is Algorithm A1 at α=0α=0. ε -greedy puts mass ε/|A| /|A| on every action, including those below reservation. Boltzmann is π(a)∝exp(u(a)/τ)π(a) (u(a)/τ) on the whole of A. Uniform is 1/|A|1/|A|. Hedge is multiplicative weights on realized u^t u_t, with no support constraint. Under Algorithms A3 and A4 the extractor does not change: only the human’s support does. Gibbs leak (Theorem 1) is visible in Algorithm A3 as soon as Boltzmann buys at p>vp>v and increments the success count sjs_j there. Lock-in. On a six-action synthetic menu with a planted δ-peak or a uniform k-set, a frequency predictor Pt(a)=(na+1)/(|A|+t)P_t(a)=(n_a+1)/(|A|+t) is updated every round. tlockt_lock is the first t at which a trailing window of 3030 one-hot hits argmaxPs=as1\ P_s=a_s\ has mean at least 0.70.7. Theorem 2 is the concentration statement behind that curve: a δ-peak with δ>1/2δ>1/2 produces a unique posterior mode in O(log(1/η)/(δ−1/2)2)O( (1/η)/(δ-1/2)^2) samples, while a uniform mix on k near-ties keeps the posterior an O(1/k)O(1/k)-peak. ε -greedy still locks quickly because 1−ε1- of its mass is a peak. Entropy of αA_α, not a generic exploration rate, is what delays identification. Appendix D Additional experimental details All simulators are NumPy/SciPy/scikit-learn CPU code. We used 9696 processes. Controlled suite (32403240 bimatrix jobs, 216216 pricing, 180180 recsys, 144144 synthetic strategic, 144144 lock-in, 452452 α-sweep rows, 132132 population runs, 45084508 recorded rows) completed with zero errors. Public source files: MovieLens-100K, all 943943 users × 77 policies (66016601 jobs, official ≥20≥ 20 ratings). MovieLens-1M, all 6,0406,040 users × 77 (42,28042,280 jobs). UCI Adult, source population 48,84248,842, d=108d=108 after one-hot, 1616 seeds × 99 policies (T=3000T=3000, pretrained logistic accuracy 0.8520.852). German Credit, source population 1,0001,000, d=61d=61, 1616 seeds × 99 (T=2000T=2000, pretrained accuracy 0.7850.785). FICO HELOC, source population 10,00010,000, 1616 seeds × 99. UCI Credit Card, source population 30,00030,000, 1616 seeds × 99. Give Me Some Credit, source population 150,000150,000, pretrained accuracy 0.9340.934, 1616 seeds × 99. UCI Bank Marketing, source population 45,21145,211, d=51d=51, pretrained accuracy 0.9010.901, 1616 seeds × 99. Bimatrix: T=1000T=1000, 1212 seeds, predictors freq,Markov,logreg\freq,Markov,logreg\. Pricing: T=500T=500, 2424 seeds, 2121-price grid. Recsys: T=300T=300, 2020 seeds, 4848 items, list size 88. Synthetic strategic: T=700T=700, 1616 seeds, 200200 agents, 88 features, 55 manipulation levels. Lock-in: T=350T=350, 66 actions, 1616 seeds. Public files are loaded without file-level subsampling. MovieLens covers every user, while tabular interactions use the stated horizons. Table A3 reports greedy vs. UNOP on every official public file we ran. MovieLens columns use α=0.1α=0.1. Tabular strategic-classification rows use α=0.15α=0.15. Table A3: Public complete datasets. N is users for MovieLens and records for the tabular sets. Utility is user surplus on MovieLens and agent payoff, approval minus gaming cost, on the strategic-classification sets. MovieLens uses α=0.1α=0.1 because the ratings have almost no slack. Tabular rows use α=0.15α=0.15, where greedy is still ahead on utility and UNOP is less readable. The Adult α=0.05α=0.05 sensitivity is utility/accuracy 0.1810.181/0.4280.428, reported in Appendix G.3. Dataset N greedy util. greedy acc. UNOP util. UNOP acc. MovieLens-100K (complete) 943 0.189 0.415 0.187 0.418 MovieLens-1M (complete) 6040 0.187 0.419 0.185 0.425 UCI Adult (complete) 48842 0.178 0.482 0.136 0.295 German Credit (complete) 1000 0.757 0.618 0.722 0.328 FICO HELOC 10000 0.501 0.779 0.470 0.312 UCI Credit Card 30000 0.082 0.981 0.046 0.334 Give Me Some Credit 150000 0.005 0.938 -0.031 0.333 UCI Bank Marketing 45211 0.061 0.815 0.024 0.327 Figure A7: (a) Pricing α-sweep. (b) Recommendation α-sweep. Shared legend at right: effectiveness rises and then falls. Buy/skip volume and scaled entropy show how the mix changes. Predictability falls once the α-set is non-singleton. Matching pennies: greedy −0.146-0.146, UNOP-α=0.15α=0.15 0.0000.000, Aware-UNOP +0.055+0.055. RPS: greedy −0.141-0.141, UNOP 0.000.00, Aware-UNOP +0.053+0.053. Zero-sum suite means remain −0.143-0.143 / 0.0000.000 / +0.054+0.054. Coordination: greedy == UNOP =1.95=1.95, uniform 1.001.00. Stag hunt: greedy == UNOP =4.88=4.88, uniform 3.963.96. Chicken: UNOP-α=0.15α=0.15 1.821.82 vs. greedy 1.771.77. Inspection: UNOP-α=0.05α=0.05 0.960.96 vs. greedy 0.920.92. Prisoner’s dilemma: all near-optimal policies collapse to defection against an adaptive counterpart (1.111.11), as expected. In the margin-seeking recommender, greedy users click a promoted item in 85.8%85.8\% of rounds after the early exploration. UNOP-α=0.15α=0.15 in 33.0%33.0\%. Aware-UNOP in 41.5%41.5\%. Skip rates rise with α (0.0640.064 greedy, 0.3400.340 at α=0.15α=0.15), which is the volume and surplus tradeoff in Figure A7. If the human’s u u is u plus independent Gaussian noise of scale σ, the α-set is a noisy near-best set. For σ≤α/2σ≤α/2, UNOP remains inside the true (2α)(2α)-optimal set with high probability on small action spaces, which is why a slightly larger α is a robust default. Bimatrix results average frequency, Markov, and online-softmax predictors. The three-case pattern is stable across the three. Replacing margin-seeking unique-best recommendation lists with vanilla CTR-UCB ranking weakens extraction: without withholding, greedy quality-maximizers are not forced onto the margin. D.1 Restatement of the claims This section restates the claims of Appendix B in the notation of the main text. The later ablations are read against that payoff shape. Start with a utility vector u∈ℝ|A|u ^|A| that the human actually believes, or at least acts as if they believe. The number u⋆=maxau(a)u = _au(a) is the one-step best-response value, and the set α(u)=a:u(a)≥u⋆−αA_α(u)=\a:u(a)≥ u -α\ is the set of actions that are never more than α worse than that value. Theorem 1 separates two constraints. First, if every action that may be played lies in αA_α, then among all such policies the uniform distribution is the unique entropy maximizer, and every such policy has expected utility at least u⋆−αu -α no matter how the rest of the vector is filled in. Second, if the only constraint is that entropy be at least some h, then the unique expected-utility maximizer is a Gibbs distribution, and Gibbs distributions put positive mass on every action as soon as the temperature is positive. The first constraint is a worst-case loss constraint and the second is a Shannon constraint. They coincide only when the utility vector is essentially flat on a subset and −∞-∞ elsewhere, which is not the situation a buyer or a recommender user is in. That distinction is not a taste in regularizers. In posted pricing, a Gibbs buyer with even a modest temperature will sometimes buy above value, because the softmax never quite hits zero on the dominated action, and a learning monopolist treats those purchases as evidence that the posted price is still acceptable. UNOP with an individual-rationality floor of zero will not do that. The ablation in Appendix F.4 is the empirical counterpart of this paragraph: when we turn the floor off, UNOP starts to look like a slightly better-behaved Boltzmann policy, and the monopolist extracts it. Proposition 2 is the matching-pennies version of the same payoff shape. If the opponent can identify a pure action and then choose the payoff-maximizing reply, then any policy that is a δ-peak for large δ is giving away the security level. Mixing on the near-optimal set is effective play in this case, and Aware-UNOP is a small refinement that puts less weight where the current forecast is already high, instead of mixing blindly. Proposition 3 is the opposite case. If the partner is trying to coordinate, a peak attains the Pareto-optimal payoff and mixing is a tax with no return from surplus-taking, so mixing is useful when there is room among near-best actions and the other side wants to take surplus. Proposition 4 is the claim that most of the empirical work is about. A greedy buyer publishes a step-function demand curve, so any revenue maximizer that knows that curve posts in the last grid cell at or below v and buyer surplus is then on the order of one grid step. An individually rational UNOP buyer publishes a band, with sure demand below v−αv-α, purchase rate 1/21/2 on [v−α,v][v-α,v], and sure refusal above v. Let p−=maxp∈:p<v−αp^-= \p :p<v-α\ and p+=maxp∈:v−α≤p≤vp^+= \p :v-α≤ p≤ v\. The sure-demand price strictly keeps the seller below every price in that half-purchase band when p−>p+/2p^->p^+/2. On a grid of spacing Δ , α<v/2−Δα<v/2- is a convenient sufficient condition for this inequality, and the buyer surplus is then Θ(α) (α) up to one grid step. Otherwise the proposition claims the demand curve, and a lower band rate is needed before the same surplus bound holds. Theorem 2 supplies the related concentration intuition, since a peak is identified in O(log(1/η))O( (1/η)) plays, whereas a uniform mixture on k near-ties leaves only an O(1/k)O(1/k) posterior peak. None of this requires the human to solve a long-horizon Stackelberg problem against a mean-based learner (Deng et al., 2019), because the policies we study are one-step mixers. A fully patient optimizer may still extract them. The experiments instead isolate the behavior of the named UCB and recommendation-list extractors. D.2 Experimental protocol This section records implementation choices that the main experiments section only summarizes. Every number in the paper comes from the released result files. The only additional runs are the ablations in Appendices F.4 to H.3. What the human sees. In bimatrix games the human does not see the optimizer’s current mixed action as a one-hot. They see the empirical mix of optimizer actions so far, form u^=UHσ u=U_H σ, and then apply π(⋅∣u^,P)π(· u,P). That is a mean-based, not a fully Bayesian, human, and it is closer to how a person looking at a platform’s recent prices actually has to reason. In pricing the menu is a single posted price, so u^=( 0,v−p) u=(\,0,\,v-p\,) is exact up to the noise ablation. In recommendation the menu is the current recommendation list plus the skip action, with skip’s utility equal to the reservation τ. We set τ to the 0.420.42 quantile of item quality in the synthetic catalog so that the outside option lies inside the quality range. On MovieLens we keep the same construction after scaling ratings from 1,2,3,4,5\1,2,3,4,5\ into [0,1][0,1]. What the optimizer sees. Bimatrix predictors are frequency counts, a first-order Markov chain, or online softmax regression on the optimizer’s own mix. The optimizer then chooses an exact best response to PtP_t, with a 5%5\% uniform random slip so that the dynamics cannot freeze on a pure profile by accident. The pricing optimizer is per-price Beta-Bernoulli UCB on p⋅ℙ(buy∣p)p·P(buy p). The recommender is a unique-best margin-seeking recommendation list: one offer is the quality-best item on the recommendation list, the rest are strictly worse filler items, and UCB is run on the offer’s acceptance. A vanilla ranking UCB that does not withhold better organic content is a weaker extractor, because it does not sit on τ. Burn-in. Reported means drop the first quarter of the horizon in bimatrix, recommendation, and strategic settings, and the first third in pricing, because we are interested in what an adaptive system does after it has had a chance to look at the human, not in the first few exploratory prices. Lock-in curves are the exception: they are the object of interest, so we keep the whole trajectory and define tlockt_lock as the first time a length-3030 rolling accuracy reaches at least 0.70.7. Seeds and pooling. Bimatrix: 1212 seeds × 33 predictors × 99 policies × 1010 games. Pricing: 2424 values of v. Recsys: 2020 catalog draws. Strategic: 1616 population draws. Public tables: MovieLens uses one job per user per policy on the official file. Adult, German, FICO HELOC, UCI Credit Card, Give Me Some Credit, and UCI Bank Marketing use 1616 seeds with a shared pretrained logistic classifier, then online updates on manipulated features. Ablations use 88 to 2424 seeds as listed in each subsection. Tables report means over seeds. Figures show error bars or seed scatters where useful. Reported effects are either large relative to seed noise or visibly null: greedy surplus near one grid step, UNOP surplus of order α, and MovieLens collapse. Compute. All of the above is NumPy/SciPy/scikit-learn on CPU. We used 9696 CPU processes. The original controlled suite is 45084508 rows. Public complete files add 66016601 MovieLens-100K jobs, 42,28042,280 MovieLens-1M jobs, and 144144 runs (1616 seeds × 99 policies) each for Adult, German, HELOC, Credit Card, Give Me Some Credit, and Bank Marketing. All completed with zero failures. Appendix E Bimatrix games E.1 Bimatrix games, one at a time Figure A8 is the predictability side of the same data. Table A4 reports mean human payoff in each of the ten games, averaged over the three predictors and twelve seeds. The ranking of policies is not a universal leaderboard. In matching pennies and rock, paper, scissors, greedy is a gift to the opponent and the mixed policies sit near or above zero. In coordination and stag hunt, greedy, small-α UNOP, and Aware-UNOP are tied at the top, and uniform is strictly worse. In prisoner’s dilemma every near-optimal policy collapses to defection, which is exactly what an adaptive counterpart does to a one-step cooperator. Table A4: Mean human payoff by bimatrix game, averaged over seeds and predictors. Mixing is mandatory in zero-sum, optional in general-sum, and costly in common-interest. Game Greedy ε -greedy UNOP α=0.15α=0.15 Aware-UNOP Uniform Matching pennies −0.146-0.146 -0.125 -0.000 0.0550.055 0.011 RPS −0.141-0.141 -0.113 0.000 0.0530.053 0.012 Prisoner’s dilemma 1.111 1.061 1.111 1.111 0.590 Chicken 1.765 1.616 1.818 1.871 1.382 Inspection 0.921 0.943 0.945 0.946 0.054 Shapley 0.339 0.345 0.391 0.412 0.333 Coordination 1.951 1.858 1.951 1.951 0.997 Stag hunt 4.878 4.789 4.878 4.878 3.959 Battle of the sexes 1.163 1.081 1.163 1.082 0.518 Pure coordination 0.965 0.902 0.964 0.964 0.329 Figure A8: Predictive accuracy by policy in the rows and game in the columns. Each cell is the numeric mean accuracy. Darker is more readable. Greedy is readable. Uniform is not. UNOP lands in between only where the game actually has room at the chosen α. A reasonable worry is that the three-case pattern comes only from using a frequency predictor. Figure A9 splits the same points by predictor. Frequency, Markov, and online softmax all produce the same qualitative cloud. Table A5 reruns greedy, UNOP-α=0.15α=0.15, and Aware-UNOP at T∈200,500,1000,2000T∈\200,500,1000,2000\ with a frequency predictor. In zero-sum games the mixed policies stay near the value as T grows, while greedy does not recover. In common-interest games the three policies that can coordinate stay coordinated. The three-case pattern is not a short-horizon illusion. Figure A9: Effectiveness against predictive accuracy, split by the optimizer’s predictor. Circles are zero-sum, squares general-sum, diamonds common-interest. The three-case pattern is not an accident of counting frequencies. Table A5: Human payoff versus horizon, grouped by game class under a frequency predictor. Zero-sum mixing remains necessary. Common-interest mixing remains a tax. Class Policy T=200T=200 T=500T=500 T=1000T=1000 T=2000T=2000 Zero-sum Greedy -0.004 -0.011 -0.008 -0.004 UNOP α=0.15α=0.15 0.035 -0.001 -0.002 0.009 Aware-UNOP 0.021 0.017 0.033 0.013 General-sum Greedy 1.206 1.196 1.197 1.201 UNOP α=0.15α=0.15 1.232 1.224 1.226 1.225 Aware-UNOP 1.240 1.228 1.228 1.228 Common-interest Greedy 2.625 2.607 2.596 2.595 UNOP α=0.15α=0.15 2.626 2.607 2.596 2.595 Aware-UNOP 2.626 2.607 2.596 2.595 In matching pennies the human’s actions are heads and tails, the optimizer wants to mismatch, and a greedy human who has a tiny preference for heads, or who simply broke a tie the same way twice, becomes a pure strategy that the optimizer can sit on. UNOP with α large enough to include both actions is just the mixed security strategy, which is why its payoff sits at 0.0000.000. Aware-UNOP does a little better (+0.054+0.054) because it is not mixing independently of PtP_t: it puts extra mass on whichever action the predictor currently underweights, and is therefore anti-correlated with a slowly adapting opponent. The gain is empirical against that learner. It is not a guarantee against an exact best responder. Appendix F Posted pricing F.1 Posted pricing: where the surplus actually goes The main-text pricing figure reports surplus and the effectiveness-predictability plane. Figure A10 shows the same runs as a scatter against the hidden value v. Greedy buyers lie on a tight band near zero surplus for every v: the monopolist found the threshold. UNOP buyers keep a wedge of surplus. Uniform and Boltzmann buyers go negative: they are sometimes paying more than v. Figure A11 is the time series: greedy posted prices climb toward v. UNOP stays below v. The hit rate should be compared with Bayes accuracy, not read as a failure to learn the mixture. Seed-level spreads appear as error bars in Figure A11. Table A1 reports means only, and the ordering is not an average of a bimodal mess. Figure A10: Left: buyer surplus against private value. Right: posted price against v, with the diagonal p=vp=v as a dashed reference. Greedy transactions hug the diagonal from below. UNOP leaves a visible gap. Boltzmann and uniform cross the diagonal. Figure A11: Pricing dynamics (rolling mean ± s.e.). (a) Posted price relative to v. (b) Predictor accuracy. (c) Running buyer surplus. Greedy prices climb toward v. UNOP stays lower. Accuracy for UNOP is bounded by the policy’s Bayes accuracy. F.2 A worked numerical example Suppose v=0.70v=0.70 and the price grid is 0.00,0.05,…,1.00\0.00,0.05,…,1.00\, so Δ=0.05 =0.05. A greedy buyer purchases at every p≤0.70p≤ 0.70 and refuses otherwise. After enough plays, UCB on revenue will prefer p=0.70p=0.70 to p=0.65p=0.65 once both empirical buy rates are near one, because 0.70⋅1>0.65⋅10.70· 1>0.65· 1, and it will prefer p=0.70p=0.70 to p=0.75p=0.75 once the latter’s empirical buy rate is near zero. Buyer surplus is then 00 when the posted price is exactly 0.700.70, or 0.050.05 if the optimizer sits one cell lower. In the main runs the mean leftover is 0.0180.018, which is a fraction of a cell, as Proposition 4 said. Now take UNOP with α=0.15α=0.15 and reservation 00. At p=0.50p=0.50, buy gives 0.200.20 and skip gives 00, so u⋆=0.20u =0.20 and u⋆−α=0.05u -α=0.05. Skip lies below that cut and buy is the unique α-optimal IR action, hence ℙ(buy)=1P(buy)=1. At p=0.60p=0.60, buy gives 0.100.10 and skip gives 00. Now u⋆−α=−0.05u -α=-0.05, so both actions lie in 0.15A_0.15 and both are IR, and the human mixes equally. At p=0.65p=0.65, buy gives 0.050.05 and skip gives 00. Both remain inside 0.15A_0.15 and both are IR, so the mix is again equal. At p=0.75p=0.75, buy gives −0.05-0.05, which is excluded by the reservation even though a Boltzmann policy would still buy with positive probability. A seller who wants demand near 11 is pushed down to p≤0.55p≤ 0.55, and the buyer keeps about 0.150.15. F.3 Grid resolution and horizon Tables A6 and A7 vary grid resolution and horizon. As the price grid goes from 1111 to 4141 cells at fixed T=500T=500, greedy surplus falls (0.0310.031 at 1111 cells, 0.0060.006 at 4141), which is Θ(Δ) ( ). UNOP-α=0.15α=0.15 remains above greedy at every resolution. Boltzmann and uniform stay negative. As T grows on the default 2121-cell grid, greedy surplus is already small at T=150T=150 and does not grow. UNOP surplus rises (0.0070.007 at 150150, 0.0610.061 at 800800), because post-burn-in means for UNOP improve once early UCB exploration is behind it. UNOP remains above greedy at every horizon in the sweep. The two sweeps should be read together. A finer grid is a larger action space for the monopolist: each additional cell needs samples before its revenue estimate is usable, so at fixed horizon a finer spacing leaves more leftover exploration and compresses surplus after the early rounds for every policy. Extending T gives the same optimizer time to finish that exploration, which is why surplus recovers as T grows even though the spacing is held fixed. In short, finite-horizon surplus tracks how far the extractor has gotten through its action set, not only how wide the human’s α-band is. The practical boundary is that UNOP still beats greedy on every grid we tried, but a very fine menu paired with a short horizon will look closer to the greedy grid step than the large-sample Θ(α) (α) band. Table A6: Buyer surplus versus the number of cells on the price grid. Policy 1111 cells 2121 cells 3131 cells 4141 cells Greedy 0.031 0.013 0.010 0.006 UNOP α=0.15α=0.15 0.065 0.036 0.021 0.012 Boltzmann 0.2 -0.010 -0.022 -0.028 -0.030 Uniform -0.140 -0.135 -0.143 -0.142 Table A7: Buyer surplus versus horizon T. Greedy tracks the grid step. UNOP remains above it. Policy T=150T=150 T=300T=300 T=500T=500 T=800T=800 Greedy 0.006 0.011 0.014 0.016 UNOP α=0.15α=0.15 0.007 0.020 0.042 0.061 Boltzmann 0.2 -0.039 -0.030 -0.020 -0.011 Uniform -0.150 -0.139 -0.135 -0.126 F.4 The individual-rationality ablation UNOP as defined in Section 3 intersects αA_α with a:u(a)≥u¯\a:u(a)≥ u\ whenever that intersection is nonempty. In pricing u¯=0 u=0 (skip). In recommendation u¯=τ u=τ. Table A8 turns that intersection off and reruns the same policies. Without the floor, UNOP-α=0.15α=0.15 in pricing goes from +0.036+0.036 to −0.018-0.018, and Aware-UNOP goes from +0.039+0.039 to −0.017-0.017. This is a separate smaller-seed ablation, while Table A1 reports 0.0400.040 with the floor on. That sign change is the whole argument for having a floor. Mixing into buy-above-value raises one-step entropy and also hands revenue to a learning monopolist. Greedy, ε -greedy, Boltzmann, and uniform are unaffected by the switch, because they never used the floor. In recommendation the same switch costs about 0.040.04 of surplus for UNOP. Table A8: Effectiveness with and without the individual-rationality floor. The floor is unused for greedy and for policies that already mix everywhere. For UNOP it separates preserved surplus from a teaching signal. Policy Pricing surplus Recsys surplus IR on IR off IR on IR off Greedy 0.012 0.012 0.050 0.050 UNOP α=0.15α=0.15 0.036 -0.018 0.136 0.094 Aware-UNOP 0.039 -0.017 0.105 0.055 Boltzmann 0.2 -0.022 -0.022 0.037 0.037 Uniform -0.135 -0.135 -0.079 -0.079 The floor is a modeling claim about extractors, not a claim that people compute a reservation and then mix above it. If a policy mixes into actions it would never take against a fixed menu, those actions teach the extractor where the cutoff still is. The reservation floor is what blocks that lesson. F.5 IR-constrained baselines A natural question is whether the gap between UNOP and Boltzmann/ε -greedy is entirely the reservation floor. Table A9 applies the same floor to those baselines: ε -greedy and Boltzmann are renormalized on a:u(a)≥u¯\a:u(a)≥ u\ whenever that set is nonempty. In pricing the floor turns Boltzmann’s negative surplus positive, but IR-Boltzmann 0.20.2 still trails UNOP-α=0.15α=0.15 (0.0280.028 vs. 0.0400.040). IR-ε -greedy remains near greedy. In recommendation, IR-Boltzmann 0.20.2 reaches 0.1130.113 surplus, still below UNOP at 0.1480.148. The floor removes teaching-signal losses. Uniform mixing on the near-optimal set is what keeps the extractor off the threshold. Table A9: Unconstrained vs. IR-constrained baselines on the same pricing and recsys loops as the main text (T=500T=500/300300). Policy Pricing Recsys surplus acc. surplus acc. Greedy 0.0180.018 0.9990.999 0.0520.052 0.7040.704 ε -greedy 0.0110.011 0.9540.954 0.0510.051 0.4890.489 IR-ε -greedy 0.0170.017 0.9700.970 0.0580.058 0.5640.564 Boltzmann 0.20.2 −0.021-0.021 0.6650.665 0.0240.024 0.1690.169 IR-Boltzmann 0.20.2 0.0280.028 0.7660.766 0.1130.113 0.3890.389 Boltzmann 0.50.5 −0.068-0.068 0.5950.595 −0.043-0.043 0.1340.134 IR-Boltzmann 0.50.5 0.0250.025 0.7380.738 0.0900.090 0.4400.440 UNOP α=0.15α=0.15 0.0400.040 0.8070.807 0.1480.148 0.3400.340 Uniform −0.133-0.133 0.4990.499 −0.094-0.094 0.1070.107 F.6 Misspecified utilities, and how much α has to absorb The theory treats u as known. In any honest deployment the human has u^=u+ξ u=u+ξ. Table A10 adds independent Gaussian noise of scale σ∈0,0.02,0.05,0.10,0.20σ∈\0,0.02,0.05,0.10,0.20\ to the utility vector before the policy is applied. In pricing, UNOP-α=0.15α=0.15 surplus falls from 0.0320.032 at σ=0σ=0 to −0.018-0.018 at σ=0.20σ=0.20. The last of those is noise larger than α, so the α-set constructed from u u is no longer a slightly wider true α-set. Raising α further at σ=0.20σ=0.20 does not recover surplus in our runs: at the same noise, α∈0.25,0.30,0.40,0.50α∈\0.25,0.30,0.40,0.50\ stays negative and slowly worsens, because a wider noisy band absorbs more wrongly signed actions through the reservation check. When σ is small relative to α, a default α on the order of twice the plausible utility noise is safer than a default that tries to be as small as the worst-case bound would suggest. When σ exceeds that scale, the binding fix is a better u u, not a larger band. Table A10: Effectiveness as a function of independent Gaussian noise on the human’s utility vector. UNOP remains useful while σ is small relative to α. At σ=0.20σ=0.20 a larger α does not restore pricing surplus. Env. Policy σ=0.00σ=0.00 σ=0.02σ=0.02 σ=0.05σ=0.05 σ=0.10σ=0.10 σ=0.20σ=0.20 Pricing Greedy 0.013 0.020 0.018 0.011 -0.012 UNOP α=0.15α=0.15 0.032 0.013 0.010 0.001 -0.018 Uniform -0.135 -0.136 -0.136 -0.136 -0.136 Recsys Greedy 0.044 0.090 0.109 0.106 0.065 UNOP α=0.15α=0.15 0.129 0.106 0.101 0.094 0.054 Uniform -0.076 -0.081 -0.081 -0.081 -0.081 Recommendation is gentler, because a typical recommendation list has several items whose qualities are already close. Even there, σ=0.20σ=0.20 is too much for α=0.15α=0.15. We do not claim a sharp phase transition. We claim that a tiny α is not enough once noise exceeds slack, and that widening the band alone is not a recovery recipe. Appendix G Recommendation and public data G.1 Margin-seeking recommendation lists, catalog size, and promoted exposure Figure A12 plots promoted-item click rate against user surplus for every synthetic recsys seed. Greedy seeds cluster at high promo and low surplus, UNOP seeds sit at lower promo and higher surplus, and uniform seeds occupy the high-promo, negative-surplus corner. Promoted exposure falls from 0.8580.858 to 0.3300.330 at α=0.15α=0.15 not because we penalized ads, but because the items that survive as unique-best offers are less often the promoted ones when τ is a band. Figure A12: Each point is one synthetic recommender seed. Horizontal: fraction of clicks on promoted items. Vertical: user surplus q−τq-τ. Table A11 asks whether that cloud depends on having chosen 4848 items and a recommendation list of 88. We crossed catalog sizes 24,48,72\24,48,72\ with recommendation-list sizes 4,8,12\4,8,12\ for greedy, UNOP-α=0.15α=0.15, and uniform. Greedy stays poor across the grid. Uniform stays worse. UNOP stays better, and it likes larger recommendation lists, because a larger recommendation list is more likely to contain a second item inside αA_α. A recommendation list of four with a unique-best offer and three filler items leaves little room: there may be nothing near-optimal to mix with except skip. Table A11: User surplus versus catalog size and recommendation-list size. The ranking does not depend on the particular (48,8)(48,8) pair used in the main text. Policy Slate cat. 2424 cat. 4848 cat. 7272 Greedy 44 0.039 0.037 0.039 88 0.055 0.055 0.064 1212 0.111 0.070 0.087 UNOP α=0.15α=0.15 44 0.125 0.092 0.052 88 0.161 0.148 0.111 1212 0.302 0.198 0.152 Uniform 44 -0.091 -0.131 -0.154 88 -0.027 -0.091 -0.127 1212 0.024 -0.049 -0.093 G.2 MovieLens, user by user, on the complete files The main text reports that on MovieLens, UNOP collapses to greedy because five-star ratings do not have slack. Figure A13 is the user-level answer on the official complete 100K file: the scatter hugs the diagonal, and the per-user difference is a spike at zero. Table A12 records the same collapse on MovieLens-1M (6,0406,040 users) and the promoted-item rates on both files. Greedy and UNOP remain close. Uniform is much higher. This is the same collapse on the complete file, without subsampling. Figure A13: Complete MovieLens-100K, all 943943 users. Left: per-user greedy versus UNOP. Right: per-user difference. The collapse is not an average over a hidden subpopulation. Table A12: Complete MovieLens files: surplus collapse and promoted exposure. UNOP uses α=0.1α=0.1. File N greedy surplus UNOP surplus greedy promo UNOP promo MovieLens-100K 943 0.189 0.187 0.085 0.096 MovieLens-1M 6040 0.187 0.185 0.065 0.084 Why is there no slack? Because a user who has rated twenty movies on a five-star grid often has a unique five-star title, and after scaling to [0,1][0,1] that title is 1.01.0 and the four-star titles are 0.750.75, so any α<0.25α<0.25 leaves a singleton αA_α. If a platform displayed utilities on a finer scale, such as a continuous rating, a dwell-time proxy, or a willingness-to-watch slider, the same protocol would look like the synthetic recsys panel. The public ratings are coarse, and the reported result is the one those ratings support. G.3 UCI Adult and German Credit Strategic classification on real tables is the other public-data pillar. The institution starts from a logistic classifier trained on unmanipulated official records (Adult accuracy 0.8520.852 on all 48,84248,842 rows after one-hot to d=108d=108, and German Credit 0.7850.785 on all 1,0001,000 rows, d=61d=61), then continues with online logistic updates on the features that agents actually present. Agents may game a small set of numeric coordinates that are at least arguably effort-like: age, education-num, capital gain/loss, and hours on Adult, and duration, credit amount, age, and existing credits on German. The action is one of five intensities along sign(w)sign(w) on those coordinates, at quadratic cost. The predictor is a frequency model over those five intensities. This is not a claim that people in the 1994 Adult file actually gamed their hours-per-week against an adaptive classifier. It is a claim that if they did, with a linear institution that maximizes accuracy, mixing would reduce readability without raising utility. Table A3 already reports agent utility and predictor accuracy on the complete UCI Adult and German Credit tables. Greedy is best or tied for best on utility. On Adult, UNOP-α=0.05α=0.05 gives utility/accuracy 0.1810.181/0.4280.428, versus 0.1780.178/0.4820.482 for greedy. Larger α costs utility without an extraction-flavored reward. Mixing does reduce readability. It just does not buy the agent anything the accuracy-seeking institution was threatening to take. Figure A14 (left) varies the quadratic cost coefficient on the synthetic strategic population. As cost rises, everyone games less and policy differences shrink. No cost in this family makes UNOP outperform greedy by a margin comparable to pricing. The counterpart maximizes accuracy rather than surplus. Figure A14: Left: synthetic strategic classification as the manipulation-cost coefficient c varies. Right: steps to 70%70\% lock-in as the action-set size k varies. Greedy and ε -greedy lock in 3030 steps at every k (the window length). UNOP and uniform do not lock within the horizon. Appendix H Lock-in and other extractors H.1 Lock-in, populations, and the public-good reading Theorem 2 is a Dirichlet multinomial calculation, and Figure A14 (right) is the empirical version as k=|A|k=|A| grows. This separate k-sweep uses T=400T=400. The main lock-in experiment uses T=350T=350. Greedy and ε -greedy hit 70%70\% rolling accuracy at t=30t=30 for every k we tried. Uniform almost never hits it within the sweep horizon. UNOP at α=0.15α=0.15 tracks uniform on that random utility draw once α admits several near-ties. The lock-in figure (Figure A15) used a vector whose near-ties were farther apart, so small α still leaves a peak. Figure A15 also shows the population overlay. A shared margin-seeking optimizer trained on a mix of greedy and UNOP users has an uneven accuracy curve: highest on a pure-greedy population and lowest around a 30%30\% UNOP minority. Greedy users’ welfare rises when they are mixed with UNOP neighbors. Figure A15: Left: steps to 70%70\% lock-in (mean ± sd) with marker color == final accuracy. Dotted line is the horizon. Right: welfare of greedy and UNOP users in a shared-optimizer population (shaded gap) and the extractor’s accuracy (right axis, min marked). Heterogeneity is itself a defense. H.2 Two tradeoffs, Gibbs leak, and why temperature is the wrong choice Figure A4 already contrasts the two tradeoffs on a six-action utility vector. The same qualitative gap (smooth Gibbs leak versus UNOP step support) appears on random vectors of dimension 44 through 1616 in the dump pareto.json. Figure A16 makes that picture concrete. The left panel is the mean size of αA_α as a function of α. The right panel is the amount of Gibbs mass that sits outside αA_α as a function of α, for several temperatures. At τ=0.2τ=0.2 and α=0.15α=0.15, more than a quarter of the Gibbs mass is already off the near-optimal set. Temperature is not a substitute for a hard support constraint. The two-panel α-sweep in Figure A7 is the corresponding empirical picture: effectiveness rises and then falls, and accuracy falls as soon as αA_α is non-singleton. Figure A16: Left: mean size of the α-optimal set on random utility vectors. Right: Gibbs mass lying outside αA_α. Temperature is not a substitute for a hard support constraint. H.3 A different extractor: Thompson sampling UCB is a particular way to turn uncertain demand into a posted price. Table A13 reruns the pricing environment with a Beta-Bernoulli Thompson monopolist that posts argmaxpp⋅θp _pp· _p for θp∼Beta(ap,bp) _p (a_p,b_p), and includes Hedge as an additional human baseline. The qualitative ranking does not flip: uniform is still a disaster, greedy is still extracted, UNOP still keeps surplus. The UNOP surplus is larger against Thompson (0.1390.139) than against UCB (0.0360.036) in this parameterization, because Thompson is more willing to post inside a noisy band. Hedge is near greedy against UCB and slightly negative against Thompson, consistent with mean-based learning not being a defense against a posted-price extractor. Table A13: Buyer surplus against a UCB monopolist and a Thompson monopolist, same price grid and values of v. The human-side ranking is stable. Hedge is not a substitute for UNOP. Policy UCB surplus Thompson surplus Greedy 0.012 0.022 UNOP α=0.15α=0.15 0.036 0.139 Uniform -0.135 -0.132 Hedge 0.003 -0.003 H.4 Putting the environments together The environments do not share a utility scale, so a single scatter is unreadable. The main-text three-case figure, the pricing/recsys tables, and Table A3 already locate each setting. If UNOP were simply the random policy, it would sit with uniform. If it were simply greedy with a different name, it would sit with greedy. On pricing and recsys it sits up and left of greedy. On both MovieLens files it sits on top of greedy. On Adult and German it sits down and left: less readable, not more effective. Predictability is correlated with policy entropy across environments, but two policies with the same entropy can be very different for the human if one spends that entropy inside αA_α and the other spends it on dominated actions. Accuracy, optimizer payoff, and human payoff are therefore the reported outcomes. Entropy alone does not determine them. H.5 A second worked example: a margin-seeking recommendation list Imagine four items with qualities (0.90,0.72,0.55,0.40)(0.90,0.72,0.55,0.40) and a reservation τ=0.60τ=0.60. The platform can show three of them. A unique-best margin-seeking recommendation list that has identified τ will offer the item with quality 0.720.72 as the unique best (it is just above τ) and fill with 0.550.55 and 0.400.40 as filler items, because 0.900.90 would give the user a large surplus and a lower margin. A greedy user clicks 0.720.72 every time, surplus 0.120.12, and the platform has sat on τ. An UNOP user with α=0.15α=0.15 sees utilities (0.12,−0.05,−0.20,0)(0.12,-0.05,-0.20,0) for the three items and skip. The α-optimal IR set is 0.72-item,skip\0.72-item,skip\, because skip is 00 and the offer is 0.120.12, which differ by less than 0.150.15, while the filler items are below reservation. The user therefore mixes equally between taking the offer and skipping. The platform now sees a 50%50\% acceptance rate on an item that would have been a sure thing if τ were 0.600.60 with a greedy user, and UCB cannot treat 0.720.72 as an identified threshold. If the catalog had also included an organic 0.850.85 item on the recommendation list, that item would have entered αA_α as well, and the mix would have included a high-quality click. Withholding is how the platform tries to prevent that. Mixing on skip is how the user refuses to certify the withheld menu. Uniform, on the same recommendation list, clicks the 0.400.40 item with positive probability, which is worse than skip, and the platform learns something even less helpful to the user: that junk is sometimes acceptable. Unpredictability without a floor is not a defense. Appendix I Limitations and further checks I.1 Limitations The response-curve commitment effect applies when the platform optimizes against that curve, when feasible menus leave more than one acceptable action, and when the utility estimate places the set correctly. Noise large enough to move the estimated set removes the surplus, which is why the pricing gain and the recommendation gain should be read together with the noise and singleton checks in Section 4.2. We list explicitly what this extra empirical work does not establish. We have not shown that UNOP cannot be exploited by a patient optimizer who knows the policy class and chooses a best response to the mixture in closed form. We have shown that the adaptive UCB and margin-seeking recommendation-list extractors used in the experiments cannot sit on a band the way they sit on a threshold. We have not shown that users will adopt α as a slider. We have shown that if they did, the effect of changing α rises and then falls, rather than more randomness always being better. We have not shown that MovieLens would look like the synthetic recsys panel under a continuous rating. We have shown that under the ratings the file actually contains, there is no slack to mix. We have not shown that Adult applicants should randomize their hours. We have shown that against an accuracy-maximizing logistic institution, randomization buys a policy that is harder to read, and not more utility. We have not run a user study or queried a deployed API. Every optimizer and every human policy in this paper is a simulator. Across these checks, mixing helps inside slack and with a reservation. It does not help when the counterpart is a collaborator or a pure accuracy maximizer. A five-star grid does not create a non-singleton α-band. I.2 The ten bimatrix games This subsection records each bimatrix game in words, together with the mean payoff over predictors and seeds. The payoffs are those in src/envs.py. The numbers are means from bimatrix.csv. Matching pennies. The human wants to match, the optimizer wants to mismatch, and there is no slack in the interesting sense because against a best responder the two actions are equally good at mixed equilibrium and equally terrible if played pure. A greedy human who breaks a tie toward heads twice in a row has published a peak. Frequency counts move, the optimizer plays tails, and the human’s continuation value is −1-1 until they mix again. In the runs, greedy finishes at about −0.15-0.15 rather than −1-1 because of the 5%5\% random slip and because identification is not instantaneous. The sign of the payoff is the result that matters. UNOP with α large enough to include both actions is the mixed security strategy and finishes at 0.0000.000. Aware-UNOP finishes at +0.054+0.054, which is the gain from avoiding a slow predictor. Uniform is statistically the same as UNOP here, because the α-set is the whole action set. In this game, remaining effective requires remaining unpredictable. Rock, paper, scissors. The same conclusion holds with three actions. Greedy that locks onto rock is giving the optimizer paper. UNOP that mixes on all three when they are near-ties is the usual security mix. The numbers track matching pennies closely (greedy about −0.14-0.14, UNOP 0.000.00, Aware-UNOP +0.05+0.05). Prisoner’s dilemma. Both players have a dominant strategy to defect if the other is held fixed, and an adaptive counterpart that can predict cooperation will defect against it. Every near-optimal policy in our suite therefore collapses to defection and gets about 1.111.11, the mutual-defection payoff, well below the cooperative payoff of 33. UNOP is not a cooperation device. If the human wants to cooperate against an extractor, they need a different theory (commitment, repeated-game arguments, outside enforcement), not a mixer on αA_α. Chicken. Swerving versus staying has an interior mixed equilibrium and two asymmetric pure equilibria. A greedy human who looks like they will stay can induce the optimizer to swerve, which is good for the human, until the optimizer’s predictor becomes confident enough to stay instead. UNOP-α=0.15α=0.15 edges greedy (1.821.82 versus 1.771.77) because mixing in the near-tie region makes it harder for the optimizer to pick the punishing pure strategy and sit on it. The gap is small. In this general-sum game a little slack is mildly useful, neither mandatory nor forbidden. Inspection. A worker who always works is wasting effort when the inspector is not inspecting. A worker who always shirks is caught once the inspector learns that. The α-set often contains both work and shirk against the empirical inspect rate, which is the support on which UNOP mixes. UNOP-α=0.05α=0.05 gets 0.960.96 against greedy’s 0.920.92. The gap is small and in the direction predicted by the theory. Shapley’s game. A cyclic general-sum game that is a standard test for learning dynamics because no pure equilibrium is stable. Mixing is almost unavoidable, and the policies that already mix do better than greedy, which keeps trying to sit on a disappearing best response. Battle of the sexes. Two pure equilibria, disagreement about which one, and a mixed equilibrium that both dislike. A greedy human who looks determined can sometimes pull the optimizer onto their preferred equilibrium, which is a case where being predictable is helpful even though the game is not common-interest. UNOP does not destroy that, because small α still leaves a peak on the currently better equilibrium, but large α mixes toward the mixed equilibrium and pays the coordination tax of Proposition 3. Coordination and pure coordination. Greedy, small-α UNOP, and Aware-UNOP all sit at the Pareto-optimal pure profile (payoff 1.951.95 on the 2,02,0 and 0,10,1 game, 0.9650.965 on three-action pure coordination, 4.884.88 on stag hunt, and suite mean 2.5982.598). Uniform falls to 1.001.00 on the two-action game. Predictor accuracy is 1.01.0 for the peaked policies. If a deployment cannot tell whether the counterpart is a collaborator or an extractor, UNOP run by default against a helper misses the meeting point. Stag hunt. The Pareto-optimal profile is (stag, stag) at 55, the safe profile is (hare, hare) at 33, and a greedy human who is predicted to hunt stag will be met there by an adaptive partner. Mixing toward hare fails to reach the stag equilibrium. Our runs put greedy and UNOP together at 4.884.88 and uniform at 3.963.96, which is the coordination tax. UNOP is not an appropriate default when the counterpart is a cooperative assistant completing a joint task, which is the content of Proposition 3. I.3 Hyperparameters Table A14 lists α and the other experimental settings. Table A14: Experimental settings. Only α is a user-facing parameter. The rest are simulation choices checked by ablations. Setting Value(s) used Role Bimatrix T 10001000 (ablation: 200200 to 20002000) Long enough for frequency counts to concentrate. Ablations check that the ranking does not flip. Bimatrix seeds 1212 Enough to see seed spread. Bimatrix random slip 5%5\% uniform Prevents exact freeze on a misidentified pure profile. Pricing T 500500 (ablation: 150150 to 800800) UCB has time to leave exploration. Longer T helps UNOP relative to greedy. Price grid 2121 points on [0,1][0,1] (ablation: 1111 to 4141) Mesh Δ≈0.05 ≈ 0.05. Finer grids extract greedy more, consistent with Proposition 4. v U[0.25,0.95]U[0.25,0.95], 2424 draws Avoids trivial v near 00 or 11. Recsys T 300300 Catalog identification is cheaper than pricing. Catalog / list size 4848 / 88 (ablation: 2424 to 7272 / 44 to 1212) Unique-best offer plus filler items. Ranking stable across the grid. Reservation τ 0.420.42-quantile of quality Interior outside option. Neither never-skip nor always-skip. Promo fraction 0.350.35 Enough ads to make withholding tempting. Strategic T 700700 synth. / 30003000 Adult / 20002000 German Online logistic has time to retrain. Manipulation levels 55 deltas Small discrete action set so UNOP’s α-set is interpretable. Cost c 0.550.55 synth. / 0.450.45 real (ablation: 0.150.15 to 1.051.05) Interior: some gaming, not always-max or never. Lock-in T / window 350350 / 3030 Window defines 70%70\% lock-in. Greedy always hits at 3030. Population users 4040, T=500T=500, 1212 seeds Shared optimizer. Fraction of UNOP from 00 to 11 in steps of 0.10.1. α main 0.05,0.15,0.3\0.05,0.15,0.3\ extraction. 0.1,0.2\0.1,0.2\ MovieLens Named slack in payoff units. MovieLens uses smaller values because ratings are already in [0,1][0,1] with 0.250.25 star gaps. IR floor 00 in pricing, τ in recsys, off in bimatrix Ablation in Appendix F.4 shows why the first two are not optional. Utility noise σ 00 main. 0.02,0.05,0.10,0.20\0.02,0.05,0.10,0.20\ ablation α absorbs small σ. σ>ασ>α does not. Workers 9696 CPU processes CPU only. The only user-facing hyperparameter is α, in payoff units (dollars, stars, minutes of extra commute, points of approval probability). The remaining choices are experimental, and the ablations indicate that they do not carry the result. I.4 Aware-UNOP Unaware UNOP is uniform on αA_α. Aware-UNOP reweights that set by (P(a)+ε)−1(P(a)+ )^-1, so that currently over-predicted near-optimal actions are downweighted and currently under-predicted ones are upweighted. It is a small step that avoids the current forecast, not a no-regret algorithm and not a full best response to the optimizer. In matching pennies it is the difference between 0.0000.000 and +0.054+0.054, because avoiding a slow predictor raises payoff. In pricing it matches unaware surplus (0.0410.041 versus 0.0400.040) at slightly lower accuracy (0.7850.785 versus 0.8070.807), which is the same-effectiveness, harder-to-read refinement. In recommendation it is the least predictable effective policy (accuracy 0.2410.241) at a small surplus cost relative to unaware α=0.15α=0.15 (0.1240.124 versus 0.1480.148). In Adult it is the least readable policy in the table and not the most effective, which is again the weak-dilemma pattern. The reason we did not make awareness the default is practical as much as theoretical. It needs a forecast PtP_t in the same action simplex the human is mixing on. A user who sees a posted price can build a crude P(buy)P(buy) from how often they have been seeing that price. A user who sees a recommendation list can less easily know the platform’s predictive distribution over clicks. Unaware UNOP only needs u and α. If an assistant can see PtP_t, it should use it. If it cannot, uniform-on-the-band is the distribution characterized by the theorems. A numerical snapshot, again with v=0.70v=0.70 and α=0.15α=0.15. Suppose the posted price is 0.650.65, so α=skip,buyA_α=\skip,buy\, and the monopolist’s current P(buy)=0.8P(buy)=0.8. Unaware mixes equally. Aware mixes in proportion to ((P(skip)+ε)−1,(P(buy)+ε)−1)≈(1/0.2, 1/0.8)=(5,1.25) ((P(skip)+ )^-1,(P(buy)+ )^-1 )≈(1/0.2,\,1/0.8)=(5,1.25) on (skip,buy)(skip,buy), i.e., more skip than buy, which is the direction that surprises a seller who has become confident that this price still sells. The next P(buy)P(buy) comes down, and the next mix tilts back. That oscillation is why awareness reduces accuracy without needing to touch dominated actions. I.5 Observed failure cases The following negative results are in the released files. First, MovieLens-100K and MovieLens-1M, complete files, UNOP indistinguishable from greedy at the surplus level that matters. The failure is the absence of slack: the five-star grid does not give 0.1A_0.1 anything to contain except the top star. Second, accuracy-seeking logistic classification on Adult and German Credit, complete tables. UNOP reduces the accuracy of a manipulation-level forecast and does not raise agent utility. The failure is of the adversary’s objective: there is no surplus to protect against an institution that is not extracting it. Third, UNOP without an IR floor, pricing environment. Surplus changes sign, from +0.036+0.036 to −0.018-0.018. The failure is of the definition: mixing into buy-above-value is a teaching signal. The definition in the main text therefore includes the floor. Fourth, utility noise σ=0.20σ=0.20 with α=0.15α=0.15. UNOP pricing surplus goes negative. The failure is that noise exceeds slack. A separate pricing sweep at the same σ with α up to 0.500.50 stays negative, so the binding fix is a better utility estimate, not a larger band. Fifth, prisoner’s dilemma and the common-interest suite, where UNOP is either irrelevant (everyone defects) or actively unhelpful (mixing is a coordination tax). The failure is of the environment class, which the three-case pattern separates from surplus extraction. Sixth, Hedge against a posted-price extractor. Near greedy against UCB, slightly negative against Thompson. No-regret on realized utilities does not protect a threshold. Mean-based learning still publishes one. Each of these cases limits where mixing helps. Slack can be absent, the counterpart can be maximizing accuracy instead of surplus, and exploration below the reservation can teach the extractor. I.6 Statistical presentation We report means over seeds, with error bars or seed scatters on the figures. We do not report p-values for the main contrasts. The pricing gap is 0.0420.042 versus 0.0180.018 surplus, with greedy’s standard deviation 0.0160.016 and a positive gap on most matched draws of v (n=24n=24). The recommendation gap is 0.1480.148 versus 0.0520.052. The MovieLens comparison is a per-user scatter on the diagonal for all 943943 users and then all 6,0406,040 users (Figure A13). The Adult and German pattern repeats on both tables and on the synthetic population. The oracle-pricing and recommendation contrasts in Section 4.2 can be recomputed from supplement/sprint, matched by value or catalog.