Paper deep dive
Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation
Serafima Lebedeva, Sumantrak Mukherjee, Ali Arshad Sadal, Ilias Ekşi, Rahul Sharma, Julia Mueller, Theresa Dombrowski, Jakob Karolus, Viktor Bengs, Eyke Hüllermeier, Sebastian Vollmer
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Many recommender services repeatedly encounter cold-start cohorts, where new users arrive with little or no interaction history. This creates two challenges: learning user preferences quickly from limited feedback and sustaining useful recommendations when each user has a finite catalog that can become repetitive or depleted over time. We propose CohortMix-TS, a warm-started mixture bandit that learns latent user groups from earlier cohorts and uses available metadata to construct group-informed priors for new users. Starting from these fixed priors, the model personalizes independently as feedback from each user becomes available. Session slates combine Thompson sampling with diversity and inventory-depletion controls. We evaluate CohortMix-TS through simulation, semi-synthetic experiments, and a 25-day randomized in-the-wild deployment with 713 registered participants in a Campus Games quiz application. Our evaluations show that cross-cohort transfer improves early recommendation quality and user-level regret, while inventory-aware slate construction helps prevent premature exhaustion of preferred items. In the field deployment, treatment users also showed a larger early-to-late change in correctness than users receiving random recommendations. Together, these results show how warm-start transfer and inventory-aware recommendations can support personalization for short-lived, repeatedly cold-starting cohorts.
Tags
Links
- Source: https://arxiv.org/abs/2609.37800v1
- Canonical: https://arxiv.org/abs/2609.37800v1
Trouble viewing inline? Open PDF directly →
Full Text
64,669 characters extracted from source content.
Expand or collapse full text
Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation Serafima Lebedeva 1,2,∗ , Sumantrak Mukherjee 1,∗ , Ali Arshad Sadal 2 , Ilias Ekşi 2 , Rahul Sharma 1,2 , Julia Mueller 2 , Theresa Dombrowski 2 , Jakob Karolus 1,2 , Viktor Bengs 1 , Eyke Hüllermeier 1,3 , Sebastian Vollmer 1,2∗ Abstract Many recommender services repeatedly encounter cold-start co- horts, where new users arrive with little or no interaction history. This creates two challenges: learning user preferences quickly from limited feedback and sustaining useful recommendations when each user has a finite catalog that can become repetitive or depleted over time. We propose CohortMix-TS, a warm-started mixture bandit that learns latent user groups from earlier cohorts and uses available metadata to construct group-informed priors for new users. Start- ing from these fixed priors, the model personalizes independently as feedback from each user becomes available. Session slates com- bine Thompson sampling with diversity and inventory-depletion controls. We evaluate CohortMix-TS through simulation, semi- synthetic experiments, and a 25-day randomized in-the-wild de- ployment with 713 registered participants in a Campus Games quiz application. Our evaluations show that cross-cohort transfer im- proves early recommendation quality and user-level regret, while inventory-aware slate construction helps prevent premature ex- haustion of preferred items. In the field deployment, treatment users also showed a larger early-to-late change in correctness than users receiving random recommendations. Together, these results show how warm-start transfer and inventory-aware recommendations can support personalization for short-lived, repeatedly cold-starting cohorts. Keywords recommender systems, multi-armed bandits, cold start, slate rec- ommendation, cross-cohort transfer 1 Introduction Recommendation systems select items in video platforms [10], re- tail services [13], news systems [21], and mobile applications [33]; broader accounts describe recommender systems [39] and their evolution [5]. In several applications, users are recruited or acti- vated in cohorts. Flash-sale periods [29] create short-lived recom- mendation tasks as available assortments change, while online- learning systems [31] repeatedly select activities for new learners [9]. Campaign-style games and health interventions [44] similarly place many participants on a common schedule. A cohort is active for a short horizon, so the recommender must personalize early enough to make the displayed items useful to each user. New cohort members commonly arrive without a current in- teraction history or a complete context for personalization. The ∗ 1 German Research Center for Artificial Intelligence (DFKI), Kaiserslautern; 2 RPTU, Kaiserslautern-Landau; 3 LMU, Munich. ∗ Equal contribution: ser- afima.lebedeva@dfki.de, sumantrak.mukherjee@dfki.de. operational setting also differs from an unconstrained ranking prob- lem: items cannot usually be repeated for a user, a user’s candi- date pool is finite, and some users disengage before the campaign ends. Learning a user’s preferences from feedback is therefore a se- quential decision problem. Multi-armed bandits provide a standard framework for this problem [4], and surveys document their use in recommendation [43]. Thompson sampling [1] balances exploration of uncertain items with exploitation of items whose current reward estimates are high; information-directed sampling [40] provides a related Bayesian decision rule. In a short campaign, however, explo- ration that would pay off only after a long horizon can consume a material share of the available interactions. Moreover, a policy that repeatedly favors one item type can use its finite per-user inven- tory too early. Item similarity could support generalization across items, but sufficiently rich item metadata are not always available to define that similarity directly. Collaborative filtering learns long-term user–item structure from earlier interactions, but a newly arriving user has not supplied the observations needed for an individual factor estimate. It also does not prescribe how to choose sequential slates or how much uncer- tainty to explore. Earlier cohorts nevertheless contain population- level preference structure that can provide a warm start for the next cohort. We use that structure as an uncertain, metadata-conditioned prior rather than as a fixed user assignment: users with the same metadata can still belong to different preference groups, and current feedback can separate them. We address this setting with CohortMix-TS, a cross-cohort mix- ture bandit. It transfers historical item and user structure into soft priors, constructs feasible slates through an arm-level Thompson rule, and performs scheduled batch updates between sessions. The empirical study is central to the paper: we pair controlled simula- tions with a 25-day randomized in-the-wild deployment of the full policy in the Campus Games of the RPTU 1 . Four questions orga- nize this study: RQ1 asks whether historical warm starts improve early decisions; RQ2 asks whether soft memberships help when users with the same metadata have different preferences; RQ3 asks whether diversity and depletion controls preserve useful items for later rounds; and RQ4 asks whether the complete policy differs from random recommendation in the in-the-wild study. This paper makes the following contributions: • We formulate cross-cohort slate recommendation with his- torical cohort data, cold-start users, delayed binary feed- back, user-level no-repeat constraints, finite per-user in- ventories, and changing active-user sets. 1 Code is available at https://github.com/etowho/university-games. arXiv:2609.37800v1 [cs.LG] 29 Sep 2026 Lebedeva and Mukherjee et al. •We introduce CohortMix-TS, which combines historical la- tent structure, soft metadata-conditioned priors, Thompson sampling, diversity and depletion controls, and scheduled approximate hierarchical updates. • We evaluate the mechanisms in two parametric environ- ments and a source-only 2024-to-2025 data-calibrated semi- synthetic environment with replication-based uncertainty estimates. • We report a randomized 25-day in-the-wild deployment of the complete policy against random recommendation with 713 registered participants. Section 2 positions the work, Section 3 defines the setting, and Section 4 describes the method. Sections 5 and 6 give the evaluation design and results, respectively. 2 Related Work Cold-start recommendation and transfer. Collaborative filtering estimates low-dimensional user–item structure from observed inter- actions [23]; implicit-feedback variants model a related signal [17]. Factorization bandits [45] and collaborative-filtering bandits [28] combine such shared representations with sequential item selec- tion. Matrix-factorization variants can represent multiple user tastes [7,8]. Long-running datasets, including MovieLens [14] and MIND [46], provide the interaction traces on which such models rely. A new user faces the cold-start problem [6]; an occasional visitor may likewise lack direct feedback at arrival [3]. Cold-start methods therefore use side information, population structure, or transferred representations to initialize a model. Meta-bandits [24], hierarchical bandits [16], and warm-start bandits [37] likewise transfer shared structure across related tasks. Dynamic collaborative filtering [18], empirical-Bayes multi-bandits [19], and empirical-Bayes meta-prior learning [36] offer related approaches. CohortMix-TS instead uses historical factors to form item arms and metadata-conditioned pri- ors, then revises arm beliefs during the short target cohort. Online, contextual, and clustered bandit recommendation. Bandits formalize the choice between gathering information and choosing items with high current reward estimates. Contextual bandits use observed features [2,26], while clustered variants share information through inferred groups [32,47]. A contextual model restricted to categorical metadata learns pooled metadata–arm effects but does not maintain a user-specific latent state; we evaluate this represen- tation with Metadata LinUCB and Metadata LinTS. Our scheduled updates are operational feedback boundaries, rather than adaptive batching [20]. Field recommender studies [12,34] and large pro- duction decision systems [35] have shown why deployed behavior can differ from offline metrics. Partial feedback complicates un- biased offline evaluation [27]; treatment-style recommendations complicate causal evaluation [41]; and evaluation surveys summa- rize these concerns [48]. This motivates both randomization and careful outcome definitions. Constrained slate recommendation. Slate-aware rankers model interactions in a displayed list [38], and slate-likelihood Thompson policies can diversify it [11]. Diversified contextual combinatorial bandits use determinantal point processes for a richer relevance– diversity trade-off [30]. These approaches are complementary to our arm-level selector: a slate-aware re-ranker could operate on feasible within-arm candidates. Multi-item mobile recommendation also exhibits slate and position effects [15]. Selecting a session slate changes future availability. Sleeping bandits motivate changing action sets [22]; rotting bandits address repeated use [25]; and non- stationary bandit methods offer related tools [42]. Our availability instead comes from user-level no-repeat constraints rather than a globally disappearing catalog. We use a penalty score rather than claim an exact constrained-bandit solution. Together, these lines leave a practical gap: a policy must transfer uncertain cohort structure, adapt within a short campaign, and construct feasible multi-item sessions from a finite per-user catalog. 3 Problem Formulation We write[푛]=1, . . .,푛. LetQdenote the fixed item catalog,푇 the number of rounds, and퐾the number of items presented to each active user in a round. The catalog is the same in every round; a user’s available subset changes only after items have been displayed to that user. 3.1 Cohorts, information, and indexing In our setting, a set of users who join during the same time period forms a cohort. LetUbe the users in the current cohort. Histor- ical dataD hist contain records from earlier cohorts of the items displayed to a user and the resulting binary outcome. Each new user푢 ∈Uarrives with an observed metadata group푔 푢 , but with no outcome from the current cohort. We index rounds by푡 ∈ [푇], items by푞 ∈ Q, and users by푢 ∈U. LetU 푡 ⊆ Ube the users who are active in round푡. The active set can shrink when users leave or do not complete a round. 3.2 Cross-cohort slate environment For each푢 ∈U 푡 , the learner presents a slate푆 푢,푡 ⊆ Qof퐾distinct items. The outcome for item푞 ∈ 푆 푢,푡 is푦 푢,푡,푞 ∈ 0,1. Before round 푡 , the items previously displayed to user푢 are Q seen 푢,푡−1 = Ø 휏<푡 푆 푢,휏 . The learner therefore selects푆 푢,푡 ⊆ Q \ Q seen 푢,푡−1 . It receives the outcomes for a slate after the round, then uses the accumulated observations to form later slates. The final slate is a set: its display order is not part of the model. 3.3 Objective and evaluation The purpose of personalization is to make useful choices for every user throughout the푇rounds. We therefore seek high reward in each round and evaluate the learner by cumulative reward over the cohort horizon: max 휋 E 휋 푇 ∑︁ 푡=1 ∑︁ 푢∈U 푡 ∑︁ 푞∈푆 푢,푡 푦 푢,푡,푞 .(1) The policy휋uses푔 푢 ,D hist , and outcomes observed before round푡 when selecting a feasible slate. In the synthetic environments, the evaluator knows the gener- ated reward probabilities and can report pseudo-regret as an addi- tional measure of how efficiently the learner personalizes. Those Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation probabilities are unavailable in the in-the-wild study, so we compare the change in observed performance for the personalized learner and the random control. For user푢, the scalar early–late change is Δ 푢 =퐶 푢 (푊 late 푢 )−퐶 푢 (푊 early 푢 ),(2) where퐶 푢 (푊)is that user’s mean of daily correctness values over observed days in time window푊. Users can also leave the study or complete only some rounds. Better personalization may improve continued participation, so we report participation and completion accounting as secondary field measures; cumulative reward remains the primary optimization metric. 4 CohortMix-TS: Warm-Started Mixture Bandits 4.1 Historical structure and cold-start priors Earlier cohorts commonly provide response records for the same finite catalog while a newly arriving user has no current-cohort outcomes. We use those records to group items into arms and to form priors that make initial personalization possible. Let푅 ∈ [0,1] 푛 퐻 ×|Q| be the sparse historical user–item response matrix, where푅 ℎ,푞 is the mean binary outcome for historical userℎand item푞. LetΩ ⊆ [푛 퐻 ]×Qbe the set of observed entries of푅. We fit a rank-푑regularized matrix factorization, where푈 ∈ R 푛 퐻 ×푑 and 푉 ∈ R |Q|×푑 are the historical-user and item factor matrices: min 푈,푉 ∑︁ (ℎ,푞)∈Ω (푅 ℎ,푞 −푈 ⊤ ℎ 푉 푞 ) 2 + 휆 푈 ∥푈∥ 2 퐹 + 휆 푉 ∥푉∥ 2 퐹 .(3) The regularization weights휆 푈 and휆 푉 penalize large historical-user and item factors, respectively. The latent dimension푑is chosen during historical preprocessing and controls the capacity of these representations. Each row푉 푞 represents an item and each row푈 ℎ represents a historical user. Clustering the item rows푉 푞 partitions the catalog into퐴item armsA=1, . . .,퐴, with arm푎contain- ingQ 푎 . Clustering historical-user rows, together with metadata where available, yields퐶latent user groupsC=1, . . .,퐶. For the deployment,퐴and퐶were selected before enrollment; Section A gives those values and the candidate ranges. For a new user, the learner observes metadata group푔 푢 but not the latent group to which the user belongs. Earlier cohorts estimate the association between metadata and latent groups. The vector푝 (0) 푢 ∈ [0,1] 퐶 therefore has entries푝 (0) 푢,푐 = b Pr(푐 | 푔 푢 )and sums to one; an unseen metadata value receives the global group frequencies. For each group–arm pair, the historical success and failure counts푠 hist 푐,푎 and푓 hist 푐,푎 give a smoothed mean and a fixed- strength Beta prior. Here,훼 0 and훽 0 are smoothing pseudo-counts, and 휅> 0 is the selected effective prior strength: b휇 푐,푎 = 훼 0 +푠 hist 푐,푎 훼 0 + 훽 0 +푠 hist 푐,푎 + 푓 hist 푐,푎 , (훼 퐻 푐,푎 ,훽 퐻 푐,푎 )=휅(b휇 푐,푎 , 1−b휇 푐,푎 ). (4) The quantities훼 퐻 푐,푎 and훽 퐻 푐,푎 are the historical success and failure pseudo-counts for group푐and arm푎. For a new user, the initial arm belief aggregates uncertain group membership, 훼 (0) 푢,푎 ,훽 (0) 푢,푎 = 퐶 ∑︁ 푐=1 푝 (0) 푢,푐 훼 퐻 푐,푎 ,훽 퐻 푐,푎 .(5) Equation (5) summarizes the metadata-conditioned mixture by a single Beta distribution. It preserves the mixture’s expected reward for arm푎, the center of the warm-started Thompson distribution be- fore the user has supplied feedback. Fixing휅gives every user–arm prior the same, controlled amount of historical influence, indepen- dent of how many observations happened to be recorded for a historical group–arm cell. This first-moment match gives compa- rable initialization across users while retaining one Beta draw per arm for slate construction. We examine the effect of the chosen prior strength in the transfer sensitivity analysis. 4.2 Constraint-aware slate construction Because an item can be displayed only once to a user, a promising arm’s finite per-user inventory can be exhausted early. Given the campaign horizon, the learner therefore paces each arm’s consump- tion while constructing a slate, balancing sampled reward now against feasible items in later rounds. Thompson sampling makes this reward–information trade-off explicit: a Beta belief represents uncertainty about an arm’s binary reward rate, and an indepen- dent draw from each arm’s belief is a plausible reward rate for the next display. A high draw favors the arm, while uncertain arms occasionally draw high enough to be explored. At selection step 푖 ∈ [퐾], letQ avail 푢,푡,푖 be the items that user푢has not seen and that have not yet been selected for the current slate. Let푚 푎 =|Q 푎 |be the per-user inventory of arm푎, letℎ 푢,푎,푡,푖 be the number of earlier selections in the current slate that used arm 푎, and define 푑 푢,푎,푡,푖 = 1− |Q 푎 ∩Q avail 푢,푡,푖 | 푚 푎 , 푟 푢,푎,푡,푖 = 1−푑 푢,푎,푡,푖 . Thus,푑 푢,푎,푡,푖 is the share of user푢’s arm-푎inventory already spent and푟 푢,푎,푡,푖 is its remaining share before selection step푖. An arm is feasible exactly whenQ 푎 ∩Q avail 푢,푡,푖 ≠∅. To pace inventory use, the learner carries a smoothed arm-specific pacing error ̄ 푒 푢,푎,푡−1 from the preceding round, initialized to zero:푒 푢,푎,푡,푖 = 휌 ̄ 푒 푢,푎,푡−1 + (1− 휌)(푑 푢,푎,푡,푖 −푡/푇). Here휌 ∈ [0,1]retains prior error;clip(푥,푙,ℎ)= minmax푥,푙,ℎ limits the adjustment to bounds휂 min and휂 max ; and휙 ≥0 sets its strength. For every feasible arm, the policy samples e 휃 푢,푎 ∼ Beta(훼 푢,푎 ,훽 푢,푎 ) and computes 퐷 푢,푎,푡,푖 = 1+휙 clip(푒 푢,푎,푡,푖 ,휂 min ,휂 max ),score 푢,푎,푡,푖 = e 휃 푢,푎 /푃 푢,푎,푡,푖 , 푃 푢,푎,푡,푖 = 1+훾ℎ 푢,푎,푡,푖 +훿(1−푟 푢,푎,푡,푖 )퐷 푢,푎,푡,푖 . (6) Here,퐷 푢,푎,푡,푖 is the pacing multiplier and푃 푢,푎,푡,푖 is the total penalty that divides the Thompson draw. The coefficient훾 ≥0 discourages repeated arm use within one slate, while훿 ≥0 weights the depletion penalty. The controller compares consumption with the uniform plan푡/푇; a positive error means an arm is being used faster than plan and strengthens its depletion term. After the slate, it stores ̄ 푒 푢,푎,푡 = 휌 ̄ 푒 푢,푎,푡−1 + (1− 휌)(푑 푢,푎,푡,퐾+1 − 푡/푇). The score therefore trades immediate sampled reward against within-slate variety and later feasibility. After selecting the highest-scoring arm, the policy samples one item uniformly from that arm’s available pool and removes it before the next selection step. Uniform sampling treats items within an arm as exchangeable. An arm may appear more than once when it still has unseen items, which is necessary when 퐾> 퐴. Lebedeva and Mukherjee et al. Algorithm 1 Constraint-aware slate construction for user푢 Require:Current Beta parameters, unseen arm pools, slate size퐾 1: 푆 ←∅ andV ←Q seen 푢,푡−1 2: for 푖= 1 to 퐾 do 3: 퐹 ←푎 :Q 푎 ∩V≠∅ 4: for each 푎 ∈ 퐹 do 5:Sample e 휃 푢,푎 and compute Equation (6) 6: end for 7: 푎 ∗ ← arg max 푎∈퐹 score 푢,푎,푡,푖 8:Sample 푞 uniformly fromQ 푎 ∗ ∩V 9: Set푆 ← 푆∪푞, setV ←V\푞, and update the within- slate state 10: end for 11: return 푆 4.3 Posterior adaptation across rounds Recommendations are commonly committed as a full slate before any item in that slate has been answered, and operational workflows can prepare slates periodically from one parameter snapshot. We therefore hold the model fixed while a round is served and update it only at scheduled feedback checkpoints. All active users receive their퐾-item slates from the same snapshot, so feedback from one item cannot change another item in that slate or another user’s slate in the same round. When the round closes, the learner aggregates observed outcomes, refreshes memberships and pseudo-counts, and uses the resulting snapshot for later slates. In the deployment, a round was a daily quiz session: answers from a completed active day were batched at the daily boundary, and membership refresh began after a user had answered at least 10 items. For user푢and arm푎,푠 (푡) 푢,푎 and푓 (푡) 푢,푎 are the scalar counts of correct and incorrect outcomes in round푡. Their cumulative values are푠 (≤푡) 푢,푎 = Í 휏≤푡 푠 (휏) 푢,푎 and 푓 (≤푡) 푢,푎 = Í 휏≤푡 푓 (휏) 푢,푎 . At each such feedback checkpoint, the learner computes the membership-probability vector푝 (푡) 푢 ∈ [ 0,1] 퐶 from the registration- time mixture and cumulative evidence: ℓ (푡) 푢,푐 = log푝 (0) 푢,푐 + ∑︁ 푎 log퐵 푠 (≤푡) 푢,푎 + 훼 퐻 푐,푎 , 푓 (≤푡) 푢,푎 + 훽 퐻 푐,푎 − ∑︁ 푎 log퐵(훼 퐻 푐,푎 ,훽 퐻 푐,푎 ), 푝 (푡) 푢,푐 = softmax 푐 ℓ (푡) 푢,푐 . (8) Here,ℓ (푡) 푢 ∈ R 퐶 is the vector of log-membership scores and퐵is the Beta function. Under the arm-level conditional-exchangeability model, the Beta-function difference is the log marginal likelihood of a user’s group–arm counts after integrating a Bernoulli rate under the historical Beta prior; the binomial coefficient cancels across groups. Thus,푝 (푡) 푢 is a soft reassignment under a working arm-level model. The entries of푝 (푡) 푢 sum to one. Between checkpoints, the learner uses the most recently computed membership vector. It also maintains two nonnegative퐶 × 퐴matrices,휉 +,(푡) and휉 −,(푡) , for cumulative fractional correct and incorrect evidence assigned to each group–arm pair. The shared-update weight휆 ∈ [0,1]deter- mines the fraction of a round’s evidence placed in these group-level Algorithm 2 Cross-cohort CohortMix-TS workflow Require: Historical logsD hist , current cohort metadata 1:Fit Equation (3); cluster item and historical-user representations 2: Estimate 푝 (0) 푢,푐 and group–arm priors with Equations (4)–(5) 3: for each round 푡 do 4: for each active user푢 do 5:Serve the slate returned by Algorithm 1 6: end for 7:Collect delayed feedback after the round 8: Update memberships, shared pseudo-counts, and user be- liefs with Equations (8)–(10) 9: end for ledgers. The ledgers evolve as 휉 ±,(0) 푐,푎 = 0, 휉 ±,(푡) 푐,푎 = 휉 ±,(푡−1) 푐,푎 + 휆 ∑︁ 푢∈U 푡 푝 (푡) 푢,푐 푥 ±,(푡) 푢,푎 ,(9) where푥 +,(푡) 푢,푎 = 푠 (푡) 푢,푎 and푥 −,(푡) 푢,푎 = 푓 (푡) 푢,푎 . For each user, the learner also retains that user’s fractional contributions휁 +,(푡) 푢,푐,푎 and휁 −,(푡) 푢,푐,푎 to the two ledgers, initialized to zero and incremented by휆푝 (푡) 푢,푐 푠 (푡) 푢,푎 and 휆푝 (푡) 푢,푐 푓 (푡) 푢,푎 , respectively. This produces a leave-one-user-out group mean for the next round: ̄ 휇 (푡) 푢,푐,푎 = 훼 퐻 푐,푎 + 휉 +,(푡) 푐,푎 − 휁 +,(푡) 푢,푐,푎 훼 퐻 푐,푎 + 훽 퐻 푐,푎 + 휉 +,(푡) 푐,푎 + 휉 −,(푡) 푐,푎 − 휁 +,(푡) 푢,푐,푎 − 휁 −,(푡) 푢,푐,푎 , ̄ 휇 (푡) 푢,푎 = ∑︁ 푐 푝 (푡) 푢,푐 ̄ 휇 (푡) 푢,푐,푎 , 훼 (푡) 푢,푎 =휅 ̄ 휇 (푡) 푢,푎 +푠 (≤푡) 푢,푎 , 훽 (푡) 푢,푎 =휅(1− ̄ 휇 (푡) 푢,푎 )+ 푓 (≤푡) 푢,푎 . (10) Thus, scheduled feedback updates the center of a fixed-strength group-informed prior while each user’s cumulative outcomes enter its Beta parameters directly. Equations (8)–(10) define the scheduled soft-mixture update. 5 Experimental Setup 5.1 Baselines Within each generated environment, policies use the same catalog, slate size, user-level no-repeat rule, and delayed-feedback schedule. At each selection step, a policy chooses an arm and samples an unseen item from that arm. Full CohortMix-TS. The full policy uses a metadata-conditioned soft mixture of historical group priors, refreshes memberships from each user’s cumulative feedback, adds a fraction휆of feedback to shared group-level ledgers, and applies slate controls when they are studied. Every non-random policy also updates its own user–arm belief from that user’s feedback. It is the deployment treatment. Cold-start TS.. Cold-start Thompson sampling initializes ev- ery user–arm belief asBeta(1,1)and updates it from that user’s outcomes. Metadata-only contextual baselines. Metadata LinUCB and Metadata LinTS use the one-hot interaction푥 푢,푎 = 푒 푔 푢 ⊗푒 푎 : 20 fea- tures in transfer and 136 in the year-split benchmark. Both receive the same source estimates and metadata association as CohortMix- TS, collapsed to a metadata–arm mean at휅=10, and pool feedback Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation by metadata–arm cell after each session. They have no user, latent- group, item, or factor features. LinUCB uses훼= .5; LinTS uses Gaussian scale휈= .5. They appear only in transfer and the year- split benchmark. Transfer ablations. Warm TS (fixed mixture) retains푝 (0) 푢 and updates individual arm beliefs, but has neither reassignment nor a group ledger. Hard-cluster TS uses fixedarg max 푐 푝 (0) 푢,푐 as- signments; its semi-synthetic counterpart is Hard membership. Static source mixture ranks the initial metadata-conditioned his- torical mean and never updates. Semi-synthetic ablations. Global prior replaces metadata- conditioned frequencies with population frequencies. In the real- catalog scarcity diagnostic, No depletion sets훿=0 while retaining the remaining personalization components. Random recommen- dation samples uniformly from unseen items and is the deployed control. Slate-control variants. The finite-inventory environment fixes highly concentrated, correct beliefs and compares only slate con- struction. No controls sets all terms to zero; Diversity only sets 훾= .5; Depletion only sets훿= 휙=1; Full selector uses both; and No adaptive controller changes only휙to zero. Full selector denotes the slate-control rule, not the full cross-cohort policy. Feasible oracle. In the transfer and year-split semi-synthetic environments, an oracle knows the simulated reward means while respecting each user’s item inventory. It provides the pseudo-regret reference rather than a deployable policy. 5.2 Three environments The environments answer different parts of the problem while keeping the interaction protocol fixed: users receive 10 unseen items in each of 25 sessions, feedback arrives after the session, and policies are compared on matched generated cohorts. The first asks whether prior-cohort information helps cold-start personalization; the second asks how to use a scarce per-user inventory; and the third asks whether the transfer result persists under a real catalog and a year-to-year split. Parametric transfer environment. This is a clean test of histor- ical transfer rather than inventory management. Each observed metadata value contains two hidden user types with opposing pre- ferred arms. Thus, metadata supplies a useful starting point but cannot determine one person’s best arm. Every arm contains more items than a user can receive in 25 sessions, and slate controls are inactive. We vary the alignment휒between historical and target preference profiles:휒=1 means that the prior cohort has the same profile, whereas휒=0 means that it carries no directional informa- tion about the target profile. We compare CohortMix-TS with the metadata-only contextual baselines, Warm TS (fixed mixture), Hard- cluster TS, Cold-start TS, and Static source mixture. Appendix A gives the generator, potential-outcome protocol, and prior-strength interaction. Parametric constraint environment. This is a clean test of slate controls rather than a test of learning from history. All policies begin with accurate arm beliefs, but one high-reward arm has a limited number of unseen items for each user. Repeatedly choosing that arm within one session also lowers the reward of later selections in that session. We vary its inventory and compare No controls, Diversity only, Depletion only, Full selector, and No adaptive controller. The resulting trade-off is between taking the scarce arm immediately, spreading choices within a slate, and retaining useful items for later sessions. Year-split data-calibrated semi-synthetic environment. The third environment joins the previous two questions to observed cohort data without replaying any participant’s actual recommendation history. A disjoint 2024 cohort supplies the historical factorization, groups, metadata association, and arm priors. A 2025 cohort cali- brates hidden target preferences for freshly generated users. Both cohorts use the same 1,347-item catalog, partitioned into eight arms of sizes 128, 239, 137, 84, 298, 98, 243, and 120. The policy sees only 2024-derived structure, a new user’s faculty, and simulated feed- back; the evaluator alone uses the 2025 records. Each main cohort has 288 users balanced across three source-only preference types, so a majority type cannot mask performance for the others. The main benchmark turns slate penalties off to focus on transfer. Fig- ure 1(c) separately uses the real 98-item arm to study depletion. It compares CohortMix-TS, the metadata-only contextual baselines, the listed ablations, and Random. Appendix A gives the calibration, potential-outcome protocol, and sensitivities. 5.3 Randomized in-the-wild deployment To test whether personalized allocation can match questions to participants, the completed in-the-wild deployment used a 25-day Campus Games quiz. Correctness was the binary reward: questions differ in difficulty and subject familiarity, so a question that suits one participant or group may be a poor match for another. The daily quiz delivered퐾=10 multiple-choice items from a catalog of푄=1,347 items, with a 20-second limit per item. A prior cohort of 859 users over the same catalog supplied the transferred struc- ture. Historical model selection yielded five item arms and 23 user groups. The five arm sizes were 273, 190, 516, 76, and 292 items. This randomized study evaluates the complete deployed policy against random allocation, rather than an isolated method component. The previous cohort’s questions were sampled independently of recommendations, so no exposure correction is required. For adaptive historical logs with known display propensities, the fac- torization and group–arm rates can instead use inverse-propensity weighting [27], avoiding a policy-induced exposure pattern being treated as preference. Assignment occurred once at registration. The 356 treatment users received CohortMix-TS slates intended to increase their estimated probability of answering correctly, while 357 control users received random unseen items; quiz length, time limit, scor- ing, and catalog were identical between arms. Recommendations were planned on a rolling three-day horizon, with assigned items reserved until used or returned to the eligible pool. The deployed settings were훾=0.5,훿=1.0,휙=1.0,휂 min = −0.3,휂 max =0.7, 휌=0.3,휆=0.3, and휅=10. Membership updates began only after a user had supplied at least 10 answers. The deployment did not vary 휅. Lebedeva and Mukherjee et al. 17142125 Session 0.500 0.525 0.550 0.575 0.600 0.625 0.650 0.675 0.700 Expected reward / item (a) Transfer (χ=0.75) CohortMix-TS Cold-start TS Metadata LinUCB Metadata LinTS 17142125 Session 0.0 0.2 0.4 0.6 0.8 1.0 Preferred inventory spent (b) Finite-inventory pacing No controls Diversity only Depletion only Full selector 5.505.756.006.256.50 Expected 10-item slate reward 0 20 40 60 80 100 Optimal-arm availability (%) (c) Real-catalog scarcity Full Cold TS No depletion Random Figure 1: One diagnostic for each generated environment. (a) Transfer expected reward at alignment휒=0.75 for CohortMix- TS, Cold-start TS, Metadata LinUCB, and Metadata LinTS; higher휒means closer agreement between historical and target preferences. (b) Preferred-arm inventory use when that arm contains 20% of the푇퐾item opportunities for No controls, Diversity only, Depletion only, and Full selector. (c) Real-catalog scarcity stress in the 2024–2025 year-split environment. Each line joins five consecutive five-session blocks. Both coordinates average over users whose generated optimal arm is the actual 98-item arm: expected 10-item slate reward (horizontal) and the percentage with an unseen item remaining in that arm before the session slate is selected (vertical). Open markers denote sessions 1–5 and filled terminal markers denote sessions 21–25. Shaded bands in (a)–(b) are percentile bootstrap 95% intervals across 160 cohorts; (c) reports cohort-average trajectories across 96 cohorts. 6 Results and Empirical Analysis 6.1 Cross-cohort transfer depends on prior alignment Figure 1(a) and Table 1 answer RQ1 and RQ2. When historical and target preferences align, CohortMix-TS has the strongest early and campaign reward and the lowest high-regret tail. The transferred mixture gives it useful structure before a user supplies feedback; Cold-start TS begins with diffuse arm beliefs and must first explore. A fixed mixture or hard assignment transfers some structure, but cannot use feedback to resolve initial latent-type uncertainty. Figure 1(a) exposes a failure mode of standard contex- tual models based only on observed metadata: Metadata LinUCB and Metadata LinTS pool faculty–arm outcomes, so their curves level off when one faculty contains users with opposing hidden preferences. Figure 3 shows the corresponding source-data pattern, with group–arm correctness varying within metadata categories. The appendix sensitivity check shows that the mean early-reward benefit grows with historical–target alignment; under severe mis- match, its advantage for high-regret users disappears. Thus, RQ1’s benefit depends on reliable cross-cohort structure, and RQ2’s soft memberships protect against within-metadata heterogeneity. 6.2 Slate controls trade early exploitation for later feasibility Figure 1(b) answers RQ3 in the finite-inventory environment. With- out depletion control, a policy spends the scarce preferred arm early; variants with the depletion term instead pace it through the Table 1: Parametric transfer results at alignment휒=0.75, averaged over 160 generated cohorts. Rewards are expected per displayed item; Minority is campaign reward for the lower-probability hidden subtype. P90 pseudo-regret is the 90th percentile of per-user pseudo-regret at the final session. Bold marks the best result and underline the second-best result in each column. PolicyEarly↑ Campaign↑ Minority↑ 푃 90 regret↓ Full CohortMix-TS0.6220.6690.65319.18 Warm TS (fixed mixture)0.5790.642 0.62824.39 Hard-cluster TS0.5920.6090.50662.77 Cold-start TS0.5760.6390.63824.73 Static source mixture0.5850.5850.40087.47 Metadata LinUCB0.5410.5960.50565.23 Metadata LinTS0.5920.6070.50266.82 final week. The full slate controller accepts less immediate concen- tration than Diversity only, then has stronger later reward once the scarce arm would otherwise be unavailable. The sensitivity grid in Appendix A shows that this trade-off weakens when the preferred inventory is less scarce. Depletion is therefore useful for the stated short-inventory setting, rather than a universally reward-maximizing penalty. 6.3 Year-split data-calibrated semi-synthetic benchmark Table 2 answers RQ1 and RQ2 in the year-split environment. The proposed policy improves on Cold-start TS because the historical mixture starts each user with informative arm preferences, while Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation Table 2: Year-split data-calibrated semi-synthetic results over 96 generated cohorts. Source-only 2024 structure initializes each policy; 2025 records calibrate hidden target preferences. Reward entries are expected per displayed item; pseudo- regret is per user over 250 displayed items. The main bench- mark turns slate penalties off to isolate transfer; Figure 1(c) reports the separate real-catalog scarcity stress. Bold marks the best result and underline the second-best result in each column. PolicyEarly↑ Campaign↑ Late↑ Pseudo-regret↓ CohortMix-TS0.5970.627 0.63237.71 Cold-start TS0.584 0.6180.62839.94 Metadata LinUCB0.5800.5900.60146.84 Metadata LinTS0.5800.5940.59845.99 Hard membership0.5830.6150.62540.59 Global prior0.584 0.6150.62440.67 Random0.5650.5650.56553.12 cold start first explores from a uniform Beta prior. The metadata- only contextual policies improve their faculty-level estimates but cannot separate individual preferences within a faculty, which lim- its their later gains. Hard membership and a global prior remove, respectively, adaptive uncertainty about type and the metadata sig- nal; both patterns support the value of a soft, metadata-conditioned warm start. Figure 1(c) answers RQ3 with the actual 98-item arm. The No depletion trajectory gains reward early by repeatedly selecting the preferred arm, then exhausts that arm for many affected users and loses late slate quality. The tempered controller preserves avail- ability into the final block. Random preserves the arm by rarely selecting it, but supplies lower-reward slates throughout. The figure therefore shows the intended early–late pacing trade-off, not a claim that depletion control improves every campaign-wide outcome. 6.4 Randomized in-the-wild deployment Real-world participation constraints substantially reduced the size of the correctness analysis. Of the 713 registered users, 385 inter- acted with at least one quiz item. The outcome analysis excluded activity before the official start and two dates affected by technical failures, then retained participants with at least 30 answered items. This answer-count filter produced an analysis set of 211 users: 109 treatment and 102 control. Because the filter occurs after assign- ment, the resulting comparison is not an intention-to-treat estimate for all registrants. The four largest metadata strata contained 58 treatment and 61 control users. ComputingΔ 푢 additionally requires observed out- comes in both the early and late windows; only 23 treatment and 29 control users met this requirement. MeanΔ 푢 was 0.0455 for treat- ment and−0.0168 for control, a difference of 0.0623 (bootstrap 95% interval[0.005,0.119]). A two-sided Mann–Whitney rank-sum test returned푝=0.043. Figure 2 shows this distribution and descriptive daily context. Broader responder analyses weakened the evidence: the all-faculty answer-qualified set (푛=93) had a 0.034 difference (푝=0.146), and all complete-window responders (푛=96) had a 0.033 difference (푝=0.153); answer-weighting the restricted out- come gave 0.057 (푝=0.053). The small complete-window subset and post-assignment filtering limit precision and interpretation; the result is sensitive to the responder definition and does not isolate individual method components. We also compared two randomized-arm participation indica- tors using all 713 registrants. At least one quiz item was started by 53.1% of treatment users (189/356) and 54.9% of control users (196/357; two-sided Fisher test푝=0.652). At least 30 items were answered by 30.6% of treatment users (109/356) and 28.6% of control users (102/357;푝=0.566). These are intention-to-treat operational outcomes; neither comparison provided evidence of a treatment difference. Correctness is undefined for users who did not answer, so the study data do not identify an all-randomized correctness effect without a missingness model. We have no pre-assignment quiz outcome. We do not use the daily count of correct answers as a treatment- effect outcome. Such counts combine answer correctness with changing participation and therefore cannot distinguish improved matching from greater exposure. 7 Discussion and Limitations The field study randomized the complete policy against random recommendation, but its correctness result is conditional on post- assignment responses. The all-registrant participation comparisons are intention-to-treat operational outcomes, not correctness or re- tention effects. Nonresponders have no correctness value to in- clude, and we did not fit a pre-specified missing-data or longitu- dinal model; we also have no pre-assignment quiz outcome. The restricted result is therefore compatible with improved item match- ing over time, but not with a population-wide causal claim about learning, interest, satisfaction, or retention. The휅sweeps show that a fixed concentration must be cali- brated to transfer alignment: stronger priors help under aligned transfer but can harm under severe source–target mismatch. The finite-inventory isolation deliberately removes prior uncertainty, and the deployment uses one fixed value. Equation (5) is not a second-moment match, and we did not fit휅by empirical Bayes, as in empirical-Bayes meta-prior learning [36]. The uniform푡/푇plan is the only pacing target evaluated: we did not test front-loaded, concave, value-aware, attendance-aware, or annealed schedules. The real-catalog stress shows a late-availability benefit from deple- tion, but it does not establish a general choice of controller weight. Likewise, no difficulty-bucket or item-level posterior was compared with uniform within-arm sampling. The comparison set is mechanism-specific rather than a broad recommender benchmark. It includes Metadata LinUCB and Meta- data LinTS in the transfer and year-split semi-synthetic benchmarks, initialized from the same metadata–arm source estimates and eval- uated with the same feasibility and slate constraints. These policies pool feedback within metadata–arm cells and therefore do not represent a user-specific latent state. We do not compare against contextual models with common user–item feature representations or against hierarchical Bayesian TS; Hard-cluster TS is an ablation, not a substitute. The arm partition is fixed throughout the study; handling catalog changes requires a principled transfer of priors across revised arms. Future work should compare richer models under common historical inputs and pre-specify longitudinal field outcomes. Lebedeva and Mukherjee et al. Random n= 29 CohortMix n= 23 −0.4 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 Δ daily correctness Δ = + 0.062 p= 0.043 (a) 17142124 Expected day 0.45 0.50 0.55 0.60 0.65 0.70 Mean correct / answered All registrants; descriptive Day 3 excluded (b) Control CohortMix-TS 17142124 0 25 50 75 100 Daily active users All registrants; descriptive Day 3 excluded (c) 05101520 Active quiz days 0.0 0.2 0.4 0.6 0.8 1.0 Share ≥ x All registrants (n= 713) (d) Figure 2: In-the-wild deployment outcomes. (a) Participant-level early–late daily-correctness changeΔ 푢 in the restricted, complete-window analysis (푛=52; treatment=23, control=29). Points are users; diamonds and bars are group means and bootstrap 95% intervals. The treatment–control difference is 0.062 (95% interval[0.005,0.119];푝=0.043). (b) Daily mean correct/answered rate, averaged within each responding user-day; (c) daily active users; and (d) active-day survival. Panels (b)–(d) are descriptive all-registrant traces (푛=713) after removing technical-failure dates and have changing daily denominators. Blue denotes random control and orange denotes CohortMix-TS. 8 Conclusion We developed CohortMix-TS for short-horizon, finite-inventory cohort recommendation, transferred historical structure into soft priors, and deployed the complete policy in a randomized campaign. The controlled environments and 2024–2025 data-calibrated bench- mark identify settings in which historical warm starts improve early recommendation and a scarce arm benefits from pacing. In our in-the-wild deployment, we observed a larger early–late cor- rectness change only in the reported restricted subset. Future work will test richer representations, adaptive priors and pacing, and longer-running services with pre-specified longitudinal outcomes. A Detailed Environments and Ablations Common protocol. All policies construct every user’s slate from the same pre-session state and receive feedback only after that session is complete. Within each generated cohort, policies share the same user metadata, item pools, and Bernoulli potential outcomes. We summarize uncertainty over independently generated cohorts; “early” denotes sessions 1–5 and “late” denotes sessions 19–25. Ablations. Warm TS (fixed mixture) keeps the registration-time metadata mixture and updates only a user’s local arm counts. Hard membership replaces that mixture with its most likely historical group, while Static source mixture serves the highest-ranked source arm without feedback updates. Global prior replacesPr(푐 | 푔 푢 ) with population group frequencies. Metadata LinUCB and Metadata LinTS pool one-hot metadata–arm estimates and do not maintain a user-specific latent state. In the scarcity diagnostic, No depletion changes only훿from.25 to zero; Random samples feasible unseen items uniformly. Transfer environment and membership ablations. The transfer environment has four metadata values, eight hidden groups, and five arms. Each metadata value places prior mass.65 and.35 on two groups with opposing preferred arms, so metadata alone cannot identify a user’s best arm. Target group profiles interpolate between the historical profile and an orthogonal profile using휒:휒=1 denotes aligned source and target profiles, whereas휒=0 removes this alignment. We draw 400 historical binary outcomes per group– arm cell, generate 480 target users, and give each user 300 items per arm; the catalog is deliberately generous, so slate controls are inactive. We use푇=25 sessions and퐾=10 items per session. The comparison includes fixed-mixture, hard-membership, static- source, cold-start, and metadata-only contextual alternatives. The 휅 × 휒grid in Table 3 shows the early-reward contrast between CohortMix-TS and Cold-start TS. Finite-inventory ablations. The finite-inventory environment has 256 users, one preferred arm with mean reward.70, and four fall- back arms with mean reward.50. The preferred arm has capacity 푚 ∈ 25,50,75,100, while each fallback arm has capacity 125. Thus 푚/(푇퐾) ∈ .10, .20, .30, .40is the fraction of a user’s campaign slots that the preferred arm could fill. To make slate diversity rele- vant, every earlier use of an arm within the same slate lowers the reward logit by.25. We vary diversity훾 ∈ 0, .25, .5,1, depletion weight훿 ∈ 0, .5,1,2, and controller strength휙 ∈ 0, .5,1,2. Ta- ble 3 reports the component ablation at푚/(푇퐾)= .20. The훿grid shows that stronger depletion control is needed when the preferred arm supplies only.10 of the campaign: early exhaustion falls from 1.00 at훿 ≤1 to .27 at훿=2, while at.20 it is zero for훿 ≥ .5. At looser capacities, the diversity-only policy can have higher cam- paign reward, which is why the controller is presented as a pacing trade-off rather than a universal improvement. Sensitivity patterns. At휒=0, the warm start still improves aver- age early reward, but it has a worse 90th-percentile user pseudo- regret tail; at휅=10, the Full-minus-Cold P90 difference is+12.27. At휒=1, the corresponding difference is−19.62. Thus, alignment chiefly determines whether historical structure also protects users with high regret. In the high-alignment year-split environment, a separate 32-cohort prior sweep over휅 ∈ 2,5,10,20improved campaign reward as prior strength increased. A휆 ∈ 0, .1, .3, .5 Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation Table 3: Ablation and sensitivity details. (Left) finite-inventory component ablation at푚/(푇퐾)= .20; rewards are expected per displayed item and Early exhaust is the fraction of users whose preferred arm is exhausted before the final week. “No adaptive controller” retains훾= .5 and훿=1 but sets휙=0. (Right) the early-reward difference between CohortMix-TS and Cold-start TS in the transfer environment; positive values favor CohortMix-TS. Bold marks the best and underline the second-best result within each left-hand reward column. PolicyEarly↑Campaign↑Late↑Early exhaust↓ No controls.4386.4333.43201.00 Diversity only.5244.5002 .45061.00 Depletion only.4756.4790.4834.00 No adaptive controller.5177.5081.4983.00 Full selector.5168.5083 .4996.00 휒= 0 휒= .5 휒= 1 휅= 1.033.029.034 휅= 5.032.034.073 휅= 10.029.035.083 휅= 20.025.035.090 휅= 40.015.032.098 check was nearly flat, so it does not identify.3 as a uniquely pre- ferred sharing weight. These are targeted sensitivity checks, not a search over all configurations. Year-split data-calibrated semi-synthetic environment. The dis- joint 2024 source contains 49,675 answers from 561 users, and the 2025 target calibration contains 23,838 answers from 347 different users. We first shrink source user–arm response profiles toward 2024 arm means and cluster those eight-dimensional profiles into three source-only groups. These groups determine the historical Beta priors and b Pr(푐 | faculty). The 2025 records are used only by the generator to obtain hidden target profiles, user residuals, and target group–arm means; policies receive neither these records nor the target labels. Source-calibrated question effects and sampled target user residuals create fresh potential outcomes, shared across policies. The source–target group–arm correlation is .985. Each co- hort has 288 users balanced across hidden types and draws from the actual 1,347-question, eight-arm catalog without replacement for 25 sessions of 10 items. The transfer benchmark fixes휅=10 and휆= .3 with slate penalties inactive to isolate transfer. The scarcity diag- nostic uses the real 98-item arm and compares(훾,훿,휙)=(0, .25,1) with훿=0; it evaluates availability only among users for whom that arm is hidden-optimal. Field deployment details. The deployment was a 25-day daily quiz within a Campus Games quiz. On an active day, a participant received 10 previously unseen multiple-choice items from a 1,347- item catalog, with a 20-second limit; incorrect or expired answers received reward zero. The preceding 859-user cohort used the same catalog. Historical preprocessing selected five item arms and 23 user groups from candidate counts 2–7 and 2–39. Their (size, historical correctness) pairs were(273, .555),(190, .560),(516, .500),(76, .524), and(292, .570). Faculty was the deployment-time metadata field; 17 categories occurred in the current cohort. Randomization occurred once at registration. Control users re- ceived uniformly sampled unseen items; treatment users received CohortMix-TS slates. Recommendations were planned over a rolling three-day horizon: assigned items were reserved, excluded from later candidate pools, and released if unused. The policy used 훾= .5,훿= 휙=1,휂 min =−.3,휂 max = .7,휌= .3,휅=10, and휆= .3; membership updates began after 10 answers. The de-identified records include event type, time, correctness, and response time; our analysis removed pre-event activity and two unreliable dates. Enrollment records show control/treatment counts of 357/356 registered, 196/189 with any activity, and 102/109 with at least 30 answers. The analysis retained 61/58 users in the four largest metadata strata and 29/23 with both windows. Equa- tion (2) uses expected days 1–7 and 18–24. All-faculty thresholded responders gave a 0.034 difference (푝=0.146); all complete-window responders gave 0.033 (푝=0.153). Complete-window response was 13.7% in control and 13.2% in treatment (푝=0.913), but it does not identify missing correctness outcomes. Arm 1Arm 2Arm 3Arm 4Arm 5 Historical item arm A1 (5.2k) A2 (1.3k) A3 (0.8k) B1 (3.3k) B2 (1.7k) B3 (1.2k) C1 (3.8k) C2 (1.5k) C3 (0.3k) Inferred source group (answers) +10+10+11-4+12 -4-0-7+19-11 -15-21-12-16-9 +9+8+3+1+1 -5-3-2-8+3 -8-9-2+10-5 -5-4-1-6+6 +3+3-2-0-9 +9+6+15 n< 30 +7 −20 0 20 Correctness residual (p) Figure 3: Source-only group–arm variation conditional on metadata. Within the three highest-support metadata cat- egories that split into multiple inferred groups, each cell is a group’s leave-group-out correctness residual (percent- age points) relative to other historical users with the same metadata category and item arm. A–C denote anonymized metadata categories; rows are ordered by source answer sup- port. The gray cell has fewer than 30 observations. Historical variation conditional on metadata. Figure 3 gives a descriptive source-only view of group–arm variation after condi- tioning on the observed metadata category. It is not an independent validation of latent grouping, because the historical grouping pro- cedure also used metadata; the controlled transfer environment isolates that question. Lebedeva and Mukherjee et al. Ethical Considerations The deployment used voluntary participation and a review process appropriate to research involving human participants. The analysis uses de-identified event logs, minimizes retained metadata, and should restrict access to personnel who need the data for the study. Any shared study data should remove identifiers, dates that permit linkage, and small strata that could enable re-identification. Optimizing correctness can favor easier items for some users and can create unequal experiences if historical metadata encode unequal access, preparation, or participation. The method uses metadata as an uncertain prior rather than a fixed assignment, but this does not remove that risk. Before deployment, operators should define privacy-preserving, adequately sized monitoring groups and report recommendation exposure, correctness, and participation by group alongside aggregate outcomes. We do not make subgroup- effect claims from the deployment: its restricted complete-window analysis contains 52 users across four metadata strata. The system should not use quiz outcomes for grading, discipline, or high-stakes decisions. Operators should provide an explanation of personaliza- tion, an opt-out path where feasible, and monitoring for unexpected disparities. Acknowledgement This work was funded by the Ministerium für Wissenschaft und Gesundheit des Landes Rheinland- Pfalz through the project AI4Nof1 under grant 724-0019#2024/0001-150115404. References [1]Shipra Agrawal and Navin Goyal. 2012. Further Optimal Regret Bounds for Thompson Sampling, In International Conference on Artificial Intelligence and Statistics. International Conference on Artificial Intelligence and Statistics. [2]Shipra Agrawal and Navin Goyal. 2012. Thompson Sampling for Contextual Bandits with Linear Payoffs, In International Conference on Machine Learning. International Conference on Machine Learning 28, 127–135. [3] M. Aharon, A. Kagian, Y. Kaplan, R. Nissim, and O. Somekh. 2015. Serving Ads to “Yahoo Answers” Occasional Visitors, In The Web Conference. The Web Conference, 1257–1262. doi:10.1145/2740908.2741997 [4] Andrea Barraza-Urbina and D. Głowacka. 2020. Introduction to Bandits in Recommender Systems, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems, 748–750. doi:10.1145/3383313.3411547 [5]M. Blake. 2017. Two Decades of Recommender Systems at Amazon.Com. (2017). [6]Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Jesús Bernal. 2012. A Collaborative Filtering Approach to Mitigate the New User Cold Start Problem. Knowledge-Based Systems 26 (2012), 225–238. doi:10.1016/j.knosys.2011.07.021 [7] Chao Chen, Dongsheng Li, Qin Lv, Jun Yan, S. M. Chu, and Li Shang. 2016. MPMA: Mixture Probabilistic Matrix Approximation for Collaborative Filtering, In International Joint Conference on Artificial Intelligence. International Joint Conference on Artificial Intelligence. [8]Chao Chen, Dongsheng Li, Yingying Zhao, Qin Lv, and Li Shang. 2015. WE- MAREC: Accurate and Scalable Recommendation through Weighted and En- semble Matrix Approximation, In Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. doi:10.1145/2766462.2767718 [9]Youngduck Choi, Youngnam Lee, Junghyun Cho, Jineon Baek, Byungsoo Kim, Yeongmin Cha, Dongmin Shin, Chan Bae, and Jaewe Heo. 2020. Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing, In ACM Conference on Learning @ Scale. ACM Conference on Learning @ Scale, 341–344. doi:10.1145/3386527.3405945 [10]James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, T. Vleet, U. Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, et al. 2010. The YouTube Video Recommendation System, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems, 293–296. doi:10.1145/1864708.1864770 [11]Simen Eide, David S. Leslie, and Arnoldo Frigessi. 2021. Dynamic Slate Recom- mendation with Gated Recurrent Units and Thompson Sampling. Data Mining and Knowledge Discovery 36, 5 (2021), 1756–1786. doi:10.1007/s10618-022-00849- w [12]Florent Garcin, Boi Faltings, Olivier Donatsch, Ayar Alazzawi, Christophe Bruttin, and Amr Huber. 2014. Offline and Online Evaluation of News Recommender Systems at Swissinfo.Ch, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems, 169–176. doi:10.1145/2645710.2645745 [13]Carlos A. Gomez-Uribe and Neil Hunt. 2016. The Netflix Recommender System: Algorithms, Business Value, and Innovation. (2016). [14] F. Harper, J. Konstan, and J. A. 2016. The MovieLens Datasets: History and Context. TIIS (2016). doi:10.1145/2827872 [15] Xu HE, Bo An, Yanghua Li, Haikai Chen, Qingyu Guo, Xin Li, and Zhirong Wang. 2020. Contextual User Browsing Bandits for Large-Scale Online Mobile Recom- mendation, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems, 63–72. doi:10.1145/3383313.3412234 [16] Joey Hong, Branislav Kveton, Manzil Zaheer, and Mohammad Ghavamzadeh. 2021. Hierarchical Bayesian Bandits, In International Conference on Artificial Intelligence and Statistics. International Conference on Artificial Intelligence and Statistics 151, 7724–7741. [17]Yuqing Hu. 2024. Matrix Factorization Model in Collaborative Filtering Algo- rithms Based on Feedback Datasets. In Applied Economics and Policy Studies. Springer Nature Singapore, 1405–1412. doi:10.1007/978-981-97-0523-8_128 [18]Shion Ishikawa, Young joo Chung, and Yu Hirate. 2022. Dynamic Collaborative Filtering Thompson Sampling for Cross-domain Advertisements Recommenda- tion, In arXiv.org. arXiv.org. arXiv:2208.11926 doi:10.48550/arXiv.2208.11926 [19]Xia Jiang and Rong J. B. Zhu. 2025. Empirical Bayesian Multi-Bandit Learning. arXiv.org (2025). arXiv:2510.26284 doi:10.48550/arXiv.2510.26284 [20]Cem Kalkanli and Ayfer Özgür. 2021. Batched Thompson Sampling. In Advances in Neural Information Processing Systems, Vol. 34. 29984–29994. [21]Mozhgan Karimi, Dietmar Jannach, and Michael Jugovac. 2018. News Recom- mender Systems – Survey and Roads Ahead. Information Processing & Manage- ment 54, 6 (2018), 1203–1227. doi:10.1016/j.ipm.2018.04.008 [22] Robert Kleinberg, Alexandru Niculescu-Mizil, and Yogeshwer Sharma. 2010. Regret bounds for sleeping experts and bandits. Machine-mediated learning 80, 2-3 (2010), 245–272. doi:10.1007/s10994-010-5178-7 [23]Y. Koren and Robert M. Bell. 2011. Advances in Collaborative Filtering. In Recommender Systems Handbook. Springer. doi:10.1007/978-0-387-85820-3_5 [24] B. Kveton, Mikhail Konobeev, M. Zaheer, Chih-Wei Hsu, Martin Mladenov, Craig Boutilier, and Csaba Szepesvari. 2021. Meta-Thompson Sampling, In International Conference on Machine Learning. International Conference on Machine Learning 139, 404–420. doi:10.1017/9781108571401.044 [25] Nir Levine, Koby Crammer, and Shie Mannor. 2017. Rotting bandits. Neural Information Processing Systems 30 (2017). [26] Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A Contextual- Bandit Approach to Personalized News Article Recommendation, In The Web Conference. The Web Conference, 661–670. doi:10.1145/1772690.1772758 [27] Lihong Li, Wei Chu, John Langford, and Xuanhui Wang. 2010. Unbiased Offline Evaluation of Contextual-Bandit-Based News Article Recommendation Algo- rithms, In Web Search and Data Mining. Web Search and Data Mining, 297–306. doi:10.1145/1935826.1935878 [28]Shuai Li, Alexandros Karatzoglou, and Claudio Gentile. 2016. Collaborative Filtering Bandits. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 539–548. doi:10. 1145/2911451.2911548 [29]Zhi Li, Daichi Amagata, Yihong Zhang, Takuya Maekawa, Takahiro Hara, Kei Yonekawa, and Mori Kurokawa. 2022. HML4Rec: Hierarchical Meta-Learning for Cold-Start Recommendation in Flash Sale E-Commerce. Knowledge-Based Systems 255 (2022), 109674. doi:10.1016/j.knosys.2022.109674 [30] Yong Liu, Yingtai Xiao, Qiong Wu, Chunyan Miao, Juyong Zhang, Binqiang Zhao, and Haihong Tang. 2020. Diversified Interactive Recommendation with Implicit Feedback, In AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence 34, 04, 4932–4939. doi:10.1609/aaai.v34i04.5931 [31]Yinping Ma, Rongbin Ouyang, Xinzheng Long, Zhitong Gao, Tianping Lai, and Chun Fan. 2023. DORIS: Personalized Course Recommendation System Based on Deep Learning. PLOS ONE 18, 6 (2023), e0284687. doi:10.1371/journal.pone. 0284687 [32]Kanak Mahadik, Qingyun Wu, Shuai Li, and Amit Sabne. 2020. Fast Distributed Bandits for Online Recommendation Systems, In International Conference on Supercomputing. International Conference on Supercomputing, 1–13. doi:10.1145/ 3392717.3392748 [33]M. H. Maqbool, Umar Farooq, Adib Mosharrof, A. B. Siddique, and Hassan Foroosh. 2023.MobileRec: A Large Scale Dataset for Mobile Apps Rec- ommendation, In Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 3007–3016. doi:10.1145/3539618.3591906 [34]James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, and Rishabh Mehrotra. 2018. Explore, Exploit, and Explain: Personalizing Explainable Recommendations with Bandits, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems. doi:10.1145/3240323.3240354 Challenges and Solutions for Bandits in the Wild: Warm-Started Mixture Bandits for Cross-Cohort Slate Recommendation [35]H. Brendan McMahan, Gary Holt, D. Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches, In Knowledge Discovery and Data Mining (Chicago, Illinois, USA). Knowledge Discovery and Data Mining, 1222–1230. doi:10.1145/2487575.2488200 [36]Sareh Nabi, Houssam Nassif, Joseph Hong, Hamed Mamani, and Guido Imbens. 2020. Bayesian Meta-Prior Learning Using Empirical Bayes. Management Sciences 68, 3 (2020), 1737–1755. doi:10.1287/mnsc.2021.4136 [37] Bastian Oetomo, R. Malinga Perera, Renata Borovica-Gajic, and Benjamin I. P. Rubinstein. 2021. Cutting to the Chase with Warm-Start Contextual Bandits. Industrial Conference on Data Mining 65, 9 (2021), 3533–3565. doi:10.1007/s10115- 023-01861-2 [38] Yi Ren, Xiao Han, Xu Zhao, Shenzheng Zhang, and Yan Zhang. 2023. Slate-Aware Ranking for Recommendation, In Web Search and Data Mining. Web Search and Data Mining, 499–507. doi:10.1145/3539597.3570380 [39]F. Ricci, L. Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. 35 pages. doi:10.1007/978-0-387-85820-3_1 [40] Daniel Russo and Benjamin Van Roy. 2014. An Information-Theoretic Analysis of Thompson Sampling. Journal of Machine Learning Research (2014). [41]Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation, In International Conference on Machine Learning. International Conference on Machine Learning, 1670–1679. [42] Julien Seznec, Andrea Locatelli, Alexandra Carpentier, Alessandro Lazaric, and Michal Valko. 2018. Rotting bandits are no harder than stochastic ones, In International Conference on Artificial Intelligence and Statistics. International Conference on Artificial Intelligence and Statistics, 2564–2572. [43]Nícollas Silva, Heitor Werneck, Thiago Silva, Adriano C. M. Pereira, and Leonardo Rocha. 2022. Multi-Armed Bandits in Recommendation Systems: A Survey of the State-of-the-Art and Future Directions. Expert Systems with Applications 197 (2022), 116669. doi:10.1016/j.eswa.2022.116669 [44] Jennifer Sween, S. Wallington, V. Sheppard, T. Taylor, A. Llanos, and L. Adams- Campbell. 2014. The Role of Exergaming in Improving Physical Activity: A Review. Journal of Physical Activity and Health 11, 4 (2014), 864–870. doi:10. 1123/jpah.2011-0425 [45]Huazheng Wang, Qingyun Wu, and Hongning Wang. 2017. Factorization Bandits for Interactive Recommendation, In AAAI Conference on Artificial Intelligence. AAAI Conference on Artificial Intelligence 31, 1, 2695–2702. doi:10.1609/aaai.v31i1. 10936 [46]Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, et al.2020. MIND: A Large- scale Dataset for News Recommendation, In Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Annual Meeting of the Association for Computational Lin- guistics, 3597–3606. doi:10.18653/v1/2020.acl-main.331 [47]Liu Yang, Bo Liu, Leyu Lin, Feng Xia, Kai Chen, and Qiang Yang. 2020. Exploring Clustering of Bandits for Online Recommendation System, In ACM Conference on Recommender Systems. ACM Conference on Recommender Systems, 120–129. doi:10.1145/3383313.3412250 [48]Eva Zangerle and Christine Bauer. 2022. Evaluating Recommender Systems: Survey and Framework. Comput. Surveys 55, 8 (2022), 1–38. doi:10.1145/3556536