Paper deep dive

Sustainable Multi-Agent Crowdsourcing via Physics-Informed Bandits

Chayan Banerjee

Year: 2026Venue: arXiv preprintArea: cs.MAType: PreprintEmbeddings: 73

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 7/20/2026, 10:58:08 AM

Summary

The paper introduces FORGE, a physics-grounded multi-agent simulator for crowdsourcing that models contractors as rational agents with strategic fatigue-based availability decisions, addressing the Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma. It proposes a Neural-Linear UCB allocator with a Physics-Informed Covariance Prior to achieve high reward with low workforce utilization (7.6%) while mitigating burnout and cold-start regret.

Entities (9)

FORGE · simulator · 98%Neural-Linear UCB Allocator · algorithm · 97%Physics-Informed Covariance Prior · method · 96%Two-Tower embedding network · model-architecture · 95%Restless Multi-Armed Bandit (RMAB) · problem-formulation · 94%Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma · concept · 93%COALESCE · prior-framework · 90%LinUCB · baseline-algorithm · 88%TOPSIS · baseline-algorithm · 88%

Relation Signals (6)

Neural-Linear UCB Allocator → utilizes → Physics-Informed Covariance Prior

confidence 96% · we propose a Neural-Linear UCB allocator that fuses a Two-Tower embedding network with a Physics-Informed Covariance Prior

FORGE → implements → Restless Multi-Armed Bandit (RMAB)

confidence 95% · FORGE... converting the standard passive Restless Multi-Armed Bandit (RMAB) into a genuine Stackelberg game.

Neural-Linear UCB Allocator → uses → Two-Tower embedding network

confidence 95% · Built on a Two-Tower neural architecture, the allocator maps high-dimensional task and contractor representations into a shared embedding space

Physics-Informed Covariance Prior → addresses → Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma

confidence 92% · The prior simultaneously warm-starts skill-cluster geometry... providing a geometry-aware belief state... that measurably reduces cold-start regret.

FORGE → extends → COALESCE

confidence 90% · extending the task-contractor embedding structure and surge pricing mechanics of COALESCE [4]

LinUCB → comparedwith → Neural-Linear UCB Allocator

confidence 85% · we evaluate LinUCB directly as a primary baseline... observe that while it achieves near-zero burnout it does so only by engaging 100% of the contractor pool.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Crowdsourcing platforms face a four-way tension between allocation quality, workforce sustainability, operational feasibility, and strategic contractor behaviour--a dilemma we formalise as the Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma. Existing methods resolve at most two of these tensions simultaneously: greedy heuristics and multi-criteria decision making (MCDM) methods achieve Day-1 quality but cause catastrophic burnout, while bandit algorithms eliminate burnout only through operationally infeasible 100% workforce this http URL address this, we introduce FORGE, a physics-grounded $K+1$ multi-agent simulator in which each contractor is a rational agent that declares its own load-acceptance threshold based on its fatigue state, converting the standard passive Restless Multi-Armed Bandit (RMAB) into a genuine Stackelberg game. Operating within FORGE, we propose a Neural-Linear UCB allocator that fuses a Two-Tower embedding network with a Physics-Informed Covariance Prior derived from offline simulator interactions. The prior simultaneously warm-starts skill-cluster geometry and UCB exploration landscape, providing a geometry-aware belief state from episode 1 that measurably reduces cold-start this http URL $T = 200$ cold-start episodes, the proposed method achieves the highest reward of all non-oracle methods ($\text{LRew} = 0.555 \pm 0.041$) at only 7.6% workforce utilisation--a combination no conventional baseline achieves--while maintaining robustness to workforce turnover up to 50% and observation noise up to $\sigma = 0.20$.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

71,552 characters extracted from source content.

Expand or collapse full text

Sustainable Multi-Agent Crowdsourcing via Physics-Informed Bandits Chayan Banerjee Chayan Banerjee is with the School of Electrical Engineering and Robotics, Queensland University of Technology (QUT), Brisbane, QLD, Australia. (email:c.banerjee@qut.edu.au). Abstract Crowdsourcing platforms face a four-way tension between allocation quality, workforce sustainability, operational feasibility, and strategic contractor behaviour—a dilemma we formalise as the Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma. Existing methods resolve at most two of these tensions simultaneously: greedy heuristics and multi-criteria decision making (MCDM) methods achieve Day-1 quality but cause catastrophic burnout, while bandit algorithms eliminate burnout only through operationally infeasible 100% workforce utilisation. To address this, we introduce FORGE, a physics-grounded K+1K+1 multi-agent simulator in which each contractor is a rational agent that declares its own load-acceptance threshold based on its fatigue state, converting the standard passive Restless Multi-Armed Bandit (RMAB) into a genuine Stackelberg game. Operating within FORGE, we propose a Neural-Linear UCB allocator that fuses a Two-Tower embedding network with a Physics-Informed Covariance Prior derived from offline simulator interactions. The prior simultaneously warm-starts skill-cluster geometry and UCB exploration landscape, providing a geometry-aware belief state from episode 1 that measurably reduces cold-start regret. Over T=200T=200 cold-start episodes, the proposed method achieves the highest reward of all non-oracle methods (LRew=0.555±0.041LRew=0.555± 0.041) at only 7.6% workforce utilisation—a combination conventional baseline achieves—while maintaining robustness to workforce turnover up to 50% and observation noise up to σ=0.20σ=0.20. I Introduction I-A Motivation Task allocation in modern digital gig economies and spatial crowdsourcing platforms is fundamentally a challenge of sustainable agentic allocation [25]. A central allocating agent must continuously match complex user requests to a decentralised pool of autonomous sub-agents (workers). This dynamic routing problem is constrained by a critical tension among four traditionally isolated objectives: short-term allocation quality, long-term workforce sustainability, operational feasibility, and the strategic behaviour of contractors themselves. In this paper, we unify these competing constraints into a novel framework, which we term the Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma. To avoid the cold-start problem, legacy platforms frequently rely on greedy heuristics or Multi-Criteria Decision Making (MCDM) algorithms such as TOPSIS [30, 14]. While these methods require zero training data and perform well on Day 1 (i.e., immediately upon deployment before any interaction data is gathered) by selecting the most suitable candidates, they are inherently “fatigue-blind.” They disproportionately route tasks to the top-performing agents, leading to severe workload imbalances and worker overload [2]. Even when contractors can partially self-protect by reducing their own load acceptance, our experiments show that TOPSIS and Greedy (Max-Reputation) still accumulate 2323–2929 burnout events over a 200-episode horizon— equivalent to roughly 12–14% of all allocations—even when every contractor can self-protect. Furthermore, they severely bottleneck platform sustainability, concentrating workforce utilisation on as few as 6 contractors from a pool of 100. Conversely, modelling this environment as a Restless Multi-Armed Bandit (RMAB)—where worker fatigue and readiness are treated as hidden Markov states [15]—using statistical learning algorithms such as LinUCB [17] and Thompson Sampling [7] naturally balances the workload over time. However, these exploratory agents face two practical barriers in live commercial environments. First, they incur a meaningful cold-start penalty, requiring hundreds of early-phase interactions to converge [5]. Second, and more critically, they achieve low burnout only through 100% workforce utilisation—requiring every contractor in the pool to be regularly engaged. In real markets where contractor re-engagement carries non-trivial operational friction and financial cost [18], this is structurally infeasible regardless of its burnout performance. I-B Scope and High-Level Approach This work addresses a gap at the intersection of two insufficiencies: heuristic methods that are fatigue-blind, and bandit methods that are operationally infeasible. Neither camp models the most practically important feature of real crowdsourcing markets—contractors are not passive arms; they observe their own state and make strategic availability decisions to protect their long-run earnings [6, 28, 19]. We therefore begin not with an allocator, but with a richer problem structure: a K+1K+1 agent system in which a central allocating principal and K contractor agents are simultaneous decision-makers, each with its own local state, local action set, and local objective. This structure demands two distinct contributions, developed jointly and evaluated together. The FORGE Simulator : No existing benchmark simultaneously captures the features that make sustainable allocation genuinely hard: restless fatigue dynamics that evolve continuously regardless of selection, endogenous surge pricing that feeds back into allocation incentives, and strategic contractor agency in which each sub-agent declares a load acceptance multiplier at,kc∈0.5,1.0a^c_t,k∈\0.5,1.0\ based on its current fatigue state. We construct a data-derived, physics-grounded environment—extending the task-contractor embedding structure and surge pricing mechanics of COALESCE [4] with continuous restless fatigue dynamics, a burnout threshold, and strategic contractor agency, none of which are present in the original framework—that provides all three. Sub-agents possess latent capabilities encoded as sentence embeddings, hidden fatigue trajectories governed by a load–recovery differential, and a Stackelberg-like interaction structure [31] in which contractors act first and the allocator observes and responds. FORGE formalizes the Cold-Start, Burnout, Utilisation, and Strategic Agency dilemma in a single reproducible simulation that strictly exceeds the expressiveness passive RMAB benchmark. The Neural-Linear UCB Allocator with Physics-Informed Prior : Operating within this harder environment, we propose a Hybrid Contextual Bandit that resolves the four-way dilemma without architectural complexity proportional to its difficulty. Built on a Two-Tower neural architecture, the allocator maps high-dimensional task and contractor representations into a shared embedding space, bypassing the explicit transition matrices required by Whittle Index solutions [29, 20]. The strategic availability signal at,kca^c_t,k enters purely as an additional scalar in the observable context vector—no structural modification is needed to handle the K+1K+1 environment. The allocator learns to read partial availability as a leading indicator of approaching burnout, producing emergent load-balancing without hard-coded rules. To address the cold-start penalty, the allocator is warm-started via an offline-to-online transfer paradigm [27, 21]. The Physics Prior—a gradient feature covariance matrix pre-computed from synthetic FORGE interactions—serves two simultaneous functions: it initialises the neural backbone with the skill-cluster geometry of the contractor pool so predictions are meaningful from episode one, and it pre-warps the UCB confidence ellipsoid so early exploration concentrates on genuinely ambiguous contractors rather than treating all arms as uniformly unknown. The result is not elimination of the cold-start penalty but a geometry-aware belief state that confers measurable robustness to observation noise and workforce turnover throughout the allocation horizon. Together, these contributions occupy a qualitatively distinct operating point on the burnout–reward–utilisation surface that no conventional baseline reaches: FORGE defines the problem faithfully; the allocator solves it sustainably. Our specific contributions are: • Empirical Diagnosis of the Four-Way Dilemma: We demonstrate that purely exploitative heuristics (Greedy, TOPSIS) cause concentrated burnout even when contractors can self-protect, while standard bandits eliminate burnout only by requiring 100% % workforce utilization, an operationally infeasible constraint. Furthermore, all existing methods treat contractors as passive arms, failing to model the strategic availability decisions that arise in real gig-economy markets. • Neural-Linear Bandit Architecture: We develop a Two-Tower NeuralUCB agent capable of operating over highly non-linear, high-dimensional capability spaces while maintaining the sample efficiency of linear bandit upper-confidence bounds. The context vector transparently incorporates contractor availability as an observable signal. • Physics-Informed Covariance Prior: We introduce a practical offline-to-online transfer methodology. By pre-computing the gradient feature gram matrix from synthetic FORGE interactions, we shape the UCB exploration landscape before any live interaction, conferring robustness to observation noise, workforce turnover, and strategic contractor behaviour rather than claiming elimination of the cold-start penalty. • Pareto-Distinct Operating Point: Comprehensive benchmarking in the strategic multi-agent environment demonstrates that the proposed method occupies a qualitatively distinct position on the burnout–reward–utilisation surface: the highest late-stage reward of any non-oracle method (LRew=0.555±0.041LRew=0.555±0.041), lowest early regret (26.40±4.9026.40±4.90), and selective 7.6% workforce utilisation with tighter cross-seed variance than all baselines—a combination no existing baseline achieves. I Related Work I-A Contextual Bandits and UCB-Based Exploration The multi-armed bandit problem and its UCB solution were placed on rigorous finite-time foundations by Auer et al. [3], establishing the exploration–exploitation regret bounds that all subsequent contextual work extends. However, UCB1 assumes stationary reward distributions, a core limitation in dynamic marketplace environments where agent fatigue renders reward non-stationary by construction. Li et al. [17] introduced LinUCB, the standard linear contextual bandit, demonstrating strong personalisation performance for news article recommendation. The linear reward assumption is explicitly insufficient for the non-linear fatigue–capability interaction modelled in this work; we evaluate LinUCB directly as a primary baseline and observe that while it achieves near-zero burnout it does so only by engaging 100% of the contractor pool. Zhou et al. [33] extended UCB exploration to neural function approximators via gradient-based feature maps. In practice, we adopt the Neural-Linear UCB variant of Riquelme et al. [24], which uses the final shared representation layer as the feature map rather than the full parameter gradient, keeping the covariance matrix at 64×6464×64 and updates tractable. Our contributions beyond both works are the Physics-Informed covariance prior (A0A_0) and the Two-Tower inductive bias for task–contractor matching, neither of which is addressed in the original formulations. Garivier and Moulines [10] proposed Sliding-Window UCB (SW-UCB) for non-stationary environments by windowing the interaction history. While this partially handles fatigue as exogenous non-stationarity, SW-UCB treats state transitions as unpredictable rather than modelling the endogenous load–recovery dynamics our formulation captures. Chapelle and Li [7] established Thompson Sampling as a competitive alternative to UCB for contextual bandits. Our evaluation shows Thompson Sampling achieves zero burnout events; however, this comes at the cost of 100% contractor utilisation, which we argue is an operationally infeasible requirement for real crowdsourcing markets. I-B Restless Multi-Armed Bandits and Workforce Scheduling Whittle [29] defined the RMAB problem and proposed the Whittle Index as a tractable allocation heuristic, requiring explicit Markov transition probability matrices and indexability conditions. Papadimitriou and Tsitsiklis [20] showed that the general RMAB is PSPACE-hard, formally motivating approximate methods for large contractor pools. Glazebrook et al. [11] extended Whittle indexability conditions, but their structural assumptions are not satisfied by the continuous fatigue dynamics used in this work—directly motivating the neural approximation approach. Jakher et al. [15] address task assignment in distributed supply chains with worker downtime modelled as discrete hidden Markov states. Our environment uses a continuous fatigue trajectory ft,k∈[0,1]f_t,k∈[0,1] with a hard burnout threshold, enabling finer-grained burnout detection than discrete-state RMAB formulations permit. I-C Crowdsourcing Platform Allocation and MCDM Xie et al. [30] propose TOPSIS-based QoS evaluation for knowledge-intensive crowdsourcing, validated for static quality ranking. The method does not model temporal fatigue or endogenous pricing, and our experiments document this limitation quantitatively at 634634–638638 burnout events over 2000 episodes. Ho and Vaughan [13] study dynamic task assignment under worker heterogeneity and budget constraints but assume workers are stationary and available on demand. The fatigue and burnout dynamics central to our work are absent, establishing a boundary condition that static assignment frameworks cannot address. Bhatt et al. [4] introduced COALESCE, a framework for skill-based task outsourcing among autonomous LLM agents, built around sentence-embedding-based capability representation, a TOPSIS-based contractor selection mechanism, and a demand-surge pricing model. This work inherits these three components as the marketplace scaffold for the FORGE simulator. However, COALESCE was designed for GPU cost optimisation between stateless LLM agents; restless fatigue dynamics, burnout thresholds, and strategic contractor agency are entirely absent. FORGE introduces all three, converting the COALESCE scaffold into a K+1K+1 multi-agent environment with restless internal states. The Neural-Linear UCB allocator with Physics-Informed Prior, also absent from COALESCE, provides the allocation intelligence layer that operates sustainably within it. I-D Offline-to-Online Transfer and Warm-Starting Rashidinejad et al. [21] established pessimism-based offline RL with provable online transfer guarantees, focusing on policy-level transfer. Our contribution is specifically the transfer of the covariance structure (A0A_0)—a more lightweight form of prior injection suited to bandit settings that does not require full policy distillation. Schweighofer et al. [1] analyse how offline dataset coverage affects online transfer quality, directly relevant to our robustness experiments in which the workforce turnover grid (ρ=0ρ=0–50%50\%) operationalises the coverage gap they describe analytically. Van Remmerden et al. [27] apply offline RL to static job shop scheduling environments. Their setting lacks the restless continuous fatigue dynamics and the covariance warm-start mechanism that distinguish our approach. I-E Neural Representation for Matching and Recommendation Covington et al. [8] introduced the Two-Tower architecture for industrial recommendation, demonstrating its effectiveness at separating query and item feature encoding. We adopt this architectural inductive bias to decouple task and contractor representations before their interaction is scored; the UCB exploration layer and workforce sustainability objective are absent from the original recommendation setting. Reimers and Gurevych [22] developed Sentence-BERT and the all-MiniLM-L6-v2 backbone used for task and contractor capability embeddings. The embedding model is treated as a fixed feature extractor; adapting it to marketplace dynamics is left to the UCB and neural layers. Yi et al. [32] address training distribution shift in Two-Tower models under streaming updates, establishing that the architecture supports stable incremental updates. This underpins the feasibility of our sliding-window online gradient descent in the live allocation phase. Positioning Summary. Existing work divides into two camps that are insufficient in isolation. MCDM and greedy heuristics achieve Day-1 stability but are fatigue-blind, producing systematic burnout in restless agent pools. Bandit and RL methods achieve long-run optimality through exploration but require either explicit RMAB transition matrices, stationary reward assumptions, or exhaustive contractor utilisation. The offline transfer literature provides theoretical grounding for pre-training but has not been applied to covariance warm-starting in neural bandits. Critically, all prior work in this space treats contractors as passive arms with fixed, externally imposed parameters—no existing method models the contractor as a rational agent with its own fatigue state and load acceptance decision. The proposed Neural-Linear UCB with Physics-Informed Prior addresses this intersection, inheriting representational power from Two-Tower matching, non-stationary awareness from UCB exploration, a lightweight offline initialisation requiring no live interactions, and robustness to the strategic availability decisions of K simultaneously acting contractor agents. Figure 1: FORGE Simulator — K+1K+1 Multi-Agent System. The offline phase pre-computes a Physics-Informed Prior from dataset simD_sim, injecting initial weights 0 θ_0 and covariance 0−1A_0^-1 into the Allocator at t=1t=1. During live allocation, each contractor independently declares availability at,kca^c_t,k via a fatigue-threshold policy before the Allocator acts. Observable state variables and the availability signal are concatenated with the task query into a 493-dimensional context vector t,kx_t,k, which drives the Allocator’s selection ata_t. The Marketplace Environment processes this decision through three parallel dynamics—success probability, fatigue (RMAB), and surge pricing—feeding reward rtr_t back to the Allocator and updated states back into the context vector each episode. I Problem Formulation I-A The Multi-Agent Marketplace and POMDP Tuple We model the task allocation marketplace as a K+1K+1 agent system: a centralised principal routing tasks to a decentralised set of K independent sub-agents (contractors), denoted C=c1,c2,…,cKC=\c_1,c_2,…,c_K\. Unlike prior RMAB formulations that treat contractors as passive arms, each contractor is a rational agent with its own local state, local action set, and local objective (Section I-C). At each discrete time step t, a user submits a task query qtq_t. First, every contractor declares its availability; then the central allocator selects a single contractor to execute the task. The environment operates as a Partially Observable Markov Decision Process (POMDP) [16] from the allocator’s perspective, formally defined by the tuple (,,,ℛ)(S,A,P,R): • State Space (S): The global system state encompasses hidden and observable variables. The hidden state contains the true continuous latent capabilities ϕk∈ℝdq _k ^d_q. The observable state tracks ft,kf_t,k (fatigue), pt,kp_t,k (dynamic price), Rt,kR_t,k (reputation), and at,kca^c_t,k (declared availability) for each contractor. • Action Space (A): At each step t the allocator takes at∈Ca_t∈ C, selecting exactly one contractor to execute qtq_t. • Transition Dynamics (P): The environment transitions deterministically in price and reputation for the selected agent, and autonomously and restlessly in the fatigue states of all K agents (both selected and unselected). The fatigue increment of the selected contractor is scaled by its declared availability at,kca^c_t,k (see Section I-C). • Reward Function (ℛR): The immediate reward is the binary task outcome rt∈0,1r_t∈\0,1\, drawn from the underlying true probability of success Pactual(rt=1∣qt,cat)P_actual(r_t=1 q_t,c_a_t). I-B Restless State Dynamics and Economic Feedback Unlike standard contextual bandits, the contractors in our system are modelled as Restless Multi-Armed Bandits (RMABs) [29]. Each contractor possesses a continuous, dynamically evolving fatigue state ft,k∈[0,1]f_t,k∈[0,1]. The fatigue state transitions autonomously based on the allocator’s actions and the contractor’s declared availability (Section I-C). If contractor ckc_k is selected (at=ka_t=k), fatigue increases by a load scaled by the contractor’s availability multiplier; if not selected (at≠ka_t≠ k), it recovers at a natural rate: ft+1,k=min⁡(1.0,ft,k+at,kc⋅loadk)if at=kmax⁡(0.0,ft,k−recoveryk)if at≠kf_t+1,k= cases (1.0,\;f_t,k+a^c_t,k·load_k)&if a_t=k\\ (0.0,\;f_t,k-recovery_k)&if a_t≠ k cases (1) where at,kc∈0.5,1.0a^c_t,k∈\0.5,1.0\ is the contractor’s declared availability multiplier. In the original passive RMAB setting, at,kc≡1.0a^c_t,k≡ 1.0 for all k, recovering the standard linear fatigue update. To model the “Star Performer” problem, the environment imposes a structural penalty when a contractor’s fatigue exceeds the critical burnout threshold θburnout _burnout, collapsing the actual probability of success to a fraction of the base: Pactual=Pbaseif ft,k≤θburnout0.1×Pbaseif ft,k>θburnoutP_actual= casesP_base&if f_t,k≤ _burnout\\ 0.1× P_base&if f_t,k> _burnout cases (2) Note that at,kca^c_t,k modulates the fatigue trajectory but does not directly affect task success probability on the current episode; a partial-availability contractor delivers the same quality on an assigned task while accumulating reduced fatigue for future episodes. The system also features endogenous economic feedback modelled on demand-surge pricing dynamics [26]. The current price pt,kp_t,k surges with frequent selection: pt,k=Pbase,k×(1+γDt,kSk)p_t,k=P_base,k× (1+γ D_t,kS_k ) (3) where Pbase,kP_base,k is the fixed baseline cost, Dt,kD_t,k is a decaying demand counter incremented upon selection, SkS_k is the contractor’s baseline supply capacity, and γ is the surge multiplier. Each contractor’s observable reputation Rt,kR_t,k is maintained as a moving average of past task outcomes. I-C Contractor Agency: The Strategic Availability Decision In real crowdsourcing markets, contractors are not passive arms. They observe their own state and can modulate their engagement to protect their long-run earnings. We formalise this with a minimal contractor agency model that requires no new learning algorithm or reward function for contractors. Contractor action set. At each episode t, before the central allocator acts, every contractor ckc_k independently declares a load acceptance multiplier: at,kc∈c=0.5, 1.0a^c_t,k ^c=\0.5,\;1.0\ (4) A value of 1.01.0 signals full availability (the contractor accepts the task at full load); 0.50.5 signals partial availability (the contractor accepts but requests reduced load, accumulating half the normal fatigue increment via Equation 1). Contractor objective and threshold policy. The contractor’s implicit objective is to maximise cumulative selection frequency—and therefore surge revenue pt,kp_t,k—while avoiding the burnout collapse that reduces PactualP_actual to 0.1×Pbase0.1× P_base. This objective is already embedded in the existing economic model; no new utility function is introduced. A contractor with fatigue ft,kf_t,k approaching θburnout _burnout has an incentive to signal partial availability to reduce its fatigue increment; a rested contractor accepts full load to remain attractive to the allocator. We implement this as a deterministic threshold policy: at,kc=0.5if ft,k>ζ⋅θburnout1.0otherwisea^c_t,k= cases0.5&if f_t,k>ζ· _burnout\\ 1.0&otherwise cases (5) where ζ=0.75ζ=0.75 is the self-protection trigger (fatigue exceeds 75% of the burnout threshold). This policy is rational: it reduces the probability of hitting burnout while accepting only a modest reduction in immediate attractiveness to the allocator. Why this creates a genuine multi-agent dynamic. The system now has K+1K+1 simultaneous decision-makers. The central allocator optimises cumulative task success. Each contractor optimises its own long-run availability. The joint behaviour is emergent: as the allocator learns to interpret at,kc=0.5a^c_t,k=0.5 as a signal of approaching fatigue, it routes around partially available contractors proactively, further reducing burnout concentration. This constitutes a legitimate Stackelberg-like interaction—contractors act first, allocator observes and selects—that is absent from all existing bandit and RMAB allocation frameworks. Minimal design. The formulation is deliberately minimal. Equation 1 already supports a multiplicative load factor; fixing it to 1.0 recovers exact backward compatibility. The availability signal at,kca^c_t,k is appended to the context vector as a single scalar observable. The central allocator requires no structural modification. I-D Optimization Objective: The Four-Way Trade-off The central allocator’s primary objective is to maximise the cumulative expected success rate over horizon T, equivalently formulated as minimising the expected cumulative regret ℛ(T)R(T). Let at∗∈Ca_t^*∈ C denote the oracle-optimal allocation at time t (the agent with the highest true probability of success for query qtq_t, assuming zero fatigue). The cumulative regret is: ℛ(T)=∑t=1T[Pactual(rt=1∣qt,cat∗)−Pactual(rt=1∣qt,cat)]R(T)= _t=1^TE\! [P_actual(r_t=1 q_t,c_a_t^*)-P_actual(r_t=1 q_t,c_a_t) ] (6) To minimise ℛ(T)R(T), the allocator must accurately estimate the latent capability ϕk _k for all agents while also anticipating their strategic availability decisions. Achieving this in a Restless Strategic Multi-Agent Environment introduces a fundamental four-way tension. • High-Capacity Agent Burnout (The Cost of Exploitation): If the allocator relies on greedy heuristics—rapidly identifying and continuously exploiting the most capable agents (“Star Performers”)—it structurally degrades the workforce. Continuous exploitation prevents natural recovery, driving ft,kf_t,k past θburnout _burnout, after which PactualP_actual collapses to 0.1×Pbase0.1× P_base. Contractor self-protection partially mitigates but does not eliminate this risk: heuristics that do not observe at,kca^c_t,k cannot anticipate when a contractor will switch to partial availability. • The Cold-Start Penalty (The Cost of Exploration): At initialisation (t=0t=0) the allocator has zero historical interactions and maximum uncertainty regarding ϕk _k. In our dynamic marketplace, naive early exploration not only yields immediate regret but artificially triggers surge pricing (pt,kp_t,k) and wastes system capacity without guaranteeing successful completions. • Operational Utilisation Constraint: Pure exploration-based methods (e.g., LinUCB, Thompson Sampling) resolve burnout by distributing allocations uniformly across the full contractor pool, achieving 100% workforce utilisation. In real crowdsourcing markets, re-engaging every contractor on a regular basis carries non-trivial operational and logistics cost. Platforms require selective utilisation—engaging a high-quality subset— rather than exhaustive coverage. • Strategic Contractor Behaviour: Contractors are not passive. A contractor signalling partial availability (at,kc=0.5a^c_t,k=0.5) imposes an additional information asymmetry on the allocator: the reduced availability is observable but the underlying fatigue trajectory that caused it may not be fully trusted. Allocators that cannot interpret this signal correctly will either over-select fatigued contractors (ignoring the signal) or under-utilise capable ones (over-reacting to it). The allocator therefore faces a strict four-way optimisation trade-off. Resolving this tension simultaneously forms the fundamental problem addressed in this work. IV The FORGE Simulation Environment To rigorously evaluate allocation policies against the problem formulated in Section I, standard static datasets are fundamentally insufficient because they lack counterfactuals and emergent state transitions [9]. Therefore, we construct a custom data-derived simulator based on the COALESCE framework [4]. IV-A Task and Capability Embeddings Task queries qtq_t and latent capabilities ϕk _k are instantiated as high-dimensional text embeddings in ℝdqR^d_q (dq=384d_q=384), derived from the all-MiniLM-L6-v2 sentence encoder [22]. While a contractor holds a broad observable ontological tag (e.g., “Medical”), the 384-dimensional embedding encodes their precise hidden niche (e.g., Paediatric Neurology vs. Cardiology). Task compatibility is computed via the cosine similarity between the task query and the contractor’s latent capability: st,k=qt⋅ϕk‖qt‖‖ϕk‖s_t,k= q_t· _k\|q_t\|\,\| _k\| (7) Following standard practice in click-through-rate modelling [23], the baseline probability of success PbaseP_base is modelled via a shifted sigmoid function simulating the non-linear competency threshold: Pbase(rt=1∣qt,ck)=11+exp⁡(−(αst,k−β))P_base(r_t=1 q_t,c_k)= 11+ (-(α s_t,k-β)) (8) where α and β are empirically tuned to control the sharpness and difficulty of the marketplace physics. IV-B Agent Initialisation and Hyperparameters The simulation generates K=100K=100 contractors, each initialised with a $20 minimum task fee (Pbase,kP_base,k) and a 50 ms geographic network delay (lkl_k) simulating real-world platform constraints. The observable reputation Rt,kR_t,k is maintained as a moving average of past task outcomes. IV-C The Observable Context Space The central allocator cannot observe ϕk _k directly; it makes decisions based on an observable context vector constructed from all available signals, including the contractor’s declared availability. With contractor agency enabled, xt,k∈ℝ493x_t,k ^493: xt,k=[qt‖Tagonehot‖IDonehot‖ft,k‖p~t,k‖l~k‖at,kc]x_t,k= [q_t _onehot _onehot f_t,k p_t,k l_k a^c_t,k ] (9) where p~t,k p_t,k and l~k l_k are the normalised current price and latency, and at,kc∈0.5,1.0a^c_t,k∈\0.5,1.0\ is the contractor’s declared availability for this episode (Section I-C). This vector is the sole input to all allocation agents evaluated in Section VI. The availability feature is appended as the 493rd scalar, leaving the 384-dimensional query embedding and the 108-dimensional contractor profile slice (Tag(5) + ID(100) + Scalars(3)) identical to the original 492-D design. Setting at,kc≡1.0a^c_t,k≡ 1.0 for all k exactly recovers the original passive-contractor context, enabling direct ablation. V Proposed Methodology: Neural-Linear Contextual Allocation To resolve the four-way dilemma defined in Section I, we propose a Neural-Linear Contextual Bandit agent augmented with a Physics-Informed Prior and static heuristic fusion. This approach fuses the representational power of deep learning—necessary to map the high-dimensional continuous context space—with the principled uncertainty quantification of UCB exploration and the Day-1 constraint satisfaction of multi-criteria decision making. The contractor availability signal at,kca^c_t,k is incorporated transparently as an additional observable feature; no structural change to the allocator is required to handle the strategic K+1K+1 agent environment. V-A Two-Tower Neural Representation (The Approximator) Because the allocator cannot directly observe ϕk _k, it approximates the expected reward (task success probability) from the observable context xt,kx_t,k. Given the highly non-linear relationship between the task embedding qtq_t and the contractor’s dynamic state (fatigue ft,kf_t,k and surged price pt,kp_t,k), linear bandit models such as LinUCB are structurally insufficient. We adopt a Two-Tower neural architecture [8], parameterised by weights θ∈ℝmθ ^m: r^t,k=h(xt,k;θt) r_t,k=h(x_t,k;\, _t) where h(⋅)h(·) projects the task query features and the contractor profile features into a shared dense representation space before computing their interaction score. Following the Neural-Linear UCB framework [24], we use the final shared representation layer as the feature map. For a given context xt,kx_t,k, the 64-dimensional feature vector is the element-wise (Hadamard) product of the two tower embeddings: ϕt,k=t⊙t,k∈ℝd,d=64 _t,k=q_t _t,k\;∈\;R^d, d=64 where t=QueryTower(qt)∈ℝ64q_t=QueryTower(q_t) ^64 and t,k=ContractorTower(xt,k(c))∈ℝ64c_t,k=ContractorTower(x_t,k^(c)) ^64 are the bounded (tanh ) outputs of the respective towers. Using the last-layer representation rather than the full parameter gradient keeps the covariance matrix at d×d=64×64d×d=64×64, making Sherman–Morrison updates and periodic full re-inversion computationally tractable. V-B Fatigue-Aware Upper Confidence Bound To prevent “Star Performer” collapse, the allocator must dynamically route tasks away from highly capable agents before their fatigue ft,kf_t,k breaches θburnout _burnout. We achieve this through a principled exploration–exploitation trade-off. The agent maintains a regularised gram matrix At∈ℝd×dA_t ^d× d (d=64d=64) of observed feature vectors. The uncertainty bonus for contractor ckc_k is: σt,k=ϕt,k⊤At−1ϕt,k _t,k= _t,k A_t^-1 _t,k The Neural-Linear UCB score for each contractor is: UCBt,knl=r^t,k+βσt,kUCB^nl_t,k= r_t,k+β\, _t,k where β>0β>0 is the exploration coefficient (β=0.06β=0.06 in all experiments). Mechanism of Burnout Prevention and Availability Adaptation: Because xt,kx_t,k explicitly includes the continuous fatigue state ft,kf_t,k, dynamic price pt,kp_t,k, and declared availability at,kca^c_t,k, the neural network h(⋅)h(·) learns the steep drop in PactualP_actual that occurs past θburnout _burnout and simultaneously correlates at,kc=0.5a^c_t,k=0.5 with approaching fatigue. As a “Star Performer” is repeatedly selected, their ft,kf_t,k increases, the network predicts a lower r^t,k r_t,k, and simultaneously the exploration bonus σt,j _t,j for rested alternative contractors cjc_j grows. When a contractor begins signalling partial availability (at,kc=0.5a^c_t,k=0.5), this further suppresses its predicted reward in the neural model, reinforcing the rotation toward rested alternatives. The UCB objective thus produces emergent load balancing that respects both the physical fatigue dynamics and the strategic self-protection signals of individual contractors, without requiring hard-coded RMAB transition matrices. V-C Offline Pre-training: The Physics-Informed Prior Standard online learning algorithms, including NeuralUCB, initialise the neural weights θ0 _0 randomly and the covariance matrix as an isotropic identity A0=λIA_0=λ I. This induces a cold-start penalty: the agent requires significant early-phase exploration, artificially surging prices and consuming workforce capacity before converging on a viable allocation policy. To substantially mitigate this penalty, we introduce the Physics Prior via offline-to-online transfer learning [21]. Before live deployment, the data-derived simulator (Section IV) generates a large offline dataset sim=(xi,ri)i=1ND_sim=\(x_i,r_i)\_i=1^N. The allocator is pre-trained in two phases: 1. Weight Initialisation: The initial network weights θ0 _0 are computed via supervised learning on simD_sim. Labels are set to the raw success probability ri=Pbase(ck,qi)∈(0,1)r_i=P_base(c_k,q_i)∈(0,1) rather than sampled binary outcomes, reducing gradient variance. The loss is binary cross-entropy with logits: ℒ(θ)=−1N∑i=1N[rilog⁡σ(hi)+(1−ri)log⁡(1−σ(hi))]L(θ)=- 1N _i=1^N [r_i σ(h_i)+(1-r_i) (1-σ(h_i)) ] where hi=h(xi;θ)h_i=h(x_i;θ) is the network’s scalar logit output and σ(⋅)σ(·) is the sigmoid function. Fatigue is zeroed for every sample in simD_sim and contractor ID features are set to zero (xi(ID)←x_i^(ID) 0), so the prior encodes structural skill-cluster geometry only, not transient fatigue states or contractor identities. This makes the prior transferable to new contractors entering the pool during live deployment. Upon loading θ0 _0 into the live allocator, the ID weight columns of the contractor tower are re-initialised (Kaiming uniform) so identity information can be learned from scratch during online operation. 2. Prior Covariance Matrix (A0A_0): Rather than an uninformative identity matrix, we construct an offline gram matrix summarising the structural geometry of the simulated workforce. The initial covariance is computed over the 64-D feature embeddings of all pre-training samples: A0=λI+∑i=1Nϕiϕi⊤,ϕi=i⊙iA_0=λ I+ _i=1^N _i\, _i , _i=q_i _i The stored artifact is the scaled inverse: A0−1=α⋅(λI+∑iϕiϕi⊤)−1,α=10.0A_0^-1=α· (λ I+ _i _i _i )^-1, α=10.0 The scale factor α inflates the initial UCB bonus by α≈3.16× α≈ 3.16×, encouraging early workload rotation before online data accumulates. By injecting this Physics Prior (A0−1A_0^-1 and θ0 _0) at t=1t=1, the allocator enters the live environment with a geometry-aware uncertainty estimate. The confidence ellipsoid is pre-warped to reflect the skill-cluster geometry of the simulated workforce, reducing σt,k _t,k for well-characterised sub-optimal agents and concentrating exploration on genuinely ambiguous contractors. The validated benefit of this prior is robustness: as demonstrated in Section VII-B, it maintains a stable regret profile under observation noise and workforce turnover conditions where identity-initialised variants degrade substantially. V-D Hybrid Fusion Strategy (Integrating TOPSIS) While the Physics Prior initialises the neural network effectively, MCDM algorithms such as TOPSIS offer mathematically guaranteed adherence to hard static business constraints (e.g., ontological tag matching and cost minimisation) on Day 1. To combine Day-1 constraint satisfaction with long-term learned adaptability, our final allocation policy computes a fused acquisition score. For each agent k, we compute the TOPSIS closeness coefficient Ct,kC_t,k from the COALESCE framework. The final selection score is: Ut,k=r^t,k+βσt,k⏟Neural-Linear UCB+ηtCt,k⏟TOPSISU_t,k= r_t,k+β\, _t,k_Neural-Linear UCB+ _t\,C_t,k_TOPSIS where ηt=η0⋅δt _t= _0·δ^t decays multiplicatively each episode (δ=0.9995δ=0.9995), so TOPSIS provides strong constraint satisfaction on Day 1 and fades as the neural model converges. The allocator selects at=arg⁡maxk∈C⁡Ut,ka_t= _k∈ CU_t,k, with off-tag contractors masked to −∞-∞ before the argmax. V-E Online Adaptation and Update Rule After selecting contractor ata_t and observing rt∈0,1r_t∈\0,1\, the agent updates its internal state. The gram matrix receives a rank-one feature update: At=At−1+ϕt,atϕt,at⊤A_t=A_t-1+ _t,a_t\, _t,a_t The inverse At−1A_t^-1 is maintained via the Sherman–Morrison formula [12]: At−1=At−1−1−(At−1−1ϕt,at)(At−1−1ϕt,at)⊤1+ϕt,at⊤At−1−1ϕt,atA_t^-1=A_t-1^-1- (A_t-1^-1 _t,a_t)(A_t-1^-1 _t,a_t) 1+ _t,a_t A_t-1^-1 _t,a_t avoiding the O(d3)O(d^3) cost of full re-inversion each step. Every τ=100τ=100 steps At−1A_t^-1 is recomputed from AtA_t directly to reset floating-point drift accumulated as θ changes. Simultaneously, θt _t is updated via mini-batch BCE gradient descent over a bounded replay buffer of the B=100B=100 most recent interactions, enforcing recency bias in the non-stationary environment. Algorithm 1 Full Hybrid Allocation: Neural-Linear UCB + TOPSIS + Physics Prior (K+1K+1 Agent) 0: Offline simulator dataset simD_sim, Two-Tower network h(⋅;θ)h(·;θ), UCB scalar β>0β>0, TOPSIS weight η0>0 _0>0, decay δ∈(0,1]δ∈(0,1], availability threshold ζ=0.75ζ=0.75. 0: Phase 1: Simulator Pre-training (Offline) 1: Initialise θ randomly. 2: Train θ0←arg⁡minθ−1N∑i=1N[rilog⁡σ(hi)+(1−ri)log⁡(1−σ(hi))] _0← _θ- 1N _i=1^N [r_i σ(h_i)+(1-r_i) (1-σ(h_i)) ] BCE with soft labels; fatigue zeroed, IDs zeroed in simD_sim 3: Initialise A0←λIA_0←λ I 4: for each (xi,⋅)∈sim(x_i,·) _sim do 5: ϕi←QueryTower(xi(q))⊙ContractorTower(xi(c)) _i (x_i^(q)) (x_i^(c)) 64-D Hadamard embedding 6: A0←A0+ϕiϕi⊤A_0← A_0+ _i _i 7: end for 8: A0−1←α⋅A0−1A_0^-1←α· A_0^-1, cache A0−1A_0^-1 α=10.0α=10.0 widens initial bonus; re-init ID weights Kaiming uniform 8: Phase 2: Live Marketplace Allocation (Online) 9: for t=1,2,…,Tt=1,2,…,T do 9: ⊳ Step 0: Contractor availability declarations 10: for each contractor ck∈Cc_k∈ C do 11: at,kc←0.5a^c_t,k← 0.5 if ft,k>ζ⋅θburnoutf_t,k>ζ· _burnout, else 1.01.0 12: end for 13: Observe incoming task query qtq_t 14: for each available contractor ck∈Cc_k∈ C do 15: Observe fatigue ft,kf_t,k, surged price pt,kp_t,k, availability at,kca^c_t,k 15: ⊳ 1. Static Heuristic (TOPSIS) 16: Compute closeness score Ct,kC_t,k 16: ⊳ 2. Neural Representation 17: xt,k←[qt‖Tag‖ID‖ft,k‖p~t,k‖l~k‖at,kc]x_t,k←[q_t f_t,k p_t,k l_k a^c_t,k] 18: r^t,k←σ(h(xt,k;θt−1)) r_t,k←σ(h(x_t,k; _t-1)) sigmoid of logit output 19: ϕt,k←QueryTower(xt,k(q))⊙ContractorTower(xt,k(c)) _t,k (x_t,k^(q)) (x_t,k^(c)) 19: ⊳ 3. Hybrid Fusion 20: σt,k←ϕt,k⊤At−1−1ϕt,k _t,k← _t,k A_t-1^-1 _t,k 21: Ut,k←r^t,k+βσt,k+ηtCt,kU_t,k← r_t,k+β\, _t,k+ _tC_t,k 22: end for 23: at←arg⁡maxk⁡Ut,ka_t← _kU_t,k 24: Execute task; observe reward rt∈0,1r_t∈\0,1\ 24: ⊳ Fatigue: ft+1,at=min⁡(1,ft,at+at,atc⋅loadat)f_t+1,a_t= (1,f_t,a_t+a^c_t,a_t·load_a_t) 25: At←At−1+ϕt,atϕt,at⊤A_t← A_t-1+ _t,a_t _t,a_t 26: Update At−1A_t^-1 via Sherman–Morrison 27: Update θt _t via BCE replay gradient descent 28: ηt←max⁡(ηt−1⋅δ, 0) _t← ( _t-1·δ,\;0) 29: end for VI Experimental Setup To empirically validate the proposed Hybrid NeuralUCB allocator against the four-way dilemma, we evaluate the system within the data-derived FORGE simulator with contractor agency enabled. All baselines are evaluated in the same K+1K+1 agent environment: every contractor in every condition declares its availability via the threshold policy (Section I-C) before each episode. This ensures that all methods are compared under equivalent strategic conditions—no baseline is disadvantaged by the agency model, but none is architecturally designed to interpret the availability signal either. VI-A Evaluation Baselines and Model Ablation We benchmark against a comprehensive array of standard heuristics and statistical learning baselines. Static Heuristics (Fatigue-Blind): • Greedy (Max-Reputation): Selects the contractor with the highest historical reputation Rt,kR_t,k within the required task tag. • Greedy (Min-Price): Selects the cheapest available contractor arg⁡mink⁡p~t,k _k p_t,k, inadvertently driving emergent round-robin load balancing via surge pricing mechanics. • TOPSIS: The standard MCDM baseline from the COALESCE framework [30], selecting ckc_k by computing the maximum closeness coefficient Ct,kC_t,k across tag match, semantic similarity, reputation, and costs. Statistical Learning Bandits: • LinUCB (Disjoint): The standard contextual bandit modelling expected reward as a linear combination of xt,kx_t,k [17]. • Sliding Window UCB (SW-UCB): A non-stationary adaptation of LinUCB computing exploration bounds from a recent interaction window, designed to rapidly discount fatigued workers [10]. • Thompson Sampling (Linear): A probabilistic matching algorithm sampling the allocation decision from a posterior distribution of context weights [7]. Model Ablation Variants: • Hybrid (No Prior): NeuralUCB initialised with random weights θ0 _0 and identity covariance A0=λIA_0=λ I. • Hybrid + Prior (Proposed): The full architecture with offline pre-training (θ0 _0, A0−1A_0^-1). • Hybrid + Prior (No Fatigue): Oracle control variant with fatigue updates disabled (Δft,k=0 f_t,k=0), establishing the performance upper bound. VI-B Evaluation Metrics We define a unified set of criteria measuring allocation performance, long-term sustainability, and strategic routing quality in the K+1K+1 agent environment. For any evaluation window of length ΔT T: • Average Reward (r¯ r): Mean true probability of success of selected contractors. • Cumulative Regret (ℛR): Accumulated loss against the zero-fatigue Oracle at∗a_t^*. • Burnout Events (ℬB): ℬ=∑t(ft,at>θburnout)B= _tI(f_t,a_t> _burnout), gross count of allocations to a burned-out contractor. • Workforce Utilisation (U): Cardinality of unique contractors selected, measuring the breadth of load distribution. • Strategic Misrouting Rate (SMR): The fraction of allocations in which the selected contractor was already in self-protection mode at the time of selection: SMR=1T∑t=1T(at,atc=0.5)×100%SMR= 1T _t=1^TI\! (a^c_t,a_t=0.5 )× 100\% A high SMR indicates the allocator is systematically routing into contractors who have already signalled fatigue, wasting load capacity and risking burnout. A low SMR indicates proactive avoidance of fatigued contractors before the burnout threshold is crossed. This metric is only meaningful in the K+1K+1 agent setting. • Mean Pre-Selection Fatigue (MPF): The average fatigue level of the selected contractor at the moment of selection: MPF=1T∑t=1Tft,atMPF= 1T _t=1^Tf_t,a_t MPF measures the operating point at which the allocator engages contractors. Methods that consistently select contractors with high MPF are operating close to the burnout threshold and have little recovery headroom. Methods with low MPF maintain a healthy fatigue buffer across the workforce. All metrics are reported as mean ± standard deviation across 5 independent evaluation seeds. SMR and MPF are computed over the full T=200T=200 episode horizon. VI-C Experiment 1: Cold-Start Evaluation The primary evaluation runs T=200T=200 episodes, modelling approximately one working week of platform operation (∼50 50 allocations/day). This horizon is chosen to isolate the cold-start regime where the Physics Prior’s warm-start advantage is most relevant. Each variant is evaluated in a fully independent, identically-seeded replica of the FORGE environment. The timeline is partitioned into: • Phase 1A: Cold-Start (t∈[1,100]t∈[1,100]): Isolates convergence ability without historical data. • Phase 1B: Stabilisation (t∈[150,200]t∈[150,200]): Evaluates allocation quality after early exploration has settled. VI-D Experiment 2: Robustness to Environmental Stress Allocators are subjected to a robustness grid covering three stress factors: • Traffic Surge (ωsurge _surge): A scalar multiplier applied to the fatigue accumulation rate (loadk×ωsurgeload_k× _surge). • Observation Noise (σnoise _noise): Gaussian noise injected into observable fatigue and pricing features, simulating the Sim-to-Real gap. • Workforce Turnover (ρ): Random replacement of 0%–50% of the workforce to test adaptability of the pre-trained prior to non-stationary contractor pools. VII Results and Discussion VII-A Cold-Start Continuous Evaluation (Experiment 1) Can a hybrid bandit allocator learn to interpret contractor availability signals and route away from fatigued workers—without exhausting the entire workforce? Table I summarises performance over T=200T=200 episodes—equivalent to approximately one working week of platform operation at 50 allocations per day. Results are reported as mean ± standard deviation across 5 seeds. Early metrics correspond to Phase 1A (t∈[1,100]t∈[1,100]) and Late metrics to Phase 1B (t∈[150,200]t∈[150,200]). TABLE I: Comparative performance over T=200T=200 episodes with contractor agency enabled (mean ± std, 5 seeds). ERew/EReg = Early Reward/Regret (Phase 1A, t∈[1,100]t\!∈\![1,100]); LRew/LReg = Late Reward/Regret (Phase 1B, t∈[150,200]t\!∈\![150,200]); Burn = Total Burnout Events; Util = Unique Contractors Used; SMR = Strategic Misrouting Rate (% episodes where selected contractor signalled ac=0.5a^c\!=\!0.5); MPF = Mean Pre-Selection Fatigue. Bold = best per column (excluding oracle). Method ERew EReg LRew LReg Burn Util SMR% MPF Greedy (Max-Rep) 0.454±0.0390.454±0.039 37.97±3.8137.97±3.81 0.441±0.0800.441±0.080 19.65±3.4619.65±3.46 22.8±3.522.8±3.5 11.8±1.211.8±1.2 36.2±1.136.2±1.1 0.456±0.0080.456±0.008 Greedy (Min-Price) 0.464±0.0570.464±0.057 37.30±6.5237.30±6.52 0.463±0.0420.463±0.042 18.86±1.9618.86±1.96 6.2±4.16.2±4.1 13.8±2.113.8±2.1 8.9±3.98.9±3.9 0.162±0.0490.162±0.049 TOPSIS 0.534±0.0360.534±0.036 30.32±3.5130.32±3.51 0.538±0.0860.538±0.086 15.11±4.2715.11±4.27 28.8±3.728.8±3.7 5.8±0.75.8±0.7 38.4±2.438.4±2.4 0.485±0.0160.485±0.016 LinUCB 0.442±0.0150.442±0.015 39.48±1.4339.48±1.43 0.518±0.0260.518±0.026 16.10±1.4816.10±1.48 0.2±0.40.2±0.4 99.6±0.599.6±0.5 0.4±0.50.4±0.5 0.023±0.0060.023±0.006 SW-UCB 0.452±0.0130.452±0.013 38.51±0.7538.51±0.75 0.455±0.0120.455±0.012 19.24±0.5919.24±0.59 0.0±0.00.0±0.0 62.4±2.662.4±2.6 0.0±0.00.0±0.0 0.005±0.0030.005±0.003 Thompson Sampling 0.449±0.0230.449±0.023 39.00±3.0739.00±3.07 0.449±0.0220.449±0.022 19.12±0.8719.12±0.87 0.0±0.00.0 0.0 96.6±2.196.6±2.1 0.0±0.00.0 0.0 0.006±0.0030.006±0.003 Hybrid (No Prior) 0.552±0.0820.552±0.082 28.93±8.7328.93±8.73 0.548±0.0710.548±0.071 14.53±2.9014.53±2.90 28.2±2.828.2±2.8 6.8±1.86.8±1.8 38.7±2.638.7±2.6 0.488±0.0200.488±0.020 Hybrid + Prior (Proposed) 0.577±0.0380.577 0.038 26.40±4.9026.40 4.90 0.555±0.0410.555 0.041 14.17±1.8314.17 1.83 25.8±2.625.8±2.6 7.6±1.77.6 1.7 37.4±2.637.4±2.6 0.478±0.0230.478±0.023 Oracle (No Fatigue) 0.655±0.0360.655±0.036 18.63±3.8318.63±3.83 0.635±0.0390.635±0.039 10.19±1.9710.19±1.97 0.0±0.00.0±0.0 5.6±0.85.6±0.8 0.0±0.00.0±0.0 0.000±0.0000.000±0.000 Heuristic Blindness to Strategic Signals. TOPSIS and Greedy (Max-Reputation) route into contractors already in self-protection mode in 3636–38%38\% of episodes, selecting at a mean pre-selection fatigue of 0.4560.456–0.4850.485—nearly halfway to the burnout threshold on every allocation. Because these methods score contractors by reputation or multi-criteria closeness with no representation of current fatigue state, the availability signal provides them no benefit, resulting in 2323–2929 burnout events across a 200-episode horizon. The Utilisation–Misrouting Frontier of Bandit Methods. Thompson Sampling and LinUCB achieve near-zero SMR and MPF by exhaustively covering the contractor pool (Util ≈97≈ 97–100%100\%): every contractor is rested at selection because no contractor is selected often enough to accumulate significant fatigue. Burnout is avoided structurally, not through intelligent routing. SW-UCB tells a similar story with Util =62%=62\%. The critical distinction is that these methods pay for sustainability with low reward—Thompson Sampling’s late reward of 0.4490.449 and LinUCB’s 0.5180.518 are well below both hybrid variants—because exhaustive coverage forces allocation to many mediocre contractors. Hybrid + Prior Leads Across All Reward and Regret Metrics. In the short-horizon regime (T=200T=200), Hybrid + Prior is the strongest performer overall. It achieves the highest early reward (0.5770.577) and lowest early regret (26.4026.40), and maintains that advantage through Phase 1B (LRew =0.555=0.555, LReg =14.17=14.17)—with standard deviations that are notably tighter than Hybrid (No Prior), indicating more consistent behaviour across seeds. The Physics Prior warm-starts the covariance structure from the first episode, enabling selective allocation of high-capability contractors before sufficient online data accumulates to identify them. Hybrid (No Prior) achieves comparable reward but with roughly twice the variance (ERew =0.552±0.082=0.552±0.082 vs. 0.577±0.0380.577±0.038), reflecting the slower, noisier exploration phase that an uninformative covariance initialisation requires. Both hybrid variants substantially outperform all baselines on reward, while maintaining moderate burnout levels (2525–2929 events) that are consistent with targeted high-capability routing rather than exhaustive pool coverage. The Greedy (Min-Price) Anomaly. Greedy (Min-Price) achieves SMR =8.9%=8.9\% and MPF =0.162=0.162 despite being fatigue-blind, because surge pricing (pt,k∝Dt,kp_t,k D_t,k) makes frequently selected contractors expensive and inadvertently rotates load to cheaper, less-fatigued alternatives. Burnout is the lowest among non-bandit methods (6.2±4.16.2±4.1), but late-stage reward (0.4630.463) remains weak—price-driven rotation achieves sustainability by accident rather than by identifying capable contractors. Oracle Interpretation. The Oracle (No Fatigue) removes fatigue accumulation entirely, achieving zero burnout and zero SMR by construction. Its reward of 0.6550.655 (early) establishes the upper bound for selective high-quality allocation under a horizon of T=200T=200; its low Util of 5.6%5.6\% shows that the skill-matched contractor pool is small, making accurate identification from limited data especially valuable. VII-B Robustness to Environmental Stress (Experiment 2) When contractors leave, new ones arrive, and observations are corrupted by noise, which variant better preserves regret? Table I reports cumulative regret over T=200T=200 episodes (10 repeats per cell) under varying workforce turnover (ρ) and observation noise (σ). TABLE I: Cumulative Regret under varying workforce turnover and observation noise with contractor agency enabled (10 repeats/cell, T=200T=200). Lower is better. Bold = best per cell (excluding TOPSIS baseline). Turnover / Noise σ 0.00 0.05 0.10 0.20 Variant A: TOPSIS 0% 59.5 59.5 59.5 59.5 10% 57.8 57.8 57.8 57.8 30% 61.2 61.2 61.2 61.2 50% 54.5 54.5 54.5 54.5 Variant B: Hybrid (No Prior) 0% 60.8 61.1 61.0 61.4 10% 54.8 55.3 55.7 55.9 30% 54.0 54.7 54.3 53.5 50% 50.0 50.4 49.6 49.2 Variant C: Hybrid + Prior (Proposed) 0% 56.4 56.4 55.1 57.5 10% 55.6 55.6 54.5 56.5 30% 55.3 56.0 55.2 54.8 50% 48.2 48.3 48.5 47.2 Turnover-Dependent Ordering Between Hybrid Variants. The two hybrid variants show a turnover-dependent crossover. At zero turnover (ρ=0%ρ=0\%), Hybrid + Prior achieves lower regret across all noise levels (56.456.4–57.557.5 vs. 60.860.8–61.461.4): with a stable pool, the prior’s structural geometry is accurate and reduces unnecessary exploration. At moderate turnover (ρ=10%ρ=10\%–30%30\%), Hybrid (No Prior) takes the lead (54.854.8–55.955.9 vs. 54.554.5–56.556.5), as new contractors diverge from the prior’s initialised geometry. At high turnover (ρ=50%ρ=50\%), Hybrid + Prior recovers its advantage (48.248.2–48.548.5 vs. 49.249.2–50.450.4): the large influx of rested contractors aligns well with the prior’s full-availability geometry, restoring its routing advantage. Both variants substantially outperform TOPSIS (54.554.5–61.261.2) across all conditions, confirming that online learning is the primary driver of robustness under workforce disruption. Notably, 50% turnover rows consistently show lower regret than 0% rows for both variants, because freshly arriving, rested contractors provide routing opportunities that online learners exploit efficiently. Noise Sensitivity. TOPSIS is entirely insensitive to observation noise (all four σ columns are identical), because it scores by fixed contractor attributes rather than observed outcomes. Both hybrid variants show modest noise sensitivity, with changes of less than 2 regret units across σ=0σ=0 to σ=0.20σ=0.20. The prior’s early exploration benefit—visible in Table I—does not amplify noise in this short-horizon regime; sensitivity differences between the two hybrid variants are small and within sampling variation. VII-C Fatigue and Surge Stress Analysis (Experiment 3) When demand spikes and fatigue accumulates faster, which variant better contains burnout while sustaining reward? Tables I and IV report burnout events and mean reward under increasing traffic surge factors (ωsurge _surge) and observation noise (10 repeats/cell). TABLE I: Burnout Events under systemic surge stress with contractor agency enabled (10 repeats/cell). Lower is better. Bold = best per cell (excluding TOPSIS baseline). Surge (ω) / Noise (σ) 0.00 0.10 0.20 TOPSIS (Baseline) 1.0× 28.80 28.80 28.80 1.5× 75.20 75.20 75.20 2.0× 97.40 97.40 97.40 Hybrid (No Prior) 1.0× 28.20 28.60 28.40 1.5× 61.20 62.20 67.60 2.0× 76.20 78.80 87.60 Hybrid + Prior (Proposed) 1.0× 25.80 25.40 26.40 1.5× 68.00 68.40 70.80 2.0× 84.40 86.40 91.00 TABLE IV: Mean Reward under systemic surge stress with contractor agency enabled (10 repeats/cell). Higher is better. Bold = best per cell (excluding TOPSIS baseline). Surge (ω) / Noise (σ) 0.00 0.10 0.20 TOPSIS (Baseline) 1.0× 0.54 0.54 0.54 1.5× 0.41 0.41 0.41 2.0× 0.35 0.35 0.35 Hybrid (No Prior) 1.0× 0.54 0.54 0.54 1.5× 0.46 0.46 0.43 2.0× 0.41 0.41 0.37 Hybrid + Prior (Proposed) 1.0× 0.56 0.56 0.56 1.5× 0.44 0.44 0.43 2.0× 0.39 0.39 0.37 Burnout Under Surge. Both hybrid variants dramatically outperform TOPSIS at baseline surge (1.0×1.0\!×): 25.825.8–28.228.2 burnout events vs. 28.828.8 for TOPSIS, with the gap widening sharply as surge increases. At 2.0×2.0\!× surge, TOPSIS reaches 97.497.4 burnout events; both hybrid variants contain this to 7676–8787, a 1212–22%22\% reduction attributable to online fatigue tracking. Hybrid (No Prior) records fewer burnouts than Hybrid + Prior at every surge–noise cell, with a gap of ≈8≈\!8–1414 events at high surge. This pattern is consistent with the covariance bias mechanism identified in Experiment 1: the prior’s initialisation causes the allocator to revisit a preferred contractor subset more often, and under surge each assignment increments fatigue more aggressively, compounding this bias into additional burnout events. Observation noise has a modest but directional effect on both variants: burnout increases by 55–1111 events between σ=0σ=0 and σ=0.20σ=0.20 at 2.0×2.0\!× surge, as corrupted signals delay the fatigue-aware correction. Reward Under Surge. At baseline surge (1.0×1.0\!×), Hybrid + Prior achieves slightly higher mean reward (0.560.56 vs. 0.540.54), consistent with the advantage observed in Table I. However, this advantage narrows under increasing surge and noise: at 2.0×2.0\!× surge and σ=0.20σ=0.20, both variants converge to the same reward of 0.370.37, and Hybrid (No Prior) equals or exceeds Hybrid + Prior in all other high-surge cells. This indicates that the prior’s reward advantage is conditional on the environment being close to the conditions under which the prior was generated. Summary of Experimental Findings. Across all three experiments a consistent picture emerges. Within the cold-start horizon (T=200T=200), the Physics Prior provides a meaningful advantage: lower regret, higher reward, and tighter variance across seeds in Experiment 1. However, this advantage is contingent on a stable environment close to prior assumptions. Under workforce turnover (Experiment 2), Hybrid (No Prior) is more robust, adapting faster to new contractors without the structural commitment of a pre-warped covariance. Under demand surge (Experiment 3), both variants outperform TOPSIS substantially, but Hybrid (No Prior) better contains burnout when fatigue accumulates rapidly. Taken together, these results support the Physics Prior as a principled cold-start mechanism, while motivating future work on adaptive prior weighting—annealing the prior’s influence as online evidence accumulates—to preserve the early-exploration benefit without the long-horizon rigidity. VIII Conclusion We presented a Neural-Linear UCB allocator for sustainable crowdsourcing that resolves the Cold-Start, Burnout, Utilisation, and Strategic Agency Dilemma through three joint contributions: the FORGE simulator, which formalises the K+1K+1 multi-agent environment in which each contractor declares a fatigue-threshold availability at,kc∈0.5,1.0a^c_t,k∈\0.5,1.0\; a Two-Tower architecture that learns the fatigue–capability–availability relationship from observable context without structural modification; and a Physics-Informed Covariance Prior that provides geometry-aware uncertainty initialisation from episode 1. The method achieves the highest late-stage reward among non-oracle methods (LRew=0.555±0.041LRew=0.555± 0.041) at 7.6% workforce utilisation with early regret 26.40±4.9026.40± 4.90, occupying a Pareto-distinct position on the burnout–reward–utilisation surface that no baseline reaches. The prior transfers robustly across 50% workforce turnover and σ=0.20σ=0.20 observation noise, though a turnover-dependent crossover motivates future work on adaptive prior weighting. Future directions include heterogeneous contractor threshold policies connecting to incentive-compatible mechanism design, formal Whittle indexability conditions for continuous fatigue dynamics under strategic availability, and extension to multi-task simultaneous allocation to test whether the prior geometry generalises across concurrent task streams. References [1] A. Agnihotri, R. Jain, D. Ramachandran, and Z. Wen (2024) Online bandit learning with offline preference data for improved rlhf. arXiv preprint arXiv:2406.09574. Cited by: §I-D. [2] A. A. Alabbadi and M. F. Abulkhair (2021) Multi-objective task scheduling optimization in spatial crowdsourcing. Algorithms 14 (3), p. 77. Cited by: §I-A. [3] P. Auer, N. Cesa-Bianchi, and P. Fischer (2002) Finite-time analysis of the multiarmed bandit problem. Machine learning 47 (2), p. 235–256. Cited by: §I-A. [4] M. Bhatt, R. F. Del Rosario, V. S. Narajala, and I. Habler (2025) Coalesce: economic and security dynamics of skill-based task outsourcing among team of autonomous llm agents. In 2025 Cyber Awareness and Research Symposium (CARS), p. 1–9. Cited by: §I-B, §I-C, §IV. [5] D. Bouneffouf, I. Rish, and C. Aggarwal (2020) Survey on applications of multi-armed and contextual bandits. In 2020 IEEE congress on evolutionary computation (CEC), p. 1–8. Cited by: §I-A. [6] R. Bruns, J. Dötterl, J. Dunkel, and S. Ossowski (2023) Evaluating collaborative and autonomous agents in data-stream-supported coordination of mobile crowdsourcing. Sensors 23 (2), p. 614. Cited by: §I-B. [7] O. Chapelle and L. Li (2011) An empirical evaluation of thompson sampling. Advances in neural information processing systems 24. Cited by: §I-A, §I-A, 3rd item. [8] P. Covington, J. Adams, and E. Sargin (2016) Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, p. 191–198. Cited by: §I-E, §V-A. [9] G. Dulac-Arnold, D. Mankowitz, and T. Hester (2019) Challenges of real-world reinforcement learning. arXiv preprint arXiv:1904.12901. Cited by: §IV. [10] A. Garivier and E. Moulines (2011) On upper-confidence bound policies for switching bandit problems. In International conference on algorithmic learning theory, p. 174–188. Cited by: §I-A, 2nd item. [11] K. D. Glazebrook, D. Ruiz-Hernandez, and C. Kirkbride (2006) Some indexable families of restless bandit problems. Advances in Applied Probability 38 (3), p. 643–672. Cited by: §I-B. [12] W. W. Hager (1989) Updating the inverse of a matrix. SIAM review 31 (2), p. 221–239. Cited by: §V-E. [13] C. Ho and J. Vaughan (2012) Online task assignment in crowdsourcing markets. In Proceedings of the AAAI conference on artificial intelligence, Vol. 26, p. 45–51. Cited by: §I-C. [14] Z. Huiqi, A. Khan, X. Qiang, S. Nazir, Y. Ali, and F. Ali (2021) MCDM approach for assigning task to the workers by selected features based on multiple criteria in crowdsourcing. Scientific Programming 2021 (1), p. 4600764. Cited by: §I-A. [15] H. Jakher, D. Singhvi, and S. Singhvi (2025) Task assignments in distributed supply chains when downtime hurts: a data-driven approach. SSRN Electronic Journal. External Links: Document Cited by: §I-A, §I-B. [16] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra (1998) Planning and acting in partially observable stochastic domains. Artificial intelligence 101 (1-2), p. 99–134. Cited by: §I-A. [17] L. Li, W. Chu, J. Langford, and R. E. Schapire (2010) A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, p. 661–670. Cited by: §I-A, §I-A, 1st item. [18] M. Mahato, N. Kumar, and L. K. Jena (2021) Re-thinking gig economy in conventional workforce post-covid-19: a blended approach for upholding fair balance. Journal of work-applied management 13 (2), p. 261–276. Cited by: §I-A. [19] J. Niño-Mora (2023) Markovian restless bandits and index policies: a review. Mathematics 11 (7), p. 1639. Cited by: §I-B. [20] C. H. Papadimitriou and J. N. Tsitsiklis (1999) The complexity of optimal queuing network control. Mathematics of Operations Research 24 (2), p. 293–305. Cited by: §I-B, §I-B. [21] P. Rashidinejad, B. Zhu, C. Ma, J. Jiao, and S. Russell (2021) Bridging offline reinforcement learning and imitation learning: a tale of pessimism. Advances in Neural Information Processing Systems 34, p. 11702–11716. Cited by: §I-B, §I-D, §V-C. [22] N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p. 3982–3992. Cited by: §I-E, §IV-A. [23] M. Richardson, E. Dominowska, and R. Moore (2007) Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web, p. 521–530. Cited by: §IV-A. [24] C. Riquelme, G. Tucker, and J. Snoek (2018) Deep bayesian bandits showdown: an empirical comparison of bayesian deep networks for thompson sampling. arXiv preprint arXiv:1802.09127. Cited by: §I-A, §V-A. [25] J. Sha, M. Song, G. Sui, H. Sun, and D. Dong (2026) A multi-agent reinforcement learning scheduling algorithm integrating state graph and task graph structural modeling for ride-sharing dispatching. Scientific Reports. Cited by: §I-A. [26] K. T. Talluri and G. J. Van Ryzin (2006) The theory and practice of revenue management. Vol. 68, Springer Science & Business Media. Cited by: §I-B. [27] J. van Remmerden, Z. Bukhsh, and Y. Zhang (2024) Offline reinforcement learning for learning to dispatch for job shop scheduling. Machine Learning. Cited by: §I-B, §I-D. [28] F. L. Vinella, J. Hu, I. Lykourentzou, and J. Masthoff (2022) Crowdsourcing team formation with worker-centered modeling. Frontiers in artificial intelligence 5, p. 818562. Cited by: §I-B. [29] P. Whittle (1988) Restless coin tosses. Journal of Applied Probability 25 (A), p. 287–298. Cited by: §I-B, §I-B, §I-B. [30] S. Xie, X. Wang, B. Yang, L. Li, and J. Yu (2022) Evaluating and visualizing qos of service providers in knowledge-intensive crowdsourcing: a combined mcdm approach. International Journal of Intelligent Computing and Cybernetics 15 (2), p. 198–223. Cited by: §I-A, §I-C, 3rd item. [31] Y. Xu et al. (2023) Incentive mechanism for spatial crowdsourcing with unknown social-aware workers: A three-stage Stackelberg game approach. IEEE Transactions on Mobile Computing 22 (8), p. 4664–4681. External Links: Document Cited by: §I-B. [32] X. Yi, J. Yang, L. Hong, D. Z. Cheng, L. Heldt, A. Kumthekar, Z. Zhao, L. Wei, and E. Chi (2019) Sampling-bias-corrected neural modeling for large corpus item recommendations. In Proceedings of the 13th ACM conference on recommender systems, p. 269–277. Cited by: §I-E. [33] D. Zhou, L. Li, and Q. Gu (2020) Neural contextual bandits with UCB-based exploration. In Proceedings of the 37th International Conference on Machine Learning, p. 11492–11502. Cited by: §I-A.

Instant research discovery