Paper deep dive
Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation
Jianglin Qiao, Siyi Hu, Thien Hoang Nguyen, Zehong Cao, Salah Sukkarieh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/13/2026, 3:41:59 AM
Summary
The paper introduces Staggered Participation Learning (SPL), a training-time augmentation for cooperative Multi-Agent Reinforcement Learning (MARL) that addresses 'staggered participation' (SP), where agents act at different times. SPL uses prospective acquisition supervision for early agents to gather useful information and outcome-supervised receiver learning for later agents to utilize that information. Evaluated across 60 MPE/RWARE settings and a UAV-UGV environment, SPL improves mean task completion by an average of 14.1% compared to vanilla baselines.
Entities (9)
Relation Signals (7)
Staggered Participation → isasettingin → Multi-Agent Reinforcement Learning
confidence 95% · In cooperative Multi-Agent Reinforcement Learning (MARL)... We study this setting as staggered participation (SP)...
Staggered Participation Learning → uses → Outcome-Supervised Receiver Learning
confidence 95% · SPL addresses these two parts with prospective acquisition supervision for earlier agents and outcome-supervised receiver learning for later agents.
Staggered Participation Learning → uses → Prospective Acquisition Supervision
confidence 95% · SPL addresses these two parts with prospective acquisition supervision for earlier agents and outcome-supervised receiver learning for later agents.
Staggered Participation Learning → evaluatedon → Isaac Lab
confidence 92% · The gains also extend to eight-agent teams and a physics-based UAV-UGV environment in Isaac Lab...
Staggered Participation Learning → evaluatedon → MPE
confidence 90% · We evaluate SPL across multiple policy-based MARL backbones... Across 60 MPE/RWARE backbone setting comparisons...
Staggered Participation Learning → evaluatedon → RWARE
confidence 90% · We evaluate SPL across multiple policy-based MARL backbones... Across 60 MPE/RWARE backbone setting comparisons...
Staggered Participation Learning → improves → Task Completion
confidence 90% · SPL achieves higher observed mean task completion in every case, with an average difference of 14.1%.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In cooperative Multi-Agent Reinforcement Learning (MARL), agents are often trained under concurrent participation, while in many tasks some agents act earlier and leave task-relevant information that becomes useful to agents participating later. We study this setting as staggered participation (SP), which introduces a cross-time, cross-agent learning dependency because an early action may affect the return through the information it provides and the later policy that uses it. Learning under SP therefore requires both identifying what information is useful for future decisions and learning how later agents should use it. We propose Staggered Participation Learning (SPL), a training-time augmentation that addresses these two parts with prospective acquisition supervision for earlier agents and outcome-supervised receiver learning for later agents. We evaluate SPL across multiple policy-based MARL backbones, environments, and staggered-participation patterns. Across 60 MPE/RWARE backbone setting comparisons, SPL achieves higher observed mean task completion in every case, with an average difference of 14.1%. The gains also extend to eight-agent teams and a physics-based UAV-UGV environment in Isaac Lab, providing evidence across algorithmic, temporal, and embodied settings.
Tags
Links
- Source: https://arxiv.org/abs/2610.07578v1
- Canonical: https://arxiv.org/abs/2610.07578v1
Trouble viewing inline? Open PDF directly →
Full Text
149,568 characters extracted from source content.
Expand or collapse full text
Cooperating with Future Collaborators: Multi-Agent RL under Staggered Participation Jianglin Qiao Affiliation: ACFR, The University of Sydney, Sydney, Australia Siyi Hu Affiliation: School of EECMS, Curtin University, Bentley, Australia Thien Hoang Nguyen Affiliation: ACFR, The University of Sydney, Sydney, Australia Zehong Cao Affiliation: School of CSIT, Adelaide University, Adelaide, Australia Salah Sukkarieh Affiliation: jianglin.qiao,thienhoang.nguyen,salah.sukkarieh@sydney.edu.au siyi.hu@curtin.edu.au jimmy.cao@adelaide.edu.au Affiliation: ACFR, The University of Sydney, Sydney, Australia Abstract In cooperative Multi-Agent Reinforcement Learning (MARL), agents are often trained under concurrent participation, while in many tasks some agents act earlier and leave task-relevant information that becomes useful to agents participating later. We study this setting as staggered participation (SP), which introduces a cross-time, cross-agent learning dependency because an early action may affect the return through the information it provides and the later policy that uses it. Learning under SP therefore requires both identifying what information is useful for future decisions and learning how later agents should use it. We propose Staggered Participation Learning (SPL), a training-time augmentation that addresses these two parts with prospective acquisition supervision for earlier agents and outcome-supervised receiver learning for later agents. We evaluate SPL across multiple policy-based MARL backbones, environments, and staggered-participation patterns. Across 60 MPE/RWARE backbone setting comparisons, SPL achieves higher observed mean task completion in every case, with an average difference of 14.1%. The gains also extend to eight-agent teams and a physics-based UAV–UGV environment in Isaac Lab, providing evidence across algorithmic, temporal, and embodied settings. keywordsMulti-Agent RL, Staggered Participation, Cooperative Tasks 1 Introduction Cooperative agents do not always participate over the same time interval. For instance, in multi-robot systems, one agent may begin sensing or exploration earlier, while another becomes active later for navigation or task execution [1, 2, 3]. Their participation windows may partially overlap, allowing direct interaction for some time steps, or they may be fully separated. In these cases, earlier agents can acquire or leave task-relevant information that remains useful to agents participating later. We study this temporal structure as staggered participation (SP), where agents contribute to the same cooperative objective over temporally offset participation windows. As illustrated in Fig. 1, these windows may partially overlap or be fully separated, while earlier participants can leave task-relevant information that supports later decisions. Figure 1: Simultaneous and staggered participation. In simultaneous participation, agents contribute within the same participation period. Under staggered participation, earlier participants acquire or leave information that remains available to future collaborators and supports later decisions. Participation windows may partially overlap or be fully separated. Learning under SP is challenging in cooperative MARL because the effect of an early action may only become visible through the behavior of an agent acting later. An earlier agent may take an action that acquires or reveals information for downstream decisions, but the usefulness of that action depends on how the resulting information affects later behavior. The return therefore links decisions across both time and participants. When participation windows overlap, this dependency can coexist with direct interaction; when they are disjoint, it is carried through information and other consequences of earlier actions. This gives rise to two interdependent learning requirements: (i) Information Acquisition, where earlier agents learn which actions provide useful information for later decisions, and (i) Downstream Utilization, where later agents learn how that information should influence action selection. Although a shared team return couples the two roles through a common objective, it does not provide separate supervision for acquiring useful evidence and using it effectively downstream. SP is closely related to three existing directions in cooperative MARL. First, asynchronous MARL studies agents that act or communicate at different times [4, 5, 6]. Second, dynamic and open-team MARL considers agents that join, leave, or interact with changing teammates [7, 8, 9]. Third, communication MARL learns what information is useful for decision-making. Our focus is the learning dependency that appears when information acquisition and downstream utilization are distributed across different participation windows and potentially different agents. SP therefore places these two components within the same cooperative learning problem. We introduce Staggered Participation Learning (SPL), a training-time augmentation that provides complementary supervision for the two learning requirements under SP. Prospective acquisition supervision guides earlier agents toward actions that provide useful information for future collaborators, while outcome-supervised receiver learning trains later agents to use inherited evidence based on downstream outcomes. SPL augments actor training without modifying environment rewards and introduces no additional communication or auxiliary inference at execution. We evaluate SPL across multiple policy-based MARL backbones, participation patterns, larger teams, and a physics-based UAV–UGV setting. Across the 60 primary Vanilla–SPL comparisons, SPL achieves higher observed mean task completion in every configuration, with an average gain of 14.1%. Our contributions are: (i) Staggered participation. We formulate cooperative MARL with temporally offset participation and identify the dependency between evidence acquisition and downstream utilization; (i) Staggered Participation Learning. We introduce complementary supervision for prospective acquisition and outcome-informed evidence use while preserving the underlying reward and decentralized execution; and (i) Empirical evaluation. We evaluate SPL across policy-based MARL backbones, temporal participation patterns, larger teams, and physics-based UAV–UGV simulation. 2 Related Work Simultaneous vs. Staggered Participation in Cooperative MARL. In simultaneous participation, agents contribute within the same interaction period. Cooperative MARL also studies temporal variation through asynchronous interaction, dynamic team composition, and staged decision making. Asynchronous methods consider agents that execute actions, communicate, or assign credit across asynchronous interactions [4, 5, 6, 10, 11], while dynamic and open-team approaches address agents that join, leave, or interact with changing teammates [7, 8, 9, 12]. Cooperative Consecutive Policies studies cooperation across consecutive decision stages [13]. These directions examine different aspects of when agents act and participate in a cooperative task. SP concerns temporally offset participation windows, which may partially overlap or be fully separated. In the tasks studied here, earlier participants can acquire task-relevant evidence before future collaborators begin participating, and this evidence remains available to support downstream decisions. Direct vs. Indirect Communication Protocols in Cooperative MARL. Communication methods support direct information exchange and learn what to send and with whom to communicate, including under intermittent connectivity [14, 15, 16, 17, 18, 19]. Cooperation can also occur indirectly through persistent environmental traces or another agent’s subsequent actions. Stigmergic methods support coordination through such environmental traces [20], while MAE-MAPPO addresses indirect cooperation with invisible collaborators through marginal advantage estimation [21]. Related methods provide complementary mechanisms for retaining and interpreting information: sequence-based methods retain long-horizon context [22], while SMPE2 and MA2E model or reconstruct information unavailable from local observations [23, 24]. Policy distillation transfers training-time coordination knowledge into decentralized policies [25, 26]. Informative sensing and Dec-POMDP approaches further guide information acquisition according to its expected decision value [27, 28, 29]. The form of SP studied here combines temporally offset participation with indirect cooperation through persistent task-relevant evidence. Within this setting, SPL focuses on evidence acquisition and downstream utilization, while related credit-assignment methods study how cooperative outcomes are attributed across agents or coalitions [21, 30]. SPL builds on value-aware information acquisition and policy distillation to supervise earlier participants and learn receiver preferences from attributable downstream outcomes. Persistent evidence connects the two roles without requiring additional execution-time communication. 3 Methodology 3.1 Staggered Participation (SP) Cooperative decision process. We model the cooperative task as a decentralized partially observable Markov decision process (Dec-POMDP) [31], defined by ℳ=⟨,,ii∈,ii∈,P,R,γ⟩M= ,S,\O_i\_i ,\U_i\_i ,P,R,γ , where =1,…,NN=\1,…,N\ is the set of agents and S is the global state space. At time t, the environment is in state st∈s_t and agent i receives a local observation oti∈io_t^i _i. Each agent maintains an action–observation history τti _t^i and selects an action uti∈iu_t^i _i according to πθi(uti∣τti,Mti) _ _i(u_t^i _t^i,M_t^i), where MtiM_t^i denotes the legal-action mask. The joint action t=(ut1,…,utN)u_t=(u_t^1,…,u_t^N) produces the shared reward rt=R(st,t)r_t=R(s_t,u_t) and the next state is sampled according to st+1∼P(⋅∣st,t)s_t+1 P(· s_t,u_t). All agents optimize the common discounted team return G=∑t=0T−1γtrtG= _t=0^T-1γ^tr_t. Staggered participation. Agents may participate over different temporal intervals, so teammates need not be active throughout the same period. Let tpart⊆N^part_t denote the set of agents participating at time t; agents outside this set take a null action and make no policy decision. For each agent i, let [tistart,tiend)[t_i^start,t_i^end) denote its participation interval, where tistartt_i^start is the first participating time and tiendt_i^end is the exclusive end time. We call agent j a future collaborator of agent i if tistart<tjstartt_i^start<t_j^start and both contribute to the same cooperative objective. Their participation intervals may be disjoint, with tiend≤tjstartt_i^end≤ t_j^start, or may partially overlap, with tjstart<tiendt_j^start<t_i^end. In both cases, agent i can act before agent j begins participating. Figure 2: Overview of SP and SPL. Earlier participants receive prospective acquisition supervision, while future collaborators learn evidence-conditioned preferences from downstream outcomes. SPL is used only during training. Learning problem under SP Earlier information-gathering actions can change the evidence available to future collaborators and thereby influence downstream decisions and the shared return. This creates two interdependent learning requirements: what evidence earlier participants should acquire and how future collaborators should use it. Because the effect of an earlier action is realized through persistent evidence and subsequent decisions by another policy, its learning signal is delayed and indirect. We therefore focus on improving both evidence preparation and downstream evidence use. The next subsection introduces SPL to support these two requirements. 3.2 Staggered Participation Learning (SPL) Under SP, earlier participants and future collaborators are coupled through persistent task-relevant evidence. SPL augments the native cooperative MARL objective with two complementary actor-side training signals. Prospective acquisition supervision guides earlier participants toward evidence-gathering actions expected to support future decisions, while outcome-supervised receiver learning teaches future collaborators how inherited evidence should influence downstream action selection. Both augment native actor training while leaving the environment reward and backbone critic objective unchanged, as illustrated in Fig. 2. Structured training supervision. SPL uses structured task knowledge available during training to construct its auxiliary targets. Prospective acquisition supervision uses task-family validity priors, the fixed valid-task constraint, and the probe observation model to estimate the downstream value of candidate evidence-gathering actions. Receiver supervision uses fixed task-level utilities for successful and unsuccessful execution, while candidate-specific success probabilities are learned from attributable outcomes observed during training. These quantities describe task-generation, sensing, and outcome semantics; they do not reveal realized task validity, future outcomes, or a predefined correct action. In particular, the fixed acquisition model does not provide receiver success probabilities. The acquisition and receiver objectives are complementary but separately constructed: the acquisition target is fixed by the structured task model and is not updated from the learned receiver policy or outcome model. SPL therefore assumes that such task structure is available from the training environment or task specification. All SPL-specific models and targets remain training-only and are unavailable during execution. Implementation details are provided in Appendix B.4. 3.2.1 Prospective Acquisition Supervision Prospective evidence value. Consider an eligible evidence-acquisition decision of an earlier participant i. Let ctic_t^i denote its actor-visible acquisition context and MtiM_t^i the corresponding legal-action mask. For each legal task-directed action u, SPL uses a fixed training-time task model to estimate Δti(u) _t^i(u), the expected increase in downstream decision value associated with the evidence produced by that action. SPL combines this prospective value with the corresponding travel or acquisition distance dti(u)d_t^i(u): sti(u)=log(maxΔti(u),ϵacq)−dti(u)τd,s_t^i(u)= \! ( \ _t^i(u), _acq\ )- d_t^i(u) _d, (1) where τd _d is a fixed distance temperature and ϵacq>0 _acq>0 is a numerical stabilizer. A deterministic transform acqT_acq masks illegal actions, normalizes the scores of positive-gain task actions, and reserves a small fixed probability mass for the scan action, which gathers local evidence without targeting a specific candidate. If no positive-gain task action is available, the scan action receives all target mass. The resulting prospective acquisition target is qacqi=acq(sti,Mti)q_acq^i=T_acq(s_t^i,M_t^i). Acquisition policy supervision. For eligible earlier-participant decisions acqD_acq, SPL distils the prospective acquisition target into the decentralized policy: ℒSPLacq=(i,t)∼acq[CE(qacqi,πθi(⋅∣τti,Mti))].L_SPL^acq=E_(i,t) _acq [CE (q_acq^i, _ _i(· _t^i,M_t^i) ) ]. (2) This auxiliary loss updates only the earlier-participant actor. 3.2.2 Outcome-Supervised Receiver Learning Outcome-labeled decisions. Consider a future collaborator j making a downstream decision at time t≥tjstartt≥ t_j^start. Let xtjx_t^j denote the actor-visible information available before the decision, including retained task-relevant evidence and the collaborator’s current local context. The collaborator selects utju_t^j using its decentralized policy, and the selected task subsequently produces an attributable outcome ytj∈0,1y_t^j∈\0,1\, where ytj=1y_t^j=1 denotes successful completion and ytj=0y_t^j=0 an attributable unsuccessful attempt. Each eligible decision contributes a tuple (xtj,utj,j,ytj)(x_t^j,u_t^j,j,y_t^j) to the outcome replay set outD_out. Subsequent observations are used only to determine the outcome label; we exclude interrupted, expired, or unresolved attempts when their outcomes cannot be reliably attributed. Candidate outcome model. SPL learns how the evidence and local context available to a future collaborator relate to the outcome of each candidate action. Let gj(x,u)g_j(x,u) collect the actor-visible decision context, candidate-action features, and collaborator-role identity. A candidate-shared scorer fψf_ψ predicts pψj(u∣x)=sigmoid(fψ(gj(x,u))),p_ψ^\,j(u x)=sigmoid (f_ψ\! (g_j(x,u) ) ), (3) where pψj(u∣x)p_ψ^\,j(u x) estimates the probability that candidate u succeeds under information x. The model is trained from attributable outcomes using binary cross-entropy, ℒout(ψ)=(x,u,j,y)∼out[ℓBCE(y,pψj(u∣x))].L_out(ψ)=E_(x,u,j,y) _out [ _BCE (y,p_ψ^\,j(u x) ) ]. (4) The outcome model is therefore learned from the observed consequences of the future collaborator’s own selected actions. It receives no labels identifying which candidate action should have been selected. Evidence-conditioned action utility. Outcome probability alone does not determine the desirability of a candidate when successful and unsuccessful executions have different task-level consequences. Let cj+(u)c_j^+(u) and cj−(u)c_j^-(u) denote fixed task-level utilities associated with successful and unsuccessful execution of candidate u, respectively. SPL combines these utilities with the evidence-conditioned success probability predicted by a slowly updated target outcome model with parameters ψ¯ ψ: Uψ¯j(u∣x)=pψ¯j(u∣x)cj+(u)+[1−pψ¯j(u∣x)]cj−(u).U_ ψ^\,j(u x)=p_ ψ^\,j(u x)c_j^+(u)+ [1-p_ ψ^\,j(u x) ]c_j^-(u). (5) Because pψ¯j(u∣x)p_ ψ^\,j(u x) is conditioned on decision-time information, the same candidate can receive different utility estimates under different evidence prepared by earlier participants. Receiver target and policy supervision. Let tj=u∈j:Mtj(u)=1A_t^j=\u _j:M_t^j(u)=1\ denote the legal actions available to collaborator j. SPL standardizes the predicted utilities over this set and constructs a soft preference target: ztj(u)=Uψ¯j(u∣xtj)−μtj(σtj)2+ϵz,qSPLj(u)=exp(βztj(u))∑v∈tjexp(βztj(v)),z_t^j(u)= U_ ψ^\,j(u x_t^j)- _t^j ( _t^j)^2+ _z, q_SPL^j(u)= (β z_t^j(u)) _v _t^j (β z_t^j(v)), (6) where μtj _t^j and (σtj)2( _t^j)^2 are the mean and variance of the legal-action utilities, ϵz>0 _z>0 is a variance stabilizer, and β>0β>0 controls the strength of the induced preference. Illegal actions receive zero probability. Since the target is constructed from learned evidence-conditioned outcome predictions, it expresses relative preferences among the currently legal actions and does not require predefined correct-action labels. For eligible future-collaborator decision points recD_rec, SPL distills this soft target into the decentralized policy: ℒSPLrec=(j,t)∼rec[CE(qSPLj,πθj(⋅∣τtj,Mtj))].L_SPL^rec=E_(j,t) _rec [CE (q_SPL^j, _ _j(· _t^j,M_t^j) ) ]. (7) The target is treated as fixed during this update. Gradients therefore modify only the future-collaborator actor and do not propagate through the outcome model, recorded outcomes, or environment dynamics. Training and Execution. SPL combines ℒSPLacqL_SPL^acq for earlier participants with ℒSPLrecL_SPL^rec for future collaborators, while the MARL backbone continues to optimize its native objective from the shared team return. After each rollout, attributable outcomes are added to outD_out and used to update the outcome model once replay-readiness conditions are met; the target parameters are updated by ψ¯←(1−ρ)ψ¯+ρψ ψ←(1-ρ) ψ+ρψ. SPL actor updates are interleaved with native actor optimization, while critic optimization follows the original backbone procedure. During execution, agents act solely through uti∼πθi(⋅∣τti,Mti)u_t^i _ _i(· _t^i,M_t^i); no acquisition model, auxiliary target, outcome replay, outcome model, or utility estimate is required. We provide detailed readiness conditions, update ordering, gradient boundaries, and the complete training algorithm in Appendices B.6 and B.7. 3.3 Theoretical Analysis We provide a local analysis of the auxiliary objectives introduced by SPL. The results characterize the prospective acquisition target, calibration of the outcome-based receiver model, and local decision quality induced by the receiver target under the structured task assumptions of Sec. 3.2. The acquisition result instantiates a standard value-of-information argument for the top-k surrogate used by SPL, while the receiver results characterize calibration and distillation errors. Together, these results support the construction of the SPL training signals but do not form an end-to-end guarantee or establish global convergence of the cooperative return. Full proofs, assumptions, and guarantee boundaries are provided in Appendix A. Prospective acquisition. Consider task families represented by the fixed binary-validity model used to construct acquisition supervision. Let b=(b1,…,bn)b=(b_1,…,b_n) denote posterior validity marginals under this model, where br=Pr(Wr=1∣cti)b_r= (W_r=1 c_t^i). If a future collaborator can execute at most k tasks, define the top-k posterior completion value as Vk(b)=maxS:|S|≤k∑r∈SbrV_k(b)= _S:|S|≤ k _r∈ Sb_r. For a purely informational acquisition action u, let ZuZ_u denote the resulting observation and bZub^Z_u the updated posterior. Its model-relative prospective gain is Δti(u)=[Vk(bZu)∣cti,u]−Vk(b) _t^i(u)=E[V_k(b^Z_u) c_t^i,u]-V_k(b). Let ti=u:Mti(u)=1,u is task-directed,Δti(u)>0P_t^i=\u:M_t^i(u)=1,\;u is task-directed,\; _t^i(u)>0\ denote the legal task-directed actions with positive prospective gain. Proposition 1 (Prospective acquisition target). Assume that acquisition actions change only the available information and that posterior updating and predictive expectation are Bayesian-consistent under the acquisition model. Then Δti(u)≥0 _t^i(u)≥ 0 for every legal acquisition action. Moreover, before adding the scan mass, if acqT_acq applies Gibbs normalization over tiP_t^i, the acquisition target satisfies qacqi(u)∝exp(sti(u))q_acq^i(u) (s_t^i(u)) and uniquely maximizes the corresponding entropy-regularized score objective. Proof sketch. The top-k value VkV_k is the maximum of linear functions of the posterior marginals and is therefore convex. Bayesian consistency gives [bZu∣cti,u]=bE[b^Z_u c_t^i,u]=b, so Jensen’s inequality implies [Vk(bZu)∣cti,u]≥Vk(b)E[V_k(b^Z_u) c_t^i,u]≥ V_k(b). The Gibbs form follows from the standard entropy-regularized maximization problem. The full proof is provided in Appendix A.2. Outcome prediction and local utility. For an attributable future-collaborator decision, let Y∈0,1Y∈\0,1\ denote its downstream outcome and let C∈0,1C∈\0,1\ indicate whether that outcome is retained in outD_out. We define p∗j(u∣x)=Pr(Y=1∣X=x,U=u,J=j,C=1)p_*^\,j(u x)= (Y=1 X=x,U=u,J=j,C=1), with corresponding local utility U∗j(u∣x)=p∗j(u∣x)cj+(u)+[1−p∗j(u∣x)]cj−(u)U_*^\,j(u x)=p_*^\,j(u x)c_j^+(u)+[1-p_*^\,j(u x)]c_j^-(u). For any predictor p, let ℒout(p)L_out(p) denote its population binary-cross-entropy risk over retained outcomes and UpjU_p^j the induced local utility. Proposition 2 (Retained-outcome calibration). On the covered retained-outcome support, ℒout(p)L_out(p) is uniquely minimized pointwise by p∗jp_*^\,j. If CΔ=supu,j|cj+(u)−cj−(u)|<∞C_ = _u,j|c_j^+(u)-c_j^-(u)|<∞, then [(UpJ(U∣X)−U∗J(U∣X))2∣C=1]≤CΔ22[ℒout(p)−ℒout(p∗)].E\! [ (U_p^J(U X)-U_*^J(U X) )^2 C=1 ]≤ C_ ^22 [L_out(p)-L_out(p_*) ]. (8) Proof sketch. Excess binary-cross-entropy risk equals the Bernoulli KL divergence on the retained-outcome distribution. Since Upj−U∗j=(pj−p∗j)(cj+−cj−)U_p^j-U_*^j=(p^j-p_*^j)(c_j^+-c_j^-), Pinsker’s inequality gives Eq. 8. The result applies to retained, selected receiver actions and is conditional on C=1C=1; an unconditional interpretation additionally requires Y⟂C|X,U,JY C X,U,J. The full proof is provided in Appendix A.3. Receiver decision quality. Fix an eligible future-collaborator decision with legal action set A and K=||K=|A|. Let z∗z_* denote the reference standardized score and zψ¯z_ ψ the score produced by the target outcome model. Define δ=‖zψ¯−z∗‖∞δ=\|z_ ψ-z_*\|_∞, Rz=maxu∈z∗(u)−minu∈z∗(u)R_z= _u z_*(u)- _u z_*(u), and η=KL(qSPLj∥πtj)=CE(qSPLj,πtj)−H(qSPLj),η=KL(q_SPL^j\| _t^j)=CE(q_SPL^j, _t^j)-H(q_SPL^j), where qSPLj(u)∝exp(βzψ¯(u))q_SPL^j(u) (β z_ ψ(u)). Thus, η is directly measurable from the receiver target and actor distributions. The score error δ additionally depends on generalization across the full legal action set and score standardization; controlling it from outcome-model error requires stronger coverage conditions. Theorem 1 (Local receiver score regret). For any legal receiver policy πtj _t^j satisfying KL(qSPLj∥πtj)<∞KL(q_SPL^j\| _t^j)<∞, Regz,tj(πtj)≤logKβ+2δ+Rzη2,Reg_z,t^j( _t^j)≤ Kβ+2δ+R_z η2, (9) where Regz,tj(πtj)=maxu∈z∗(u)−u∼πtj[z∗(u)]Reg_z,t^j( _t^j)= _u z_*(u)-E_u _t^j[z_*(u)]. Proof sketch. The Boltzmann target contributes a softness gap of at most logK/β K/β. Replacing zψ¯z_ ψ with z∗z_* contributes at most 2δ2δ, and Pinsker’s inequality bounds the remaining distillation error by Rzη/2R_z η/2. The three terms therefore capture finite-temperature softness, score approximation, and incomplete receiver distillation. The full proof is provided in Appendix A.5. Figure 3: Evaluation environments and temporal participation patterns. (a) MPE, (b) RWARE, (c) the physics-based UAV–UGV environment in Isaac Lab, and (d) the five participation patterns used in MPE and RWARE. Scope. The results characterize local components of SPL under the structured training assumptions of Sec. 3.2. Proposition 1 gives a model-relative information-value guarantee: internally consistent Bayesian updating preserves non-negative prospective gain, but misspecified priors can change how this gain relates to performance under the true environment distribution; this sensitivity is evaluated in Appendix D.6. Proposition 2 controls expected local-utility error on the retained selected-action distribution. Theorem 1 separates the measurable distillation error η from the score-approximation term δ and bounds standardized local-score regret, not raw utility or full team return. Relating δ directly to outcome-model training requires stronger coverage and generalization assumptions across legal actions. Figure 4: Training TC (%) for all methods across 4 environments and 5 participation patterns averaged over 5 training seeds. 4 Experimental Setup Environments. We evaluate SPL in MPE, RWARE, and a physics-based UAV–UGV environment, where earlier participants gather task-relevant evidence before future collaborators perform downstream tasks. MPE-Base builds on the Multi-Agent Particle Environment [32], while MPE-Advanced further randomizes task assignments, spawn positions, and future-collaborator identities. RWARE-Base transfers the same evidence-mediated cooperation problem to the Multi-Robot Warehouse domain [33], and RWARE-Advanced additionally introduces task permutation, a shared candidate pool, and dynamic task claims. Isaac Lab [34] provides an embodied UAV–UGV setting with physical motion and obstacle constraints. Full environment specifications are given in Appendices B.1–B.3. For MPE and RWARE, we evaluate five patterns: Non-overlap, with a strict handoff between roles; Overlap-25 and Overlap-50, with fixed temporal overlap; Dynamic, with stochastic pair-specific handoff times; and Cascaded, with successive participation intervals. Isaac Lab uses corresponding UAV–UGV schedules. Exact participation windows are provided in Appendices B.2–B.3. Implementation We augment three PPO-based MARL backbones—IPPO, MAPPO, and HAPPO—to obtain SPL-IPPO, SPL-MAPPO, and SPL-HAPPO. MAPPO [35] is the primary backbone, while HAPPO [36] and IPPO [37] test compatibility with heterogeneous and independent learning, respectively. We additionally compare with MAE-MAPPO [21], Sable [22], and SMPE2 [23] as complementary cross-method references. The primary controlled comparisons remain between each Vanilla backbone and its SPL-augmented counterpart. Baseline adaptations and implementation details are provided in Appendix B.5. Our implementation builds on HARL [38]. SPL preserves the native actor–critic architecture and training objective of each backbone and adds only role-specific auxiliary actor updates during training; SPL-specific models are not required during execution. Optimization details, training procedures, and observed training costs are reported in Appendices B–C. Evaluation. Task completion (TC) is the primary metric. The four-agent MPE/RWARE experiments use five independent training seeds per method and configuration, while the eight-agent and Isaac Lab evaluations use three; all final evaluations contain 200 episodes per seed. Results are reported as mean ± standard error across training seeds. Throughout the results, reported TC gains denote absolute differences between percentage-valued TC scores and are written using %. The primary Vanilla–SPL comparisons are additionally examined using paired seed-level analyses where same-seed pairing is available; complete confidence intervals, Holm-adjusted tests, and pairing details are reported in Appendix D.7. Additional metrics, including full success, return, failed execution, and exploration diagnostics, are reported in Appendix D. All configurations are trained and evaluated independently, so the reported results characterize within-configuration performance rather than transfer across participation patterns or task structures. Table 1: Final task completion (TC, %) after 2M environment steps. Values are mean ± standard error over five training seeds. SPL columns are shaded; bold indicates the highest observed mean in each setting and does not denote statistical significance. Environment Participation IPPO MAPPO HAPPO Other baselines Van. SPL Van. SPL Van. SPL MAE-MAPPO Sable SMPE2 MPE-Base Non-overlap 70.50±0.0070.50± 0.00 78.38±0.3878.38± 0.38 70.50±0.0070.50± 0.00 78.73±0.4378.73± 0.43 67.45±3.0567.45± 3.05 79.18±0.2379.18± 0.23 70.50±0.0070.50± 0.00 62.70±0.2562.70± 0.25 42.73±9.2942.73± 9.29 Overlap-25 68.25±2.7568.25± 2.75 79.35±0.5779.35± 0.57 70.78±0.1470.78± 0.14 80.25±0.2180.25± 0.21 70.70±0.1870.70± 0.18 80.35±0.2980.35± 0.29 67.93±2.9567.93± 2.95 63.50±2.2463.50± 2.24 53.85±5.4353.85± 5.43 Overlap-50 67.85±3.1567.85± 3.15 78.90±0.3178.90± 0.31 69.53±1.2769.53± 1.27 78.90±0.2478.90± 0.24 69.80±0.9269.80± 0.92 78.50±0.4778.50± 0.47 70.63±0.2370.63± 0.23 64.43±1.8564.43± 1.85 63.95±3.3463.95± 3.34 Dynamic 66.20±3.0166.20± 3.01 79.65±0.1879.65± 0.18 70.23±0.5970.23± 0.59 78.88±0.1878.88± 0.18 69.53±0.9969.53± 0.99 79.63±0.3979.63± 0.39 64.85±3.0664.85± 3.06 58.30±1.1158.30± 1.11 49.70±1.0449.70± 1.04 Cascaded 66.83±3.6866.83± 3.68 78.90±0.7078.90± 0.70 70.63±0.1070.63± 0.10 78.93±0.4878.93± 0.48 70.68±0.1970.68± 0.19 78.78±0.2278.78± 0.22 70.45±0.0570.45± 0.05 38.03±0.9338.03± 0.93 27.93±5.7427.93± 5.74 MPE-Advanced Non-overlap 70.38±0.0070.38± 0.00 77.95±0.1977.95± 0.19 64.08±3.8664.08± 3.86 78.23±0.1678.23± 0.16 67.28±3.1067.28± 3.10 78.03±0.1678.03± 0.16 63.28±3.4763.28± 3.47 64.90±1.3664.90± 1.36 49.78±3.2549.78± 3.25 Overlap-25 65.65±3.5365.65± 3.53 79.95±0.5279.95± 0.52 68.28±1.5068.28± 1.50 79.38±0.2779.38± 0.27 70.93±0.5670.93± 0.56 79.55±0.4079.55± 0.40 71.40±0.2371.40± 0.23 64.30±1.6364.30± 1.63 47.83±9.0247.83± 9.02 Overlap-50 71.30±0.2271.30± 0.22 78.63±0.2378.63± 0.23 68.58±3.0568.58± 3.05 78.33±0.4078.33± 0.40 71.40±0.1471.40± 0.14 78.68±0.2578.68± 0.25 62.63±5.9262.63± 5.92 61.88±1.4861.88± 1.48 52.08±8.2552.08± 8.25 Dynamic 66.63±3.6666.63± 3.66 78.23±0.2978.23± 0.29 67.05±3.2067.05± 3.20 79.08±0.5479.08± 0.54 61.73±5.5561.73± 5.55 78.80±0.1878.80± 0.18 67.68±1.7667.68± 1.76 57.60±1.3057.60± 1.30 43.10±6.6143.10± 6.61 Cascaded 67.68±2.5867.68± 2.58 76.80±0.1376.80± 0.13 67.28±3.1067.28± 3.10 77.00±0.3377.00± 0.33 69.28±1.1069.28± 1.10 77.23±0.1877.23± 0.18 66.83±3.0266.83± 3.02 62.73±0.9862.73± 0.98 56.70±5.5056.70± 5.50 RWARE-Base Non-overlap 66.45±5.8666.45± 5.86 86.83±0.4086.83± 0.40 71.38±2.2771.38± 2.27 86.70±0.4986.70± 0.49 73.18±1.5073.18± 1.50 86.93±0.2686.93± 0.26 73.48±1.6973.48± 1.69 71.80±0.9871.80± 0.98 66.05±3.2966.05± 3.29 Overlap-25 81.10±3.4181.10± 3.41 89.15±0.1689.15± 0.16 74.25±3.4174.25± 3.41 89.08±0.3589.08± 0.35 70.15±2.9370.15± 2.93 88.35±0.5088.35± 0.50 72.50±2.8672.50± 2.86 76.70±1.1876.70± 1.18 67.28±2.8767.28± 2.87 Overlap-50 82.00±1.1282.00± 1.12 89.00±0.1989.00± 0.19 74.73±3.3574.73± 3.35 88.58±0.2888.58± 0.28 70.28±2.4170.28± 2.41 88.80±0.2388.80± 0.23 77.53±0.7277.53± 0.72 77.23±1.9777.23± 1.97 61.95±4.0961.95± 4.09 Dynamic 72.88±1.1972.88± 1.19 88.88±0.2788.88± 0.27 65.35±5.4665.35± 5.46 88.70±0.3088.70± 0.30 68.80±4.0668.80± 4.06 88.40±0.4288.40± 0.42 72.28±1.5172.28± 1.51 74.63±1.2274.63± 1.22 62.03±4.2962.03± 4.29 Cascaded 66.80±1.9866.80± 1.98 86.25±0.3886.25± 0.38 67.53±2.0567.53± 2.05 86.75±0.5086.75± 0.50 66.58±2.2466.58± 2.24 85.95±0.7985.95± 0.79 71.63±0.7371.63± 0.73 76.83±0.9376.83± 0.93 64.18±2.9464.18± 2.94 RWARE-Advanced Non-overlap 52.08±1.3752.08± 1.37 79.93±0.5979.93± 0.59 58.63±2.3158.63± 2.31 80.03±0.2380.03± 0.23 57.58±3.4457.58± 3.44 79.55±0.7979.55± 0.79 58.50±2.2058.50± 2.20 50.70±0.3250.70± 0.32 45.70±0.2545.70± 0.25 Overlap-25 64.18±1.6264.18± 1.62 83.23±0.4783.23± 0.47 68.83±1.4068.83± 1.40 83.65±0.3283.65± 0.32 63.28±1.9863.28± 1.98 83.73±0.4883.73± 0.48 64.70±1.8164.70± 1.81 54.75±0.4454.75± 0.44 47.68±0.6647.68± 0.66 Overlap-50 70.35±1.2870.35± 1.28 84.55±0.6984.55± 0.69 68.05±4.1468.05± 4.14 84.45±0.3384.45± 0.33 67.50±2.1267.50± 2.12 84.43±0.5684.43± 0.56 72.88±1.0572.88± 1.05 58.88±0.7758.88± 0.77 47.05±0.3847.05± 0.38 Dynamic 66.60±1.9566.60± 1.95 81.30±0.3481.30± 0.34 62.55±2.8362.55± 2.83 82.50±0.4382.50± 0.43 63.23±0.8363.23± 0.83 82.20±0.1682.20± 0.16 65.90±2.3365.90± 2.33 56.30±0.4956.30± 0.49 47.55±0.4047.55± 0.40 Cascaded 58.95±1.7058.95± 1.70 85.40±0.4285.40± 0.42 65.55±1.7665.55± 1.76 85.98±0.4485.98± 0.44 66.03±2.0666.03± 2.06 84.90±0.5184.90± 0.51 64.18±3.9864.18± 3.98 54.90±1.0854.90± 1.08 47.03±0.5547.03± 0.55 5 Results 5.1 Main Results Fig. 4 shows the training task-completion trajectories across the four environments and five participation patterns. In most settings, the SPL variants reach a higher task-completion regime than their corresponding Vanilla backbones and maintain this difference through 2M environment steps. Although the Vanilla IPPO, MAPPO, and HAPPO trajectories differ, their SPL counterparts often converge to similar final performance, suggesting that the observed improvement is not specific to a single PPO-based actor–critic backbone. Table 1 reports the final results for the 60 primary Vanilla–SPL comparisons. SPL achieves higher observed mean TC in every configuration, with an equal-weight descriptive mean difference of 14.09%. The corresponding mean differences are 13.93%, 14.02%, and 14.33% for IPPO, MAPPO, and HAPPO, respectively. Among the 57 configurations with paired inference, 56 pointwise 95% confidence intervals have positive lower bounds, and 29 remain significant after Holm correction over the declared family of 60 comparisons at a familywise level of 0.05. Complete statistical results are reported in Appendix D.7, while full TC, FS, return, and FE results are provided in Appendices D.1–D.4. Participation structure affects the magnitude of the observed improvement. Averaged across environments and backbones, SPL improves task completion by 15.1% under Non-overlap, 13.3% under Overlap-25, 11.7% under Overlap-50, 15.5% under Dynamic, and 14.9% under Cascaded participation. Across the fixed Non-overlap, Overlap-25, and Overlap-50 schedules, the average gain decreases as cross-role concurrency increases. Since the exact temporal constructions differ between MPE and RWARE, we treat this pattern as descriptive and do not interpret it as a controlled effect of overlap. The positive gains under both overlap settings nevertheless show that SPL does not require a strictly sequential handoff. Improvements under Dynamic and Cascaded participation further show that the observed benefit extends to stochastic and successively staged participation schedules. We additionally evaluate SPL-MAPPO in eight-agent MPE-Advanced and RWARE-Advanced settings. As shown in Table 2, SPL-MAPPO achieves higher mean task completion than MAPPO in all six matched comparisons, with an average observed gain of 17.1%. The gains range from 3.3% in MPE-Advanced Cascaded to more than 30% in the Non-overlap and Dynamic RWARE-Advanced settings. Non-overlap and Dynamic use 2M environment steps, whereas Cascaded uses 4M to accommodate its longer participation horizon; differences across participation patterns should therefore not be interpreted as controlled comparisons of training budget. SPL introduces no additional models during execution. Its training-time computational cost, however, varies across environments and team sizes: the overhead is small in several four-agent settings but becomes more pronounced in some larger-scale configurations. Detailed wall-clock measurements are reported in Appendix C. Table 2: Task completion (%) for eight-agent scaling and physics-based UAV–UGV evaluations using MAPPO. MPE-Advanced (8) RWARE-Advanced (8) Participation Vanilla SPL Vanilla SPL Non-overlap 64.21±0.6164.21± 0.61 77.77±0.3077.77± 0.30 41.10±2.6641.10± 2.66 74.38±1.2474.38± 1.24 Dynamic 68.60±2.6468.60± 2.64 77.75±0.2077.75± 0.20 44.81±1.5744.81± 1.57 76.15±0.8876.15± 0.88 Cascaded 71.79±0.2771.79± 0.27 75.13±0.1475.13± 0.14 19.73±0.7019.73± 0.70 31.75±1.0631.75± 1.06 1 UAV + 1 UGV 2 UAVs + 2 UGVs Participation Vanilla SPL Vanilla SPL Non-overlap 31.50±5.9731.50± 5.97 61.58±0.6761.58± 0.67 55.79±3.2355.79± 3.23 76.38±1.5276.38± 1.52 Dynamic 36.67±0.7136.67± 0.71 59.58±1.7959.58± 1.79 58.46±3.4058.46± 3.40 79.25±1.3079.25± 1.30 5.2 Physics-Based UAV–UGV Evaluation We further evaluate SPL in a physics-based UAV–UGV setting in Isaac Lab, where earlier UAVs navigate to sensing locations, and later UGVs navigate to and execute downstream tasks. This introduces embodied motion and obstacle constraints on both evidence acquisition and downstream execution. As shown in Table 2, SPL-MAPPO achieves higher mean task completion than MAPPO in all four team-size–participation settings. The gains range from 20.6% to 30.1% across both team sizes and occur under both Non-overlap and Dynamic participation. Attempt-level diagnostics show more resolved executions and successful completions with SPL-MAPPO, together with a lower observed failure rate per resolved attempt, while map and published evidence coverage vary with team size. These results provide complementary evidence that the observed SPL improvement persists when evidence acquisition and downstream task execution are coupled to embodied motion. Execution, coverage, and safety diagnostics are reported in Appendix D.5. Figure 5: Isaac Lab case study comparing Vanilla and SPL based on MAPPO. Qualitative case study. Figure 5 compares MAPPO and SPL-MAPPO on the same deterministic fixed-handoff rollout in the one-UAV–one-UGV Isaac setting. The hidden ground truth, shown only for interpretation, contains two decoy tasks (Tasks 1 and 3) and two serviceable tasks (Tasks 2 and 4). All candidate locations are visible to the agents, while task serviceability must be inferred from acquired evidence. The UAV acts during the first 200 steps of the 600-step episode, after which the UGV acts using the persistent shared board. In the trajectory panels, triangles and squares mark UAV and UGV starting positions, circles mark trajectory endpoints, plus/minus symbols denote published task cues, and check/cross symbols denote UGV service outcomes; trajectory color darkens with time. At handoff, both methods have mapped 48 of 144 occupancy cells, but their task-specific evidence differs substantially. MAPPO publishes no task-quality cues, after which its UGV attempts decoy Task 3 at step 453 and completes serviceable Task 2 at step 577, leaving Task 4 unserved. SPL-MAPPO instead acquires cues for Tasks 2, 3, and 4, which are published to the shared board after the five-step delay: positive for Tasks 2 and 4 and negative for Task 3. Its UGV then completes Task 4 at step 438 and Task 2 at step 484 without attempting a decoy. This example makes the cross-stage information pathway explicit: similar spatial coverage can coexist with different task-specific evidence and, consequently, different downstream decisions. 5.3 Ablation Study We evaluate SPL-MAPPO under Non-overlap participation to examine the contribution of its two training components, the role of inherited evidence in receiver learning, and the dependence on structured training information. Acquisition-only and Receiver-only retain only the corresponding SPL component. Evidence-redacted removes inherited task evidence consistently from both outcome-model inputs and receiver-target construction, whereas SKD-MAPPO retains full evidence during outcome-model training but masks predecessor-generated evidence when constructing receiver targets. Outcome-agnostic fixes candidate success probabilities to 0.50.5, while Permuted assigns predicted utilities to incorrect legal actions while preserving the receiver-update budget. Table 3 shows that full SPL achieves the highest observed mean task completion in all four environments. Relative to full SPL, Acquisition-only and Receiver-only reduce mean task completion by 10.7% and 7.7% on average, respectively. Neither component alone reproduces the complete method, supporting the complementary roles of evidence preparation and downstream evidence use. Table 3: SPL ablations under Non-overlap. Variant MPE-Base MPE-Adv. RWARE-Base RWARE-Adv. Full SPL 78.73±0.4378.73± 0.43 78.23±0.1678.23± 0.16 86.70±0.4986.70± 0.49 80.03±0.2380.03± 0.23 SKD-MAPPO 70.50±0.0070.50± 0.00 70.38±0.0070.38± 0.00 70.75±0.0070.75± 0.00 67.63±0.2367.63± 0.23 Acquisition-only 70.50±0.0070.50± 0.00 69.88±0.5069.88± 0.50 77.80±2.3777.80± 2.37 62.83±1.9862.83± 1.98 Receiver-only 73.45±1.2973.45± 1.29 72.28±0.9472.28± 0.94 74.40±1.9474.40± 1.94 72.90±1.0372.90± 1.03 Evidence-redacted 70.50±0.0070.50± 0.00 70.38±0.0070.38± 0.00 70.75±0.0070.75± 0.00 69.13±0.2169.13± 0.21 Outcome-agnostic 70.50±0.0070.50± 0.00 70.38±0.0070.38± 0.00 62.33±2.2762.33± 2.27 48.48±0.8048.48± 0.80 Permuted 37.13±0.3737.13± 0.37 37.05±0.8737.05± 0.87 26.80±2.7626.80± 2.76 28.95±0.8428.95± 0.84 SKD-MAPPO also remains below full SPL in every environment, with an average gap of 11.1%. Thus, matching SPL’s training-side structured information and auxiliary machinery without evidence-conditioned receiver supervision does not reproduce the full SPL performance. We further test sensitivity to misspecified validity priors while keeping the true environment distribution unchanged. Even with fully reversed priors (λ=1λ=1), SPL-MAPPO retains 76.73%76.73\% and 77.45%77.45\% TC in MPE-Advanced and RWARE-Advanced, compared with nominal values of 78.23%78.23\% and 80.03%80.03\%, respectively. Full success is more sensitive, particularly in RWARE-Advanced; complete results are reported in Appendix D.6. These results show that SPL retains substantial task-completion performance under validity-prior misspecification in the tested settings. Other forms of structured-model error remain outside the scope of this experiment. Receiver-target ablations further show that the information used to construct the target matters. Evidence-redacted, Outcome-agnostic, and Permuted reduce mean task completion by 10.7%, 18.0%, and 48.4% on average, respectively. The strong degradation under Permuted, despite preserving the receiver-update budget, shows that the correspondence between candidate actions and their predicted utilities is important. Preserving the auxiliary update budget alone is insufficient to recover the full SPL performance. 6 Conclusion We studied cooperative MARL under staggered participation and introduced SPL, which combines prospective evidence acquisition with downstream evidence use. SPL achieves higher observed mean task completion across multiple policy-based backbones, participation patterns, larger teams, and physics-based UAV–UGV simulation. These results support explicitly addressing both information acquisition and downstream utilization when agents contribute over staggered participation windows. References [1] F. M. Rockenbauer, J. Lim, M. G. Müller, R. Siegwart, and L. Schmid (2024) Traversing mars: cooperative informative path planning to efficiently navigate unknown scenes. IEEE Robotics and Automation Letters. Cited by: §1. [2] T. Sasaki, K. Otsu, R. Thakker, S. Haesaert, and A. Agha-mohammadi (2020) Where to map? iterative rover-copter path planning for mars exploration. IEEE Robotics and Automation Letters. Cited by: §1. [3] S. Zhang, X. Zhang, T. Li, J. Yuan, and Y. Fang (2022) Fast active aerial exploration for traversable path finding of ground robots in unknown environments. IEEE Transactions on Instrumentation and Measurement. Cited by: §1. [4] C. Yu, X. Yang, J. Gao, J. Chen, Y. Li, J. Liu, Y. Xiang, R. Huang, H. Yang, Y. Wu, and Y. Wang (2023) Asynchronous multi-agent reinforcement learning for efficient real-time multi-robot cooperative exploration. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems, Cited by: §1, §2. [5] Y. Min, J. He, T. Wang, and Q. Gu (2023) Cooperative multi-agent reinforcement learning: asynchronous communication and linear function approximation. In International Conference on Machine Learning, Cited by: §1, §2. [6] S. Dolan, S. Nayak, J. J. Aloor, and H. Balakrishnan (2025) Asynchronous cooperative multi-agent reinforcement learning with limited communication. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, Cited by: §1, §2. [7] B. Liu, Q. Liu, P. Stone, A. Garg, Y. Zhu, and A. Anandkumar (2021) Coach-player multi-agent reinforcement learning for dynamic team composition. In International Conference on Machine Learning, Cited by: §1, §2. [8] A. Rahman, I. Carlucho, N. Höpner, and S. V. Albrecht (2023) A general learning framework for open ad hoc teamwork using graph-based policy learning. Journal of Machine Learning Research. Cited by: §1, §2. [9] C. Wang (2024) N-agent ad hoc teamwork. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, Cited by: §1, §2. [10] Y. Liang, H. Wu, H. Wang, and H. Cai (2025) Asynchronous credit assignment for multi-agent reinforcement learning. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, IJCAI ’25. Cited by: §2. [11] M. Farjadnasab and S. Sirouspour (2025) Cooperative and asynchronous transformer-based mission planning for heterogeneous teams of mobile robots. Robotics and Autonomous Systems, p. 105131. Cited by: §2. [12] S. Hu, M. A. Hady, J. Qiao, J. Cao, M. Pratama, and R. Kowalczyk (2025) Adaptability in multi-agent reinforcement learning: a framework and unified review. arXiv preprint arXiv:2507.10142. Cited by: §2. [13] J. Erskine and C. Lehnert (2022) Developing cooperative policies for multi-stage reinforcement learning tasks. IEEE Robotics and Automation Letters. Cited by: §2. [14] C. Zhu, M. Dastani, and S. Wang (2024) A survey of multi-agent deep reinforcement learning with communication. Autonomous Agents and Multi-Agent Systems 38 (1), p. 4. Cited by: §2. [15] S. Hu, L. Shen, Y. Zhang, and D. Tao (2024) Learning multi-agent communication from graph modeling perspective. In International Conference on Learning Representations, Cited by: §2. [16] X. Li and J. Zhang (2024) Context-aware communication for multi-agent reinforcement learning. Proceedings of the 2024 International Conference on Autonomous Agents and Multiagent Systems. Cited by: §2. [17] M. Yang, K. Zhao, Y. Wang, R. Dong, Y. Du, F. Liu, M. Zhou, and L. H. U (2024) Team-wise effective communication in multi-agent reinforcement learning. Autonomous Agents and Multi-Agent Systems. Cited by: §2. [18] Y. Lin, J. Zhang, and X. Zeng (2026) Why and whom to communicate? a dual-objective, cost-benefit framework for multi-agent communication. In Proc. of the 25th International Conference on Autonomous Agents and Multiagent Systems, Cited by: §2. [19] E. Yao, Q. Liu, J. Sun, K. Huang, and M. Xu (2026) DIRECT: decentralized intention and latent rule emergence for multi-agent cooperation under intermittent communication. In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence, IJCAI-26, p. 437–445. Cited by: §2. [20] X. Xu, R. Li, Z. Zhao, and H. Zhang (2021) Stigmergic independent reinforcement learning for multiagent collaboration. IEEE transactions on neural networks and learning systems. Cited by: §2. [21] J. Qiao, Z. Cao, S. Hu, M. Fan, M. Pratama, and R. Kowalczyk (2026) Multi-agent RL with invisible collaborators: marginal advantage estimation for indirect cooperation. In Proceedings of the 42nd Conference on Uncertainty in Artificial Intelligence, Cited by: §2, §4. [22] O. Mahjoub, S. Abramowitz, R. J. de Kock, W. Khlifi, S. V. D. Toit, J. Daniel, L. B. Nessir, L. Beyers, J. C. Formanek, L. Clark, and A. Pretorius (2025) Sable: a performant, efficient and scalable sequence model for MARL. In Forty-second International Conference on Machine Learning, Cited by: §2, §4. [23] A. Kontogiannis, K. Papathanasiou, Y. Shen, G. Stamou, M. M. Zavlanos, and G. Vouros (2025) Enhancing cooperative multi-agent reinforcement learning with state modelling and adversarial exploration. In Forty-second International Conference on Machine Learning, Cited by: §2, §4. [24] S. Kang, Y. Lee, G. Kim, S. Chong, and S. Yun (2025) MA2^2E: addressing partial observability in multi-agent reinforcement learning with masked auto-encoder. In The Thirteenth International Conference on Learning Representations, Cited by: §2. [25] Y. Pei, T. Ren, Y. Zhang, Z. Sun, and M. Champeyrol (2025) Policy distillation for efficient decentralized execution in multi-agent reinforcement learning. Neurocomputing 626, p. 129617. Cited by: §2. [26] Z. Zhai, M. Xu, and W. Liang (2026) Distilling task-level coordination policies for generalizable multi-agent cooperation. In Forty-third International Conference on Machine Learning, Cited by: §2. [27] N. Cao, K. H. Low, and J. M. Dolan (2013) Multi-robot informative path planning for active sensing of environmental phenomena: a tale of two algorithms. In Proceedings of the 2013 International Conference on Autonomous Agents and Multi-Agent Systems, Cited by: §2. [28] M. Lauri, J. Pajarinen, and J. Peters (2019) Information gathering in decentralized pomdps by policy graph improvement. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, Cited by: §2. [29] W. Liao, T. Wu, and I. Wu (2025) Dynamic sight range selection in multi-agent reinforcement learning. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, Cited by: §2. [30] Y. Li, Z. Cao, J. Qiao, and S. Hu (2025) Nucleolus credit assignment for effective coalitions in multi-agent reinforcement learning. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems, AAMAS ’25, p. 1318–1326. Cited by: §2. [31] F. A. Oliehoek and C. Amato (2016) A concise introduction to decentralized pomdps. Springer. Cited by: §3.1. [32] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems. Cited by: §4. [33] G. Papoudakis, F. Christianos, L. Schäfer, and S. V. Albrecht (2021) Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (NeurIPS), Cited by: §4. [34] M. Mittal, P. Roth, J. Tigue, A. Richard, O. Zhang, P. Du, A. Serrano-Munoz, X. Yao, R. Zurbrüg, N. Rudin, et al. (2025) Isaac lab: a gpu-accelerated simulation framework for multi-modal robot learning. arXiv preprint arXiv:2511.04831. Cited by: §4. [35] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu (2022) The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems. Cited by: §4. [36] J. G. Kuba, R. Chen, M. Wen, Y. Wen, F. Sun, J. Wang, and Y. Yang (2022) Trust region policy optimisation in multi-agent reinforcement learning. In International Conference on Learning Representations, Cited by: §4. [37] C. S. De Witt, T. Gupta, D. Makoviichuk, V. Makoviychuk, P. H. Torr, M. Sun, and S. Whiteson (2020) Is independent learning all you need in the starcraft multi-agent challenge?. arXiv preprint arXiv:2011.09533. Cited by: §4. [38] Y. Zhong, J. G. Kuba, X. Feng, S. Hu, J. Ji, and Y. Yang (2024) Heterogeneous-agent reinforcement learning. Journal of Machine Learning Research. Cited by: §4. Appendix A Proofs and Additional Analysis for SPL This section provides full proofs for Proposition 1, Proposition 2, and Theorem 1 in the main paper, together with supporting lemmas and additional analysis of the SPL auxiliary objectives. The acquisition result applies a standard value-of-information argument to the top-k completion surrogate used by SPL. For the receiver analysis, we additionally relate the distillation term directly to the training objective and give a conditional connection between outcome-model error and standardized-score error. Section A.6 summarizes the scope of the analysis. A.1 Notation and Assumptions Agent i denotes an earlier participant and agent j a future collaborator. The actor-visible acquisition context of i is denoted by ctic_t^i, while xtjx_t^j denotes the actor-visible receiver context of j. The legal receiver action set is tj=u∈j:Mtj(u)=1.A_t^j=\u _j:M_t^j(u)=1\. For an attributable receiver decision, let Y∈0,1Y∈\0,1\ denote its downstream outcome and let C∈0,1C∈\0,1\ indicate whether that outcome is retained in outD_out. The retained-outcome probability is p∗j(u∣x)=Pr(Y=1∣X=x,U=u,J=j,C=1).p_*^\,j(u x)= (Y=1 X=x,U=u,J=j,C=1). (A.1) The corresponding local utility is U∗j(u∣x)=p∗j(u∣x)cj+(u)+[1−p∗j(u∣x)]cj−(u),U_*^\,j(u x)=p_*^\,j(u x)c_j^+(u)+ [1-p_*^\,j(u x) ]c_j^-(u), (A.2) where cj+(u)c_j^+(u) and cj−(u)c_j^-(u) are the task-level utilities associated with successful and unsuccessful execution, respectively. Illegal actions are excluded from the distributions below. Assumption 1 (Attribution and censoring). Whenever C=1C=1, the recorded outcome is correctly attributed to the selected receiver action. Receiver-side probability and utility results are conditional on C=1C=1. They admit an unconditional interpretation only when Y⟂C|X,U,J.Y C X,U,J. Assumption 2 (Coverage). Every legal context–candidate pair on which a receiver guarantee is invoked is selected with positive probability under the retained rollout distribution. A.2 Proof of Proposition 1: Prospective Acquisition Target Let ℳacqM_acq denote the fixed acquisition model used to construct SPL supervision. Probabilities and expectations in this subsection are taken with respect to this model. Consider binary task-validity variables W1,…,WnW_1,…,W_n. Given acquisition context ctic_t^i, let br=Prℳacq(Wr=1∣cti),b=(b1,…,bn).b_r= _M_acq(W_r=1 c_t^i), b=(b_1,…,b_n). If a future collaborator can execute at most k tasks, define Vk(b)=maxS⊆1,…,n:|S|≤k∑r∈Sbr.V_k(b)= _S \1,…,n\:|S|≤ k _r∈ Sb_r. (A.3) For a legal acquisition action u, let ZuZ_u denote its observation and bZub^Z_u the posterior marginals produced by ℳacqM_acq. Its model-relative prospective gain is Δti(u)=ℳacq[Vk(bZu)∣cti,u]−Vk(b). _t^i(u)=E_M_acq [V_k(b^Z_u) c_t^i,u ]-V_k(b). (A.4) SPL combines this gain with acquisition distance through sti(u)=log(maxΔti(u),ϵacq)−dti(u)τd.s_t^i(u)= \! ( \ _t^i(u), _acq\ )- d_t^i(u) _d. (A.5) Let ti=u:Mti(u)=1,u is task-directed,Δti(u)>0P_t^i= \u:M_t^i(u)=1,\;u is task-directed,\; _t^i(u)>0 \ denote the legal positive-gain task-directed actions. For completeness, the task-directed component of the SPL acquisition target is q~acqi(u)=exp(sti(u))∑v∈tiexp(sti(v)). q_acq^i(u)= (s_t^i(u)) _v _t^i (s_t^i(v)). (A.6) Equivalently, q~acqi=argmaxq∈Δ(ti)∑u∈tiq(u)sti(u)+H(q), q_acq^i= _q∈ (P_t^i) \ _u _t^iq(u)s_t^i(u)+H(q) \, (A.7) where H(q)=−∑uq(u)logq(u)H(q)=- _uq(u) q(u). Proof. For every feasible set S with |S|≤k|S|≤ k, let LS(b)=∑r∈Sbr.L_S(b)= _r∈ Sb_r. Since Vk(b)=maxS:|S|≤kLS(b),V_k(b)= _S:|S|≤ kL_S(b), VkV_k is convex. Bayesian consistency under ℳacqM_acq gives ℳacq[bZu∣cti,u]=b.E_M_acq [b^Z_u c_t^i,u ]=b. Jensen’s inequality therefore gives ℳacq[Vk(bZu)∣cti,u]≥Vk(ℳacq[bZu∣cti,u])=Vk(b),E_M_acq [V_k(b^Z_u) c_t^i,u ]≥ V_k (E_M_acq[b^Z_u c_t^i,u] )=V_k(b), which proves Δti(u)≥0. _t^i(u)≥ 0. This is the standard nonnegative value-of-information argument applied to the top-k completion surrogate. For the Gibbs characterization, consider F(q)=∑u∈tiq(u)sti(u)+H(q)F(q)= _u _t^iq(u)s_t^i(u)+H(q) over the probability simplex. Introducing a Lagrange multiplier for ∑uq(u)=1 _uq(u)=1 gives the stationary condition sti(u)−logq(u)−1+λ=0,s_t^i(u)- q(u)-1+λ=0, hence q(u)∝exp(sti(u)).q(u) (s_t^i(u)). Normalization yields Eq. (A.6). Since entropy is strictly concave and the score term is linear, this solution is the unique maximizer of Eq. (A.7). ∎ The complete acquisition target assigns the prescribed fixed mass to the scan action and distributes the remaining mass according to q~acqi q_acq^i. If tiP_t^i is empty, the scan action receives unit target mass. The implementation uses the small positive-gain threshold and numerical constants reported in Sec. B.4. The nonnegativity result is model-relative. Bayesian consistency within ℳacqM_acq does not require its assumed priors to equal the true environment priors. Prior misspecification can therefore preserve nonnegative model-relative gains while changing how those gains relate to performance in the true environment; this sensitivity is evaluated in Sec. D.6. A.3 Proof of Proposition 2: Retained-Outcome Calibration For any measurable predictor pj(u∣x)∈[0,1]p^j(u x)∈[0,1], define its population binary-cross-entropy risk over retained outcomes as ℒout(p)=[ℓBCE(Y,pJ(U∣X))∣C=1].L_out(p)=E [ _BCE (Y,p^J(U X) ) C=1 ]. (A.8) The excess population risk satisfies ℒout(p)−ℒout(p∗)=[KL(Ber(p∗J(U∣X))∥Ber(pJ(U∣X)))∣C=1].L_out(p)-L_out(p_*)=E [KL (Ber(p_*^J(U X))\|Ber(p^J(U X)) ) C=1 ]. (A.9) Let CΔ=supu,j|cj+(u)−cj−(u)|<∞.C_ = _u,j|c_j^+(u)-c_j^-(u)|<∞. The induced local utility then satisfies [(UpJ(U∣X)−U∗J(U∣X))2∣C=1]≤CΔ22[ℒout(p)−ℒout(p∗)].E [ (U_p^J(U X)-U_*^J(U X) )^2 C=1 ]≤ C_ ^22 [L_out(p)-L_out(p_*) ]. (A.10) Proof. Fix a covered triple (x,u,j)(x,u,j) and abbreviate p∗=p∗j(u∣x),p=pj(u∣x).p_*=p_*^\,j(u x), p=p^j(u x). The conditional binary-cross-entropy risk is r(p,p∗)=−p∗logp−(1−p∗)log(1−p).r(p;p_*)=-p_* p-(1-p_*) (1-p). Its excess risk is r(p;p∗)−r(p∗;p∗)=KL(Ber(p∗)∥Ber(p)),r(p;p_*)-r(p_*;p_*)=KL (Ber(p_*)\|Ber(p) ), which is non-negative and vanishes only when p=p∗p=p_*. Taking expectations over (X,U,J)|C=1(X,U,J) C=1 gives Eq. (A.9). From Eq. (A.2), Upj(u∣x)−U∗j(u∣x)=[pj(u∣x)−p∗j(u∣x)][cj+(u)−cj−(u)].U_p^j(u x)-U_*^\,j(u x)= [p^j(u x)-p_*^\,j(u x) ] [c_j^+(u)-c_j^-(u) ]. Hence |Upj−U∗j|2≤CΔ2|p−p∗|2. |U_p^j-U_*^\,j |^2≤ C_ ^2|p-p_*|^2. Pinsker’s inequality gives |p−p∗|2≤12KL(Ber(p∗)∥Ber(p)).|p-p_*|^2≤ 12KL (Ber(p_*)\|Ber(p) ). Combining the two inequalities and taking expectations proves Eq. (A.10). ∎ The result is conditional on retained attributable outcomes. Under the additional condition Y⟂C|X,U,J,Y C X,U,J, the retained-outcome probability also equals the corresponding unconditional success probability. A.4 Receiver Target and Supporting Analysis Fix an eligible receiver decision (j,t)(j,t) with legal action set =tj,K=||.A=A_t^j, K=|A|. For any local utility vector U over A, let μU _U and σU2 _U^2 denote its mean and variance over legal actions, and define the standardized score zU(u)=U(u)−μUσU2+ϵz,ϵz>0.z_U(u)= U(u)- _U _U^2+ _z, _z>0. (A.11) For a score vector z, the SPL receiver target is qSPLj(u)=exp(βz(u))∑v∈exp(βz(v)),u∈.q_SPL^j(u)= (β z(u)) _v (β z(v)), u . (A.12) Lemma 1 (Soft receiver target). For β>0β>0, the distribution in Eq. (A.12) uniquely maximizes ∑u∈q(u)z(u)+1βH(q) _u q(u)z(u)+ 1βH(q) (A.13) over q∈Δ()q∈ (A), and satisfies maxu∈z(u)−u∼qSPLj[z(u)]≤logKβ. _u z(u)-E_u q_SPL^j[z(u)]≤ Kβ. (A.14) Proof. Introducing a Lagrange multiplier for the simplex constraint gives the stationary condition z(u)−1β[logq(u)+1]+λ=0,z(u)- 1β [ q(u)+1 ]+λ=0, hence q(u)∝exp(βz(u)).q(u) (β z(u)). Since the entropy term is strictly concave, this solution is unique and equals Eq. (A.12). Let u∗∈argmaxu∈z(u).u^*∈ _u z(u). Optimality of qSPLjq_SPL^j relative to the point mass on u∗u^* gives qSPLj[z]+1βH(qSPLj)≥z(u∗).E_q_SPL^j[z]+ 1βH(q_SPL^j)≥ z(u^*). Using H(qSPLj)≤logKH(q_SPL^j)≤ K yields Eq. (A.14). ∎ Receiver distillation. For a fixed SPL target qSPLjq_SPL^j and receiver policy πtj _t^j, CE(qSPLj,πtj)=H(qSPLj)+KL(qSPLj∥πtj).CE(q_SPL^j, _t^j)=H(q_SPL^j)+KL(q_SPL^j\| _t^j). (A.15) Thus, η=KL(qSPLj∥πtj)=CE(qSPLj,πtj)−H(qSPLj),η=KL(q_SPL^j\| _t^j)=CE(q_SPL^j, _t^j)-H(q_SPL^j), (A.16) so the local distillation term is directly computable from the receiver target and actor distributions. Averaging Eq. (A.16) over receiver decisions gives an empirical diagnostic of target distillation. Connection between outcome error and score error. The calibration result in Eq. (A.10) controls utility error on selected actions, whereas receiver regret depends on scores over the full legal action set. A quantitative connection follows under stronger coverage. Let ρj(u∣x)=Pr(U=u∣X=x,J=j,C=1).ρ^j(u x)= (U=u X=x,J=j,C=1). Suppose that, on the retained context distribution considered below, there exists α>0α>0 such that ρj(u∣x)≥αρ^j(u x)≥α for every legal candidate whose utility is supplied by the outcome model. Any remaining legal actions with fixed utilities are assumed identical in the predicted and reference utility vectors. Lemma 2 (Mean score-error control). Let pψ¯p_ ψ denote the target outcome predictor and let δ(X,J)=‖zψ¯−z∗‖∞.δ(X,J)=\|z_ ψ-z_*\|_∞. Under the uniform coverage condition above, [δ(X,J)2∣C=1]≤CΔ22αϵz[ℒout(pψ¯)−ℒout(p∗)]≤CΔ22αϵzℒout(pψ¯).E [δ(X,J)^2 C=1 ]≤ C_ ^22α _z [L_out(p_ ψ)-L_out(p_*) ]≤ C_ ^22α _zL_out(p_ ψ). (A.17) Proof. For a fixed legal action set, the stabilized standardization map in Eq. (A.11) is 1/ϵz1/ _z-Lipschitz from the utility-vector ℓ2 _2 norm to the score-vector ℓ∞ _∞ norm. Hence δ(X,J)2≤‖Uψ¯−U∗‖22ϵz.δ(X,J)^2≤ \|U_ ψ-U_*\|_2^2 _z. The uniform coverage condition gives ∑u∈(Uψ¯j(u∣x)−U∗j(u∣x))2≤1α[(Uψ¯J(U∣X)−U∗J(U∣X))2∣X=x,J=j,C=1]. _u (U_ ψ^j(u x)-U_*^j(u x) )^2≤ 1αE [ (U_ ψ^J(U X)-U_*^J(U X) )^2 X=x,J=j,C=1 ]. Taking expectations over retained contexts and applying Eq. (A.10) proves the first inequality in Eq. (A.17). The second follows from ℒout(p∗)≥0.L_out(p_*)≥ 0. ∎ Equation (A.17) controls mean squared score error under the retained context distribution; it is not a uniform per-context bound. For fixed α and ϵz _z, vanishing population excess outcome risk implies vanishing mean squared score error. Held-out binary-cross-entropy provides an empirical estimate of population outcome risk, while finite-sample generalization remains separate from this population statement. A.5 Proof of Theorem 1: Local Receiver Score Regret Let z∗z_* denote the standardized score induced by the retained-outcome local utility U∗jU_*^\,j, and let zψ¯z_ ψ denote the score induced by the target outcome model. Define δ=∥zψ¯−z∗∥∞,η=KL(qSPLj∥πtj),δ=\|z_ ψ-z_*\|_∞, η=KL (q_SPL^j\| _t^j ), where qSPLj(u)∝exp(βzψ¯(u)),q_SPL^j(u) (β z_ ψ(u)), and let Rz=maxu∈z∗(u)−minu∈z∗(u).R_z= _u z_*(u)- _u z_*(u). Theorem 1 in the main paper states that Regz,tj(πtj)≤logKβ+2δ+Rzη2,Reg_z,t^j( _t^j)≤ Kβ+2δ+R_z η2, (A.18) where Regz,tj(πtj)=maxu∈z∗(u)−u∼πtj[z∗(u)].Reg_z,t^j( _t^j)= _u z_*(u)-E_u _t^j[z_*(u)]. (A.19) Proof. Add and subtract the expected reference score under qSPLjq_SPL^j: Regz,tj(πtj)=[maxuz∗(u)−qSPLj[z∗]]+[qSPLj[z∗]−πtj[z∗]].Reg_z,t^j( _t^j)= [ _uz_*(u)-E_q_SPL^j[z_*] ]+ [E_q_SPL^j[z_*]-E_ _t^j[z_*] ]. Since ‖zψ¯−z∗‖∞≤δ,\|z_ ψ-z_*\|_∞≤δ, we have maxuz∗(u)−qSPLj[z∗]≤maxuzψ¯(u)−qSPLj[zψ¯]+2δ. _uz_*(u)-E_q_SPL^j[z_*]≤ _uz_ ψ(u)-E_q_SPL^j[z_ ψ]+2δ. Applying Eq. (A.14) gives maxuz∗(u)−qSPLj[z∗]≤logKβ+2δ. _uz_*(u)-E_q_SPL^j[z_*]≤ Kβ+2δ. For the second term, qSPLj[z∗]−πtj[z∗]≤RzTV(qSPLj,πtj).E_q_SPL^j[z_*]-E_ _t^j[z_*]≤ R_z\,TV (q_SPL^j, _t^j ). Pinsker’s inequality gives TV(qSPLj,πtj)≤KL(qSPLj∥πtj)2=η2.TV (q_SPL^j, _t^j )≤ KL(q_SPL^j\| _t^j)2= η2. Combining the two bounds proves Eq. (A.18). ∎ The bound separates three local sources of receiver error: finite-temperature softness, error in the evidence-conditioned score estimate, and incomplete distillation of the SPL target into the decentralized policy. Equation (A.16) makes the distillation term directly observable during training, while Lemma 2 gives a population-level connection between score approximation and outcome-model error under uniform coverage. A.6 Scope of the Local Analysis The results above characterize the local auxiliary objectives used by SPL. They support the construction and interpretation of the training signals but do not establish global convergence or monotonic improvement of the cooperative return. Acquisition model and prior misspecification. Proposition 1 in the main paper is a model-relative value-of-information result. Bayesian consistency within the acquisition model guarantees non-negative prospective gain for the top-k surrogate, including when the model uses internally consistent but misspecified priors. Such misspecification can change the relation between the acquisition score and performance under the true environment distribution. Section D.6 evaluates this sensitivity while keeping the remaining structured model components fixed. Receiver approximation scope. Proposition 2 in the main paper controls expected local-utility error on the retained selected-action distribution. The receiver distillation quantity η is directly computable from the target and actor distributions through Eq. (A.16). Lemma 2 further connects mean squared standardized-score error to population outcome-model risk under a uniform retained-action coverage condition. It does not provide a uniform per-context bound, and empirical validation loss introduces the usual finite-sample generalization error. Theorem 1 in the main paper therefore remains a local standardized-score guarantee, not a bound on raw utility or full team return. Local utility and retained outcomes. The retained-outcome utility in Eq. (A.2) describes the task-level consequence of an attributable receiver action and need not equal the full Dec-POMDP action value. Receiver-side results are conditional on attributable outcomes retained in outD_out and admit an unconditional interpretation only under Assumption 1. Under partially overlapping participation, the receiver context may combine evidence acquired before and after participation begins; the analysis conditions on the complete actor-visible context and does not isolate the causal contribution of pre-participation evidence. Backbone optimization. SPL auxiliary actor updates are applied in addition to the native MAPPO, IPPO, or HAPPO optimization. The analysis does not provide a convergence guarantee for the resulting joint optimization; the reported task-completion and return results are empirical. Appendix B Implementation and Reproducibility Details This section provides the environment configurations, participation schedules, SPL target construction, optimization settings, and training procedure used in the reported experiments. All methods are implemented within the HARL framework. We denote the SPL extensions of MAPPO, IPPO, and HAPPO as SPL-MAPPO, SPL-IPPO, and SPL-HAPPO, respectively. Within each matched comparison, the corresponding unmodified backbone is referred to as MAPPO, IPPO, or HAPPO. B.1 Environment Configuration The primary four-agent MPE and RWARE evaluations contain two earlier participants, two future collaborators, and eight candidate tasks. Each domain includes Base and Advanced variants. We additionally evaluate SPL in physics-based UAV–UGV tasks in Isaac Lab with either one UAV and one UGV or two UAVs and two UGVs. Persistent task-relevant evidence produced by earlier participants remains available to future collaborators. The Base variants use fixed task assignments and earlier-participant–future-collaborator correspondences. MPE-Advanced randomizes balanced task shards, spawn assignments, and future-collaborator identities, while RWARE-Advanced permutes task identifiers and uses a shared candidate pool with dynamic task claims. The Isaac Lab settings introduce physical motion, obstacles, and planner-mediated downstream task execution. The eight-agent scaling configurations are described separately below. Table B.1: Four-agent MPE/RWARE and Isaac Lab configurations. Environment Earlier Future Tasks Horizon MPE-Base 2 2 8 125 (250) MPE-Advanced 2 2 8 125 (250) RWARE-Base 2 2 8 500 (1000) RWARE-Advanced 2 2 8 500 (1000) Isaac Lab (2-agent) 1 UAV 1 UGV 4 600 Isaac Lab (4-agent) 2 UAVs 2 UGVs 6 840 For the primary four-agent MPE and RWARE settings, the environments use the same eight task-validity priors and a fixed-valid-task sampling rule. Probe outcomes are noisy but persistent, and each future collaborator can execute at most two candidate tasks. Table B.2 summarizes the main task and evidence parameters. Table B.2: Task and evidence settings for MPE and RWARE. Setting MPE RWARE Candidate tasks 8 8 Validity priors (0.80,0.80,0.65,0.65,0.35,0.35,0.20,0.20)(0.80,0.80,0.65,0.65,0.35,0.35,0.20,0.20) Probe budget / earlier participant 5 6 Probe limit / task 2 2 Probe accuracy 0.85 0.85 Publication delay 5 5 Execution budget / future collaborator 2 2 Valid-task value 12,12,10,10,8,8,6,6\12,12,10,10,8,8,6,6\ 1 Probe cost 0.2 0.01 Invalid-execution cost 8.0 0.25 B.2 Temporal Participation Patterns Participation intervals follow the half-open convention [tstart,tend)[t_start,t_end). Inactive agents take a null environment action and make no effective policy decision. The Base and Advanced variants within each domain use the same participation schedules. We evaluate five participation patterns. Non-overlap uses a strict team-wide handoff from earlier participants to future collaborators. Overlap-25 and Overlap-50 introduce fixed levels of cross-role overlap. Dynamic samples pair-specific handoff times each episode while preserving aggregate role exposure. Cascaded distributes the two earlier-participant–future-collaborator pairs across successive participation intervals, with an extended horizon to preserve total role exposure. Table B.3: Four-agent participation schedules. Domain Participation Horizon P0P_0 P1P_1 R0R_0 R1R_1 MPE Non-overlap 125 [0,45)[0,45) [0,45)[0,45) [45,125)[45,125) [45,125)[45,125) MPE Overlap-25 125 [0,45)[0,45) [0,45)[0,45) [34,125)[34,125) [34,125)[34,125) MPE Overlap-50 125 [0,45)[0,45) [0,45)[0,45) [22,125)[22,125) [22,125)[22,125) MPE Dynamic 125 [0,h0)[0,h_0) [0,h1)[0,h_1) [h0,125)[h_0,125) [h1,125)[h_1,125) MPE Cascaded 250 [0,45)[0,45) [125,170)[125,170) [45,125)[45,125) [170,250)[170,250) RWARE Non-overlap 500 [0,200)[0,200) [0,200)[0,200) [200,500)[200,500) [200,500)[200,500) RWARE Overlap-25 500 [0,175)[0,175) [0,225)[0,225) [175,500)[175,500) [225,500)[225,500) RWARE Overlap-50 500 [0,150)[0,150) [0,250)[0,250) [150,500)[150,500) [250,500)[250,500) RWARE Dynamic 500 [0,h0)[0,h_0) [0,h1)[0,h_1) [h0,500)[h_0,500) [h1,500)[h_1,500) RWARE Cascaded 1000 [0,200)[0,200) [500,700)[500,700) [200,500)[200,500) [700,1000)[700,1000) Here, P0,P1P_0,P_1 denote earlier participants and R0,R1R_0,R_1 denote future collaborators. In MPE, Overlap-25 and Overlap-50 provide 11 and 23 steps of cross-role overlap, respectively; the corresponding RWARE overlaps are 50 and 100 steps. For Dynamic participation, h0=h¯+δh_0= h+δ and h1=h¯−δh_1= h-δ, where δ is sampled once per episode from a symmetric discrete uniform distribution. MPE uses h¯=45 h=45 and δ∈−11,…,11δ∈\-11,…,11\, while RWARE uses h¯=200 h=200 and δ∈−50,…,50δ∈\-50,…,50\. Consequently, Dynamic changes the pair-specific handoff times while preserving aggregate earlier-participant and future-collaborator exposure. Cascaded preserves the same role exposure while changing its temporal ordering. B.3 Isaac Lab Configuration The two-agent Isaac Lab task contains four candidate sites, of which two are serviceable in each episode, on a 12×1212× 12 occupancy grid with five barriers. The four-agent setting contains six candidate sites, of which four are serviceable. Candidate positions are public from reset, while serviceability remains hidden. UAV sensing has accuracy 0.850.85 and a five-step publication delay. UGV motion is planned from the published occupancy map, and selecting a task commits the UGV to its physical execution. Neither the actor nor the critic receives hidden serviceability. Table B.4: Isaac Lab participation and training configurations. Agents Participation Training steps Participation windows 1 UAV + 1 UGV Non-overlap 1,996,800 UAV [0,200)[0,200); UGV [200,600)[200,600) 1 UAV + 1 UGV Dynamic 1,996,800 UAV [0,h)[0,h); UGV [h,600)[h,600), h∼Unif150,…,250h \150,…,250\ 2 UAVs + 2 UGVs Non-overlap 3,010,560 UAVs [0,300)[0,300); UGVs [300,840)[300,840) 2 UAVs + 2 UGVs Dynamic 3,010,560 UAVs [0,300)[0,300); UGVs [h,h+540)[h,h+540), h∈150,180,…,300h∈\150,180,…,300\ Dynamic handoff times are sampled per episode. The two-agent setting exposes the normalized handoff step to the actors, while the four-agent setting uses a shared UGV participation window with a sampled start time. MAPPO and SPL-MAPPO use matched participation schedules and evaluation instances. Each setting uses three independent training seeds and 200 final evaluation episodes per seed. Final policies are evaluated deterministically, and checkpoints are not selected by performance. B.4 SPL Target Construction Prospective acquisition target. The frozen acquisition model constructs the task-validity posterior from the configured validity priors, the fixed number of valid tasks, and public probe outcomes available in the actor-visible context. Repeated probe observations are aggregated through their positive and negative counts using the configured probe accuracy. Hidden task validity and future outcomes are never provided to either the actor or the target-construction procedure. Posterior marginals br=Pr(Wr=1∣cti)b_r= (W_r=1 c_t^i) are used to compute the prospective gain Δti(u) _t^i(u) defined in Eq. (A.4). In the implementation, task-directed actions with Δti(u)≤10−8 _t^i(u)≤ 10^-8 receive no acquisition-target mass. For the remaining legal task-directed actions, wti(u)=maxΔti(u),ϵacqexp(−dti(u)τd),ϵacq=10−12,τd=0.50.w_t^i(u)= \ _t^i(u), _acq\ \! (- d_t^i(u) _d ), _acq=10^-12, _d=0.50. (B.1) The weights are normalized over eligible task-directed actions. The target assigns 0.950.95 total probability mass to this distribution and 0.050.05 to the scan action. If no positive-gain task-directed action is legal, the scan action receives unit target mass. The acquisition model and target-construction rule remain fixed throughout training. Outcome-supervised receiver target. For each future collaborator j, SPL uses a candidate-shared outcome model pψj(u∣x)=sigmoid(fψ(gj(x,u))),p_ψ^\,j(u x)=sigmoid (f_ψ(g_j(x,u)) ), (B.2) where gj(x,u)g_j(x,u) contains the actor-visible pre-decision context, candidate-action features, and collaborator-role identity. The same scorer fψf_ψ is applied across candidate actions. Training labels are available only for selected actions whose outcomes can be reliably attributed; unselected actions receive no counterfactual labels. A slowly updated target model pψ¯p_ ψ is used to construct the receiver target. For legal candidate action u, Uψ¯j(u∣x)=pψ¯j(u∣x)cj+(u)+[1−pψ¯j(u∣x)]cj−(u).U_ ψ^\,j(u x)=p_ ψ^\,j(u x)c_j^+(u)+ [1-p_ ψ^\,j(u x) ]c_j^-(u). (B.3) The task-level utilities are fixed during training. In MPE, cj+(u)=v(u)/12c_j^+(u)=v(u)/12 with v(u)∈12,10,8,6v(u)∈\12,10,8,6\ and cj−(u)=−8/12c_j^-(u)=-8/12. In RWARE, cj+(u)=1c_j^+(u)=1 and cj−(u)=−0.25c_j^-(u)=-0.25. In Isaac Lab, cj+(u)=1c_j^+(u)=1 and cj−(u)=0c_j^-(u)=0. The no-op utility is zero. These quantities define the local utilities used for SPL supervision and are not interpreted as the complete environment return. Utilities are standardized over the current legal action set using ϵz=10−6 _z=10^-6 as the variance stabilizer, following Eq. (A.11). The receiver target is then the Boltzmann distribution in Eq. (A.12). All reported experiments use β=1β=1. Receiver supervision uses this soft utility target directly, without mixing it with the current actor policy or converting it to a one-hot target. Table B.5: Training hyperparameters for the PPO backbones and SPL. PPO backbone SPL-specific Parameter Value Parameter Value Actor hidden layers [128,128][128,128] Outcome hidden width 128 Activation ReLU Outcome learning rate 3×10−43× 10^-4 Actor learning rate 1×10−41× 10^-4 Replay capacity 20,000 Critic learning rate 5×10−45× 10^-4 Outcome minibatch size 128 PPO epochs 3 Outcome gradient steps 4 Critic epochs 3 Target EMA rate ρ 0.05 PPO clip 0.2 Outcome-model gate 128 Entropy coefficient 0.002 Per-class model gate 16 Discount γ 0.99 Receiver-policy gate 1,024 GAE λ 0.95 Per-class receiver gate 128 Value normalization enabled Acquisition epochs 1 Recurrent policy disabled Receiver epochs 20 Parameter sharing disabled Acquisition task mass 0.95 Gradient norm cap 10 Distance temperature τd _d 0.50 Receiver target strength β 1.0 B.5 Optimization and Training Settings MAPPO, IPPO, and HAPPO use the same feed-forward actor architecture and PPO hyperparameters listed in Table B.5. Policies are not parameter-shared within a team. MAPPO and HAPPO retain centralized critics, whereas IPPO uses independent critics based on local observations. SPL does not use centralized critic state and does not modify the native reward, return estimator, critic objective, or critic target. The corresponding augmented methods are denoted SPL-MAPPO, SPL-IPPO, and SPL-HAPPO. Vanilla–SPL comparisons use the same backbone configuration, environment, participation schedule, training budget, and evaluation protocol. All primary four-agent MPE and RWARE method–environment–participation combinations are trained for two million environment steps without early stopping. Reported results use five independent training seeds per method and configuration, with 200 final evaluation episodes per seed. Training seeds are the independent replication units. Final policies are evaluated deterministically using a fixed 200-episode reset-seed schedule reused across training seeds within each environment protocol. The scaling and Isaac Lab settings use the training procedures described in their respective configuration sections. Training-seed pairing. A retrospective audit recovered all 600 expected method–training-seed records for the 60 primary Vanilla–SPL comparisons. Same-training-seed pairing was verified for 285 pairs across 57 configurations, providing five paired training seeds in each of those configurations. The Non-overlap MAPPO comparisons in MPE-Advanced, RWARE-Base, and RWARE-Advanced use different Vanilla and SPL training-seed sets. Their observed method means remain included in the descriptive results, but paired confidence intervals and paired p-values are reported as unavailable. Consequently, paired inference covers 57 of the 60 primary configurations. Full statistical details are provided in Appendix D.7. Baseline implementation and adaptation. MAE-MAPPO uses the same per-agent observations, legal task-option masks, team rewards, participation schedules, and evaluation protocol as MAPPO. For Sable and SMPE2, we retain the original learning architectures and optimization objectives and adapt only the environment interface to the SP tasks. Both methods receive the same environment-specific local observations, legal task-option masks, shared rewards, participation schedules, and episode boundaries as the PPO-based methods. A common task-option controller maps each legal high-level choice to the corresponding native environment action, while inactive agents receive only the environment-defined no-op action. Where global or state-level inputs are required, they are constructed solely from non-privileged observable information and do not expose hidden task validity. Sable retains its sequence-model rollout procedure, while SMPE2 retains its recurrent policy, critic, state-VAE, and state-filtering components. All primary four-agent baseline comparisons use the same two-million-step training budget, five independent training seeds per method and configuration, and 200 final evaluation episodes per seed. Knowledge-matched control. SKD-MAPPO uses the same structured task information, prospective acquisition supervision, task utilities, outcome-model training procedure, target-model update, readiness conditions, and auxiliary update budget as SPL-MAPPO. Its online outcome model is trained from the same complete actor-visible pre-decision context and attributable outcome labels as in SPL. When the learned target outcome model is subsequently used to construct receiver supervision, however, predecessor-generated task evidence is masked from its input. The decentralized actor observation remains unchanged. SKD-MAPPO therefore retains evidence during outcome-model learning but withholds that evidence when constructing the receiver target. This control tests the effect of removing inherited evidence specifically from the target-construction pathway while leaving the remaining SPL training machinery unchanged. Evidence-redacted control. Evidence-redacted uses the same MAPPO backbone, acquisition supervision, task utilities, attributable outcome labels, readiness conditions, receiver utility construction, and auxiliary update budget as SPL-MAPPO. Unlike SKD-MAPPO, inherited predecessor-generated task evidence is removed consistently from the feature vectors supplied to both the online outcome model during outcome-model training and the target outcome model during receiver-target construction. The replay buffer retains the original actor-visible pre-decision context, and redaction is applied when outcome-model features are constructed. The decentralized actor observation is not modified. Evidence-redacted therefore evaluates receiver learning when inherited task evidence is unavailable to both outcome prediction and receiver-target construction, while preserving the remaining training and execution pipeline. The distinction between the two controls is thus the stage at which inherited evidence is removed. SKD-MAPPO allows the outcome model to learn from the complete evidence-conditioned context but removes predecessor evidence when that model is queried to construct receiver supervision. Evidence-redacted removes the same class of inherited evidence from both outcome-model learning and receiver-target construction. Neither control removes the evidence from the decentralized actor observation, and neither exposes hidden task validity or future outcomes to the actor or critic. B.6 Training Procedure and Decentralized Execution Outcome replay and readiness. When a future collaborator selects a task-directed action, SPL stores its actor-visible pre-decision context, selected action, collaborator-role identity, and subsequently attributable outcome. Successful completion is labeled 11, while an attributable unsuccessful attempt is labeled 00. Interrupted, expired, unresolved, or otherwise ambiguous attempts are excluded and receive no synthetic labels. Information observed after the decision is used only to determine the outcome label and is not included in the stored pre-decision context. Outcome-model optimization begins once outD_out contains at least 128 retained decisions, including at least 16 examples from each outcome class. Each eligible training iteration then applies four outcome-model gradient steps with minibatches of at most 128 replay samples. Receiver-policy supervision begins once the replay contains at least 1,024 retained decisions with at least 128 examples from each class. Before this condition is met, future collaborators receive only their native MARL actor update. Prospective acquisition supervision does not depend on the receiver-side readiness conditions. Update order and gradient boundaries. After each rollout, attributable receiver outcomes are appended to outD_out. Once the outcome-model readiness condition is satisfied, the online outcome model is updated by binary cross-entropy, followed by the exponential-moving-average update ψ¯←(1−ρ)ψ¯+ρψ. ψ←(1-ρ) ψ+ρψ. When receiver supervision is ready, receiver targets are constructed from the updated target outcome model before backbone optimization. SPL actor updates are interleaved with the native actor updates. For each eligible actor, the native backbone actor update is performed first, followed by the corresponding SPL update. MAPPO processes actors in fixed agent order and updates the centralized critic after all actor updates. IPPO applies the native actor update, SPL update, and independent critic update for each agent before proceeding to the next agent. HAPPO retains its sequential heterogeneous actor-update order. For each actor, the current cumulative factor is stored before the native actor update, followed by the applicable SPL update. Action probabilities are then re-evaluated to update the cumulative factor used by subsequent actors. Thus, later actors can receive factors reflecting the post-SPL policies of earlier actors. We do not claim that SPL-HAPPO preserves HAPPO’s original monotonic-improvement guarantee. The outcome loss updates only the online outcome model ψ, the acquisition loss updates only the eligible earlier-participant actor, and the receiver loss updates only the eligible future-collaborator actor. Gradients do not propagate through the frozen acquisition model, target outcome model, utility construction, replayed outcomes, future trajectories, or environment dynamics. SPL does not modify the environment reward, transition function, observation space, action space, native return estimator, or critic objective. Decentralized execution. At execution time, each active agent acts solely through its decentralized policy, uti∼πθi(⋅∣τti,Mti).u_t^i _ _i (· _t^i,M_t^i ). (B.4) The acquisition model, outcome replay, online and target outcome models, utility calculations, and SPL target distributions are used only during training. SPL-MAPPO, SPL-IPPO, and SPL-HAPPO use the original decentralized policy architecture at execution time, without privileged state, auxiliary inference, or additional communication. B.7 Training Algorithm Algorithm 1 summarizes one SPL training iteration. SPL preserves the native optimization procedure of each MARL backbone and adds the role-appropriate auxiliary actor update. Earlier participants receive prospective acquisition supervision, while future collaborators receive receiver supervision once the corresponding replay-readiness condition is satisfied. Algorithm 1 One training iteration with Staggered Participation Learning 1: Backbone ℬB; policies πθii∈\ _ _i\_i ; frozen acquisition model; online outcome model pψp_ψ; target outcome model pψ¯p_ ψ; outcome replay outD_out 2: Collect one cooperative rollout 3: Compute native return and advantage inputs 4: Extract attributable future-collaborator outcomes 5: Add retained samples (xtj,utj,j,ytj)(x_t^j,u_t^j,j,y_t^j) to outD_out 6: if outcome-model readiness condition is satisfied then 7: Update ψ using ℒoutL_out 8: ψ¯←(1−ρ)ψ¯+ρψ ψ←(1-ρ) ψ+ρψ 9: end if 10: Construct acquisition targets for eligible earlier participants 11: if receiver-supervision readiness condition is satisfied then 12: Construct receiver targets using pψ¯p_ ψ 13: end if 14: if ℬB is MAPPO then 15: for each actor i do 16: Apply the native MAPPO actor update to i 17: Apply the eligible SPL actor update to i 18: end for 19: Update the centralized critic 20: else if ℬB is IPPO then 21: for each actor i do 22: Apply the native IPPO actor update to i 23: Apply the eligible SPL actor update to i 24: Update the local critic of i 25: end for 26: else if ℬB is HAPPO then 27: Initialize cumulative factor F←1F← 1 28: Select the HAPPO actor-update order 29: for each actor i in the selected order do 30: Store the current factor F for actor i 31: Evaluate the pre-update action probabilities 32: Apply the native HAPPO actor update to i 33: Apply the eligible SPL actor update to i 34: Evaluate the post-update action probabilities 35: Update the cumulative factor F 36: end for 37: Update the centralized critic 38: end if 39: return updated policies, critics, outcome models, and replay B.8 Eight-Agent Scaling Protocol Experimental scope. The eight-agent scaling study evaluates MPE-Advanced and RWARE-Advanced with four earlier participants P0,…,P3P_0,…,P_3, four future collaborators R0,…,R3R_0,…,R_3, and sixteen candidate tasks. We evaluate Non-overlap, Dynamic, and Cascaded participation using MAPPO and SPL-MAPPO. Non-overlap and Dynamic are trained for two million environment steps, while Cascaded uses four million steps to accommodate its extended participation horizon; all configurations use three independent training runs. Table B.6: Task and evidence configuration for the eight-agent scaling experiments. The notation x×4x^× 4 denotes four repetitions of value x. Setting MPE-Advanced RWARE-Advanced Earlier participants 4 4 Future collaborators 4 4 Candidate tasks 16 16 Fixed valid tasks 8 8 Valid tasks / four-task group 2 2 Validity priors (0.80×4,0.65×4,0.35×4,0.20×4)(0.80^× 4,0.65^× 4,0.35^× 4,0.20^× 4) Probe budget / earlier participant 5 6 Probe limit / task 2 2 Probe accuracy 0.85 0.85 Publication delay 5 5 Execution budget / future collaborator 2 2 Valid-task value (12×4,10×4,8×4,6×4)(12^× 4,10^× 4,8^× 4,6^× 4) 1 Probe cost 0.20 0.01 Invalid-execution cost 8.00 0.25 Environment configuration. The sixteen candidate tasks form four groups of four, with two valid tasks sampled in each group. In MPE-Advanced, balanced task shards are randomly assigned at reset and the earlier-participant–future-collaborator correspondence is randomly assigned at handoff. This correspondence is not available to earlier participants beforehand. Earlier participants and future collaborators spawn on opposite sides of the environment at x=−1.82x=-1.82 and x=1.82x=1.82, respectively, with y∈0.76, 0.2533,−0.2533,−0.76.y∈\0.76,\ 0.2533,\ -0.2533,\ -0.76\. The agent speed, view radius, interaction radius, and placement jitter are 1.01.0, 4.04.0, 0.180.18, and 0.020.02. Each actor observation has dimension 339, and the MAPPO critic input concatenates the eight actor observations, giving dimension 2712. Hidden task validity is unavailable to both representations. RWARE-Advanced uses one shelf row, three shelf columns of height eight, and sensor range one. Candidate task identifiers are permuted each episode, future collaborators select from a shared candidate pool, and competing claims are resolved by first arrival. Hidden task validity is excluded from both actor observations and the MAPPO critic input. Temporal participation schedules. All participation intervals follow the half-open convention [tstart,tend)[t_start,t_end). Let H denote the ordinary episode horizon, a the nominal handoff time, d the maximum Dynamic deviation, and HcasH_cas the Cascaded horizon. Table B.7 summarizes the corresponding values. Table B.7: Temporal parameters and aggregate role exposure for the eight-agent experiments. Exposure is measured in agent-steps per episode. Domain H a d HcasH_cas Earlier Future MPE-Advanced 125 45 11 500 180 320 RWARE-Advanced 500 200 50 2000 800 1200 For Non-overlap, ℐnonPi=[0,a),ℐnonRi=[a,H),i∈0,1,2,3.I^P_i_non=[0,a), ^R_i_non=[a,H), i∈\0,1,2,3\. (B.5) For Dynamic participation, two independent deviations are sampled per episode, δk∼Unif−d,…,d,k∈0,1, _k \-d,…,d\, k∈\0,1\, (B.6) with paired handoffs h2k=a+δk,h2k+1=a−δk.h_2k=a+ _k, h_2k+1=a- _k. (B.7) The corresponding intervals are ℐdynPi=[0,hi),ℐdynRi=[hi,H).I^P_i_dyn=[0,h_i), ^R_i_dyn=[h_i,H). (B.8) The handoff vector is sampled each episode and included in the actor-visible schedule information. Since ∑i=03hi=4a _i=0^3h_i=4a, Dynamic preserves the same aggregate role exposure as Non-overlap. For Cascaded participation, ℐcasPi=[iH,iH+a),ℐcasRi=[iH+a,(i+1)H),i∈0,1,2,3.I^P_i_cas=[iH,iH+a), ^R_i_cas=[iH+a,(i+1)H), i∈\0,1,2,3\. (B.9) The resulting horizon is Hcas=4H_cas=4H, preserving aggregate role exposure while distributing the four earlier-participant–future-collaborator pairs across successive intervals. Training and evaluation. The eight-agent MAPPO and SPL-MAPPO runs use the feed-forward architecture, PPO settings, and SPL-specific hyperparameters reported in Table B.5. Policies are not parameter-shared and recurrent policies are disabled. Non-overlap and Dynamic are trained for two million environment steps, whereas Cascaded is trained for four million steps; each configuration uses three independent runs and is evaluated over 200 final episodes per run. MPE-Advanced uses eight parallel rollout environments, whereas RWARE-Advanced uses one. Reported task-completion results use the final checkpoint at the corresponding training budget, without performance-based checkpoint selection. Appendix C Observed Training Cost We report observed wall-clock training time as a practical indication of computational cost. Measurements correspond to completed full-budget runs; short qualification and smoke-scale runs are excluded. Because experiments ran under varying system load and host hardware was not recorded consistently, these values are descriptive runtime measurements. They are not controlled hardware-throughput benchmarks or estimates of algorithmic complexity. Table C.1 summarizes the observed training time. In the primary four-agent experiments, all methods use two million environment steps, and Table C.1(a) reports mean minutes per completed run averaged over the ten settings formed by two environment variants and five participation patterns. Table C.1(b) reports the eight-agent scaling and Isaac Lab experiments. The eight-agent Non-overlap and Dynamic settings use 2M environment steps, whereas Cascaded uses 4M. The two- and four-agent Isaac Lab settings use approximately 2.0M and 3.0M steps, respectively. Table C.1: Observed wall-clock training time in minutes. Primary four-agent values are averages across the ten environment–participation settings in each domain; scaling and Isaac Lab values are mean ± standard error over three independent runs. Light blue shading identifies SPL results. (a) Primary four-agent experiments. Method MPE RWARE IPPO 154.1 166.4 SPL-IPPO 175.3 177.9 MAPPO 114.0 128.3 SPL-MAPPO 131.1 129.6 HAPPO 134.6 128.5 SPL-HAPPO 152.4 145.2 MAE-MAPPO 157.2 128.6 Sable 205.1 210.8 SMPE2 119.2 218.0 (b) Scaling and Isaac Lab experiments. Setting Participation Steps MAPPO SPL-MAPPO MPE-Adv. (8) Non-overlap 2.00M 76.0±0.276.0± 0.2 87.0±0.187.0± 0.1 Dynamic 2.00M 89.5±0.689.5± 0.6 103.0±0.5103.0± 0.5 Cascaded 4.00M 175.5±0.2175.5± 0.2 185.5±2.7185.5± 2.7 RWARE-Adv. (8) Non-overlap 2.00M 242.9±6.1242.9± 6.1 396.9±15.3396.9± 15.3 Dynamic 2.00M 216.6±3.6216.6± 3.6 377.5±11.0377.5± 11.0 Cascaded 4.00M 453.6±16.6453.6± 16.6 538.3±15.4538.3± 15.4 Isaac Lab (2) Non-overlap 2.00M 548.8±17.5548.8± 17.5 562.9±18.8562.9± 18.8 Dynamic 2.00M 438.1±9.9438.1± 9.9 450.0±7.0450.0± 7.0 Isaac Lab (4) Non-overlap 3.01M 1090.2±17.81090.2± 17.8 1244.7±43.21244.7± 43.2 Dynamic 3.01M 1120.1±15.11120.1± 15.1 1295.0±7.31295.0± 7.3 Observation. The practical training-time effect of SPL varies across environments and team configurations. In the primary four-agent experiments, SPL incurs moderate additional wall-clock cost for most PPO backbones, while MAPPO and SPL-MAPPO are nearly identical in RWARE. In the eight-agent experiments, the relative increase is modest in MPE and larger in RWARE; absolute times across participation patterns should be interpreted together with their different training budgets, particularly the 4M-step Cascaded settings. The two-agent Isaac Lab runs show only a small difference, whereas the four-agent settings require approximately 14–16% more observed training time with SPL-MAPPO. Overall, SPL introduces additional training-time computation whose magnitude depends on the environment and configuration, while decentralized execution remains unchanged. Appendix D Detailed Experiment Results The following tables provide the complete evaluation metrics underlying the main-paper results. Task completion (TC) measures the percentage of required downstream task objectives successfully completed within an evaluation episode, while full success (FS) is the percentage of episodes in which all required downstream tasks are completed. We also report episode return and failed execution (FE). Higher values are better for TC, FS, and return, whereas lower FE is better. MPE and RWARE values are reported as mean ± standard error over five independent training seeds and 200 final evaluation episodes per seed. A reported standard error of 0.000.00 indicates that the five seed-level evaluation means are exactly identical at full precision; further discussion is provided in Appendix D.7. D.1 MPE-Base Table D.1 reports the complete MPE-Base results across the five participation patterns, including task completion, full success, return, and failed execution. Table D.1: Complete MPE-Base four-agent results at 2M environment steps. Values are mean ± standard error over five independent training seeds and 200 final evaluation episodes per seed. Bold marks the best reported mean in each participation–metric row. For legacy MPE-Base Non-overlap HARL runs, FS is recovered from the retained per-episode completed-task counts using the same all-four-tasks definition. Participation Metric IPPO MAPPO HAPPO Other baselines Vanilla SPL Vanilla SPL Vanilla SPL MAE-MAPPO Sable SMPE2 Non-overlap TC (%) ↑ 70.50±0.0070.50± 0.00 78.38±0.3878.38± 0.38 70.50±0.0070.50± 0.00 78.73±0.4378.73± 0.43 67.45±3.0567.45± 3.05 79.18±0.2379.18± 0.23 70.50±0.0070.50± 0.00 62.70±0.2562.70± 0.25 42.73±9.2942.73± 9.29 FS (%) ↑ 17.50±0.0017.50± 0.00 31.80±0.8731.80± 0.87 17.50±0.0017.50± 0.00 32.80±1.0632.80± 1.06 14.00±3.5014.00± 3.50 34.20±0.4934.20± 0.49 17.50±0.0017.50± 0.00 9.00±0.719.00± 0.71 4.20±2.574.20± 2.57 Return ↑ 21.569±0.02921.569± 0.029 27.337±0.08327.337± 0.083 21.644±0.03621.644± 0.036 27.231±0.09127.231± 0.091 21.010±0.67521.010± 0.675 27.409±0.04827.409± 0.048 21.747±0.07621.747± 0.076 15.100±0.19915.100± 0.199 11.016±2.40911.016± 2.409 FE ↓ 1.125±0.0001.125± 0.000 0.582±0.0080.582± 0.008 1.127±0.0021.127± 0.002 0.610±0.0120.610± 0.012 1.056±0.0691.056± 0.069 0.608±0.0040.608± 0.004 1.125±0.0001.125± 0.000 1.450±0.0111.450± 0.011 0.873±0.2320.873± 0.232 Overlap-25 TC (%) ↑ 68.25±2.7568.25± 2.75 79.35±0.5779.35± 0.57 70.78±0.1470.78± 0.14 80.25±0.2180.25± 0.21 70.70±0.1870.70± 0.18 80.35±0.2980.35± 0.29 67.93±2.9567.93± 2.95 63.50±2.2463.50± 2.24 53.85±5.4353.85± 5.43 FS (%) ↑ 14.90±2.6014.90± 2.60 34.10±1.4534.10± 1.45 17.50±0.0017.50± 0.00 36.80±0.4636.80± 0.46 17.50±0.1617.50± 0.16 36.40±0.8136.40± 0.81 14.00±3.3814.00± 3.38 9.30±2.229.30± 2.22 4.30±2.534.30± 2.53 Return ↑ 20.308±1.00920.308± 1.009 27.068±0.21227.068± 0.212 21.521±0.08221.521± 0.082 27.232±0.13927.232± 0.139 21.439±0.17421.439± 0.174 27.358±0.12227.358± 0.122 20.950±0.55420.950± 0.554 15.295±1.86815.295± 1.868 12.778±2.12412.778± 2.124 FE ↓ 1.124±0.0361.124± 0.036 0.654±0.0090.654± 0.009 1.161±0.0071.161± 0.007 0.670±0.0140.670± 0.014 1.159±0.0101.159± 0.010 0.658±0.0170.658± 0.017 1.079±0.0791.079± 0.079 1.460±0.0901.460± 0.090 1.246±0.2361.246± 0.236 Overlap-50 TC (%) ↑ 67.85±3.1567.85± 3.15 78.90±0.3178.90± 0.31 69.53±1.2769.53± 1.27 78.90±0.2478.90± 0.24 69.80±0.9269.80± 0.92 78.50±0.4778.50± 0.47 70.63±0.2370.63± 0.23 64.43±1.8564.43± 1.85 63.95±3.3463.95± 3.34 FS (%) ↑ 14.00±3.5014.00± 3.50 32.20±0.6032.20± 0.60 15.70±1.3615.70± 1.36 31.90±0.7131.90± 0.71 16.20±0.9716.20± 0.97 30.70±1.1130.70± 1.11 17.50±0.1617.50± 0.16 10.30±2.1210.30± 2.12 10.60±3.2010.60± 3.20 Return ↑ 20.744±0.68020.744± 0.680 26.429±0.11226.429± 0.112 21.213±0.28721.213± 0.287 26.569±0.09826.569± 0.098 21.228±0.34021.228± 0.340 26.125±0.13426.125± 0.134 21.353±0.16621.353± 0.166 15.888±1.50315.888± 1.503 15.529±2.96615.529± 2.966 FE ↓ 1.086±0.0741.086± 0.074 0.731±0.0030.731± 0.003 1.136±0.0331.136± 0.033 0.715±0.0080.715± 0.008 1.148±0.0091.148± 0.009 0.748±0.0030.748± 0.003 1.161±0.0101.161± 0.010 1.422±0.0741.422± 0.074 1.442±0.1331.442± 0.133 Dynamic TC (%) ↑ 66.20±3.0166.20± 3.01 79.65±0.1879.65± 0.18 70.23±0.5970.23± 0.59 78.88±0.1878.88± 0.18 69.53±0.9969.53± 0.99 79.63±0.3979.63± 0.39 64.85±3.0664.85± 3.06 58.30±1.1158.30± 1.11 49.70±1.0449.70± 1.04 FS (%) ↑ 12.60±3.4312.60± 3.43 35.30±0.5135.30± 0.51 16.70±0.8016.70± 0.80 33.60±0.4333.60± 0.43 16.30±1.2016.30± 1.20 35.60±1.1435.60± 1.14 11.20±3.3911.20± 3.39 5.30±0.585.30± 0.58 0.60±0.600.60± 0.60 Return ↑ 20.694±0.60020.694± 0.600 27.643±0.09027.643± 0.090 21.364±0.23221.364± 0.232 27.306±0.12427.306± 0.124 21.153±0.36721.153± 0.367 27.499±0.14627.499± 0.146 20.613±0.61320.613± 0.613 13.970±0.83913.970± 0.839 12.579±1.59112.579± 1.591 FE ↓ 1.003±0.0791.003± 0.079 0.599±0.0090.599± 0.009 1.125±0.0101.125± 0.010 0.610±0.0120.610± 0.012 1.119±0.0151.119± 0.015 0.610±0.0140.610± 0.014 0.963±0.0770.963± 0.077 1.318±0.0381.318± 0.038 1.022±0.1391.022± 0.139 Cascaded TC (%) ↑ 66.83±3.6866.83± 3.68 78.90±0.7078.90± 0.70 70.63±0.1070.63± 0.10 78.93±0.4878.93± 0.48 70.68±0.1970.68± 0.19 78.78±0.2278.78± 0.22 70.45±0.0570.45± 0.05 38.03±0.9338.03± 0.93 27.93±5.7427.93± 5.74 FS (%) ↑ 14.00±3.5014.00± 3.50 33.70±1.7933.70± 1.79 17.50±0.0017.50± 0.00 33.50±0.8533.50± 0.85 17.40±0.1017.40± 0.10 33.00±0.5733.00± 0.57 17.50±0.0017.50± 0.00 0.50±0.320.50± 0.32 0.20±0.200.20± 0.20 Return ↑ 20.872±0.75320.872± 0.753 27.563±0.25927.563± 0.259 21.529±0.06021.529± 0.060 27.447±0.20727.447± 0.207 21.502±0.04821.502± 0.048 27.581±0.05927.581± 0.059 21.642±0.04721.642± 0.047 9.745±0.8619.745± 0.861 6.875±2.6686.875± 2.668 FE ↓ 1.040±0.0851.040± 0.085 0.577±0.0150.577± 0.015 1.138±0.0041.138± 0.004 0.590±0.0180.590± 0.018 1.144±0.0081.144± 0.008 0.578±0.0140.578± 0.014 1.127±0.0021.127± 0.002 0.850±0.0580.850± 0.058 0.650±0.0590.650± 0.059 Observation. Across all five participation patterns, SPL improves TC, FS, and return while reducing FE relative to the matched Vanilla backbone. The improvement is consistent across IPPO, MAPPO, and HAPPO, indicating that the additional gains are not specific to a single PPO-based backbone. D.2 MPE-Advanced Table D.2 reports the corresponding results in MPE-Advanced, where task assignments, spawn configurations, and future-collaborator identities vary across episodes. Table D.2: Complete MPE-Advanced four-agent results at 2M environment steps. Values are mean ± standard error over five independent training seeds and 200 final evaluation episodes per seed. Bold marks the best reported mean in each participation–metric row. Participation Metric IPPO MAPPO HAPPO Other baselines Vanilla SPL Vanilla SPL Vanilla SPL MAE-MAPPO Sable SMPE2 Non-overlap TC (%) ↑ 70.38±0.0070.38± 0.00 77.95±0.1977.95± 0.19 64.08±3.8664.08± 3.86 78.23±0.1678.23± 0.16 67.28±3.1067.28± 3.10 78.03±0.1678.03± 0.16 63.28±3.4763.28± 3.47 64.90±1.3664.90± 1.36 49.78±3.2549.78± 3.25 FS (%) ↑ 19.00±0.0019.00± 0.00 30.50±0.6530.50± 0.65 11.40±4.6511.40± 4.65 31.90±0.6231.90± 0.62 15.20±3.8015.20± 3.80 31.10±0.5331.10± 0.53 10.60±4.3510.60± 4.35 10.80±1.4110.80± 1.41 1.60±1.091.60± 1.09 Return ↑ 21.485±0.13521.485± 0.135 27.229±0.09227.229± 0.092 19.986±0.84719.986± 0.847 27.390±0.10627.390± 0.106 20.754±0.64520.754± 0.645 27.328±0.07627.328± 0.076 19.939±0.79819.939± 0.798 16.683±1.08516.683± 1.085 10.730±2.26910.730± 2.269 FE ↓ 1.125±0.0001.125± 0.000 0.589±0.0080.589± 0.008 0.989±0.0830.989± 0.083 0.579±0.0070.579± 0.007 1.056±0.0691.056± 0.069 0.580±0.0060.580± 0.006 0.971±0.0780.971± 0.078 1.369±0.0571.369± 0.057 1.238±0.2561.238± 0.256 Overlap-25 TC (%) ↑ 65.65±3.5365.65± 3.53 79.95±0.5279.95± 0.52 68.28±1.5068.28± 1.50 79.38±0.2779.38± 0.27 70.93±0.5670.93± 0.56 79.55±0.4079.55± 0.40 71.40±0.2371.40± 0.23 64.30±1.6364.30± 1.63 47.83±9.0247.83± 9.02 FS (%) ↑ 12.60±4.2812.60± 4.28 35.50±0.5735.50± 0.57 16.60±1.2716.60± 1.27 33.80±0.7333.80± 0.73 18.70±0.8018.70± 0.80 34.50±1.2134.50± 1.21 19.30±0.2019.30± 0.20 10.50±2.2010.50± 2.20 5.20±4.175.20± 4.17 Return ↑ 20.343±0.73420.343± 0.734 27.536±0.23527.536± 0.235 20.611±0.49420.611± 0.494 27.244±0.13927.244± 0.139 21.634±0.20521.634± 0.205 27.502±0.18527.502± 0.185 21.967±0.12021.967± 0.120 15.589±1.23815.589± 1.238 13.152±2.71813.152± 2.718 FE ↓ 1.011±0.0771.011± 0.077 0.622±0.0120.622± 0.012 1.101±0.0271.101± 0.027 0.641±0.0070.641± 0.007 1.126±0.0161.126± 0.016 0.611±0.0120.611± 0.012 1.131±0.0041.131± 0.004 1.427±0.0661.427± 0.066 0.901±0.2550.901± 0.255 Overlap-50 TC (%) ↑ 71.30±0.2271.30± 0.22 78.63±0.2378.63± 0.23 68.58±3.0568.58± 3.05 78.33±0.4078.33± 0.40 71.40±0.1471.40± 0.14 78.68±0.2578.68± 0.25 62.63±5.9262.63± 5.92 61.88±1.4861.88± 1.48 52.08±8.2552.08± 8.25 FS (%) ↑ 19.20±0.3019.20± 0.30 31.90±0.6031.90± 0.60 15.80±3.7015.80± 3.70 31.10±1.1131.10± 1.11 19.40±0.1019.40± 0.10 32.10±0.7632.10± 0.76 12.00±4.6012.00± 4.60 9.10±1.209.10± 1.20 6.10±4.236.10± 4.23 Return ↑ 21.669±0.08321.669± 0.083 26.481±0.08426.481± 0.084 21.005±0.70121.005± 0.701 26.358±0.17126.358± 0.171 21.567±0.09021.567± 0.090 26.446±0.09726.446± 0.097 19.743±1.44319.743± 1.443 13.716±1.26613.716± 1.266 13.018±3.41213.018± 3.412 FE ↓ 1.124±0.0071.124± 0.007 0.713±0.0070.713± 0.007 1.069±0.0661.069± 0.066 0.715±0.0100.715± 0.010 1.143±0.0061.143± 0.006 0.721±0.0050.721± 0.005 0.958±0.1150.958± 0.115 1.525±0.0591.525± 0.059 1.139±0.0671.139± 0.067 Dynamic TC (%) ↑ 66.63±3.6666.63± 3.66 78.23±0.2978.23± 0.29 67.05±3.2067.05± 3.20 79.08±0.5479.08± 0.54 61.73±5.5561.73± 5.55 78.80±0.1878.80± 0.18 67.68±1.7667.68± 1.76 57.60±1.3057.60± 1.30 43.10±6.6143.10± 6.61 FS (%) ↑ 14.70±3.6814.70± 3.68 32.30±0.6432.30± 0.64 14.70±3.6814.70± 3.68 34.10±1.3834.10± 1.38 11.10±4.5311.10± 4.53 34.20±0.6434.20± 0.64 15.60±2.4215.60± 2.42 6.50±1.106.50± 1.10 0.00±0.000.00± 0.00 Return ↑ 20.509±0.89320.509± 0.893 27.006±0.13127.006± 0.131 20.770±0.68120.770± 0.681 27.312±0.25827.312± 0.258 19.867±1.21519.867± 1.215 27.330±0.08127.330± 0.081 20.936±0.42020.936± 0.420 14.027±1.01814.027± 1.018 13.678±2.57713.678± 2.577 FE ↓ 1.037±0.0721.037± 0.072 0.608±0.0050.608± 0.005 1.047±0.0641.047± 0.064 0.608±0.0130.608± 0.013 0.915±0.1230.915± 0.123 0.596±0.0120.596± 0.012 1.047±0.0381.047± 0.038 1.272±0.0521.272± 0.052 0.608±0.0830.608± 0.083 Cascaded TC (%) ↑ 67.68±2.5867.68± 2.58 76.80±0.1376.80± 0.13 67.28±3.1067.28± 3.10 77.00±0.3377.00± 0.33 69.28±1.1069.28± 1.10 77.23±0.1877.23± 0.18 66.83±3.0266.83± 3.02 62.73±0.9862.73± 0.98 56.70±5.5056.70± 5.50 FS (%) ↑ 15.90±3.1015.90± 3.10 29.00±0.4529.00± 0.45 15.20±3.8015.20± 3.80 29.20±0.6829.20± 0.68 17.80±1.2017.80± 1.20 30.70±0.7230.70± 0.72 14.80±3.7214.80± 3.72 8.30±1.668.30± 1.66 7.10±3.727.10± 3.72 Return ↑ 20.895±0.63820.895± 0.638 25.976±0.07225.976± 0.072 20.623±0.66120.623± 0.661 25.988±0.14325.988± 0.143 21.038±0.23821.038± 0.238 26.033±0.10026.033± 0.100 20.707±0.66220.707± 0.662 14.706±0.81114.706± 0.811 18.650±1.42318.650± 1.423 FE ↓ 1.065±0.0551.065± 0.055 0.734±0.0060.734± 0.006 1.056±0.0691.056± 0.069 0.742±0.0070.742± 0.007 1.101±0.0241.101± 0.024 0.741±0.0070.741± 0.007 1.043±0.0671.043± 0.067 1.457±0.0351.457± 0.035 0.795±0.1080.795± 0.108 Observation. The same pattern persists under the additional structural variation of MPE-Advanced. SPL improves all four reported metrics for every matched backbone–participation comparison, while the three SPL backbones remain close in final performance. D.3 RWARE-Base Table D.3 reports the complete RWARE-Base results across the five participation patterns. Table D.3: Complete RWARE-Base four-agent results at 2M environment steps. Values are mean ± standard error over five independent training seeds and 200 final evaluation episodes per seed. Bold marks the best reported mean in each participation–metric row. Participation Metric IPPO MAPPO HAPPO Other baselines Vanilla SPL Vanilla SPL Vanilla SPL MAE-MAPPO Sable SMPE2 Non-overlap TC (%) ↑ 66.45±5.8666.45± 5.86 86.83±0.4086.83± 0.40 71.38±2.2771.38± 2.27 86.70±0.4986.70± 0.49 73.18±1.5073.18± 1.50 86.93±0.2686.93± 0.26 73.48±1.6973.48± 1.69 71.80±0.9871.80± 0.98 66.05±3.2966.05± 3.29 FS (%) ↑ 14.50±6.2714.50± 6.27 53.80±1.6853.80± 1.68 19.50±4.1419.50± 4.14 53.70±1.4653.70± 1.46 19.20±3.0919.20± 3.09 54.00±0.5754.00± 0.57 21.30±2.2821.30± 2.28 20.00±1.3020.00± 1.30 14.10±3.6114.10± 3.61 Return ↑ 2.342±0.2262.342± 0.226 3.292±0.0183.292± 0.018 2.548±0.1012.548± 0.101 3.286±0.0233.286± 0.023 2.618±0.0702.618± 0.070 3.297±0.0103.297± 0.010 2.641±0.0722.641± 0.072 2.471±0.0492.471± 0.049 2.301±0.1642.301± 0.164 FE ↓ 0.848±0.0880.848± 0.088 0.494±0.0130.494± 0.013 1.075±0.1101.075± 0.110 0.503±0.0190.503± 0.019 1.064±0.0571.064± 0.057 0.483±0.0110.483± 0.011 1.003±0.0681.003± 0.068 1.128±0.0391.128± 0.039 1.358±0.1311.358± 0.131 Overlap-25 TC (%) ↑ 81.10±3.4181.10± 3.41 89.15±0.1689.15± 0.16 74.25±3.4174.25± 3.41 89.08±0.3589.08± 0.35 70.15±2.9370.15± 2.93 88.35±0.5088.35± 0.50 72.50±2.8672.50± 2.86 76.70±1.1876.70± 1.18 67.28±2.8767.28± 2.87 FS (%) ↑ 38.60±7.3038.60± 7.30 61.00±0.3261.00± 0.32 25.90±7.1825.90± 7.18 60.60±0.9460.60± 0.94 16.60±4.2016.60± 4.20 58.80±1.9558.80± 1.95 19.20±5.3519.20± 5.35 28.10±3.0328.10± 3.03 16.00±2.8316.00± 2.83 Return ↑ 2.960±0.1572.960± 0.157 3.404±0.0083.404± 0.008 2.667±0.1572.667± 0.157 3.401±0.0183.401± 0.018 2.500±0.1312.500± 0.131 3.369±0.0233.369± 0.023 2.633±0.1052.633± 0.105 2.720±0.0562.720± 0.056 2.358±0.1452.358± 0.145 FE ↓ 0.728±0.1420.728± 0.142 0.411±0.0110.411± 0.011 0.935±0.1650.935± 0.165 0.410±0.0180.410± 0.018 1.075±0.0921.075± 0.092 0.424±0.0140.424± 0.014 0.877±0.1140.877± 0.114 0.932±0.0470.932± 0.047 1.309±0.1151.309± 0.115 Overlap-50 TC (%) ↑ 82.00±1.1282.00± 1.12 89.00±0.1989.00± 0.19 74.73±3.3574.73± 3.35 88.58±0.2888.58± 0.28 70.28±2.4170.28± 2.41 88.80±0.2388.80± 0.23 77.53±0.7277.53± 0.72 77.23±1.9777.23± 1.97 61.95±4.0961.95± 4.09 FS (%) ↑ 41.80±1.7441.80± 1.74 59.40±0.6859.40± 0.68 24.60±6.5524.60± 6.55 59.30±0.9459.30± 0.94 15.20±3.0315.20± 3.03 59.70±0.8059.70± 0.80 30.20±1.3230.20± 1.32 31.20±4.4731.20± 4.47 10.10±4.1510.10± 4.15 Return ↑ 3.008±0.0523.008± 0.052 3.397±0.0103.397± 0.010 2.706±0.1532.706± 0.153 3.375±0.0133.375± 0.013 2.487±0.1122.487± 0.112 3.388±0.0113.388± 0.011 2.834±0.0332.834± 0.033 2.741±0.0992.741± 0.099 2.143±0.1752.143± 0.175 FE ↓ 0.686±0.0370.686± 0.037 0.414±0.0100.414± 0.010 0.979±0.1440.979± 0.144 0.432±0.0090.432± 0.009 1.184±0.0991.184± 0.099 0.415±0.0090.415± 0.009 0.858±0.0390.858± 0.039 0.911±0.0790.911± 0.079 1.322±0.1501.322± 0.150 Dynamic TC (%) ↑ 72.88±1.1972.88± 1.19 88.88±0.2788.88± 0.27 65.35±5.4665.35± 5.46 88.70±0.3088.70± 0.30 68.80±4.0668.80± 4.06 88.40±0.4288.40± 0.42 72.28±1.5172.28± 1.51 74.63±1.2274.63± 1.22 62.03±4.2962.03± 4.29 FS (%) ↑ 19.30±1.5619.30± 1.56 60.10±0.8160.10± 0.81 14.50±5.1414.50± 5.14 58.70±1.2058.70± 1.20 16.30±4.8916.30± 4.89 58.40±1.3758.40± 1.37 18.90±2.6218.90± 2.62 24.70±2.7224.70± 2.72 10.20±4.0710.20± 4.07 Return ↑ 2.579±0.0412.579± 0.041 3.389±0.0133.389± 0.013 2.304±0.2112.304± 0.211 3.385±0.0143.385± 0.014 2.425±0.1992.425± 0.199 3.368±0.0203.368± 0.020 2.572±0.0722.572± 0.072 2.616±0.0582.616± 0.058 2.095±0.2152.095± 0.215 FE ↓ 0.910±0.1180.910± 0.118 0.423±0.0070.423± 0.007 1.119±0.1291.119± 0.129 0.415±0.0060.415± 0.006 1.166±0.2031.166± 0.203 0.438±0.0140.438± 0.014 1.106±0.0611.106± 0.061 1.015±0.0491.015± 0.049 1.519±0.1721.519± 0.172 Cascaded TC (%) ↑ 66.80±1.9866.80± 1.98 86.25±0.3886.25± 0.38 67.53±2.0567.53± 2.05 86.75±0.5086.75± 0.50 66.58±2.2466.58± 2.24 85.95±0.7985.95± 0.79 71.63±0.7371.63± 0.73 76.83±0.9376.83± 0.93 64.18±2.9464.18± 2.94 FS (%) ↑ 13.20±2.8313.20± 2.83 52.40±1.0752.40± 1.07 11.80±3.6411.80± 3.64 53.60±1.0253.60± 1.02 11.90±2.6411.90± 2.64 51.50±1.6751.50± 1.67 18.70±0.9718.70± 0.97 28.80±1.9728.80± 1.97 11.00±3.6711.00± 3.67 Return ↑ 2.346±0.0972.346± 0.097 3.281±0.0143.281± 0.014 2.417±0.0742.417± 0.074 3.299±0.0233.299± 0.023 2.322±0.1102.322± 0.110 3.269±0.0323.269± 0.032 2.573±0.0312.573± 0.031 2.721±0.0472.721± 0.047 2.207±0.1462.207± 0.146 FE ↓ 1.298±0.0791.298± 0.079 0.456±0.0120.456± 0.012 1.095±0.1411.095± 0.141 0.466±0.0110.466± 0.011 1.333±0.0881.333± 0.088 0.448±0.0080.448± 0.008 1.135±0.0291.135± 0.029 0.927±0.0370.927± 0.037 1.433±0.1181.433± 0.118 Observation. SPL produces larger gains in RWARE than in MPE, particularly in FS and FE. The increase in full success indicates that higher task completion is accompanied by more episodes in which the cooperative objective is completed in full. D.4 RWARE-Advanced Table D.4 reports the results in RWARE-Advanced, which additionally includes task permutation, a shared candidate pool, and dynamic task claims. Table D.4: Complete RWARE-Advanced four-agent results at 2M environment steps. Values are mean ± standard error over five independent training seeds and 200 final evaluation episodes per seed. Bold marks the best reported mean in each participation–metric row. Participation Metric IPPO MAPPO HAPPO Other baselines Vanilla SPL Vanilla SPL Vanilla SPL MAE-MAPPO Sable SMPE2 Non-overlap TC (%) ↑ 52.08±1.3752.08± 1.37 79.93±0.5979.93± 0.59 58.63±2.3158.63± 2.31 80.03±0.2380.03± 0.23 57.58±3.4457.58± 3.44 79.55±0.7979.55± 0.79 58.50±2.2058.50± 2.20 50.70±0.3250.70± 0.32 45.70±0.2545.70± 0.25 FS (%) ↑ 2.60±0.582.60± 0.58 40.60±0.8740.60± 0.87 4.70±0.934.70± 0.93 42.20±1.4442.20± 1.44 5.70±1.795.70± 1.79 38.70±1.7138.70± 1.71 4.30±1.194.30± 1.19 1.60±0.191.60± 0.19 1.20±0.621.20± 0.62 Return ↑ 1.654±0.0591.654± 0.059 2.950±0.0262.950± 0.026 1.972±0.0961.972± 0.096 2.951±0.0102.951± 0.010 1.891±0.1441.891± 0.144 2.928±0.0302.928± 0.030 1.951±0.0991.951± 0.099 1.510±0.0171.510± 0.017 1.346±0.0221.346± 0.022 FE ↓ 1.573±0.0431.573± 0.043 0.577±0.0130.577± 0.013 1.174±0.0981.174± 0.098 0.583±0.0110.583± 0.011 1.386±0.1151.386± 0.115 0.599±0.0200.599± 0.020 1.330±0.1191.330± 0.119 1.660±0.0191.660± 0.019 1.882±0.0251.882± 0.025 Overlap-25 TC (%) ↑ 64.18±1.6264.18± 1.62 83.23±0.4783.23± 0.47 68.83±1.4068.83± 1.40 83.65±0.3283.65± 0.32 63.28±1.9863.28± 1.98 83.73±0.4883.73± 0.48 64.70±1.8164.70± 1.81 54.75±0.4454.75± 0.44 47.68±0.6647.68± 0.66 FS (%) ↑ 7.00±0.847.00± 0.84 42.30±0.9442.30± 0.94 11.60±1.2711.60± 1.27 44.50±1.0144.50± 1.01 7.60±1.647.60± 1.64 43.50±2.1143.50± 2.11 8.40±1.718.40± 1.71 1.80±0.341.80± 0.34 1.00±0.421.00± 0.42 Return ↑ 2.175±0.0622.175± 0.062 3.069±0.0233.069± 0.023 2.360±0.0662.360± 0.066 3.092±0.0163.092± 0.016 2.122±0.0862.122± 0.086 3.095±0.0203.095± 0.020 2.193±0.0872.193± 0.087 1.636±0.0211.636± 0.021 1.331±0.0451.331± 0.045 FE ↓ 1.316±0.0541.316± 0.054 0.590±0.0170.590± 0.017 1.159±0.0611.159± 0.061 0.567±0.0180.567± 0.018 1.368±0.0731.368± 0.073 0.566±0.0110.566± 0.011 1.256±0.1001.256± 0.100 1.770±0.0131.770± 0.013 2.007±0.0392.007± 0.039 Overlap-50 TC (%) ↑ 70.35±1.2870.35± 1.28 84.55±0.6984.55± 0.69 68.05±4.1468.05± 4.14 84.45±0.3384.45± 0.33 67.50±2.1267.50± 2.12 84.43±0.5684.43± 0.56 72.88±1.0572.88± 1.05 58.88±0.7758.88± 0.77 47.05±0.3847.05± 0.38 FS (%) ↑ 13.40±1.5113.40± 1.51 45.90±1.9245.90± 1.92 12.40±3.6712.40± 3.67 45.20±0.7745.20± 0.77 11.70±3.1411.70± 3.14 45.10±1.7845.10± 1.78 17.50±1.3117.50± 1.31 4.90±0.334.90± 0.33 0.80±0.490.80± 0.49 Return ↑ 2.437±0.0562.437± 0.056 3.124±0.0343.124± 0.034 2.344±0.1842.344± 0.184 3.122±0.0173.122± 0.017 2.327±0.1042.327± 0.104 3.120±0.0243.120± 0.024 2.566±0.0512.566± 0.051 1.836±0.0391.836± 0.039 1.340±0.0151.340± 0.015 FE ↓ 1.118±0.0561.118± 0.056 0.587±0.0260.587± 0.026 1.209±0.1561.209± 0.156 0.574±0.0150.574± 0.015 1.141±0.1241.141± 0.124 0.574±0.0110.574± 0.011 1.002±0.0371.002± 0.037 1.629±0.0351.629± 0.035 2.054±0.0202.054± 0.020 Dynamic TC (%) ↑ 66.60±1.9566.60± 1.95 81.30±0.3481.30± 0.34 62.55±2.8362.55± 2.83 82.50±0.4382.50± 0.43 63.23±0.8363.23± 0.83 82.20±0.1682.20± 0.16 65.90±2.3365.90± 2.33 56.30±0.4956.30± 0.49 47.55±0.4047.55± 0.40 FS (%) ↑ 11.60±2.4311.60± 2.43 39.30±0.6439.30± 0.64 8.50±2.328.50± 2.32 42.80±0.7242.80± 0.72 7.90±0.937.90± 0.93 41.20±0.7841.20± 0.78 11.10±1.8711.10± 1.87 3.80±0.643.80± 0.64 0.60±0.400.60± 0.40 Return ↑ 2.282±0.0842.282± 0.084 2.995±0.0142.995± 0.014 2.090±0.1242.090± 0.124 3.048±0.0183.048± 0.018 2.162±0.0412.162± 0.041 3.031±0.0063.031± 0.006 2.269±0.0982.269± 0.098 1.713±0.0241.713± 0.024 1.353±0.0351.353± 0.035 FE ↓ 1.122±0.0771.122± 0.077 0.590±0.0040.590± 0.004 1.351±0.1141.351± 0.114 0.564±0.0140.564± 0.014 1.207±0.0461.207± 0.046 0.586±0.0150.586± 0.015 1.147±0.0861.147± 0.086 1.705±0.0251.705± 0.025 2.054±0.0312.054± 0.031 Cascaded TC (%) ↑ 58.95±1.7058.95± 1.70 85.40±0.4285.40± 0.42 65.55±1.7665.55± 1.76 85.98±0.4485.98± 0.44 66.03±2.0666.03± 2.06 84.90±0.5184.90± 0.51 64.18±3.9864.18± 3.98 54.90±1.0854.90± 1.08 47.03±0.5547.03± 0.55 FS (%) ↑ 4.10±0.874.10± 0.87 46.00±1.1746.00± 1.17 8.60±1.768.60± 1.76 48.30±1.5448.30± 1.54 9.50±2.139.50± 2.13 44.80±1.8544.80± 1.85 9.40±2.509.40± 2.50 2.70±0.372.70± 0.37 0.60±0.190.60± 0.19 Return ↑ 1.936±0.0761.936± 0.076 3.153±0.0213.153± 0.021 2.194±0.0782.194± 0.078 3.182±0.0213.182± 0.021 2.226±0.0932.226± 0.093 3.126±0.0263.126± 0.026 2.150±0.1672.150± 0.167 1.625±0.0541.625± 0.054 1.339±0.0351.339± 0.035 FE ↓ 1.539±0.1151.539± 0.115 0.573±0.0170.573± 0.017 1.368±0.0701.368± 0.070 0.547±0.0140.547± 0.014 1.317±0.0771.317± 0.077 0.599±0.0210.599± 0.021 1.331±0.1301.331± 0.130 1.804±0.0431.804± 0.043 2.119±0.0222.119± 0.022 Observation. The greatest improvements appear in RWARE-Advanced, where downstream decisions depend more strongly on the available task-relevant evidence. SPL increases TC, FS, and return and reduces FE across every matched backbone and participation pattern. D.5 Additional Isaac Lab Metrics Table D.5 reports the complete Isaac Lab evaluation, including task completion (TC), full success (FS), episode return, failed execution (FE), map coverage, task discovery, and wall-penetration steps. These additional diagnostics complement the TC results reported in the main paper and provide a broader view of the behaviors induced by SPL-MAPPO under embodied UAV–UGV execution. Table D.5: Isaac Lab evaluation metrics. Values are mean ± standard error over three independent training seeds and 200 final evaluation episodes per seed. Agents Metric Non-overlap Dynamic MAPPO SPL-MAPPO MAPPO SPL-MAPPO 2 TC (%) ↑ 31.50±5.9731.50± 5.97 61.58±0.6761.58± 0.67 36.67±0.7136.67± 0.71 59.58±1.7959.58± 1.79 FS (%) ↑ 5.17±3.035.17± 3.03 35.17±0.7335.17± 0.73 9.00±0.769.00± 0.76 34.50±2.2934.50± 2.29 Return ↑ 0.648±0.1470.648± 0.147 1.540±0.0161.540± 0.016 0.785±0.0220.785± 0.022 1.494±0.0591.494± 0.059 FE ↓ 0.632±0.0570.632± 0.057 0.822±0.0090.822± 0.009 0.798±0.0030.798± 0.003 0.760±0.0380.760± 0.038 Map coverage (%) ↑ 36.21±1.6936.21± 1.69 44.03±0.1744.03± 0.17 33.21±1.6033.21± 1.60 44.03±1.0744.03± 1.07 Tasks discovered ↑ 0.238±0.0090.238± 0.009 0.287±0.0030.287± 0.003 0.150±0.0200.150± 0.020 0.261±0.0160.261± 0.016 Wall-penetration steps ↓ 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 4 TC (%) ↑ 55.79±3.2355.79± 3.23 76.38±1.5276.38± 1.52 58.46±3.4058.46± 3.40 79.25±1.3079.25± 1.30 FS (%) ↑ 11.83±4.6911.83± 4.69 39.83±3.2239.83± 3.22 12.17±4.2112.17± 4.21 44.83±1.1744.83± 1.17 Return ↑ 2.235±0.1722.235± 0.172 3.330±0.0923.330± 0.092 2.343±0.1742.343± 0.174 3.499±0.0643.499± 0.064 FE ↓ 1.100±0.0161.100± 0.016 1.167±0.0271.167± 0.027 1.160±0.0231.160± 0.023 1.193±0.0201.193± 0.020 Map coverage (%) ↑ 67.96±1.4567.96± 1.45 54.90±0.3654.90± 0.36 69.25±1.5969.25± 1.59 53.84±0.7653.84± 0.76 Tasks discovered ↑ 0.470±0.0220.470± 0.022 0.200±0.0070.200± 0.007 0.512±0.0410.512± 0.041 0.190±0.0090.190± 0.009 Wall-penetration steps ↓ 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 0.000±0.0000.000± 0.000 Task performance. SPL-MAPPO achieves higher TC, FS, and return in all four team-size–participation settings. For the two-agent system, TC increases from 31.50%31.50\% to 61.58%61.58\% under Non-overlap and from 36.67%36.67\% to 59.58%59.58\% under Dynamic participation. The corresponding FS gains are also substantial, increasing from 5.17%5.17\% to 35.17%35.17\% and from 9.00%9.00\% to 34.50%34.50\%. The same pattern persists with four agents: TC increases from 55.79%55.79\% to 76.38%76.38\% under Non-overlap and from 58.46%58.46\% to 79.25%79.25\% under Dynamic participation, while FS increases from 11.83%11.83\% to 39.83%39.83\% and from 12.17%12.17\% to 44.83%44.83\%. Episode return follows the same direction in all four settings. Execution and exploration diagnostics. FE does not change uniformly with SPL. In the two-agent setting it is higher under Non-overlap and slightly lower under Dynamic participation, while in the four-agent setting it is slightly higher under both schedules. These absolute FE values occur under substantially different exploration and task-discovery behavior and should therefore be interpreted together with those diagnostics. The exploration pattern also differs with team size. With two agents, SPL-MAPPO attains greater map coverage and discovers more tasks under both participation schedules. For example, under Dynamic participation, map coverage increases from 33.21%33.21\% to 44.03%44.03\%, while task discovery increases from 0.1500.150 to 0.2610.261. This pattern is consistent with broader evidence acquisition. With four agents, SPL-MAPPO exhibits the opposite exploration pattern: map coverage decreases from 67.96%67.96\% to 54.90%54.90\% under Non-overlap and from 69.25%69.25\% to 53.84%53.84\% under Dynamic participation, while task discovery also decreases. Despite this reduced exploration, SPL-MAPPO achieves higher TC, FS, and return in both four-agent settings. This is consistent with more selective acquisition and/or more effective use of the available task-relevant evidence, although the reported diagnostics do not separate these effects causally. No wall-penetration steps are observed for either method in any of the four settings, indicating that the performance differences reported above are not accompanied by violations of this physical-safety diagnostic. D.6 Prior-Misspecification Robustness SPL uses task-family validity priors as part of the structured model for prospective acquisition supervision. We therefore evaluate how sensitive SPL is to misspecification of these priors. The true environment distribution is kept unchanged, and only the validity priors assumed by the acquisition model are perturbed. Let =(0.80,0.80,0.65,0.65,0.35,0.35,0.20,0.20)p=(0.80,0.80,0.65,0.65,0.35,0.35,0.20,0.20) denote the nominal prior vector. For misspecification level λ∈[0,1]λ∈[0,1], the acquisition model instead uses ^λ=(1−λ)+λ(1−). p_λ=(1-λ)p+λ(1-p). (D.1) Thus, λ=0λ=0 recovers the nominal priors, λ=0.5λ=0.5 produces an uninformative prior of 0.50.5 for every task, and λ=1λ=1 fully reverses the nominal prior ordering. All other structured information, including the fixed-valid-task constraint, probe observation model, task utilities, environment dynamics, and receiver learning procedure, remains unchanged. We evaluate SPL-MAPPO under Non-overlap participation in MPE-Advanced and RWARE-Advanced. Each misspecified condition uses five independent training seeds, two million environment steps, and 200 final evaluation episodes per seed, matching the primary four-agent protocol. The λ=0λ=0 values correspond to the nominal SPL-MAPPO results reported in the main experiments; the λ∈0.25,0.50,1.00λ∈\0.25,0.50,1.00\ conditions comprise 30 additional training runs. In all misspecified runs, the environment continues to sample tasks from the nominal prior distribution in p. Table D.6: Sensitivity of SPL-MAPPO to misspecified validity priors under Non-overlap participation. Values are mean ± standard error over five training seeds and 200 final evaluation episodes per seed. The environment always uses the nominal prior distribution; only the acquisition model uses ^λ p_λ. Environment λ TC (%) ↑ FS (%) ↑ Return ↑ FE ↓ MPE-Advanced 0.000.00 78.23±0.1678.23± 0.16 31.90±0.6231.90± 0.62 27.390±0.10627.390± 0.106 0.579±0.0070.579± 0.007 0.250.25 77.70±0.3977.70± 0.39 30.70±0.8030.70± 0.80 27.02±0.1527.02± 0.15 0.560±0.0060.560± 0.006 0.500.50 76.43±0.1176.43± 0.11 29.90±0.3329.90± 0.33 26.70±0.1026.70± 0.10 0.557±0.0090.557± 0.009 1.001.00 76.73±0.2176.73± 0.21 29.80±0.7229.80± 0.72 26.59±0.0526.59± 0.05 0.612±0.0070.612± 0.007 RWARE-Advanced 0.000.00 80.03±0.2380.03± 0.23 42.20±1.4442.20± 1.44 2.951±0.0102.951± 0.010 0.583±0.0110.583± 0.011 0.250.25 79.93±0.6879.93± 0.68 39.70±1.5939.70± 1.59 2.947±0.0282.947± 0.028 0.586±0.0110.586± 0.011 0.500.50 78.35±0.9478.35± 0.94 33.90±1.4333.90± 1.43 2.888±0.0402.888± 0.040 0.666±0.0230.666± 0.023 1.001.00 77.45±0.5677.45± 0.56 35.50±1.1535.50± 1.15 2.826±0.0232.826± 0.023 0.652±0.0070.652± 0.007 Observation. Prior misspecification reduces performance relative to the nominal SPL configuration, but does not cause a collapse in task completion in either tested environment. With fully reversed priors (λ=1λ=1), SPL-MAPPO achieves 76.73%76.73\% TC in MPE-Advanced and 77.45%77.45\% in RWARE-Advanced, compared with nominal values of 78.23%78.23\% and 80.03%80.03\%, respectively. These values also remain above the corresponding Vanilla MAPPO means of 64.08%64.08\% and 58.63%58.63\% reported in the main experiments. The effect is not strictly monotonic in MPE-Advanced, where TC at λ=1λ=1 is slightly higher than at λ=0.5λ=0.5. RWARE-Advanced shows a clearer reduction in TC as the assumed prior becomes less accurate, while full success is more sensitive than TC: FS decreases from 42.20%42.20\% in the nominal setting to 33.90%33.90\% at λ=0.5λ=0.5 and 35.50%35.50\% at λ=1λ=1. These results indicate that SPL retains substantial task-completion performance under validity-prior misspecification in the tested settings, while stricter episode-level success can be more sensitive. This experiment perturbs only the validity priors used by the acquisition model. The fixed-valid-task constraint, probe accuracy, task utilities, and other elements of the structured task model remain correctly specified. Accordingly, the results provide evidence of tolerance to validity-prior misspecification in the tested settings. Robustness to arbitrary errors in the full structured task model is not established. D.7 Statistical Analysis of Primary Comparisons We conduct a retrospective statistical analysis of the 60 primary Vanilla–SPL comparisons reported in the main paper. Each trained policy is summarized by its mean task completion (TC) over 200 final evaluation episodes, and independent training seeds are treated as the statistical replication units. Throughout this subsection, ΔTC denotes the absolute difference between percentage-valued SPL and Vanilla TC means, TCSPL−TCVanillaTC_SPL-TC_Vanilla, and is reported using %. Same-training-seed pairing is available for 57 of the 60 configurations, providing five paired seed-level differences in each testable configuration. For these comparisons, we report the mean paired difference, a two-sided paired Student t test with four degrees of freedom, and a pointwise 95% t confidence interval. The Non-overlap MAPPO comparisons in MPE-Advanced, RWARE-Base, and RWARE-Advanced use different Vanilla and SPL training-seed sets. Their observed five-seed method means remain available for descriptive comparison, but no paired confidence interval or paired p-value is reported. To account for multiplicity, we apply Holm correction over the declared family of all 60 primary comparisons at familywise level 0.050.05. The three configurations without verified seed pairing are assigned p=1p=1 internally for the correction and are displayed as unavailable below. This preserves the original 60-comparison family and yields a conservative adjustment. Pointwise confidence intervals are not multiplicity-adjusted. Table D.7: Retrospective statistical analysis for MPE-Base. Each training seed is a replication unit. Δ is the absolute SPL-minus-Vanilla difference in percentage-valued TC scores. Confidence intervals are pointwise 95% intervals from five paired training seeds. Holm p values are adjusted over the declared family of 60 primary comparisons. Participation Backbone Δ (%) 95% CI (%) Raw p Holm p Non-overlap IPPO 7.88 [6.82, 8.93] 3.17×10−53.17× 10^-5 0.0017 MAPPO 8.23 [7.03, 9.42] 4.48×10−54.48× 10^-5 0.0023 HAPPO 11.73 [3.01, 20.44] 0.0202 0.3028 Overlap-25 IPPO 11.10 [2.11, 20.09] 0.0266 0.3073 MAPPO 9.48 [8.74, 10.21] 3.56×10−63.56× 10^-6 2.10×10−42.10× 10^-4 HAPPO 9.65 [8.86, 10.44] 4.61×10−64.61× 10^-6 2.67×10−42.67× 10^-4 Overlap-50 IPPO 11.05 [2.81, 19.29] 0.0204 0.3028 MAPPO 9.38 [5.41, 13.34] 0.0028 0.0752 HAPPO 8.70 [6.28, 11.12] 5.65×10−45.65× 10^-4 0.0237 Dynamic IPPO 13.45 [5.36, 21.54] 0.0099 0.2081 MAPPO 8.65 [7.41, 9.89] 4.24×10−54.24× 10^-5 0.0022 HAPPO 10.10 [6.97, 13.23] 8.59×10−48.59× 10^-4 0.0326 Cascaded IPPO 12.08 [2.41, 21.74] 0.0256 0.3073 MAPPO 8.30 [7.06, 9.54] 5.01×10−55.01× 10^-5 0.0025 HAPPO 8.10 [7.25, 8.95] 1.23×10−51.23× 10^-5 7.01×10−47.01× 10^-4 Table D.8: Retrospective statistical analysis for MPE-Advanced. Notation and statistical procedure follow Table D.7. The Non-overlap MAPPO comparison uses different Vanilla and SPL training-seed sets; its Δ is therefore descriptive and paired inference is unavailable. Participation Backbone Δ (%) 95% CI (%) Raw p Holm p Non-overlap IPPO 7.58 [7.05, 8.10] 2.26×10−62.26× 10^-6 1.36×10−41.36× 10^-4 MAPPO 14.15 – – – HAPPO 10.75 [2.06, 19.44] 0.0264 0.3073 Overlap-25 IPPO 14.30 [5.25, 23.35] 0.0118 0.2358 MAPPO 11.10 [7.48, 14.72] 0.0010 0.0378 HAPPO 8.63 [6.13, 11.12] 6.59×10−46.59× 10^-4 0.0264 Overlap-50 IPPO 7.33 [6.43, 8.22] 2.23×10−52.23× 10^-5 0.0012 MAPPO 9.75 [1.26, 18.24] 0.0332 0.3073 HAPPO 7.28 [6.31, 8.24] 3.09×10−53.09× 10^-5 0.0017 Dynamic IPPO 11.60 [0.73, 22.47] 0.0414 0.3073 MAPPO 12.03 [3.78, 20.27] 0.0155 0.2784 HAPPO 17.08 [1.68, 32.47] 0.0370 0.3073 Cascaded IPPO 9.13 [1.73, 16.52] 0.0266 0.3073 MAPPO 9.73 [0.98, 18.47] 0.0366 0.3073 HAPPO 7.95 [5.10, 10.80] 0.0015 0.0492 – indicates that paired inference is unavailable because Vanilla and SPL use different training-seed sets. The corresponding untestable comparison is assigned p=1p=1 internally for the 60-comparison Holm correction. Table D.9: Retrospective statistical analysis for RWARE-Base. Notation and statistical procedure follow Table D.7. The Non-overlap MAPPO comparison uses different Vanilla and SPL training-seed sets; its Δ is descriptive and paired inference is unavailable. Participation Backbone Δ (%) 95% CI (%) Raw p Holm p Non-overlap IPPO 20.38 [4.50, 36.25] 0.0235 0.3056 MAPPO 15.33 – – – HAPPO 13.75 [9.92, 17.58] 5.67×10−45.67× 10^-4 0.0237 Overlap-25 IPPO 8.05 [-1.30, 17.40] 0.0751 0.3073 MAPPO 14.83 [6.33, 23.32] 0.0084 0.1842 HAPPO 18.20 [9.60, 26.80] 0.0042 0.1049 Overlap-50 IPPO 7.00 [3.73, 10.27] 0.0040 0.1043 MAPPO 13.85 [4.26, 23.44] 0.0160 0.2784 HAPPO 18.53 [12.18, 24.87] 0.0013 0.0440 Dynamic IPPO 16.00 [12.30, 19.70] 2.76×10−42.76× 10^-4 0.0124 MAPPO 23.35 [7.88, 38.82] 0.0138 0.2624 HAPPO 19.60 [8.63, 30.57] 0.0077 0.1774 Cascaded IPPO 19.45 [13.18, 25.72] 0.0010 0.0371 MAPPO 19.23 [12.63, 25.82] 0.0013 0.0440 HAPPO 19.38 [11.51, 27.24] 0.0024 0.0693 Table D.10: Retrospective statistical analysis for RWARE-Advanced. Notation and statistical procedure follow Table D.7. The Non-overlap MAPPO comparison uses different Vanilla and SPL training-seed sets; its Δ is descriptive and paired inference is unavailable. Participation Backbone Δ (%) 95% CI (%) Raw p Holm p Non-overlap IPPO 27.85 [23.14, 32.56] 8.06×10−58.06× 10^-5 0.0040 MAPPO 21.40 – – – HAPPO 21.98 [11.18, 32.77] 0.0048 0.1160 Overlap-25 IPPO 19.05 [14.95, 23.15] 2.08×10−42.08× 10^-4 0.0096 MAPPO 14.83 [11.13, 18.52] 3.71×10−43.71× 10^-4 0.0163 HAPPO 20.45 [16.11, 24.79] 1.98×10−41.98× 10^-4 0.0093 Overlap-50 IPPO 14.20 [9.08, 19.32] 0.0015 0.0492 MAPPO 16.40 [4.79, 28.01] 0.0172 0.2784 HAPPO 16.93 [10.26, 23.59] 0.0021 0.0639 Dynamic IPPO 14.70 [8.73, 20.67] 0.0024 0.0693 MAPPO 19.95 [12.23, 27.67] 0.0020 0.0620 HAPPO 18.98 [16.46, 21.49] 3.05×10−53.05× 10^-5 0.0017 Cascaded IPPO 26.45 [21.10, 31.80] 1.63×10−41.63× 10^-4 0.0078 MAPPO 20.43 [15.09, 25.76] 4.43×10−44.43× 10^-4 0.0190 HAPPO 18.88 [13.19, 24.56] 7.69×10−47.69× 10^-4 0.0300 Summary. SPL has a higher observed mean TC than its corresponding Vanilla backbone in all 60 primary configurations. The equal-weight descriptive mean difference across these configurations is 14.09%14.09\%; the corresponding backbone-level mean differences are 13.93%13.93\%, 14.02%14.02\%, and 14.33%14.33\% for IPPO, MAPPO, and HAPPO, respectively. Among the 57 configurations with paired inference, 56 pointwise 95% confidence intervals have a positive lower bound. The single exception is RWARE-Base Overlap-25 with IPPO, whose pointwise interval is [−1.30,17.40]%[-1.30,17.40]\%. After Holm correction over the declared family of 60 comparisons, 29 configurations remain significant at familywise level 0.050.05, while 28 testable comparisons do not. The three remaining configurations have no paired test because same-training-seed pairing is unavailable. These results distinguish descriptive consistency from multiplicity-adjusted statistical evidence. Higher observed SPL means occur in all 60 configurations, but this does not imply that all 60 comparisons are individually statistically significant. Similarly, failure to reject the null hypothesis after Holm correction should not be interpreted as evidence of equivalence between SPL and the corresponding Vanilla method. Evaluation design and zero standard errors. Final evaluation uses deterministic policies and a fixed set of 200 reset seeds within each environment protocol, reused across training seeds and methods. This controls variation due to the sampled evaluation instances while retaining independent policy training across training seeds. Four TC cells in the primary results table have an exact seed-level standard error of zero. These are Vanilla IPPO and Vanilla MAPPO under MPE-Base Non-overlap, MAE-MAPPO under MPE-Base Non-overlap, and Vanilla IPPO under MPE-Advanced Non-overlap. In each case, the five independently trained policies produce exactly identical 200-episode mean TC values at full precision. The reported zero therefore concerns variability among the five seed-level evaluation means; it does not imply that individual evaluation episodes have zero variability. The available records establish this numerical plateau under the fixed deterministic evaluation schedule but do not establish a particular behavioral mechanism responsible for it. Limitations of the statistical analysis. This analysis is retrospective and was conducted on the existing completed runs without additional training or evaluation. The paired tests treat the five training seeds as independent replications and assume that the seed-level differences are approximately normally distributed. With only five pairs per configuration, this normality assumption cannot be assessed convincingly. Furthermore, because the same fixed evaluation-instance schedule is reused across training seeds and methods, the reported inference is conditional on this evaluation design and does not quantify uncertainty over an unrestricted distribution of deployment tasks. The three Non-overlap MAPPO configurations without verified same-seed pairing prevent a complete 60-comparison paired analysis. Assigning these unavailable tests p=1p=1 internally makes the reported Holm correction conservative while preserving the declared family size, but does not replace the missing paired evidence. Pointwise confidence intervals should also be interpreted separately from the Holm-adjusted hypothesis tests.