Paper deep dive
Distributed Team Orchestration via Supervisor Networks: Convergence, Optimality, and Resilience
Juntian Zhu, Guanpu Chen, Tongtian Zhu, Miguel de Carvalho, Zhouwang Yang, Fengxiang He
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/13/2026, 3:30:02 AM
Summary
This paper proposes the Distributed Team-Orchestrating Algorithm (DTOA) for zero-sum potential team games with a supervisor network. It addresses challenges from belief-estimation errors and Byzantine attacks by combining team fictitious play with distributed belief learning. The authors prove convergence to a near team-Nash equilibrium (TNE) and develop a Byzantine-resilient variant (BR-DTOA) with probabilistic guarantees for identifying malicious teams.
Entities (7)
Relation Signals (6)
BR-DTOA → isvariantof → DTOA
confidence 96% · develop a Byzantine-resilient DTOA... propose the Byzantine-resilient DTOA (BR-DTOA)
DTOA → uses → Team Fictitious Play
confidence 95% · DTOA, which combines team fictitious play with supervisor-based distributed belief learning.
BR-DTOA → mitigates → Byzantine Attack
confidence 94% · In the Byzantine setting... develop a Byzantine-resilient DTOA.
DTOA → appliesto → Zero-Sum Potential Team Game
confidence 92% · We propose the distributed team-orchestrating algorithm (DTOA)... for zero-sum potential team games with a supervisor network.
DTOA → convergesto → Team-Nash Equilibrium
confidence 90% · establish that the induced learning dynamics converge to a near team-Nash equilibrium (TNE)
Supervisor Network → enables → Distributed Belief Learning
confidence 88% · supervisor network provides a basis for distributed learning under incomplete information
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In this paper, we study zero-sum potential team games with a supervisor network, where agents rely on supervisor-provided belief information rather than accurate common beliefs. The main challenge is that such belief information can be inaccurate because of supervisors' belief-estimation errors and the misreporting of joint actions by Byzantine teams. We propose the distributed team-orchestrating algorithm (DTOA), which combines team fictitious play with supervisor-based distributed belief learning. We prove the convergence of supervisors' belief estimates and establish that the induced learning dynamics converge to a near team-Nash equilibrium (TNE) in terms of the team-Nash gap (TNG). In the Byzantine setting, we consider a misreporting attack model and develop a Byzantine-resilient DTOA. We further provide probabilistic guarantees for Byzantine-team identification and establish an asymptotic bound on the honest TNG. Numerical experiments illustrate the theoretical findings, compare DTOA with baseline learning methods, and evaluate its performance in a Markov decision process setting.
Tags
Links
- Source: https://arxiv.org/abs/2608.09256v1
- Canonical: https://arxiv.org/abs/2608.09256v1
Trouble viewing inline? Open PDF directly →
Full Text
113,770 characters extracted from source content.
Expand or collapse full text
Distributed Team Orchestration via Supervisor Networks: Convergence, Optimality, and Resilience Juntian Zhu Guanpu Chen Tongtian Zhu Miguel de Carvalho Zhouwang Yang Fengxiang He Thanks: J. Zhu and Z. Yang are with the School of Artificial Intelligence and Data Science, University of Science and Technology of China, Hefei 230026, China. Email: zjt1229@mail.ustc.edu.cn, yangzw@ustc.edu.cn. Thanks: G. Chen is with the School of Automation, Southeast University, Nanjing 210096, China. Email: guanpu_chen@seu.edu.cn. Thanks: T. Zhu is with the College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China. Email: raiden@zju.edu.cn. Thanks: M. de Carvalho is with the School of Mathematics, University of Edinburgh, Edinburgh EH9 3FD, Scotland. Email: Miguel.deCarvalho@ed.ac.uk. Thanks: F. He is with the School of Informatics, University of Edinburgh, Edinburgh EH8 9AB, Scotland. Email: fhe@ed.ac.uk. Abstract In this paper, we study zero-sum potential team games with a supervisor network, where agents rely on supervisor-provided belief information rather than accurate common beliefs. The main challenge is that such belief information can be inaccurate because of supervisors’ belief-estimation errors and the misreporting of joint actions by Byzantine teams. We propose the distributed team-orchestrating algorithm (DTOA), which combines team fictitious play with supervisor-based distributed belief learning. We prove the convergence of supervisors’ belief estimates and establish that the induced learning dynamics converge to a near team-Nash equilibrium (TNE) in terms of the team-Nash gap (TNG). In the Byzantine setting, we consider a misreporting attack model and develop a Byzantine-resilient DTOA. We further provide probabilistic guarantees for Byzantine-team identification and establish an asymptotic bound on the honest TNG. Numerical experiments illustrate the theoretical findings, compare DTOA with baseline learning methods, and evaluate its performance in a Markov decision process setting. Index Terms: Team game, distributed learning, algorithmic convergence, equilibrium, Byzantine resilience I Introduction With the rapid development of artificial intelligence (AI), decision-making problems in multi-agent systems have received increasing attention. A large body of work has investigated cooperative and competitive interactions in multi-agent systems, which can be broadly classified into three categories: purely cooperative single-team problems, single-team problems involving external adversaries, and multi-team games. Single-team problems, with or without external adversaries, consider settings in which all agents cooperate to achieve a common objective [3, 24, 6, 34]. However, many complex decision-making environments involve multiple teams whose members cooperate internally while competing strategically with other teams, and such settings are naturally modeled by multi-team games [19, 10, 37]. Existing studies develop theoretical and algorithmic tools for multi-team interactions from several perspectives. Feng et al. [13] study a two-team mean-field game that incorporates both within-team cooperation and inter-team competition. Other studies consider coalition games in which each fixed coalition is treated as a team and analyze the corresponding equilibrium properties [40, 23, 41, 9]. Approaches based on mean-field games and coalition games typically take intra-team cooperation as a premise induced by a team-level objective. In contrast, team fictitious play (team-FP) focuses on team-level learning dynamics under complete observation of all agents. Dönmez et al. [10] study a multi-team setting in which self-interested agents learn team-level cooperative behavior and propose team-FP, which is shown to converge to a team-Nash equilibrium (TNE). However, team-FP [10] does not address multi-team learning in which self-interested agents learn to exhibit team-level cooperative behavior under incomplete and possibly unreliable information about opponents. This setting is relevant to applications such as market competition [31] and security problems [2], where self-interested agents may not naturally coordinate toward a team-level objective and decisions are often made under incomplete information about other teams. In addition, the information available for learning can be unreliable or deliberately manipulated, for example, through Byzantine attacks, which may affect agents’ decisions and long-term strategic behavior. The challenge is to establish how self-interested agents can learn team-level cooperative behavior when incomplete information and Byzantine attacks affect their decisions and, consequently, their long-term behavior. To address this challenge, it is important to introduce supervisors as high-level information intermediaries and construct a supervisor network for distributed information learning. The design of supervisors is motivated by information-intermediary roles studied in organizational management and networked systems [28, 5]. Each supervisor is associated with a subset of teams, receives the joint actions reported by these teams, and provides estimates of team-related information to the agents in these teams. Supervisors then exchange information over the supervisor network and update their estimates of team-related information for all teams. The supervisor network provides a basis for distributed learning under incomplete information and Byzantine attacks in multi-team games. The above motivates the development of algorithms for team orchestration based on distributed information learning over the supervisor network. For algorithm design, supervisors estimate and exchange information about all teams, and correspondingly, agents choose their actions using the information provided by supervisors. In addition, algorithmic resilience against Byzantine attacks should be taken into account since such attacks can corrupt supervisors’ information learning and affect agents’ strategy optimization. On this basis, we will explore the convergence of the learning dynamics and investigate whether self-interested agents can learn team-level cooperative behavior, as well as the resilience against Byzantine attacks. The primary contributions of this paper are as follows: • We design a supervisor-network mechanism for distributed learning of team-related information. Compared with existing studies on multi-team games [13, 10, 9], the proposed mechanism does not require direct observation of agents in other teams or a priori intra-team cooperation. This enables agents to optimize their strategies using supervisor-mediated belief estimates, thereby supporting team orchestration. • We develop the distributed team-orchestrating algorithm (DTOA) for team-FP learning over the supervisor network. We establish the convergence of the belief-estimation errors over the uniformly connected supervisor network using a gossip-matrix contraction argument (Theorem 1). We further derive an upper bound on the team-Nash gap (TNG) by comparing the actual learning dynamics with ideal and reference dynamics (Theorem 2). • We investigate a Byzantine attack setting where some teams misreport their actions to supervisors. We propose the Byzantine-resilient DTOA (BR-DTOA) and show the convergence of the belief-estimation errors for honest teams (Theorem 3). An upper bound on the honest TNG is revealed, and the near-TNE convergence for honest teams is resiliently preserved against Byzantine attacks with high-probability identification guarantees (Theorem 4). The rest of this paper is organized as follows. Section I reviews related work. Section I formulates zero-sum potential team games (ZSPTGs) with a supervisor network and their Byzantine extension. Section IV presents DTOA and establishes convergence and near-TNE guarantees. Section V develops BR-DTOA and analyzes its convergence and resilience. Section VI presents the experimental results. Section VII concludes this paper. The code is available at https://github.com/zjt-1229/team_game_with_supervisor_network. I Related Work This section provides a literature review. Mean-field games (MFG). Studies on MFGs primarily characterize optimal responses of individuals or representative agents to the mean field in large-scale weakly coupled systems, together with a consistency condition between the induced aggregate statistics and the hypothesized mean field. In standard MFGs, participants are typically modeled as anonymously coupled individuals through aggregate distributions [17]. Subsequent studies further investigate population-level consistency induced by mean-field responses [26], as well as associated learning and computational methods [36]. In parallel, the mean-field team literature focuses on team-optimal decision-making under mean-field coupling [30] and extends this line to more general uncertainty settings [12]. More recently, mixed cooperative–competitive mean-field models have incorporated both cooperation and competition into the mean-field framework [13, 1]. Related studies have also used mixed-coalition formulations with intra-group cooperation and inter-group competition [16] and large-scale competitive team learning [18] to characterize richer collective interactions. Despite these advances, mean-field-based approaches typically characterize agents’ responses to aggregate population statistics rather than team-level equilibrium learning among strategically interacting teams. In contrast, our work addresses the complementary problem of equilibrium formation in repeated multi-team games under limited information, where agents’ strategy optimization depends on estimated information acquired through the learning process. Coalition games. Coalitional-game studies take coalitions as the basic units of analysis, focusing either on coalition formation, coalition values, and the stability of payoff allocations, or, under a fixed coalition partition, on equilibria induced by coalition-level objectives. The early coalition-game literature mainly developed around hedonic preferences [11] and broader modeling and stability analysis of coalition formation [15]. Recent studies further examine the endogenous grouping of self-interested agents and the structural stability of the resulting partitions from a group-formation perspective [33], and provide a systematic account of major classes of coalitional games and their distributed-network applications [29]. In parallel, the literature on coalition games with fixed coalition partitions treats each coalition as a composite agent and studies the existence and distributed computation of generalized Nash equilibria (GNEs) and variational generalized Nash equilibria (vGNEs), together with extensions to nonsmooth, constrained, and dynamic settings [40, 23, 41, 9]. However, studies on multi-coalition games with fixed coalition partitions typically model each coalition as a unified agent with a prescribed coalition-level objective and therefore do not capture how team-level behavior emerges from self-interested agent-level interactions. In contrast, our work studies equilibrium formation among teams induced by agent-level learning, which is relevant to multi-team systems without centralized team control. Byzantine-resilient mechanisms. Uncertainty is an unavoidable issue in many multi-agent game-theoretic settings, and seeking robust or resilient equilibria has become a common approach for preserving desirable performance in uncertain environments [42, 7, 8]. Among different sources of uncertainty, Byzantine attacks represent a particularly challenging form, in which malicious agents can distort the information available to others by sending falsified or inconsistent reports. Research on Byzantine-resilient multi-agent systems mainly focuses on how normal agents maintain coordination, identify malicious information sources, and suppress the influence of such sources when adversarial agents disrupt the system by sending falsified messages [35]. Early work focuses primarily on resilient consensus and establishes convergence mechanisms in the presence of adversarial or Byzantine nodes [20]. Building on this line, subsequent studies introduce mechanisms such as distributed detection [39]. These ideas have also been extended to more general control and learning settings, including Byzantine-resilient output regulation [38] and multi-agent reinforcement learning [22]. These works mainly develop Byzantine-resilient mechanisms for consensus, control, and cooperative learning, with an emphasis on filtering malicious information and preserving coordination among normal agents. They provide useful insights for the Byzantine-resilient algorithm developed in this paper for multi-team games with self-interested agents. I Problem Formulation In this section, we first revisit ZSPTGs and then formulate ZSPTGs with a supervisor network. We also specify the Byzantine attack setting considered in this paper. I-A Revisiting Zero-sum Potential Team Game In multi-team games, teams compete against others, while agents in each team cooperate. A multi-team game is characterized by the tuple =(ℐ,,ii∈ℐ,uii∈ℐ)G\!=\!(I,T,\A^i\_i ,\\!u^i\!\_i ), where ℐI and T denote the index sets of agents and teams, respectively. Let ℐmI^m denote the index set of agents in team m∈m\!∈\!T. These sets form a partition of ℐI, i.e., ℐm∩ℐm′=∅I^m\! ^m \!\!\!=\!\! for m≠m′m\!\!≠\!\!m and ⋃m∈ℐm=ℐ _m∈\!T\!I^m\!\!=\!\!I. Here, iA^i denotes the finite action set of agent i, ≜∏i∈ℐiA\! \! _i A^i denotes the finite joint action set of all agents, and ui:→ℝu^i\!\!:\!A\!→\!R denotes the utility function of agent i. Let ¯m≜∏i∈ℐmi A^m\!\! \! _i ^mA^i denote the joint action set of team m∈m . The mixed-strategy spaces of agent i, team m, and all agents are Δ(i) (A^i), Δ(¯m) ( A^m), and Δ() (A), respectively. We next recall the definition of ZSPTGs, TNG and TNE [10]. Definition 1 (Zero-sum Potential Team Game). A multi-team game is a ZSPTG if, for every team m∈m , there exists a potential function ϕm:→ℝφ^m:A such that ϕm(a^i,a−i,a¯−m)−ϕm(a)=ui(a^i,a−i,a¯−m)−ui(a), φ^m( a^i\!,a^-i\!, a^-m)\!-\!φ^m(a)\!=\!u^i( a^i\!,a^-i\!, a^-m)\!-\!u^i(a), (1) for all (a^i,a)∈i×( a^i,a) ^i×A and all i∈ℐmi ^m, where a−i≜ajj∈ℐm∖ia^-i \a^j\_j ^m \i\ and a¯−m≜a¯ll∈∖m a^-m \ a^l\_l \m\. Moreover, the potential functions satisfy the zero-sum condition ∑m∈ϕm(a)=0,∀a∈. Σ _m φ^m(a)=0, ∀ a . (2) The cross-team interactions are network-separable, so the potential and utility functions can be decomposed as ϕm=∑l≠mϕml, and ui=∑l≠muil,∀i∈ℐm, φ^m=Σ _l≠ mφ^ml, and u^i=Σ _l≠ mu^il,\ ∀ i ^m, (3) for some ϕml:¯mׯl→ℝφ^ml: A^m× A^l and uil:¯mׯl→ℝu^il: A^m× A^l . Definition 2 (Team-Nash Gap). Given a team strategy profile π=πm∈Δ(¯m)m∈π\!=\!\π^m\!∈\! ( A^m)\_m , the TNG for each team m∈m is defined as TNGm(π)≜maxπ′∈Δ(¯m)ϕm(π′,π−m)−ϕm(π). TNG^m(π) _π ∈ ( A^m)\φ^m(π ,π^-m)\-φ^m(π). The TNG is then defined as TNG(π)≜∑m∈TNGm(π)TNG(π)\!\! \!\! _m \!TNG^m(π), where π−m=πll∈∖mπ^-m=\π^l\_l m. Correspondingly, π is called a TNE if TNG(π)=0TNG(π)=0. Furthermore, for any ε>0 >0, π is called an ε -TNE if TNG(π)<εTNG(π)< . Intuitively, a small TNG means that the induced team strategy profile is nearly stable against unilateral deviations of any single team from its current strategy. When the team strategy profile has a small TNG, the resulting behavior reflects an approximate pattern of within-team cooperation and inter-team competition. Thus, we use the TNG to evaluate the extent to which self-interested agents in each team behave cooperatively while competing with other teams. To study how self-interested agents maximize their own utilities, we next revisit the smoothed best response used in the subsequent learning algorithms. Definition 3 (Smoothed Best Response). Given a finite set X, for a distribution D∈Δ(X)D\!\!∈\!\! (\!X\!) and a function f:X→ℝf\!:\!X\!\!→\!\!R, let f(D)=∑x∈XD(x)f(x)f(D)\!\!=\!\! _x∈\!X\!D(x)f(x). For a temperature parameter τ>0τ>0, the smoothed best response to f is defined as brτ(f)(x)=ef(x)τ∑x~∈Xef(x~)τ,∀x∈X. br_τ(f)(x)= e f(x)τ _ x∈ Xe f( x)τ,\ ∀ x∈ X. Equivalently, brτ(f)=argmaxD∈Δ(X)f(D)+τℋ(D)br_τ(f)\!\!=\!\! _D∈ (\!X\!)\!\! \f(D)\!+\! (D) \ with entropy regularization ℋ(D)≜∑x∈X−D(x)logD(x)H(D)\!\! \!\! _x∈\!X\!-\!D(\!x\!)\! \!D(\!x\!). The smoothed best response provides a regularized decision rule that assigns positive probability to all feasible actions and balances objective optimization with exploration. The temperature parameter τ controls the degree of smoothing: smaller values make the response closer to a pure best response, whereas larger values induce more exploratory behavior. The maximizer is unique because f is linear and, for τ>0τ>0, the entropy regularization term makes the objective strictly concave on Δ(X) (X). Note that the utilities of all agents need not sum to zero. Although the definition of a ZSPTG requires the team potentials to sum to zero, the same analysis applies when their sum is a constant independent of the action profile a∈a\!∈\!A and agents choose actions according to Definition 3. Indeed, the potentials can be normalized by subtracting this constant from any one team potential, which preserves all potential differences and therefore leaves the smoothed best-response structure unchanged. I-B ZSPTG with a supervisor network In this paper, we focus on ZSPTGs with a supervisor network, where supervisors serve as distributed information intermediaries. A multi-team game with a supervisor network is characterized by the tuple =(ℐ,,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ)G=(I,T,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ), where ℐI, T, and S denote the index sets of agents, teams, and supervisors, respectively. We follow the notation in Section I-A: ℐmI^m, iA^i, ¯m A^m, A, and ui:→ℝu^i:A denote the index set of agents in team m, the action set of agent i, the joint action set of team m, the joint action set of all agents, and the utility function of agent i, respectively. We assume that each agent observes only the actions of agents in the same team. The matrix ST∈0,1||×||ST∈\0,1\^|T|×|S| denotes the supervision relationships between teams and supervisors, and the matrix SNk∈0,1||×||SN_k∈\0,1\^|S|×|S| denotes the communication structure among supervisors at round k. Each supervisor supervises a subset of teams: it receives action reports from these teams and provides their agents with information about other teams. Specifically, ST(m,s)=1ST(m,s)\!\!=\!\!1 indicates that team m is supervised by supervisor s, and ST(m,s)=0ST(m,s)\!\!=\!\!0 otherwise. Similarly, SNk(s,s′)=1SN_k(s,s )=1 indicates that supervisors s and s′s exchange information at round k, and SNk(s,s′)=0SN_k(s,s )=0 otherwise. Each agent aims to maximize its utility based on the information provided by the supervisors. For each supervisor s∈s\!∈\!S, let s⊆T^s\! \!T denote the set of teams supervised by s. The supervisor communication structure at round k can be equivalently represented by an undirected graph Gk=(,Ek)G_k\!\!=\!\!(S,\!E_k), where S is the node set and Ek=s,s′:SNk(s,s′)=E_k\!\!=\!\! \\!\s,s \\!:\!SN_k(s,s )\!\!=\!\!1\! \ is the edge set, for all k≥0k\!≥\!0. In the team-FP learning dynamics, agents update their actions based on common beliefs about other teams’ strategies [10], which are formed through direct observation of the actions of other agents. In ZSPTGs with a supervisor network, however, these beliefs are provided by supervisors that receive action reports from supervised teams and exchange information over the supervisor network. Consequently, belief-estimation errors can arise because each supervisor receives reports from only a subset of teams. It is therefore crucial to characterize how such errors affect the convergence of the learning dynamics and the induced TNG. This motivates the following problem. Problem 1. How to design a distributed team-orchestrating algorithm for ZSPTGs with a supervisor network while providing theoretical guarantees? This problem concerns whether the common-belief requirement in team-FP can be relaxed through distributed belief learning over the supervisor network. The key challenge is to control the long-term effect of supervisors’ belief-estimation errors on the induced team behavior. We first characterize the belief-estimation process over the supervisor network and then quantify its effect on the induced learning dynamics and the TNG. We impose the following assumptions. Assumption 1. Every team is supervised by at least one supervisor, i.e., ⋃s∈s= _s T^s=T. Assumption 2. The supervisor communication graph Gk≜(,Ek)G_k (S,E_k), which represents the supervisor network, is time-varying and uniformly connected over time [27]. Specifically, let B be the smallest positive integer such that, for any n∈ℕ+n ^+, the union graph G(n)≜(,⋃k=nB(n+1)B−1Ek)G_(n) (S, _k=nB^(n+1)B-1E_k) is connected. Assumption 1 ensures that every team has at least one reporting link to the supervisor network and hence no team is completely isolated from it. Assumption 2 ensures that information can propagate among all supervisors within each finite communication window. I-C Byzantine Attack We consider a Byzantine attack setting in which some teams can misreport their actions to the supervisors at each round. Given a ZSPTG with a supervisor network =(ℐ,,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ)G=(I,T,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ), honest teams and Byzantine teams are defined as follows. Definition 4 (Honest Team and Byzantine Team). A team m is called honest if it reports its joint action truthfully to the supervisors that supervise it, i.e., a¯k,rm=a¯km a_k,r^m= a_k^m for all k≥0k≥ 0, where a¯k,rm a_k,r^m denotes the joint action reported by team m at round k. In contrast, a team m is called Byzantine if it can misreport its joint action to the supervisors that supervise it, i.e., a¯k,rm≠a¯km a_k,r^m≠ a_k^m for some k≥0k≥ 0. Let ℋ⊆H and ℬ⊆B denote the sets of honest and Byzantine teams, respectively. Definition 5 (Honest Team-Nash Gap). Given a team strategy profile π=πm∈Δ(¯m)m∈π=\π^m∈ ( A^m)\_m , the honest TNG is defined as TNGℋ(π)≜∑m∈ℋTNGm(π)TNG_H(π) _m TNG^m(π). Correspondingly, the team strategy profile π is called an ε -honest TNE if TNGℋ(π)<εTNG_H(π)< . In the definition of the honest TNG, the strategies of Byzantine teams are treated as fixed exogenous factors, and the gap is evaluated only over honest teams. Since our objective is to optimize the performance of honest teams under Byzantine attacks, including the gaps for Byzantine teams in the honest TNG does not reflect the intended performance measure. In this Byzantine attack setting, the learning dynamics are affected by Byzantine teams, whose misreports can distort supervisors’ belief learning processes and influence the long-term behavior of agents in honest teams. This leads to the following Byzantine-resilient learning problem. Problem 2. How to design a distributed resilient algorithm for ZSPTGs with a supervisor network against Byzantine attacks? The key challenge is to identify Byzantine teams using limited information while preventing their misreports from disrupting the long-term learning behavior of honest teams. We address this challenge by developing a supervisor-based mechanism and quantifying the effect of Byzantine teams on the learning dynamics of honest teams. To this end, each supervisor can check whether the action reports from its supervised teams are consistent with their true actions. The checking outcome is not necessarily accurate: an honest report can be incorrectly regarded as a misreport, and a misreport can fail to be detected. IV Distributed Team-Orchestrating Algorithm This section addresses Problem 1 by developing the DTOA. We then analyze the convergence of supervisors’ belief-estimation errors and derive an upper bound on the TNG. IV-A Algorithm Design Algorithm 1 Distributed Team-Orchestrating Algorithm (DTOA) 1: Initialize: πs,0mm∈\ _s,0^m\_m and a−1ii∈ℐ\a_-1^i\_i arbitrarily 2: while round k=0,1,…k=0,1,… do 3: for m∈m do 4: select agent i∈ℐmi ^m in team m randomly 5: select s∈s:i∈ss∈ \s:i ^s \ randomly to provide belief estimates about other teams 6: agent i updates its action based on ak−1−ia_k-1^-i and πk−m,s _k^-m,s: aki∼brτ(ui(⋅,ak−1−i,πkm,s))a_k^i br_τ (u^i(·,a_k-1^-i, _k^m,s) ) 7: for agent j∈ℐm\ij ^m \i\ do 8: repeat the last action: akj=ak−1ja_k^j=a_k-1^j 9: end for 10: end for 11: for s∈s and m∈m do 12: if m∈sm ^s then 13: update: πk+1m,s=πkm,s+αk(a¯km−πkm,s) _k+1^m,s= _k^m,s+ _k ( a_k^m- _k^m,s ) 14: else 15: update: πk+1m,s=1|ks|∑s′∈ksπkm,s′,for |ks|>0πkm,s,for |ks|=0 _k\!+\!1^m,s\!\!=\!\! cases\!\! 1|N^s_k|\!\! _s \! ^s_k\! _k^m,s \!\!\!,\!\!&for |N^s_k|\!\!>\!\!0\\ \! _k^m,s,\!\!&for |N^s_k|\!\!=\!\!0 cases 16: end if 17: end for 18: end while We first specify the main components of DTOA for ZSPTGs with a supervisor network. At round k, let akia_k^i denote the action of agent i, and let a¯km=(aki)i∈ℐm a_k^m= (a_k^i )_i ^m denote the joint action of team m. To compensate for the lack of complete opponent-action information, DTOA uses supervisors to provide belief estimates. Let πkm,s∈Δ(¯m) _k^m,s∈ ( A^m ) denote the belief of supervisor s regarding the strategy of team m. To induce team-level strategic behavior under incomplete opponent-action information, agent i in team m updates its action according to the smoothed best response in Definition 3, using the previous actions of the other agents in the same team and the beliefs provided by a supervisor s that supervises team m: aki∼brτ(ui(⋅,ak−1−i,πk−m,s)), a_k^i br_τ (u^i(·,a_k-1^-i, _k^-m,s) ), (4) where aki∼pa_k^i p means that akia_k^i is sampled from p∈Δ(i)p∈ (A^i ). The rule above specifies the action update of a selected agent. In DTOA, one agent per team is randomly selected at each round to update its action, while the other agents in the same team keep their actions unchanged. When such coordination is unavailable, we consider an independent variant, called independent DTOA (iDTOA), in which each agent updates its action independently with probability δ∈(0,1)δ∈(0,1). To maintain belief estimates of teams’ strategies, we design two belief-update rules: one for teams directly supervised by a given supervisor and the other for teams not supervised by that supervisor. Given a supervisor s∈s , for a team m∈sm ^s, supervisor s directly updates its belief using the joint action reported by team m: πk+1m,s=πkm,s+αk(a¯km−πkm,s), _k+1^m,s= _k^m,s+ _k ( a_k^m- _k^m,s ), where αkk≥0 \\! _k\! \_\!k≥ 0 is the step-size sequence used by all supervisors, and a¯km a_k^m denotes the one-hot representation in Δ(¯m) ( A^m ) for notational simplicity. Let ks=s′∈:SNk(s,s′)=N^s_k\!\!=\!\!\\!s \!\!∈\!\!S\!\!:\!SN_k(\!s,\!s \!)\!\!=\!\!1\!\ denote the neighbor set of supervisor s at round k. For a team m∉sm\!\!∉\!\!T^s, if |ks|>0|N^s_k|\!\!>\!0, supervisor s updates its beliefs using information from its neighbors: πk+1m,s=1|ks|∑s′∈ksπkm,s′. _k+1^m,s= 1|N^s_k| _s ^s_k _k^m,s . If |ks|=0|N^s_k|=0, supervisor s keeps its beliefs unchanged, i.e., πk+1m,s=πkm,s _k+1^m,s= _k^m,s for all m∉sm ^s. The implementation of DTOA is summarized in Algorithm 1. IV-B Convergence Analysis We now turn to the convergence analysis of DTOA. Following the step-size conditions considered in 10, we impose the following assumption. Assumption 3. The step-size sequence αkk≥0\ _k\_k≥ 0 satisfies the following conditions: • αk∈[0,1] _k∈[0,1], and αk→0 _k→ 0 as k→∞k→∞; • ∑k=0∞αk=∞ _k=0^∞ _k=∞ and ∑k=0∞αk2<∞ _k=0^∞ _k^2<∞; • limk→∞αk/αk+1=1 _k→∞ _k/ _k+1=1 and αk−αk+1≥αkαk+1 _k- _k+1≥ _k _k+1. The first two conditions in Assumption 3 ensure that the belief updates continue to incorporate new action information while the effect of sampling fluctuations is asymptotically averaged out. The last condition further ensures that action information from adjacent periods has comparable influence on the beliefs. A standard choice satisfying these conditions is αk=1/(k+1) _k\!\!=\!\!1/(k\!+\!1), which corresponds to empirical averaging over past actions. To quantify supervisors’ belief-estimation errors, we define the true belief updates in the full-supervision case where each supervisor supervises all teams as πk+1m=πkm+αk(a¯km−πkm),∀m∈. _k+1^m= _k^m+ _k ( a_k^m- _k^m ),\ ∀ m . (5) Under Assumption 3, the effect of the initial beliefs vanishes asymptotically. Hence, the initialization does not affect the asymptotic convergence results. For ease of analysis, we set π0m,s=π0m _0^m,s= _0^m for all m∈m and s∈s . Remark 1. In the full-supervision case, for any supervisor s∈s\!∈\!S and any team m∈m\!∈\!T, we have πkm,s=πkm _k^m,s\!=\! _k^m for all k≥0k≥ 0. By contrast, under a general supervision structure, a supervisor s∈s may not directly supervise a team m∈m\!∈\!T, i.e., m∉sm\!∉\!T^s, in which case the equality πkm,s=πkm _k^m,s\!=\! _k^m may no longer hold. For supervisor s∈s\!∈\!S and team m∈m\!∈\!T, we define the belief-estimation error with respect to the true belief as ‖πkm,s−πkm‖∞\| _k^m,s\!-\! _k^m\!\|_∞. The following theorem establishes the asymptotic convergence of all supervisors’ belief-estimation errors under DTOA. The proof is provided in Appendix -A. Theorem 1. For a ZSPTG with a supervisor network =(ℐ,,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ), =(I,T,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ), under Assumptions 1 and 2, the supervisors’ belief-estimation errors in DTOA converge to zero; specifically, ‖πkm,s−πkm‖∞≤O(ρ[k2])+O(α[k2]),∀s∈,∀m∈, \| _k^m,s\!-\! _k^m\|_∞\!≤\!O(ρ^[ k2])\!+\!O( _[ k2]),∀ s\!∈\!S,∀ m\!∈\!T, (6) where ρ=(−(1||)(||+1)B)1(||+1)B<1ρ\!=\! (\!1\!-\! ( 1|S| )^(|S|+1)B ) 1(|S|+1)B\!<\!1. If Assumption 3 also holds, (6) simplifies to ‖πkm,s−πkm‖∞≤O(α[k2])\| _k^m,s\!-\! _k^m\|_∞\!≤\!O( _[ k2]). Theorem 1 shows that the convergence rate of the supervisors’ beliefs depends on the supervisor network SNkk≥0\SN_k\_k≥ 0, the number of supervisors |||S|, and the step-size sequence αkk≥0\ _k\_k≥ 0. Here, B characterizes the length of the communication window over which information is propagated among supervisors. The first term in (6) indicates that, for fixed B and αkk≥0\ _k\_k≥ 0, a larger number of supervisors |||S| leads to slower convergence. We next fix |||S| and αkk≥0\ _k\_k≥ 0 and examine how B affects the convergence rate. Since the logarithm is strictly increasing, the monotonicity of ρ with respect to B is equivalent to the monotonicity of logρ ρ with respect to B. For all ||>1|S|>1 and all B>0B>0, we have ∂(logρ)∂B=(||+1)B⋅log||||(||+1)B−1(||+1)2B2>0. ∂( ρ)∂ B= (|S|+1)B· |S||S|^(|S|+1)B-1(|S|+1)^2B^2>0. This implies that ρ increases with B; hence, a larger B leads to a slower convergence rate. The second term in (6) further shows that the step-size sequence αkk≥0\\! _k\!\_\!k≥ 0 also affects the convergence rate of the supervisors’ beliefs. IV-C TNG Analysis Having established the convergence of supervisors’ belief estimates, we next analyze TNG convergence under DTOA. The main challenge in proving TNG convergence for DTOA is that supervisors’ belief-estimation errors affect agents’ actions at each round, thereby influencing subsequent belief updates and action decisions. Although supervisors’ belief-estimation errors converge to zero as k→∞k→∞, it remains nontrivial to show that their cumulative influence on agents’ long-term behavior also vanishes asymptotically. To address this difficulty, we introduce an ideal scenario for the analysis of DTOA and refer to the original setting with belief-estimation errors as the actual scenario. The repeated play is divided into epochs, each consisting of T rounds. In the ideal scenario, at the beginning of each epoch, all supervisors are initialized with the history of all agents’ actions from the actual scenario and supervise all teams throughout the epoch. Consequently, they share common beliefs at each round, which are referred to as ideal beliefs. Since the actual and ideal scenarios use different supervision structures during epoch n, they generally induce different distributions over the joint actions of all teams at round k of epoch n. Remark 2. The ideal beliefs generally differ from the true beliefs. Specifically, the true beliefs depend on the action history in the actual scenario, whereas the ideal beliefs depend on both the actions during the current epoch in the ideal scenario and the action history generated before epoch n in the actual scenario. Nevertheless, at the beginning of each epoch, the ideal beliefs coincide with the true beliefs because both are constructed from the action history generated before epoch n in the actual scenario. The key idea is to use an epoch-wise comparison between the ideal and actual scenarios to quantify how supervisors’ belief-estimation errors propagate to the induced action distributions. Let νk,(n)m _k,(n)^m and a-νk,(n)ma- _k,(n)^m denote the joint-action distributions of team m at round k of epoch n in the ideal and actual scenarios, respectively. The following lemma quantifies how the difference between νk,(n)m _k,(n)^m and a-νk,(n)ma- _k,(n)^m is bounded in terms of supervisors’ belief-estimation errors. Lemma 1. For a ZSPTG with a supervisor network =(ℐ,,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ)G=(I,T,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ), under Assumptions 1 and 2, the difference between the induced action distributions for DTOA in the actual and ideal scenarios can be bounded as ‖νk,(n)m−a-νk,(n)m‖1≤Cδ(n), \| _k,(n)^m-a- _k,(n)^m\|_1≤ C _(n), where δ(n)=maxk∈[nT−1,(n+)T−1]∑m∈maxs∈‖πkm,s−πkm‖1 _(n)\!\!=\!\! _k∈[nT\!-\!1,(\!n\!+\!1\!)T\!-\!1]\! _m \! _s \!\| _k^m,s\!-\! _k^m\|_1 denotes the maximum aggregate supervisor belief-estimation error in epoch n. Since Theorem 1 implies δ(n)→0 _(n)\!\!→\!0 as n→∞n\!\!→\!∞, it follows that ‖νk,(n)m−a-νk,(n)m‖1→0 \| _k,(n)^m\!\!-\!a- _k,(n)^m\! \|_1\!\!→\!0. Lemma 1 shows that the difference between the action distributions induced by the actual and ideal beliefs is bounded by a term proportional to the maximum aggregate supervisor belief-estimation error in epoch n. This is crucial for analyzing agents’ long-term behavior in the actual scenario. The proof is provided in Appendix -B. Remark 3. At round k=nTk\!=\!nT, Remark 2 and the properties of the smoothed best response yield that ‖νnT,(n)m−a-νnT,(n)m‖1\| _nT,(n)^m\!-\!a- _nT,(n)^m\|_1 can be controlled by ‖πnTm,s−πnTm‖1\| _nT^m,s\!\!-\!\! _nT^m\|_1. However, this argument does not directly extend to nT<k≤(n+)T−1nT\!\!\!<\!\!k\!\!≤\!\!(\!n\!+\!1\!)T\!\!-\!1, since the action distributions are affected by the accumulated discrepancy between the actual and ideal scenarios. We now present the TNG result. Building on Theorem 1 and Lemma 1, the following theorem establishes an almost-sure upper bound on the TNG under DTOA. Theorem 2. For a ZSPTG with a supervisor network =(ℐ,,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ)G=(I,T,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ), under Assumptions 1, 2, and 3, DTOA satisfies, ∀s∈∀ s , lim supk→∞TNG(πks)≤τlog||,for DTOA,τlog||+||2ϕ¯Λ(δ,εϕ),for iDTOA, _k→∞T\!N\!G( _k^s)\!\!≤\!\! cases\!τ\! \!|A|,\!\!\!\!\!&for DTOA,\\ \!τ\! \!|A|\!+\!|T|^2 φ (δ, _φ),\!\!\!&for iDTOA, cases where ϕ¯=max(m,l,a)|ϕml(a)| φ\!\!=\!\! _(m,l,a)|φ^ml(a)|, πks=πkm,sm∈ _k^s\!\!=\!\!\ _k^m,s\_m , and Λ(δ,εϕ) (δ, _φ) is a function decaying to zero as δ→0+δ→ 0^+ for any 0<εϕ≤mina∈brτ(a,a−i)0< _φ≤ _a br_τ (a,a^-i ). To proceed with the TNG convergence analysis, we introduce a reference scenario following [10]. In this reference scenario, at any round k in epoch n, agents update their actions using the true beliefs at the beginning of epoch n. Specifically, for nT≤k≤(n+1)T−1nT\!≤\!k\!≤\!(n\!+\!1)T\!-\!1, the selected agent i follows the reference update rule aki∼brτ(ui(⋅,ak−1−i,πnT−m))a_k^i\! \!br_τ (u^i(·,a_k-1^-i, _nT^-m) ). The comparison between the actual and reference scenarios can then be decomposed into two parts: the comparison between the actual and ideal scenarios and the comparison between the ideal and reference scenarios. The proof is in Appendix -B. Theorem 2 shows that the long-term behavior of all agents induces a near TNE. This implies that agents in the same team learn to cooperate with one another while competing against other teams, even though each agent only knows its own utility. The resulting TNG bound matches that in Theorem 4.2 of [10]. In contrast, our analysis explicitly accounts for belief-estimation errors induced by supervisor-network learning. Accordingly, DTOA replaces the common-belief requirement in team-FP with supervisor-based distributed belief learning. DTOA enables agents to optimize their strategies for team orchestration while ensuring the convergence of belief-estimation errors and providing a small-TNG guarantee. These results provide an affirmative answer to Problem 1. V Byzantine resilience This section addresses Problem 2 by developing a supervisor-based Byzantine-identification mechanism and adapting DTOA to the Byzantine attack setting. We consider a Byzantine attack setting for ZSPTGs with a supervisor network, adapted from Byzantine-resilient multi-agent learning [21] and imperfect verification [14]. In this setting, each team is either honest or Byzantine; that is, ℋ∩ℬ=∅H = and ℋ∪ℬ=H =T. Agents in honest teams update their actions according to (4) and report their joint actions truthfully to their supervisors, whereas agents in Byzantine teams update their actions arbitrarily and can misreport their joint actions. Supervisors aim to identify Byzantine teams and mitigate their influence on honest teams. The supervisors’ checking policy and the Byzantine teams’ attack policy are specified below. Supervisors’ checking policy. For a supervisor s∈s and a team m∈sm ^s, supervisor s checks, with a prescribed probability, whether the joint action reported by team m is consistent with the joint action actually taken by team m. We use a binary variable vkm,s∈0,1v_k^m,s∈\0,1\ to indicate this decision, where vkm,s=1v_k^m,s=1 means that supervisor s checks team m at round k. After checking team m at round k, supervisor s receives a verification signal zkm,s∈fail,passz_k^m,s∈\fail,pass\. The verification signals zkm,sz_k^m,s are not necessarily accurate. Specifically, we assume that there exist constants ηFP∈(0,1] _FP∈(0,1] and ηFN∈[0,1) _FN∈[0,1) such that, for any s∈s , m∈sm ^s, k≥0k≥ 0, ℙ(zkm,s=fail∣vkm,s=1,a¯k,rm=a¯km)≤ηFP, (z_k^m,s=fail v_k^m,s=1,\ a_k,r^m= a_k^m )≤ _FP, ℙ(zkm,s=pass∣vkm,s=1,a¯k,rm≠a¯km)≤ηFN. (z_k^m,s=pass v_k^m,s=1,\ a_k,r^m≠ a_k^m )≤ _FN. Here, ηFP _FP and ηFN _FN denote upper bounds on the false-positive and false-negative probabilities, respectively. Byzantine attack policy. A Byzantine team can be identified more readily if it misreports its joint action at each round. We therefore assume that there exists plie∈[0,1]p_lie∈[0,1] such that each Byzantine team misreports its joint action to its supervisors with probability pliep_lie at each round. If a Byzantine team does not misreport at a given round, then it reports its true joint action at that round. In this case, the Byzantine team is indistinguishable from an honest team from the supervisors’ perspective, because the verification signal depends only on whether the reported joint action is consistent with the joint action actually taken. Moreover, supervisors have no access to any agent’s utility function or any team’s potential function. In practice, a verification signal is meaningful only if it is informative about whether a team misreports. We impose the following consistency condition on the verification signals. Specifically, ηFP _FP, ηFN _FN, and pliep_lie satisfy ηFP<(1−plie)ηFP+plie(1−ηFN). _FP<(1-p_lie) _FP+p_lie(1- _FN). (7) Condition (7) means that a checked Byzantine team has a higher probability of generating a failfail signal than a checked honest team. Equivalently, supervisors are more likely to receive a passpass signal after checking an honest team than after checking a Byzantine team. The Byzantine attack setting described above is referred to as a Byzantine ZSPTG (B-ZSPTG) with a supervisor network and is defined by the tuple B=(ℐ,ℋ,ℬ,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ,p), _B=(I,H,B,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ,p), where ℋH and ℬB denote the sets of honest and Byzantine teams, respectively, and p=(ηFP,ηFN,plie)p=( _FP, _FN,p_lie) denotes the parameters satisfying (7). This formulation serves as the basis for the subsequent analysis of Problem 2. V-A Algorithm Design Algorithm 2 Byzantine-resilient DTOA (BR-DTOA) 1: Initialise: πs,0mm∈\ _s,0^m\_m and a−1ii∈ℐ\a_-1^i\_i arbitrarily, F0m,s=0F_0^m,s=0 2: while round k=0,1,…k=0,1,… do 3: for m∈m do 4: select agent i∈ℐmi ^m in team m randomly 5: if m is an honest team then 6: select s∈s:i∈ms∈ \s:i ^m \ randomly to provide belief estimates about other teams 7: agent i updates its action based on ak−1−ia_k-1^-i and πkm,s _k^m,s: aki∼brτ(ui(⋅,ak−1−i,πkm,s))a_k^i br_τ (u^i(·,a_k-1^-i, _k^m,s) ) 8: else 9: agent i updates its action randomly 10: end if 11: for j∈ℐm\ij ^m \i\ do 12: repeat the last action: akj=ak−1ja_k^j=a_k-1^j 13: end for 14: report the joint action a¯k,rm a_k,r^m to supervisors mT^m 15: end for 16: for s∈s and m∈m do 17: if m∈sm ^s then 18: check team m for misreporting with probability q and receive zkm,s∈pass,failz_k^m,s∈\pass,fail\ 19: update Fkm,s=Fk−1m,s+1(zkm,s=fail)F_k^m,s=F_k-1^m,s+1 (z_k^m,s=fail ) 20: update fkm,s=Fkm,s/kf_k^m,s=F_k^m,s/k 21: if k<Kk<K or fkm,s<f_k^m,s<f then 22: update: πk+1m,s=πkm,s+αk(a¯k,rm−πkm,s) _k+1^m,s\!=\! _k^m,s\!+ _k\! (\! a_k,r^m\!-\! _k^m,s\! ) 23: end if 24: else 25: update: πk+1m,s=1|ks|∑s′∈ksπkm,s′,for |ks|>0πkm,s,for |ks|=0 _k+1^m,s\!\!=\!\! cases\! 1|N^s_k|\!\! _s \! ^s_k\!\! _k^m,s \!\!\!,\!\!&for |N^s_k|\!\!>\!\!0\\ _k^m,s,\!\!&for |N^s_k|\!\!=\!\!0 cases 26: end if 27: end for 28: end while To address the Byzantine attack setting described above, we extend DTOA with a Byzantine-resilient mechanism that operates at the supervisor-team level. The resulting algorithm, referred to as BR-DTOA, is summarized in Algorithm 2. Byzantine-resilient mechanism. Since each checking outcome can be inaccurate, supervisors do not rely on a single failfail verification signal to identify Byzantine teams. Instead, they accumulate the evidence over time. For a supervisor s∈s\!∈\!S and a team m∈sm\!∈\!T^s, let Fkm,sF_k^m,s denote the number of times that supervisor s has received a failfail signal from team m up to round k. Supervisor s checks team m with probability q and updates Fkm,sF_k^m,s as Fkm,s=Fk−1m,s+vkm,s=1,zkm,s=fail. F_k^m,s=F_k-1^m,s+1\v_k^m,s=1,\ z_k^m,s=fail\. The empirical frequency of receiving a failfail signal from team m is then defined as fkm,s=Fkm,s/kf_k^m,s\!=\!F_k^m,s/k. To reduce the effect of early-stage fluctuations, supervisors start labeling teams only after a burn-in period of K rounds. Given a threshold f satisfying qηFP<f<q[(−plie)ηFP+plie(−ηFN)]q _FP\!<\!f\!<\!q [(1\!-\!p_lie) _FP\!+\!p_lie(1\!-\! _FN) ], if k>Kk>K and fkm,s>f_k^m,s>f hold, then supervisor s labels team m as Byzantine. Such an f exists by Condition (7). Once a team m is labeled as Byzantine by a supervisor s, supervisor s keeps this label in all subsequent rounds and stops providing belief estimates to team m. This label is also used to prevent identified Byzantine teams from further affecting supervisors’ belief estimates. At round k, supervisor s updates its belief regarding team m according to πk+1m,s=πkm,s+ωkm,sαk(a¯k,rm−πkm,s),for m∈s,1|ks|∑s′∈ksπkm,s′,for m∉sand |ks|>0,πkm,s,for m∉sand |ks|=0, _k\!+\!1^m,s\!\!\!=\!\! cases\! _k^m,s\!\!\!+\! _k^m,s\! _k\!\! (\! a_k,r^m\!\!-\! _k^m,s\! )\!,&\!\!\!\!for m\!∈\!T^s,\\ \! 1|N^s_k| _s ^s_k _k^m,s ,&\!\!\!\!for m\!\!∉\!\!T^sand |N^s_k|\!\!>\!\!0,\\ \! _k^m,s,&\!\!\!\!for m\!\!∉\!\!T^sand |N^s_k|\!\!=\!\!0, cases where ωkm,s=1 _k^m,s=1 if team m has not been labeled as Byzantine by supervisor s, and ωkm,s=0 _k^m,s=0 otherwise. V-B Resilience Analysis We analyze BR-DTOA in two steps. We first examine the Byzantine-resilient mechanism and then study the convergence of BR-DTOA and the associated honest TNG. The following lemma provides exponential bounds on Byzantine-team identification errors. Lemma 2. Given a B-ZSPTG with a supervisor network B=(ℐ,ℋ,ℬ,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ,p)G_B=(I,H,B,S,ST,\SN_k\_k≥ 0,\A^i\_i ,\u^i\_i ,p), for any supervisor s and any team m∈sm ^s, the following statements hold under BR-DTOA: • if m∈ℋm , then ℙ(fkm,s>f)≤exp(−kD(f∥qηFP))P (f_k^m,s\!>\!f )\!≤\! (-kD(f\|q _FP) ); • if m∈ℬm , then ℙ(fkm,s<f)≤exp(−kD(f∥q[(−plie)ηFP+plie(−ηFN)])); (f_k^m,s\!\!\!<\!\!f )\!\!≤\!\! \! (\!\!\!-\!\!kD (f\|q\! [(\!1\!\!-\!\!p_lie\!) _F\!P\!+\!p_lie(\!1\!\!-\!\! _F\!N\!) ]\! )\! ); where D(x∥y)≜xlog(xy)+(1−x)log(1−x1−y)D(x\|y) x ( xy )+(1-x) ( 1-x1-y ) denotes the binary relative entropy. Lemma 2 shows that the probabilities of falsely labeling an honest team as Byzantine and failing to identify a Byzantine team by round k decay exponentially as k→∞k→∞. A larger checking probability q and smaller error rates ηFP _FP and ηFN _FN can improve the identification performance by reducing the misidentification probabilities. The threshold f controls the trade-off between missed identifications of Byzantine teams and false alarms for honest teams. Increasing f reduces the probability of falsely labeling an honest team as Byzantine but increases the probability of failing to identify a Byzantine team. Conversely, decreasing f facilitates Byzantine-team identification but raises the risk of misclassifying honest teams. We now derive convergence guarantees for BR-DTOA from the identification result in Lemma 2 and the convergence analysis of DTOA in the previous section. The first result concerns supervisors’ belief-estimation errors for honest teams. Theorem 3. For a Byzantine ZSPTG with a supervisor network B=(ℐ,ℋ,ℬ,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ,p)G_\!B\!\!=\!\! (I,\!H,\!B,\!S,\!ST,\!\\!SN_k\!\_k≥ 0,\!\\!A^i\!\_i ,\!\\!u^i\!\_i ,\!p ), under the conditions of Theorem 1, for any supervisor s∈s , the following statement holds for BR-DTOA with probability at least 1−δB1- _B: ‖πkm,s−πkm‖∞≤O(ρ[k2])+O(α[k2]),∀m∈ℋ, \| _k^m,s- _k^m\|_∞≤ O(ρ^[ k2])+O( _[ k2]), ∀ m , (8) where δB=δBℋ+δBℬ _B= _B^H+ _B^B with δBℋ=|ℋ|⋅exp(−K⋅D(f∥qηFP))1−exp(−D(f∥qηFP)) _B^H=|H|· (-K· D(f\|q _FP) )1- (-D(f\|q _FP) ) and δBℬ=|ℬ|⋅exp(−K⋅D(f∥q[(1−plie)ηFP+plie(1−ηFN)])) _B^B=|B|· (-K· D (f\|q [(1-p_lie) _FP+p_lie(1- _FN) ] ) ), and ρ is defined as in Theorem 1. If Assumption 3 also holds, (8) simplifies to ‖πkm,s−πkm‖∞≤O(α[k2])\| _k^m,s- _k^m\|_∞≤ O( _[ k2]). Lemma 2 shows that, after the burn-in period, BR-DTOA avoids falsely labeling honest teams and identifies Byzantine teams with high probability. On this event, Theorem 1 can be applied to bound the belief-estimation errors for honest teams, yielding Theorem 3. Thus, the convergence rate remains the same as in the non-Byzantine case, while the guarantee holds with probability at least 1−δB1- _B because of possible identification errors. Moreover, δB→0 _B→ 0 as K→∞K→∞, so a longer burn-in period improves the reliability. The next result establishes the honest-TNG convergence guarantee under BR-DTOA. (a) Three supervisors. (b) Five supervisors. (c) Rounds needed for small errors. Fig. 1: Convergence of belief-estimation errors. Theorem 4. For a Byzantine ZSPTG with a supervisor network B=(ℐ,ℋ,ℬ,,ST,SNkk≥0,ii∈ℐ,uii∈ℐ,p)G_\!B\!\!=\!\! (I,\!H,\!B,\!S,\!ST,\!\\!SN_k\!\_k≥ 0,\!\\!A^i\!\_i ,\!\\!u^i\!\_i ,\!p ), under the conditions of Theorem 2, for any supervisor s∈s , the following inequality holds for BR-DTOA with probability at least 1−δB1- _B lim supk→∞TNGℋ(πks) _k→∞TNG_H ( _k^s ) ≤ ≤ τlog|ℋ|,for BR-DTOA,τlog|ℋ|+|ℋ|2ϕ¯Λ(δ,ϵ),for BR-iDTOA, casesτ |A_H |,&for BR-DTOA,\\ τ |A_H |+ |H |^2 φ (δ,ε),&for BR-iDTOA, cases where δB _B is defined as in Theorem 3, and ℋ=∏m∈ℋ¯mA_H= _m A^m is the joint action set of honest teams. Theorem 4 extends the TNG result in Theorem 2 to the Byzantine attack setting. Conditional on the high-probability event characterized by Lemma 2, the honest-team learning dynamics satisfy an honest-TNG bound of the same form as in the non-Byzantine case, but with the gap evaluated only over honest teams. This yields a sharper guarantee than treating all teams uniformly. Together, BR-DTOA can identify Byzantine teams with vanishing error probability and the Byzantine resilience of BR-DTOA is further established by showing that, in the presence of Byzantine teams, it preserves the convergence of belief-estimation errors for honest teams and provides an honest-TNG guarantee. These results affirmatively answer Problem 2. VI Experimental Validation In this section, we present numerical experiments to illustrate the theoretical results in Sections IV and V, including the convergence, optimality, and Byzantine resilience of the proposed algorithms. We also examine the scalability of DTOA. We further compare DTOA with two baseline methods and numerically extend DTOA to the Markov decision process (MDP) setting. (a) DTOA. (b) Independent DTOA. Fig. 2: Convergence of TNG. (a) More teams. (b) More agents per team. Fig. 3: Scalability tests. Unless otherwise specified, the experiments are conducted on a repeated ZSPTG with a supervisor network consisting of five teams, three supervisors, and two agents in each team. Each agent has the binary action set 0,1\0,1\. The utility and potential functions are chosen separately for different experiments to reflect the corresponding scenarios while preserving the ZSPTG structure. The supervision relationship between teams and supervisors is given by 1=1,2T^1\!\!=\!\!\1,2\, 2=3,4T^2\!\!=\!\!\3,4\, and 3=5T^3\!\!=\!\!\5\. We set the step size to αk=1/(k+1) _k\!=\!1/(k\!+\!1) and the temperature parameter to τ=0.1τ\!=\!0.1. The shaded regions in the figures indicate the variability across ten independent runs. The code is available at https://github.com/zjt-1229/team_game_with_supervisor_network. VI-A Numerical Results for DTOA We first present numerical results for DTOA. These experiments aim to illustrate the convergence behavior and TNG performance characterized by the theoretical analysis and to examine the scalability of DTOA. (a) Honest TNG. (b) Fail-signal frequencies. (c) Effect of q. (d) Effect of pliep_lie. Fig. 4: Performance of the BR-DTOA. Fig. 5: Comparison. Fig. 6: MDP setting. Hypothesis I: convergence of belief-estimation errors. Theorem 1 shows that supervisors’ belief-estimation errors converge to zero, with slower convergence when the number of supervisors |||S| or the communication window length B increases. We examine the convergence of supervisors’ belief-estimation errors in two instances: one with three supervisors and six teams, and the other with five supervisors and ten teams. In both instances, each supervisor supervises two teams, each team contains two agents, and each agent has the binary action set 0,1\0,1\. Each agent randomly selects an action at every round, so the experiment isolates supervisor-network belief learning under random action sequences. Fig. 1 reports the results. The errors in Fig. 1b converge more slowly than those in Fig. 1a, illustrating that increasing |||S| slows the convergence rate. Moreover, in both instances, a larger B leads to slower convergence, as reflected in Fig. 1c by the increased number of rounds required to reach a small error level. These observations are consistent with Theorem 1. Hypothesis I: upper bound on TNG. Theorem 2 implies that DTOA and independent DTOA converge to a near TNE in terms of the TNG, with an asymptotic bound comparable to that of team-FP [10], while sparser supervisor communication can slow convergence. We examine TNG convergence under DTOA and independent DTOA. The independent update probability in independent DTOA is set to δ=0.5δ=0.5. Fig. 2 shows that both DTOA and independent DTOA eventually attain TNGs comparable to those of the corresponding team-FP benchmarks, which is consistent with Theorem 2. The numerical results also show that the TNG curves under DTOA and independent DTOA converge more slowly than those under team-FP. The slower convergence can be attributed to belief-estimation errors induced by distributed belief learning over the supervisor network. A larger communication window length B further slows convergence in both instances, suggesting that sparser supervisor communication delays the reduction of the TNG. Scalability tests. To examine the scalability, we consider two larger instances: one with increased numbers of supervisors and teams, and the other with an increased number of agents per team. In the first instance, we consider eight supervisors and fifteen teams with two agents per team; in the second, we consider three supervisors and five teams with five agents per team. Fig. 3 shows that DTOA still ensures a small TNG in both larger instances, consistent with the small-scale experiments. VI-B Numerical Results for BR-DTOA We next present numerical results for BR-DTOA. These experiments illustrate the implications of the theoretical analysis for Byzantine-team identification and honest-TNG convergence. Hypothesis I: convergence of honest TNG. The theoretical results indicate that BR-DTOA identifies Byzantine teams through accumulated failfail-signal frequencies and provides an honest-TNG guarantee under Byzantine attacks. We consider an instance with two Byzantine teams. The parameters are set as ηFP=ηFN=0.05 _FP= _FN=0.05, K=50K=50, f=0.1f=0.1, q=0.5q=0.5, and plie=0.5p_lie=0.5. To examine parameter sensitivity, we also vary pliep_lie and q separately while keeping all other parameters unchanged. Fig. 4a shows that the honest TNG decreases to a small value under Byzantine attacks, which is consistent with Theorem 4. By contrast, team-FP does not include a Byzantine-resilient mechanism and cannot distinguish Byzantine teams from honest teams. Consequently, the overall TNG under team-FP remains high in the Byzantine attack setting, indicating that team-FP fails to provide an effective performance guarantee for honest teams. Fig. 4b shows that the failfail-signal frequencies of honest and Byzantine teams become separated after the burn-in period K=50K=50. Figs. 4c and 4d show that the failfail-signal frequencies of Byzantine teams increase as pliep_lie and q increase, respectively. These observations are consistent with Lemma 2. VI-C Comparison With Baseline Methods We compare DTOA with multiplicative weights update (MWU) and smoothed fictitious play (SFP) on a two-team ZSPTG instance with one supervisor assigned to each team. For MWU and SFP, we relax the information constraint by allowing access to full opponent information, whereas DTOA relies on supervisor-based distributed belief learning. Fig. 6 shows the results. In this instance, DTOA attains a lower TNG after sufficiently many rounds, while MWU exhibits persistent oscillations and SFP stabilizes at a higher TNG level. This suggests that DTOA can maintain a smaller TNG even under a more restrictive information structure. VI-D Extension to an MDP Setting Although the theoretical analysis in this paper focuses on repeatedly played ZSPTGs, we also examine the proposed supervisor-network learning framework in an MDP setting. The experiment considers three teams and three supervisors, with each team supervised by one supervisor. Fig. 6 shows the results. Both the model-based and model-free variants reduce the TNG through learning, suggesting that the proposed framework can be applied numerically in an MDP setting. Compared with the corresponding results in [10], the reduction in the TNG is slower, which may be attributed to the additional belief-estimation errors induced by the supervisor network. VII Conclusions In this paper, we studied team-orchestrating learning in repeatedly played ZSPTGs with a supervisor network under distributed belief information. We proposed the DTOA, which combines team-FP with distributed belief learning over a supervisor network, established the convergence of supervisors’ belief-estimation errors, and showed that the induced learning dynamics converged to a near TNE with a small TNG. We further considered a Byzantine attack setting, where Byzantine teams could misreport their joint actions and developed the BR-DTOA by integrating DTOA with a supervisor-based identification mechanism. For BR-DTOA, we established the convergence of supervisors’ belief-estimation errors for honest teams and derived an honest TNG guarantee. Numerical simulations illustrated the convergence of the proposed algorithms and the effectiveness of the Byzantine-resilient mechanism. References [1] S. Aggarwal, M. A. u. Zaman, M. Bastopcu, and T. Başar (2026) Semantic communication in multiteam dynamic games: a mean field perspective. IEEE Transactions on Automatic Control 71 (1), p. 49–64. External Links: Document Cited by: §I. [2] G. Alcantara-Jiménez and J. B. Clempner (2020) Repeated Stackelberg security games: learning with incomplete state information. Reliability Engineering & System Safety 195, p. 106695. Cited by: §I. [3] A. Bagchi and T. Basar (1980) Team decision theory for linear continuous-time systems. IEEE Transactions on Automatic Control 25 (6), p. 1154–1161. External Links: Document Cited by: §I. [4] L. E. Blume (1993) The statistical mechanics of strategic interaction. Games and economic behavior 5 (3), p. 387–424. Cited by: §-B. [5] F. Boutcher, W. Berta, R. Urquhart, and A. R. Gagliardi (2022) The roles, activities and impacts of middle managers who function as knowledge brokers to improve care delivery and outcomes in healthcare organizations: a critical interpretive synthesis. BMC Health Services Research 22 (1), p. 11. Cited by: §I. [6] R. Cao and Y. Zhao (2026) Distributed average tracking over directed communication networks: a nonsmooth surplus approach. IEEE Transactions on Automatic Control 71 (4), p. 2450–2465. External Links: Document Cited by: §I. [7] G. Chen, Y. Ming, Y. Hong, and P. Yi (2021) Distributed algorithm for ε -generalized Nash equilibria with uncertain coupled constraints. Automatica 123, p. 109313. Cited by: §I. [8] G. Chen, G. Xu, F. He, D. Tao, T. Parisini, and K. H. Johansson (2025) Inverse learning of black-box aggregator for robust Nash equilibrium. IEEE Transactions on Automatic Control. Cited by: §I. [9] Z. Deng and J. Luo (2024) Distributed algorithm for nonsmooth multi-coalition games and its application in electricity markets. Automatica 161, p. 111494. Cited by: 1st item, §I, §I. [10] A. Dönmez, Y. Arslantaş, and M. Sayin (2024) Team-fictitious play for reaching team-Nash equilibrium in multi-team games. Advances in Neural Information Processing Systems 37, p. 111515–111543. Cited by: §-B, §-B, §-B, 1st item, §I, §I, §I, §I-A, §I-B, §IV-B, §IV-C, §IV-C, §VI-A, §VI-D. [11] J. H. Dreze and J. Greenberg (1980) Hedonic coalitions: optimality and stability. Econometrica: Journal of the Econometric Society, p. 987–1003. Cited by: §I. [12] X. Feng, J. Huang, Y. Jia, and M. Yu (2025) Mean-field team in backward linear-quadratic control problems with model uncertainty. Science China Information Sciences 68 (11), p. 210203. Cited by: §I. [13] X. Feng, J. Huang, and Z. Qiu (2021) Mixed social optima and Nash equilibrium in linear-quadratic-gaussian mean-field system. IEEE Transactions on Automatic Control 67 (12), p. 6858–6865. Cited by: 1st item, §I, §I. [14] A. Haeberlen, P. Kouznetsov, and P. Druschel (2006) The case for Byzantine fault detection. In Proceedings of the 2nd Conference on Hot Topics in System Dependability, p. 5–5. Cited by: §V. [15] J. Hajduková (2006) Coalition formation games: a survey. International Game Theory Review 8 (04), p. 613–641. Cited by: §I. [16] J. Huang, Z. Qiu, S. Wang, and Z. Wu (2024) Linear quadratic mean-field game-team analysis: a mixed coalition approach. Automatica 159, p. 111358. Cited by: §I. [17] M. Huang, R. P. Malhamé, and P. E. Caines (2006) Large population stochastic dynamic games: closed-loop Mckean-Vlasov systems and the Nash certainty equivalence principle. Communication Information Systems 6 (1), p. 221–252. Cited by: §I. [18] B. Jeloka, Y. Guan, and P. Tsiotras (2025) Learning large-scale competitive team behaviors with mean-field interactions. In The Seventeenth Workshop on Adaptive and Learning Agents, Cited by: §I. [19] J. Kim, T. R. Palfrey, and J. R. Zeidel (2022) Games played by teams of players. American Economic Journal: Microeconomics 14 (4), p. 122–157. Cited by: §I. [20] H. J. LeBlanc, H. Zhang, X. Koutsoukos, and S. Sundaram (2013) Resilient asymptotic consensus in robust networks. IEEE Journal on Selected Areas in Communications 31 (4), p. 766–781. Cited by: §I. [21] J. Li, W. Abbas, M. Shabbir, and X. Koutsoukos (2022) Byzantine resilient distributed learning in multirobot systems. IEEE Transactions on Robotics 38 (6), p. 3550–3563. Cited by: §V. [22] S. Li, J. Guo, J. Xiu, R. Xu, X. Yu, J. Wang, A. Liu, Y. Yang, and X. Liu (2024) Byzantine robust cooperative multi-agent reinforcement learning as a Bayesian game. In The 12th International Conference on Learning Representations, Cited by: §I. [23] T. Ma, Z. Deng, and C. Hu (2022) A fully distributed Nash equilibrium seeking algorithm for N-coalition games of euler–lagrange players. IEEE Transactions on Control of Network Systems 10 (1), p. 205–213. Cited by: §I, §I. [24] A. A. Malikopoulos (2023) On team decision problems with nonclassical information structures. IEEE Transactions on Automatic Control 68 (7), p. 3915–3930. External Links: Document Cited by: §I. [25] J. R. Marden and J. S. Shamma (2012) Revisiting log-linear learning: asynchrony, completeness and payoff-based implementation. Games and Economic Behavior 75 (2), p. 788–808. Cited by: §-B. [26] B. Moll and L. Ryzhik (2026) Mean field games without rational expectations. Communications in Contemporary Mathematics, p. 2640007. Cited by: §I. [27] L. Moreau (2005) Stability of multiagent systems with time-dependent communication links. IEEE Transactions on automatic control 50 (2), p. 169–182. Cited by: Assumption 2. [28] J. W. Neal, Z. P. Neal, and B. Brutzman (2022) Defining brokers, intermediaries, and boundary spanners: a systematic review. Evidence & policy 18 (1), p. 7–24. Cited by: §I. [29] W. Saad, Z. Han, T. Basar, M. Debbah, and A. Hjorungnes (2010) Hedonic coalition formation for distributed task allocation among wireless agents. IEEE Transactions on Mobile Computing 10 (9), p. 1327–1344. Cited by: §I. [30] S. Sanjari and S. Yüksel (2020) Optimal solutions to infinite-player stochastic teams and mean-field teams. IEEE Transactions on Automatic Control 66 (3), p. 1071–1086. Cited by: §I. [31] Y. Shi and B. Zhang (2020) Multi-agent reinforcement learning in Cournot games. In 2020 59th IEEE Conference on Decision and Control, p. 3561–3566. Cited by: §I. [32] M. J. Wainwright (2019) High-dimensional statistics: a non-asymptotic viewpoint. Vol. 48, Cambridge university press. Cited by: §-C. [33] C. Wang, M. Moharrami, K. Jin, D. Kempe, P. J. Brantingham, and M. Liu (2021) Structural stability of a family of group formation games. In 2021 60th IEEE Conference on Decision and Control, Vol. , p. 3080–3085. External Links: Document Cited by: §I. [34] J. Wang, D. W. C. Ho, X. Jin, F. Li, and Y. Tang (2025) Multi-agent target-attacker-defender differential games with anomalous defenders under limited perception. IEEE Transactions on Automatic Control (), p. 1–16. External Links: Document Cited by: §I. [35] J. Wang, X. Deng, J. Guo, and Z. Zeng (2023) Resilient consensus control for multi-agent systems: a comparative survey. Sensors 23 (6), p. 2904. Cited by: §I. [36] Z. Wu, M. Laurière, S. J. C. Chua, M. Geist, O. Pietquin, and A. Mehta (2024) Population-aware online mirror descent for mean-field games by deep reinforcement learning. In Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, p. 2561–2563. Cited by: §I. [37] J. Xiao, C. Qiu, Y. Xu, J. Zhang, S. Qi, and X. Wang (2025) Solving equilibrium for adversarial team games utilizing fictitious team play with refined team plans. Expert Systems with Applications, p. 129496. Cited by: §I. [38] J. Yan, C. Deng, and C. Wen (2021) Resilient output regulation in heterogeneous networked systems under Byzantine agents. Automatica 133, p. 109872. Cited by: §I. [39] L. Yuan and H. Ishii (2021) Secure consensus with distributed detection via two-hop communication. Automatica 131, p. 109775. Cited by: §I. [40] X. Zeng, J. Chen, S. Liang, and Y. Hong (2019) Generalized Nash equilibrium seeking strategy for distributed nonsmooth multi-cluster game. Automatica 103, p. 20–26. Cited by: §I, §I. [41] H. Zhang, G. Chen, and Y. Hong (2023) Distributed algorithm for continuous-type Bayesian Nash equilibrium in subnetwork zero-sum games. IEEE Transactions on Control of Network Systems 11 (2), p. 915–927. Cited by: §I, §I. [42] Q. Zhu and T. Basar (2015) Game-theoretic methods for robustness, security, and resilience of cyberphysical control systems: games-in-games principle for optimal cross-layer resilient control systems. IEEE Control Systems Magazine 35 (1), p. 46–65. Cited by: §I. -A Proof of Theorem 1 We prove Theorem 1 by first establishing a technical result on the supervisor networks SNkk≥0\SN_k\_k≥ 0 and then proving the convergence of the belief-estimation errors. Lemma 3. Let Wk∈ℝ||×||W_k ^|S|×|S| denote the gossip matrix associated with the supervisor network at round k, where Wk(s,s′)=SNk(s,s′)‖SNk(s,⋅)‖1,if ‖SNk(s,⋅)‖1>0,s′=s,if ‖SNk(s,⋅)‖1=0. W_k(s,s )\!=\! cases SN_k(s,s )\|SN_k(s,·)\|_1,&\!\!\!if \|SN_k(s,·)\|_1\!>\!0,\\ 1\s \!=\!s\,&\!\!\!if \|SN_k(s,·)\|_1\!=\!0. cases If Assumption 2 holds, there exists some L∈ℕ+L _+ such that, for any supervisors s, s′s and any k≥0k≥ 0, the following statement holds: [Wk+L⋯Wk](s,s′)≥[1||]L [W_k+L·s W_k ](s,s )≥ [ 1|S| ]^L. Proof. We consider a setting in which Wk(s,s′)W_k(s,s ) represents the probability of s reaching s′s at round k; therefore, WkW_k is a transition matrix. A path on the dynamic supervisor network SNkk≥0\SN_k\_k≥ 0 is defined as =(Xk∈)k≥0P=(X_k )_k≥ 0 such that Wk(Xk,Xk+1)>0W_k(X_k,X_k+1)>0. Given k≥0k≥ 0 and s=Xks=X_k, there exists n∈ℕn such that nB≤k<(n+1)BnB≤ k<(n+1)B, and we have [W(n+1)B⋯Wk](s,s)≥[1||](n+1)B−k≥[1||]B [\!W_(n+1)B·s W_k\! ]\!(s,s)\!\!≥\!\! [\! 1|S|\! ]^(n\!+\!1)\!B\!-\!k\!\!≥\!\! [\! 1|S|\! ]^B. Let s(l)N^s(l) denote the supervisors that can be reached with positive probability at time step (n+)B+l(\!n\!+\!1\!)\!B\!+\!l. It follows that s(l)⊆s(l+1)N^s\!(l)\!\! \!\!N^s(l\!+\!1) since Wl(s′,s′)>0W_l(\!s \!,s )\!\!>\!\!0 for all s′∈s(l)s \!\!∈\!\!N^s(l). If s(n′B)≠N^s\!(n \!B)\!\!≠\!\!S, there exists n′B≤l′<(n′+1)Bn \!B\!≤\!\!l \!\!<\!\!(n \!+\!1)B, s∈s(n′B)s\!\!∈\!\!N^s\!(n \!B) and s′∉s(n′B)s \!\!∉\!\!N^s(n \!B) such that s,s′∈El \\!s,s \! \\!\!∈\!\!E_l due to Assumption 2. Hence, we obtain s(n′B)⊊s(l′)⊆s((n′+)B)N^s\!(n \!B)\!\! \!\!N^s(l )\!\! \!\!N^s (\!(\!n \!+\!1\!)\!B\! ). Then we can conclude that s(||B)=N^s\!(|S|B)\!\!=\!\!S. Let L=(||+1)BL\!\!=\!\!(|S|\!+\!1)\!B, then we have ℙ(Xk+L=s′|Xk=s)>0P\! (\!X_k\!+\!L\!\!=\!\!s |\!X_k\!\!=\!\!s\! )\!\!>\!\!0. By the definition of Wl′W_l , Wl(s,s′)>0W_l(s,s )\!\!>\!\!0 implies Wl′(s,s′)≥1||W_l (s,s )\!\!≥\!\! 1|S|. It follows that there exists at least one path satisfying ℙ(Xk+L=s′|Xk=s)>0P (\!X_k+L\!\!=\!\!s |\!X_k\!\!=\!\!s\! )\!\!>\!\!0, namely, s,s′,L=(Xk=s,…,Xl′=s,Xl′+1=s′,…,Xk+L=s′)P_\!s,s \!,L\!\!=\!\! (\!X_k\!\!=\!\!s,\!…\!,X_l \!\!=\!\!s,X_l \!+\!1\!\!=\!\!s \!,\!…\!,X_k\!+\!L\!\!=\!\!s ). Thus, we have ℙ(Xk+L=s′|Xk=s)≥ℙ(s,s′,L)≥[1||]LP (\!X_k\!+\!L\!\!=\!\!s \! |\!X_k\!\!=\!\!s\! )\!\!≥\!\!P\! (P_\!s,s \!,L )\!\!≥\!\! [\! 1|S|\! ]^\!L. This completes the proof. ∎ Now we can prove Theorem 1. We first define the error of πkm,s _k^m,s as the difference between πkm,s _k^m,s and πkm _k^m: ekm,s=πkm,s−πkm,∀m∈,∀s∈, e_k^m,s= _k^m,s- _k^m, ∀ m ,\ ∀ s , (9) and then we analyze the decay rate of ekm,se_k^m,s. Given a team m, we introduce the following notation: Dm=diag(ST(m,1),…,ST(m,||))D^m=diag (ST(m,1),…,ST(m,|S|) ). Then we obtain (πk+1m,1⋮πk+1m,||)=Dm(πk+1m⋮πk+1m)+(I−Dm)Wk(πkm,1⋮πkm,||). (\!\! array[]c _k+1^m,1\\[-3.0pt] \\[-3.0pt] _k+1^m,|S|\\ array\!\! )\!\!=\!D^m\!\! (\!\! array[]c _k+1^m\\[-3.0pt] \\[-3.0pt] _k+1^m\\ array\!\! )\!\!+\!(I\!-\!D^m)W_k\! (\!\! array[]c _k^m,1\\[-3.0pt] \\[-3.0pt] _k^m,|S|\\ array\!\! ). Since ‖Wk(s,⋅)‖1>0\|W_k(s,·)\|_1>0, combining (9) and (-A), we obtain (ek+1m,1⋮ek+1m,||)=(πk+1m,1⋮πk+1m,||)−(πk+1m⋮πk+1m) ( array[]ce_k+1^m,1\\[-3.0pt] \\[-3.0pt] e_k+1^m,|S|\\ array )= ( array[]c _k+1^m,1\\[-3.0pt] \\[-3.0pt] _k+1^m,|S|\\ array )- ( array[]c _k+1^m\\[-3.0pt] \\[-3.0pt] _k+1^m\\ array ) = = (I−Dm)Wk(ekm,1⋮ekm,||)+(I−Dm)(πkm−πk+1m⋮πkm−πk+1m). (I\!-\!D^m)W_k\! (\!\! array[]ce_k^m,1\\[-3.0pt] \\[-3.0pt] e_k^m,|S|\\ array\!\! )\!+\!(I\!-\!D^m)\!\! (\!\! array[]c _k^m- _k+1^m\\[-3.0pt] \\[-3.0pt] π^m_k- _k+1^m\\ array\!\! )\!. Let ekm≜(ekm,1⋮ekm,||)e_k^m\!\! \!\!\! (\!\!\! array[]ce_k^m,1\\[-5.0pt] \\[-5.0pt] e_k^m,|S|\\ array\!\!\! ), bk+1m≜(I−Dm)(πkm−πk+1m⋮πkm−πk+1m)b_k+1^m\!\! \!\!(I-D^m)\!\! (\!\!\! array[]c _k^m\!-\! _k+1^m\\[-5.0pt] \\[-5.0pt] π^m_k\!-\! _k+1^m\\ array\!\!\! ) and Bkm≜(I−Dm)WkB^m_k\!\! \!\!(I\!-\!D^m)W_k. Then we rewrite (-A) as ek+1m=Bkmekm+bk+1me_k+1^m\!=\!B^m_ke_k^m\!+\!b_k+1^m. Using the above recursion, we obtain ekm=ℬm(k,0)e0m+∑l=0k−1ℬm(k,l+1)blme_k^m\!=\!B^m(k,0)e_0^m\!+\! _l=0^k-1B^m(k,l\!+\!1)b_l^m, where ℬm(k,l)=Bk−1m⋯BlmB^m(k,l)\!=\!B^m_k-1·s B_l^m. By reordering the indices, WkmW_k^m and DmD^m can be written as Wkm=(Wk,UmUmWk,UmOmWk,OmUmWk,OmOm),D~m=(000I). W^m_k= ( array[]cW_k,U^mU^m&W_k,U^mO^m\\ W_k,O^mU^m&W_k,O^mO^m\\ array ),\ D^m= ( array[]c0&0\\ 0&I\\ array ). Then we have Bkm=(Wk,UmUmWk,UmOm00),ℬm(k,l)=(BUUm(k,l)BUOm(k,l)00), B_k^m\!\!\!=\!\!\! (\!\!\! array[]cW_k,U^m\!U^m&\!\!W_k,U^m\!O^m\\ 0&0\\ array\!\!\!\! )\!\!,B^m\!(\!k,\!l)\!\!=\!\!\! (\!\!\! array[]cB^m_U\!U\!(\!k,\!l)&\!\!B^m_U\!O\!(\!k,\!l)\\ 0&0\\ array\!\!\! )\!\!, where BUUm(k,l)=Wk−1,UmUm⋯Wl,UmUmB^m_U\!U\!(k,l)\!\!=\!\!W_k\!-\!1,U^m\!U^m\!·s\!W_l,U^m\!U^m and BUOm(k,l)=Wk−1,UmOm⋯Wl,UmOmB^m_UO(k,l)\!\!=\!\!W_k-1,U^mO^m·s W_l,U^mO^m. We denote ‖Wm(k,l)‖∞=Rm(k,l)=maxs∈Um∑s′∈Um[Wm(k,l)](s,s′)\|\!W^m\!(k,\!l)\!\|_∞\!\!=\!\!R^m\!(\!k,\!l)\!\!=\!\! _s∈ U^m\!\! _s \!∈\!U^m\!\! [W^m\!(\!k,\!l) ]\!(\!s,\!s ) and Wm(k,l)=Wk−1,UmUm⋯Wl,UmUmW^m(k,l)\!\!=\!\!W_\!k\!-\!1,U^\!m\!U^\!m\!·s\!W_\!l,U^\!m\!U^\!m. By Lemma 3, we have Rm(l+L,l)≤−αR^m\!(l\!+\!L,l)\!\!≤\!\!1\!\!-\!\!α with α=[1||]Lα\!\!=\!\!\! [\! 1|S|\! ]^\!\!L. Thus, we get Rm(k,l)≤α[k−lL]≤C′ρk−lR^m\!(k,l)\!\!≤\!\!α^[ k\!-\!lL]\!\!≤\!\!C \!ρ^k\!-\!l, where C′=(−α)−1>1C \!\!=\!\!(1\!\!-\!\!α)^\!-\!1\!\!>\!\!1 and ρ=(−α)1L<1ρ\!\!=\!\!(1\!\!-\!\!α)^\! 1L\!\!<\!\!1. For any x=(xUm,xOm)T∈ℝmx\!\!=\!\!(x_U^m,x_O^m)^T\!\!∈\!\!R^m, we get Blmx=(yUm,0)TB^m_lx\!\!=\!\!(y_U^m,0)^T. Then we obtain ℬm(k,l)x=(Wm(k,l+1)yUm,0)TB^m(k,l)x\!\!=\!\!(W^m(k,l\!+\!1)y_U^m,0)^T and have ‖ℬm(k,l)x‖∞ \|B^m(k,l)x\|_∞ ≤‖Wm(k,l+1)‖∞‖yUm‖∞ ≤\|W^m(k,l+1)\|_∞\|y_U^m\|_∞ ≤C′ρk−l−1‖yUm‖∞. ≤ C \!ρ^k-l-1\|y_U^m\|_∞. (34) From yUm=Wl,UmUmxUm+Wl,UmOmxOm≜M(l)xy_U^m\!\!=\!\!W_l,U^mU^mx_U^m\!\!+\!\!W_l,U^mO^mx_O^m\!\! \!\!M(l)x and ‖M(l)‖∞≤1\|\!M(l)\!\|_∞\!\!≤\!\!1, we get that ‖yUm‖∞≤‖x‖∞\|y_U^m\|_∞\!\!≤\!\!\|x\|_∞, combing which and (-A), we have ‖ℬm(k,l)‖∞≤C′ρk−l\|B^m(k,l)\!\|_∞\!\!≤\!\!C \!ρ^k-l. Hence, we obtain ‖ekm‖∞=‖ℬm(k,0)e0m+∑l=0k−1ℬm(k,l+1)blm‖∞ \|e_k^m\|_∞=\|B^m(k,0)e_0^m+ _l=0^k-1B^m(k,l+1)b_l^m\|_∞ ≤ ≤ C′ρk‖e0m‖∞+∑l=1k−1C′ρk−l−1‖blm‖∞ C ρ^k\|e_0^m\|_∞+ _l=1^k-1C ρ^k-l-1\|b_l^m\|_∞ = = C′ρk‖e0m‖∞+∑l=1[k2]−1C′ρk−l−1‖blm‖∞+∑l=[k2]k−1C′ρk−l−1‖blm‖∞ C \!\!ρ^k\|e_0^m\!\|_∞\!\!+\!\!\! _l=1^[ k2]-1\!\!C \!ρ^k-l-1\|b_l^m\!\|_∞\!\!+\!\!\! _l=[ k2]^k-1\!\!C \!ρ^k-l-1\!\|b_l^m\!\|_∞ ≤ ≤ C′ρk‖e0m‖∞+C′∑l=k−[k2]∞ρlsupl≥0‖blm‖∞+C′∑l=0∞ρlsupl≥[k2]‖blm‖∞ C \!\!ρ^k\!\|e_0^m\!\|_∞\!\!+\!C \!\!\!\! _l=k-[ k2]^∞\!\!\!ρ^l\! sup_l≥ 0\!\|b_l^m\!\|_∞\!\!+\!C \! _l=0^∞\!ρ^l\!\! sup_l≥[ k2]\!\|b_l^m\!\|_∞ ≤ ≤ C′ρk‖e0m‖∞+C′supl≥0‖blm‖∞ρk−[k2]1−ρ+C′supl≥[k2]‖blm‖∞11−ρ C \!\!ρ^k\!\|e_0^m\!\|_∞\!\!+\!C \! sup_l≥ 0\!\|b_l^m\!\|_∞ ρ^k-[ k2]1-ρ\!\!+\!C \!\! sup_l≥[ k2]\!\!\|b_l^m\!\|_∞ 11-ρ ≤ ≤ C′ρk‖e0m‖∞+C′supl≥0‖blm‖∞ρk21−ρ+C′supl≥[k2]‖blm‖∞11−ρ C \!ρ^k\!\|e_0^m\!\|_∞\!\!+\!C \! sup_l≥ 0\!\|b_l^m\!\|_∞ ρ k21-ρ\!\!+\!C \!\! sup_l≥[ k2]\!\|b_l^m\!\|_∞ 11-ρ = = C′ρk‖e0m‖∞+C′supl≥0αl−1‖(I−Dm)(a¯l−1m−πl−1m)‖∞ρ[k2]1−ρ C \!ρ^k\!\|e_0^m\!\|_∞\!\!+\!C \!\! sup_l≥ 0 _l-1\!\|(\!I\!\!-\!\!D^m\!)( a_l-1^m\!\!-\!\! _l-1^m\!)\|_∞ ρ^[ k2]1-ρ +C′1−ρsupl≥[k2]αl−1∥(I−Dm)(a¯l−1m−πl−1m)∥∞ + C 1-ρ sup_l≥[ k2] _l-1\|(I-D^m)( a_l-1^m- _l-1^m)\|_∞ ≤ ≤ C′ρk‖e0m‖∞+2C′α0⋅ρ[k2]1−ρ+2C′1−ρ⋅α[k2]−1 C \!ρ^k\!\|e_0^m\!\|_∞\!\!+\!\!2C _0· ρ^[ k2]1-ρ+ 2C 1-ρ· _[ k2]-1 = = O(ρ[k2])+O(α[k2]). O(ρ^[ k2])+O( _[ k2]). Finally, we can conclude ‖πkm,s−πkm‖∞≤O(ρ[k2])+O(α[k2])\| _k^m,s- _k^m\|_∞≤ O(ρ^[ k2])+O( _[ k2]) and complete the proof of Theorem 1. -B Proof of Theorem 2 The TNG of Algorithm 1 is closely related to the true belief πkm _k^m. The proof of Theorem 2 proceeds in two steps: 1) constructing a reference scenario where true beliefs are frozen and showing that members within a team can learn to team up in spite of the errors caused by the supervisor networks; 2) bounding the TNG by leveraging stochastic differential inclusion approximations. Given a team strategy profile π=πm∈Δ(¯m)m∈π\!\!=\!\! \\!π^m\!\!∈\!\! \! ( A^m )\!\! \_\!m∈\!T, for any agent i∈ℐmi\!\!∈\!\!I^m and any a−i∈∏j∈ℐm,j≠ija^\!-\!i\!\!∈\!\! _j ^m\!,j≠ i\!A^j, (1) implies brτ(ui(⋅,a−i,π−m))=brτ(ϕm(⋅,a−i,π−m))br_\!τ\! (\!u^i\!(\!·,a^\!-\!i\!,\!π^\!-\!m\!)\! )\!\!=\!\!br_\!τ (\!φ^m\!(\!·,a^\!-\!i\!,\!π^\!-\!m\!)\! ), where π−m≜πll≠mπ^\!-\!m\!\! \!\!\π^l\_\!l≠ m. We divide the learning process into epochs of length T. By accumulating the true belief update (5) from nTnT to (n+)T−1(\!n\!+\!1\!)T\!-\!1, we obtain π(n+)Tm=[∏k=nT(n+)T−1(−αk)]πnTm+∑k=nT(n+)T−1αk[∏l=k+1(n+)T−1(−αl)]a¯km, _(\!n\!+\!1\!)T^m\!\!=\!\!\! [\! _k=nT^(\!n\!+\!1\!)T\!-\!1\!\!\!\!(\!1\!\!-\!\! _k\!)\! ]\!\! _nT^m\!+\!\!\! _k=nT^(\!n\!+\!1\!)T\!-\!1\!\!\!\!\! _k\!\! [\! _l=k\!+\!1^(\!n\!+\!1\!)T\!-\!1\!\!\!\!(\!1\!\!-\!\! _l\!)\! ]\!\! a_k^m\!, (35) where n=0,1,…n\!\!=\!\!0,\!1,\!… is the epoch index. Let π(n)m≜πnTmπ^m_(n)\!\! \!\!π^m_nT and π(n)m,s≜πnTm,sπ^m,s_(n)\!\! \!\!π^m,s_nT denote the true belief and the actual belief about team m after learning n epochs. We further define βk≜αk∏l=k+1(n+1)T−1(1−αl) and β(n)≜∑k=nT(n+1)T−1βk. _k _k _l=k+1^(n+1)T-1\!\!(1- _l)\ and \ _(n) _k=nT^(n+1)T-1\!\! _k. (36) Then, we rewrite (35) as π(n+1)Tm=(−β(n))⋅π(n)m+β(n)[∑k=nT(n+1)T−1βkβ(n)⋅a¯km]. _(n\!+\!1)T^m\!=\! (\!1\!-\! _(n)\! )\!·\! _(n)^m\!+\! _(n)\!\! [ _k=nT^(n\!+\!1)T\!-\!1\!\! _k _(n)· a_k^m ]. (37) By Assumption 3 and Lemma 5.4 of [10], we obtain β(n)→0 as n→∞,∑n=0∞β(n)=∞and∑n=0∞β(n)2<∞. _(n)\!\!→\!0 as n\!\!→\!∞,\ _n=0^∞\! _(n)\!=\!∞\ and\ _n=0^∞\! _(n)^2\!\!<\!∞. (38) Let ℱ(n)F_(n) denote the filtration generated by the σ-algebra σ(,ST,SNkk≥0,A0,…,AnT−1)σ(S,ST,\SN_k\_k≥ 0,A_0,…,A_nT-1), where AtA_t denotes the joint action profile of all teams at round t, i.e., At=(a¯t1,…,a¯t||)A_t=( a_t^1,…, a_t^|T|). Note that π(n)m _(n)^m and π(n)m,s _(n)^m,s are ℱ(n)F_(n)-measurable. The joint action distributions of team m based on the true beliefs, i.e., in the ideal scenario, are defined for DTOA and iDTOA at time k in epoch n as team-ν(n),km≜[a¯km|ℱ(n)],indp-ν(n),km≜[a¯km|ℱ(n)] - _(n),k^m [ a_k^m |F_(n) ],\ indp- _(n),k^m [ a_k^m |F_(n) ] for k=nT,…,(n+1)T−1k=nT,…,(n\!+\!1)T\!-\!1. We also define the distributions of the joint action of team m based on the actual beliefs at round k in epoch n for DTOA and iDTOA as a-team-ν(n),km≜[a¯km|ℱ(n)],a-indp-ν(n),km≜[a¯km|ℱ(n)] -team- _(n),k^m\! \!E [ a_k^m |F_(n) ]\!\!,a-indp- _(n),k^m\! \!E [ a_k^m |F_(n) ] for k=nT,…,(n+)T−1k\!\!=\!\!nT\!,…,\!(\!n\!+\!1\!)T\!\!-\!\!1. For notational simplicity, we use ν(n),km _(n),k^m to represent team-ν(n),kmteam- _(n),k^m for DTOA and indp-ν(n),kmindp- _(n),k^m for iDTOA, and use a-ν(n),kma- _(n),k^m to represent a-team-ν(n),kma-team- _(n),k^m for DTOA and a-indp-ν(n),kma-indp- _(n),k^m for iDTOA. Then, we rewrite (37) in form of stochastic approximation: π(n+)m=(−β(n))π(n)m+β(n)[∑k=nT(n+)T−1βkβ(n)ν(n),km+a-ω(n+)m], _(\!n\!+\!1\!)^m\!\!=\!\! (\!1\!\!-\!\! _(\!n\!)\! )\! _(\!n\!)^m\!\!+\!\! _(\!n\!)\!\!\! [\! _k=nT^(\!n\!+\!1\!)T\!-\!1\!\!\!\! _k _(\!n\!)\! _(\!n\!),k^m\!\!+\!a- _(\!n\!+\!1\!)^m\! ]\!\!, (39) where a-ω(n+1)ma- _(n+1)^m is defined as a-ω(n+1)m≜ - _(n+1)^m ∑k=nT(n+1)T−1βkβ(n)[a¯km−ν(n),km] _k=nT^(n+1)T-1 _k _(n) [ a_k^m- _(n),k^m ] = = ∑k=nT(n+)T−1βkβ(n)[a¯km−a-ν(n),km]+∑k=nT(n+)T−1βkβ(n)[a-ν(n),km−ν(n),km] \!\!\! _k\!=\!nT^(\!n\!+\!1\!)\!T\!-\!1\!\!\!\! _k _(\!n\!)\!\! [ a_k^m\!\!-\!\!a- _(\!n\!),k^m\! ]\!\!+\!\!\!\!\!\! _k\!=\!nT^(\!n\!+\!1\!)\!T\!-\!1\!\!\!\! _k _(\!n\!)\!\! [a- _(n),k^m\!\!-\!\! _(\!n\!),k^m\! ] = = ω(n+1)m+a-e(n+1)m, _(n+1)^m+a-e_(n+1)^m, (40) where ω(n+)m≜∑k=nT(n+)T−1βkβ(n)[a¯km−a-ν(n),km] _(\!n\!+\!1\!)^m\!\! \!\! _\!k\!=\!nT^\!(\!n\!+\!1\!)T\!-\!1\!\! _k _(\!n\!)\!\! [\! a_k^m\!\!-\!\!a- _(\!n\!)\!,k^m\! ] is a martingale difference sequence and we define the actual error as a-e(n+1)m≜∑k=nT(n+1)T−1βkβ(n)[a-ν(n),km−ν(n),km]a-e_(n\!+\!1)^m\!\! \!\! _k=nT^(n\!+\!1)T\!-\!1\!\! _k _(n)\! [\!a- _(n),k^m\!\!-\!\! _(n),k^m\! ]. It follows that [ω(n+1)m|ℱ(n)]=0E\! [ _(n+1)^m |F_(n) ]\!\!\!\!=0. Then we are ready to prove Lemma 1. Recall the result in Lemma 1: the difference between a-ν(n),kma- _(n),k^m and ν(n),km _(n),k^m can be bounded as ‖a-ν(n),km−ν(n),km‖1≤Cδ(n) \|a- _(n),k^m\!\!-\!\! _(n),k^m \|_1\!\!≤\!\!C _(n), where δ(n)→0 _(n)\!\!→\!0 as n→∞n\!\!→\!∞. Proof. We first define the joint-action process from the start of epoch n. We denote the joint-action profiles of all teams by ωkk≥nT≜(a¯km)m∈|ℱ(n)\ω_k\_k≥ nT \ ( a_k^m )_m |F_(n) \, and view each joint-action profile ωkω_k as a state. The transitions between states depend only on the beliefs about all teams, STST, and SNkk≥0\SN_k\_k≥ 0. Let P(n),kP_(n),k denote the transition probabilities between states and let a-ν(n),ka- _(n),k denote the state distribution at round k in the actual scenario. Let P(n)∗P^*_(n) denote the transition probabilities between states and let ν(n),k _(n),k denote the state distribution at round k in the ideal scenario. Then, the joint-action distributions of team m at round k within epoch n in the actual and ideal scenarios are defined as a-ν(n),km(a)=∑ω:am=a-ν(n),k(ω),ν(n),km(a)=∑ω:am=aν(n),k(ω). a- _(\!n\!),k^m(a)\!\!=\!\!\!\!\!\! _ω:a^m\!=\!a\!\!\!a- _(\!n\!),k(ω), _(\!n\!),k^m(a)\!\!=\!\!\!\!\!\! _ω:a^m\!=\!a\!\!\! _(\!n\!),k(ω). (41) Since ϕmφ^m is linear in π, the smoothed best response is Lipschitz in π. Thus, there exists Lπ>0L_π>0 such that ‖P(n),k(⋅|ωk,…,ωnT;πk)−P(n),k∗(⋅|ωk,…,ωnT;πk)‖TV \|\!P_(n),k\! (· | _k,\!…\!, _nT;\! _k^S )\!\!-\!\!P^*_(n),k\! (· | _k,\!…\!, _nT;\! _k )\! \|_TV ≤ ≤ Lπ∑m∈maxs∈‖πkm,s−πkm‖1, L_π _m _s \| _k^m,s- _k^m\|_1, where πk _k^S denotes the belief estimates provided by the supervisor network, based on which agents choose their actions. Consider the error Dk≜‖a-ν(n),k−ν(n),k‖1D_k\!\! \!\! \|a- _(n),k\!\!-\!\!ν_(n),k \|_1. Since a-ν(n),k+1=a-ν(n),kP(n),ka- _(\!n\!),k\!+\!1\!\!=\!\!a- _(\!n\!),kP_(\!n\!),k and ν(n),k+1=ν(n),kP(n),k∗ν_(\!n\!),k\!+\!1\!\!=\!\!ν_(\!n\!),kP_(\!n\!),k^*, we have Dk+1= D_k+1= ‖a-ν(n),kP(n),k−ν(n),kP(n),k∗‖1 \|a- _(n),kP_(n),k-ν_(n),kP_(n),k^*\|_1 = = ‖(a-ν(n),k−ν(n),k)P(n),k∗+a-ν(n),k(P(n),k−P(n),k∗)‖1. \|\!\! (a- _(n),k\!\!-\!\!ν_(n),k )\!\!P_(n),k^*\!\!+\!a- _(n),k\!\! (\!\!P_(n),k\!\!-\!\!P_(n),k^*\! )\!\! \|_1\!. Using the fact that ‖xP‖1≤‖x‖1\|xP\|_1≤\|x\|_1 and the above decomposition, we obtain Dk+1≤ D_k+1≤ ‖(a-ν(n),k−ν(n),k)P(n),k∗‖1+‖a-ν(n),k(P(n),k−P(n),k∗)‖1 \|\! (a- _(\!n\!),k\!-\!ν_(\!n\!),k )\!\!P_(\!n\!),k^*\! \|_1\!\!+\!\! \|a- _(\!n\!),k\!\! (\!\!P_\!(\!n\!),k\!\!-\!\!P_\!(\!n\!),k^*\! )\! \|_1 ≤ ≤ Dk+‖a-ν(n),k(P(n),k−P(n),k∗)‖1 D_k+ \|a- _(n),k (P_(n),k-P_(n),k^* ) \|_1 ≤ ≤ Dk+2supω‖P(n),k(ω,⋅)−P(n),k∗(ω,⋅)‖TV D_k+2 _ω\|P_(n),k(ω,·)-P_(n),k^*(ω,·)\|_TV ≤ ≤ Dk+2Lπ∑m∈maxs∈‖πkm,s−πkm‖1. D_k+2L_π _m _s \| _k^m,s- _k^m\|_1. By the definition of the maximum belief-estimation error in epoch n, δ(n)=maxk∈[nT−1,(n+)T−1]∑m∈maxs∈‖πkm,s−πkm‖1 _(\!n\!)\!\!=\!\! _k∈\![nT\!\!-\!\!1,(\!n\!+\!1\!)T\!-\!1]\!\! _m \! _s \!\!\| _k^m,s\!\!-\!\! _k^m\|_1, we have Dk≤DnT+2TLπδ(n)D_k≤ D_nT+2TL_π _(n). Moreover, DnT≤∑m∈C′‖πnT−1m,s−πnT−1m‖1D_nT\!≤\! _m C \| _nT-1^m,s\!-\! _nT-1^m\|_1. Thus we have Dk≤Cδ(n)D_k≤ C _(n), where C=C′+2TLπC\!=\!C \!+\!2TL_π. From (41), we have ‖a-ν(n),km−ν(n),km‖1≤‖a-ν(n),k−ν(n),k‖1=Dk≤Cδ(n). \|a- _(n),k^m\!\!-\!\! _(n),k^m\|_1\!\!≤\!\!\|a- _(n),k\!\!-\!\! _(n),k\|_1\!\!=\!\!\!D_k\!\!≤\!\!C _(n). (42) This completes the proof of Lemma 1. ∎ Finally, we can get ‖a-e(n+1)m‖1≤∑k=nT(n+1)T−1βkβ(n)Cδ(n)=Cδ(n)\|a-e_(n\!+\!1)^m\|_1\!≤\! _k=nT^(n+1)T-1\! _k _(n)C _(n)\!\!=C _(n). Furthermore, δ(n)→0 _(n)→ 0 as n→∞n→∞ by Theorem 1. Now we introduce a reference scenario to facilitate the analysis. Let π^tm π_t^m denote the belief for team m at round t in the reference scenario. In the reference scenario, π^tm π_t^m is only updated at the end of each epoch. In other words, for all nT≤t≤(n+1)T−1nT\!≤\!t\!≤\!(n\!+\!1)T\!-\!1 and m∈m\!∈\!T, we have π^tm=π(n)m π_t^m\!=\! _(n)^m. Since the beliefs are fixed, DTOA dynamics reduce to log-linear learning in the reference scenario. Let a^(n),km a^m_(n),k denote the joint action of team m at round k under the fixed beliefs in the reference scenario. Due to the nature of log-linear learning, a^(n),kmk=nT∞\ a_(n),k^m\_k=nT^∞ forms a homogeneous Markov chain (MC). In contrast, the actual action profiles a¯kmk=nT∞\ a_k^m\_k=nT^∞ do not. We define the stationary distributions of the MC in the reference scenario as νˇ(n),∗m ν_(n),*^m and ν^(n),∗m ν_(n),*^m for DTOA and iDTOA, respectively. By [4, 25], it follows that νˇ(n),∗m=brτ(ϕm(⋅,π(n)−m)) ν_(n),*^m\!\!=\!br_\!τ\!\! (\!φ^m(·, _(n)^-\!m)\! ). Therefore, we write the true belief update (39) as π(n+)m=(−β(n))π(n)m+β(n)[brτ(ϕm(⋅,π(n)−m))+ω(n+)m+e(n)m+a-e(n+)m]. _\!(\!n\!+\!1\!)^\!m\!\!=\!\! (\!1\!\!-\!\! _(\!n\!)\! )\! _\!(\!n\!)^\!m\!\!+\!\! _(\!n\!)\!\! [\!br_\!τ\!\! (\!\!φ^m\!(\!·,\! _\!(\!n\!)^\!\!-\!m\!)\!\! )\!\!+\!\! _\!(\!n\!+\!1\!)^\!m\!\!\!+\!\!e_\!(\!n\!)^\!m\!\!\!+\!\!a-e_\!(\!n\!+\!1\!)^\!m\! ]\!\!. (43) The error for DTOA is team-e(n)m -e_(n)^m =∑k=nT(n+1)T−1βkβ(n)(team-ν(n),km−νˇ(n),∗m). = _k=nT^(n+1)T-1\!\! _k _(n) (team- _(n),k^m\!-\! ν_(n),*^m ). (44) The error for iDTOA is indp-e(n)m≜e^(n)m+eˇ(n)mindp-e_(n)^m\!\! \!\! e^m_(n)\!\!+\!\! e^m_(n), where eˇ(n)m≜ν^(n),∗m−νˇ(n),∗m e^m_(n)\!\! \!\! ν_(n),*^m\!\!-\!\! ν_(n),*^m, and e^(n)m=∑k=nT(n+1)T−1βkβ(n)(indp-ν(n),km−ν^(n),∗m) e^m_(n)\!\!=\!\! _k=nT^(n\!+\!1)T\!-\!1\! _k _(n)\! (\!indp- _(n),k^m\!\!-\!\! ν_(n),*^m\! ). Lemma 4. There exist constants c,d,ρ≥0c,d,ρ≥ 0 such that ‖team-ν(n),km−νˇ(n),∗m‖1≤cρk−nT+dTαnT,∀m∈, \|team- _(n),k^m\!-\! ν_(n),*^m\|_1\!≤\!cρ^k-nT\!+\!dT _nT,\ ∀ m , ‖indp-ν(n),km−ν^(n),∗m‖1≤cρk−nT+dTαnT,∀m∈. \|indp- _(n),k^m\!-\! ν_(n),*^m\|_1\!≤\!cρ^k-nT\!+\!dT _nT,\ ∀ m . The proof of Lemma 4 can be found in Appendix B.1 of [10]. Let team-e(n)mteam-e_(n)^m and indp-e(n)mindp-e_(n)^m denote the errors for DTOA and iDTOA, respectively. Lemma 4 yields that team-e(n)mteam-e_(n)^m and e^(n)m e_(n)^m can be bounded by ‖team-e(n)m‖1≤c∑k=nT(n+1)T−1βkβ(n)ρk−nT+dRαnT, \|team-e_(n)^m\|_1≤ c _k=nT^(n+1)T-1 _k _(n)ρ^k-nT+dR _nT, (45) ‖e^(n)m‖1≤c∑k=nT(n+1)T−1βkβ(n)ρk−nT+dRαnT. \| e_(n)^m\|_1≤ c _k=nT^(n+1)T-1 _k _(n)ρ^k-nT+dR _nT. (46) With Assumption 3 and (36), we have βk+1βk≤1 _k\!+\!1 _k\!\!≤\!\!1. Thus, βkβ(n)≤βnTβ(n+1)T−1≤αnTα(n+1)T. _k _(n)≤ _nTT _(n+1)T-1≤ _nTT _(n+1)T. (47) By Assumption 3, we obtain limn→∞αnTα(n+1)T=1Tlimn→∞∏k=nT(n+1)T−1αkαk+1=1T. _n→∞ _nTT _(n+1)T= 1T _n→∞ _k=nT^(n+1)T-1 _k _k+1= 1T. (48) Using (45), (46), (47), (48), and the decay property of αk _k, we obtain, for all m∈m , lim supn→∞‖team-e(n)m‖1≤cT1−ρ and lim supn→∞‖e^(n)m‖1≤cT1−ρ. _n→∞\! \|team-e_(\!n\!)^m\! \|_1\!\!≤\!\! cT 11\!\!-\!\!ρ and _n→∞\! \| e_(\!n\!)^m\! \|_1\!\!≤\!\! cT 11\!\!-\!\!ρ. Let ν ν and νˇ ν denote the stationary distributions induced by the classical and independent log-linear learning, respectively. A small δ>0δ>0 implies close stationary distributions in the classical and independent settings: ‖ν^−νˇ‖1≤Λ(δ,εϕ), \| ν- ν\|_1≤ (δ, _φ), (49) for some function Λ , and the difference ‖ν^−νˇ‖1\| ν- ν\|_1 decays to zero as δ→0+δ→ 0^+ for any εϕ>0 _φ>0. Based on (49), we can bound eˇ(n)m e_(n)^m as eˇ(n)m≤Λ(δ,ε) e_(n)^m≤ (δ, ) for some ε>0 >0. Hence, given ε>0 >0, there exists Nε∈ℕ+N_ _+ such that for any n>Nεn>N_ , ‖team-e(n)m‖1<Cteam(ε,T), \|team-e_(n)^m\|_1<C_team( ,T), (50) ∥indp-e(n)m∥1<Cindp(ε,T),∥a-e(n+1)m∥1≤ε, \|indp-e_(n)^m\|_1<C_indp( ,T),\|a-e_(n+1)^m\|_1≤ , where Cteam(ε,T)=ε+cT11−ρC_team( ,T)\!\!=\!\! \!+\! cT\! 11-ρ and Cindp(ε,T)=ε+cT11−ρ+Λ(δ,ε)C_indp( ,T)\!\!=\!\! \!+\! cT\! 11-ρ\!+\! (δ, ). Note that Cteam(ε,T)→0C_team( ,T)\!→\!0 as ε→0,T→∞ \!→\!0,\ T\!→\!∞, and Cindp(ε,T)→Λ(δ,ε)C_indp( ,T)\!→\! (δ, ) as ε→0,T→∞ \!→\!0,\ T\!→\!∞. We then analyze the convergence of the TNG. To this end, we define the set-valued mapping F(π) F(π) ≜(brτ(ϕm(⋅,π−m))−πm+em)m∈|∀m∈: \\!\! (br_τ (φ^m(·,π^-m) )\!-\!π^m\!+\!e^m )_m |\ ∀ m : ∥em∥1≤C(ε,T),brτ(ϕm(⋅,π−m))+em∈Δ(¯m) \|e^m\|_1\!≤\!C( ,T),\ br_τ (φ^m(·,π^-m) )\!+\!e^m\!∈\! \! ( A^m )\!\! \ for all π∈∏m∈Δ(¯m)≜Ππ∈ _m ( A^m ) , where C(ε,T)=Cteam(ε,T)+εfor DTOA,Cindp(ε,T)+εfor iDTOA. C( ,T)= casesC_team( ,T)+ DTOA,\\ C_indp( ,T)+ iDTOA. cases Then, (43) and (50) imply that, for sufficiently large n, π(n+1)−π(n)−β(n)⋅ω(n+1)∈β(n)⋅F(π(n)). _(n+1)\!-\! _(n)\!-\! _(n)\!· _(n+1)∈ _(n)\!· F ( _(n) ). The following argument is similar to the proof in Appendix A.2 of [10]. Finally, we get lim supk→∞TNG(πks)≤τlog||,for DTOA,τlog||+||2ϕ¯Λ(δ,εϕ),for iDTOA, _k→∞\!T\!N\!G( _k^s)\!\!≤\!\! cases\!τ\! \!|A|,\!\!\!\!&for DTOA,\\ \!τ\! \!|A|\!+\!|T|^2 φ (δ, _φ),\!\!\!\!&for iDTOA, cases This completes the proof of Theorem 2. -C Proof of Lemma 2 and Theorem 4 Proof of Lemma 2. Given a team m∈m and its supervisor s, define Xkm,s=vkm,s=1,zkm,s=failX_k^m,s\!\!=\!\!1\\!v_k^m,s\!\!=\!\!1,z_k^m,s\!\!=\!fail\. We have ℙ(Xkm,s=1)=q⋅ηFP,m∈ℋ,q⋅[(−plie)ηFP+plie(−ηFN)],m∈ℬ. (X_k^m,s\!=\!1 )\!=\! casesq\!·\! _FP,\!\!\!\!&m\!∈\!H,\\ q\!·\! [(1\!-\!p_lie) _FP\!+\!p_lie(1\!-\! _FN) ]\!,\!\!\!\!&m\!∈\!B. cases For any t>0t\!>\!0, Ftm,s=∑k=1tXkm,sF_t^m,s\!\!= _k=1^tX_k^m,s is the number of failfail signals up to round t. Using the KL-form Chernoff bound (see, e.g., [32], Chapter 2) and (7), we get ℙ(Ftm,st>f)≤exp(−tD(f∥qηFP)),m∈ℋ, and \!\! (\!\! F_t^m,st\!\!>\!\!f\!\! )\!\!≤\!\! \! (\!\!\!-\!\!tD(f\|q _FP)\! ),m , and ℙ(Ftm,st<f)≤exp(−tD(f∥q[(−plie)ηFP+plie(−ηFN)])),m∈ℬ, \!\! (\!\! F_t^m,st\!\!<\!\!f\!\! )\!\!\!≤\!\! \! (\!\!\!\!-\!\!tD\! (f\|q[(\!1\!\!-\!\!p_lie\!)\! _F\!P\!\!+\!\!p_lie\!(\!1\!\!-\!\! _F\!N\!)]\! )\!\! )\!,m\!\!∈\!\!B, where D(x∥y)≜xlogxy+(1−x)log1−x1−yD(x\|y) x xy+(1-x) 1-x1-y. This completes the proof of Lemma 2. Proof of Theorem 4. For m∈ℋm , ℙ(∃t≥t0:Ftm,st>f) (∃ t≥ t_0: F_t^m,st>f ) ≤∑t=t0∞exp(−tD(f∥qηFP)) ≤ _t=t_0^∞ (-tD(f\|q _FP) ) =exp(−t0D(f∥qηFP))1−exp(−D(f∥qηFP)). = (-t_0D(f\|q _FP) )1- (-D(f\|q _FP) ). We define the following events: ℰ0(t0)=⋂m∈ℋ∀t≥t0:Ftm,st≤f, _0(t_0)= _m \∀ t≥ t_0: F_t^m,st≤ f \, ℰ1(t1)=⋂m∈ℬ∃t≤t1:Ftm,st>f. _1(t_1)= _m \∃ t≤ t_1: F_t^m,st>f \. Then we obtain ℙ(ℰ0(t0))≥1−|ℋ|⋅exp(−t0D(f∥qηFP))1−exp(−D(f∥qηFP))=1−δt0, (E_0(t_0) )≥ 1-|H|· (-t_0D(f\|q _FP) )1- (-D(f\|q _FP) )=1- _t_0, ℙ(ℰ1(t1))≥1−|ℬ|⋅exp(−t1D(f∥q[(1−plie)ηFP (E_1(t_1) )≥ 1-|B|· (-t_1D (f\|q [(1-p_lie) _FP +plie(1−ηFN)]))=1−δt1. +p_lie(1- _FN) ] ) )=1- _t_1. Define the good event as ℰ(t0,t1)=ℰ0(t0)∩ℰ1(t1)E(t_0,t_1)=E_0(t_0) _1(t_1). Then we have ℙ(ℰ(t0,t1))≥1−δt0−δt1=1−δB.P(E(t_0,t_1))≥ 1- _t_0- _t_1=1- _B. On the good event ℰE, for k>t1k\!>\!t_1, the beliefs of Byzantine teams are frozen, while those of honest teams are not. Following the proof of Theorem 2, we can conclude lim supk→∞TNGℋ(πks) _k→∞TNG_H ( _k^s ) ≤ ≤ τlog|ℋ|,for BR-DTOA,τlog|ℋ|+|ℋ|2ϕ¯Λ(δ,ϵ),for BR-iDTOA, casesτ |A_H |,&for BR-DTOA,\\ τ |A_H |+ |H |^2 φ (δ,ε),&for BR-iDTOA, cases This completes the proof of Theorem 4. Juntian Zhu is a Ph.D. candidate with the School of Artificial Intelligence and Data Science, University of Science and Technology of China. She received the B.Sc. degree in mathematics from Sichuan University, Chengdu, China, in 2022. Her research interests are in (1) learning theory in uncertain AI environments; (2) multi-agent learning, decision-making and game-theoretic analysis under limited information; and (3) uncertainty-aware reasoning in large language models. Guanpu Chen received his B.Sc. degree from University of Science and Technology of China, Hefei, China, in 2017, and Ph.D. degree from Academy of Mathematics and Systems Science, Chinese Academy of Sciences (CAS), Beijing, China, in 2022. He is currently a Professor with the School of Automation, Southeast University, Nanjing, China. He used to be a postdoctoral researcher with the School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden. His research interests include multi-agent systems, network games, as well as robustness, resilience, and security in cyber-physical systems. Prof. Chen was the recipient of the President Award of CAS, the Best Paper Award at IEEE International Conference on Control and Automation (ICCA), Guan Zhao-Zhi Award at Chinese Control Conference (C), and the Best Student Paper Honorable Mention at IEEE CSS TCSP. Tongtian Zhu is a Ph.D. candidate with the College of Computer Science and Technology, Zhejiang University. His research interests are in (1) understanding modern deep learning systems from an optimization-theoretic perspective; and (2) the theoretical foundations and algorithms for scalable and autonomous decentralized, distributed, and multi-agent learning. He has made pioneering contributions to the study of implicit sharpness bias and data influence in decentralized learning, with related findings published at top-tier machine learning conferences, including ICML, NeurIPS, and ICLR, as oral and spotlight presentations. Miguel de Carvalho is the Professor and Chair of Statistical Data Science, a Fellow of Generative AI Laboratory, and the Co-Director of the Edinburgh Centre for Financial Innovations at the University of Edinburgh. He is also an Honorary Professor at Universidade de Aveiro. He received a BSc in Mathematics from the NOVA School of Science and Technology in 2004, an MSc in Economics from the NOVA School of Business and Economics in 2009, and a PhD in Mathematics with emphasis on Statistics from the NOVA School of Science and Technology in 2009. His research is in Extreme Value Theory, Interfaces between Statistics and AI, and Bayesian Analysis. He is the Editor-in-Chief of the Springer book series on Courses in Advanced Statistics and Data Science, and an Associate Editor of American Statistician, the Annals of Applied Statistics, Bayesian Analysis, Computational Statistics & Data Analysis, Extremes, the Journal of the American Statistical Association, and Statistics and Public Policy. Zhouwang Yang is a Professor in the School of Mathematical Sciences at University of Science and Technology of China (USTC). He received his Bachelor degree, Master degree and PhD degree in Mathematics from USTC in 1997, 2000 and 2005, respectively. He worked at Seoul National University as a postdoctoral researcher from December 2006 to November 2007. He was a visiting scholar in School of Industrial and Systems Engineering (ISyE) at Georgia Institute of Technology during August 2010 - August 2011. He has been working on geometric modeling and processing; optimization methods and their applications in sparse recovery. He also has a particular interest in the integration of optimization, machine learning and statistics for solving big data problems. Fengxiang He is a Lecturer at the University of Edinburgh, and a Fellow of its Generative AI Laboratory. He received a BSc in statistics from the University of Science and Technology of China, an MPhil and a PhD in computer science from the University of Sydney in 2017, 2019, and 2021, respectively. His research interests are in (1) understanding AI from both learning-theoretical and game-theoretical views; (2) new models and algorithms leveraging symmetries in data, task, environment; (3) collaboration and interactions between AI agents in decentralized and multi-agent settings; and (4) applications in economics and finance. He is an Area Chair of ICML, NeurIPS, ICLR, UAI, and AISTATS, an Editorial Board member of Machine Learning, and an Associate Editor of Pattern Recognition.