Paper deep dive
Joint Communication-Control Strategy Optimization with Partially Nested Information Structures: The Linear-Quadratic Case
Haoyi You, Kaiqing Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/14/2026, 3:56:28 AM
Summary
This paper formalizes the Joint Communication-Control Strategy Optimization (JCCO) problem for multi-agent linear systems with quadratic costs under the Common-Information-Based (CIB) framework. It focuses on Partially Nested (PN) Information Structures to ensure computational tractability. The authors derive conditions for preserving partial nestedness under additional communication strategies and develop a dynamic-programming-based approach using closed-form Riccati Equations to compute optimal control strategies for both open-loop and closed-loop communication strategies.
Entities (8)
Relation Signals (6)
Joint Communication-Control Strategy Optimization → uses → Common-Information-Based Framework
confidence 95% · formalize a joint communication-control strategy optimization (JCCO) problem ... under the common-information-based (CIB) framework
Joint Communication-Control Strategy Optimization → constrains → Partially Nested Information Structure
confidence 92% · focus on such JCCO problems with partially nested (PN) information structures (ISs)
Partially Nested Information Structure → ensures → Linearity of Optimal Strategies
confidence 90% · partial nestedness is known to be a favorable IS that not only ensures the linearity of an optimal control strategy
Dynamic programming → yields → Riccati Equations
confidence 90% · develop a dynamic-programming-based approach ... which yields a set of closed-form Riccati Equations
Open-Loop Communication Strategies → enables → Linear Optimal Control Strategies
confidence 88% · partial nestedness is preserved ... ensures the linearity of an optimal control strategy
Joint Communication-Control Strategy Optimization → extendsto → Closed-Loop Communication Strategies
confidence 85% · extend such an approach to JCCOs with closed-loop communication strategies
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs, under the common-information-based (CIB) framework from decentralized stochastic control. For computational tractability, we focus on such JCCO problems with partially nested (PN) information structures (ISs). In particular, with a baseline communication protocol that leads to a PN IS, we establish a series of conditions under which the partial nestedness is preserved under the (additional) communication strategies to be optimized, while violating them may cause nonlinearity of the optimal strategies in general, with open-loop communication strategies. We then develop a dynamic-programming-based approach to compute the optimal control strategies of JCCO with open-loop communication strategies, which yields a set of closed-form Riccati Equations. As a byproduct of independent interest, such an approach also offers a way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework. Finally, we extend such an approach to JCCOs with closed-loop communication strategies, yielding a more tractable dynamic program than an infinite-dimensional CIB-belief-based one.
Tags
Links
- Source: https://arxiv.org/abs/2608.13535v1
- Canonical: https://arxiv.org/abs/2608.13535v1
Trouble viewing inline? Open PDF directly →
Full Text
246,758 characters extracted from source content.
Expand or collapse full text
Joint Communication-Control Strategy Optimization with Partially Nested Information Structures: The Linear-Quadratic Case Haoyi YouKaiqing Zhang Thanks: The authors are affiliated with the University of Maryland, College Park, MD, USA, 20742. Emails: yuriiyou,˜kaiqing@umd.edu. Abstract In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs, under the common-information-based (CIB) framework from decentralized stochastic control. For computational tractability, we focus on such JCCO problems with partially nested (PN) information structures (ISs). In particular, with a baseline communication protocol that leads to a PN IS, we establish a series of conditions under which the partial nestedness is preserved under the (additional) communication strategies to be optimized, while violating them may cause nonlinearity of the optimal strategies in general, with open-loop communication strategies. We then develop a dynamic-programming-based approach to compute the optimal control strategies of JCCO with open-loop communication strategies, which yields a set of closed-form Riccati Equations. As a byproduct of independent interest, such an approach also offers a way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework. Finally, we extend such an approach to JCCOs with closed-loop communication strategies, yielding a more tractable dynamic program than an infinite-dimensional CIB-belief-based one. I. Introduction The design of communication protocols for control has been extensively studied in both the control theory and multi-agent learning literature, from different perspectives and under different models. For example, [29, 21] investigated the design of communication channels with capacity constraints in networked control systems; [17, 16] studied communication architecture and control strategy co-design via norm minimization; [28, 6, 15] investigated the joint optimization of the quantization and control policies, with the quantized signals being communicated to the controller; and [5, 24] developed heuristic learning-to-communicate algorithms to jointly learn the control and communication strategies for accumulated reward maximization. In this paper, we formalize a joint communication-control strategy optimization (JCCO) problem in multi-agent linear systems with quadratic costs. Motivated by the empirical studies [5, 24], we model communication as some actions/strategies to be optimized jointly with the control strategies, under an optimal control formulation to minimize some accumulated costs. The agents may follow some fixed baseline communication protocols for information sharing, and then share their private information through additional sharing, following their communication strategies. We formalize the problem under the common-information-based (CIB) framework from decentralized stochastic control [20, 7], which has also been adopted in several recent studies on control-communication strategy co-optimization [23, 9, 11]. In comparison, [23, 9] focused specifically on sharing instantaneous observations and on systems with decoupled state dynamics. Our formalism in [11] is more general and allows the sharing of private-information histories in discrete-space problems, together with computational and sample complexity analyses. We here focus on an instantiation of our formalism in [11] in the continuous-space, linear-quadratic (LQ) setting, which yields fundamentally different technical challenges as detailed below. To facilitate tractable computation of the optimal strategies, we focus on deriving dynamic-programming (DP)-based approaches to solve JCCO. To this end, we concentrate on such co-optimization problems with partially nested (PN) information structures (ISs) [8]. Indeed, with a fixed open-loop communication strategy (and thus a fixed IS), partial nestedness is known to be a favorable IS that not only ensures the linearity of an optimal control strategy, but also yields a convex reformulation of the decentralized LQ control problem [8]. However, as pointed out in [10], the reformulated convex program can be too large to be computationally efficient, which precisely motivated the DP-based solution therein. Moreover, the DP-based approach can provide more insights into the structure of the optimal controller [10], by identifying the proper sufficient statistics for decision-making [10, 20, 14, 19]. Yet, the results in [10, 19] do not apply to our setting, as they focused on the special cases with factorized states, and with either state-feedback [10] or multi-tree coupling and communication graphs [19]. Under the CIB framework [20], [14] considered the output-feedback setting, and showed that when control strategies are restricted to be linear, and the private-information component of the strategy is fixed, the resulting problem is linear-quadratic-Gaussian (LQG) control, which can be solved via Riccati Equations. However, it remains unclear how to solve the overall decentralized LQ control problem by further optimizing over the private-information component. Since given a fixed communication strategy, our JCCO problem reduces to a decentralized LQ control one, we need to advance these results in order to fully address JCCO. We thus make the following contributions: Contributions. (i) We formalize joint communication-control strategy optimization as an optimal control problem with linear systems and quadratic costs, under the common-information-based framework [20]. (i) For better computational tractability, we focus on JCCO with partially nested baseline ISs [8], and identify structural conditions under which partial nestedness is preserved under additional information sharing, while violating them may cause nonlinearity or non-existence of the optimal control strategies under open-loop communication strategies. (i) We then develop a closed-form dynamic program, i.e., a set of Riccati Equations, to solve for the optimal control strategy with respect to fixed open-loop communication strategies in JCCO. This program is also of independent interest: it may be viewed as a new way to solve decentralized linear-quadratic control with PN ISs and output feedback, under the CIB framework, which thus advances the results in [14]. The key is to identify a novel connection between the (strictly) PN IS [8] and the strategy-independent CIB belief (SI-CIB) condition [7]. (iv) We then extend such an approach to JCCO with closed-loop communication strategies, yielding reduced-dimensional sufficient statistics and a more tractable dynamic program than an infinite-dimensional belief-based one when applying the CIB framework to general LQ settings directly. I. Preliminaries Notation. Random variables are denoted by bold upper case letters, and their realizations are denoted by the corresponding non-bold upper case letters. For any matrix X, we use X†X to denote the pseudo-inverse of X if X⪰0X 0. For any two integers 0≤a<b0≤ a<b, we denote [a:b]:=a,a+1,⋯,b[a:b]:=\a,a+1,·s,b\, and denote [a]=[1:a][a]=[1:a]. For any vector space X, we denote by ()P(X) the space of all probability distributions over X. I-A Problem formulation For a team of n>1n>1 agents, a joint communication-control strategy optimization problem with output feedback can be described by the following tuple: =⟨H,,ii∈[n],ii∈[n],ℳi,hi∈[n],h∈[H],Ahh∈[H],Bi,hi∈[n],h∈[H],Ei,hi∈[n],h∈[H],Qh1h∈[H+1],Qh2h∈[H],hh∈[H]⟩D= H,X,\Y_i\_i∈[n],\U_i\_i∈[n],\M_i,h\_i∈[n],h∈[H],\A_h\_h∈[H],\B_i,h\_i∈[n],h∈[H],\E_i,h\_i∈[n],h∈[H],\Q_h^1\_h∈[H+1],\Q_h^2\_h∈[H], \K_h\_h∈[H] , where H is the time horizon and h∈=ℝdx X_h =R^d_x is the state. At each timestep h∈[H]h∈[H], each agent i∈[n]i∈[n] receives a noisy observation i,h∈i=ℝdyi Y_i,h _i=R^d_y^i of the state h X_h, and chooses a control action i,h∈i=ℝdui U_i,h _i=R^d_u^i. At timestep h∈[H]h∈[H], we denote by h=[1,h⊤2,h⊤⋯n,h⊤]⊤ U_h= bmatrix U_1,h & U_2,h &·s& U_n,h bmatrix the joint control action of all the n agents, and by =ℝ∑i=1nduiU=R _i=1^nd_u^i the joint control action space; we denote by h=[1,h⊤2,h⊤⋯n,h⊤]⊤ Y_h= bmatrix Y_1,h & Y_2,h &·s& Y_n,h bmatrix the joint observation, and by =ℝ∑i=1ndyiY=R _i=1^nd_y^i the joint observation space. The system evolves as follows for each timestep h∈[H]h∈[H]: h+1=Ahh+∑i=1nBi,hi,h+0,h,i,h=Ei,hh+i,h,∀i∈[n], X_h+1=A_h X_h+ _i=1^nB_i,h U_i,h+ W_0,h, Y_i,h=E_i,h X_h+ W_i,h,∀ i∈[n], (I.1) where for each i∈[0:n],h∈[H]i∈[0:n],h∈[H], i,h W_i,h is a Gaussian random variable with distribution (,�i,h)N( 0, _i,h) for some covariance matrix �i,h⪰0 _i,h 0, while Ah,Ei,hA_h,E_i,h are matrices of appropriate dimensions. We define Bh:=[B1,hB2,h⋯Bn,h]B_h:= bmatrixB_1,h&B_2,h&·s&B_n,h bmatrix and Eh:=[E1,h⊤E2,h⊤⋯En,h⊤]⊤E_h:= bmatrixE_1,h &E_2,h &·s&E_n,h bmatrix . Here, we assume that the initial state 1 X_1 follows a Gaussian distribution (,�1)N( 0, _1) with covariance matrix �1⪰0 _1 0, and that 1 X_1 and i,hi∈[0:n],h∈[H]\ W_i,h\_i∈[0:n],h∈[H] are mutually independent. At timestep h∈[H]h∈[H], each agent will share part of her information with other agents. The shared information h:=hb∪ha Z_h:= Z_h^b∪ Z_h^a consists of two parts, the baseline-sharing part hb Z_h^b, which originates from some existing sharing protocol, and the additional-sharing part ha Z_h^a, which is decided/learned by agents, with joint additional-sharing information ha:=∪i=1ni,ha Z_h^a:= _i=1^n Z_i,h^a. The baseline-sharing part is introduced for generality (i.e., hb Z_h^b may be set as ∅ ), and some benign information structures to be introduced later may require a certain amount of baseline sharing; see [12, 11] for concrete examples. At timestep h, the common information among all the agents is thus defined as the union of all the shared information so far: h−=∪t=1h−1t∪hb,h+=∪t=1ht C_h^-= _t=1^h-1 Z_t∪ Z_h^b, C_h^+= _t=1^h Z_t, where h− C_h^- and h+ C_h^+ denote the common information before and after additional sharing, respectively. The private information of agent i at timestep h before and after additional sharing is denoted by i,h−,i,h+ P_i,h^-, P_i,h^+, respectively, where i,h−⊆1:h,1:h−1 −,i,h+⊆1:h,1:h−1 + P_i,h^- \ Y_1:h, U_1:h-1\ C_h^-, P_i,h^+ \ Y_1:h, U_1:h-1\ C_h^+. Then, we define i,h−:=i,h−∪h−,i,h+:=i,h+∪h+ I_i,h^-:= P_i,h^-∪ C_h^-, I_i,h^+:= P_i,h^+∪ C_h^+ as the information available to agent i before and after additional sharing, respectively. We denote by h−:=[1,h−⊤⋯n,h−⊤]⊤ P_h^-:=[ P_1,h^- ~·s~ P_n,h^- ] the joint private information at timestep h before additional sharing, and similarly by h+ P_h^+ the joint private information after additional sharing. We denote by h−,h+,i,h−,i,h+,h−,h+,ℐi,h−,ℐi,h+C_h^-,C_h^+,P_i,h^-,P_i,h^+,P_h^-,P_h^+,I_i,h^-,I_i,h^+ the sets of all possible values of the respective random variables h−,h+,i,h−,i,h+,h−,h+,i,h−,i,h+ C_h^-, C_h^+, P_i,h^-, P_i,h^+, P_h^-, P_h^+, I_i,h^-, I_i,h^+. At each timestep h, in addition to the control action, each agent i needs to choose a communication action i,h∈ℳi,h M_i,h _i,h to determine what information i,ha Z_i,h^a she will share, as specified below. We use h:=(1,h,⋯,n,h)∈ℳh M_h:=( M_1,h,·s, M_n,h) _h to denote the joint communication action at timestep h. We assume that the evolution of common and private information in D follows the rules below, where we adopt the convention that any quantity at timestep 00 is empty/null. Assumption I.1 (Information Evolution). (a) (Baseline sharing) The baseline sharing evolves as hb=χh((h−1)+,h−1,h) Z_h^b= _h( P_(h-1)^+, U_h-1, Y_h) for some fixed projection function χh _h. (b) (Additional sharing) For each i∈[n]i∈[n], the additional sharing evolves as i,ha=ϕi,h(i,h,i,h−) Z_i,h^a= _i,h( M_i,h, P_i,h^-) for some function ϕi,h _i,h, where for each realization Mi,h∈ℳi,hM_i,h _i,h, ϕi,h(Mi,h,⋅) _i,h(M_i,h,·) is a fixed projection function. The joint additional sharing ha Z_h^a is thus generated by ha=ϕh(h,h−) Z_h^a= _h( M_h, P_h^-) for some function ϕh _h. (c) For each i∈[n],i,h−=ζi,h(i,(h−1)+,i,h−1,i,h)i∈[n], P_i,h^-= _i,h( P_i,(h-1)^+, U_i,h-1, Y_i,h) for some fixed projection function ζi,h _i,h, and the joint private information thus evolves as h−=ζh((h−1)+,h−1,h) P_h^-= _h( P_(h-1)^+, U_h-1, Y_h) for some projection function ζh _h. (d) For each i∈[n],i,h+=i,h− ,hai∈[n], P_i,h^+= P_i,h^- Z_i,h^a. (e) For each i∈[n]i∈[n] and h∈[H]h∈[H], i,h∈i,h− Y_i,h∈ I_i,h^- and i,h−⊆i,h+ I_i,h^- I_i,h^+; for h∈[H−1]h∈[H-1], i,h+⊆i,(h+1)− I_i,h^+ I_i,(h+1)^-. Note that (a) and (c) on baseline sharing follow from those in [7, 12, 11]. (b) and (d) on additional sharing dictate how the communication action affects the additional sharing. For example, a common choice of (ℳi,h,ϕi,h)(M_i,h, _i,h) is that ℳi,h=0,1maxPi,h−∈i,h−|Pi,h−|M_i,h=\0,1\ _P_i,h^- _i,h^-|P_i,h^-|, where |Pi,h−||P_i,h^-| denotes the number of elements in Pi,h−P_i,h^-, and for any Pi,h−∈i,h−,Mi,h∈ℳi,hP_i,h^- _i,h^-,M_i,h _i,h, ϕi,h(i,h−=Pi,h−,i,h=Mi,h) _i,h( P_i,h^-=P_i,h^-, M_i,h=M_i,h) consists of the k-th element (k∈[|Pi,h−|])(k∈[|P_i,h^-|]) of Pi,h−P_i,h^- if and only if the k-th element of Mi,hM_i,h is 1. Condition (e) means that each agent has full memory of the available information, and has a closed-loop (baseline) information structure with i,h∈i,h− Y_i,h∈ I_i,h^-, which are standard assumptions also made in [11, 27]. I-B Objective and strategies At each timestep h∈[H]h∈[H], agents will incur two types of costs, communication cost κh _h and control cost chc_h. The control cost inherits the cost of a decentralized LQG problem without communication, see e.g., [14, 10], and is defined as ch=h⊤Qh1h+h⊤Qh2hc_h= X_h Q_h^1 X_h+ U_h Q_h^2 U_h, with a terminal cost of cH+1=H+1⊤QH+11H+1c_H+1= X_H+1 Q_H+1^1 X_H+1. Here, for every h∈[H]h∈[H], Qh1,Qh2⪰0Q_h^1,Q_h^2 0 and QH+11⪰0Q_H+1^1 0 are symmetric matrices of appropriate dimensions. The communication cost is determined by the additional sharing and is defined as κh=h(h) _h=K_h( M_h) for some function h:ℳh→ℝ+K_h:M_h _+, depending on the communication action h M_h chosen from a finite set ℳhM_h. At each timestep h, each agent i needs to choose a communication action i,h M_i,h and a control action i,h U_i,h, based on her communication strategy gi,hmg_i,h^m and control strategy gi,hag_i,h^a, respectively. The class of open-loop communication strategies is defined as i,hm:=gi,hm:∗→ℳi,hG_i,h^m:=\g_i,h^m:\ \ _i,h\. We identify each such strategy with its single value gi,hm(∗)g_i,h^m( ), so i,h M_i,h is pre-selected. In this case, we consider the control strategies gi,ha∈i,ha:=gi,ha:ℐi,h+→ig_i,h^a _i,h^a:=\g_i,h^a:I_i,h^+ _i\. Also, we may extend the setting to the class of closed-loop communication strategies, i,hm:=gi,hm:ℐi,h−×ℳ1:h−1→ℳi,hG_i,h^m:=\g_i,h^m:I_i,h^-×M_1:h-1 _i,h\ where i,h M_i,h is thus chosen based on i,h− I_i,h^- and 1:h−1 M_1:h-1. The communication actions 1:h M_1:h are publicly observed. In this case, we consider the control strategies gi,ha∈i,ha:=gi,ha:ℐi,h+×ℳ1:h→ig_i,h^a _i,h^a:=\g_i,h^a:I_i,h^+×M_1:h _i\ for generality. We will focus on the open-loop communication strategies throughout and extend our approach to the closed-loop ones in §V. We use ghm:=(g1,hm,⋯,gn,hm)g_h^m:=(g_1,h^m,·s,g_n,h^m) and gha:=(g1,ha,⋯,gn,ha)g_h^a:=(g_1,h^a,·s,g_n,h^a) to denote the joint communication and control strategies, respectively. We denote by ha,hmG_h^a,G_h^m the spaces of these joint strategies. The total cost (objective) of all the agents is defined as J(g1:Hm,g1:Ha):=[∑h=1H(ch+κh)+cH+1|g1:Hm,g1:Ha],J_D(g_1:H^m,g_1:H^a):=E [ _h=1^H(c_h+ _h)+c_H+1\, |\,g_1:H^m,g_1:H^a ], (I.2) which is the expected accumulated sum of the control and communication costs over H timesteps. With this objective, we define the solution concept of team optimality below. Definition I.2. Given a JCCO problem D, a strategy (g1:Hm,∗,g1:Ha,∗)(g_1:H^m, ,g_1:H^a, ) is team-optimal if ∀g1:Hm∈1:Hm,g1:Ha∈1:Ha∀ g_1:H^m _1:H^m,g_1:H^a _1:H^a, J(g1:Hm,g1:Ha)≥J(g1:Hm,∗,g1:Ha,∗),J_D(g_1:H^m,g_1:H^a)≥ J_D(g_1:H^m, ,g_1:H^a, ), with g1:Hm,∗,g1:Ha,∗g_1:H^m, ,g_1:H^a, being referred to as the optimal communication and control strategies, respectively. Note that the team-optimal strategies depend on the (communication) strategy space being used (i.e., open-loop versus closed-loop), which will be clear later from the context. I-C Information structures Information structure is a well-studied notion in decentralized stochastic control that captures who knows what and when [26, 13]. For any JCCO problem D, as the additional sharing via communication will also affect the IS and is not determined beforehand, when we discuss the IS of D, we will refer to that of the problem without additional sharing, which is essentially a decentralized LQG control problem (with potential baseline information sharing), denoted by ˇ D. We refer to ˇ D as the decentralized LQG control problem induced by D, and define the IS of D as that of ˇ D. Partially nested [8] ISs are an important subclass of ISs well studied in decentralized stochastic control. We extend such a categorization to JCCO. Formally, we call a JCCO problem D PN if the induced decentralized LQG control problem ˇ D is PN. Namely, for any i1,i2∈[n],h1<h2i_1,i_2∈[n],h_1<h_2, if there is no additional sharing and i1,h1 U_i_1,h_1 influences i2,h2− I_i_2,h_2^-, then agent (i2,h2)(i_2,h_2) can access i1,h1− I_i_1,h_1^-, i.e., i1,h1−⊆i2,h2− I_i_1,h_1^- I_i_2,h_2^-. I. Linearity and Structural Conditions for JCCO with Open-Loop Communication Strategies We now focus on the setting with open-loop communication strategies. Given any such fixed gm1:Hg^m_1:H (and thus pre-selected M1:HM_1:H), the problem becomes a decentralized LQG control one. However, it is known that the optimal control strategy of a decentralized LQG control problem may be nonlinear in general [25], making it computationally challenging. Fortunately, with partially nested ISs, the optimal control strategies can be linear [8]11 1 For completeness, we provide Corollary A.2 to establish the extension needed here for possibly singular Gaussian covariances and positive-semidefinite state and control cost matrices Qh1h∈[H+1]\Q_h^1\_h∈[H+1] and Qh2h∈[H]\Q_h^2\_h∈[H].. Hence, it is natural to ask: when it comes to JCCO (with open-loop gm1:Hg^m_1:H), when would the optimal control strategy gi,hag^a_i,h still be linear, i.e., a linear function of the available information i,h+ I_i,h^+, for all i∈[n],h∈[H]i∈[n],h∈[H]? To answer this question, first, if the IS of JCCO (i.e., the IS of baseline sharing) is not even PN, then in general it is hard for the IS of the decentralized LQG to be PN after the optimal additional sharing, especially with (high) communication costs. This may thus further break the linearity of the optimal control strategies. To formalize this intuition, we introduce the following result, whose proof, together with other omitted proofs for this section, can be found in §B. Lemma I.1. There exists a JCCO problem D with open-loop communication strategies that is not PN but satisfies Assumptions I.1, I.2, and I.4, such that the optimal control strategy is nonlinear. Due to Lemma I.1, we focus on PN JCCO problems, in order to obtain linearity of the control strategies and thus computational tractability. However, even when the baseline sharing is PN, the additional sharing may break the PN IS, and thus cause computational hardness again. Specifically, suppose agent i1i_1’s control action i1,h1 U_i_1,h_1 does not influence another agent i2i_2’s information i2,h2 I_i_2,h_2 in baseline sharing, but her control action i1,h1 U_i_1,h_1 is shared with others via additional sharing. Then, i1,h1 U_i_1,h_1 starts to influence i2,h2 I_i_2,h_2 and the PN IS may thus break. Here, the non-influence of i1,h1 U_i_1,h_1 in baseline sharing may come from two cases: 1) i1,h1 U_i_1,h_1 does not influence the underlying state h1+1 X_h_1+1; or 2) i1,h1 U_i_1,h_1 does influence the underlying state h1+1 X_h_1+1, but the effect is not observed by other agents. To avoid the impact of such non-influential actions, we make the following two assumptions, followed by justifications of their necessity regarding the existence/linearity of the optimal control strategies. For case 1) above, we make the following assumption that such useless actions will not be used or shared later. In fact, for the linear systems considered in (I.1), a useless control action will not even appear as a random variable in the system evolution, and it is natural to disregard it in the available information of later agents. Moreover, we highlight that such an assumption has been implicitly made in the common examples in decentralized stochastic control [10, 12]. Assumption I.2. For any i∈[n],h∈[H]i∈[n],h∈[H], if Bi,h=B_i,h=0, then i,h<j,(h′)−,i,h<j,(h′)+,∀h′∈[H] with h′>h,j∈[n] U_i,h∉ I_j,(h )^-, U_i,h∉ I_j,(h )^+,∀ h ∈[H] with h >h,\ j∈[n]. The significance of Assumption I.2 in JCCO is reflected in two respects. First, it helps to preserve the PN IS (as to be shown formally in Theorem I.6). Second, without it, the optimal control strategy may either not exist or be nonlinear, as shown in the following lemma. Lemma I.3. There exist PN JCCO problems 1,2D_1,D_2 with open-loop communication strategies that satisfy Assumptions I.1, I.4 but not Assumption I.2, such that the optimal control strategy of 1D_1 is nonlinear, and the team-optimal strategy of 2D_2 does not exist. For case 2) above, the PN IS may break due to the non-informativeness of agents’ observations. We thus make the following assumption. Assumption I.4. For any h∈[H−1],i∈[n]h∈[H-1],i∈[n], if Bi,h,0B_i,h 0, then E−i,h+1Bi,h,0E_-i,h+1B_i,h 0, where E−i,h+1:=[E1,h+1⊤⋯Ei−1,h+1⊤Ei+1,h+1⊤⋯En,h+1⊤]⊤E_-i,h+1:= bmatrixE_1,h+1 &·s&E_i-1,h+1 &E_i+1,h+1 &·s&E_n,h+1 bmatrix . Assumption I.4 means that, for any i∈[n]i∈[n] and h∈[H−1]h∈[H-1], if Bi,h,0B_i,h 0, then Ej,h+1Bi,h,0E_j,h+1B_i,h 0 for at least one j,ij≠ i. Thus, it does not require E−i,h+1E_-i,h+1 to have full rank or the other agents to jointly observe the whole state h+1 X_h+1. For instance, if Bi,h=B_i,h=0, then no condition is imposed on E−i,h+1E_-i,h+1. As shown below, Assumption I.4 is necessary to ensure the linearity of the optimal control strategy of D. Lemma I.5. There exists a PN JCCO problem D with open-loop communication strategies that satisfies Assumptions I.1, I.2 but not Assumption I.4, such that the optimal control strategy of D is nonlinear. We have thus justified the importance of the above assumptions and the partial nestedness of baseline sharing for JCCO problems, by showing that missing any one of them (while keeping others) may cause the nonlinearity (or even non-existence) of the optimal control strategy. On the other hand, for any fixed open-loop communication strategy g1:Hm∈1:Hmg_1:H^m _1:H^m, we denote by (g1:Hm)D(g_1:H^m) the subproblem of finding the optimal control strategy with respect to g1:Hmg_1:H^m, where we add ¯~ x~ to the notation in (g1:Hm)D(g_1:H^m) to distinguish it from the notation in the original JCCO problem. We remove ++ in the timestep index for all the notation in (g1:Hm)D(g_1:H^m) for convenience, as the IS is now fixed and only the control problem will be analyzed. Then, we can show that if D satisfies all the assumptions above, the optimal control strategy of D is linear. Indeed, under any fixed open-loop g1:Hm∈1:Hmg_1:H^m _1:H^m, the subproblem (g1:Hm)D(g_1:H^m) can be shown to be a decentralized LQG problem with PN information structures (see §A for a formal definition). Theorem I.6. Consider a PN JCCO problem D that satisfies Assumptions I.1, I.2, and I.4. For any open-loop communication strategy g1:Hm∈1:Hmg_1:H^m _1:H^m, the subproblem (g1:Hm)D(g_1:H^m) is a decentralized LQG control problem with a PN IS, and its information evolution rules satisfy: for each i∈[n]i∈[n] and h∈[H]h∈[H], ¯h=¯h−1∪¯h,¯h=χ¯h(¯h−1,¯h−1,¯h),¯i,h∈¯i,h, C_h= C_h-1∪ Z_h,~~~ Z_h= χ_h( P_h-1, U_h-1, Y_h),~~~ Y_i,h∈ I_i,h, ¯i,h=ζ¯i,h(¯i,h−1,¯i,h−1,¯i,h),¯i,h−1⊆¯i,h, P_i,h= ζ_i,h( P_i,h-1, U_i,h-1, Y_i,h), I_i,h-1 I_i,h, for some projection functions χ¯h χ_h and ζ¯i,hi∈[n]\ ζ_i,h\_i∈[n]. Furthermore, there exists an optimal strategy (g1:Hm,∗,g1:Ha,∗)(g_1:H^m, ,g_1:H^a, ) of D with open-loop communication strategies such that g1:Ha,∗g_1:H^a, is linear. Theorem I.6 provides a way to solve D by finding the optimal control strategy g1:Ha,∗g_1:H^a, with respect to any fixed open-loop communication strategy g1:Hmg_1:H^m, i.e., solving (g1:Hm)D(g_1:H^m), and then optimizing over the open-loop communication strategies g1:Hmg_1:H^m. We will thus investigate how to solve the problem (g1:Hm)D(g_1:H^m) next. IV. Dynamic Programming for Decentralized LQG with Partially Nested Information Structures Since (g1:Hm)D(g_1:H^m) is PN under the assumptions in Theorem I.6, it now suffices to develop an algorithm that can solve PN decentralized LQG control problems. For notational convenience, in this section, we will omit the superscript a for the control strategies g¯a1:H g^a_1:H for (g1:Hm)D(g_1:H^m). The algorithm to be introduced can be viewed as a dynamic-programming approach for solving a class of PN decentralized LQG problems with output feedback under the common-information-based framework [20, 18], which thus advances the results in [14] and may be of independent interest. Detailed proofs of results in this section are deferred to §C. Specifically, under PN IS, the known dynamic-programming-based approaches and sufficient statistics in [10, 19] do not apply here, as they focused on the special settings with factorized states, and with either state-feedback [10] or multi-tree coupling and communication graphs [19]. On the other hand, under the common-information-based framework [20], [14] showed that if agents are restricted to using linear control strategies, then the decentralized control problem can be reformulated as a centralized LQG one when fixing the private-information component of the strategies. However, it is not clear how to further optimize the private-information component, which induces a non-convex optimization problem in general [14]. Furthermore, [7] showed that, if additionally the CIB beliefs from [20] are strategy independent [7, Section 2.4], then the CIB beliefs can be further simplified to a finite-dimensional conditional mean of the state and private information in this linear-quadratic setting22 2 Note that [7] focused on a game setting with strategic agents, instead of the team setting in [14, 10] and the present paper.. Inspired by [7], and to address the non-convexity issue in [14], we exploit the PN IS to obtain a finite-dimensional, conditional-mean-based sufficient statistic, by establishing a new connection between the PN IS and the SI-CIB condition. To this end, we adapt our technique in [11] to this linear-quadratic setting, while addressing several fundamentally different challenges. First, we expand the PN IS into a strictly PN one, where the actions of agents who affect some other agent will also be known by the affected agent (in addition to their information). More formally, we construct a new problem ~(g1:Hm) D(g_1:H^m), whose elements are almost identical to those in (g1:Hm)D(g_1:H^m), except that the agents’ information is now expanded as follows: for any h∈[H]h∈[H] ~h=¯h∪¯i,t|i∈[n],t<h,¯i,t⊆¯h,B¯i,t,0,∀j∈[n],~j,h=¯j,h\¯j,t|t<h,¯j,t⊆¯h, C_h= C_h∪\ U_i,t\,|\,i∈[n],t<h, I_i,t C_h, B_i,t 0\, ∀ j∈[n], P_j,h= P_j,h \ U_j,t\,|\,t<h, I_j,t C_h\, (IV.1) where we add ~~ ~ to the notation in ~(g1:Hm) D(g_1:H^m), and the incremental common information is defined by ~h:=~h\~h−1,∀h∈[H] Z_h:= C_h C_h-1,∀ h∈[H]. Through such an expansion, we can leverage the new problem ~(g1:Hm) D(g_1:H^m) to solve the original (g1:Hm)D(g_1:H^m), as formalized in the following lemma. Lemma IV.1. Let D be PN and satisfy Assumptions I.1, I.2, and I.4, and for any fixed open-loop g1:Hm∈1:Hmg_1:H^m _1:H^m, let (g1:Hm)D(g_1:H^m) and ~(g1:Hm) D(g_1:H^m) be the decentralized LQG control problems constructed above. Then, there exists a function φ such that for any linear optimal control strategy g~1:H∗ g_1:H of the problem ~(g1:Hm) D(g_1:H^m), g¯1:H∗=φ(g~1:H∗,(g1:Hm)) g_1:H = ( g_1:H ,D(g_1:H^m)) is a linear optimal control strategy of (g1:Hm)D(g_1:H^m), and J(g1:Hm)(g¯1:H∗)=J~(g1:Hm)(g~1:H∗)J_D(g_1:H^m)( g_1:H )=J_ D(g_1:H^m)( g_1:H ), where J(g1:Hm)(g¯1:H∗)J_D(g_1:H^m)( g_1:H ) is the expected accumulated cost of (g1:Hm)D(g_1:H^m) under the strategy g¯1:H∗ g_1:H , and J~(g1:Hm)(g~1:H∗)J_ D(g_1:H^m)( g_1:H ) is defined similarly (cf. §A for the formal definitions). Before proceeding further, we introduce some additional notation in ~(g1:Hm) D(g_1:H^m). For any h∈[H]h∈[H], we define ~h:=[~h⊤~1,h⊤⋯~n,h⊤]⊤ S_h:= bmatrix X_h & P_1,h &·s& P_n,h bmatrix and use ~h:=~×~1,h×⋯×~n,h S_h:= X× P_1,h×·s× P_n,h to denote the space of ~h S_h. Then, we use ~h∈(~h) B_h ( S_h) with ~h(d~h):=ℙ~(g1:Hm)(d~h|~h,g~1:h−1) B_h(d S_h):=P D(g_1:H^m)(d S_h\,|\, C_h, g_1:h-1) to denote the conditional probability measure of ~h S_h given the common information ~h C_h and the past strategies g~1:h−1 g_1:h-1. Hence, we have ~h=�h(~h,g~1:h−1) B_h= _h( C_h, g_1:h-1) for some functional �h _h. The benefit of solving ~(g1:Hm) D(g_1:H^m) instead of (g1:Hm)D(g_1:H^m) is that the former satisfies the strategy-independence condition for CIB beliefs under the stated assumptions, as shown below. Theorem IV.2. If D is PN and satisfies Assumptions I.1, I.2, and I.4, then for any fixed open-loop g1:Hm∈1:Hmg_1:H^m _1:H^m, ~(g1:Hm) D(g_1:H^m) constructed as above is (strictly) PN and satisfies the SI-CIB condition: for any h∈[H]h∈[H], any realization C~h∈~h C_h∈ C_h, and any two control strategies g~1:h−1,g~1:h−1′ g_1:h-1, g_1:h-1 that can reach C~h C_h, it holds that �h(C~h,g~1:h−1)=�h(C~h,g~1:h−1′) _h( C_h, g_1:h-1)= _h( C_h, g_1:h-1 ). Meanwhile, ~(g1:Hm) D(g_1:H^m) has the information evolution rules as follows: for each i∈[n]i∈[n] and h∈[H]h∈[H], ~h C_h =~h−1∪~h,~h=χ~h(~h−1,~h−1,~h), = C_h-1∪ Z_h, Z_h= χ_h( P_h-1, U_h-1, Y_h), (IV.2) ~i,h P_i,h =ζ~i,h(~i,h−1,~i,h−1,~i,h),~i,h∈~i,h,~i,h−1⊆~i,h, = ζ_i,h( P_i,h-1, U_i,h-1, Y_i,h), Y_i,h∈ I_i,h, I_i,h-1 I_i,h, for some projection functions χ~h χ_h and ζ~i,hi∈[n]\ ζ_i,h\_i∈[n]. Furthermore, we define the prescription [20] of each agent i at timestep h as γ~i,h:~i,h→~i γ_i,h: P_i,h→ U_i. We denote by γ~h=(γ~1,h,⋯,γ~n,h) γ_h=( γ_1,h,·s, γ_n,h) the joint prescription of all the agents, and by �~i,h _i,h and �~h _h the spaces of agent i’s and the joint prescriptions at timestep h, respectively. For notational convenience, we write γ~h(~h)=[γ~1,h(~1,h)⊤⋯γ~n,h(~n,h)⊤]⊤ γ_h( P_h)= bmatrix γ_1,h( P_1,h) &·s& γ_n,h( P_n,h) bmatrix . Following [7], under the SI-CIB condition, the belief ~h B_h in ~(g1:Hm) D(g_1:H^m) is a Gaussian distribution with mean �~h=[[~h|~h]⊤[~1,h|~h]⊤⋯[~n,h|~h]⊤]⊤ _h= bmatrixE[ X_h\,|\, C_h] &E[ P_1,h\,|\, C_h] &·s&E[ P_n,h\,|\, C_h] bmatrix and covariance �~h _h that evolve over time, as formalized below. Lemma IV.3 (Adapted from Lemmas 2.2 & 2.3 in [7]). Consider a decentralized LQG problem ~(g1:Hm) D(g_1:H^m) constructed by the strict expansion in Theorem IV.2, that has information evolution rules in Equation (IV.2) and satisfies the SI-CIB condition. Then the CIB belief ~h B_h admits a Gaussian distribution (�~h,�~h)N( _h, _h), and the conditional mean �~h _h and the conditional covariance �~h _h of ~h B_h evolve as follows: for any h∈[2:H]h∈[2:H], �~h= ~h1(�~h−1,~h),�~h= ~h2(�~h−1), _h= _h^1( _h-1, Z_h), _h= _h^2( _h-1), where ~h1 _h^1 is a fixed linear function, ~h2 _h^2 is a fixed deterministic function, and neither depends on the strategies g~1:h−1 g_1:h-1. In particular, �~h _h is deterministic and strategy-independent. According to Lemma IV.3, we know that �~h _h depends only on ~tt=1h\ Z_t\_t=1^h. Hence, there exists a fixed linear transformation ~h3 _h^3 such that �~h= ~h3(~h) _h= _h^3( C_h). Then, following [7], we can construct a new problem ~‡(g1:Hm) D (g_1:H^m): At timestep h∈[H]h∈[H], the state of ~‡(g1:Hm) D (g_1:H^m) is defined as �~h∈~h _h∈ S_h and the action is defined as γ~h∈�~h γ_h∈ _h. The stage cost is defined as ch‡(�~h,γ~h):=[~h⊤Q~h1~h+γ~h(~h)⊤Q~h2γ~h(~h)+[h=H]~H+1⊤Q~H+11~H+1],c_h ( _h, γ_h):=E [ X_h Q_h^1 X_h+ γ_h( P_h) Q_h^2 γ_h( P_h)+ 1[h=H] X_H+1 Q_H+1^1 X_H+1 ], where the expectation is taken over [~h⊤~h⊤]⊤∼(�~h,�~h)[ X_h ~ P_h ] ( _h, _h) and, when h=Hh=H, also over the final state ~H+1 X_H+1 generated by the system dynamics. The admissible strategy at timestep h in ~‡(g1:Hm) D (g_1:H^m) is denoted by gh‡:~h→�~hg _h: C_h→ _h, and h‡G_h is the set of all admissible strategies. The objective of ~‡(g1:Hm) D (g_1:H^m) is then defined by J~‡(g1:Hm)(g1:H‡):=[∑h=1Hch‡(�~h,γ~h)]. J_ D (g_1:H^m)(g_1:H ):=E [ _h=1^Hc_h ( _h, γ_h) ]. As shown in the following theorem, the constructed problem ~‡(g1:Hm) D (g_1:H^m) is a Markov decision problem, and the optimal strategy of ~‡(g1:Hm) D (g_1:H^m) can be used to compute the team-optimal strategy of ~(g1:Hm) D(g_1:H^m), and vice versa. Theorem IV.4. Let ~‡(g1:Hm) D (g_1:H^m) be the problem constructed from ~(g1:Hm) D(g_1:H^m) satisfying the SI-CIB condition. Then, ~‡(g1:Hm) D (g_1:H^m) is a Markov decision problem with horizon H, state �~hh∈[H]\ _h\_h∈[H], control action γ~hh∈[H]\ γ_h\_h∈[H], and cost function ch‡h∈[H]\c_h \_h∈[H]. Moreover, there exist bijections ςh:~h→h‡ _h: G_h _h for all h∈[H]h∈[H] that can be defined as follows: for any strategy g~h∈~h g_h∈ G_h and realization of common information C~h∈~h C_h∈ C_h, ςh(g~h)(C~h)(⋅)=g~h(C~h,⋅) _h( g_h)( C_h)(·)= g_h( C_h,·). These bijections also satisfy the following equality for every strategy g~1:H∈~1:H g_1:H∈ G_1:H: J~(g1:Hm)(g~1:H)=J~‡(g1:Hm)(g1:H‡)J_ D(g_1:H^m)( g_1:H)=J_ D (g_1:H^m)(g _1:H), where gh‡=ςh(g~h),∀h∈[H]g_h = _h( g_h),∀ h∈[H]. By Theorem IV.4 and Lemma IV.5 below, an optimal strategy exists and may be chosen Markovian, with gh‡,∗g_h , depending on the common record only through �~h _h. In particular, for any h∈[H]h∈[H], we can write gh‡,∗g_h , as gh‡,∗(~h)=h∗( ~h3(~h))g_h , ( C_h)= G_h ( _h^3( C_h)) for some function h∗ G_h . Furthermore, if we define the strategy g~h∗=ςh−1(gh‡,∗),∀h∈[H] g_h = _h^-1(g_h , ),∀ h∈[H], then g~1:H∗ g_1:H is an optimal strategy of ~(g1:Hm) D(g_1:H^m). Therefore, there is no loss of optimality in restricting attention to g~i,h g_i,h that chooses action ~i,h U_i,h depending only on �~h _h and ~i,h P_i,h for any h∈[H],i∈[n]h∈[H],i∈[n] in solving the problem ~(g1:Hm) D(g_1:H^m). We categorize this type of strategy as common-information-based Markovian (CIB-Markovian) strategy. Moreover, we can define the associated value functions for every h∈[H]h∈[H] by V~h(�~h):=ming~h:H[∑t=hH(~t⊤Q~t1~t+~t⊤Q~t2~t)+~H+1⊤Q~H+11~H+1|�~h,g~h:H], V_h( _h):= _ g_h:HE [ _t=h^H( X_t Q_t^1 X_t+ U_t Q_t^2 U_t)+ X_H+1 Q_H+1^1 X_H+1\, |\, _h, g_h:H ], which satisfy the following Bellman Equations: V~h(�~h)=minγ~h∈�~hch‡(�~h,γ~h)+[V~h+1(�~h+1)|�~h,γ~h],h∈[H−1],minγ~H∈�~HcH‡(�~H,γ~H),h=H. V_h( _h)= cases _ γ_h∈ _h \c_h ( _h, γ_h)+E[ V_h+1( _h+1)\,|\, _h, γ_h] \,&h∈[H-1],\\[5.69054pt] _ γ_H∈ _Hc_H ( _H, γ_H),&h=H. cases (IV.3) Then, we can derive an approach to obtain the optimal strategy g~1:H∗ g_1:H by dynamic programming: for all h∈[H]h∈[H] and C~h∈~h C_h∈ C_h, g~h∗(C~h,⋅) g_h ( C_h,·) minimizes the right-hand side of Equation (IV.3) at �~h= ~h3(C~h) _h= _h^3( C_h). However, there are uncountably many possible C~h C_h (or �~h _h), making this approach computationally intractable without a closed-form solution. For computational tractability, we leverage the linearity property of the optimal strategy. Although it is known that there exists a linear optimal strategy under PN IS (cf. Corollary A.2), and if it exists, the optimal strategy can be CIB-Markovian under the SI-CIB condition (cf. Theorem IV.4), the existence of an optimal strategy with both properties remains unclear. We show the existence of such an optimal strategy below. Lemma IV.5. Suppose ~(g1:Hm) D(g_1:H^m) is a decentralized LQG control problem constructed from a PN JCCO problem D that satisfies Assumptions I.1, I.2, and I.4. Then, there exists an optimal linear strategy g~1:H∗ g_1:H , and matrices ℰ~i,h∗i∈[n],h∈[H],ℱ~i,h∗i∈[n],h∈[H]\ E_i,h \_i∈[n],h∈[H],\ F_i,h \_i∈[n],h∈[H] such that γ~i,h∗(⋅)=ℰ~i,h∗�~h+ℱ~i,h∗⋅ γ_i,h (·)= E_i,h _h+ F_i,h ·, γ~h∗ γ_h minimizes the right-hand side of Equation (IV.3) at every CIB mean �~h _h reachable under a preceding strategy, for each h∈[H]h∈[H], and g~i,h∗(C~h,P~i,h)=ℰ~i,h∗ ~h3(C~h)+ℱ~i,h∗P~i,h g_i,h ( C_h, P_i,h)= E_i,h _h^3( C_h)+ F_i,h P_i,h for any i∈[n],h∈[H],C~h∈~h,P~i,h∈~i,hi∈[n],h∈[H], C_h∈ C_h, P_i,h∈ P_i,h. Lemma IV.5 shows that it suffices to compute the optimal strategy that is linear in �~h _h and ~i,h P_i,h. Then, for any i∈[n],h∈[H]i∈[n],h∈[H], we can parameterize the strategy g~i,h g_i,h by two matrices ℰ~i,h,ℱ~i,h E_i,h, F_i,h, where g~i,h(~h,~i,h)=ℰ~i,h ~h3(~h)+ℱ~i,h~i,h g_i,h( C_h, P_i,h)= E_i,h _h^3( C_h)+ F_i,h P_i,h. Define ℰ~h∗:=[(ℰ~1,h∗)⊤⋯(ℰ~n,h∗)⊤]⊤ E_h := bmatrix( E_1,h ) &·s&( E_n,h ) bmatrix and ℱ~h∗:=diag(ℱ~1,h∗,⋯,ℱ~n,h∗) F_h := diag( F_1,h ,·s, F_n,h ). With these definitions, we can find the optimal control strategy g~h∗ g_h by solving for the optimal matrices: (ℰ~h∗,ℱ~h∗)∈argminℰ~h,ℱ~hch‡(�~h,γ~h)+[V~h+1(�~h+1)|�~h=�~h,ℰ~h,ℱ~h],h∈[H−1],cH‡(�~H,γ~H),h=H, ( E_h , F_h )∈ argmin_ E_h, F_h casesc_h ( _h, γ_h)+E[ V_h+1( _h+1)\,|\, _h= _h, E_h, F_h],&h∈[H-1],\\ c_H ( _H, γ_H),&h=H, cases (IV.4) As shown in the following theorem, such optimal matrices can be computed through a set of Riccati Equations with better computational tractability. Theorem IV.6. Suppose ~(g1:Hm) D(g_1:H^m) is a decentralized LQG control problem constructed from a PN JCCO problem D that satisfies Assumptions I.1, I.2, and I.4. For h∈[H−1]h∈[H-1], the conditional mean evolves as �~h+1=Kh+11�~h+Kh+12~h+Kh+13~h+Kh+14~h+1 _h+1=K_h+1^1 _h+K_h+1^2 P_h+K_h+1^3 U_h+K_h+1^4 Y_h+1, where these matrices are computed by a centralized Kalman filter. For h∈[H−1]h∈[H-1], define L~h1=L~h3:=Kh+13+Kh+14E~h+1B~h, L_h^1= L_h^3:=K_h+1^3+K_h+1^4 E_h+1 B_h, L~h2:=Kh+11+Kh+12p,h+Kh+14E~h+1A~hx,h, L_h^2:=K_h+1^1+K_h+1^2I_p,h+K_h+1^4 E_h+1 A_hI_x,h, L~h4:=Kh+12p,h+Kh+14E~h+1A~hx,h, L_h^4:=K_h+1^2I_p,h+K_h+1^4 E_h+1 A_hI_x,h, and set L~H1=L~H3:=B~H L_H^1= L_H^3:= B_H and L~H2=L~H4:=A~Hx,H L_H^2= L_H^4:= A_HI_x,H. Then, the value function V~h(�~h) V_h( _h) has the quadratic form V~h(�~h)=�~h⊤R~h�~h+c~h V_h( _h)= _h R_h _h+ c_h for some matrix R~h R_h and constant c~h c_h that do not depend on �~h _h, which can be computed through the following backward recursion: R~h R_h =(ℰ^h∗)⊤Q~h2ℰ^h∗+(L~h1ℰ^h∗+L~h2)⊤R~h+1(L~h1ℰ^h∗+L~h2)+x,h⊤Q~h1x,h, =( E_h ) Q_h^2 E_h +( L_h^1 E_h + L_h^2) R_h+1( L_h^1 E_h + L_h^2)+I_x,h Q_h^1I_x,h, (IV.5) R~H+1 R_H+1 =Q~H+11, = Q_H+1^1, ℰ^h∗ E_h =−(Q~h2+(L~h1)⊤R~h+1L~h1)†(L~h1)⊤R~h+1L~h2, =-( Q_h^2+( L_h^1) R_h+1 L_h^1) ( L_h^1) R_h+1 L_h^2, where x,h,p,hI_x,h,I_p,h are the projection matrices that project vectors in ~h=~×~h S_h= X× P_h to the corresponding vectors in ~ X and ~h P_h, respectively. For h∈[H]h∈[H], define �ty:=diag(�~1,t,…,�~n,t),�~h:=Kh+14(E~h+1�0,hE~h+1⊤+�h+1y)(Kh+14)⊤,h∈[H−1],�0,H,h=H. _t^y:= diag( _1,t,…, _n,t), _h:= casesK_h+1^4( E_h+1 _0,h E_h+1 + _h+1^y)(K_h+1^4) ,&h∈[H-1],\\ _0,H,&h=H. cases Also, ℱ~i,h∗i∈[n],c~h\ F_i,h \_i∈[n], c_h can be computed as follows: ℱ~h∗∈argminℱ~h=diag(ℱ~1,h,⋯,ℱ~n,h)Tr(ℱ~hp,h�~hp,h⊤ℱ~h⊤Q~h2)+Tr((L~h3ℱ~hp,h+L~h4)�~h(L~h3ℱ~hp,h+L~h4)⊤R~h+1), F_h ∈ argmin_ F_h= diag( F_1,h,·s, F_n,h) Tr( F_hI_p,h _hI_p,h F_h Q_h^2)+ Tr(( L_h^3 F_hI_p,h+ L_h^4) _h( L_h^3 F_hI_p,h+ L_h^4) R_h+1), (IV.6) ℱ~h∗:=diag(ℱ~1,h∗,⋯,ℱ~n,h∗), F_h := diag( F_1,h ,·s, F_n,h ), c~h=Tr(ℱ~h∗p,h�~hp,h⊤(ℱ~h∗)⊤Q~h2)+Tr(x,h�~hx,h⊤Q~h1)+Tr((L~h3ℱ~h∗p,h+L~h4)�~h(L~h3ℱ~h∗p,h+L~h4)⊤R~h+1) c_h= Tr( F_h I_p,h _hI_p,h ( F_h ) Q_h^2)+ Tr(I_x,h _hI_x,h Q_h^1)+ Tr(( L_h^3 F_h I_p,h+ L_h^4) _h( L_h^3 F_h I_p,h+ L_h^4) R_h+1) +Tr(�~hR~h+1)+c~h+1, + Tr( _h R_h+1)+ c_h+1, c~H+1=0. c_H+1=0. Finally, ℰ~h∗:=[(ℰ~1,h∗)⊤⋯(ℰ~n,h∗)⊤]⊤ E_h := bmatrix( E_1,h ) &·s&( E_n,h ) bmatrix is computed as ℰ~h∗=ℰ^h∗−diag(ℱ~1,h∗,ℱ~2,h∗,⋯,ℱ~n,h∗)p,h. E_h = E_h - diag( F_1,h , F_2,h ,·s, F_n,h )I_p,h. (IV.7) Solving JCCO with open-loop gm1:Hg^m_1:H: Now we are ready to solve the problem (g1:Hm)D(g^m_1:H). First, from Theorem IV.7, we can solve for g~1:H∗ g_1:H as g~i,h∗(~h,~i,h)=ℰ~i,h∗ ~h3(~h)+ℱ~i,h∗~i,h g_i,h ( C_h, P_i,h)= E_i,h _h^3( C_h)+ F_i,h P_i,h for any i∈[n],h∈[H]i∈[n],h∈[H], and obtain the optimal value of ming~1:HJ~(g1:Hm)(g~1:H)=[V~1(�~1)]:=c~0 _ g_1:HJ_ D(g_1:H^m)( g_1:H)=E[ V_1( _1)]:= c_0. Note that �~1= ~13(~1)=K11~1,~1=~1=χ~1(~1)=K12~1,~1=E~1~1+~1:n,1 _1= _1^3( C_1)=K_1^1 C_1, C_1= Z_1= χ_1( Y_1)=K_1^2 Y_1, Y_1= E_1 X_1+ W_1:n,1, where ~1∼(,�1) X_1 (0, _1) for some matrices K11,K12K_1^1,K_1^2. Therefore, we can write c~0=Tr(E~1⊤(K12)⊤(K11)⊤R~1K11K12E~1�1)+Tr((K12)⊤(K11)⊤R~1K11K12diag(�1,1,⋯,�n,1))+c~1 c_0= Tr( E_1 (K_1^2) (K_1^1) R_1K_1^1K_1^2 E_1 _1)+ Tr((K_1^2) (K_1^1) R_1K_1^1K_1^2 diag( _1,1,·s, _n,1))+ c_1. Then, by Lemma IV.1, we can obtain an optimal control strategy of (g1:Hm)D(g^m_1:H) as g¯1:H∗=φ(g~1:H∗,(g1:Hm)) g_1:H = ( g_1:H ,D(g_1:H^m)) such that J(g1:Hm)(g¯1:H∗)=J~(g1:Hm)(g~1:H∗)J_D(g_1:H^m)( g_1:H )=J_ D(g_1:H^m)( g_1:H ). If we assign g1:H∗=g¯1:H∗g_1:H = g_1:H , then g1:H∗g_1:H is an optimal control strategy with respect to the communication strategy g1:Hmg_1:H^m in D. Let M1:HM_1:H be the communication actions chosen by g1:Hmg_1:H^m such that Mh=ghmM_h=g^m_h. Then, J(g1:Hm,g1:H∗)=J~(g1:Hm)(g~1:H∗)+∑h=1Hh(ghm)J_D(g_1:H^m,g_1:H )=J_ D(g_1:H^m)( g_1:H )+ _h=1^HK_h(g^m_h). Therefore, we can finally solve the JCCO problem by solving g1:Hm,∗∈argming1:HmJ~(g1:Hm)(g~1:H∗)+∑h=1Hh(ghm)g_1:H^m, ∈ argmin_g_1:H^mJ_ D(g_1:H^m)( g_1:H )+ _h=1^HK_h(g^m_h). V. Extension to JCCO with Closed-loop Communication Strategies In this section, we extend our study to the JCCO problem with closed-loop communication strategies, i.e., for any h∈[H],i∈[n],gi,hm:ℐi,h−×ℳ1:h−1→ℳi,h∈[H],i∈[n],g_i,h^m:I_i,h^-×M_1:h-1 _i,h, with associated (closed-loop) control strategies gi,ha:ℐi,h+×ℳ1:h→ig_i,h^a:I_i,h^+×M_1:h _i. Since each agent may influence others through the additional sharing determined by the communication strategy gi,hmg_i,h^m, while others may not access the input of gi,hmg_i,h^m, PN IS may break. Therefore, the SI-CIB condition does not hold in general and the common-information-based beliefs need not be Gaussian distributions that can be characterized by the finite-dimensional conditional mean and covariance. For the recursion at h=1h=1, identify (1,0+)=(1,∅)( X_1, P_0^+)=( X_1, ) with 1 X_1 and set 0+:=(,�1) B_0^+:=N( 0, _1), �0+:= _0^+:= 0, and �0+:=�1 _0^+:= _1. Since the strict expansion has the same initial law, set ~0+:=0+ B_0^+:= B_0^+, �~0+:=�0+ _0^+:= _0^+, �~0+:=�0+ _0^+:= _0^+, and γ0a=γ~0a:=∅ _0^a= γ_0^a:= . Fortunately, we can still apply the approach designed in [11], which expanded the problem into a new one that satisfies the SI-CIB condition, and then solved it via dynamic programming. To guarantee the SI-CIB condition, [11] introduced the following assumption. Assumption V.1. The communication strategies take common information and past communication actions as input, i.e., ∀i∈[n],h∈[H],gi,hm:h−×ℳ1:h−1→ℳi,h.∀ i∈[n],h∈[H],g_i,h^m:C_h^-×M_1:h-1 _i,h. To this end, we first introduce some additional notation. We denote by h−(M1:h−1)⊆h−P_h^-(M_1:h-1) _h^- the set of all possible Ph−P_h^- given M1:h−1M_1:h-1. Similarly, we denote by h+(M1:h)⊆h+P_h^+(M_1:h) _h^+ the set of all possible Ph+P_h^+ given M1:hM_1:h. Then, given any M1:h−1M_1:h-1, we can define h−∈(×h−(M1:h−1)) B_h^- (X×P_h^-(M_1:h-1)) as the conditional probability measure of state h X_h and private information h− P_h^-, given the past strategies g1:h−1m,g1:h−1ag_1:h-1^m,g_1:h-1^a, communication actions M1:h−1M_1:h-1, and common information h− C_h^- before additional sharing. Similarly, we define h+∈(×h+(M1:h)) B_h^+ (X×P_h^+(M_1:h)) as the conditional probability measure of h X_h and h+ P_h^+ given g1:hm,g1:h−1a,1:hg_1:h^m,g_1:h-1^a, M_1:h, and h+ C_h^+. Hence, we can write h−=�h−(h−,1:h−1,g1:h−1m,g1:h−1a) B_h^-= _h^-( C_h^-, M_1:h-1,g_1:h-1^m,g_1:h-1^a) and h+=�h+(h+,1:h,g1:hm,g1:h−1a) B_h^+= _h^+( C_h^+, M_1:h,g_1:h^m,g_1:h-1^a) for some functionals �h− _h^- and �h+ _h^+, respectively. Then, we can extend the SI-CIB condition as follows. Definition V.2 (SI-CIB Condition of D). We say that a JCCO problem D satisfies the strategy-independent common-information-based belief condition if • For any h∈[H]h∈[H], any realization Ch−∈h−,M1:h−1∈ℳ1:h−1C_h^- _h^-,M_1:h-1 _1:h-1 and any two strategies (g1:h−1m,g1:h−1a)(g_1:h-1^m,g_1:h-1^a) and (g1:h−1m,′,g1:h−1a,′)(g_1:h-1^m, ,g_1:h-1^a, ) that can reach Ch−C_h^- and M1:h−1M_1:h-1, it holds that �h−(Ch−,M1:h−1,g1:h−1m,g1:h−1a)=�h−(Ch−,M1:h−1,g1:h−1m,′,g1:h−1a,′). _h^-(C_h^-,M_1:h-1,g_1:h-1^m,g_1:h-1^a)= _h^-(C_h^-,M_1:h-1,g_1:h-1^m, ,g_1:h-1^a, ). • For any h∈[H]h∈[H], any realization Ch+∈h+,M1:h∈ℳ1:hC_h^+ _h^+,M_1:h _1:h and any two strategies (g1:hm,g1:h−1a)(g_1:h^m,g_1:h-1^a) and (g1:hm,′,g1:h−1a,′)(g_1:h^m, ,g_1:h-1^a, ) that can reach Ch+C_h^+ and M1:hM_1:h, it holds that �h+(Ch+,M1:h,g1:hm,g1:h−1a)=�h+(Ch+,M1:h,g1:hm,′,g1:h−1a,′). _h^+(C_h^+,M_1:h,g_1:h^m,g_1:h-1^a)= _h^+(C_h^+,M_1:h,g_1:h^m, ,g_1:h-1^a, ). Under Assumption V.1, for both timesteps h−h^- and h+h^+, we can expand the problem D into a new one by adding some actions that affect other agents into the common information, similarly to Equation (IV.1). After the strict expansion, the new problem satisfies the SI-CIB condition, and the team-optimal strategy of the expanded JCCO problem can be used to obtain that of the original one. More importantly, satisfying the SI-CIB condition ensures that ~h− B_h^- and ~h+ B_h^+ are Gaussian distributions, as shown below. Theorem V.3. Let ~ D be the problem obtained by the above strict expansion from a JCCO problem D with PN IS that satisfies Assumptions I.1, I.2, I.4, and V.1. Then, ~ D is a JCCO problem satisfying the SI-CIB condition. Moreover, for any h∈[H]h∈[H], ~h−,~h+ B_h^-, B_h^+ are Gaussian distributions. Formally, ~h−=(�~h−,�~h−),~h+=(�~h+,�~h+) B_h^-=N( _h^-, _h^-), B_h^+=N( _h^+, _h^+), and �~h−= ~h−1(�~(h−1)+,~1:h−1,~hb),�~h−= ~h−2(~1:h−1), _h^-= _h^-^1( _(h-1)^+, M_1:h-1, Z_h^b), _h^-= _h^-^2( M_1:h-1), (V.1) �~h+= ~h+1(�~h−,~1:h,~ha),�~h+= ~h+2(~1:h), _h^+= _h^+^1( _h^-, M_1:h, Z_h^a), _h^+= _h^+^2( M_1:h), for some functions ~h−1, ~h+1, ~h−2, ~h+2 _h^-^1, _h^+^1, _h^-^2, _h^+^2 that do not depend on the strategies g~1:hm,g~1:ha g_1:h^m, g_1:h^a. Theorem V.3 provides a way to solve JCCO with closed-loop communication strategies via dynamic programming. Specifically, we can conduct dynamic programming over the spaces of (M~1:h,�~h+)∈ℳ~1:h×(~×~h+(M~1:h))( M_1:h, _h^+)∈ M_1:h×( X× P_h^+( M_1:h)) and the spaces of (M~1:h−1,�~h−)∈ℳ~1:h−1×(~×~h−(M~1:h−1))( M_1:h-1, _h^-)∈ M_1:h-1×( X× P_h^-( M_1:h-1)), to solve for the optimal strategies g~ha,∗ g_h^a, and g~hm,∗ g_h^m, , respectively. The overall algorithm, under its stated attainment assumption, is tabulated as Algorithm 1 in §D. VI. Numerical Example and Experimental Results In this section, we demonstrate the proposed method through concrete JCCO examples. We first present a numerical example that illustrates the implementation of our method step by step. We then evaluate the approach experimentally and analyze its performance. VI-A Numerical example We illustrate the method using a linear-quadratic JCCO problem D with n=2n=2, H=2H=2, and open-loop communication strategies. The baseline sharing of D is one-step measurement delay; that is, for any i∈[2]i∈[2] and h∈[2]h∈[2], h−=1:h−1 C_h^-=\ Y_1:h-1\ and i,h−=i,h P_i,h^-=\ Y_i,h\. The communication action spaces are ℳi,h=0,1M_i,h=\0,1\ for all i∈[2]i∈[2] and h∈[2]h∈[2], where i,h=1 M_i,h=1 indicates i,ha=i,h Z_i,h^a=\ Y_i,h\ and i,h=0 M_i,h=0 indicates i,ha=∅ Z_i,h^a= . The communication cost is h(h)=0.2(1,h+2,h)K_h( M_h)=0.2( M_1,h+ M_2,h). The system is defined by =i=i=ℝX=Y_i=U_i=R for all i∈[2]i∈[2], with the following data: A1=1.25,A2=1.15,B1,1=1,B2,1=0.9,B1,2=0.85,B2,2=1.1,E1,1=1.1,E2,1=0.8,E1,2=0.9,E2,2=1.2, A_1=1.25,A_2=1.15,B_1,1=1,B_2,1=0.9,B_1,2=0.85,B_2,2=1.1,E_1,1=1.1,E_2,1=0.8,E_1,2=0.9,E_2,2=1.2, �0,1=1.5,�0,2=2,�1,1=1,�2,1=1.5,�1,2=2,�2,2=0.8,�1=2,Q11=5,Q21=6,Q31=8,Q12=Q22=2. _0,1=1.5, _0,2=2, _1,1=1, _2,1=1.5, _1,2=2, _2,2=0.8, _1=2,Q_1^1=5,Q_2^1=6,Q_3^1=8,Q_1^2=Q_2^2=I_2. The problem D has PN IS and satisfies Assumptions I.1, I.2, and I.4. We solve it as follows. Step 1: There are 16 possible communication strategies with gi,hm∈0,1g_i,h^m∈\0,1\ for all h∈[2]h∈[2] and i∈[2]i∈[2]. For each g1:2mg_1:2^m, we construct the associated decentralized LQG problem (g1:2m)D(g_1:2^m). Step 2: We expand (g1:2m)D(g_1:2^m) to ~(g1:2m) D(g_1:2^m) and compute the corresponding filter matrices and covariance matrices from the information structure of ~(g1:2m) D(g_1:2^m). To illustrate the computation, consider g1,1m=g2,2m=1g_1,1^m=g_2,2^m=1 and g2,1m=g1,2m=0g_2,1^m=g_1,2^m=0. This choice yields ~1=~1,1,~2=~1,~1,~2,2,~2,1=~2,1,~1,2=~1,2 C_1=\ Y_1,1\, C_2=\ Y_1, U_1, Y_2,2\, P_2,1=\ Y_2,1\, P_1,2=\ Y_1,2\, and ~1,1=~2,2=∅ P_1,1= P_2,2= . For timestep h=2h=2, K2jj∈[4]\K_2^j\_j∈[4] and �~2 _2 are given by K21=E1,2ext(1−P̊2E2,22E2,22P̊2+�2,2)A1P̊1E1,12�1+�1,1�1�1,1[10],K22=E1,2ext(1−P̊2E2,22E2,22P̊2+�2,2)A1P̊1E2,1�2,1, K_2^1=E_1,2^ext (1- P_2E_2,2^2E_2,2^2 P_2+ _2,2 )A_1 P_1 E_1,1^2 _1+ _1,1 _1 _1,1 bmatrix1&0 bmatrix,K_2^2=E_1,2^ext (1- P_2E_2,2^2E_2,2^2 P_2+ _2,2 )A_1 P_1 E_2,1 _2,1, (VI.1) K23=E1,2ext(1−P̊2E2,22E2,22P̊2+�2,2)B1,K24=E1,2extP̊2E2,2[01]E2,22P̊2+�2,2,�~2=[P̊2�2,2E2,22P̊2+�2,2E1,2P̊2�2,2E2,22P̊2+�2,2E1,2P̊2�2,2E2,22P̊2+�2,2�1,2+E1,22P̊2�2,2E2,22P̊2+�2,2], K_2^3=E_1,2^ext (1- P_2E_2,2^2E_2,2^2 P_2+ _2,2 )B_1,K_2^4=E_1,2^ext P_2E_2,2 bmatrix0&1 bmatrixE_2,2^2 P_2+ _2,2, _2= bmatrix P_2 _2,2E_2,2^2 P_2+ _2,2& E_1,2 P_2 _2,2E_2,2^2 P_2+ _2,2\\ E_1,2 P_2 _2,2E_2,2^2 P_2+ _2,2& _1,2+ E_1,2^2 P_2 _2,2E_2,2^2 P_2+ _2,2 bmatrix, where P̊1=(1�1+E1,12�1,1+E2,12�2,1)−1 P_1=( 1 _1+ E_1,1^2 _1,1+ E_2,1^2 _2,1)^-1, P̊2=A12P̊1+�0,1 P_2=A_1^2 P_1+ _0,1 and E1,2ext=[1E1,2]⊤E_1,2^ext= bmatrix1&E_1,2 bmatrix . The initial quantities K11,K12K_1^1,K_1^2, and �~1 _1 are computed similarly. Step 3: Apply the backward recursion and finite-dimensional convex quadratic minimizations in Theorem IV.7 to compute c~0 c_0 and g~1:2∗ g_1:2 for the selected g1:2mg_1:2^m, and define f(g1:2m):=ming1:2aJ(g1:2m,g1:2a)=c~0+1(g1m)+2(g2m)f(g_1:2^m):= _g_1:2^aJ_D(g_1:2^m,g_1:2^a)= c_0+K_1(g_1^m)+K_2(g_2^m). Step 4: Compute g1:2m,∗∈argming1:2m∈1:2mf(g1:2m)g_1:2^m, ∈ argmin_g_1:2^m _1:2^mf(g_1:2^m). The optimizer is g1,1m,∗=g2,2m,∗=1g_1,1^m, =g_2,2^m, =1 and g2,1m,∗=g1,2m,∗=0g_2,1^m, =g_1,2^m, =0. The optimal strategy g~1:2∗ g_1:2 for the problem ~(g1:2m,∗) D(g_1:2^m, ) is33 3 All numerical values are rounded to three decimal places. g~1,1∗(~1,1)=−0.410~1,1,g~2,1∗(~2,1)=−0.219~1,1−0.292~2,1, g_1,1 ( I_1,1)=-0.410 Y_1,1, g_2,1 ( I_2,1)=-0.219 Y_1,1-0.292 Y_2,1, g~1,2∗(~1,2)=−0.038~1,1−0.019~2,1−0.060~1,1−0.054~2,1−0.196~1,2−0.200~2,2, g_1,2 ( I_1,2)=-0.038 Y_1,1-0.019 Y_2,1-0.060 U_1,1-0.054 U_2,1-0.196 Y_1,2-0.200 Y_2,2, g~2,2∗(~2,2)=−0.079~1,1−0.038~2,1−0.123~1,1−0.110~2,1−0.410~2,2. g_2,2 ( I_2,2)=-0.079 Y_1,1-0.038 Y_2,1-0.123 U_1,1-0.110 U_2,1-0.410 Y_2,2. The optimal objective of ~(g1:2m,∗) D(g_1:2^m, ) is c~0=45.947 c_0=45.947. We then compute g1:2a,∗=φ(g~1:2∗,(g1:2m,∗))g_1:2^a, = ( g_1:2 ,D(g_1:2^m, )), where g1a,∗=g~1∗g_1^a, = g_1 , and g1,2a,∗(1,2+)=−0.0021,1−0.0032,1−0.1961,2−0.2002,2, g_1,2^a, ( I_1,2^+)=-0.002 Y_1,1-0.003 Y_2,1-0.196 Y_1,2-0.200 Y_2,2, g2,2a,∗(2,2+)=−0.0041,1−0.0062,1−0.4102,2, g_2,2^a, ( I_2,2^+)=-0.004 Y_1,1-0.006 Y_2,1-0.410 Y_2,2, -5.69054pt The optimal objective of D is J(g1:2m,∗,g1:2a,∗)=46.347J_D(g_1:2^m, ,g_1:2^a, )=46.347. VI-B Experimental results (a) (b) Figure 1: Average values over 10 random seeds under different baseline sharing protocols and values of α. Figure (a): For each bar, the dark and light portions correspond to the control cost and communication cost, respectively. Figure (b): Each value represents the total number of additionally shared elements. (a) (b) Figure 2: Total cost under different values of α and communication strategies. Each line corresponds to a different communication strategy, with N representing the number of elements shared through additional sharing. Figure (a): One-Step-Delay baseline sharing. Figure (b): One-Direction-One-Step-Delay baseline sharing. We evaluate our approach on JCCO problems with the following four information structures. In the descriptions below, labeled observations and actions with nonpositive time indices are omitted, and 0+=0=∅ C_0^+= U_0= . • Single-Controller: Only agent 1 can control the underlying state h X_h, i.e., Bi,h=B_i,h=0 for all h∈[H]h∈[H] and i∈[2:n]i∈[2:n]. Agent 1 shares all of its information with the other agents, while the remaining agents share their information with a d-step delay for some d>1d>1; that is, h−=(h−1)+∪1,h,1,h−1,2:n,h−d C_h^-= C_(h-1)^+∪\ Y_1,h, U_1,h-1, Y_2:n,h-d\, 1,h−=∅ P_1,h^-= , and i,h−=i,(h−1)+∪i,h∖i,h−d P_i,h^-= P_i,(h-1)^+∪\ Y_i,h\ \ Y_i,h-d\ for all h∈[H]h∈[H] and i∈[2:n]i∈[2:n]. In the experiments, we set n=3,H=6n=3,H=6. • Turn-based-Controllers: For each h∈[H]h∈[H] with the unique representation h=kn+ih=kn+i, where k∈ℤ≥0k _≥ 0 and i∈[n]i∈[n], only agent i can control the state at timestep h, i.e., Bj,h=B_j,h=0 for all j,ij≠ i. For h<Hh<H, at timestep h+1h+1, agent i shares its past observations and actions with the other agents; that is, (h+1)−=h+∪i,h−n+1:h,i,h C_(h+1)^-= C_h^+∪\ Y_i,h-n+1:h, U_i,h\. In the experiments, we set n=3,H=6n=3,H=6. • One-Step-Delay: Through baseline sharing, each agent shares its past observations and actions with the other agents, i.e., h−=(h−1)+∪h−1,h−1 C_h^-= C_(h-1)^+∪\ Y_h-1, U_h-1\ and i,h−=i,h P_i,h^-=\ Y_i,h\ for all h∈[H]h∈[H] and i∈[n]i∈[n]. In the experiments, we set n=3,H=7n=3,H=7. • One-Direction-One-Step-Delay: Agent 1 shares all of its information with the other agents, while the remaining agents share their past observations and actions through baseline sharing; that is, h−=(h−1)+∪1,h,2:n,h−1,h−1,1,h−=∅,i,h−=i,h C_h^-= C_(h-1)^+∪\ Y_1,h, Y_2:n,h-1, U_h-1\, P_1,h^-= , P_i,h^-=\ Y_i,h\ for all h∈[H]h∈[H] and i∈[2:n]i∈[2:n]. In the experiments, we set n=3,H=7n=3,H=7. For additional sharing, each agent decides whether to share each element of its private information with the other agents. The communication cost is α times the number of elements shared additionally, where α∈0.05,0.1,0.2,0.5α∈\0.05,0.1,0.2,0.5\. We set =i=i=ℝX=U_i=Y_i=R for all i∈[n]i∈[n]. The system matrices are sampled using a random seed β. Each nonzero block of the dynamics matrices A1:H,B1:H,E1:HA_1:H,B_1:H,E_1:H is sampled uniformly from [0.8,1.3][0.8,1.3], [0.5,1.2][0.5,1.2], and [0.7,1.3][0.7,1.3], respectively. The noise covariance matrices �i,hi∈[0:n],h∈[H]\ _i,h\_i∈[0:n],h∈[H] are set to be block diagonal, and each diagonal block is sampled uniformly from [0.5,2.0][0.5,2.0]. The covariance matrix �1 _1 is also set to be block diagonal, with each diagonal block sampled uniformly from [1.0,3.0][1.0,3.0]. We run experiments for all four information structures, the four values of α in 0.05,0.1,0.2,0.5\0.05,0.1,0.2,0.5\, and 10 random seeds β=1,…,10β=1,…,10. The results are presented in Figure 1, which shows that lower communication costs encourage agents to share more information, thereby reducing both the control cost and the total objective value. We also evaluate the total cost under different values of α and different communication strategies, each paired with its optimal control strategy, as presented in Figure 2. The results show that when α is small, strategies with more additional sharing can attain lower total costs. Conversely, when α is large, the communication penalty makes strategies with less additional sharing preferable. VII. Concluding Remarks In this paper, we formalized joint communication-control strategy optimization in decentralized multi-agent LQG systems. We identified structural conditions under which additional information sharing preserves partial nestedness and showed that excluding them can yield either nonlinearity or non-existence of the optimal control strategies. For each fixed open-loop communication strategy, we derived Riccati-type dynamic-programming recursions for the optimal controller, which, of independent interest, can also be applied to decentralized LQG problems with partially nested information structures and output feedback, thus advancing the results in [14]. We then further extended the approach to JCCO with closed-loop communication strategies. Our work opens several important directions for future research, including extensions to non-Gaussian noise and non-cooperative settings, as well as relaxations of the assumptions for specific system structures such as those with factorized states. Acknowledgement The authors acknowledge the valuable feedback from the anonymous reviewers of IEEE CDC 2026, and acknowledge the support from the Army Research Office grant W911NF-24-1-0085, the NSF CAREER Award 2443704, the AFOSR YIP Award FA 9550-25-1-0258, a JP Morgan Faculty Research Award, and an AI Safety Research Award from Coefficient Giving. References [1] S. Axler (2024) Linear algebra done right. 4 edition, Springer. Cited by: §B–2. [2] H. Chen, P. R. Kumar, and J. H. van Schuppen (1989) On Kalman filtering for conditionally gaussian systems with random matrices. Systems & Control Letters 13 (5), p. 397–404. External Links: Document Cited by: §C–6. [3] C. Eckart and G. Young (1936) The approximation of one matrix by another of lower rank. Psychometrika 1 (3), p. 211–218. External Links: Document Cited by: §B–2. [4] A. Ferrante and L. Ntogramatzidis (2012) The generalised discrete algebraic riccati equation arising in lq optimal control problems: part i. In 2012 IEEE 51st IEEE Conference on Decision and Control (CDC), p. 6394–6399. Cited by: §C–6. [5] J. Foerster, I. A. Assael, N. De Freitas, and S. Whiteson (2016) Learning to communicate with deep multi-agent reinforcement learning. In NeurIPS, Cited by: §I, §I. [6] M. Fu (2012) Lack of separation principle for quantized linear quadratic Gaussian control. IEEE Transactions on Automatic Control 57 (9), p. 2385–2390. Cited by: §I. [7] A. Gupta, A. Nayyar, C. Langbort, and T. Basar (2014) Common information based Markov perfect equilibria for linear-gaussian games with asymmetric information. SIAM Journal on Control and Optimization 52 (5), p. 3228–3260. Cited by: §C–3, §I, §I, §I-A, Lemma IV.3, §IV, §IV, §IV, §IV, footnote 2. [8] Y. Ho and K. Chu (1972) Team decision theory and information structures in optimal control problems – part I. IEEE Trans. Autom. Control 17, p. 15–22. Cited by: Corollary A.2, Appendix A, Appendix A, §I, §I, §I-C, §I. [9] D. Kartik, S. Sudhakara, R. Jain, and A. Nayyar (2022) Optimal communication and control strategies for a multi-agent system in the presence of an adversary. In IEEE Conf. on Dec. and Control, Cited by: §I. [10] A. Lamperski and L. Lessard (2015) Optimal decentralized state-feedback control with sparsity and delays. Automatica, p. 143–151. Cited by: §I, §I, §I-B, §I, §IV, footnote 2. [11] X. Liu, H. You, and K. Zhang (2025) Principled learning-to-communicate with quasi-classical information structures. In 2025 64th IEEE Conference on Decision and Control (CDC), Cited by: §I, §I-A, §I-A, §IV, §V. [12] X. Liu and K. Zhang (2026) Partially observable multiagent reinforcement learning with information sharing. SIAM Journal on Control and Optimization 64 (2), p. 673–697. Cited by: §I-A, §I-A, §I. [13] A. Mahajan, N. C. Martins, M. C. Rotkowitz, and S. Yüksel (2012) Information structures in optimal decentralized control. In IEEE Conf. on Dec. and Control, Cited by: §I-C. [14] A. Mahajan and A. Nayyar (2015) Sufficient statistics for linear control strategies in decentralized systems with partial history sharing. IEEE Trans. Autom. Control 60 (8), p. 2046–2056. Cited by: §C–5, §I, §I, §I, §I-B, §IV, §IV, §IV, §VII, footnote 2. [15] D. Maity and P. Tsiotras (2021) Optimal controller synthesis and dynamic quantizer switching for linear-quadratic-gaussian systems. IEEE Transactions on Automatic Control 67 (1), p. 382–389. Cited by: §I. [16] N. Matni and V. Chandrasekaran (2016) Regularization for design. IEEE Trans. Autom. Control 61 (12), p. 3991–4006. Cited by: §I. [17] N. Matni (2015) Communication delay co-design in ℋ2H_2-distributed control using atomic norm minimization. IEEE Transactions on Control of Network Systems 4 (2), p. 267–278. Cited by: §I. [18] A. Nayyar, A. Gupta, C. Langbort, and T. Başar (2013) Common information based Markov perfect equilibria for stochastic games with asymmetric information: finite games. IEEE Trans. Autom. Control 59, p. 555–570. Cited by: §IV. [19] A. Nayyar and L. Lessard (2015) Structural results for partially nested LQG systems over graphs. In 2015 American Control Conference (ACC), p. 5457–5464. Cited by: §I, §I, §IV. [20] A. Nayyar, A. Mahajan, and D. Teneketzis (2013) Decentralized stochastic control with partial history sharing: a common information approach. IEEE Trans. Autom. Control 58 (7), p. 1644–1658. Cited by: §I, §I, §I, §I, §IV, §IV, §IV. [21] C. Peng and T. C. Yang (2013) Event-triggered communication and H∞H_∞ control co-design for networked control systems. Automatica 49 (5), p. 1326–1332. Cited by: §I. [22] D. Rappaport and L. M. Silverman (1971) Structure and stability of discrete-time optimal systems. IEEE Transactions on Automatic Control 16 (3), p. 227–233. External Links: Document Cited by: §C–6. [23] S. Sudhakara, D. Kartik, R. Jain, and A. Nayyar (2024) Optimal communication and control strategies in a cooperative multiagent MDP problem. IEEE Transactions on Automatic Control 69 (10), p. 6959–6966. Cited by: §I. [24] S. Sukhbaatar, A. Szlam, and R. Fergus (2016) Learning multiagent communication with backpropagation. In NeurIPS, Cited by: §I, §I. [25] H. S. Witsenhausen (1968) A counterexample in stochastic optimum control. SIAM Journal on Control 6 (1), p. 131–147. Cited by: Proposition B.2, §I. [26] H. S. Witsenhausen (1971) On information structures, feedback and causality. SIAM Journal on Control 9 (2), p. 149–160. Cited by: §I-C. [27] S. Yüksel and T. Başar (2023) Stochastic teams, games, and control under information constraints. Springer Nature. Cited by: §I-A. [28] S. Yüksel (2013) Jointly optimal LQG quantization and control policies for multi-dimensional systems. IEEE Trans. Autom. Control 59, p. 1612–1617. Cited by: §I. [29] L. Zhang and D. Hristu-Varsakelis (2006) Communication and control co-design for networked control systems. Automatica 42 (6), p. 953–958. Cited by: §I. Appendix A Decentralized Linear-Quadratic-Gaussian Control Problem For n>1n>1 agents, a (cooperative) decentralized linear-quadratic-Gaussian (decentralized LQG) control problem ˇ D is described by a tuple ⟨H,,ii∈[n],ii∈[n],Ahh∈[H],Bi,hi∈[n],h∈[H],Ei,hi∈[n],h∈[H],Qh1h∈[H+1],Qh2h∈[H]⟩ H,X,\Y_i\_i∈[n],\U_i\_i∈[n],\A_h\_h∈[H],\B_i,h\_i∈[n],h∈[H],\E_i,h\_i∈[n],h∈[H],\Q_h^1\_h∈[H+1],\Q_h^2\_h∈[H] , where H is the time horizon and h∈=ℝdx X_h =R^d_x is the state. At each timestep h∈[H]h∈[H], each agent i∈[n]i∈[n] receives a noisy observation i,h∈i=ℝdyi Y_i,h _i=R^d_y^i of the state h X_h, and chooses a control action i,h∈i=ℝdui U_i,h _i=R^d_u^i. At timestep h∈[H]h∈[H], we denote by h=[1,h⊤2,h⊤⋯n,h⊤]⊤ U_h= bmatrix U_1,h & U_2,h &·s& U_n,h bmatrix the joint control action of all the n agents, and by =ℝ∑i=1nduiU=R _i=1^nd_u^i the joint control action space; we denote by h=[1,h⊤2,h⊤⋯n,h⊤]⊤ Y_h= bmatrix Y_1,h & Y_2,h &·s& Y_n,h bmatrix the joint observation, and by =ℝ∑i=1ndyiY=R _i=1^nd_y^i the joint observation space. The system evolves as follows: for each timestep h∈[H]h∈[H] h+1 X_h+1 =Ahh+Bhh+0,h=Ahh+∑i=1nBi,hi,h+0,h, =A_h X_h+B_h U_h+ W_0,h=A_h X_h+ _i=1^nB_i,h U_i,h+ W_0,h, (A.1) i,h Y_i,h =Ei,h+i,h,∀i∈[n], =E_i,h X_h+ W_i,h,∀ i∈[n], where i,h W_i,h is a random variable taking values in some real-vector space i,hW_i,h for all h∈[H],i=0,1,⋯,nh∈[H],i=0,1,·s,n. Ah,Bh,Ei,hA_h,B_h,E_i,h are matrices of appropriate dimensions, with Bh=[B1,hB2,h⋯Bn,h]B_h= bmatrixB_1,h&B_2,h&·s&B_n,h bmatrix. Here, i,h∼(,�i,h) W_i,h (0, _i,h) for some covariance matrices �i,h⪰0 _i,h 0 for all i∈[0:n],h∈[H]i∈[0:n],h∈[H] and 1∼(,�1) X_1 (0, _1) for some covariance matrix �1⪰0 _1 0. The random variables 1 X_1 and i,hi∈[0:n],h∈[H]\ W_i,h\_i∈[0:n],h∈[H] are mutually independent. At timestep h∈[H]h∈[H], each agent can access some information i,h⊆1:h,1:h−1 I_i,h \ Y_1:h, U_1:h-1\, and the collection of all possible such available information is denoted by ℐi,hI_i,h. We use h I_h to denote the joint available information at timestep h. Meanwhile, agents may share part of their information with each other, and the shared information is denoted by h Z_h. At timestep h, the common information among all agents is defined as the union of all the shared information so far: h=∪t=1ht C_h= _t=1^h Z_t. The private information of agent i at timestep h is thus defined as i,h=i,h P_i,h= I_i,h C_h, and the joint private information is denoted by h:=[1,h⊤⋯n,h⊤]⊤ P_h:=[ P_1,h ~·s~ P_n,h ] . We denote by h,h,i,h,ℐi,h,ℐhC_h,P_h,P_i,h,I_i,h,I_h the sets of values of the random variables h,h,i,h,i,h,h C_h, P_h, P_i,h, I_i,h, I_h for each h∈[H],i∈[n]h∈[H],i∈[n]. Each agent i at timestep h chooses the control action i,h U_i,h based on some strategy gi,h:ℐi,h→ig_i,h:I_i,h _i. We denote by gh:=(g1,h,g2,h,⋯,gn,h)g_h:=(g_1,h,g_2,h,·s,g_n,h) the joint control strategy of all the agents, and by g1:h:=(g1,g2,⋯,gh),∀h∈[H]g_1:h:=(g_1,g_2,·s,g_h),∀ h∈[H] the sequence of joint strategies from timesteps 11 to h. We use i,hG_i,h to denote the strategy space of gi,hg_i,h, and use h,1:hG_h,G_1:h to denote the corresponding joint strategy spaces. At each timestep h∈[H]h∈[H], the cost is defined as ch=h⊤Qh1h+h⊤Qh2hc_h= X_h Q_h^1 X_h+ U_h Q_h^2 U_h, and the cost of final timestep is cH+1=H+1⊤QH+11H+1c_H+1= X_H+1 Q_H+1^1 X_H+1. Here, ∀h∈[H],Qh1,Qh2⪰0,QH+11⪰0∀ h∈[H],Q_h^1,Q_h^2 0,Q_H+1^1 0 are symmetric matrices with appropriate dimensions. The objective among all the agents is thus defined as Jˇ(g1:H):=[∑h=1Hch+cH+1|g1:H]. J_ D(g_1:H):=E [ _h=1^Hc_h+c_H+1\, |\,g_1:H ]. With this objective, we define the notion of team optimality for the problem ˇ D as follows. Definition A.1 (Team optimality). We call a joint strategy g1:H∗∈1:Hg_1:H _1:H a team-optimal strategy of the decentralized LQG control problem ˇ D if ∀g1:H∈1:H,Jˇ(g1:H)≥Jˇ(g1:H∗).∀ g_1:H _1:H, J_ D(g_1:H)≥ J_ D(g_1:H ). (A.2) Corollary A.2 ([8] with positive semidefinite matrices). Every finite-horizon decentralized LQG problem above with a partially nested information structure admits a team-optimal linear strategy, even when the primitive Gaussian covariance matrices and the state and control cost matrices Qh1h∈[H+1]\Q_h^1\_h∈[H+1] and Qh2h∈[H]\Q_h^2\_h∈[H] are positive semidefinite and possibly singular. Proof. We follow Theorems 1 and 2 in [8]. Regarding each agent-time pair (i,h)(i,h) as a decision maker, partial nestedness allows the effects of preceding actions to be recursively subtracted from each information vector, exactly as in Eqs. (28) and (29) of [8]. Denote the resulting static information by ^i,h I_i,h. This transformation and its reverse preserve the actions and the value of JˇJ_ D. It therefore remains to establish linear optimality for the resulting static team. Let W stack 1 X_1 and all the process and observation noises, and let 1:H+1 X_1:H+1 and 1:H U_1:H denote the stacked state and action vectors. Recursively substituting the dynamics gives fixed matrices A and ℬB such that 1:H+1=+ℬ1:H. X_1:H+1=A W+B U_1:H. Consequently, Jˇ=[1:H⊤Q¯1:H+21:H⊤S¯]+c0,J_ D=E\! [ U_1:H Q U_1:H+2 U_1:H S W ]+c_0, where Q¯ Q :=ℬ⊤diag(Q11,…,QH+11)ℬ+diag(Q12,…,QH2)⪰0, :=B diag(Q_1^1,…,Q_H+1^1)B+ diag(Q_1^2,…,Q_H^2) 0, S¯ S :=ℬ⊤diag(Q11,…,QH+11), :=B diag(Q_1^1,…,Q_H+1^1)A, and c0c_0 is independent of the strategy. Write [⊤]=LL⊤,=L,∼(,I),E[ W W ]=L , W=L ξ, ξ ( 0,I), with L having full column rank. Since ^i,h I_i,h is linear in W, remove its linearly dependent coordinates and denote the resulting information by ¯i,h=C¯i,h. I_i,h= C_i,h ξ. The vectors ^i,h I_i,h and ¯i,h I_i,h determine one another through fixed linear maps, while every nonempty ¯i,h I_i,h has a positive-definite covariance matrix. Consider static linear strategies i,h=Ki,h¯i,h U_i,h=K_i,h I_i,h. As a function of the entries of Ki,hi,h\K_i,h\_i,h, JˇJ_ D is a finite-dimensional quadratic function with a positive-semidefinite quadratic part. It is bounded below because Jˇ≥0J_ D≥ 0. Hence, its linear part vanishes on the null space of its quadratic part, since otherwise, the objective would be unbounded below along a null direction. Its normal equations are therefore consistent, and a minimizing collection Ki,h∗i,h\K_i,h \_i,h exists. Let i,h∗:=Ki,h∗¯i,h U_i,h :=K_i,h I_i,h and define ∗:=Q¯1:H∗+S¯L. G := Q U_1:H + SL ξ. If i,h∗ G_i,h denotes the block corresponding to i,h∗ U_i,h , the normal equation for Ki,hK_i,h gives [i,h∗¯i,h⊤]=0.E[ G_i,h I_i,h ]=0. Since (i,h∗,¯i,h)( G_i,h , I_i,h) is jointly centered Gaussian, this implies [i,h∗∣¯i,h]=0E[ G_i,h I_i,h]=0. For empty effective information, the same conclusion follows from centeredness. Now let g1:Hg_1:H be any admissible static strategy and set �i,h:=gi,h(¯i,h)−i,h∗. U_i,h:=g_i,h( I_i,h)- U_i,h . Expanding the quadratic objective and conditioning on ¯i,h I_i,h gives Jˇ(g1:H)−Jˇ(g1:H∗)=2∑h=1H∑i=1n[�i,h⊤i,h∗]+[�1:H⊤Q¯�1:H]=[�1:H⊤Q¯�1:H]≥0. J_ D(g_1:H)-J_ D(g_1:H )=2 _h=1^H _i=1^nE[ U_i,h G_i,h ]+E[ U_1:H Q U_1:H]=E[ U_1:H Q U_1:H]≥ 0. Thus, the linear strategy is team-optimal for the static problem. Finally, reverse the reduction in [8] over the precedence groups. For the first group, the static and original information vectors coincide, so the corresponding actions are linear in the original information. Suppose the actions in all preceding groups have been reconstructed as linear functions of their original information. Whenever a preceding action appears in the subtraction defining the current static information, partial nestedness makes the information determining that action available to the current decision maker. Hence the preceding action, and therefore the current static information, is linear in the current original information. The current static linear action is consequently linear in the original information. Induction gives a linear dynamic strategy. The forward and reverse reductions generate the same actions and costs for every realization, so the reconstructed strategy is team-optimal for the original dynamic problem. ∎ Appendix B Deferred Details of §I We start with the following auxiliary lemma. Lemma B.1. For any n1,n2∈ℕn_1,n_2 , let Q1∈ℝn1×n1Q_1 ^n_1× n_1 and Q2∈ℝn2×n2Q_2 ^n_2× n_2 be symmetric positive-semidefinite matrices, and let A,BA,B be matrices of compatible dimensions. Then, ker(Q1+B⊤Q2B)⊆ker(A⊤Q2B) (Q_1+B Q_2B) (A Q_2B). Proof. For any v∈ker(Q1+B⊤Q2B)v∈ (Q_1+B Q_2B), we have v⊤Q1v+v⊤B⊤Q2Bv=0v Q_1v+v B Q_2Bv=0. Since Q1⪰0,Q2⪰0Q_1 0,Q_2 0, we know v⊤Q1v=v⊤B⊤Q2Bv=0v Q_1v=v B Q_2Bv=0. Then, from v⊤B⊤Q2Bv=0v B Q_2Bv=0, we have Q212Bv=0Q_2 12Bv=0 and thus Q2Bv=0Q_2Bv=0, which further implies that A⊤Q2Bv=0A Q_2Bv=0 and thus v∈ker(A⊤Q2B)v∈ (A Q_2B). This completes the proof. ∎ B–1 Proof of Lemma I.1 Proof. To prove this lemma, we leverage the following proposition. Proposition B.2 (Adapted from Theorems 1 and 2 of [25]). There exists a decentralized LQG control problem with n=2,H=2n=2,H=2 and parameters k and σ0 _0 such that no linear control strategy is team-optimal: All random variables are scalar, 1∼(0,σ02),2=1+1,1,3=2−2,2, random variables are scalar, X_1 (0, _0^2), X_2= X_1+ U_1,1, X_3= X_2- U_2,2, 1,1=1,2,1=0,1,2=0,2,2=2+2,2,2,2∼(0,1), Y_1,1= X_1, Y_2,1=0, Y_1,2=0, Y_2,2= X_2+ W_2,2, W_2,2 (0,1), c1=k21,12,c2=0,c3=32,no information sharing between agents. c_1=k^2 U_1,1^2,c_2=0,c_3= X_3^2,~~no information sharing between agents. Then, using the parameters k and σ0 _0 specified in Proposition B.2, we construct a JCCO problem D with n=2,H=2n=2,H=2, with the system dynamics being defined as 1∼(0,σ02),2=1+1,1,3=2−2,2, X_1 (0, _0^2), X_2= X_1+ U_1,1, X_3= X_2- U_2,2, 1,1=1,2,1=0,1,2=0,2,2=2+2,2,2,2∼(0,1). Y_1,1= X_1, Y_2,1=0, Y_1,2=0, Y_2,2= X_2+ W_2,2, W_2,2 (0,1). For the communication part, suppose the baseline sharing is null and ℳi,1=0,1,ℳ1,2=0,13,ℳ2,2=0,12M_i,1=\0,1\,M_1,2=\0,1\^3,M_2,2=\0,1\^2, such that i,1 M_i,1 represents whether agent i shares i,1 Y_i,1, each digit of 1,2 M_1,2 represents whether agent 11 shares 1,1,1,1,1,2 Y_1,1, U_1,1, Y_1,2, and each digit of 2,2 M_2,2 represents whether agent 22 shares 2,1,2,2 Y_2,1, Y_2,2. For control costs, we set c1=k21,12,c2=0,c3=32c_1=k^2 U_1,1^2,c_2=0,c_3= X_3^2, where k is specified as in Proposition B.2; for communication costs, we set κh=(σ02+1)[ha,∅],∀h∈[2] _h=( _0^2+1) 1[ Z_h^a≠ ],∀ h∈[2]. The information available to each agent at each timestep is defined as: ∀i∈[2],i,1−=i,1,i,1+=i,1−∪1a∀ i∈[2], I_i,1^-=\ Y_i,1\, I_i,1^+= I_i,1^-∪ Z_1^a, and 1,2−=1,1+∪1,1,1,2,2,2−=2,1+∪2,2,i,2+=i,2−∪2a I_1,2^-= I_1,1^+∪\ U_1,1, Y_1,2\, I_2,2^-= I_2,1^+∪\ Y_2,2\, I_i,2^+= I_i,2^-∪ Z_2^a. Then, we can verify that: • D satisfies Assumption I.1. • D satisfies Assumption I.2: The only zero input coefficient relevant to a later decision is the scalar B2,1=0B_2,1=0, and 2,1 U_2,1 does not appear in any later information set. • D satisfies Assumption I.4: It holds that rank(E1,2B2,1)=rank(B2,1)=0,rank(E2,2B1,1)=rank(B1,1)=1rank(E_1,2B_2,1)=rank(B_2,1)=0,rank(E_2,2B_1,1)=rank(B_1,1)=1. • D does not have PN IS: If there is no additional sharing, agent (1,1)(1,1) influences agent (2,2)(2,2), but 1,1−⊈2,2− I_1,1^- I_2,2^-. For any communication strategy g1:2mg_1:2^m, if it chooses any i,h M_i,h with any digit non-zero (namely, sharing some information via additional sharing), then it cannot achieve the optimum, since it will suffer from a communication cost of κh≥σ02+1 _h≥ _0^2+1, which is larger than the total expected cost when the agents choose 1,1=2,2=0 U_1,1= U_2,2=0, and they do not have any additional sharing. Therefore, the optimal communication strategy g1:2m,∗g_1:2^m, must yield no additional sharing. The remaining control subproblem is exactly Proposition B.2; hence every team-optimal control strategy is nonlinear. This completes the proof. ∎ B–2 Proof of Lemma I.3 Proof. This proof consists of two Parts. Part 1: we construct a JCCO problem 1D_1 whose optimal control strategy is nonlinear; Part 2: we construct another JCCO problem 2D_2 whose team-optimal strategy does not exist. Part 1: Consider an H=2,n=2H=2,n=2 JCCO problem 1D_1 with the system dynamics being defined as ∀h∈[2],h∈ℝ2,1,1,1,2∈ℝ,2,1,2,2∈ℝ2,2=1,3=2−2,2, ∀ h∈[2], X_h ^2, U_1,1, U_1,2 , U_2,1, U_2,2 ^2, X_2= X_1, X_3= X_2- U_2,2, 1∼(,),1,1=1,1,2=2,1=2,2=0,c1=c2=0,c3=3⊤3. X_1 (0,I), Y_1,1= X_1, Y_1,2= Y_2,1= Y_2,2=0,c_1=c_2=0,c_3= X_3 X_3. For the communication part, suppose the baseline sharing is null and ℳi,1=0,1,ℳi,2=0,13,∀i∈[2]M_i,1=\0,1\,M_i,2=\0,1\^3,∀ i∈[2], where i,1 M_i,1 represents whether agent i shares i,1 Y_i,1 and each digit of i,2 M_i,2 represents whether agent i shares i,1,i,1 Y_i,1, U_i,1, and i,2 Y_i,2, respectively. Set κ1=[1,1=1 or 2,1=1] _1= 1[ M_1,1=1 or M_2,1=1] and κ2=[1,2<(0,0,0),(0,1,0) or 2,2,(0,0,0)] _2= 1[ M_1,2∉\(0,0,0),(0,1,0)\ or M_2,2≠(0,0,0)]. Thus (0,1,0)(0,1,0) shares only 1,1 U_1,1 at zero cost, while every other nonempty sharing incurs cost one. The information available to each agent at each timestep is defined as ∀i∈[2],i,1−=i,1,i,1+=i,1−∪1a∀ i∈[2], I_i,1^-=\ Y_i,1\, I_i,1^+= I_i,1^-∪ Z_1^a, and 1,2−=1,1+∪1,1,1,2,2,2−=2,1+∪2,1,2,2,i,2+=i,2−∪2a I_1,2^-= I_1,1^+∪\ U_1,1, Y_1,2\, I_2,2^-= I_2,1^+∪\ U_2,1, Y_2,2\, I_i,2^+= I_i,2^-∪ Z_2^a. Then, we can verify that: • 1D_1 satisfies Assumption I.1. • 1D_1 has PN IS: If there is no additional sharing, then for any i1,i2∈[2]i_1,i_2∈[2], i1,1−⊆i2,2− I_i_1,1^- I_i_2,2^- if i1=i2i_1=i_2, and otherwise agent (i1,1)(i_1,1) does not influence agent (i2,2)(i_2,2). • 1D_1 satisfies Assumption I.4: For any i∈[2]i∈[2], rank(E−i,2Bi,1)=rank(Bi,1)=0rank(E_-i,2B_i,1)=rank(B_i,1)=0. • 1D_1 does not satisfy Assumption I.2: B1,1=B2,1=0B_1,1=B_2,1=0, but 1,1∈1,2− U_1,1∈ I_1,2^- and 2,1∈2,2− U_2,1∈ I_2,2^-. Now, we aim to show that the optimal control strategy is nonlinear. Firstly, we can construct a strategy (g1:2m,∗,g1:2a,∗)(g_1:2^m, ,g_1:2^a, ) as g1,1m,∗=g2,1m,∗=0,g1,2m,∗=(0,1,0),g2,2m,∗=(0,0,0),g1,1a,∗(1,1+)=φbi(1,1),g2,2a,∗(2,2+)=φbi−1(1,1), g_1,1^m, =g_2,1^m, =0,g_1,2^m, =(0,1,0),g_2,2^m, =(0,0,0),g_1,1^a, ( I_1,1^+)= _bi( Y_1,1),g_2,2^a, ( I_2,2^+)= _bi^-1( U_1,1), where φbi _bi is a Borel isomorphism that pushes (0,I2)N(0,I_2) to (0,1)N(0,1) (such an isomorphism exists between atomless standard probability spaces), and the other two controls are zero. Hence the strategy is square-integrable and has total cost zero almost surely. Since every cost is nonnegative, it is team-optimal. Such a Borel bijection cannot be linear: every linear :ℝ2→ℝT:R^2 has nontrivial kernel by the rank-nullity theorem [1]. However, we will show that for any strategy (g1:2m,g1:2a)(g_1:2^m,g_1:2^a) such that g1:2ag_1:2^a is linear, (g1:2m,g1:2a)(g_1:2^m,g_1:2^a) cannot be team-optimal, i.e., J1(g1:2m,g1:2a)>0J_D_1(g_1:2^m,g_1:2^a)>0. We prove it by contradiction, and suppose that (g1:2m,′,g1:2a,′)(g_1:2^m, ,g_1:2^a, ) satisfies that J1(g1:2m,′,g1:2a,′)=0J_D_1(g_1:2^m, ,g_1:2^a, )=0 and g1:2a,′g_1:2^a, is linear. Then, it must hold that g1,1m,′=g2,1m,′=0,g2,2m,′=(0,0,0),g1,2m,′=(0,0,0)g_1,1^m, =g_2,1^m, =0,g_2,2^m, =(0,0,0),g_1,2^m, =(0,0,0) or (0,1,0)(0,1,0), since otherwise the communication cost is at least 1. If g1,2m,′=(0,0,0)g_1,2^m, =(0,0,0), changing it to (0,1,0)(0,1,0) preserves the original controls as feasible prescriptions, adds no communication cost, and therefore preserves the zero value. We may thus assume g1,2m,′=(0,1,0)g_1,2^m, =(0,1,0). Then i,1+=i,1 I_i,1^+=\ Y_i,1\ for i∈[2]i∈[2] and 2,2+=2,1,2,1,2,2,1,1 I_2,2^+=\ Y_2,1, U_2,1, Y_2,2, U_1,1\. Since g1:2a,′g_1:2^a, is linear, we can write: 1,1=G1,11,1,y1,1,2,1=G2,12,1,y2,1,2,2=G2,22,1,y2,1+G2,22,2,y2,2+G2,21,1,u1,1+G2,22,1,u2,1, U_1,1=G_1,1^1,1,y Y_1,1, U_2,1=G_2,1^2,1,y Y_2,1, U_2,2=G_2,2^2,1,y Y_2,1+G_2,2^2,2,y Y_2,2+G_2,2^1,1,u U_1,1+G_2,2^2,1,u U_2,1, for some matrices G1,11,1,y,G2,12,1,y,G2,22,1,y,G2,22,2,y,G2,21,1,u,G2,22,1,uG_1,1^1,1,y,G_2,1^2,1,y,G_2,2^2,1,y,G_2,2^2,2,y,G_2,2^1,1,u,G_2,2^2,1,u with proper dimensions. Then, we can write 2,2 U_2,2 as 2,2=G2,21,1,uG1,11,1,y1,1 U_2,2=G_2,2^1,1,uG_1,1^1,1,y Y_1,1. Then, we can write J1(g1:2m,′,g1:2a,′)=[((−G2,21,1,uG1,11,1,y)1)⊤(−G2,21,1,uG1,11,1,y)1)]=Tr((−G2,21,1,uG1,11,1,y)⊤(−G2,21,1,uG1,11,1,y))J_D_1(g_1:2^m, ,g_1:2^a, )=E[((I-G_2,2^1,1,uG_1,1^1,1,y) X_1) (I-G_2,2^1,1,uG_1,1^1,1,y) X_1)]= Tr((I-G_2,2^1,1,uG_1,1^1,1,y) (I-G_2,2^1,1,uG_1,1^1,1,y)). Note that G2,21,1,uG_2,2^1,1,u is a 2×12× 1 matrix and G1,11,1,yG_1,1^1,1,y is a 1×21× 2 matrix. Therefore, we have rank(G2,21,1,uG1,11,1,yG_2,2^1,1,uG_1,1^1,1,y)≤1≤ 1, and then Tr((−G2,21,1,uG1,11,1,y)⊤(−G2,21,1,uG1,11,1,y))≥1 Tr((I-G_2,2^1,1,uG_1,1^1,1,y) (I-G_2,2^1,1,uG_1,1^1,1,y))≥ 1 due to Eckart-Young Theorem [3]. This means that J1(g1:2m,′,g1:2a,′)>0J_D_1(g_1:2^m, ,g_1:2^a, )>0 and leads to the contradiction. Hence, we know that there exists a team-optimal strategy of 1D_1 and the control strategy of any team-optimal strategy is nonlinear, which completes the proof of Part 1. Part 2: Consider an n=2,H=2n=2,H=2 JCCO problem 2D_2 with the system dynamics being defined as ∀h∈[2],i∈[2],h,3,i,h,i,h∈ℝ,2=1,3=2−2,2, ∀ h∈[2],i∈[2], X_h, X_3, Y_i,h, U_i,h , X_2= X_1, X_3= X_2- U_2,2, 1,1=1,1,2=2,1=2,2=0,1∼(0,14),c1=1,12,c2=0,c3=32. Y_1,1= X_1, Y_1,2= Y_2,1= Y_2,2=0, X_1 (0, 14),c_1= U_1,1^2,c_2=0,c_3= X_3^2. For the communication part, similarly to Part 1, suppose the baseline sharing is null and ℳi,1=0,1,ℳi,2=0,13,∀i∈[2]M_i,1=\0,1\,M_i,2=\0,1\^3,∀ i∈[2], where i,1 M_i,1 represents whether agent i shares i,1 Y_i,1 and each digit of i,2 M_i,2 represents whether agent i shares i,1,i,1 Y_i,1, U_i,1, and i,2 Y_i,2, respectively. We set the communication costs as κ1=[1a,∅],κ2=[2a\1,1,∅] _1= 1[ Z_1^a≠ ], _2= 1[ Z_2^a \ U_1,1\≠ ]. This communication cost means that if any information except 1,1 U_1,1 is shared, then all agents will incur a communication cost of 11. The information available to each agent at each timestep is defined as ∀i∈[2],i,1−=i,1,i,1+=i,1−∪1a,i,2−=i,1+∪i,1,i,2,i,2+=i,2−∪2a∀ i∈[2], I_i,1^-=\ Y_i,1\, I_i,1^+= I_i,1^-∪ Z_1^a, I_i,2^-= I_i,1^+∪\ U_i,1, Y_i,2\, I_i,2^+= I_i,2^-∪ Z_2^a. Then, we can verify that: • 2D_2 satisfies Assumption I.1. • 2D_2 has PN IS: If there is no additional sharing, then for any i1,i2∈[2]i_1,i_2∈[2], i1,1−⊆i2,2− I_i_1,1^- I_i_2,2^- if i1=i2i_1=i_2, and otherwise agent (i1,1)(i_1,1) does not influence agent (i2,2)(i_2,2). • 2D_2 satisfies Assumption I.4: For any i∈[2]i∈[2], rank(E−i,2Bi,1)=rank(Bi,1)=0rank(E_-i,2B_i,1)=rank(B_i,1)=0. • 2D_2 does not satisfy Assumption I.2: B1,1=B2,1=0B_1,1=B_2,1=0, but 1,1∈1,2− U_1,1∈ I_1,2^- and 2,1∈2,2− U_2,1∈ I_2,2^-. We prove it by contradiction. We assume (g1:2m,∗,g1:2a,∗)(g_1:2^m, ,g_1:2^a, ) is a team-optimal strategy of 2D_2. Firstly, it holds that g1,1m,∗=g2,1m,∗=0,g2,2m,∗=(0,0,0),g1,2m,∗=(0,0,0)g_1,1^m, =g_2,1^m, =0,g_2,2^m, =(0,0,0),g_1,2^m, =(0,0,0) or (0,1,0)(0,1,0), since otherwise J2(g1:2m,∗,g1:2a,∗)≥[κ1+κ2|g1:2m,∗,g1:2a,∗]≥1J_D_2(g_1:2^m, ,g_1:2^a, ) [ _1+ _2\,|\,g_1:2^m, ,g_1:2^a, ]≥ 1; however, if we consider strategy (g1:2m,g1:2a)(g_1:2^m,g_1:2^a) that shares nothing and chooses i,h=0,∀i∈[2],h∈[2] U_i,h=0,∀ i∈[2],h∈[2], then J2(g1:2m,g1:2a)=[32|g1:2m,g1:2a]=[12]=14J_D_2(g_1:2^m,g_1:2^a)=E[ X_3^2\,|\,g_1:2^m,g_1:2^a]=E[ X_1^2]= 14. Secondly, we can assume g1,2m,∗=(0,1,0)g_1,2^m, =(0,1,0), otherwise we can change it to be (0,1,0)(0,1,0) and it is still a team-optimal strategy, since additionally sharing 1,1 U_1,1 enlarges the 2,2+ I_2,2^+ but incurs no communication cost. Therefore, we know that under g1:2m,∗g_1:2^m, , we have i,1+=i,1,∀i∈[2],2,2+=2,1,2,1,2,2,1,1 I_i,1^+=\ Y_i,1\,∀ i∈[2], I_2,2^+=\ Y_2,1, U_2,1, Y_2,2, U_1,1\. Thirdly, if ℙ(1,1=0|g1:2m,∗,g1:2a,∗)=1P( U_1,1=0\,|\,g_1:2^m, ,g_1:2^a, )=1, then we consider the realization I2,2+=2,1=0,2,1=g2,1a,∗(0),2,2=0,1,1=0I_2,2^+=\ Y_2,1=0, U_2,1=g_2,1^a, (0), Y_2,2=0, U_1,1=0\, and let U2,2=g2,2a,∗(I2,2+)U_2,2=g_2,2^a, (I_2,2^+) be the realization of 2,2 U_2,2 when g2,2a,∗g_2,2^a, takes the realization I2,2+I_2,2^+ as input. Then, it holds that ℙ(2,2=U2,2|g1:2m,∗,g1:2a,∗)=1P( U_2,2=U_2,2\,|\,g_1:2^m, ,g_1:2^a, )=1, and we have J2(g1:2m,∗,g1:2a,∗)≥[32|g1:2m,∗,g1:2a,∗]=[(1−2,2)2|g1:2m,∗,g1:2a,∗]=[(1−U2,2)2|g1:2m,∗,g1:2a,∗]≥[12]≥14. J_D_2(g_1:2^m, ,g_1:2^a, ) [ X_3^2\,|\,g_1:2^m, ,g_1:2^a, ]=E[( X_1- U_2,2)^2\,|\,g_1:2^m, ,g_1:2^a, ]=E[( X_1-U_2,2)^2\,|\,g_1:2^m, ,g_1:2^a, ] [ X_1^2]≥ 14. However, we consider the strategy (g1:2m,∗,g1:2a,′)(g_1:2^m, ,g_1:2^a, ) with g1:2a,′g_1:2^a, being defined as g1,1a,′(1,1+)=1,12,g2,2a,′(2,2+)=21,1g_1,1^a, ( I_1,1^+)= Y_1,12,g_2,2^a, ( I_2,2^+)=2 U_1,1 if 1,1∈2,2+ U_1,1∈ I_2,2^+ and otherwise 00, and choose arbitrary g2,1a,′,g1,2a,′g_2,1^a, ,g_1,2^a, . Then, we can verify that J2(g1:2m,∗,g1:2a,′)=[c1+c3|g1:2m,∗,g1:2a,′]=[1,12+32|g1:2m,∗,g1:2a,′]=[141,12+(1−2⋅1,12)2]=[141,12]=116. J_D_2(g_1:2^m, ,g_1:2^a, )=E[c_1+c_3\,|\,g_1:2^m, ,g_1:2^a, ]=E[ U_1,1^2+ X_3^2\,|\,g_1:2^m, ,g_1:2^a, ]=E[ 14 Y_1,1^2+( X_1-2· Y_1,12)^2]=E[ 14 Y_1,1^2]= 116. Therefore, we know that it holds that ℙ(1,1=0|g1:2m,∗,g1:2a,∗)<1P( U_1,1=0\,|\,g_1:2^m, ,g_1:2^a, )<1, and then [1,12|g1:2m,∗,g1:2a,∗]>0E[ U_1,1^2\,|\,g_1:2^m, ,g_1:2^a, ]>0. Lastly, we can construct a new strategy g1:2ag_1:2^a based on g1:2a,∗g_1:2^a, as g1,1a(1,1+)=12g1,1a,∗(1,1+), g_1,1^a( I_1,1^+)= 12g_1,1^a, ( I_1,1^+), g2,2a(2,2+)=g2,2a(2,1,2,1,2,2,1,1)=g2,2a,∗(2,1,2,1,2,2,21,1) if 2,2+=2,1,2,1,2,2,1,1, otherwise 0. g_2,2^a( I_2,2^+)=g_2,2^a( Y_2,1, U_2,1, Y_2,2, U_1,1)=g_2,2^a, ( Y_2,1, U_2,1, Y_2,2,2 U_1,1) if $ I_2,2^+=\ Y_2,1, U_2,1, Y_2,2, U_1,1\$, otherwise 0. All other control rules remain unchanged. This construction means that g1,1ag_1,1^a will choose 1,1 U_1,1 to be half of the 1,1 U_1,1 chosen by g1,1a,∗g_1,1^a, , but g2,2a,∗,g2,2ag_2,2^a, ,g_2,2^a choose the same 2,2 U_2,2. Therefore, we can verify that J2(g1:2m,∗,g1:2a) J_D_2(g_1:2^m, ,g_1:2^a) =[1,12|g1:2m,∗,g1:2a]+[(1−2,2)2|g1:2m,∗,g1:2a] =E[ U_1,1^2\,|\,g_1:2^m, ,g_1:2^a]+E[( X_1- U_2,2)^2\,|\,g_1:2^m, ,g_1:2^a] =[(1,1/2)2|g1:2m,∗,g1:2a,∗]+[(1−2,2)2|g1:2m,∗,g1:2a,∗] =E[( U_1,1/2)^2\,|\,g_1:2^m, ,g_1:2^a, ]+E[( X_1- U_2,2)^2\,|\,g_1:2^m, ,g_1:2^a, ] <[1,12|g1:2m,∗,g1:2a,∗]+[(1−2,2)2|g1:2m,∗,g1:2a,∗]=J2(g1:2m,∗,g1:2a,∗), <E[ U_1,1^2\,|\,g_1:2^m, ,g_1:2^a, ]+E[( X_1- U_2,2)^2\,|\,g_1:2^m, ,g_1:2^a, ]=J_D_2(g_1:2^m, ,g_1:2^a, ), where the inequality holds because [1,12|g1:2m,∗,g1:2a,∗]>0E[ U_1,1^2\,|\,g_1:2^m, ,g_1:2^a, ]>0 as proved before. This means that for any team-optimal strategy g1:2m,∗,g1:2a,∗g_1:2^m, ,g_1:2^a, , we can construct a strategy (g1:2m,∗,g1:2a)(g_1:2^m, ,g_1:2^a) that is strictly better than it. Therefore, the team-optimal strategy does not exist. ∎ B–3 Proof of Lemma I.5 Proof. Consider a JCCO problem D with n=2,H=2n=2,H=2, parameters k,σ0k, _0 specified in Proposition B.2, and the system dynamics being defined as ∀h∈[2],i∈[2],h,3,i,h,i,h∈ℝ,2=1+1,1,3=2−2,2, ∀ h∈[2],i∈[2], X_h, X_3, Y_i,h, U_i,h , X_2= X_1+ U_1,1, X_3= X_2- U_2,2, 1,1=1,1,2=2+1,2,2,1=2,2=0,1∼(0,σ02),1,2∼(0,1). Y_1,1= X_1, Y_1,2= X_2+ W_1,2, Y_2,1= Y_2,2=0, X_1 (0, _0^2), W_1,2 (0,1). For the communication part, baseline sharing is null and ℳi,1=0,1,ℳ1,2=0,13,ℳ2,2=0,12, _i,1=\0,1\,M_1,2=\0,1\^3,M_2,2=\0,1\^2, where i,1 M_i,1 represents whether agent i shares i,1 Y_i,1. Each digit of 1,2 M_1,2 represents whether agent 11 shares 1,1,1,1 Y_1,1, U_1,1, and 1,2 Y_1,2. Each digit of 2,2 M_2,2 represents whether agent 22 shares 2,1 Y_2,1 and 2,2 Y_2,2. For the control costs, we set c1=k21,12,c2=0,c3=32c_1=k^2 U_1,1^2,c_2=0,c_3= X_3^2; for the communication costs, we set κ1=(σ02+1)[1a,∅],κ2=(σ02+1)[2a\1,2,∅] _1=( _0^2+1) 1[ Z_1^a≠ ], _2=( _0^2+1) 1[ Z_2^a \ Y_1,2\≠ ]. This communication cost means that if any information except 1,2 Y_1,2 is shared through additional sharing, it will incur a communication cost of σ02+1 _0^2+1. The information available to each agent at each timestep is defined as: ∀i∈[2],i,1−=i,1,i,1+=i,1−∪1a,∀ i∈[2], I_i,1^-=\ Y_i,1\, I_i,1^+= I_i,1^-∪ Z_1^a, and 1,2−=1,1+∪1,1,1,2,2,2−=2,1+∪2,2,i,2+=i,2−∪2a I_1,2^-= I_1,1^+∪\ U_1,1, Y_1,2\, I_2,2^-= I_2,1^+∪\ Y_2,2\, I_i,2^+= I_i,2^-∪ Z_2^a. Then, we can verify that: • D satisfies Assumption I.1. • D has PN IS: If there is no additional sharing, then for any i1,i2∈[2]i_1,i_2∈[2], i1,1−⊆i2,2− I_i_1,1^- I_i_2,2^- if i1=i2i_1=i_2; otherwise, agent (i1,1)(i_1,1) does not influence agent (i2,2)(i_2,2). • D satisfies Assumption I.2: The only zero input coefficient relevant to a later decision is B2,1=0B_2,1=0, and 2,1 U_2,1 does not appear in any later information set. • D does not satisfy Assumption I.4: It holds that rank(E2,2B1,1)=0,1=rank(B1,1)rank(E_2,2B_1,1)=0≠ 1=rank(B_1,1). First, no team-optimal communication strategy shares any information other than 1,2 Y_1,2 through additional sharing, since otherwise it will incur a communication cost κh≥σ02+1 _h≥ _0^2+1, which is larger than the total cost when each agent i chooses i,h=0,∀h∈[2] U_i,h=0,∀ h∈[2] and shares nothing via additional sharing. Second, we analyze the optimal control strategy under two types of communication strategies. 1) Suppose the optimal communication strategy g1:2m,∗g_1:2^m, yields that agents share 1,2 Y_1,2 through additional sharing, i.e. 1a=∅,2a=1,2 Z_1^a= , Z_2^a=\ Y_1,2\. Then, finding a team-optimal strategy of D can be reduced to finding an optimal strategy of the Decentralized LQG problem in Proposition B.2, where the optimal control strategy is nonlinear. 2) Suppose the optimal communication strategy g1:2m,∗g_1:2^m, yields that agents share nothing through additional sharing, i.e., 1a=2a=∅ Z_1^a= Z_2^a= . Then, let g1:2a,∗g_1:2^a, be the optimal control strategy, and let g1:2m,′g_1:2^m, be the communication strategy that additionally shares 1,2 Y_1,2. Since both g1:2m,′g_1:2^m, and g1:2m,∗g_1:2^m, lead to no communication cost and g1:2m,′g_1:2^m, enlarges 2,2+ I_2,2^+, it follows that J(g1:2m,∗,g1:2a,∗)≥J(g1:2m,′,g1:2a,∗)J_D(g_1:2^m, ,g_1:2^a, )≥ J_D(g_1:2^m, ,g_1:2^a, ), which means (g1:2m,′,g1:2a,∗)(g_1:2^m, ,g_1:2^a, ) is an optimal strategy. Therefore, from case 1) we know that g1:2a,∗g_1:2^a, is nonlinear, which completes the proof. ∎ B–4 Proof of Theorem I.6 Proof. Fix any communication strategy g1:Hm∈1:Hmg_1:H^m _1:H^m and let M1:HM_1:H be the communication actions chosen by g1:Hm∈1:Hmg_1:H^m _1:H^m. Then, D becomes a decentralized LQG problem denoted by (g1:Hm)D(g_1:H^m), with the same system dynamics as D and the following information evolution: for any h∈[H]h∈[H] (a) The common information in (g1:Hm)D(g_1:H^m) evolves as ¯h=∪t=1h¯t C_h= _t=1^h Z_t, and ¯h=χ¯h(¯h−1,¯h−1,¯h) Z_h= χ_h( P_h-1, U_h-1, Y_h) for some fixed projection function χ¯h χ_h. Here, χ¯h χ_h is defined as χ¯h(¯h−1,¯h−1,¯h)=χh(¯h−1,¯h−1,¯h)∪ϕh(Mh,ζh(¯h−1,¯h−1,¯h)). χ_h( P_h-1, U_h-1, Y_h)= _h( P_h-1, U_h-1, Y_h)∪ _h(M_h, _h( P_h-1, U_h-1, Y_h)). Recall that χh,ζh,ϕh _h, _h, _h are defined in Assumption I.1. Such a χ¯h χ_h is a fixed projection function since χh,ϕi,h(Mi,h,⋅)i∈[n] _h,\ _i,h(M_i,h,·)\_i∈[n] are fixed projection functions, and for any Ph−∈h−,ϕh(Mh,Ph−)=∪i=1nϕi,h(Mi,h,Pi,h−)P_h^- _h^-, _h(M_h,P_h^-)= _i=1^n _i,h(M_i,h,P_i,h^-). (b) The private information in (g1:Hm)D(g_1:H^m) evolves as follows. For any i∈[n],¯i,h=ζ¯i,h(¯i,h−1,¯i,h−1,¯i,h)i∈[n], P_i,h= ζ_i,h( P_i,h-1, U_i,h-1, Y_i,h) for some fixed projection function ζ¯i,h ζ_i,h and thus ¯h=ζ¯h(¯h−1,¯h−1,¯h) P_h= ζ_h( P_h-1, U_h-1, Y_h) for some fixed projection function ζ¯h ζ_h. Here, ζ¯i,h ζ_i,h is defined as ζ¯i,h(¯i,h−1,¯i,h−1,¯i,h)=ζi,h(¯i,h−1,¯i,h−1,¯i,h)\ϕi,h(Mi,h,ζi,h(¯i,h−1,¯i,h−1,¯i,h)). ζ_i,h( P_i,h-1, U_i,h-1, Y_i,h)= _i,h( P_i,h-1, U_i,h-1, Y_i,h) _i,h(M_i,h, _i,h( P_i,h-1, U_i,h-1, Y_i,h)). Recall that ζi,h,ϕi,h _i,h, _i,h are defined in Assumption I.1. Such a ζ¯i,h ζ_i,h is a fixed projection function since ζi,h,ϕi,h(Mi,h,⋅) _i,h, _i,h(M_i,h,·) are fixed projection functions. (c) For each agent i∈[n]i∈[n], ¯i,h∈¯i,h Y_i,h∈ I_i,h for h∈[H]h∈[H], and ¯i,h⊆¯i,h+1 I_i,h I_i,h+1 for h∈[H−1]h∈[H-1]. These properties still hold since additional sharing only moves some private information into common information but does not remove any information. Then, we know that (g1:Hm)D(g_1:H^m) is a decentralized LQG problem with some information sharing due to both the baseline sharing and the communication actions M1:HM_1:H. It remains to prove partial nestedness. Use ˇ~ ~ for the baseline problem ˇ D. Suppose that ¯i1,h1 U_i_1,h_1 influences ¯i2,h2 I_i_2,h_2, where h1<h2h_1<h_2. If Bi1,h1=0B_i_1,h_1=0, the action has no state effect, and Assumption I.2 excludes its later appearance as a shared label, a contradiction. Hence Bi1,h1,0B_i_1,h_1≠ 0. Assumption I.4 gives some j,i1j≠ i_1 with Ej,h1+1Bi1,h1,0E_j,h_1+1B_i_1,h_1≠ 0, so ˇi1,h1 U_i_1,h_1 influences ˇj,h1+1∈ˇj,h1+1 Y_j,h_1+1∈ I_j,h_1+1. Baseline partial nestedness yields ˇi1,h1⊆ˇj,h1+1 I_i_1,h_1 I_j,h_1+1. Since j,i1j≠ i_1, every label private to agent i1i_1 can enter agent j’s information only through common information due to Assumption I.1, then it holds ˇi1,h1⊆ˇh1+1 I_i_1,h_1 C_h_1+1. Moreover, every label in ¯i1,h1∖ˇi1,h1 I_i_1,h_1 I_i_1,h_1 arrived through additional sharing and lies in ¯h1 C_h_1, and then it holds ¯i1,h1⊆¯h1+1⊆¯h2⊆¯i2,h2. I_i_1,h_1 C_h_1+1 C_h_2 I_i_2,h_2. Thus (g1:Hm)D(g_1:H^m) is partially nested. Corollary A.2 supplies a linear team optimum for every fixed schedule. Since the open-loop schedule set is finite, a minimizing schedule exists, and its associated linear control strategy proves the final assertion. ∎ Appendix C Deferred Details of §IV For the appendix derivations only, we use the following cost-free terminal convention after the last decision: ~H+1:=~H∪~H,~H+1,~i,H+1:=∅. C_H+1:= C_H∪\ U_H, X_H+1\, P_i,H+1:= . Equivalently, set ~H+1:=~H+1 Y_H+1:= X_H+1, E~H+1:=I E_H+1:=I, ~1:n,H+1:= W_1:n,H+1:= 0, and ~H+1:=(~H,~H+1) Z_H+1:=( U_H, Y_H+1). Then ~H+1=�~H+1=~H+1 S_H+1= _H+1= X_H+1, �~H+1=0 _H+1=0, and x,H+1=II_x,H+1=I. This convention is only a device for writing the terminal-cost calculation in the same form as the preceding backward steps; it introduces no additional decision and changes neither the strategy space nor the cost. C–1 Proof of Lemma IV.1 Proof. From the construction, the system dynamics and control cost are the same for both decentralized LQG problems, and it holds that ¯i,h⊆~i,h I_i,h I_i,h for any i∈[n],h∈[H]i∈[n],h∈[H]. Therefore, the agents in ~(g1:Hm) D(g_1:H^m) have larger strategy spaces than those in (g1:Hm)D(g_1:H^m), which implies ming~1:H∈~1:HJ~(g1:Hm)(g~1:H)≤ming¯1:H∈¯1:HJ(g1:Hm)(g¯1:H) _ g_1:H∈ G_1:HJ_ D(g_1:H^m)( g_1:H)≤ _ g_1:H∈ G_1:HJ_D(g_1:H^m)( g_1:H). Now, we want to show that for any optimal g~1:H∗ g_1:H of problem ~(g1:Hm) D(g_1:H^m), we can construct g¯1:H∗=φ(g~1:H∗,(g1:Hm)) g_1:H = ( g_1:H ,D(g_1:H^m)) to be a linear optimal strategy of (g1:Hm)D(g_1:H^m), and J(g1:Hm)(g¯1:H∗)=J~(g1:Hm)(g~1:H∗)J_D(g_1:H^m)( g_1:H )=J_ D(g_1:H^m)( g_1:H ). For any optimal linear strategy g~1:H∗∈argming~1:H∈~1:HJ~(g1:Hm)(g~1:H) g_1:H ∈ argmin_ g_1:H∈ G_1:HJ_ D(g_1:H^m)( g_1:H) of ~(g1:Hm) D(g_1:H^m), we recursively construct a strategy g¯1:H∗=φ(g~1:H∗,(g1:Hm)) g_1:H = ( g_1:H ,D(g_1:H^m)) of (g1:Hm)D(g_1:H^m). For h=1h=1 and any i∈[n]i∈[n], from the construction of ~(g1:Hm) D(g_1:H^m), we know that ~i,1=¯i,1 I_i,1= I_i,1 always holds. Then we can define g~i,1∗(I~i,1)=g¯i,1∗(I~i,1) g_i,1 ( I_i,1)= g_i,1 ( I_i,1) for any i∈[n],I~i,1∈ℐ~i,1=ℐ¯i,1i∈[n], I_i,1∈ I_i,1= I_i,1. Then, for any i∈[n]i∈[n], g~i,1∗ g_i,1 and g¯i,1∗ g_i,1 output the same control action, namely, ~i,1=¯i,1 U_i,1= U_i,1, and g¯i,1∗ g_i,1 is linear. Now, for any h∈[2:H]h∈[2:H], assume that we have already constructed the linear strategy g¯1:h−1∗ g_1:h-1 from g~1:h−1∗ g_1:h-1 such that for any i∈[n]i∈[n] and t<ht<h, g¯i,t∗ g_i,t and g~i,t∗ g_i,t output the same actions. Now, we aim to construct g¯h∗ g_h from g~h∗ g_h . From the construction of problem ~(g1:Hm) D(g_1:H^m) from (g1:Hm)D(g_1:H^m), for any i∈[n]i∈[n], ¯i,h⊆~i,h I_i,h I_i,h and the only additional variables are the actions ~j,t∈~i,h\¯i,h U_j,t∈ I_i,h I_i,h, where j∈[n]j∈[n] and t<ht<h. Furthermore, if ~j,t∈~i,h\¯i,h U_j,t∈ I_i,h I_i,h, then B¯j,t,0 B_j,t 0 and ¯j,t⊆¯h I_j,t C_h. Since g¯1:h−1∗ g_1:h-1 outputs the same actions as g~1:h−1∗ g_1:h-1 , we can write ~j,t=¯j,t=g¯j,t∗(¯j,t) U_j,t= U_j,t= g_j,t ( I_j,t), which means that we can obtain ~j,t U_j,t from g¯j,t∗ g_j,t and ¯j,t I_j,t. Since g¯j,t∗ g_j,t is linear, we write g¯j,t∗(¯j,t)=∑¯k,s∈¯j,tG¯j,tk,s,y¯k,s+∑¯k,s∈¯j,tG¯j,tk,s,u¯k,s g_j,t ( I_j,t)= _ Y_k,s∈ I_j,t G_j,t^k,s,y Y_k,s+ _ U_k,s∈ I_j,t G_j,t^k,s,u U_k,s for any j∈[n]j∈[n] and t<ht<h. Here, the sum ∑¯k,s∈¯j,t _ Y_k,s∈ I_j,t ranges over all k∈[n]k∈[n] and s∈[H]s∈[H] such that ¯k,s∈¯j,t Y_k,s∈ I_j,t, and the other sums are interpreted similarly. Meanwhile, from the linearity of g~i,h∗ g_i,h for any i∈[n]i∈[n], we can write g~i,h∗(~i,h)=∑~j,t∈~i,hG~i,hj,t,y~j,t+∑~j,t∈~i,hG~i,hj,t,u~j,t g_i,h ( I_i,h)= _ Y_j,t∈ I_i,h G_i,h^j,t,y Y_j,t+ _ U_j,t∈ I_i,h G_i,h^j,t,u U_j,t for some matrices G~i,hj,t,y,G~i,hj,t,u G_i,h^j,t,y, G_i,h^j,t,u. Then, we have g~i,h∗(~i,h)=∑~j,t∈~i,hG~i,hj,t,y~j,t+∑~j,t∈~i,hG~i,hj,t,u~j,t=∑¯j,t∈¯i,hG~i,hj,t,y¯j,t+∑¯j,t∈¯i,hG~i,hj,t,u¯j,t+∑¯j,t∈~i,h\¯i,hG~i,hj,t,u¯j,t g_i,h ( I_i,h)= _ Y_j,t∈ I_i,h G_i,h^j,t,y Y_j,t+ _ U_j,t∈ I_i,h G_i,h^j,t,u U_j,t= _ Y_j,t∈ I_i,h G_i,h^j,t,y Y_j,t+ _ U_j,t∈ I_i,h G_i,h^j,t,u U_j,t+ _ U_j,t∈ I_i,h I_i,h G_i,h^j,t,u U_j,t =∑¯j,t∈¯i,hG~i,hj,t,y¯j,t+∑¯j,t∈¯i,hG~i,hj,t,u¯j,t+∑¯j,t∈~i,h\¯i,hG~i,hj,t,u(∑¯k,s∈¯j,tG¯j,tk,s,y¯k,s+∑¯k,s∈¯j,tG¯j,tk,s,u¯k,s). = _ Y_j,t∈ I_i,h G_i,h^j,t,y Y_j,t+ _ U_j,t∈ I_i,h G_i,h^j,t,u U_j,t+ _ U_j,t∈ I_i,h I_i,h G_i,h^j,t,u ( _ Y_k,s∈ I_j,t G_j,t^k,s,y Y_k,s+ _ U_k,s∈ I_j,t G_j,t^k,s,u U_k,s ). The second equality in the first line holds because g~1:h−1∗ g_1:h-1 and g¯1:h−1∗ g_1:h-1 output the same control actions, and the system dynamics of both problems are the same, so the observations are the same before agents take actions at timestep h. The equality between the first and the second lines follows from ~j,t=g¯j,t∗(¯j,t) U_j,t= g_j,t ( I_j,t) and substitution of the explicit form of g¯j,t∗ g_j,t . Also, if ~j,t∈~i,h\¯i,h U_j,t∈ I_i,h I_i,h, then ¯j,t⊆¯i,h I_j,t I_i,h, and any ¯k,s∈¯j,t,¯k,s∈¯j,t Y_k,s∈ I_j,t, U_k,s∈ I_j,t will also be included in ¯i,h I_i,h. Therefore, we have g~i,h∗(~i,h) g_i,h ( I_i,h) =∑¯j,t∈¯i,h(G~i,hj,t,y+∑¯k,s∈~i,h\¯i,hG~i,hk,s,u[¯j,t∈¯k,s]G¯k,sj,t,y)¯j,t = _ Y_j,t∈ I_i,h ( G_i,h^j,t,y+ _ U_k,s∈ I_i,h I_i,h G_i,h^k,s,u 1[ Y_j,t∈ I_k,s] G_k,s^j,t,y ) Y_j,t +∑¯j,t∈¯i,h(G~i,hj,t,u+∑¯k,s∈~i,h\¯i,hG~i,hk,s,u[¯j,t∈¯k,s]G¯k,sj,t,u)¯j,t. + _ U_j,t∈ I_i,h ( G_i,h^j,t,u+ _ U_k,s∈ I_i,h I_i,h G_i,h^k,s,u 1[ U_j,t∈ I_k,s] G_k,s^j,t,u ) U_j,t. Note that in the equation above, if ¯j,t<¯k,s Y_j,t∉ I_k,s, then it means that ¯k,s U_k,s is not based on ¯j,t Y_j,t, and we can just assign G¯k,sj,t,y= G_k,s^j,t,y=0. Also, if ¯j,t<¯k,s U_j,t∉ I_k,s, then we assign G¯k,sj,t,u= G_k,s^j,t,u=0. Therefore, we can construct g¯i,h∗ g_i,h for any i∈[n]i∈[n] as g¯i,h∗(¯i,h) g_i,h ( I_i,h) =∑¯j,t∈¯i,hG¯i,hj,t,y¯j,t+∑¯j,t∈¯i,hG¯i,hj,t,u¯j,t, = _ Y_j,t∈ I_i,h G_i,h^j,t,y Y_j,t+ _ U_j,t∈ I_i,h G_i,h^j,t,u U_j,t, ∀j∈[n],t≤h,G¯i,hj,t,y ∀ j∈[n],t≤ h, G_i,h^j,t,y :=G~i,hj,t,y+∑¯k,s∈~i,h\¯i,hG~i,hk,s,u[¯j,t∈¯k,s]G¯k,sj,t,y, := G_i,h^j,t,y+ _ U_k,s∈ I_i,h I_i,h G_i,h^k,s,u 1[ Y_j,t∈ I_k,s] G_k,s^j,t,y, ∀j∈[n],t≤h,G¯i,hj,t,u ∀ j∈[n],t≤ h, G_i,h^j,t,u :=G~i,hj,t,u+∑¯k,s∈~i,h\¯i,hG~i,hk,s,u[¯j,t∈¯k,s]G¯k,sj,t,u. := G_i,h^j,t,u+ _ U_k,s∈ I_i,h I_i,h G_i,h^k,s,u 1[ U_j,t∈ I_k,s] G_k,s^j,t,u. Then, g¯h∗ g_h is linear and outputs the same actions as g~h∗ g_h . Recursively, we can construct g¯h∗ g_h from h=1h=1 to H based on g~1:H∗ g_1:H and the problem (g1:Hm)D(g_1:H^m), and we denote this construction by a function φ , so that g¯1:H∗=φ(g~1:H∗,(g1:Hm)) g_1:H = ( g_1:H ,D(g_1:H^m)). Since the dynamics and control costs are the same in the two problems, and g¯1:H∗ g_1:H and g~1:H∗ g_1:H output the same actions, J~(g1:Hm)(g~1:H∗)=J(g1:Hm)(g¯1:H∗)J_ D(g_1:H^m)( g_1:H )=J_D(g_1:H^m)( g_1:H ). Also, we know that J~(g1:Hm)(g~1:H∗)=ming~1:H∈~1:HJ~(g1:Hm)(g~1:H)≤ming¯1:H∈¯1:HJ(g1:Hm)(g¯1:H)≤J(g1:Hm)(g¯1:H∗). J_ D(g_1:H^m)( g_1:H )= _ g_1:H∈ G_1:HJ_ D(g_1:H^m)( g_1:H)≤ _ g_1:H∈ G_1:HJ_D(g_1:H^m)( g_1:H)≤ J_D(g_1:H^m)( g_1:H ). Therefore, g¯1:H∗ g_1:H is a linear optimal strategy of (g1:Hm)D(g_1:H^m). ∎ C–2 Proof of Theorem IV.2 Proof. The proof consists of three Parts: Part 1: ~(g1:Hm) D(g_1:H^m) is PN; Part 2: ~(g1:Hm) D(g_1:H^m) has the information evolution rules; Part 3: ~(g1:Hm) D(g_1:H^m) satisfies the SI-CIB condition. Part 1: We want to prove that ~(g1:Hm) D(g_1:H^m) is PN. Fix i1,i2∈[n]i_1,i_2∈[n] and h1<h2h_1<h_2 in [H][H], and suppose that ~i1,h1 U_i_1,h_1 influences ~i2,h2 I_i_2,h_2. If B~i1,h1=0 B_i_1,h_1=0, the action has no state effect, while Assumption I.2 prevents it from appearing in any later information set, contradicting the assumed influence. Hence B~i1,h1=B¯i1,h1,0 B_i_1,h_1= B_i_1,h_1≠ 0. Assumption I.4 gives some i,i1i≠ i_1 such that ¯i1,h1 U_i_1,h_1 influences ¯i,h1+1 Y_i,h_1+1. Since h1+1≤h2h_1+1≤ h_2 and information is retained, ¯i1,h1 U_i_1,h_1 influences ¯i,h2 I_i,h_2. Theorem I.6 therefore gives ¯i1,h1⊆¯i,h2 I_i_1,h_1 I_i,h_2. Since i,i1i≠ i_1, every label private to agent i1i_1 can enter agent i’s information only through common information from Assumption I.1, so it holds ¯i1,h1⊆¯h2 I_i_1,h_1 C_h_2. Moreover, every label in ~i1,h1∖¯i1,h1 I_i_1,h_1 I_i_1,h_1 is common by construction. Thus, we have ~i1,h1⊆~h2⊆~i2,h2 I_i_1,h_1 C_h_2 I_i_2,h_2, proving that ~(g1:Hm) D(g^m_1:H) is PN. Part 2: We want to prove ~(g1:Hm) D(g_1:H^m) has the information evolution rules. From Theorem I.6, we know that the problem (g1:Hm)D(g_1:H^m) has the information evolution rules with some projection functions χ¯hh∈[H],ζ¯i,hi∈[n],h∈[H]\ χ_h\_h∈[H],\ ζ_i,h\_i∈[n],h∈[H]. For any i∈[n],h∈[H]i∈[n],h∈[H], because the corresponding label sets in ~h Z_h and ~i,h P_i,h are fixed, it suffices to prove that ~h⊆~h−1∪~h−1,~h,~i,h⊆~i,h−1∪~i,h−1,~i,h Z_h P_h-1∪\ U_h-1, Y_h\, P_i,h P_i,h-1∪\ U_i,h-1, Y_i,h\. Firstly, we show that the common information of ~(g1:Hm) D(g_1:H^m) has the evolution rule. For any h∈[H]h∈[H], consider the ~h Z_h. If any ~i,t∈~h,i∈[n],t≤h Y_i,t∈ Z_h,i∈[n],t≤ h, then ~i,t∈~h Y_i,t∈ C_h and ~i,t<~h−1 Y_i,t∉ C_h-1. Then ¯i,t∈¯h Y_i,t∈ C_h and ¯i,t<¯h−1 Y_i,t∉ C_h-1 since the construction does not change any observation in the information. Therefore, ¯i,t∈¯h⊆¯h−1∪¯h−1,¯h Y_i,t∈ Z_h P_h-1∪\ U_h-1, Y_h\ due to the evolution rule of problem (g1:Hm)D(g_1:H^m); since ¯i,t Y_i,t is an observation label, it belongs to ¯h−1∪¯h P_h-1∪\ Y_h\, and then ~i,t∈~h−1∪~h Y_i,t∈ P_h-1∪\ Y_h\. If any ~i,t∈~h U_i,t∈ Z_h, then B¯i,t,0 B_i,t 0 from Assumption I.2 and construction of ~(g1:Hm) D(g_1:H^m). If t<h−1t<h-1, we know ¯i,t U_i,t influences ¯t+1 X_t+1 and thus influences ¯j,t+1 Y_j,t+1 for some j,ij≠ i due to Assumption I.4. Partial nestedness gives ¯i,t⊆¯j,t+1 I_i,t I_j,t+1. Due to j,ij≠ i and Assumption I.1, it holds that ¯i,t⊆¯t+1 I_i,t C_t+1, and the strict expansion therefore places ~i,t U_i,t in ~t+1 C_t+1. From t<h−1t<h-1 we know ~i,t<~h U_i,t∉ Z_h, which leads to a contradiction. Hence, if ~i,t∈~h U_i,t∈ Z_h, it is only possible that t=h−1t=h-1. In conclusion, ~h⊆~h−1∪~h−1,~h Z_h P_h-1∪\ U_h-1, Y_h\. Secondly, we show that the private information of ~(g1:Hm) D(g_1:H^m) satisfies the evolution rule. For any i∈[n],h∈[H]i∈[n],h∈[H], consider the ~i,h P_i,h. If any ~i,t∈~i,h,t≤h Y_i,t∈ P_i,h,t≤ h, then ¯i,t∈¯i,h Y_i,t∈ P_i,h since the construction does not change any observation in the information. Therefore, ¯i,t∈¯i,h−1∪¯i,h Y_i,t∈ P_i,h-1∪\ Y_i,h\ due to the evolution rule of problem (g1:Hm)D(g_1:H^m), and then ~i,t∈~i,h−1∪~i,h Y_i,t∈ P_i,h-1∪\ Y_i,h\. If any ~i,t∈~i,h U_i,t∈ P_i,h, then B¯i,t,0 B_i,t 0 by Assumption I.2. Then, we know ¯i,t U_i,t influences ¯t+1 X_t+1 and thus influences ¯j,t+1 Y_j,t+1 for some j,ij≠ i due to Assumption I.4. Partial nestedness gives ¯i,t⊆¯j,t+1 I_i,t I_j,t+1. Due to j,ij≠ i and Assumption I.1, it holds that ¯i,t⊆¯t+1 I_i,t C_t+1, and the strict expansion therefore places ~i,t U_i,t in ~t+1 C_t+1. Thus, we have ~i,t<~i,h U_i,t∉ P_i,h, which leads to a contradiction. In conclusion, ~i,h⊆~i,h−1∪~i,h P_i,h P_i,h-1∪\ Y_i,h\. Part 3: ~(g1:Hm) D(g_1:H^m) satisfies the SI-CIB condition. We show this by induction. If h=1h=1, then the belief ~1 B_1 does not depend on any strategy, and the SI-CIB condition holds automatically. For each h≥2h≥ 2, the space ~h=~×~1,h×⋯×~n,h S_h= X× P_1,h×·s× P_n,h is finite-dimensional because the collection of random variables comprising each ~i,h P_i,h, i∈[n]i∈[n], is fixed. For any fixed Borel set h⊆~hQ_h S_h and any fixed common-information realization C~h∈~h C_h∈ C_h that can be reached by two control strategies g~1:h−1 g_1:h-1 and g~′1:h−1 g _1:h-1, define ℙ1=ℙ(~h∈h|~h=C~h,g~1:h−1),ℙ2=ℙ(~h∈h|~h=C~h,g~′1:h−1). _1=P ( S_h _h\,|\, C_h= C_h, g_1:h-1 ),~~P_2=P ( S_h _h\,|\, C_h= C_h, g _1:h-1 ). It suffices to prove that ℙ1=ℙ2P_1=P_2. We first suppose that g~1:h−1 g_1:h-1 and g~′1:h−1 g _1:h-1 differ only at one pair (i1,h1)(i_1,h_1), where i1∈[n]i_1∈[n] and h1<h_1<h. If B~i1,h1=B¯i1,h1,0 B_i_1,h_1= B_i_1,h_1 0, then ¯i1,h1 U_i_1,h_1 influences ¯h1+1 X_h_1+1 in (g1:Hm)D(g_1:H^m). By Assumption I.4, there exists i2,i1i_2≠ i_1 such that ¯i1,h1 U_i_1,h_1 influences ¯i2,h1+1 Y_i_2,h_1+1. Since (g1:Hm)D(g_1:H^m) has a PN IS, we have ¯i1,h1⊆¯i2,h1+1 I_i_1,h_1 I_i_2,h_1+1. Due to i2,i1i_2≠ i_1 and Assumption I.1, we have ¯i1,h1⊆¯h1+1 I_i_1,h_1 C_h_1+1. By the construction of ~(g1:Hm) D(g_1:H^m), ~i1,h1∈~h1+1⊆~h. U_i_1,h_1∈ C_h_1+1 C_h. Moreover, ~i1,h1∖¯i1,h1⊆~h1,¯i1,h1⊆¯h1+1⊆~h1+1⊆~h, I_i_1,h_1 I_i_1,h_1 C_h_1, I_i_1,h_1 C_h_1+1 C_h_1+1 C_h, and hence ~i1,h1⊆~h I_i_1,h_1 C_h. Conditioning on ~h=C~h C_h= C_h therefore fixes both the information at which g~i1,h1 g_i_1,h_1 and g~i1,h1′ g _i_1,h_1 are evaluated and the resulting action. Since both strategies reach the same common-information realization, they produce the same recorded action from the same recorded information. With these quantities fixed, the system dynamics and information evolution are identical under the two strategies. Therefore, ℙ1=ℙ2P_1=P_2. If B~i1,h1=B¯i1,h1= B_i_1,h_1= B_i_1,h_1=0, then ~i1,h1 U_i_1,h_1 does not affect the system dynamics. Moreover, Assumption I.2 prevents ¯i1,h1 U_i_1,h_1 from appearing in any later information set. Since the strict expansion only adds actions with nonzero input coefficients, ~i1,h1 U_i_1,h_1 does not appear in ~h′ C_h or ~i,h′ P_i,h for any h′∈[H]h ∈[H] with h′>h1h >h_1 and any i∈[n]i∈[n]. Thus, changing g~i1,h1 g_i_1,h_1 affects neither the state nor any later common or private information. It follows again that ℙ1=ℙ2P_1=P_2. Finally, consider any two arbitrary g~1:h−1 g_1:h-1 and g~′1:h−1 g _1:h-1 that can both reach a realization C~h C_h. Let C~h−1 C_h-1 be the common information realization at timestep h−1h-1 such that C~h−1⊆C~h C_h-1 C_h, then both g~1:h−1 g_1:h-1 and g~1:h−1′ g_1:h-1 can reach C~h−1 C_h-1. From induction, we know that the belief B~h−1 B_h-1 generated by C~h−1 C_h-1 under both g~1:h−1 g_1:h-1 and g~1:h−1′ g_1:h-1 are the same. Also, let g~1:h′ g_1:h be the strategy that replace g~1,h g_1,h by g~1,h′ g_1,h in g~1:h g_1:h, then g~1:h′ g_1:h can also reach C~h C_h. Therefore, we connect g~1:h′ g_1:h and g~1,h g_1,h by finitely many intermediate strategies, replacing one component control law at a time in this way. Consequently, we can use the result above and show that ℙ1=ℙ2P_1=P_2, which proves the SI-CIB condition. ∎ C–3 Proof of Lemma IV.3 The proof follows the arguments in Appendices B and C of [7], with the minor modification needed for possibly singular Gaussian distributions. Proof. We first prove Gaussianity: At timestep h=1h=1, the state, private information, and common information are linear functions of the primitive Gaussian variables. Hence, (~1,~1)( S_1, C_1) is jointly Gaussian, possibly singular, and the conditional belief ~1 B_1 is Gaussian. Fix h≥2h≥ 2 and suppose that g~1:h−1 g_1:h-1 is affine. Due to the linearity of the system dynamics, the state ~h X_h, private information ~h P_h, and common information ~h C_h are jointly Gaussian. Consequently, for every common information realization C~h C_h that can be reached under g~1:h−1 g_1:h-1, the conditional distribution of ~h S_h given ~h=C~h C_h= C_h is Gaussian. This conclusion remains valid when the joint distribution is singular: the conditional-Gaussian formula, with the Moore-Penrose pseudo-inverse in place of the inverse, gives a Gaussian conditional distribution whose covariance does not depend on C~h C_h. If g~1:h−1 g_1:h-1 is not affine, then for every common information realization C~h C_h that can be reached under g~1:h−1 g_1:h-1, there exist realizations Y~1:h,U~1:h−1 Y_1:h, U_1:h-1 such that (Y~1:h,U~1:h−1,C~h)( Y_1:h, U_1:h-1, C_h) is reachable under g~1:h−1 g_1:h-1. Then, we construct affine strategies g~1:h−1′∈~1:h−1 g_1:h-1 ∈ G_1:h-1 as: ∀i∈[n],t<h,∀I~i,t∈ℐ~i,t,g~i,t′(I~i,t)≡U~i,t∀ i∈[n],t<h,∀ I_i,t∈ I_i,t, g_i,t ( I_i,t)≡ U_i,t. Then, (Y~1:h,U~1:h−1,C~h)( Y_1:h, U_1:h-1, C_h) is also reachable under g~1:h−1′ g_1:h-1 . Since the CIB belief ~h B_h is strategy independent, its value at C~h C_h is the same under the original and comparison strategies. Thus, ~h B_h is Gaussian under any admissible strategy g~1:h−1 g_1:h-1. We next prove that �~h _h and �~h _h satisfy the evolution rules. The system dynamics and Equation (IV.2) give the usual fixed CIB-belief update from the preceding belief, the past control laws, and ~h Z_h. The SI-CIB condition removes the dependence on the control laws, so (�~h,�~h)=F~h−1((�~h−1,�~h−1),~h),( _h, _h)= F_h-1 (( _h-1, _h-1), Z_h ), where F~h−1 F_h-1 is a fixed transformation that does not depend on the control strategies. To determine the covariance component, again take affine control laws. The variables ~h S_h and ~h C_h are jointly Gaussian, and the conditional covariance of jointly Gaussian variables does not depend on the realized conditioning value, including in the singular case. Similarly to the argument above, for any strategy, we can replace it by a constant affine strategy. Constant control values change only the conditional mean, not the conditional covariance. Hence, �~h= ~h2(�~h−1) _h= _h^2( _h-1) for a fixed deterministic function ~h2 _h^2. Under the same affine control laws, [~h|~h]E[ S_h\,|\, C_h] is an affine function of ~h C_h, and [~h−1|~h−1]E[ S_h-1\,|\, C_h-1] is an affine function of ~h−1 C_h-1. Combining these facts with the preceding fixed belief update gives �~h= ~h1(�~h−1,~h) _h= _h^1( _h-1, Z_h) for a fixed affine function ~h1 _h^1. Since the dynamics and observation equations are homogeneous and the relevant known controls are included in ~h Z_h, the constant term is zero, so ~h1 _h^1 is linear. Neither update depends on the control strategies, which completes the proof. ∎ C–4 Proof of Theorem IV.4 Proof. Part 1: For h∈[H−1]h∈[H-1], given (�~h,γ~h)( _h, γ_h), draw ~h∼(�~h,�~h) S_h ( _h, _h) and the independent Gaussian noises and then apply the dynamics, χ~h+1 χ_h+1, and Lemma IV.3. The law of �~h+1 _h+1 therefore depends only on (�~h,γ~h)( _h, γ_h). At h=Hh=H, the final-state cost has already been included in cH‡c_H . Hence, ~‡(g1:Hm) D (g_1:H^m) is Markov. Part 2: We aim to prove that for any strategy g~1:H∈~1:H,J~(g1:Hm)(g~1:H)=J~‡(g1:Hm)(g1:H‡) g_1:H∈ G_1:H,J_ D(g_1:H^m)( g_1:H)=J_ D (g_1:H^m)(g _1:H), where gh‡=ςh(g~h),∀h∈[H]g_h = _h( g_h),∀ h∈[H]. For every h∈[H−1]h∈[H-1], [ch‡|g1:H‡]=[[ch‡(�~h,gh‡(~h))|~h,�~h= ~h3(~h)]|g1:h‡] [c_h \,|\,g_1:H ]=E [E[c_h ( _h,g _h( C_h))\,|\, C_h, _h= _h^3( C_h)]\, |\,g_1:h ] (C.1) =[[~h⊤Q~h1~h+[γ~1,h(~1,h)⊤⋯γ~n,h(~n,h)⊤]Q~h2[γ~1,h(~1,h)⋯γ~n,h(~n,h)]|~h,γ~h=gh‡(~h)]|g1:h‡] =E [E [ X_h Q_h^1 X_h+ bmatrix γ_1,h( P_1,h) &·s& γ_n,h( P_n,h) bmatrix Q_h^2 bmatrix γ_1,h( P_1,h)\\ ·s\\ γ_n,h( P_n,h) bmatrix\, |\, C_h, γ_h=g_h ( C_h) ]\, |\,g_1:h ] (C.2) =[[~h⊤Q~h1~h+[γ~1,h(~1,h)⊤⋯γ~n,h(~n,h)⊤]Q~h2[γ~1,h(~1,h)⋯γ~n,h(~n,h)]|~h,γ~h(⋅)=g~h(~h,⋅)]|g1:h‡] =E [E [ X_h Q_h^1 X_h+ bmatrix γ_1,h( P_1,h) &·s& γ_n,h( P_n,h) bmatrix Q_h^2 bmatrix γ_1,h( P_1,h)\\ ·s\\ γ_n,h( P_n,h) bmatrix\, |\, C_h, γ_h(·)= g_h( C_h,·) ]\, |\,g_1:h ] (C.3) =[[~h⊤Q~h1~h+[γ~1,h(~1,h)⊤⋯γ~n,h(~n,h)⊤]Q~h2[γ~1,h(~1,h)⋯γ~n,h(~n,h)]|~h,γ~h(⋅)=g~h(~h,⋅)]|g~1:h] =E [E [ X_h Q_h^1 X_h+ bmatrix γ_1,h( P_1,h) &·s& γ_n,h( P_n,h) bmatrix Q_h^2 bmatrix γ_1,h( P_1,h)\\ ·s\\ γ_n,h( P_n,h) bmatrix\, |\, C_h, γ_h(·)= g_h( C_h,·) ]\, |\, g_1:h ] (C.4) =[~h⊤Q~h1~h+~h⊤Q~h2~h|g~1:H]. =E[ X_h Q_h^1 X_h+ U_h Q_h^2 U_h\,|\, g_1:H]. (C.5) Equation (C.1) and Equation (C.5) are due to the tower property; Equation (C.2) is from the definition of ch‡c_h ; Equation (C.3) is from the definition of ςh _h; Equation (C.4) is because ~h C_h has the same distribution under both strategies g~1:h g_1:h and g1:h‡g_1:h , i.e., for any Borel set ~h⊆~h C_h C_h, it holds that ℙ(~h∈~h|g1:h‡)=[[~h∈~h]|g1:h‡]=[[[~h∈~h]|~h−1,γ~h−1=gh−1‡(~h−1)]|g1:h−1‡] ( C_h∈ C_h\,|\,g_1:h )=E[ 1[ C_h∈ C_h]\,|\,g_1:h ]=E [E [ 1[ C_h∈ C_h]\,|\, C_h-1, γ_h-1=g_h-1 ( C_h-1) ]\,|\,g_1:h-1 ] =[[⋯[[[~h∈~h]|~h−1,γ~h−1=gh−1‡(~h−1)]|~h−2,γ~h−2=gh−2‡(~h−2)]⋯|~1,γ~1=g1‡(~1)]] =E [E [·sE [E [ 1[ C_h∈ C_h]\, |\, C_h-1, γ_h-1=g_h-1 ( C_h-1) ]\, |\, C_h-2, γ_h-2=g_h-2 ( C_h-2) ]·s\, |\, C_1, γ_1=g_1 ( C_1) ] ] =[[⋯[[[~h∈~h]|~h−1,γ~h−1(⋅)=g~h−1(~h−1,⋅)]|~h−2,γ~h−2(⋅)=g~h−2(~h−2,⋅)]⋯|~1,γ~1(⋅)=g~1(~1,⋅)]] =E [E [·sE [E [ 1[ C_h∈ C_h]\,|\, C_h-1, γ_h-1(·)= g_h-1( C_h-1,·) ]\,|\, C_h-2, γ_h-2(·)= g_h-2( C_h-2,·) ]·s\, |\, C_1, γ_1(·)= g_1( C_1,·) ] ] =[[~h∈~h]|g~1:h]=ℙ(~h∈~h|g~1:h). =E [ 1[ C_h∈ C_h]\, |\, g_1:h ]=P( C_h∈ C_h\,|\, g_1:h). The same calculation at h=Hh=H, now using the definition of cH‡c_H , gives [cH‡|g1:H‡]=[~H⊤Q~H1~H+~H⊤Q~H2~H+~H+1⊤Q~H+11~H+1|g~1:H].E[c_H \,|\,g_1:H ]=E[ X_H Q_H^1 X_H+ U_H Q_H^2 U_H+ X_H+1 Q_H+1^1 X_H+1\,|\, g_1:H]. Therefore, J~‡(g1:Hm)(g1:H‡)=[∑h=1Hch‡|g1:H‡]=J~(g1:Hm)(g~1:H),J_ D (g_1:H^m)(g_1:H )=E [ _h=1^Hc_h \,|\,g_1:H ]=J_ D(g_1:H^m)( g_1:H), which completes the proof. ∎ C–5 Proof of Lemma IV.5 Proof. For convenience, given any matrices ℰ¯i,hi∈[n],h∈[H]\ E_i,h\_i∈[n],h∈[H] and ℱ~i,hi∈[n],h∈[H]\ F_i,h\_i∈[n],h∈[H] with appropriate dimensions, we denote by g~1:H=�(ℰ¯i,hi∈[n],h∈[H],ℱ~i,hi∈[n],h∈[H]) g_1:H= (\ E_i,h\_i∈[n],h∈[H],\ F_i,h\_i∈[n],h∈[H]) the linear strategy generated by ℰ¯i,hi∈[n],h∈[H],ℱ~i,hi∈[n],h∈[H]\ E_i,h\_i∈[n],h∈[H],\ F_i,h\_i∈[n],h∈[H], namely for any i∈[n],h∈[H],g~i,h(~h,~i,h)=ℰ¯i,h~h+ℱ~i,h~i,hi∈[n],h∈[H], g_i,h( C_h, P_i,h)= E_i,h C_h+ F_i,h P_i,h. Theorem IV.2 and Corollary A.2 give an optimal linear strategy g~1:H∗=�(ℰ¯i,h∗,ℱ~i,h∗) g_1:H = (\ E_i,h \,\ F_i,h \). If we fix ℱ~i,h∗i∈[n],h∈[H]\ F_i,h \_i∈[n],h∈[H], then ℰ¯i,h∗i∈[n],h∈[H]\ E_i,h \_i∈[n],h∈[H] minimizes J~(g1:Hm)J_ D(g_1:H^m) over ℰ¯i,hi∈[n],h∈[H]\ E_i,h\_i∈[n],h∈[H] with this private-information component fixed, i.e., ℰ¯i,h∗i∈[n],h∈[H]∈argminℰ¯i,hi∈[n],h∈[H]J~(g1:Hm)(�(ℰ¯i,hi∈[n],h∈[H],ℱ~i,h∗i∈[n],h∈[H])). \ E_i,h \_i∈[n],h∈[H]∈ argmin_\ E_i,h\_i∈[n],h∈[H]J_ D(g_1:H^m) ( (\ E_i,h\_i∈[n],h∈[H],\ F_i,h \_i∈[n],h∈[H]) ). From Theorem IV.2, we know that for any h∈[H]h∈[H] ~h=χ~h(~h−1,~h−1,~h),~h=ζ~h(~h−1,~h−1,~h), Z_h= χ_h( P_h-1, U_h-1, Y_h), P_h= ζ_h( P_h-1, U_h-1, Y_h), where χ~h,ζ~h χ_h, ζ_h are fixed projection functions. Therefore, there exist matrices such that ~h=~h1~h−1+~h2~h−1+~h3~h,~h=ℒ~h1~h−1+ℒ~h2~h−1+ℒ~h3~h, Z_h= T_h^1 P_h-1+ T_h^2 U_h-1+ T_h^3 Y_h, P_h= L_h^1 P_h-1+ L_h^2 U_h-1+ L_h^3 Y_h, for some matrices ~h1,~h2,~h3,ℒ~h1,ℒ~h2,ℒ~h3 T_h^1, T_h^2, T_h^3, L_h^1, L_h^2, L_h^3. As shown in [14], if we fix ℱ~i,h∗i∈[n],h∈[H]\ F_i,h \_i∈[n],h∈[H], we can construct an H-step centralized linear control problem ˘ D based on ~(g1:Hm) D(g_1:H^m), where ˘ D is an LQG system with linear dynamics, linear observations, quadratic costs, and Gaussian noises. For completeness, we include the construction below. Specifically, we can construct the centralized LQG problem ˘ D based on ~(g1:Hm) D(g_1:H^m) and fixed ℱ~i,h∗i∈[n],h∈[H]\ F_i,h \_i∈[n],h∈[H], where the state ˘h∈˘h X_h∈ X_h, the action ˘h∈˘h U_h∈ U_h, and the observation ˘h∈˘h Y_h∈ Y_h evolve as follows: ∀h∈[H],˘h=~×~h,˘h=~h,˘h=~h,˘h+1=A˘h˘h+B˘h˘h+˘h, ∀ h∈[H],~ X_h= X× P_h,~~~ U_h= U_h,~~~ Y_h= Z_h,~~~ X_h+1= A_h X_h+ B_h U_h+ W_h, ˘h=E˘h˘h−1+F˘h˘h−1+˘h(for h>1),˘1=˘1, Y_h= E_h X_h-1+ F_h U_h-1+ V_h~~(for h>1),~~ Y_1= V_1, ˘H+1=~H+1,˘h=[~h~h],˘h=~h−diag(ℱ~1,h∗,⋯,ℱ~n,h∗)~h,˘h=~h. X_H+1= X_H+1,~~~ X_h= bmatrix X_h\\ P_h bmatrix,~~~ U_h= U_h- diag( F_1,h ,·s, F_n,h ) P_h,~~~ Y_h= Z_h. Defining ℱ~h∗=diag(ℱ~1,h∗,⋯,ℱ~n,h∗) F_h = diag( F_1,h ,·s, F_n,h ), the matrices A˘h,B˘h,E˘h,F˘h A_h, B_h, E_h, F_h and noises ˘h,˘h W_h, V_h are given by ∀h∈[H−1],A˘h=[A~hB~hℱ~h∗ℒ~h+13E~h+1A~hℒ~h+11+ℒ~h+12ℱ~h∗+ℒ~h+13E~h+1B~hℱ~h∗],B˘h=[B~hℒ~h+12+ℒ~h+13E~h+1B~h]; ∀ h∈[H-1], A_h= bmatrix A_h& B_h F_h \\ L_h+1^3 E_h+1 A_h& L_h+1^1+ L_h+1^2 F_h + L_h+1^3 E_h+1 B_h F_h bmatrix, B_h= bmatrix B_h\\ L_h+1^2+ L_h+1^3 E_h+1 B_h bmatrix; ∀h∈[2:H],E˘h=[~h3E~hA~h−1~h1+~h2ℱ~h−1∗+~h3E~hB~h−1ℱ~h−1∗],F˘h=[~h2+~h3E~hB~h−1]; ∀ h∈[2:H], E_h= bmatrix T_h^3 E_h A_h-1& T_h^1+ T_h^2 F_h-1 + T_h^3 E_h B_h-1 F_h-1 bmatrix, F_h= bmatrix T_h^2+ T_h^3 E_h B_h-1 bmatrix; ∀h∈[H−1],˘h=[~0,hℒ~h+13(E~h+1~0,h+~1:n,h+1)],˘h+1=~h+13(E~h+1~0,h+~1:n,h+1); ∀ h∈[H-1], W_h= bmatrix W_0,h\\ L_h+1^3( E_h+1 W_0,h+ W_1:n,h+1) bmatrix, V_h+1= T_h+1^3( E_h+1 W_0,h+ W_1:n,h+1); A˘H=[A~HB~Hℱ~H∗],B˘H=B~H,˘H=~0,H,˘1=~13(E~1~1+~1:n,1), A_H= bmatrix A_H& B_H F_H bmatrix, B_H= B_H, W_H= W_0,H, V_1= T_1^3( E_1 X_1+ W_1:n,1), where we recall that ∀h∈[H−1],~1:n,h+1=[~1,h+1⋯~n,h+1]∀ h∈[H-1], W_1:n,h+1= bmatrix W_1,h+1\\ ·s\\ W_n,h+1 bmatrix. The cost of each stage h∈[H]h∈[H] is c˘h=[˘h⊤˘h⊤]⊤[Q˘h1N˘hN˘h⊤Q˘h2][˘h˘h] c_h= bmatrix X_h & U_h bmatrix bmatrix Q_h^1& N_h\\ N_h & Q_h^2 bmatrix bmatrix X_h\\ U_h bmatrix and c˘H+1=˘H+1⊤Q˘H+11˘H+1 c_H+1= X_H+1 Q^1_H+1 X_H+1, where the matrices Q˘h1,Q˘h2,N˘h,Q˘H+11 Q_h^1, Q_h^2, N_h, Q^1_H+1 are defined as ∀h∈[H],Q˘h1=[Q~h100(ℱ~h∗)⊤Q~h2ℱ~h∗],Q˘h2=Q~h2,N˘h=[0(ℱ~h∗)⊤Q~h2],Q˘H+11=Q~H+11. ∀ h∈[H], Q_h^1= bmatrix Q_h^1&0\\ 0&( F_h ) Q_h^2 F_h bmatrix, Q_h^2= Q_h^2, N_h= bmatrix0\\ ( F_h ) Q_h^2 bmatrix, Q^1_H+1= Q^1_H+1. The strategy of ˘ D at each stage h∈[H]h∈[H] is defined by g˘h:∏t=1h˘t→˘h g_h: _t=1^h Y_t→ U_h and the objective is defined as J˘(g˘1:H)=[∑h=1H+1c˘h|g˘1:H]J_ D( g_1:H)=E[ _h=1^H+1 c_h\,|\, g_1:H]. Thus, an optimal strategy satisfies g˘1:H∗∈argming˘1:HJ˘(g˘1:H) g_1:H ∈ argmin_ g_1:HJ_ D( g_1:H). Applying the Bellman projection argument below recursively to this centralized construction shows that an optimal strategy can be chosen linear even when the induced noises are correlated and singular and the quadratic weights are semidefinite. Under this strategy, the aggregate variables of ˘ D are jointly Gaussian, possibly singular, so the Moore-Penrose conditional-Gaussian formula gives matrices K˘h K_h such that ˘^h=[˘h|˘1:h]=K˘h˘1:h X_h=E[ X_h\,|\, Y_1:h]= K_h Y_1:h. Hence there exist matrices ℰ˘h∗ E_h such that g˘h∗(˘1:h)=ℰ˘h∗˘^h=ℰ˘h∗K˘h˘1:h g_h ( Y_1:h)= E_h X_h= E_h K_h Y_1:h for every h∈[H]h∈[H]. Also, one can verify that for any linear strategy g˘1:H g_1:H with the form g˘h(˘1:h)=ℰ˘h˘1:h,∀h∈[H] g_h( Y_1:h)= E_h Y_1:h,∀ h∈[H] and ℰ˘h=[ℰ˘1,h⋯ℰ˘n,h] E_h= bmatrix E_1,h\\ ·s\\ E_n,h bmatrix, we can construct a linear strategy of ~(g1:Hm) D(g_1:H^m) as g~1:H:=�(ℰ˘i,hi∈[n],h∈[H],ℱ~i,h∗i∈[n],h∈[H]) g_1:H:= (\ E_i,h\_i∈[n],h∈[H],\ F_i,h \_i∈[n],h∈[H]) such that J˘(g˘1:H)=J~(g1:Hm)(g~1:H)J_ D( g_1:H)=J_ D(g_1:H^m)( g_1:H), and vice versa. Therefore, g~1:H∗:=�(ℰ˘i,h∗K˘hi∈[n],h∈[H],ℱ~i,h∗i∈[n],h∈[H]) g_1:H := (\ E_i,h K_h\_i∈[n],h∈[H],\ F_i,h \_i∈[n],h∈[H]) is an optimal strategy of ~(g1:Hm) D(g_1:H^m), where ℰ˘h∗=[ℰ˘1,h∗⋯ℰ˘n,h∗] E_h = bmatrix E_1,h \\ ·s\\ E_n,h bmatrix for any h∈[H]h∈[H]. From the construction of ˘ D, we know that ˘h=[~h~h],˘h=~h X_h= bmatrix X_h\\ P_h bmatrix, Y_h= Z_h for any h∈[H]h∈[H], so ˘^h=[˘h|˘1:h]=[[~h~h]|~1:h]=[[~h|~h][~h|~h]]=�~h X_h=E [ X_h\,|\, Y_1:h ]=E [ bmatrix X_h\\ P_h bmatrix\, |\, Z_1:h ]= bmatrixE[ X_h\,|\, C_h]\\ E[ P_h\,|\, C_h] bmatrix= _h, and ~h3(~h)=K˘h~h _h^3( C_h)= K_h C_h. The centralized construction proves global optimality. The stronger pointwise Bellman assertion follows by backward induction. Under the appendix terminal convention, V~H+1(�~)=�~⊤Q~H+11�~ V_H+1( )= Q_H+1^1 . Suppose V~h+1 V_h+1 is quadratic with positive-semidefinite quadratic part, fix a reachable mean �~h=θ _h=θ, and write ~h=θ+ω~h S_h=θ+ ω_h, where ω~h∼(0,�~h) ω_h (0, _h). For an arbitrary admissible prescription, let δ~i,h:=~i,h−[~i,h∣�~h=θ]δ P_i,h:= P_i,h-E[ P_i,h _h=θ] and take the L2L^2 projection of ~i,h U_i,h onto the affine functions of δ~i,hδ P_i,h ~i,h=ai,h+ℱ~i,hδ~i,h+ri,h,[ri,h]=0,[ri,hδ~i,h⊤]=0. U_i,h=a_i,h+ F_i,hδ P_i,h+r_i,h, [r_i,h]=0, [r_i,hδ P_i,h ]=0. Since (ω~h,δ~i,h)( ω_h,δ P_i,h) is jointly Gaussian, possibly singular, its conditional mean is linear on its support. Hence [ri,hω~h⊤]=0E[r_i,h ω_h ]=0, and therefore ri,hr_i,h is also orthogonal to every δ~j,hδ P_j,h; it is orthogonal to the fresh independent noises as well. The Bellman objective is a positive-semidefinite quadratic in the state, controls, and fresh noises. Expanding it makes all terms linear in the stacked projection residual rhr_h vanish, while the remaining residual term is nonnegative. Thus an affine prescription dominates every admissible prescription at this fixed mean. Stacking the affine projections gives ~h=ah+ℱ~hp,hω~h U_h=a_h+ F_hI_p,h ω_h with block-diagonal ℱ~h F_h. Its Bellman objective separates into a mean quadratic in aha_h and a centered quadratic in ℱ~h F_h. Both are bounded-below finite-dimensional convex quadratics, so their generalized normal equations are consistent. The centered problem is independent of θ. Writing the mean-dependent part as ah⊤hah+2ah⊤hθ+θ⊤hθa_h G_ha_h+2a_h C_hθ+θ D_hθ, the mean quadratic is positive semidefinite in (ah,θ)(a_h,θ), so its block matrix is positive semidefinite. Hence ker(h)⊆ker(h⊤) ( G_h) ( C_h ), equivalently range(h)⊆range(h)range( C_h) ( G_h); therefore ah∗=−h†hθ=:ℰ^h∗θa_h =- G_h C_hθ=: E_h θ. Thus ℱ~h∗ F_h and ℰ^h∗ E_h minimize simultaneously for every reachable mean. Since ~h=p,h(θ+ω~h) P_h=I_p,h(θ+ ω_h), set ℰ~h∗:=ℰ^h∗−ℱ~h∗p,h E_h := E_h - F_h I_p,h. The same prescription minimizes at every CIB mean reachable under a preceding strategy, and its linear formula defines the strategy elsewhere. Its quadratic value has a positive-semidefinite quadratic part, so backward induction yields g~i,h∗(C~h,P~i,h)=ℰ~i,h∗ ~h3(C~h)+ℱ~i,h∗P~i,h g_i,h ( C_h, P_i,h)= E_i,h _h^3( C_h)+ F_i,h P_i,h, simultaneously linear, CIB-Markovian, globally optimal, and Bellman optimal over all prescriptions at every such reachable mean. ∎ C–6 Proof of Theorem IV.7 Proof. The proof consists of two Parts. Part 1: we aim to show the evolution of �~h _h. For every h∈[H]h∈[H], we define ¯h:=~h=[~h~h],¯h:=~h X_h:= S_h= bmatrix X_h\\ P_h bmatrix, Y_h:= Z_h, and ¯h:=~he=[[B~1,h,0]~1,h[B~2,h,0]~2,h⋯[B~n,h,0]~n,h], U_h:= U_h^e= bmatrix 1[ B_1,h 0] U_1,h\\ 1[ B_2,h 0] U_2,h\\ ·s\\ 1[ B_n,h 0] U_n,h bmatrix, where ~he U_h^e is defined by replacing the ~i,h U_i,h part of ~h U_h by 0 for all i∈[n]i∈[n] such that B~i,h= B_i,h=0. We claim that such a system is a linear system with the evolution rules given by ¯h+1=A¯h¯h+B¯h¯h+¯h,h∈[H−1], X_h+1= A_h X_h+ B_h U_h+ W_h, h∈[H-1], ¯t=E¯t¯t−1+F¯t¯t−1+¯t,t∈[2:H], Y_t= E_t X_t-1+ F_t U_t-1+ V_t, t∈[2:H], for some matrices A¯h,B¯h,E¯h,F¯h A_h, B_h, E_h, F_h defined below. Firstly, from the system evolution, we know that ∀h∈[H],~h+1=A~h~h+B~h~h+~0,h,~h=E~h~h+~1:n,h,E~h=[E~1,h⋯E~n,h],~1:n,h=[~1,h⋯~n,h]. ∀ h∈[H], X_h+1= A_h X_h+ B_h U_h+ W_0,h, Y_h= E_h X_h+ W_1:n,h, E_h= bmatrix E_1,h\\ ·s\\ E_n,h bmatrix, W_1:n,h= bmatrix W_1,h\\ ·s\\ W_n,h bmatrix. Also, from Theorem IV.2, we know that ~h,~h Z_h, P_h evolve as ~h=~h1~h−1+~h2~h−1+~h3~h,~h=ℒ~h1~h−1+ℒ~h2~h−1+ℒ~h3~h, Z_h= T_h^1 P_h-1+ T_h^2 U_h-1+ T_h^3 Y_h, P_h= L_h^1 P_h-1+ L_h^2 U_h-1+ L_h^3 Y_h, for some matrices ~hkk∈[3]\ T_h^k\_k∈[3] and ℒ~hkk∈[3]\ L_h^k\_k∈[3]. Then, we can show that the system dynamics are defined for h∈[H−1]h∈[H-1] as A¯h:=[A~h0ℒ~h+13E~h+1A~hℒ~h+11],B¯h:=[B~hℒ~h+12+ℒ~h+13E~h+1B~h],¯h=[~0,hℒ~h+13(E~h+1~0,h+~1:n,h+1)]. A_h:= bmatrix A_h&0\\ L_h+1^3 E_h+1 A_h& L_h+1^1 bmatrix,~ B_h:= bmatrix B_h\\ L_h+1^2+ L_h+1^3 E_h+1 B_h bmatrix, W_h= bmatrix W_0,h\\ L_h+1^3( E_h+1 W_0,h+ W_1:n,h+1) bmatrix. The observation quantities are defined for t∈[2:H]t∈[2:H] by E¯t:=[~t3E~tA~t−1~t1],F¯t:=~t2+~t3E~tB~t−1,¯t:=~t3(E~t~0,t−1+~1:n,t). E_t:= bmatrix T_t^3 E_t A_t-1& T_t^1 bmatrix, F_t:= T_t^2+ T_t^3 E_t B_t-1, V_t:= T_t^3( E_t W_0,t-1+ W_1:n,t). Note that based on Assumption I.2, one can verify that if B~i,h= B_i,h=0, then ~i,h+12=,ℒ~i,h+12= T_i,h+1^2=0, L_i,h+1^2=0, where ~i,h+12 T_i,h+1^2 and ℒ~i,h+12 L_i,h+1^2 are the corresponding agent i’s column blocks of ~h+12,ℒ~h+12 T_h+1^2, L_h+1^2. Therefore, it holds that for any h∈[H−1]h∈[H-1], B¯h~h=B¯h~he,F¯h+1~h=F¯h+1~he B_h U_h= B_h U_h^e,~~ F_h+1 U_h= F_h+1 U_h^e. Defining �¯h:=[¯h|¯1:h,¯1:h−1]=[~h|~1:h,~1:h−1] _h:=E[ X_h\,|\, Y_1:h, U_1:h-1]=E[ S_h\,|\, Z_1:h, U_1:h-1], the standard linear-system formulas give for h∈[H−1]h∈[H-1] �¯h+1=A¯h�¯h+B¯h¯h+(A¯h�¯hpE¯h+1⊤+S¯h)[E¯h+1�¯hpE¯h+1⊤+�h+1v]†(¯h+1−E¯h+1�¯h−F¯h+1¯h),†footnotemark: _h+1= A_h _h+ B_h U_h+( A_h ^p_h E_h+1 + S_h)[ E_h+1 ^p_h E_h+1 + _h+1^v] ( Y_h+1- E_h+1 _h- F_h+1 U_h), with �¯h+1p:=A¯h�¯hpA¯h⊤+�hw−(A¯h�¯hpE¯h+1⊤+S¯h)[E¯h+1�¯hpE¯h+1⊤+�h+1v]†(E¯h+1�¯hpA¯h⊤+S¯h⊤), ~ ^p_h+1:= A_h ^p_h A_h + _h^w-( A_h ^p_h E_h+1 + S_h)[ E_h+1 ^p_h E_h+1 + _h+1^v] ( E_h+1 ^p_h A_h + S_h ), :=1diag(�1,�~1,1,…,�~n,1),M1:=[I0ℒ~13E~1ℒ~13],N1:=[~13E~1~13], _1:= diag( _1, _1,1,…, _n,1), M_1:= bmatrixI&0\\ L_1^3 E_1& L_1^3 bmatrix, N_1:= bmatrix T_1^3 E_1& T_1^3 bmatrix, �¯1=M1N⊤11(N1N⊤11)†~1,�¯1p=M1M⊤11−M1N⊤11(N1N⊤11)†N1M⊤11, _1=M_1_1N_1 (N_1_1N_1 ) Z_1, _1^p=M_1_1M_1 -M_1_1N_1 (N_1_1N_1 ) N_1_1M_1 , �ty:=diag(�~1,t,…,�~n,t),t∈[H],0dx×dx,t=H+1, _t^y:= cases diag( _1,t,…, _n,t),&t∈[H],\\ 0_d_x× d_x,&t=H+1, cases S¯h:=cov(¯h,¯h+1)=[�~0,hE~h+1⊤ℒ~h+13(E~h+1�~0,hE~h+1⊤+�h+1y)](~h+13)⊤,h∈[H−1], S_h:=cov( W_h, V_h+1)= bmatrix _0,h E_h+1 \\ L_h+1^3( E_h+1 _0,h E_h+1 + _h+1^y) bmatrix( T_h+1^3) , h∈[H-1], �tv:=cov(¯t)=~t3(E~t�~0,t−1E~t⊤+diag(�~1,t,⋯,�~n,t))(~t3)⊤,t∈[2:H], _t^v:=cov( V_t)= T_t^3( E_t _0,t-1 E_t + diag( _1,t,·s, _n,t))( T_t^3) , t∈[2:H], �hw:=cov(¯h)=[�~0,h�~0,hE~h+1⊤(ℒ~h+13)⊤ℒ~h+13E~h+1�~0,hℒ~h+13(E~h+1�~0,hE~h+1⊤+�h+1y)(ℒ~h+13)⊤],h∈[H−1]. _h^w:=cov( W_h)= bmatrix _0,h& _0,h E_h+1 ( L_h+1^3) \\ L_h+1^3 E_h+1 _0,h& L_h+1^3( E_h+1 _0,h E_h+1 + _h+1^y)( L_h+1^3) bmatrix, h∈[H-1]. †footnotetext: Let h+1:=E¯h+1�¯hpE¯h+1⊤+�h+1v_h+1:= E_h+1 _h^p E_h+1 + _h+1^v. Positivity of the joint innovation/state-error covariance implies ker()h+1⊆ker(A¯h�¯hpE¯h+1⊤+S¯h) (_h+1) ( A_h _h^p E_h+1 + S_h). Hence, these Moore-Penrose gains give the canonical singular Kalman update [2]. For h∈[H−1]h∈[H-1], strict partial nestedness implies that if B~i,h,0 B_i,h≠ 0, then both ~i,h I_i,h and ~i,h U_i,h are contained in ~h+1 C_h+1. Hence a projection Prj~hz,u Prj_h^z,u satisfies ~he=Prj~hz,u~h+1 U_h^e= Prj_h^z,u Z_h+1 for h<Hh<H. For every such effective action, its information is also revealed at h+1h+1; after deleting redundant action coordinates and subtracting their known contribution, the remaining signal is a linear Gaussian observation with noise independent of the past. Thus the known-input Kalman formula below does not treat a private, state-correlated action as exogenous. At h=Hh=H, the synthetic terminal observation is ~H+1=~H+1 Y_H+1= X_H+1, so directly set KH+11=KH+12=KH+13=0K_H+1^1=K_H+1^2=K_H+1^3=0 and KH+14=IK_H+1^4=I; no known-input Kalman argument is used. Finally, for h∈[H−1]h∈[H-1], we can write �¯h+1=A¯h�¯h+B¯hPrj~hz,u¯h+1+(A¯h�¯hpE¯h+1⊤+S¯h)[E¯h+1�¯hpE¯h+1⊤+�h+1v]†(¯h+1−E¯h+1�¯h−F¯h+1Prj~hz,u¯h+1). _h+1= A_h _h+ B_h Prj_h^z,u Y_h+1+( A_h ^p_h E_h+1 + S_h)[ E_h+1 ^p_h E_h+1 + _h+1^v] ( Y_h+1- E_h+1 _h- F_h+1 Prj_h^z,u Y_h+1). (C.6) Based on Equation (C.6), we know that �¯h _h is a linear combination of ¯1:h=~1:h=~h Y_1:h= Z_1:h= C_h. Then, we have �¯h=[¯h|¯1:h,¯1:h−1]=[¯h|¯1:h]=[~h|~h]=�~h _h=E[ X_h\,|\, Y_1:h, U_1:h-1]=E[ X_h\,|\, Y_1:h]=E[ S_h\,|\, C_h]= _h. Furthermore, we obtain �~h=�¯hp _h= ^p_h. Therefore, we can conclude �~h+1=A¯h�~h+B¯hPrj~hz,u~h+1+(A¯h�¯hpE¯h+1⊤+S¯h)[E¯h+1�¯hpE¯h+1⊤+�h+1v]†(~h+1−E¯h+1�~h−F¯h+1Prj~hz,u~h+1), _h+1= A_h _h+ B_h Prj_h^z,u Z_h+1+( A_h ^p_h E_h+1 + S_h)[ E_h+1 ^p_h E_h+1 + _h+1^v] ( Z_h+1- E_h+1 _h- F_h+1 Prj_h^z,u Z_h+1), (C.7) �~h+1=A¯h�~hA¯h⊤+�hw−(A¯h�~hE¯h+1⊤+S¯h)[E¯h+1�~hE¯h+1⊤+�h+1v]†(E¯h+1�~hA¯h⊤+S¯h⊤). _h+1= A_h _h A_h + _h^w-( A_h _h E_h+1 + S_h)[ E_h+1 _h E_h+1 + _h+1^v] ( E_h+1 _h A_h + S_h ). Note that for h<Hh<H, Equation (C.7) gives precisely the functions ~h+11, ~h+12 _h+1^1, _h+1^2 in Lemma IV.3. For h<Hh<H, from Theorem IV.2, we know that ~h+1=χ~h+1(~h,~h,~h+1) Z_h+1= χ_h+1( P_h, U_h, Y_h+1) for some projection function χ~h+1 χ_h+1. Then, we can replace ~h+1 Z_h+1 by a linear combination of ~h,~h,~h+1 P_h, U_h, Y_h+1 and then obtain Kh+1kk∈[4]\K_h+1^k\_k∈[4] directly. The terminal matrices are the direct values stated above. Substituting those terminal values gives L~H1=L~H3=B~H L_H^1= L_H^3= B_H, L~H2=L~H4=A~Hx,H L_H^2= L_H^4= A_HI_x,H, and �~H=�0,H _H= _0,H, as stated in Theorem IV.7. For h<Hh<H, the two fresh-noise trace terms below combine as Tr(�~hR~h+1) Tr( _h R_h+1). Part 2: we aim to show that the value function V~h(�~h) V_h( _h) has a quadratic form, and the value function and the matrices ℰ~h∗,ℱ~h∗ E_h , F_h of the optimal control strategy can be computed via Riccati Equations. We prove this by induction. Under the appendix terminal convention, �~H+1=~H+1 _H+1= X_H+1, �~H+1=0 _H+1=0, and x,H+1=II_x,H+1=I. Thus V~H+1(�~H+1)=�~H+1⊤Q~H+11�~H+1 V_H+1( _H+1)= _H+1 Q_H+1^1 _H+1, so R~H+1=Q~H+11 R_H+1= Q_H+1^1 and c~H+1=0 c_H+1=0. At h=Hh=H, this is exactly the final-state term already included in cH‡c_H in the main text. For the calculation below, we equivalently decompose cH‡c_H into the ordinary stage-H cost and the auxiliary terminal value V~H+1(~H+1) V_H+1( X_H+1). Now, for any given h∈[H]h∈[H], we assume that V~h+1(�~h+1)=�~h+1⊤R~h+1�~h+1+c~h+1 V_h+1( _h+1)= _h+1 R_h+1 _h+1+ c_h+1 for some R~h+1⪰0 R_h+1 0 and c~h+1 c_h+1. From Lemma IV.3, we know that the CIB belief ~h B_h is a Gaussian distribution (�~h,�~h)N( _h, _h), so we can write ~h=�~h+ω~h S_h= _h+ ω_h, where ω~h∼(,�~h) ω_h (0, _h). From the definition of ~h S_h, we can write ~h=x,h~h,~h=p,h~h X_h=I_x,h S_h, P_h=I_p,h S_h for the projection matrices x,hI_x,h and p,hI_p,h. Also, for any i∈[n]i∈[n], since the action of agent i has the form ~i,h=ℰ~i,h�~h+ℱ~i,h~i,h U_i,h= E_i,h _h+ F_i,h P_i,h, we can write the joint control action as ~h=ℰ~h�~h+ℱ~h~h U_h= E_h _h+ F_h P_h with ℰ~h:=[ℰ~1,h⋯ℰ~n,h] E_h:= bmatrix E_1,h\\ ·s\\ E_n,h bmatrix and ℱ~h:=diag(ℱ~1,h,⋯,ℱ~n,h) F_h:= diag( F_1,h,·s, F_n,h). Therefore, ~h=ℰ~h�~h+ℱ~hp,h(�~h+ω~h)=(ℰ~h+ℱ~hp,h)�~h+ℱ~hp,hω~h U_h= E_h _h+ F_hI_p,h( _h+ ω_h)=( E_h+ F_hI_p,h) _h+ F_hI_p,h ω_h. Now, we define ℰ^h:=ℰ~h+ℱ~hp,h E_h:= E_h+ F_hI_p,h, and then optimizing (ℰ~h,ℱ~i,hi∈[n])( E_h,\ F_i,h\_i∈[n]) is equivalent to optimizing (ℰ^h,ℱ~i,hi∈[n])( E_h,\ F_i,h\_i∈[n]) with respect to the corresponding spaces. This is because for any (ℰ~h,ℱ~i,hi∈[n])( E_h,\ F_i,h\_i∈[n]), we can find the corresponding (ℰ^h,ℱ~i,hi∈[n])( E_h,\ F_i,h\_i∈[n]), and vice versa. Then, we can write the stage-cost as minℰ~h,ℱ~i,hi∈[n][~h⊤Q~h1~h+~h⊤Q~h2~h|�~h=�~h,ℰ~h,ℱ~i,hi∈[n]] _ E_h,\ F_i,h\_i∈[n]E[ X_h Q_h^1 X_h+ U_h Q_h^2 U_h\,|\, _h= _h, E_h,\ F_i,h\_i∈[n]] =minℰ^h,ℱ~i,hi∈[n][~h⊤Q~h1~h+~h⊤Q~h2~h|�~h=�~h,ℰ^h,ℱ~h] = _ E_h,\ F_i,h\_i∈[n]E[ X_h Q_h^1 X_h+ U_h Q_h^2 U_h\,|\, _h= _h, E_h, F_h] =minℰ^h,ℱ~i,hi∈[n]�~h⊤x,h⊤Q~h1x,h�~h+�~h⊤ℰ^h⊤Q~h2ℰ^h�~h+Tr(x,h�~hx,h⊤Q~h1)+Tr(ℱ~hp,h�~hp,h⊤ℱ~h⊤Q~h2). = _ E_h,\ F_i,h\_i∈[n] _h I_x,h Q_h^1I_x,h _h+ _h E_h Q_h^2 E_h _h+ Tr(I_x,h _hI_x,h Q_h^1)+ Tr( F_hI_p,h _hI_p,h F_h Q_h^2). For h<Hh<H, Theorem IV.2 gives the information evolution ~h+1=χ~h+1(~h,~h,~h+1) Z_h+1= χ_h+1( P_h, U_h, Y_h+1), and Lemma IV.3 gives �~h+1= ~h+11(�~h,~h+1) _h+1= _h+1^1( _h, Z_h+1). Then, we can write �~h+1= ~h+14(�~h,~h,~h,~h+1) _h+1= _h+1^4( _h, P_h, U_h, Y_h+1) for some linear function ~h+14 _h+1^4. Therefore, we can write �~h+1=Kh+11�~h+Kh+12~h+Kh+13~h+Kh+14~h+1 _h+1=K_h+1^1 _h+K_h+1^2 P_h+K_h+1^3 U_h+K_h+1^4 Y_h+1 for some matrices Kh+11,Kh+12,Kh+13,Kh+14K_h+1^1,K_h+1^2,K_h+1^3,K_h+1^4. At h=Hh=H, the same expression follows from the appendix terminal convention and the terminal matrices stated above. Also, we know that ~h=p,h(�~h+ω~h),~h=(ℰ~h+ℱ~hp,h)�~h+ℱ~hp,hω~h,~h+1=E~h+1~h+1+~h+1=E~h+1(A~h~h+B~h~h+~0,h)+~1:n,h+1 P_h=I_p,h( _h+ ω_h), U_h=( E_h+ F_hI_p,h) _h+ F_hI_p,h ω_h, Y_h+1= E_h+1 X_h+1+ W_h+1= E_h+1( A_h X_h+ B_h U_h+ W_0,h)+ W_1:n,h+1, where E~h+1=[E~1,h+1⋯E~n,h+1],~1:n,h+1=[~1,h+1⋯~n,h+1] E_h+1= bmatrix E_1,h+1\\ ·s\\ E_n,h+1 bmatrix, W_1:n,h+1= bmatrix W_1,h+1\\ ·s\\ W_n,h+1 bmatrix. Therefore, we can write �~h+1 _h+1 as �~h+1 _h+1 =Kh+11�~h+Kh+12p,h(�~h+ω~h)+Kh+13(ℰ^h�~h+ℱ~hp,hω~h) =K_h+1^1 _h+K_h+1^2I_p,h( _h+ ω_h)+K_h+1^3( E_h _h+ F_hI_p,h ω_h) +Kh+14(E~h+1(A~hx,h(�~h+ω~h)+B~h(ℰ^h�~h+ℱ~hp,hω~h)+~0,h)+~1:n,h+1), +K_h+1^4( E_h+1( A_hI_x,h( _h+ ω_h)+ B_h( E_h _h+ F_hI_p,h ω_h)+ W_0,h)+ W_1:n,h+1), =(Kh+11+Kh+12p,h+Kh+13ℰ^h+Kh+14E~h+1A~hx,h+Kh+14E~h+1B~hℰ^h)�~h+Kh+14~1:n,h+1 =(K_h+1^1+K_h+1^2I_p,h+K_h+1^3 E_h+K_h+1^4 E_h+1 A_hI_x,h+K_h+1^4 E_h+1 B_h E_h) _h+K_h+1^4 W_1:n,h+1 +(Kh+12p,h+Kh+13ℱ~hp,h+Kh+14E~h+1(A~hx,h+B~hℱ~hp,h))ω~h+Kh+14E~h+1~0,h. +(K_h+1^2I_p,h+K_h+1^3 F_hI_p,h+K_h+1^4 E_h+1( A_hI_x,h+ B_h F_hI_p,h)) ω_h+K_h+1^4 E_h+1 W_0,h. Since ω~h ω_h depends only on primitive randomness through time h, it is independent of ~0,h W_0,h and ~1:n,h+1 W_1:n,h+1. Then, we can define the parameters as L~h1 L_h^1 :=Kh+13+Kh+14E~h+1B~h,L~h2:=Kh+11+Kh+12p,h+Kh+14E~h+1A~hx,h, :=K_h+1^3+K_h+1^4 E_h+1 B_h,~~ L_h^2:=K_h+1^1+K_h+1^2I_p,h+K_h+1^4 E_h+1 A_hI_x,h, L~h3 L_h^3 :=Kh+13+Kh+14E~h+1B~h,L~h4:=Kh+12p,h+Kh+14E~h+1A~hx,h, :=K_h+1^3+K_h+1^4 E_h+1 B_h,~~ L_h^4:=K_h+1^2I_p,h+K_h+1^4 E_h+1 A_hI_x,h, and have �~h+1=(L~h1ℰ^h+L~h2)�~h+(L~h3ℱ~hp,h+L~h4)ω~h+Kh+14E~h+1~0,h+Kh+14~1:n,h+1. _h+1=( L_h^1 E_h+ L_h^2) _h+( L_h^3 F_hI_p,h+ L_h^4) ω_h+K_h+1^4 E_h+1 W_0,h+K_h+1^4 W_1:n,h+1. Thus, from the induction hypothesis, we have [V~h+1(�~h+1)|�~h,ℰ~h,ℱ~h]=[V~h+1(�~h+1)|�~h,ℰ^h,ℱ~h]=[�~h+1⊤R~h+1�~h+1+c~h+1|�~h,ℰ~h,ℱ~h] [ V_h+1( _h+1)\,|\, _h, E_h, F_h]=E[ V_h+1( _h+1)\,|\, _h, E_h, F_h]=E[ _h+1 R_h+1 _h+1+ c_h+1\,|\, _h, E_h, F_h] =�~h⊤(L~h1ℰ^h+L~h2)⊤R~h+1(L~h1ℰ^h+L~h2)�~h+Tr((L~h3ℱ~hp,h+L~h4)�~h(L~h3ℱ~hp,h+L~h4)⊤R~h+1) = _h ( L_h^1 E_h+ L_h^2) R_h+1( L_h^1 E_h+ L_h^2) _h+ Tr(( L_h^3 F_hI_p,h+ L_h^4) _h( L_h^3 F_hI_p,h+ L_h^4) R_h+1) +Tr(Kh+14E~h+1�0,hE~h+1⊤(Kh+14)⊤R~h+1)+Tr(Kh+14�h+1y(Kh+14)⊤R~h+1)+c~h+1. + Tr(K_h+1^4 E_h+1 _0,h E_h+1 (K_h+1^4) R_h+1)+ Tr(K_h+1^4 _h+1^y(K_h+1^4) R_h+1)+ c_h+1. If we define the following quantities: Jh1(�~h):=�~h⊤(x,h⊤Q~h1x,h+ℰ^h⊤Q~h2ℰ^h+(L~h1ℰ^h+L~h2)⊤R~h+1(L~h1ℰ^h+L~h2))�~h, J_h^1( _h):= _h (I_x,h Q_h^1I_x,h+ E_h Q_h^2 E_h+( L_h^1 E_h+ L_h^2) R_h+1( L_h^1 E_h+ L_h^2)) _h, Jh2:=Tr(ℱ~hp,h�~hp,h⊤ℱ~h⊤Q~h2)+Tr((L~h3ℱ~hp,h+L~h4)�~h(L~h3ℱ~hp,h+L~h4)⊤R~h+1), J_h^2:= Tr( F_hI_p,h _hI_p,h F_h Q_h^2)+ Tr(( L_h^3 F_hI_p,h+ L_h^4) _h( L_h^3 F_hI_p,h+ L_h^4) R_h+1), Jh3:=Tr(x,h�~hx,h⊤Q~h1)+Tr(Kh+14E~h+1�0,hE~h+1⊤(Kh+14)⊤R~h+1)+Tr(Kh+14�h+1y(Kh+14)⊤R~h+1)+c~h+1, J_h^3:= Tr(I_x,h _hI_x,h Q_h^1)+ Tr(K_h+1^4 E_h+1 _0,h E_h+1 (K_h+1^4) R_h+1)+ Tr(K_h+1^4 _h+1^y(K_h+1^4) R_h+1)+ c_h+1, then we can write [~h⊤Q~h1~h+~h⊤Q~h2~h+V~h+1(�~h+1)|�~h,ℰ~h,ℱ~h]=Jh1(�~h)+Jh2+Jh3E[ X_h Q_h^1 X_h+ U_h Q_h^2 U_h+ V_h+1( _h+1)\,|\, _h, E_h, F_h]=J_h^1( _h)+J_h^2+J_h^3. Note that Jh1(�~h)J_h^1( _h) only depends on �~h _h and ℰ^h E_h, Jh2J_h^2 only depends on ℱ~h F_h, and Jh3J_h^3 is just a constant. Both minimizations below admit solutions. Indeed, the coefficient matrix Gh:=Q~h2+(L~h1)⊤R~h+1L~h1G_h:= Q_h^2+( L_h^1) R_h+1 L_h^1 is positive semidefinite and Lemma B.1 gives range((L~h1)⊤R~h+1L~h2)⊆range(Gh)range(( L_h^1) R_h+1 L_h^2) (G_h). Hence the canonical least-squares gain −Gh†(L~h1)⊤R~h+1L~h2-G_h ( L_h^1) R_h+1 L_h^2 minimizes the first quadratic for every mean vector simultaneously. The private-gain objective is a bounded-below finite-dimensional convex quadratic, so its normal equations are consistent and it also has a minimizer. Therefore, if ℰ^h∗,ℱ~h∗ E_h , F_h satisfy ℰ^h∗∈argminℰ^hJh1(�~h),∀�~h∈~h,ℱ~h∗∈argminℱ~hJh2, E_h ∈ argmin_ E_hJ_h^1( _h),~~∀ _h∈ S_h, F_h ∈ argmin_ F_hJ_h^2, then ℰ~h∗:=ℰ^h∗−ℱ~h∗p,h E _h:= E_h - F_h I_p,h and ℱ~h∗ F_h will satisfy (ℰ~h∗,ℱ~h∗)∈argminℰ~h,ℱ~h[~h⊤Q~h1~h+~h⊤Q~h2~h+V~h+1(�~h+1)|�~h,ℰ~h,ℱ~h]. ( E_h , F_h )∈ argmin_ E_h, F_hE[ X_h Q_h^1 X_h+ U_h Q_h^2 U_h+ V_h+1( _h+1)\,|\, _h, E_h, F_h]. Firstly, we intend to minimize Jh1(�~h)J_h^1( _h) and find an optimal ℰ^h∗ E_h . Since Q~h1,Q~h2,R~h+1⪰0 Q_h^1, Q_h^2, R_h+1 0, we can write Jh1(�~h)=‖(Q~h1)12x,h�~h‖22+‖(Q~h2)12ℰ^h�~h‖22+‖R~h+112(L~h1ℰ^h+L~h2)�~h‖22, J_h^1( _h)=||( Q_h^1) 12I_x,h _h||_2^2+||( Q_h^2) 12 E_h _h||_2^2+|| R_h+1 12( L_h^1 E_h+ L_h^2) _h||_2^2, which is a convex function of ℰ^h E_h given �~h _h. Therefore, we can compute the optimal ℰ^h E_h via the first-order condition. Taking the gradient of Jh1(�~h)J_h^1( _h) with respect to ℰ^h E_h, we obtain ∇ℰ^hJh1(�~h)=(2(Q~h2+(L~h1)⊤R~h+1L~h1)ℰ^h+2(L~h1)⊤R~h+1L~h2)�~h�~h⊤. _ E_hJ_h^1( _h)=(2( Q_h^2+( L_h^1) R_h+1 L_h^1) E_h+2( L_h^1) R_h+1 L_h^2) _h _h . Since it holds that ker(Q~h2+(L~h1)⊤R~h+1L~h1)⊆ker((L~h2)⊤R~h+1L~h1) ( Q_h^2+( L_h^1) R_h+1 L_h^1) (( L_h^2) R_h+1 L_h^1) due to Lemma B.1, and Q~h2+(L~h1)⊤R~h+1L~h1⪰0 Q_h^2+( L_h^1) R_h+1 L_h^1 0, we know that this gradient is zero if we choose ℰ^h∗=−(Q~h2+(L~h1)⊤R~h+1L~h1)†(L~h1)⊤R~h+1L~h2. E_h =-( Q_h^2+( L_h^1) R_h+1 L_h^1) ( L_h^1) R_h+1 L_h^2. Then, we know that such a ℰ^h∗ E_h minimizes Jh1(�~h)J_h^1( _h), and we can compute R~h=x,h⊤Q~h1x,h+(ℰ^h∗)⊤Q~h2ℰ^h∗+(L~h1ℰ^h∗+L~h2)⊤R~h+1(L~h1ℰ^h∗+L~h2). R_h=I_x,h Q_h^1I_x,h+( E_h ) Q_h^2 E_h +( L_h^1 E_h + L_h^2) R_h+1( L_h^1 E_h + L_h^2). This is a sum of positive-semidefinite Gram terms, so R~h⪰0 R_h 0, closing the induction. Note that the use of pseudo-inverse here precisely corresponds to the proof of generalized Riccati Equations in [22, 4]. Secondly, we intend to minimize Jh2J_h^2 and find an optimal ℱ~h∗ F_h . Hence, the optimal ℱ~h∗ F_h can be found via the following optimization problem ℱ~h∗ F_h ∈argminℱ~h=diag(ℱ~1,h,⋯,ℱ~n,h)Tr(ℱ~hp,h�~hp,h⊤ℱ~h⊤Q~h2)+Tr((L~h3ℱ~hp,h+L~h4)�~h(L~h3ℱ~hp,h+L~h4)⊤R~h+1). ∈ argmin_ F_h= diag( F_1,h,·s, F_n,h) \ Tr( F_hI_p,h _hI_p,h F_h Q_h^2)+ Tr(( L_h^3 F_hI_p,h+ L_h^4) _h( L_h^3 F_hI_p,h+ L_h^4) R_h+1) \. (C.8) Then, plugging in the optimal ℱ~h∗ F_h , we can get c~h c_h =Tr(ℱ~h∗p,h�~hp,h⊤(ℱ~h∗)⊤Q~h2)+Tr((L~h3ℱ~h∗p,h+L~h4)�~h(L~h3ℱ~h∗p,h+L~h4)⊤R~h+1)+Tr(Kh+14E~h+1�0,hE~h+1⊤(Kh+14)⊤R~h+1) = Tr( F_h I_p,h _hI_p,h ( F_h ) Q_h^2)+ Tr(( L_h^3 F_h I_p,h+ L_h^4) _h( L_h^3 F_h I_p,h+ L_h^4) R_h+1)+ Tr(K_h+1^4 E_h+1 _0,h E_h+1 (K_h+1^4) R_h+1) +Tr(Kh+14�h+1y(Kh+14)⊤R~h+1)+Tr(x,h�~hx,h⊤Q~h1)+c~h+1, + Tr(K_h+1^4 _h+1^y(K_h+1^4) R_h+1)+ Tr(I_x,h _hI_x,h Q_h^1)+ c_h+1, which completes Part 2. Combining Part 1 and Part 2, we complete the proof. ∎ Solution of Equation (C.8): From the fact that �~h,Q~h2,R~h+1⪰0 _h, Q_h^2, R_h+1 0, we can write Jh2=‖(Q~h2)12ℱ~hp,h(�~h)12‖F2+‖(R~h+1)12(L~h3ℱ~hp,h+L~h4)(�~h)12‖F2, J_h^2=||( Q_h^2) 12 F_hI_p,h( _h) 12||_F^2+||( R_h+1) 12( L_h^3 F_hI_p,h+ L_h^4)( _h) 12||_F^2, where ‖X‖F||X||_F denotes the Frobenius norm of X for any matrix X. Therefore, we know that Jh2J_h^2 is a convex function with respect to ℱ~h F_h, and the optimal ℱ~h∗ F_h can be computed via the first-order condition. By taking the gradient of Jh2J_h^2 with respect to ℱ~h F_h, we obtain ∇ℱ~hJh2=2(Q~h2ℱ~hp,h�~hp,h⊤+(L~h3)⊤R~h+1L~h3ℱ~hp,h�~hp,h⊤+(L~h3)⊤R~h+1L~h4�~hp,h⊤). _ F_hJ_h^2=2( Q_h^2 F_hI_p,h _hI_p,h +( L_h^3) R_h+1 L_h^3 F_hI_p,h _hI_p,h +( L_h^3) R_h+1 L_h^4 _hI_p,h ). Then, we partition ∇ℱ~hJh2 _ F_hJ_h^2 into n×n× n blocks and set its diagonal blocks to zero, namely, let [∇ℱ~hJh2]i,i=[ _ F_hJ_h^2]_i,i=0 for all i∈[n]i∈[n]. Here the block partitions follow their natural row and column variables: [Q~h2]i,k[ Q_h^2]_i,k and [Th1]i,k[T_h^1]_i,k have dimensions dim(~i)×dim(~k) ( U_i)× ( U_k), [�^h]k,i[ _h]_k,i has dimension dim(~k)×dim(~i) ( P_k)× ( P_i), and [Th2]i,i[T_h^2]_i,i and [∇ℱ~hJh2]i,i[ _ F_hJ_h^2]_i,i have dimensions dim(~i)×dim(~i) ( U_i)× ( P_i). We define �^h:=p,h�~hp,h⊤ _h:=I_p,h _hI_p,h , Th1:=(L~h3)⊤R~h+1L~h3T_h^1:=( L_h^3) R_h+1 L_h^3, and Th2:=(L~h3)⊤R~h+1L~h4�~hp,h⊤T_h^2:=( L_h^3) R_h+1 L_h^4 _hI_p,h , where �^h _h is the covariance matrix of ~h P_h. Then, for any i∈[n]i∈[n], it holds [∇ℱ~hJh2]i,i=2(∑k=1n([Q~h2]i,k+[Th1]i,k)ℱ~k,h[�^h]k,i+[Th2]i,i), [ _ F_hJ_h^2]_i,i=2 ( _k=1^n ([ Q_h^2]_i,k+[T_h^1]_i,k ) F_k,h[ _h]_k,i+[T_h^2]_i,i ), and we can obtain the optimal ℱ~h F_h by solving the equations ∑k=1n([Q~h2]i,k+[Th1]i,k)ℱ~k,h[�^h]k,i+[Th2]i,i=. _k=1^n ([ Q_h^2]_i,k+[T_h^1]_i,k ) F_k,h[ _h]_k,i+[T_h^2]_i,i=0. Appendix D Deferred Details of §V For the backward recursion in this appendix only, we use a cost-free post-decision terminal convention: (H+1)−:=H+∪H,H+1,(H+1)−:=∅, C_(H+1)^-:= C_H^+∪\ U_H, X_H+1\, P_(H+1)^-:= , and the same convention with every quantity tilded for the strict expansion. Thus ~(H+1)−=δ~H+1 B_(H+1)^-= _ X_H+1, �~(H+1)−=~H+1 _(H+1)^-= X_H+1, and �~(H+1)−= _(H+1)^-=0. This is only terminal bookkeeping after all decisions and changes neither the feasible strategies nor the objective. D-A Deferred details of results For h∈[H]h∈[H] and M1:h∈ℳ1:hM_1:h _1:h, let �i,h(M1:h) _i,h(M_1:h) denote the admissible prescriptions γi,ha:i,h+(M1:h)→i _i,h^a:P_i,h^+(M_1:h) _i, and write γha _h^a and �ha(M1:h) _h^a(M_1:h) for their joint versions. Lemma D.1. At a realized common history, let μi,h(Pi,h−):=gi,hm(Ch−∪Pi,h−,M1:h−1) _i,h(P_i,h^-):=g_i,h^m(C_h^-∪ P_i,h^-,M_1:h-1) and let μh _h be the joint communication prescription. If D satisfies Assumptions I.1 and I.2, then h−=�h1((h−1)+,hb,γh−1a),h+=�h2(h−,ha,h,μh). B_h^-= _h^1( B_(h-1)^+, Z_h^b, _h-1^a), B_h^+= _h^2( B_h^-, Z_h^a, M_h, _h). (D.1) Under Assumption V.1, μh _h is constant in the private information; conditional on h M_h, it drops out, giving h+=�h2(h−,ha,h) B_h^+= _h^2( B_h^-, Z_h^a, M_h). Proof. Given (h−1)+ B_(h-1)^+, the fixed model maps, independent noises, and γh−1a _h-1^a determine the joint law of (h,h−,hb)( X_h, P_h^-, Z_h^b); conditioning on hb Z_h^b gives �h1 _h^1. For the post-sharing update, impose h=μh(h−) M_h= _h( P_h^-) and ha=ϕh(h,h−) Z_h^a= _h( M_h, P_h^-); conditioning on (h,ha)( M_h, Z_h^a) gives �h2(h−,ha,h,μh) _h^2( B_h^-, Z_h^a, M_h, _h). Under Assumption V.1, μh _h is constant in h− P_h^-, so conditioning on h M_h adds no private-information likelihood and μh _h may be omitted. ∎ Given any JCCO problem D with closed-loop communication strategies, we can expand it into another JCCO problem ~ D (with notation system ~~ ~). First, for any h∈[H]h∈[H], we define the set �h−:=(i,t)|i∈[n],t<h,i,t+⊆h− under any communication strategy, and Bi,t,0 _h^-:=\(i,t)\,|\,i∈[n],t<h, I_i,t^+ C_h^- under any communication strategy, and B_i,t 0\ and �h+:=(i,t)|i∈[n],t<h,i,t+⊆h+ under any communication strategy, and Bi,t,0 _h^+:=\(i,t)\,|\,i∈[n],t<h, I_i,t^+ C_h^+ under any communication strategy, and B_i,t 0\. The system variables are identified across the two problems; in particular, ~1:H+1:=1:H+1 X_1:H+1:= X_1:H+1, ~j,h:=j,h Y_j,h:= Y_j,h, and ~j,h:=j,h U_j,h:= U_j,h for every j∈[n]j∈[n] and h∈[H]h∈[H]. Then, we can expand D to ~ D as follows: for any h∈[H]h∈[H] ~h−=h−∪~i,t|i∈[n],t<h,(i,t)∈�h−,~h+=h+∪~i,t|i∈[n],t<h,(i,t)∈�h+, C_h^-= C_h^-∪\ U_i,t\,|\,i∈[n],t<h,(i,t)∈ _h^-\, C_h^+= C_h^+∪\ U_i,t\,|\,i∈[n],t<h,(i,t)∈ _h^+\, (D.2) ∀i∈[n],~i,h−=i,h−\~j,t|j∈[n],t<h,(j,t)∈�h−,~i,h+=i,h+\~j,t|j∈[n],t<h,(j,t)∈�h+. ∀ i∈[n], P_i,h^-= P_i,h^- \ U_j,t\,|\,j∈[n],t<h,(j,t)∈ _h^-\, P_i,h^+= P_i,h^+ \ U_j,t\,|\,j∈[n],t<h,(j,t)∈ _h^+\. Then, we have the following lemma. Lemma D.2. Let D be a JCCO problem with PN IS that satisfies Assumptions I.1, I.2, I.4, and V.1, and let ~ D be the JCCO expanded from D according to Equation (D.2). Then, ~ D satisfies Assumption I.1, and the two problems have the same set of achievable costs. Hence either problem admits a team-optimal strategy if and only if the other does. Moreover, there exists a function φc ^c such that for any team-optimal strategy g~1:Hm,∗,g~1:Ha,∗ g_1:H^m, , g_1:H^a, of ~ D, (g1:Hm,∗,g1:Ha,∗)=φc(g~1:Hm,∗,g~1:Ha,∗,)(g_1:H^m, ,g_1:H^a, )= ^c( g_1:H^m, , g_1:H^a, ,D) is a team-optimal strategy of D, and J(g1:Hm,∗,g1:Ha,∗)=J~(g~1:Hm,∗,g~1:Ha,∗)J_D(g_1:H^m, ,g_1:H^a, )=J_ D( g_1:H^m, , g_1:H^a, ). Proof. This proof consists of two Parts: Part 1: ~ D satisfies Assumption I.1; Part 2: we can construct optimal strategies of D from the optimal strategies of ~ D, and the optimal values of the two problems are the same. Part 1: To begin with, we discuss a property of the problem ~ D. For any i∈[n],h∈[H]i∈[n],h∈[H], if Bi,h=B_i,h=0, then from Assumption I.2, i,h<j,(h′)−,i,h<j,(h′)+,∀h′∈[H] with h′>h,j∈[n] U_i,h∉ I_j,(h )^-, U_i,h∉ I_j,(h )^+,∀ h ∈[H] with h >h,\ j∈[n]. Also from expansion, we know that ~i,h=i,h U_i,h= U_i,h will never be added into common information. Therefore, ~i,h<~j,(h′)−,~i,h<~j,(h′)+,∀h′∈[H] with h′>h,j∈[n] U_i,h∉ I_j,(h )^-, U_i,h∉ I_j,(h )^+,∀ h ∈[H] with h >h,\ j∈[n]. For h∈[H−1]h∈[H-1], if Bi,h,0B_i,h 0, Assumption I.4 gives j,ij≠ i such that i,h U_i,h influences j,h+1 Y_j,h+1 and hence j,(h+1)− I_j,(h+1)^-. Partial nestedness gives i,h−⊆j,(h+1)− I_i,h^- I_j,(h+1)^-. Since j,ij≠ i, every label private to agent i can enter agent j’s information only through common information due to Assumption I.1, and then it holds i,h−⊆(h+1)− I_i,h^- C_(h+1)^-. Moreover, i,h+∖i,h−⊆ha⊆(h+1)− I_i,h^+ I_i,h^- Z_h^a C_(h+1)^-, so i,h+⊆(h+1)− I_i,h^+ C_(h+1)^-. Consequently (i,h)∈�(h′)−∩�(h′)+(i,h)∈ _(h )^-∩ _(h )^+ for every h′>h >h, and the expansion adds ~i,h U_i,h to ~(h+1)− C_(h+1)^- immediately. No such argument is needed for h=Hh=H, because Equation (D.2) only adds actions with t<h≤Ht<h≤ H. Then, based on the property discussed above, we show that ~ D satisfies Assumption I.1. (a) Since D satisfies Assumption I.1, hb=χh((h−1)+,h−1,h) Z_h^b= _h( P_(h-1)^+, U_h-1, Y_h) for a fixed projection function χh _h. We claim that ~hb⊆~(h−1)+∪~h−1,~h Z_h^b P_(h-1)^+∪\ U_h-1, Y_h\: This is because it holds that hb⊆(h−1)+∪h−1,h Z_h^b P_(h-1)^+∪\ U_h-1, Y_h\; from the property discussed above, ~hb Z_h^b Z_h^b only consists of some control actions at timestep h−1h-1; from expansion, ((h−1)+\~(h−1)+)⊆~(h−1)+( P_(h-1)^+ P_(h-1)^+) C_(h-1)^+, so ((h−1)+\~(h−1)+)∩~hb=∅( P_(h-1)^+ P_(h-1)^+)∩ Z_h^b= . Also, since χh _h is a projection function, and the sets �h−,�h+ _h^-, _h^+ are predefined and are not affected by the realization of each random variable, there exists a fixed projection function χ~h χ_h such that ~hb=χ~h(~(h−1)+,~h−1,~h) Z_h^b= χ_h( P_(h-1)^+, U_h-1, Y_h). (b) For each realization Mi,hM_i,h, obtain ϕ~i,h(Mi,h,⋅) φ_i,h(M_i,h,·) from the projection ϕi,h(Mi,h,⋅) _i,h(M_i,h,·) by deleting exactly the labels already moved to ~h− C_h^- by the expansion. The retained labels lie in ~i,h− P_i,h^-, while the deleted set is fixed by �h−,�h+ _h^-, _h^+ and is independent of all realizations. Hence ϕ~i,h(Mi,h,⋅) φ_i,h(M_i,h,·) is a fixed projection and ~i,ha=ϕ~i,h(~i,h,~i,h−) Z_i,h^a= φ_i,h( M_i,h, P_i,h^-). Stacking over agents gives ~ha=ϕ~h(~h,~h−) Z_h^a= φ_h( M_h, P_h^-). (c) Since D satisfies Assumption I.1, for any i∈[n],i,h−=ζi,h(i,(h−1)+,i,h−1,i,h)i∈[n], P_i,h^-= _i,h( P_i,(h-1)^+, U_i,h-1, Y_i,h) for a fixed projection function ζi,h _i,h. We claim that ~i,h−⊆~i,(h−1)+∪~i,h−1,~i,h P_i,h^- P_i,(h-1)^+∪\ U_i,h-1, Y_i,h\: This is because it holds that i,h−⊆i,(h−1)+∪i,h−1,i,h P_i,h^- P_i,(h-1)^+∪\ U_i,h-1, Y_i,h\; from expansion, it holds that ~i,h−⊆i,h− P_i,h^- P_i,h^-, and for any t<ht<h such that (i,t)∈�(h−1)+(i,t)∈ _(h-1)^+, it must hold that (i,t)∈�h−(i,t)∈ _h^-, which means if ~i,t∈i,(h−1)+\~i,(h−1)+ U_i,t∈ P_i,(h-1)^+ P_i,(h-1)^+, then ~i,t∈i,h−\~i,h− U_i,t∈ P_i,h^- P_i,h^-, so ~i,t<~i,h− U_i,t∉ P_i,h^-. Also, since ζi,h _i,h is a projection function, and the sets �h−,�h+ _h^-, _h^+ are predefined and are not affected by the realization of each random variable, there exists a fixed projection function ζ~i,h ζ_i,h such that ~i,h−=ζ~i,h(~i,(h−1)+,~i,h−1,~i,h) P_i,h^-= ζ_i,h( P_i,(h-1)^+, U_i,h-1, Y_i,h). Therefore, there exists a function ζ~h ζ_h such that ~h−=ζ~h(~(h−1)+,~h−1,~h) P_h^-= ζ_h( P_(h-1)^+, U_h-1, Y_h). (d) For each i∈[n],~i,h+=~i,h−\~i,hai∈[n], P_i,h^+= P_i,h^- Z_i,h^a. (e) For each i∈[n]i∈[n], the construction of ~ D only adds some actions into ~h−,~h+ C_h^-, C_h^+ and does not change the observations. Thus ~i,h−⊆~i,h+ I_i,h^- I_i,h^+ and ~i,h∈~i,h− Y_i,h∈ I_i,h^- for h∈[H]h∈[H], while ~i,h+⊆~i,(h+1)− I_i,h^+ I_i,(h+1)^- for h∈[H−1]h∈[H-1]. Part 2: From the construction of ~ D, we know that the system dynamics and communication cost are the same for both D and ~ D, and it holds that i,h−⊆~i,h− I_i,h^- I_i,h^- and i,h+⊆~i,h+ I_i,h^+ I_i,h^+ for any i∈[n],h∈[H]i∈[n],h∈[H]. So the agents in ~ D have larger strategy spaces than those in D. Thus every cost achievable in D is achievable in ~ D. More strongly, for every feasible strategy (g~1:Hm,g~1:Ha)( g_1:H^m, g_1:H^a) of ~ D, we recursively construct a feasible strategy (g1:Hm,g1:Ha)(g_1:H^m,g_1:H^a) of D with the same samplewise communication and control actions. For h=1h=1 and any i∈[n]i∈[n], note that i,1−=~i,1−,i,1+=~i,1+ I_i,1^-= I_i,1^-, I_i,1^+= I_i,1^+ always holds. Then, set gi,1m:=g~i,1mg_i,1^m:= g_i,1^m and gi,1a:=g~i,1ag_i,1^a:= g_i,1^a. It is immediate that these two strategies output the same communication and control actions. Now assume that g~1:h−1m,g~1:h−1a g_1:h-1^m, g_1:h-1^a and g1:h−1m,g1:h−1ag_1:h-1^m,g_1:h-1^a output the same actions. The only additional variables in ~h− − C_h^- C_h^- are actions ~j,t U_j,t with j∈[n]j∈[n] and t<ht<h. For each such action, the expansion gives j,t+⊆h− I_j,t^+ C_h^- under any additional sharing. Meanwhile, since gj,tag_j,t^a outputs the same control action as g~j,ta g_j,t^a, we can recover ~j,t=gj,ta(j,t+,1:t) U_j,t=g_j,t^a( I_j,t^+, M_1:t). Therefore, we can construct ~h− C_h^- from h−,1:h−1 C_h^-, M_1:h-1 and g1:h−1ag_1:h-1^a by recovering all possible ~j,t∈~h− − U_j,t∈ C_h^- C_h^-. We then construct gi,hm(h−,1:h−1)=g~i,hm(~h−,~1:h−1)g_i,h^m( C_h^-, M_1:h-1)= g_i,h^m( C_h^-, M_1:h-1), and these two strategies will output the same communication action. Similarly, assume that g~1:hm,g~1:h−1a g_1:h^m, g_1:h-1^a and g1:hm,g1:h−1ag_1:h^m,g_1:h-1^a output the same actions. For any i, the only additional variables in ~i,h+ ,h+ I_i,h^+ I_i,h^+ are actions ~j,t U_j,t with j∈[n]j∈[n] and t<ht<h. For each such action, the expansion gives Bj,t,0B_j,t 0 and j,t+⊆h+ I_j,t^+ C_h^+ under any additional sharing. Meanwhile, since gj,tag_j,t^a outputs the same control action as g~j,ta g_j,t^a, we can recover ~j,t=gj,ta(j,t+,1:t) U_j,t=g_j,t^a( I_j,t^+, M_1:t). Therefore, we can construct ~i,h+ I_i,h^+ from i,h+,1:h I_i,h^+, M_1:h and g1:h−1ag_1:h-1^a by recovering all possible ~j,t∈~i,h+ ,h+ U_j,t∈ I_i,h^+ I_i,h^+. We then construct gi,ha(i,h+,1:h)=g~i,ha(~i,h+,~1:h)g_i,h^a( I_i,h^+, M_1:h)= g_i,h^a( I_i,h^+, M_1:h), and these two strategies will output the same control action. Recursion defines (g1:Hm,g1:Ha)=φc(g~1:Hm,g~1:Ha,)(g_1:H^m,g_1:H^a)= ^c( g_1:H^m, g_1:H^a,D). The two problems then have identical samplewise actions, dynamics, and costs, so J(g1:Hm,g1:Ha)=J~(g~1:Hm,g~1:Ha)J_D(g_1:H^m,g_1:H^a)=J_ D( g_1:H^m, g_1:H^a) for every feasible expanded strategy. Thus every cost achievable in ~ D is achievable in D. Together with the preceding inclusion, the achievable cost sets coincide, and team optimality and its existence transfer in both directions. ∎ Now, we introduce the following theorem as a full version of Theorem V.3. Theorem D.3 (Full version of Theorem V.3). Let D be a JCCO problem with PN IS that satisfies Assumptions I.1, I.2, I.4, and V.1, and let ~ D be the JCCO expanded from D. Then ~ D is a JCCO problem satisfying the SI-CIB condition, and Assumptions I.1, I.2, I.4, V.1. Moreover, for any h∈[H]h∈[H], ~h−,~h+ B_h^-, B_h^+ admit Gaussian distributions. Formally, ~h−=(�~h−,�~h−),~h+=(�~h+,�~h+) B_h^-=N( _h^-, _h^-), B_h^+=N( _h^+, _h^+), and �~h− _h^- = ~h−1(�~(h−1)+,~1:h−1,~hb),�~h+= ~h+1(�~h−,~1:h,~ha), = _h^-^1( _(h-1)^+, M_1:h-1, Z_h^b),~~ _h^+= _h^+^1( _h^-, M_1:h, Z_h^a), (D.3) �~h− _h^- = ~h−2(~1:h−1),�~h+= ~h+2(~1:h), = _h^-^2( M_1:h-1),~~ _h^+= _h^+^2( M_1:h), for some functions ~h−1, ~h+1, ~h−2, ~h+2 _h^-^1, _h^+^1, _h^-^2, _h^+^2 that do not depend on the strategies g~1:hm,g~1:ha g_1:h^m, g_1:h^a. Therefore, it holds that �~h−= ~h−3(~h−,~1:h−1),�~h+= ~h+3(~h+,~1:h), _h^-= _h^-^3( C_h^-, M_1:h-1), _h^+= _h^+^3( C_h^+, M_1:h), (D.4) for some functions ~h−3, ~h+3 _h^-^3, _h^+^3 that do not depend on the strategies g~1:hm,g~1:ha g_1:h^m, g_1:h^a. Proof. This proof consists of three Parts: Part 1: ~ D satisfies Assumptions I.1, I.2, I.4, and V.1; Part 2: ~ D satisfies the SI-CIB condition; Part 3: Beliefs ~h−,~h+ B_h^-, B_h^+ admit Gaussian distributions, and the mean and covariance satisfy Equation (D.3) and (D.4). Throughout this proof, untilded quantities belong to the original problem D, whereas tilded quantities belong to its strict expansion ~ D. Part 1: From Lemma D.2, we know that ~ D satisfies Assumption I.1; Since for any i∈[n],h∈[H],t<hi∈[n],h∈[H],t<h, we add control actions ~j,t U_j,t into common information at timestep h in the expansion procedure only if B~j,t=Bj,t,0 B_j,t=B_j,t 0, ~ D satisfies Assumption I.2; Since expansion procedure does not change the system dynamics and E~i,h=Ei,h E_i,h=E_i,h, ∀i∈[n],h∈[H]∀ i∈[n],h∈[H], ~ D satisfies Assumption I.4; After expansion, g~i,hm,∀i∈[n],h∈[H] g_i,h^m,∀ i∈[n],h∈[H] can still only take (~h−,~1:h−1)( C_h^-, M_1:h-1) as input, and ~ D satisfies Assumption V.1. Part 2: Condition on a reachable common history and message sequence. Communication rules then add no private-information likelihood. If B~i,t=0 B_i,t=0, Assumption I.2 makes the corresponding control irrelevant to later states and information. If B~i,t,0 B_i,t≠ 0, the argument in Part 1 gives ~i,t+⊆~(t+1)− I_i,t^+ C_(t+1)^-, and the strict expansion also places ~i,t U_i,t there; the control rule’s message-history input is among the separately conditioned messages. Thus past strategy rules impose only compatibility with conditioned variables and do not change either conditional belief, almost surely. Hence ~ D satisfies SI-CIB. Part 3: For a fixed message sequence, conditioning on the common history fixes the strategy inputs and outputs identified in Part 2. After these compatibility relations are removed, the remaining model variables are affine functions of the primitive Gaussian variables. Hence ~h− B_h^- and ~h+ B_h^+ are Gaussian almost surely, possibly singular. Conditioned control values affect only affine offsets, so the covariances depend only on ~1:h−1 M_1:h-1 and ~1:h M_1:h, respectively; thus �~h−= ~h−2(~1:h−1) _h^-= _h^-^2( M_1:h-1) and �~h+= ~h+2(~1:h) _h^+= _h^+^2( M_1:h). At H+1H+1, the synthetic terminal disclosure gives ~(H+1)−=δ~H+1 B_(H+1)^-= _ X_H+1, �~(H+1)−=~H+1 _(H+1)^-= X_H+1, and �~(H+1)−= _(H+1)^-=0. Because ~ D satisfies Assumptions I.1, I.2, and V.1, Lemma D.1 implies that for each h∈[H]h∈[H], ~h−=�~h1(~(h−1)+,~hb,γ~h−1a),~h+=�~h2(~h−,~ha,~h), B_h^-= _h^1( B_(h-1)^+, Z_h^b, γ_h-1^a), B_h^+= _h^2( B_h^-, Z_h^a, M_h), for some functions �~h1,�~h2 _h^1, _h^2. Meanwhile, we know that ~h−,~h+ B_h^-, B_h^+ are Gaussian distributions that can be characterized by the means �~h−,�~h+ _h^-, _h^+ and the covariances �~h−,�~h+ _h^-, _h^+, respectively. Then, it holds that (�~h−,�~h−)=�~h−3(�~(h−1)+,�~(h−1)+,~hb,γ~h−1a),(�~h+,�~h+)=�~h+3(�~h−,�~h−,~ha,~h)( _h^-, _h^-)= _h^-^3( _(h-1)^+, _(h-1)^+, Z_h^b, γ_h-1^a),( _h^+, _h^+)= _h^+^3( _h^-, _h^-, Z_h^a, M_h) for some functions �~h−3,�~h+3 _h^-^3, _h^+^3. From �~h−= ~h−2(~1:h−1),�~h+= ~h+2(~1:h) _h^-= _h^-^2( M_1:h-1), _h^+= _h^+^2( M_1:h), we have �~h−= ~h−1(�~(h−1)+,~1:h−1,~hb,γ~h−1a),�~h+= ~h+1(�~h−,~1:h,~ha), _h^-= _h^-^1( _(h-1)^+, M_1:h-1, Z_h^b, γ_h-1^a), _h^+= _h^+^1( _h^-, M_1:h, Z_h^a), for some functions ~h−1, ~h+1 _h^-^1, _h^+^1. Furthermore, since ~ D satisfies the SI-CIB condition, �~h− _h^- does not depend on γ~h−1a γ_h-1^a, and ~h−1, ~h+1 _h^-^1, _h^+^1 do not depend on the strategies (g~1:hm,g~1:ha)( g_1:h^m, g_1:h^a). Finally, since ~h−=(∪t=1h~tb)∪(∪t=1h−1~ta) C_h^-=( _t=1^h Z_t^b)∪( _t=1^h-1 Z_t^a), and ~h+=(∪t=1h~tb)∪(∪t=1h~ta) C_h^+=( _t=1^h Z_t^b)∪( _t=1^h Z_t^a), we know that �~h−= ~h−3(~h−,~1:h−1),�~h+= ~h+3(~h+,~1:h), _h^-= _h^-^3( C_h^-, M_1:h-1), _h^+= _h^+^3( C_h^+, M_1:h), for some functions ~h−3, ~h+3 _h^-^3, _h^+^3 that do not depend on the strategies g~1:hm,g~1:ha g_1:h^m, g_1:h^a. This completes the proof. ∎ For ~ D, the common-information coordinator states are (�~h−,~1:h−1)( _h^-, M_1:h-1) before sharing and (�~h+,~1:h)( _h^+, M_1:h) after sharing. Lemma D.1, SI-CIB, and the coordinator-policy correspondence give Algorithm 1. We assume that all displayed Bellman minima have choices defining admissible strategies. D-B Algorithm to solve PN JCCO with closed-loop communication strategies Expand D into ~ D by Equation (D.2). Under the preceding assumption, Theorem D.3 and Algorithm 1 yield a team-optimal strategy of ~ D, which Lemma D.2 maps to one of D. Algorithm 1 Dynamic Programming for SI-CIB JCCO with Closed-loop Communication Strategies 0: JCCO ~ D satisfying the SI-CIB condition and Assumptions I.1, I.2, I.4, and V.1. 1: for each M~1:H∈ℳ~1:H M_1:H∈ M_1:H, and each realization �~(H+1)−=X~H+1∈~ _(H+1)^-= X_H+1∈ X do 2: V~(H+1)−(�~(H+1)−,M~1:H)←�~(H+1)−⊤Q~H+11�~(H+1)− V_(H+1)^-( _(H+1)^-, M_1:H)← _(H+1)^- Q_H+1^1 _(H+1)^- 3: end for 4: for h=Hh=H to 1 do 5: for each M~1:h∈ℳ~1:h M_1:h∈ M_1:h, and each realization �~h+∈~×~h+(M~1:h) _h^+∈ X× P_h^+( M_1:h) do 6: Choose γ~ha,∗∈argminγ~ha∈�~ha(M~1:h) γ_h^a, ∈ argmin_ γ_h^a∈ _h^a( M_1:h)[~h⊤Q~h1~h+~h⊤Q~h2~h+V~(h+1)−(�~(h+1)−,M~1:h)|�~h+=�~h+,~1:h=M~1:h,γ~ha] [ X_h Q_h^1 X_h+ U_h Q_h^2 U_h+ V_(h+1)^-( _(h+1)^-, M_1:h)\,|\, _h^+= _h^+, M_1:h= M_1:h, γ_h^a] 7: V~h+(�~h+,M~1:h)← V_h^+( _h^+, M_1:h)←[~h⊤Q~h1~h+~h⊤Q~h2~h+V~(h+1)−(�~(h+1)−,M~1:h)|�~h+=�~h+,~1:h=M~1:h,γ~ha,∗] [ X_h Q_h^1 X_h+ U_h Q_h^2 U_h+ V_(h+1)^-( _(h+1)^-, M_1:h)\,|\, _h^+= _h^+, M_1:h= M_1:h, γ_h^a, ] 8: for each C~h+ C_h^+ such that ~h+3(C~h+,M~1:h)=�~h+ _h^+^3( C_h^+, M_1:h)= _h^+ do 9: for i∈[n]i∈[n] do 10: g~i,ha,∗(C~h+,⋅,M~1:h)←γ~i,ha,∗(⋅) g_i,h^a, ( C_h^+,·, M_1:h)← γ_i,h^a, (·) 11: end for 12: end for 13: end for 14: for each M~1:h−1∈ℳ~1:h−1 M_1:h-1∈ M_1:h-1, and each realization �~h−∈~×~h−(M~1:h−1) _h^-∈ X× P_h^-( M_1:h-1) do 15: Choose M~h∗∈argminM~h∈ℳ~h[~h(M~h)+V~h+(�~h+,M~1:h)|�~h−=�~h−,~1:h−1=M~1:h−1,~h=M~h] M_h ∈ argmin_ M_h∈ M_hE[ K_h( M_h)+ V_h^+( _h^+, M_1:h)\,|\, _h^-= _h^-, M_1:h-1= M_1:h-1, M_h= M_h] 16: V~h−(�~h−,M~1:h−1)←[~h(M~h∗)+V~h+(�~h+,(M~1:h−1,M~h∗))|�~h−=�~h−,~1:h−1=M~1:h−1,~h=M~h∗] V_h^-( _h^-, M_1:h-1) [ K_h( M_h )+ V_h^+( _h^+,( M_1:h-1, M_h ))\,|\, _h^-= _h^-, M_1:h-1= M_1:h-1, M_h= M_h ] 17: for each C~h− C_h^- such that ~h−3(C~h−,M~1:h−1)=�~h− _h^-^3( C_h^-, M_1:h-1)= _h^- do 18: for i∈[n]i∈[n] do 19: g~i,hm,∗(C~h−,M~1:h−1)←M~i,h∗ g_i,h^m, ( C_h^-, M_1:h-1)← M_i,h 20: end for 21: end for 22: end for 23: end for 24: return (g~1:Hm,∗,g~1:Ha,∗)( g^m, _1:H, g^a, _1:H)