Paper deep dive

Test-Time Collaborative Classification over Multi-Agent Networks

Ping Hu, Mert Kayaalp, Ali H. Sayed

Year: 2026Venue: arXiv preprintArea: cs.MAType: PreprintEmbeddings: 255

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 8/26/2026, 3:44:04 AM

Summary

The paper proposes a framework for distributed binary classification over multi-agent networks where agents train local models independently and collaborate during test-time inference. By exchanging local decision statistics via a DeGroot-based distributed learning protocol, agents form collective predictions. The study provides theoretical guarantees on classification error and generalization bounds under various communication constraints (finite rounds, finite precision) and model heterogeneity.

Entities (10)

Ali H. Sayed · person · 99%Mert Kayaalp · person · 99%Ping Hu · person · 99%Test-Time Collaborative Classification · method · 98%EPFL · organization · 95%IDSIA · organization · 95%Multi-Agent Networks · system · 95%DeGroot Model · algorithm · 92%Cooperative Inference · process · 90%Independent Training · process · 90%

Relation Signals (7)

Ping Hu → affiliatedwith → EPFL

confidence 95% · Ping Hu and Ali H. Sayed are with the School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL)

Ali H. Sayed → affiliatedwith → EPFL

confidence 95% · Ping Hu and Ali H. Sayed are with the School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL)

Mert Kayaalp → affiliatedwith → IDSIA

confidence 95% · Mert Kayaalp is now with the DTI, SUPSI, Dalle Molle Institute for Artificial Intelligence (IDSIA USI-SUPSI)

Test-Time Collaborative Classification → appliesto → Multi-Agent Networks

confidence 95% · distributed binary classification over multi-agent networks

Test-Time Collaborative Classification → uses → DeGroot Model

confidence 92% · adopt the classical DeGroot model [18] as an interpretable and analytically tractable mechanism for information aggregation during inference

Test-Time Collaborative Classification → combines → Independent Training

confidence 90% · proposes a collective prediction framework... which combines independent local training with collaborative inference

Test-Time Collaborative Classification → combines → Cooperative Inference

confidence 90% · proposes a collective prediction framework... which combines independent local training with collaborative inference

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The increasing heterogeneity of multi-agent systems poses significant challenges for jointly training a global model across agents. At the same time, cooperative inference between agents has long been recognized as a powerful mechanism for distributed decision making over networks. Motivated by these observations, we propose a collaboration framework for distributed binary classification over multi-agent networks, where a set of independently trained agents, potentially differing in architecture, feature space, or modality, coordinate their actions during test time to form collective predictions. This coordination is achieved by exchanging local decision statistics through a distributed learning protocol. We develop a theoretical and experimental study of this independent training and cooperative inference paradigm, and examine its performance under different communication budgets and distributed learning rules. We establish classification error guarantees under sufficient, finite-round, and finite-precision communication, together with PAC-style generalization bounds. These results capture the influence of model heterogeneity, network topology, combination policy, and communication constraints on prediction accuracy. Taken together with the experimental results, they reveal both the price of independent training and the benefit of collective prediction for the proposed distributed decision making framework with models learned from data.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

251,589 characters extracted from source content.

Expand or collapse full text

Test-Time Collaborative Classification over Multi-Agent Networks Ping Hu Mert Kayaalp and Ali H. Sayed †thanks: Ping Hu and Ali H. Sayed are with the School of Engineering, École Polytechnique Fédérale de Lausanne (EPFL), 1015 Lausanne, Switzerland (email: ping.hu@epfl.ch; ali.sayed@epfl.ch). Mert Kayaalp is now with the DTI, SUPSI, Dalle Molle Institute for Artificial Intelligence (IDSIA USI-SUPSI), 6962 Lugano, Switzerland, where he is affiliated with the UBS-IDSIA AI Lab (email: mert.kayaalp@idsia.ch). This work was done while he was a Ph.D. student and Post-doc at the Adaptive Systems Laboratory at EPFL. Mert Kayaalp’s work was supported by UBS Switzerland AG and its affiliates through the UBS-IDSIA AI Lab. A preliminary version of this work was presented at the Algorithmic Collective Action Workshop, NeurIPS 2025. Abstract The increasing heterogeneity of multi-agent systems poses significant challenges for jointly training a global model across agents. At the same time, cooperative inference between agents has long been recognized as a powerful mechanism for distributed decision making over networks. Motivated by these observations, we propose a collaboration framework for distributed binary classification over multi-agent networks, where a set of independently trained agents, potentially differing in architecture, feature space, or modality, coordinate their actions during test time to form collective predictions. This coordination is achieved by exchanging local decision statistics through a distributed learning protocol. We develop a theoretical and experimental study of this independent training and cooperative inference paradigm, and examine its performance under different communication budgets and distributed learning rules. We establish classification error guarantees under sufficient, finite-round, and finite-precision communication, together with PAC-style generalization bounds. These results capture the influence of model heterogeneity, network topology, combination policy, and communication constraints on prediction accuracy. Taken together with the experimental results, they reveal both the price of independent training and the benefit of collective prediction for the proposed distributed decision making framework with models learned from data. Index Terms: Distributed learning and inference, collective intelligence, classification, probability of error, ensemble learning I Introduction Distributed learning and inference is a key paradigm for large-scale and distributed systems, where data is generated and processed across multiple agents.11 1 Throughout this paper, we use the term “distributed” to describe systems without a central coordinator. Closely related architectures are also termed “decentralized” in the machine learning literature. Within this context, two major lines of research have emerged: distributed inference, extensively studied in the statistics, signal processing, and control communities [1, 2, 3], and distributed machine learning, primarily advanced within the machine learning (ML) and data science communities [4, 5]. In distributed inference, a collection of agents is connected through a communication network. Each agent owns a family of statistical models that describes the distribution of its observations under a common, shared latent variable (e.g., a hidden state or hypothesis). The collective objective is to infer this latent variable through local computations and communication. To this end, agents iteratively update their beliefs or estimates by combining private observations with information received from their neighbors. Classical instances of this paradigm include distributed estimation [2, 1, 3], distributed detection [6, 7, 8], and social learning [9]. A key feature of these problems is that collaboration takes place during the inference phase. In contrast, distributed ML has mainly focused on collaboration during the training phase, as exemplified by federated learning and decentralized learning. In this setting, agents jointly train a global model on distributed data, which is later deployed for inference either centrally or across devices. The inference stage is thus governed by a single unified model, and the system operates as a single decision maker. This is fundamentally different from distributed inference, where each agent acts as an autonomous decision maker, and collaboration serves to improve the quality of individual or consensus predictions. While these two research lines have evolved largely independently, they share some core motivation (e.g., scalability, decentralization, and privacy), and often employ similar tools, particularly those from distributed optimization [10, 11]. Recently, efforts have been made to bridge the two fields in order to leverage their complementary benefits. A promising line of work is inference with machine-learned models, as studied in [12, 13, 14]. In [12], this idea is examined in the context of hypothesis testing within a centralized setting involving a single agent. More relevantly, [13] and [14] study the social learning task where multiple agents train local classifiers on their own data to extract discriminative information for inference in the absence of explicit statistical models. In both cases, inference is performed using a stream of unlabeled observations. In this work, we consider instead a distributed binary classification setting in which each agent has access to only a single local testing sample associated with a common, unknown class label. The agents are collectively tasked with identifying this label from their own observations. This setting naturally arises in applications such as sensor fusion (e.g., vehicles capturing different views of the same object) and multi-view classification (e.g., image and text modalities of the same entity). The goal of this work is to develop a distributed classification framework for this setting. I-A Our work To this end, we propose a collective prediction framework for distributed classification over multi-agent networks, which combines independent local training with collaborative inference. Specifically, during the training phase, each agent trains a local classifier using its own dataset. This independent training process is motivated by several trends and challenges in modern ML systems. First, it has become increasingly common in large-scale and privacy-sensitive applications, such as edge computing [15] and foundation models [16]. For example, large language models or domain-specific experts may be trained independently across different entities (e.g., companies, institutions, clients, or devices) and later coordinated at inference time [17]. Second, feature and model heterogeneity often make joint training impractical. For example, agents may process different input modalities or rely on distinct model architectures tailored to their local data. During the testing phase, each agent receives a private testing sample and participates in a distributed inference protocol to infer the shared class label by exchanging predictive information with its neighbors. This collaborative framework introduces several challenges that are not present in the standard supervised classification setting: (i) How should agents aggregate predictions at test time? Unlike standard ML pipelines that rely on a single trained model, each agent in our setting is equipped with an independently trained local model. This raises the question of how agents can effectively aggregate their individual predictions during inference. (i) How does agent heterogeneity govern the generalization performance? While classical supervised generalization theory typically analyzes a single learned predictor, our setting involves collective predictions based on multiple independently trained local classifiers. This raises the question of how agents’ inherent heterogeneity and inter-agent interactions shape the generalization behavior of the entire distributed system. (i) How does limited communication affect prediction? Collective prediction typically relies on sufficient information exchange. When communication is limited in either the number of rounds or message precision, the resulting predictions may deviate from the sufficient-communication limit. This raises the question of how communication constraints affect the individual prediction accuracy. To introduce our framework, we assume that agents produce soft predictions using their local classifiers and adopt the classical DeGroot model [18] as an interpretable and analytically tractable mechanism for information aggregation during inference. In this model, agents iteratively update their predictions by taking a weighted average of their neighbors’ predictions according to a combination policy defined over a fixed communication topology [18]. The DeGroot model has been extensively studied in the distributed inference literature and is well known for achieving consensus in multi-agent systems [19, 20]. DeGroot updating has also been examined experimentally in human social networks [21]. Moreover, its linear update structure makes it particularly well-suited for distributed implementation with finite communication rounds. The novelty of the present work is not in proposing a new information aggregation rule, but in formulating and analyzing this inference-time collaboration framework. Accordingly, this work develops a theoretical and experimental study with the following contributions: (i) We establish theoretical guarantees for distributed classification over multi-agent networks under the independent training and collaborative inference paradigm. The derived bounds characterize how network structure, local model quality, and data heterogeneity influence predictive accuracy. (i) We analyze how communication constraints affect classification, deriving agent-dependent bounds for both finite-round and finite-precision communication. These results characterize how incomplete consensus, network topology, combination policy, and communication precision influence prediction accuracy. (i) We evaluate the proposed framework on two complementary benchmarks and empirically demonstrate the benefits of cooperative inference under different observation settings. These contributions provide theoretical foundations for test-time collaboration in distributed classification and highlight connections between distributed inference and distributed ML. I-B Related Literature I-B1 Distributed machine learning Most existing work in this domain has focused on collaboratively training a shared global model using data distributed across multiple devices or computational nodes. Two dominant approaches are federated learning, which relies on a central server to aggregate locally trained updates [22, 23], and decentralized learning, which dispenses with the server and synchronizes models through peer-to-peer communication over a fixed network graph [24, 25]. Variants of these methods address challenges such as non-i.i.d. data [26, 27], agent sampling [28], personalization [29], and communication efficiency during training [30, 31]. Despite their topological differences, in their canonical forms, both paradigms aim to produce a shared global model during training. At inference time, this model is either executed on a central server or deployed across devices, so the system effectively operates as a centralized decision maker, even when implementation is distributed. These joint-training frameworks are therefore complementary to the setting studied here: they aim to produce a shared global model during training, whereas our work focuses on inference-time collaboration among independently trained local models. A small body of work has explored collaboration during inference, where a set of independently trained models exchange information with each other to improve accuracy via trust-score design [32, 33]. Yet they all assume that all agents observe the same test input, which restricts agents’ heterogeneity to model parameters or inductive biases. This setup closely parallels ensemble learning and multiple classifier combination discussed later. In contrast, we study a setting in which each agent receives its own private testing sample, and these samples may be statistically dependent across agents. This formulation naturally accommodates heterogeneous feature spaces (e.g., multimodal sensors or domain-specific models) and enables distributed decision-making without feature alignment. Collaborative inference in this setting has received little attention, and our work provides a formal theoretical and experimental treatment. I-B2 Distributed inference Distributed inference studies how a network of agents collaborates to estimate a latent variable based on private observations and local communication [2, 1, 3]. Applications include state estimation in smart power grids, cooperative perception in multi-robot systems, and opinion dynamics in social networks. These applications motivate different modeling frameworks depending on the nature of the latent variable. For discrete hypothesis spaces, a widely studied paradigm is social learning (SL), in which agents iteratively exchange local beliefs to identify the true hypothesis. SL naturally accommodates heterogeneous agents, since each agent may observe different types of signals modeled by distinct statistical distributions [9]. Classical SL methods assume that agents possess a family of likelihood models, which are used in updating beliefs via Bayes’ rule (or suitable variants) [34, 35, 36, 37, 38]. While analytically elegant, this assumption is restrictive in real-world problems, where data distributions are complex and rarely admit closed-form models. To address this limitation, [13] proposed a social machine learning strategy, where each agent first trains a local classifier using its own labeled examples, and then performs the SL rule based on the trained classifier. The asymptotic and non-asymptotic performance of this strategy has been examined in [13, 14]. Our work follows this data-driven perspective but differs in the task formulation. In SL, agents typically process streams of unlabeled i.i.d. data to infer a shared latent hypothesis. In the supervised classification setting studied here, each agent receives only a single testing sample for collective prediction, and collaboration takes place through a finite-round exchange of learned local decision statistics at test time. This difference changes the operational role of communication and leads to analytical questions that are not directly addressed in existing SL studies or in classical single-agent supervised classification. I-B3 Multiple classifier combination This is a classical line of research in ML that studies how to aggregate the predictions of multiple models to improve performance. While our framework is distributed and collaborative in nature, it bears a strong conceptual connection to this literature. Specifically, when all agents follow the DeGroot model for information aggregation at test time, their predictions converge (in the limit of infinite communication) to a consensus that is a weighted average of the local classifier outputs. This resembles the effect of a centralized fusion rule, a central topic in the literature on multiple classifier combination. Classical techniques in this area include voting schemes, bagging, boosting, and stacking [39, 40]. These approaches form the foundation of what is often referred to as ensemble learning, where multiple base classifiers are combined to improve generalization and robustness [41, 42]. More advanced strategies, such as mixture-of-experts models, combine classifiers through input-dependent gating [16]. These approaches, however, typically operate in a centralized setting during test time and assume a common feature representation across experts. In contrast, our framework operates in a fully-distributed environment during both training and testing phases. When communication is limited to a finite number of rounds, consensus may not be reached, and predictions remain agent-dependent. This dynamic, iterative process differs fundamentally from the static, one-shot aggregation used in most ensemble methods. Notation: We use boldface font to denote random variables and normal font for their realizations, e.g., h and h. E and ℙP denote the expectation and probability operators, respectively. [⋅] 1[·] denotes the indicator function, and 1 denotes the all-ones vector. I Problem Formulation We are given a network of K agents indexed by k and a binary classification task with class label γ. The sets of agents and labels are denoted by ≜1,2,…,KK \1,2,…,K\ and Γ≜+1,−1 \+1,-1\, respectively. We assume that the semantic meaning of the labels is fixed and shared across the network during both training and inference. Each agent k holds a local training set kD_k of NkN_k labeled examples (k,n,k,n)( h_k,n, γ_k,n), where k,n h_k,n is the n-th feature vector and k,n∈Γ γ_k,n∈ is its label. Let ℋkH_k denote the feature space of agent k. A network observation consists of one local view from each agent and therefore takes values in the product space ℋ≜ℋ1×⋯×ℋK.H _1×·s×H_K. (1) For each γ∈Γγ∈ , let PγP_γ denote the class-conditional distribution of a network observation =(1,…,K)∈ℋ h=( h_1,…, h_K) , and let Pγ,kP_γ,k denote its k-th marginal. When this marginal admits a density or probability mass function, we denote it by pk(h|γ)p_k(h|γ). For the binary classification task, we assume a uniform class prior. Then, PγP_γ induces the joint distribution Q of (,)( h, γ): ℙQ(=γ)=12,∣=γ∼Pγ,γ∈Γ.P_Q( γ=γ)= 12, h \ γ=γ\ P_γ, γ∈ . (2) We denote by QkQ_k the corresponding marginal distribution of (k,)( h_k, γ) at agent k. Conditional on γ, the local observations in h may be statistically dependent across agents. The agents are heterogeneous as they may have different feature spaces ℋkH_k (e.g., different views in multi-view learning) or distinct class-conditional models pk(h|γ)p_k(h|γ) (e.g., different local distributions in personalized federated learning). The posterior log-ratio is sufficient for binary decision making. Under the uniform prior in (2), Bayes’ rule gives Λk(h)≜log⁡pk(+1|h)pk(−1|h)=log⁡pk(h|+1)pk(h|−1),∀h∈ℋk _k(h) p_k(+1|h)p_k(-1|h)= p_k(h|\!+1)p_k(h|\!-1), ∀ h _k (3) where pk(γ|h)p_k(γ|h) denotes the posterior probability. The function Λk _k is also known as the logit function in the literature. I-A The Training Phase In the training phase, each agent k uses its own dataset kD_k to train a local classifier. For each agent k, the samples in kD_k are independently drawn according to QkQ_k. Moreover, samples with the same index n may be statistically dependent across agents, while the corresponding cross-agent sample collections are independent across indices. We denote by fk:ℋk→ℝf_k:H_k a generic real-valued score function from an admissible function class ℱkF_k. The function fkf_k serves as an approximation to the logit function Λk _k in (3). Formally, fk(h)=log⁡p^k(+1|h)p^k(−1|h)f_k(h)= p_k(+1|h) p_k(-1|h) (4) where p^k(γ|h) p_k(γ|h) denotes the estimated posterior probability. Given kD_k, the empirical risk associated with a candidate score function fk∈ℱkf_k _k is k,(fk)≜1Nk∑n=1NkΦ⁡(k,nfk(k,n)) R_k, emp(f_k) 1N_k _n=1^N_k ( γ_k,nf_k( h_k,n)) (5) where Φ denotes the loss function used for training. Let kA_k denote the local training algorithm at agent k and let kS_k collect its internal randomness. The resulting trained score function is denoted by ^k≜k(k,k)∈ℱk. f_k _k(D_k,S_k) _k. (6) We measure the optimization accuracy of ^k f_k by its empirical optimality gap, namely, the difference between its empirical risk and the infimum empirical risk over ℱkF_k: Gapk,opt(^k)≜k,(^k)−inffk∈ℱk,(fk)≥0.Gap_k,opt( f_k) R_k, emp( f_k)- _f_k _k R_k, emp(f_k)≥ 0. (7) A zero gap corresponds to exact empirical risk minimization (ERM), while a positive gap quantifies the residual optimization error of the trained score function. For the theoretical analysis, we use the following assumptions on the loss function Φ and the function class ℱkF_k, which are commonly adopted in statistical learning theory [43, 44]. Assumption 1 (Conditions on the loss function). The loss function Φ:ℝ→ℝ+ :R _+ is convex, non-increasing and differentiable at 00 with Φ′(0)<0 (0)<0. Also, it is LΦL_ -Lipschitz. ∎ Assumption 2 (Boundedness of functions). There exists a constant β>0β>0 such that |fk(h)|≤β|f_k(h)|≤β for every k∈k , fk∈ℱkf_k _k, and h∈ℋkh _k. ∎ The training phase is completed by empirically centering the learned score ^k f_k, following [13, 14]. Since independently trained local classifiers may exhibit different empirical offsets, we subtract each score’s empirical training mean to better align the local outputs before collaboration. Specifically, we define the empirical training mean as k,(fk)≜1Nk∑n=1Nkfk(k,n),∀fk∈ℱk. μ_k, emp(f_k) 1N_k _n=1^N_kf_k( h_k,n), ∀ f_k _k. (8) The resulting classifier generates a centered score: k(h)≜^k(h)−k,(^k),∀h∈ℋk. c_k(h) f_k(h)- μ_k, emp( f_k), ∀ h _k. (9) Therefore, after training, the sign of k(h) c_k(h) determines the local prediction at agent k for any unseen feature vector h∈ℋkh _k. The role of empirical centering is examined in Section I-A. I-B The Prediction Phase In the prediction phase, the network receives a fresh testing example (∗,∗)( h^*, γ^*) drawn from Q independently of the training data and local training-algorithm randomness, with each agent k observing only its local component k∗∈ℋk h_k^* _k. These local views may remain statistically dependent conditioned on ∗ γ^*. The goal of the agents is to make a collective prediction about ∗ γ^* by collaborating with their neighbors. We use t to index the communication rounds and denote by k,t λ_k,t the decision statistic of agent k at round t, whose sign determines its prediction. Following the DeGroot model, the agents iteratively combine their decision statistics over the network. Upon observing k∗ h_k^*, agent k initializes its decision statistic using the local classifier k c_k: k,0≜k(k∗). λ_k,0 c_k( h_k^*). (10) Then, each agent communicates with its neighbors and updates its decision statistic by aggregating the local decision statistics in the neighborhood [18]: k,t=∑ℓ=1Kaℓkℓ,t−1 λ_k,t= _ =1^Ka_ k λ_ ,t-1 (11) where aℓka_ k denotes the combination weight agent k assigns to its neighbor ℓ , which satisfies ∑ℓ=1Kaℓk=1,aℓk>0∀ℓ∈k, and aℓk=0∀ℓ∉k _ =1^Ka_ k=1,\;\;a_ k>0\;\;∀ _k, and a_ k=0\;\;∀ _k (12) with kN_k denoting the neighboring set of agent k. Communication proceeds in synchronous rounds over this fixed graph, and the exchanged messages are assumed to be delivered reliably. Accordingly, the communication model considered here does not include asynchronous updates, time-varying connectivity, or packet losses. Moreover, we impose the following assumption on the topology of the communication network to ensure information diffusion throughout the network. Assumption 3 (Strongly-connected graph). The graph of the communication network is strongly connected. That is, there exist paths with positive combination weights between any two distinct agents in both directions (the two paths need not be the same), and at least one agent has a self-loop, i.e., akk>0a_k>0 for some agent k [10]. ∎ Under this assumption and from the Perron-Frobenius theorem [45], the combination matrix A=[aℓk]A=[a_ k] is primitive and admits a Perron vector π satisfying Aπ=π,∑k=1Kπk=1, and πk>0,∀k∈.Aπ=π,\;\; _k=1^K _k=1, and _k>0,\;\;∀ k . (13) The entries of π quantify the relative influence of the agents following the adopted combination policy. In network science terminology [46, 47], this Perron vector corresponds to the notion of eigenvector centrality associated with the matrix A. Moreover, the second-largest magnitude among all eigenvalues of A, denoted by σA _A, is strictly smaller than 1. To evaluate the performance of this framework, we first introduce a feasibility condition associated with the classification task, defined in terms of the target risk for training. For each agent k, the target risk ko R_k^o is the infimum of the expected risk over the local function class ℱkF_k: ko≜inffk∈ℱk(k,)∼QkΦ(fk(k)). R_k^o _f_k _kE_( h_k, γ) Q_k ( γf_k( h_k)). (14) The target risk for the network is defined as a weighted average of the individual target risks involving the Perron vector π: o≜∑k=1Kπkko. R^o _k=1^K _k R_k^o. (15) The reason for introducing π in defining o R^o will be clear in our subsequent analysis for the classification error. The feasibility of the classification task requires the following assumption. Assumption 4 (Feasibility). The network target risk satisfies o<Φ⁡(0) R^o< (0).∎ We note that Φ⁡(0) (0) represents the expected risk of the model fk=0f_k=0, which corresponds to the case where agent k assigns labels +1+1 and −1-1 with equal probability to all feature vectors. Hence, ko=Φ⁡(0) R_k^o= (0) implies that the feature vectors of agent k provide no useful information for classification within the prescribed function class ℱkF_k. Assumption 4 is a network-level feasibility condition: it does not require every agent to be individually informative, but only that the network target risk satisfies o<Φ⁡(0) R^o< (0). Thus, the collective observations available in the network must contain enough class-discriminative information to outperform random guessing. If this condition fails, then the classification problem is effectively uninformative at the network level. I Performance Guarantees This section develops four principal performance guarantees for the proposed framework: sufficient-communication classification, finite-round communication, finite-precision communication, and probably approximately correct (PAC)-style generalization. The analysis of the first three guarantees proceeds in two stages. We first characterize the training performance of the classifier network through an expected network margin. Two supporting propositions are derived for clarifying the role of empirical centering in (9) and establishing when approximate local training attains such a margin with high probability. We then use this common margin event to characterize the prediction error under the three communication settings. The final PAC-style theorem provides a complementary guarantee that relates population error directly to the performance observed on the training data. We conclude by clarifying the relation to conventional social learning recursions and summarizing the corresponding results without empirical centering. I-A Network Margin and Training Guarantee To define the expected network margin used throughout the analysis, we first identify the aggregate classifier obtained after sufficient communication. Following the DeGroot averaging rule (11), agents agree on a common decision statistic denoted by λ_ ave after sufficient communication [18]. That is, it holds almost surely (a.s.) that ≜limt→∞k,t=a.s.∑k=1Kπkk(k∗). λ_ ave _t→∞ λ_k,t a.s.= _k=1^K _k c_k( h_k^*). (16) Accordingly, the collective prediction under sufficient communication is ^≜sgn() γ ( λ_ ave) (17) where any fixed tie rule may be used at zero. Equivalently, the distributed rule (11) functions as an aggregate classifier: (∗)≜∑k=1Kπkk(k∗). c_ ave( h^*) _k=1^K _k c_k( h_k^*). (18) This aggregate classifier resembles a π-weighted soft decision combination of K independently trained local classifiers. The analysis of c_ ave is nontrivial because the informativeness of the local scores may vary across agents and the local views may be statistically dependent. Existing theoretical results on soft decision combination often rely on specific distributional assumptions on the local scores [48]. We therefore develop an analysis that accommodates these heterogeneous and statistically dependent local observations. Since the deployed local scores are empirically centered, we first examine how the empirical centering operation in (9) affects the aggregate score relative to the common zero decision threshold. For a generic collection of local score functions f=(f1,…,fK)∈ℱ≜ℱ1×⋯×ℱK,f=(f_1,…,f_K) _1×·s×F_K, (19) we define the uncentered aggregate score by f()≜∑k=1Kπkfk(k).f_ ave( h) _k=1^K _kf_k( h_k). (20) Let rawγ(f)≜(γ)f() μ_ raw^γ(f) ^(γ)f_ ave( h) denote its class-conditional mean, where (γ)E^(γ) denotes expectation under PγP_γ. We introduce the following network-level quantities associated with rawγ(f) μ_ raw^γ(f): μmid(f) _ mid(f) ≜raw+(f)+raw−(f)2, μ_ raw^+(f)+ μ_ raw^-(f)2, (21) D⁡(f) D(f) ≜raw+(f)−raw−(f)2. μ_ raw^+(f)- μ_ raw^-(f)2. (22) Moreover, we define the network empirical training mean by (f)≜∑k=1Kπkk,(fk). μ_ emp(f) _k=1^K _k μ_k, emp(f_k). (23) Under the uniform class prior, for any fixed f independent of the training samples, the training setup in Section I-A implies ⁡[(f)]=μmid(f)E[ μ_ emp(f)]= _ mid(f). Hence, μmid(f) _ mid(f) is the population counterpart of the network empirical training mean. With the above definitions, we establish the following proposition. Proposition 1 (Effect of additive centering). For arbitrary additive centers χk _k, let χπ≜∑kπkχk _π _k _k _k and define sχ,f()≜∑k=1Kπk[fk(k)−χk].s_χ,f( h) _k=1^K _k [f_k( h_k)- _k ]. (24) Its expected margin over both classes at threshold zero is Δχ(f) _χ(f) ≜min⁡(+1)sχ,f(),−(−1)sχ,f() \E^(+1)s_χ,f( h),-E^(-1)s_χ,f( h) \ =D⁡(f)−|χπ−μmid(f)|. =D(f)-| _π- _ mid(f)|. (25) Hence the population midpoint χπ=μmid(f) _π= _ mid(f) maximizes this margin. In particular, the margins under raw and empirically centered scores satisfy Δraw(f) _ raw(f) =D⁡(f)−|μmid(f)|, =D(f)-| _ mid(f)|, (26) Δcen(f) _ cen(f) =D⁡(f)−|(f)−μmid(f)|. =D(f)-| μ_ emp(f)- _ mid(f)|. (27) Therefore, empirical centering (9) improves the expected zero-threshold margin if and only if |(f)−μmid(f)|<|μmid(f)|.| μ_ emp(f)- _ mid(f)|<| _ mid(f)|. (28) Moreover, empirical centering is invariant to the agent-specific additive offsets: if fk′(h)=fk(h)+okf _k(h)=f_k(h)+o_k, then fk′(h)−k,(fk′)=fk(h)−k,(fk).f _k(h)- μ_k, emp(f _k)=f_k(h)- μ_k, emp(f_k). (29) Consequently, the centered DeGroot recursion (11) stays unchanged by these offsets at every communication round. Proof. See Appendix . ∎ For the learned classifiers k c_k from Section I-A, Proposition 1 applies with fk=^kf_k= f_k and χk=k,(^k) _k= μ_k, emp( f_k). It shows that empirical centering acts as a threshold alignment mechanism that changes the location of the aggregate score relative to the zero decision threshold while leaving the class-conditional mean separation 2D(^)2D( f) unchanged. Its effect on the expected zero-threshold margin depends on how closely the realized empirical training mean (^) μ_ emp( f) aligns with the corresponding population midpoint μmid(^) _ mid( f), as characterized by (28). Moreover, empirical centering removes agent-specific additive offsets from the local scores and hence from the subsequent collaborative recursion. We next characterize the expected network margin attained by the centered local classifiers. For any fk∈ℱkf_k _k, define its empirically centered score ck,fk(h)≜fk(h)−k,(fk),c_k,f_k(h) f_k(h)- μ_k, emp(f_k), (30) and the corresponding class-conditional means k+(fk)≜k(+1)ck,fk(k), μ_k^+(f_k) _ h_k^(+1)c_k,f_k( h_k), (31) k−(fk)≜k(−1)ck,fk(k), μ_k^-(f_k) _ h_k^(-1)c_k,f_k( h_k), (32) where k(γ)E_ h_k^(γ) denotes expectation under Pγ,kP_γ,k. The corresponding network means are +(f)=∑k=1Kπkk+(fk),−(f)=∑k=1Kπkk−(fk). μ^+(f)= _k=1^K _k μ_k^+(f_k), μ^-(f)= _k=1^K _k μ_k^-(f_k). (33) Before conditioning on the training phase, these quantities are random because the deployed local functions and empirical training means k,(fk) μ_k, emp(f_k) depend on the realized training data kD_k and, when applicable, local optimization randomness kS_k. The inclusion of the Perron vector π in (33) is consistent with the collective nature of the prediction phase, where the same weights determine the common decision statistic in (16). For a collection of learned functions ^=(^1,…,^K) f=( f_1,…, f_K), we say that the classifier network satisfies the δ-margin consistent training condition if +(^)>δand−(^)<−δ μ^+( f)>δ μ^-( f)<-δ (34) where δ≥0δ≥ 0 is the prescribed decision margin. In view of (18) and (30)–(33), this condition requires that the class-conditional mean of the aggregate score (∗) c_ ave( h^*) lies on the correct side of the zero decision threshold for both classes, with a margin exceeding δ. The performance of the training phase is then characterized by the probability that condition (34) holds for the classifier network. Define the corresponding event δ≜+(^)>δ,−(^)<−δ,C_δ \ μ^+( f)>δ,\, μ^-( f)<-δ \, (35) and let Pc,δ≜ℙ⁡(δ),P_c,δ (C_δ), (36) where the probability is taken over the randomness from training. Analyzing Pc,δP_c,δ is non-trivial, as it depends on various factors, including the size of the training sets, the complexity of the function class, the loss function, and the combination policy. A lower bound on Pc,δP_c,δ under exact ERM, i.e., zero optimization gap in (7), was established in [14] using tools from [13]. We next extend it to approximate local optimization. The auxiliary quantities NmaxN_ , α, ρ, EΦ(r,δ) E_ (r,δ), and δmax(r) _ (r) appearing below are defined in Appendix . Proposition 2 (δ-margin consistent training under approximate optimization). Suppose that, for every k, there exists a deterministic tolerance εk,≥0 _k, opt≥ 0 such that Gapk,(^k)≤εk,Gap_k, opt( f_k)≤ _k, opt (37) almost surely, and define ε≜∑k=1Kπkεk,. _ opt _k=1^K _k _k, opt. (38) If o+ε<Φ⁡(0) R^o+ _ opt< (0), 0≤δ<δmax(o+ε)0≤δ< _ ( R^o+ _ opt), and ρ<EΦ(o+ε,δ)ρ< E_ ( R^o+ _ opt,δ), then under Assumptions 1–4, Pc,δ≥1−2exp⁡−8Nmaxα2β2(EΦ(o+ε,δ)−ρ)2.P_c,δ≥ 1-2 \- 8N_ α^2β^2 ( E_ ( R^o+ _ opt,δ)-ρ )^2 \. (39) Proof. See Appendix . ∎ The key implication of Proposition 2 is that Pc,δP_c,δ admits an exponential guarantee when the network Rademacher complexity ρ is smaller than EΦ(o+ε,δ) E_ ( R^o+ _ opt,δ). The relevant properties of these auxiliary quantities are collected in Appendix . In particular, for a fixed feasible decision margin δ, EΦ(r,δ) E_ (r,δ) decreases as the risk argument r increases, while the feasible margin range characterized by δmax(r) _ (r) also becomes smaller. Consequently, increasing ε _ opt reduces EΦ(o+ε,δ) E_ ( R^o+ _ opt,δ), making the condition ρ<EΦ(o+ε,δ)ρ< E_ ( R^o+ _ opt,δ) more restrictive. Moreover, whenever this condition remains satisfied, the quantity EΦ(o+ε,δ)−ρ E_ ( R^o+ _ opt,δ)-ρ decreases, thereby reducing the exponential rate in (39). Thus, a larger optimization residual makes it more difficult to certify a prescribed decision margin and weakens the resulting training guarantee. Setting εk,=0 _k, opt=0 for all agents recovers the exact-ERM specialization in [14]. The bound also helps characterize the statistical cost of independent local training as reflected in the training guarantee, a defining feature of our framework when data pooling or joint training is unavailable. To quantify this cost, we compare independent training with a centralized benchmark under a fixed budget of available training samples. In the homogeneous setting considered in Appendix , under matched optimization accuracy and when the relevant Rademacher complexity decreases with the number of training samples as described there, specializing Proposition 2 shows that the centralized benchmark yields a tighter lower bound on Pc,δP_c,δ than an even split of the same samples among independently trained agents. The detailed comparison is provided in Appendix . I-B Sufficient-Communication Classification Guarantee We next turn to the classification performance of the trained network under sufficient communication. Section I-A characterizes the training performance through the probability that the learned network attains a prescribed decision margin δ. We now examine how this margin condition translates into the probability of classification error for a fresh testing example. To distinguish the randomness arising from training from that of prediction, let ≜σ⁡(1,…,K,1,…,K),T σ(D_1,…,D_K,S_1,…,S_K), (40) which collects the training data and the randomness of the local training algorithms. Conditional on T, the learned classifiers k c_k and their empirical training means are therefore fixed. Moreover, the testing sample (∗,∗)( h^*, γ^*) is independent of T. Since the training event δC_δ associated with (34) is determined by the training phase, it is T-measurable. From (17), under sufficient communication, the collective prediction γ is determined by the sign of the aggregate score (∗) c_ ave( h^*). Let ℳ≜∗(∗)≤0M \ γ^* c_ ave( h^*)≤ 0\ (41) denote the event that the signed aggregate score is nonpositive. Under any fixed tie-breaking rule at zero, the actual misclassification event is contained in ℳM. With Pe≜ℙ⁡(^≠∗)P_e ( γ≠ γ^*), the tower property and the T-measurability of δC_δ give Pe P_e ≤ℙ⁡(ℳ)=⁡[ℙ⁡(ℳ|)] (M)=E\! [P(M 1.0mu| 1.0muT) ] =⁡[[δ]ℙ(ℳ|)]+⁡[[δ¯]ℙ(ℳ|)] =E\! [ 1[C_δ]P(M 1.0mu| 1.0muT) ]+E\! [ 1[ C_δ]P(M 1.0mu| 1.0muT) ] ≤⁡[[δ]ℙ(ℳ|)]+ℙ⁡(δ¯). \! [ 1[C_δ]P(M 1.0mu| 1.0muT) ]+P( C_δ). (42) This decomposition separates the conditional prediction error for training realizations in δC_δ from the probability that training fails to attain the prescribed δ-margin condition in (34). The latter is characterized by Proposition 2. Based on the event δC_δ, the class-conditional mean of the aggregate score (∗) c_ ave( h^*) lies on the correct side of the zero decision threshold with a margin greater than δ. The remaining prediction error depends on the variability of the realized aggregate score (∗) c_ ave( h^*) around its corresponding class-conditional mean. The local observations 1∗,…,K∗ h_1^*,…, h_K^* are different views of the same testing example and may therefore remain statistically dependent even after conditioning on the class label. Rather than requiring conditional independence, we allow their joint class-conditional distribution PγP_γ to satisfy the following approximate tensorization property [49]. For a probability law P and a bounded nonnegative function G, define EntP⁡(G)≜P[Glog⁡G]−P[G]log⁡P[G].Ent_P(G) _P[G G]-E_P[G] _P[G]. (43) For each k, let −k h_-k collect all components of h except k h_k, and let Pγk|−k(⋅|−k)P_γ^k|-k(·| h_-k) denote the conditional law of k h_k given the remaining views under PγP_γ. Assumption 5 (Class-conditional approximate tensorization of entropy (ATE)). For every γ∈Γγ∈ , there exists τγ≥1 _γ≥ 1 such that EntPγ(G)≤τγ∑k=1KPγ[EntPγk|−k(⋅|−k)(G(⋅,−k))]Ent_P_γ(G)≤ _γ _k=1^KE_P_γ\! [Ent_P_γ^k|-k(·| h_-k) (G(·, h_-k) ) ] (44) for every bounded nonnegative measurable G.∎ Assumption 5 allows statistically dependent local views across agents beyond the product case. The coefficient τγ _γ serves as a dependence parameter for the ATE condition: product class-conditional distributions satisfy the condition with τγ=1 _γ=1, while larger values accommodate greater departures from this product structure. For convenience, define τmax≜maxγ∈Γ⁡τγ. _ _γ∈ _γ. (45) Together with the training guarantee in Proposition 2, the ATE condition (44) leads to the following sufficient-communication classification guarantee. Theorem 1 (Sufficient-communication classification guarantee). Suppose Assumptions 2, 3, and 5 hold, and let δ≥0δ≥ 0. If the agents label their testing samples k∗ h_k^* with γ according to (17), then, almost surely on δC_δ, ℙ⁡(ℳ|)≤exp⁡−δ22τmaxβ2∑k=1Kπk2.P(M 1.0mu| 1.0muT)≤ \- δ^22 _ β^2 _k=1^K _k^2 \. (46) If, in addition, the conditions of Proposition 2 hold, combining the two guarantees through (42) yields an explicit upper bound on PeP_e. Proof. See Appendix . ∎ Theorem 1 establishes the connection between the training performance and the reliability of subsequent collaborative prediction. On δC_δ, the conditional prediction error is controlled by the decision margin δ, the dependence factor τmax _ , and the Perron weights, while Proposition 2 characterizes the probability of attaining this favorable training event. Through (42), the two guarantees jointly characterize the overall classification performance. We discuss several implications of this result. I-B1 Relation to margin theory The decision margin δ plays complementary roles in training and prediction. Proposition 2 shows that requiring a larger δ makes the training condition more difficult to attain, whereas Theorem 1 shows that, once it is attained, a larger δ yields a stronger conditional prediction guarantee. This tradeoff is reminiscent of the role of margins in classical margin-based generalization theory [44]. Given the classifier c_ ave from (18), the quantity ∗(∗) γ^* c_ ave( h^*) is known as the “margin” associated with the testing sample ∗ h^*. The class-conditional expectations of the margin distribution associated with c_ ave are +(^) μ^+( f) under class +1+1 and −(^)- μ^-( f) under class −1-1. Therefore, the δ-margin condition in (34) provides a first-order statistical characterization of the margin distribution for classifier c_ ave. According to Theorem 1, this characterization leads to a classification guarantee without specifying the exact distributions of the local decision statistics, in contrast to some distribution-specific soft combination analyses [48]. I-B2 Role of the Perron weights From (16), the Perron entries πk _k determine the relative influence of the local classifiers in the sufficient-communication aggregate. Fusion weight design has been widely studied in ensemble learning, typically for classifiers sharing the same feature representation [50, 51, 40]. The extensions to heterogeneous representations are more problem-dependent [39, 40]. Here, rather than optimizing the fusion weights under arbitrary heterogeneity, we characterize the effect of the Perron weights induced by the communication policy. Their role extends beyond the factor ∑kπk2 _k _k^2 in the conditional prediction bound (46): the training quantities ρ, o R^o, α, and ε _ opt, as well as the achievable decision margin, also depend on π. Consequently, although the uniform Perron vector minimizes ∑kπk2 _k _k^2 when δ and τmax _ are fixed, it need not optimize the overall classification performance. Importantly, the simple averaging rule for constructing A, where each agent assigns equal combination weights to all neighbors, does not guarantee uniform π on arbitrary topologies. Several useful rules for constructing doubly-stochastic A are available in [10]. I-B3 Benefits of cooperative prediction To isolate the benefit of cooperation during prediction, suppose that the same target decision margin δ is attained collaboratively and by a non-cooperative agent. The conditional error bound in Theorem 1 is tighter under collaboration whenever τmax∑k=1Kπk2<1. _ _k=1^K _k^2<1. (47) This condition reflects the balance between the gain from averaging the local scores and the penalty induced by their statistical dependence. In particular, for product class-conditional distributions, we may take τmax=1 _ =1, and (47) reduces to ∑kπk2<1 _k _k^2<1, which holds for any nontrivial network satisfying Assumption 3. Cooperation also provides a distinct benefit through the network-level margin condition. The event δC_δ requires only the π-weighted class-conditional means to attain the prescribed two-sided margin; individual classifiers need not satisfy the same condition. Consequently, agents with weak local evidence can be compensated by sufficiently informative or complementary observations from other agents. This is consistent with Assumption 4, which likewise requires informativeness only at the network level. I-C Finite-Round Communication Theorem 1 characterizes the sufficient-communication limit. When communication is restricted to a finite number of rounds, however, consensus may not yet be reached and the prediction remains agent-dependent. We next characterize how limited communication affects the classification performance at each agent. According to (11), after t communication rounds, agent k uses the following decision statistic k,t=∑ℓ=1K[At]ℓkℓ(ℓ∗), λ_k,t= _ =1^K[A^t]_ k c_ ( h_ ^*), (48) and predicts the label ^k,t≜sgn(k,t). γ_k,t ( λ_k,t). (49) Define the corresponding event ℳk,t≜∗k,t≤0.M_k,t \ γ^* λ_k,t≤ 0\. (50) Under any fixed tie-breaking rule at zero, the misclassification event is contained in ℳk,tM_k,t. Thus, with Pk,t≜ℙ⁡(^k,t≠∗)P_k,t ( γ_k,t≠ γ^*), applying the same decomposition as in (42) gives Pk,t≤ℙ⁡(ℳk,t)≤⁡[[δ]ℙ(ℳk,t|)]+ℙ⁡(δ¯). P_k,t (M_k,t) \! [ 1[C_δ]P(M_k,t 1.0mu| 1.0muT) ]+P( C_δ). (51) On the event δC_δ, a positive decision margin is guaranteed for the π-weighted aggregate under sufficient communication (18). At a finite communication round, from (48), agent k combines the local scores ℓ(ℓ∗) c_ ( h_ ^*) using the weights [At]ℓk[A^t]_ k. To characterize how much of this guaranteed margin is retained after t rounds, fix any σ∈(σA,1)σ∈( _A,1). Matrix theory ensures that there exists a constant C⁡(A,σ)>0C(A,σ)>0 such that [52] |[At]ℓk−πℓ|≤C⁡(A,σ)σt|[A^t]_ k- _ |≤ C(A,σ)σ^t (52) for all agents k,ℓk, and communication rounds t. Accordingly, define the residual finite-round margin rt(δ)≜δ−2βKC(A,σ)σt.r_t(δ) δ-2β KC(A,σ)σ^t. (53) The second term bounds the possible reduction of the guaranteed margin after only t communication rounds. Theorem 2 (Finite-round classification guarantee). Suppose Assumptions 2, 3 and 5 hold, and let δ>0δ>0. Fix σ∈(σA,1)σ∈( _A,1) and a corresponding C⁡(A,σ)C(A,σ). Then, for every agent k and every integer t≥0t≥ 0 such that rt(δ)>0r_t(δ)>0, almost surely on δC_δ, ℙ⁡(ℳk,t|)≤exp⁡−rt2(δ)2τmaxβ2∑ℓ=1K([At]ℓk)2.P(M_k,t 1.0mu| 1.0muT)≤ \- r_t^2(δ)2 _ β^2 _ =1^K([A^t]_ k)^2 \. (54) If, in addition, the conditions of Proposition 2 hold, combining the two guarantees through (51) yields an explicit upper bound on Pk,tP_k,t. Proof. See Appendix . ∎ The condition rt(δ)>0r_t(δ)>0 provides a sufficient communication horizon for retaining a positive guaranteed margin. In particular, when 2βKC(A,σ)>δ2β KC(A,σ)>δ, it is equivalent to t>log⁡δ2βKC(A,σ)log⁡σ.t> δ2β KC(A,σ) σ. (55) Thus, faster decay of the transient term C⁡(A,σ)σtC(A,σ)σ^t allows a positive margin to be guaranteed with fewer communication rounds. The bound in (54) is both round- and agent-dependent. Finite communication affects the prediction guarantee in two ways. First, the residual margin rt(δ)r_t(δ) reflects the reduction in the guaranteed decision margin caused by incomplete mixing. Second, the column [At]ℓkℓ=1K\[A^t]_ k\_ =1^K specifies the fusion profile at agent k and determines the corresponding concentration term. Therefore, finite-round performance depends on the full combination matrix A, through both the transient fusion weights and their convergence toward the Perron vector. The second-largest eigenvalue magnitude σA _A determines the admissible geometric rates in (52), while C⁡(A,σ)C(A,σ) captures the associated matrix-power constant. This is consistent with the role of spectral properties in the convergence behavior of distributed averaging [53]. As t→∞t→∞, At→π⊤A^t→π 1 and rt(δ)→δr_t(δ)→δ. Accordingly, the finite-round conditional bound converges to the sufficient-communication bound in Theorem 1. I-D Finite-Precision Communication Theorem 2 considers finite-round communication with real-valued messages. In practical implementations, however, transmitted scalars are typically represented with finite precision. We therefore introduce a finite-bit version of the prediction-stage communication rule and characterize its effect on classification performance. The local training procedures, empirical centering, and the training event δC_δ remain unchanged. Since both a bounded score and its empirical training mean lie in [−β,β][-β,β], every centered score satisfies |ck,fk(h)|≤2β.|c_k,f_k(h)|≤ 2β. (56) Let b≥1b≥ 1 denote the available bits per transmitted scalar, and let 2≤Lb≤2b2≤ L_b≤ 2^b reconstruction levels be used. Over the interval [−2β,2β][-2β,2β], consider the uniform alphabet b _b =vj=−2β+jΔb:j=0,…,Lb−1, =\v_j=-2β+j _b:\ j=0,…,L_b-1\, Δb _b =4βLb−1. = 4βL_b-1. (57) We consider here an adjacent-level stochastic rounding protocol [54]. For u∈[vj,vj+1]u∈[v_j,v_j+1], j=0,…,Lb−2j=0,…,L_b-2, the quantizer b Q_b returns one of the two adjacent reconstruction levels vjv_j and vj+1v_j+1 according to b(u)=vj,with probability vj+1−uΔb,vj+1,with probability u−vjΔb. Q_b(u)= casesv_j,&with probability\; v_j+1-u _b,\\ v_j+1,&with probability\; u-v_j _b. cases (58) An important feature of this protocol is that the quantizer b Q_b is unbiased, that is, [b(u)|u]=u.E[ Q_b(u) 1.0mu| 1.0muu]=u. (59) Let xk,t(b)x_k,t^(b) denote the finite-precision decision statistic at agent k after t communication rounds. Starting from xk,0(b)=k(k∗)x_k,0^(b)= c_k( h_k^*), each agent quantizes its current decision statistic before transmission, and the receiving agents combine the quantized values according to qℓ,t(b)=b(xℓ,t(b)),xk,t+1(b)=∑ℓ=1Kaℓkqℓ,t(b).q_ ,t^(b)= Q_b(x_ ,t^(b)), x_k,t+1^(b)= _ =1^Ka_ kq_ ,t^(b). (60) The random draws used for stochastic rounding are independent across agents and communication rounds, with each agent sending the same quantized value to all of its neighbors. After t rounds, agent k predicts according to ^k,t(b)≜sgn(xk,t(b)). γ_k,t^(b) (x_k,t^(b)). (61) Define the event for the finite-precision decision statistic ℳk,t(b)≜∗xk,t(b)≤0,M_k,t^(b) \ γ^*x_k,t^(b)≤ 0\, (62) and let Pk,t(b)≜ℙ⁡(^k,t(b)≠∗)P_k,t^(b) ( γ_k,t^(b)≠ γ^*) (63) denote the classification error probability. As in the real-valued finite-round case, the decomposition in (42), with ℳM replaced by ℳk,t(b)M_k,t^(b), separates the conditional prediction error ℙ⁡(ℳk,t(b)|)P(M_k,t^(b) 1.0mu| 1.0muT) on δC_δ from the probability of the unfavorable training event δ¯ C_δ. Since quantization is performed at every communication round, the perturbations introduced at earlier rounds are propagated through subsequent neighbor-aggregation steps. To state the resulting finite-precision guarantee compactly, define the accumulated quantization term Vk,tq V_k,t q ≜Δb24∑r=1t∑j=1K([Ar]jk)2 _b^24 _r=1^t _j=1^K([A^r]_jk)^2 =4β2(Lb−1)2∑r=1t∑j=1K([Ar]jk)2. = 4β^2(L_b-1)^2 _r=1^t _j=1^K([A^r]_jk)^2. (64) The quantity Vk,tqV_k,t q depends jointly on the communication precision, the number of communication rounds, and the network combination policy. Theorem 3 (Finite-precision classification guarantee). Suppose Assumptions 2, 3, and 5 hold, and let δ>0δ>0. Suppose the agents follow the stochastic finite-precision recursion (60). Fix σ∈(σA,1)σ∈( _A,1) and a corresponding C⁡(A,σ)C(A,σ). Then, for every agent k and every integer t≥0t≥ 0 such that rt(δ)>0r_t(δ)>0, almost surely on δC_δ, ℙ⁡(ℳk,t(b)|)≤exp⁡−rt2(δ)2[τmaxβ2∑ℓ=1K([At]ℓk)2+Vk,tq].P(M_k,t^(b) 1.0mu| 1.0muT)≤ \- r_t^2(δ)2 [ _ β^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \. (65) If, in addition, the conditions of Proposition 2 hold, combining the two bounds in (65) and (39) yields an explicit upper bound on Pk,t(b)P_k,t^(b). Proof. See Appendix . ∎ Theorem 3 extends the real-valued finite-round guarantee (54) established in Theorem 2 by quantifying the additional effect of finite communication precision. The same residual finite-round margin term rt(δ)r_t(δ) from (53) appears in both guarantees, while finite precision introduces the additional term Vk,tqV_k,t q in the denominator of the prediction exponent (65). This result also reveals a tradeoff pertaining to the number of communication rounds. Increasing t increases the guaranteed residual margin rt(δ)r_t(δ) by reducing the effect of incomplete mixing, but also allows additional quantization effects to accumulate through Vk,tqV_k,t q. Moreover, the combination policy A influences both effects through the finite-round fusion weights and the propagation of quantization noise across rounds. Consequently, at fixed precision, the bound need not improve monotonically with the number of communication rounds. The communication precision also determines the communication cost. Let Eoff(A)≜∑k=1K∑ℓ=1ℓ≠kK[aℓk>0]E_ off(A) _k=1^K _ subarrayc =1\\ ≠ k subarray^K 1[a_ k>0] (66) denote the number of active directed off-diagonal links. If each quantizer index is represented by a fixed-length b-bit word, the per-round and total communication costs per testing example are Bround=bEoff(A),Btotal(t)=tbEoff(A),B_ round=bE_ off(A), B_ total(t)=tbE_ off(A), (67) respectively, excluding packet headers and coding overhead. Self-loops correspond to the local operations and are therefore not counted. Moreover, for any fixed t, we have from (64) that Vk,tq=O⁡((Lb−1)−2).V_k,t q=O ((L_b-1)^-2 ). (68) Here, the O⁡(⋅)O(·) notation describes the decay with respect to LbL_b for fixed t. Under the full-code convention Lb=2bL_b=2^b, this becomes Vk,tq=O⁡(2−2b)V_k,t q=O(2^-2b). Therefore, increasing the bit width increases the communication cost linearly, while the finite-precision term in the bound (65) decays as O⁡(2−2b)O(2^-2b). For fixed t, as b→∞b→∞, the finite-precision guarantee converges to the corresponding real-valued finite-round guarantee. I-E PAC-Style Generalization Guarantee The preceding results characterize collaborative prediction through the lens of expected network margin. We now consider a complementary PAC-style perspective that relates the population classification error of the realized classifiers to their performance observed on the training data. This is the core idea of classical margin-based generalization bounds [44]. Specifically, to quantify this performance, for η>0η>0, we define the η-margin loss [44] Φη(x)=min⁡(1,max⁡(0,1−xη)). _η(x)= (1, (0,1- xη ) ). (69) It satisfies [sgn(f())≠]≤Φη(f())≤[f()≤η]. 1[ (f( h))≠ γ]≤ _η( γf( h))≤ 1[ γf( h)≤η]. (70) The margin parameter η used here is distinct from the decision-margin level δ in (34). For a training set (hn,γn)n=1m\(h_n, _n)\_n=1^m, the empirical margin loss for an arbitrary function f is given by Pe,η(f)≜1m∑n=1mΦη(γnf(hn)).P_e, emp^η(f) 1m _n=1^m _η( _nf(h_n)). (71) By (70), Pe,η(f)P_e, emp^η(f) upper bounds the empirical classification error on the same sample. When the classifiers depend on the training data, their classification errors on a fresh example are understood conditionally on the realized training information. Specifically, for a local classifier k c_k and the aggregate classifier c_ ave, respectively, we define Pe(k) P_e( c_k) ≜ℙ⁡(sgn(k(k∗))≠∗|), ( ( c_k( h_k^*))≠ γ^* 2.0mu| 2.0muT ), (72) Pe() P_e( c_ ave) ≜ℙ⁡(sgn((∗))≠∗|). ( ( c_ ave( h^*))≠ γ^* 2.0mu| 2.0muT ). (73) To evaluate the empirical margin loss of the aggregate classifier c_ ave with (71), the training samples must provide the aligned collection of local views on which c_ ave operates. Therefore, for the PAC analysis, we specialize the training setup to equal local sample sizes Nk=N_k=N and aligned network examples. Specifically, let ≜(n,n)n=1ND \( h_n, γ_n) \_n=1^N (74) consist of N i.i.d. draws from the joint distribution Q, where n=(1,n,…,K,n) h_n=( h_1,n,…, h_K,n). The local training set at agent k is the corresponding projection k≜(k,n,n)n=1N.D_k \( h_k,n, γ_n) \_n=1^N. (75) This specialization leaves the local training procedures in Section I-A unchanged. The network examples are independent across n, whereas the K views within each example may remain statistically dependent. Based on the network training set D in (74), we establish the following PAC-style generalization bounds for the classifiers k c_k and c_ ave using their empirical margin losses. Here, ρk _k and ρ denote the individual and network Rademacher complexities defined in Appendix , evaluated at the common sample size Nk=N_k=N. Theorem 4 (PAC-style margin generalization guarantee). Under Assumption 2 and the sampling setup above, fix any η>0η>0. Then, for any ϵ∈(0,1)ε∈(0,1) and every fixed agent k, Pe(k)≤Pe,η(k)+4ηρk+ℛ⁡(ϵ,N)P_e( c_k)≤ P_e, emp^η( c_k)+ 4η _k+R(ε,N) (76) with probability at least 1−ϵ1-ε. Moreover, Pe()≤Pe,η()+4ηρ+ℛ⁡(ϵ,N)P_e( c_ ave)≤ P_e, emp^η( c_ ave)+ 4ηρ+R(ε,N) (77) with probability at least 1−ϵ1-ε, where ℛ⁡(ϵ,N)≜4βηN(1+2log⁡1ϵ).R(ε,N) 4βη N (1+ 2 1ε ). (78) Proof. See Appendix . ∎ Theorem 4 provides a complementary guarantee for the trained classifiers without investigating the prescribed network-margin event δC_δ. It also does not require Assumption 5: dependence among the local views within each network example is incorporated through their joint distribution Q. Moreover, the guarantee does not require exact empirical risk minimization or a prescribed optimization gap. Therefore, it holds uniformly across classifiers returned by the local training procedures in Section I-A. From (76) and (77), the local and collaborative bounds have the same form, but differ in their empirical margin losses and complexity terms. Structurally, k c_k operates on a single local feature space ℋkH_k, whereas c_ ave operates on the joint feature space ℋ1×⋯×ℋKH_1×·s×H_K. The empirical margins of c_ ave can therefore reflect how information from different local views is combined. The theorem does not, however, impose a universal ordering between the local and collaborative bounds. Any collaborative advantage must be reflected in the aggregate margin loss Pe,η()P_e, emp^η( c_ ave) relative to the corresponding complexity terms ρ. This differs from the conventional ensemble setting, where multiple models typically operate on the same feature representation and the benefit of combining their outputs is often discussed in terms of variance reduction [44, 55]. I-F Relation to Social Learning The DeGroot rule (11) and conventional social learning (SL) strategies both aggregate information across a network, but differ in how evidence enters the inference process. In the present framework, agents repeatedly communicate decision statistics initialized from a fixed testing example, whereas conventional SL recursions continually incorporate newly arriving observations. To clarify the relation between these two inference mechanisms, Appendix considers a standard geometric-pooling SL recursion under a static-observation specialization, in which each agent reuses the same local testing observation at every communication round. This specialization places the SL rule in the same single-testing-example setting as the DeGroot rule and enables a direct comparison, while remaining distinct from the conventional streaming operation of SL. Under this specialization, repeated SL updates accumulate the same local evidence rather than new temporal evidence. After normalization by the number of communication rounds, the SL decision statistic converges to the same π-weighted aggregate c_ ave as the DeGroot rule (11). Consequently, whenever (∗)≠0 c_ ave( h^*)≠ 0, the two strategies yield the same prediction under sufficient communication. Their finite-round decision statistics are generally different, however, and therefore lead to different finite-round guarantees. The complete comparison and the corresponding finite-round SL guarantee are provided in Appendix . Raw-score counterparts. The preceding analysis considers the empirically centered local scores k(k) c_k( h_k) defined in (9). As shown by Proposition 1, centering affects threshold alignment rather than being a structural requirement for collaboration. We refer to the learned scores ^k(k) f_k( h_k) used without empirical centering as raw scores. In Appendix , we provide an analogous analysis for raw scores. This analysis shows that the main collaborative guarantees persist without empirical centering, although the conditions for attaining a prescribed zero-threshold margin and the constants appearing in the finite-round bounds change. Neither representation universally dominates the other: centering can improve threshold alignment, whereas raw scores avoid estimating the empirical center and have a smaller worst-case score magnitude than the centered scores. IV Numerical Simulations We evaluate the proposed test-time collaborative classification framework under two complementary observation models. The first is a controlled patch-partition benchmark based on CIFAR-10, where each agent observes a spatial patch extracted from a common image. The second is a multi-view benchmark constructed from rendered objects in ModelNet40, where each agent observes the same 3D object from a different viewpoint. The two benchmarks introduce different forms of local heterogeneity, namely spatial partitioning and viewpoint variation. We further examine dependence across agents, heterogeneous local models, communication topology, quantization, adaptive stopping, and corrupted reports. Unless otherwise stated, all curves are averaged over multiple Monte Carlo repetitions and the error bars indicate 95%95\% confidence intervals. Additional implementation details and extended numerical results are provided in Appendix . TABLE I: Training and inference coordination of the methods considered in the main baseline comparisons. Method Training coordination Additional fitting Inference coordination Non-cooperative None None None Avg-stat / Vote None None One-shot centralized averaging / voting Learned (simplex) fusion None Fusion rule fitted on aligned validation outputs One-shot centralized fusion AdaBoost Centralized Boosting-based fitting of local predictors One-shot weighted fusion Ours / Ours (no centering) None None Iterative peer-to-peer scalar exchange VFL-JT Joint training on aligned labeled samples Server-side fusion head learned jointly One-shot server fusion Central oracle Centralized full-information training None Centralized full-information prediction IV-A CIFAR-10 patch-partition benchmark We begin with a controlled patch-partition benchmark based on CIFAR-10 [56]. We consider the binary task of distinguishing cats from dogs. Each 32×3232× 32 RGB image is divided into a 3×33× 3 grid of non-overlapping spatial patches, yielding K=9K=9 agents, and agent k receives only the patch associated with its grid location. In this way, the agents observe different local regions of the same image and collaborate only during inference. The communication network is taken to be a directed Erdös–Rényi graph [46], generated randomly at the beginning of the experiment. The observation map of the nine agents and the communication topology are illustrated in Fig. 1. (a) (b) Fig. 1: CIFAR-10 patch-partition benchmark. Panel (a) shows the local patch observed by each agent for a representative cat image, and panel (b) shows the communication topology used for collaboration. At each agent, a convolutional neural network (CNN) is trained independently on the corresponding patch dataset. For simplicity, we use the same local labeled dataset size N0N_0 for all agents. For each value of N0N_0, we construct balanced local datasets, train the classifiers independently, perform collaborative classification at test time, and average the results over multiple repetitions. The detailed training protocol is provided in Appendix . (a) Decision margin (b) Probability of error (c) Instantaneous probability of error Fig. 2: Main CIFAR-10 patch-partition results. Panel (a) reports the empirical class-conditional decision statistics, panel (b) compares the probability of error under different values of N0N_0, and panel (c) shows the error evolution over the communication rounds for N0=10000N_0=10000. Figure 2 summarizes the main performance results for this CIFAR-10 benchmark. Panel (a) reports the empirical class-conditional decision statistics under different values of N0N_0. In the notation of Section I, these curves correspond to the empirical behavior of +(^) μ^+( f) and −(^) μ^-( f) in (33). As N0N_0 increases, +(^) μ^+( f) moves farther above zero and −(^) μ^-( f) moves farther below zero, so that the separation between the two curves grows overall, consistent with a larger achievable decision margin in the sense of condition (34). This suggests that the network of trained classifiers becomes more informative as more training samples are available, which is beneficial for classification at test time. Panel (b) shows the corresponding classification performance. We compare the proposed method with several rules from ensemble learning and decision fusion, together with the non-cooperative baseline, an explicit centering ablation Ours (no centering), the VFL-JT baseline, and a central oracle trained on the full image as a full-information benchmark. The VFL-JT baseline uses a score-level joint-training scheme inspired by [57], where the local models and a server-side fusion head are optimized jointly using aligned labeled samples. Table I summarizes the training and inference coordination of the methods considered in the main baseline comparison. Since several of these reference rules depend on centralized aggregation of local outputs, the comparison is carried out in the regime of sufficient communication, where the distributed recursion (11) approaches its π-weighted consensus limit given in (16). From Fig. 2b, the probability of error decreases as N0N_0 increases, consistent with the increased decision margins observed in Fig. 2a. At the same time, the proposed method consistently improves upon non-cooperative prediction and remains competitive with the stronger fusion baselines. With joint training, VFL-JT achieves lower error than the proposed method over the tested range. We also note that under sufficient communication, Avg-stat coincides with the proposed method when A is doubly-stochastic, while Ours and Ours (no centering) exhibit similar performance over the tested range. Panel (c) illustrates the evolution of Pk,tP_k,t during the collaboration process. As the communication round t grows, the decision statistic k,t λ_k,t converges to λ_ ave, and Pk,tP_k,t approaches the error of the limiting aggregate classifier. It is worth noting that Pk,0P_k,0 at t=0t=0 corresponds to the error incurred by agent k in the non-cooperative scenario. The light grey dashed horizontal lines in Fig. 2c indicate these baseline error levels for reference. The average of Pk,tP_k,t across all K agents is also shown in Fig. 2c, which decreases over the communication rounds before stabilizing. This highlights the improvement in the network-wide performance under collective prediction. We next present two additional studies for the CIFAR-10 patch-partition benchmark. IV-A1 Conditional mutual information between patches (a) Conditional mutual information matrix (b) Grouped conditional mutual information Fig. 3: Conditional mutual information between CIFAR-10 patches. Panel (a) shows the estimated pairwise conditional mutual information matrix across the nine patch agents. White stars mark patch pairs that are adjacent on the 3×33× 3 grid shown in Fig. 1a, while red boxes indicate, in each row, the largest off-diagonal entries whose number matches the number of grid neighbors of the corresponding patch. Panel (b) shows the grouped summary over same-row, same-column, and far patch pairs. Since all patches are extracted from the same natural image, nearby patches may remain statistically dependent even after conditioning on the class label. To quantify this effect, we estimate the class-conditional mutual information for each label value and then average according to the class probabilities: ICMI(hk,hℓ)≜∑γ∈Γℙ⁡(=γ)I(hk;hℓ|=γ)I_CMI(h_k,h_ ) _γ∈ P( γ=γ)\,I(h_k;h_ 1.0mu| 1.0mu γ=γ) (79) where hkh_k denotes the raw pixel vector of the patch observed by agent k, γ denotes the class label, and I⁡(hk;hℓ|=γ)I(h_k;h_ 1.0mu| 1.0mu γ=γ) is the mutual information under the conditional distribution given =γ γ=γ. The reported estimates are computed by first reducing the patch dimension through principal component analysis (PCA) [45] and then applying a k-nearest-neighbor (kNN) mutual information estimator [58]. More details about this estimation process are provided in Appendix . Figure 3 shows that the estimated conditional mutual information is clearly nonzero for many patch pairs and tends to be strongest among spatially related patches. Thus, the CIFAR-10 patch-partition benchmark induces a structured observation model in which nearby local views exhibit stronger statistical dependence than distant ones. This observation motivates the dependence-aware prediction analysis in Section I. While the CMI diagnostic is not an estimate of the ATE coefficient in Assumption 5, it does provide empirical evidence that the local views are not independent in this benchmark. IV-A2 Heterogeneous local model families We next relax the assumption that all agents use the same local architecture. Different agents are then assigned different local model families, while the communication graph and the collaboration rule remain unchanged. The local models include the patch-level convolutional network used in the main CIFAR-10 benchmark, a logistic-regression classifier on raw patch pixels, and a small residual convolutional network [59]. The detailed protocol and the precise model specifications are given in Appendix . Figure 4 reports a balanced assignment study in which the model families are rotated across the 3×33× 3 grid so that each family appears equally often at every spatial location. The results indicate that collaboration remains beneficial across all three model families and across all patch locations, although the magnitude of the improvement varies with both the family and the position. Furthermore, under one representative heterogeneous assignment, the main performance results summarized in Fig. 5 show that the same qualitative behavior observed in the homogeneous benchmark is preserved here: the empirical decision margins become more separated as N0N_0 increases, the collaborative classifier improves over non-cooperative inference, and most of the performance gain is realized within the first few communication rounds. Fig. 4: Collaboration gain under heterogeneous local model families on CIFAR-10. Each cell shows the relative gain when the corresponding model family is assigned to that patch location under the balanced rotation protocol. (a) Decision margin (b) Probability of error (c) Instantaneous probability of error Fig. 5: Performance of heterogeneous local model families on CIFAR-10. The curve labels in panel (b) and the value of N0N_0 in panel (c) are the same as those in Fig. 2. IV-B ModelNet40 multi-view benchmark In this section, we consider a multi-view benchmark derived from ModelNet40 [60]. Unlike the CIFAR-10 patch-partition setting, where the agents observe different spatial regions of the same image, here each agent receives one rendered view of the same 3D object. The heterogeneity across agents therefore arises from viewpoint rather than patch location. (a) (b) Fig. 6: ModelNet40 multi-view benchmark. Panel (a) shows the 12 rendered views associated with a nightstand object, and panel (b) shows the communication topology used for collaboration. For each object, we generate K=12K=12 RGB renderings from fixed camera positions whose azimuth angles are uniformly spaced around the object, with a common elevation angle of 30∘30 . One rendered view is then assigned to each agent. In the experiments, we consider the binary task pair of dresser and nightstand. The multi-view construction and a randomly-generated communication topology used in our experiments are illustrated in Fig. 6. Each local classifier is a lightweight convolutional network, referred to as MicroCNN; further experimental details are given in Appendix . (a) Decision margin (b) Probability of error (c) Instantaneous probability of error Fig. 7: Main ModelNet40 multi-view results. Panel (a) reports the empirical class-conditional decision statistics, panel (b) compares the probability of error under different values of N0N_0, and panel (c) shows the error evolution over the communication rounds for N0=400N_0=400. Figure 7 summarizes the main performance results for this ModelNet40 benchmark. In panel (a), the empirical statistics +(^) μ^+( f) and −(^) μ^-( f) remain separated across N0N_0, indicating that the classifier network remains informative under the viewpoint heterogeneity. Compared with the CIFAR-10 benchmark in Fig. 2, their variation with N0N_0 is milder and exhibits more visible fluctuations, which is consistent with the smaller amount of training data available in this benchmark. The corresponding probability of error curves in Fig. 7b also decrease overall as N0N_0 increases, in agreement with the larger decision margins attained at larger values of N0N_0. Meanwhile, the improvement of the proposed method is more limited at smaller N0N_0, where some local training sets may be too small for the agents to learn informative classifiers from their views. In this regime, some non-cooperative agents can achieve lower error when acting alone than under collaboration. The relative ordering of the score-fusion baselines is also more sensitive than in the CIFAR-10 benchmark, which may reflect the fact that classifiers trained on different viewpoints need not produce scores on comparable scales. In particular, since Avg-stat corresponds to a special case of the proposed method under sufficient communication when A is doubly-stochastic, the gap between Avg-stat and Ours indicates that the choice of the combination policy A can have a visible effect on collaborative performance. The instantaneous error curves in Fig. 7c further show that most of the performance gain is achieved within the first few communication rounds. IV-B1 Temperature scaling and score calibration The sensitivity of the score fusion baselines in Fig. 7 suggests that score calibration across viewpoints deserves closer examination. To this end, we consider temperature scaling as a post-hoc calibration step in our experiments [61]. For each agent, a positive temperature is fitted on the local validation set by minimizing the cross-entropy loss of the rescaled logits, as detailed in Appendix . The local decision statistic is then formed from the difference between the two calibrated logits, followed by the same centering step (9) as in the uncalibrated setting. Since temperature scaling does not change the predicted class of an individual local classifier, its role here is not to alter the local decisions themselves, but to improve the comparability of the real-valued decision statistics exchanged during collaboration. (a) Decision margin (b) Probability of error (c) Instantaneous probability of error Fig. 8: Performance of temperature scaling on ModelNet40. The curve labels in panel (b) and the value of N0N_0 in panel (c) are the same as those in Fig. 7. Figure 8 shows that, relative to Fig. 7, the class-conditional decision statistics become more clearly separated after calibration, and the probability of error curve of the proposed method is slightly improved over most of the tested range of N0N_0. This suggests that, under viewpoint heterogeneity, score calibration can improve the alignment of the exchanged decision statistics across agents and thereby improve collaborative prediction. VFL-JT is not included in this study because temperature scaling of its local scores is coupled with the jointly optimized fusion head. Additional figures are provided in Appendix . IV-B2 Controlled correlation stress test In the multi-view setting, different agents observe the same underlying object and can therefore produce statistically dependent local decision statistics. To examine how the collaborative classifier behaves as this dependence becomes stronger, we perform a controlled stress test directly on the local decision statistics, with more details provided in Appendix . Let xi,kx_i,k denote the clean local decision statistic produced by agent k for the i-th testing sample. To control the correlation between xi,kx_i,k across agents, we consider a perturbed local decision statistic at each agent, denoted by xi,k′x _i,k: xi,k′=xi,k+σpert(ωc(r)ui+ωi(r)ei,k)x _i,k=x_i,k+ _pert\! ( _c(r)\,u_i+ _i(r)\,e_i,k ) (80) where ui∼⁡(0,1)u_i (0,1) is a shared standard Gaussian perturbation for sample i, ei,k∼⁡(0,1)e_i,k (0,1) are independent standard Gaussian perturbations across samples and agents, and ωc(r)=r1+r2,ωi(r)=11+r2. _c(r)= r 1+r^2, _i(r)= 1 1+r^2. (81) Here, r≥0r≥ 0 controls the relative strength of the common-mode component, while σpert>0 _pert>0 controls the overall perturbation scale. Since ωc(r)2+ωi(r)2=1 _c(r)^2+ _i(r)^2=1, varying r changes the balance between common and idiosyncratic perturbations without substantially changing the total injected variance. This construction follows a standard common-factor decomposition with shared and idiosyncratic components. At test time, the agents employ the same rule (11), but with the perturbed local decision statistics xi,k′x _i,k in place of the clean statistics xi,kx_i,k. (a) Average pairwise correlation (b) Collaboration gain Fig. 9: Controlled correlation stress test on ModelNet40. Panel (a) shows the average pairwise Pearson correlation across agent decision statistics, both unconditional and label-conditional, and panel (b) shows the collaboration gain after t=20t=20 communication rounds. Figure 9 shows that the average pairwise correlations across agents, both unconditional and label-conditional, increase with r. The corresponding collaboration gain, which is measured by the error reduction relative to non-cooperative inference after t=20t=20 communication rounds, decreases with r for each fixed perturbation amplitude. This is consistent with the fact that stronger dependence makes the local decision statistics more redundant and therefore reduces the benefit of information sharing across the network. At the same time, the degradation is gradual, indicating that the collaborative classifier continues to provide measurable gains over a broad range of dependence levels. IV-C Communication design, adaptive stopping, and robustness In this section, we examine several aspects of the communication protocol using the ModelNet40 benchmark. In particular, we study how collaborative performance is influenced by the graph topology, the combination policy, the communication precision, adaptive stopping rules, and corrupted messages. IV-C1 Topology, combination policy, and communication precision We begin by examining the impact of the communication design across three graph topologies: ring, grid, and directed Erdös–Rényi. For each topology, the combination policy A is constructed using two different schemes: the uniform averaging rule and the Metropolis rule. For this quantization study only, we represent the already trained probabilistic classifier by the bounded posterior-probability difference gk(h)≜p^k(+1|h)−p^k(−1|h)=tanh⁡(^k(h)2)∈[−1,1].g_k(h) p_k(+1|h)- p_k(-1|h)= \! ( f_k(h)2 )∈[-1,1]. (82) No retraining process is required for this representation. Under this transformation, the resulting centered local score ckg(h)≜gk(h)−1Nk∑n=1Nkgk(hk,n)c_k^g(h) g_k(h)- 1N_k _n=1^N_kg_k(h_k,n) (83) satisfies |ckg(h)|≤2|c_k^g(h)|≤ 2. We compare the resulting matched full-precision recursion with 4-, 6-, and 8-bit stochastic communication. Further details of the quantization protocol are provided in Appendix . Fig. 10: Effect of topology, the construction rule of A, and quantization noise. Figure 10 reports the evolution of the average probability of error under these communication designs. All settings benefit from communication in the early rounds, but both the finite-round improvement and the error level eventually reached depend visibly on the topology, the construction rule of A, and the communication precision. Among the graph families considered here, the grid generally attains the lowest error, while the ring also performs competitively. By contrast, the directed Erdös–Rényi topology is more sensitive to the choice of combination policy and communication precision. The precision hierarchy is systematic: 8-bit stochastic communication is practically indistinguishable from the matched bounded full-precision reference, 6-bit communication produces a modest intermediate degradation, and 4-bit communication produces the largest loss. More detailed grouped views of these results are provided in Appendix . IV-C2 Adaptive stopping rules In the previous experiments, the collaboration framework is evaluated under a fixed communication budget t chosen in advance. In practice, however, selecting this budget is not straightforward: a larger t increases communication cost, while a smaller t may be insufficient for collaboration. This motivates the study of adaptive stopping rules, in which the communication pattern is allowed to depend on the evolution of local decision statistics. We consider two local event-triggered families: a Δ -trigger rule based on the change in the communicated statistic, and a label-stability with confidence rule based on the local prediction and a confidence threshold (see Appendix ). Since transmissions may stop at different times for different agents and different samples, we measure the resulting communication cost by the total number of transmissions per testing sample. (a) tmax=2t_ =2 (b) tmax=5t_ =5 (c) tmax=10t_ =10 (d) tmax=20t_ =20 Fig. 11: Tradeoff between the probability of error and the total number of transmissions (per testing sample) under local adaptive stopping rules. Figure 11 illustrates the tradeoff between the probability of error and the total communication cost for four values of the maximum communication budget tmaxt_ . In each panel, only the Pareto-efficient operating points are shown, namely those for which no alternative configuration attains both lower communication cost and lower probability of error. Both stopping rule families exhibit a clear communication–accuracy tradeoff, but the label-stability rule generally yields a better Pareto frontier than the Δ -trigger rule, especially in the low communication regime. The star markers denote the best points selected from the two families of stopping rules. Their positions indicate that communication can often be terminated well before reaching the maximum communication budget. IV-C3 Robustness to noisy, faulty, and adversarial agents We next examine how the collaborative inference mechanism behaves when a subset of agents transmits corrupted messages at test time. The perturbation is introduced only at the level of the local decision statistics before communication begins. For each corruption scenario, we compare the resulting probability of error with that of the corresponding clean run obtained from the same trained models, the same testing set, and the same combination policy. The degradation is measured by the paired excess error, namely the increase in probability of error relative to the clean run within the same Monte Carlo repetition. In each repetition, we corrupt exactly m∈1,…,Km∈\1,…,K\ agents and consider three families of perturbations: benign Gaussian noise, faulty agents (including stuck-at-zero and constant-bias failures), and adversarial reports modeled by scaled sign flips. Additional details on the perturbation models are provided in Appendix . (a) Benign Gaussian noise (b) Faulty agents (c) Adversarial reports Fig. 12: Robustness of collaborative classification to corrupted agents, measured by the paired excess error at the communication round t=20t=20. Panel (a) varies the Gaussian noise multiplier ηnoise _noise, panel (b) varies the fault type and the normalized bias level ηbias _bias, and panel (c) varies the adversarial scale ηadv _adv. Figure 12 summarizes the robustness results after 20 rounds of communication. In all three perturbation settings, the excess error increases with both the number of corrupted agents and the perturbation severity. Among the three cases, adversarial scaled sign flips are the most damaging, indicating that persistently misleading decision statistics can have a particularly strong effect on collaborative prediction. By contrast, benign Gaussian noise and faulty agents yield a more gradual degradation, especially when only a few agents are affected. V Concluding Remarks In this work, we proposed a distributed test-time collaborative classification framework for multi-agent networks. We considered a group of independently trained agents, potentially with heterogeneous feature spaces and model architectures, that collaborates at inference time by exchanging local beliefs with graph neighbors to produce collective predictions. Our analysis established classification error bounds that characterize the impact of the agent heterogeneity, network topology, combination policies, and communication budgets on collective prediction performance. These bounds provide theoretical insight into the interplay between local training and distributed inference in multi-agent systems, in addition to highlighting the benefits of collaborative classification relative to the non-cooperative scenario. Importantly, this study shows that cooperation can be valuable at test time even when the local models are trained independently, which is becoming an increasingly important paradigm in real-world implementations of distributed machine learning. Future directions include extensions to multi-class settings, theoretical analysis of adaptive and communication-efficient protocols, privacy-preserving implementations, and more general communication settings, including asynchronous updates, time-varying topologies, and lossy communication. References [1] A. H. Sayed, S. Tu, J. Chen, X. Zhao, and Z. J. Towfic (2013) Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior. IEEE Signal Processing Magazine 30 (3), p. 155–171. Cited by: §I-B2, §I, §I. [2] S. Kar and J. M. F. Moura (2013) Consensus + innovations distributed inference over networks: cooperation and sensing in networked systems. IEEE Signal Processing Magazine 30 (3), p. 99–109. Cited by: §I-B2, §I, §I. [3] M. I. Jordan, J. D. Lee, and Y. Yang (2019) Communication-efficient distributed statistical inference. Journal of the American Statistical Association 114 (526), p. 668–681. Cited by: §I-B2, §I, §I. [4] J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and J. S. Rellermeyer (2020) A survey on distributed machine learning. ACM Computing Surveys 53 (2), p. 1–33. Cited by: §I. [5] D. Jin, N. Kannengießer, S. Rank, and A. Sunyaev (2024) Collaborative distributed machine learning. ACM Computing Surveys 57 (4), p. 95:1–95:36. External Links: Document Cited by: §I. [6] V. Matta, P. Braca, S. Marano, and A. H. Sayed (2016) Diffusion-based adaptive distributed detection: Steady-state performance in the slow adaptation regime. IEEE Transactions on Information Theory 62 (8), p. 4710–4732. Cited by: §I. [7] D. Bajović, J. M. F. Moura, J. Xavier, and B. Sinopoli (2016) Distributed inference over directed networks: Performance limits and optimal design. IEEE Transactions on Signal Processing 64 (13), p. 3308–3323. Cited by: §I. [8] S. Shahrampour, A. Rakhlin, and A. Jadbabaie (2016) Distributed detection: Finite-time analysis and impact of network topology. IEEE Transactions on Automatic Control 61 (11), p. 3256–3268. Cited by: §I. [9] V. Matta, V. Bordignon, and A. H. Sayed (2025) Social learning: opinion formation and decision-making over graphs. NOW Book Series on Information and Learning Sciences, Now Publishers. Cited by: §I-B2, §I. [10] A. H. Sayed (2014) Adaptation, learning, and optimization over networks. Foundations and Trends® in Machine Learning 7 (4-5), p. 311–801. Cited by: §I, §I-B2, Assumption 3, §H.3.1. [11] A. Nedic (2020) Distributed gradient methods for convex machine learning problems in networks: Distributed optimization. IEEE Signal Processing Magazine 37 (3), p. 92–101. Cited by: §I. [12] P. Braca, L. M. Millefiori, A. Aubry, S. Marano, A. De Maio, and P. Willett (2022) Statistical hypothesis testing based on machine learning: Large deviations analysis. IEEE Open Journal of Signal Processing 3, p. 464–495. Cited by: §I. [13] V. Bordignon, S. Vlaski, V. Matta, and A. H. Sayed (2023) Learning from heterogeneous data based on social interactions over graphs. IEEE Transactions on Information Theory 69 (5), p. 3347–3371. External Links: Document Cited by: §I-B2, §I, §I-A, §I-A, §A.3, §A.3, §A.4, §F.1. [14] P. Hu, V. Bordignon, M. Kayaalp, and A. H. Sayed (2025) Non-asymptotic performance of social machine learning under limited data. Signal Processing 230, p. 109849. Cited by: §I-B2, §I, §I-A, §I-A, §I-A, §A.1, §A.1, §A.1, §A.3, §A.3, §A.3, §A.3, §A.3, §F.1, §H.1.1. [15] S. F. Yilmaz, B. Hasircioğlu, L. Qiao, and D. Gündüz (2025) Private collaborative edge inference via over-the-air computation. IEEE Transactions on Machine Learning in Communications and Networking 3, p. 215–231. Cited by: §I-A. [16] P. Yadav, C. Raffel, M. Muqeeth, L. Caccia, H. Liu, T. Chen, M. Bansal, L. Choshen, and A. Sordoni (2025) A survey on model MoErging: recycling and routing among specialized experts for collaborative learning. Transactions on Machine Learning Research. Cited by: §I-A, §I-B3. [17] C. Mavromatis, P. Karypis, and G. Karypis (2024) Pack of LLMs: Model fusion at test-time via perplexity optimization. In Conference on Language Modeling, Cited by: §I-A. [18] M. H. DeGroot (1974) Reaching a consensus. Journal of the American Statistical Association 69 (345), p. 118–121. Cited by: §I-A, §I-B, §I-A. [19] R. Olfati-Saber, J. A. Fax, and R. M. Murray (2007) Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE 95 (1), p. 215–233. Cited by: §I-A. [20] D. Acemoglu and A. Ozdaglar (2011) Opinion dynamics and learning in social networks. Dynamic Games and Applications 1 (1), p. 3–49. Cited by: §I-A. [21] A. G. Chandrasekhar, H. Larreguy, and J. P. Xandri (2020) Testing models of social learning on networks: Evidence from two experiments. Econometrica 88 (1), p. 1–32. Cited by: §I-A. [22] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, p. 1273–1282. Cited by: §I-B1. [23] J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon (2016) Federated learning: Strategies for improving communication efficiency. arXiv:1610.05492. Cited by: §I-B1. [24] X. Lian, C. Zhang, H. Zhang, C. Hsieh, W. Zhang, and J. Liu (2017) Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §I-B1. [25] Z. Jiang, A. Balu, C. Hegde, and S. Sarkar (2017) Collaborative deep learning in fixed topology networks. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §I-B1. [26] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra (2018) Federated learning with non-iid data. arXiv:1806.00582. Cited by: §I-B1. [27] T. Lin, L. Kong, S. U. Stich, and M. Jaggi (2020) Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems, Vol. 33, p. 2351–2363. Cited by: §I-B1. [28] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al. (2021) Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14 (1–2), p. 1–210. Cited by: §I-B1. [29] A. Fallah, A. Mokhtari, and A. Ozdaglar (2020) Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Advances in Neural Information Processing Systems, Vol. 33, p. 3557–3568. Cited by: §I-B1. [30] D. Wen, K. Jeon, and K. Huang (2022) Federated dropout–A simple approach for enabling federated learning on resource constrained devices. IEEE wireless communications letters 11 (5), p. 923–927. Cited by: §I-B1. [31] S. Xie, D. Wen, X. Liu, C. You, T. Ratnarajah, and K. Huang (2024) Convergence analysis for federated dropout. In GLOBECOM 2024 - 2024 IEEE Global Communications Conference, p. 5313–5318. Cited by: §I-B1. [32] C. Mendler-Dünner, W. Guo, S. Bates, and M. I. Jordan (2021) Test-time collective prediction. In Advances in Neural Information Processing Systems, Vol. 34, p. 13719–13731. Cited by: §I-B1. [33] O. A. Alzubi, J. A. A. Alzubi, S. Tedmori, H. Rashaideh, and O. Almomani (2018) Consensus-based combining method for classifier ensembles. The International Arab Journal of Information Technology 15 (1), p. 76–86. Cited by: §I-B1. [34] A. Jadbabaie, P. Molavi, A. Sandroni, and A. Tahbaz-Salehi (2012) Non-Bayesian social learning. Games and Economic Behavior 76 (1), p. 210–225. Cited by: §I-B2. [35] A. Lalitha, T. Javidi, and A. D. Sarwate (2018) Social learning and distributed hypothesis testing. IEEE Transactions on Information Theory 64 (9), p. 6161–6179. Cited by: §I-B2, §F.1. [36] A. Nedic, A. Olshevsky, and C. A. Uribe (2017) Fast convergence rates for distributed non-Bayesian learning. IEEE Transactions on Automatic Control 62 (11), p. 5538–5553. Cited by: §I-B2. [37] V. Bordignon, V. Matta, and A. H. Sayed (2021) Adaptive social learning. IEEE Transactions on Information Theory 67 (9), p. 6053–6081. Cited by: §I-B2. [38] M. Kayaalp, Y. İnan, E. Telatar, and A. H. Sayed (2024) On the arithmetic and geometric fusion of beliefs for distributed inference. IEEE Transactions on Automatic Control 69 (4), p. 2265–2280. Cited by: §I-B2. [39] R. Polikar (2006) Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 6 (3), p. 21–45. Cited by: §I-B3, §I-B2. [40] L. I. Kuncheva (2014) Combining pattern classifiers: methods and algorithms. John Wiley & Sons. Cited by: §I-B3, §I-B2. [41] P. L. Bartlett, Y. Freund, W. S. Lee, and R. E. Schapire (1998) Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics 26 (5), p. 1651–1686. Cited by: §I-B3. [42] Z. Zhou (2012) Ensemble methods: foundations and algorithms. 1 edition, Chapman and Hall/CRC. Cited by: §I-B3. [43] P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe (2006) Convexity, classification, and risk bounds. Journal of the American Statistical Association 101 (473), p. 138–156. Cited by: §I-A. [44] M. Mohri, A. Rostamizadeh, and A. Talwalkar (2018) Foundations of machine learning. 2 edition, MIT press. Cited by: §I-A, §I-B1, §I-E, §I-E, Appendix D, Appendix E, Appendix E. [45] A. H. Sayed (2022) Inference and learning from data. Cambridge University Press. Cited by: §I-B, §IV-A1, §H.1.3. [46] M. Newman (2018) Networks. Oxford University Press. Cited by: §I-B, §IV-A. [47] T. G. Lewis (2011) Network science: theory and applications. John Wiley & Sons. Cited by: §I-B. [48] L. I. Kuncheva (2002) A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2), p. 281–286. Cited by: §I-A, §I-B1. [49] P. Caputo, G. Menz, and P. Tetali (2015) Approximate tensorization of entropy at high temperature. Annales de la Faculté des Sciences de Toulouse: Mathématiques 24 (4), p. 691–716. External Links: Document Cited by: §I-B, §B.1. [50] G. Fumera and F. Roli (2005) A theoretical and experimental analysis of linear combiners for multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (6), p. 942–956. Cited by: §I-B2. [51] T. Heskes (1997) Selecting weighting factors in logarithmic opinion pools. In Advances in Neural Information Processing Systems, Vol. 10. Cited by: §I-B2. [52] R. A. Horn and C. R. Johnson (2012) Matrix analysis. 2 edition, Cambridge University Press. Cited by: §I-C. [53] L. Xiao and S. Boyd (2004) Fast linear iterations for distributed averaging. Systems & Control Letters 53 (1), p. 65–78. Cited by: §I-C. [54] T. C. Aysal, M. J. Coates, and M. G. Rabbat (2007) Distributed average consensus using probabilistic quantization. In 2007 IEEE/SP 14th Workshop on Statistical Signal Processing, Madison, WI, USA, p. 640–644. Cited by: §I-D. [55] G. Fumera, F. Roli, and A. Serrau (2008) A theoretical analysis of bagging as a linear combination of classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (7), p. 1293–1299. Cited by: §I-E. [56] A. Krizhevsky (2009) Learning multiple layers of features from tiny images. Technical report University of Toronto. Cited by: §IV-A. [57] Y. Hu, D. Niu, J. Yang, and S. Zhou (2019) FDML: a collaborative machine learning framework for distributed features. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 2232–2240. External Links: Document Cited by: §IV-A, §H.1.2. [58] A. Kraskov, H. Stögbauer, and P. Grassberger (2004) Estimating mutual information. Physical Review E 69 (6), p. 066138. Cited by: §IV-A1, §H.1.3. [59] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 770–778. Cited by: §IV-A2, item 3. [60] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao (2015) 3D shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1912–1920. Cited by: §IV-B. [61] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger (2017) On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70, p. 1321–1330. Cited by: §IV-B1. [62] P. L. Bartlett and S. Mendelson (2002) Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research 3 (Nov), p. 463–482. Cited by: §A.4, Appendix E. [63] B. Neyshabur, R. Tomioka, and N. Srebro (2015) Norm-based capacity control in neural networks. In Conference on Learning Theory, p. 1376–1401. Cited by: §A.4. [64] S. Boucheron, G. Lugosi, and P. Massart (2003) Concentration inequalities using the entropy method. The Annals of Probability 31 (3), p. 1583–1614. External Links: Document Cited by: §B.1. [65] C. McDiarmid (1989) On the method of bounded differences. In Surveys in Combinatorics, 1989, J. Siemons (Ed.), London Mathematical Society Lecture Note Series, Vol. 141, p. 148–188. External Links: Document Cited by: Appendix E. Ping Hu received her Ph.D. in electrical engineering from EPFL, Switzerland in 2026. She is currently a postdoctoral researcher with the Adaptive Systems Laboratory, EPFL. Her research interests include distributed learning, inference and decision-making over networked systems. Mert Kayaalp received his Ph.D. in computer and communication sciences from EPFL, Switzerland in 2025. He is currently a research scientist at Dalle Molle Institute for Artificial Intelligence (IDSIA USI-SUPSI), Switzerland, where he is affiliated with the UBS-IDSIA AI Lab. His research focuses on machine learning and signal analysis in networked and dynamical systems. Ali H. Sayed is professor and former dean of engineering at EPFL, Switzerland, where he also directs the Adaptive Systems Laboratory. He served before as a distinguished professor and chair of electrical engineering with UCLA. He is a member of the US National Academy of Engineering (NAE) and The World Academy of Sciences (TWAS). He served as a president of the IEEE Signal Processing Society in 2018 and 2019. An author of over 650 scholarly publications and 10 books, his research involves several areas including adaptation and learning theories, statistical inference, and multi-agent systems. His work has been recognized with several awards including the 2022 IEEE Fourier Technical Field Award and the 2020 IEEE Wiener Society Award. He is also a fellow of EURASIP and the American Association for the Advancement of Science (AAAS). Supplementary Material for “Test-Time Collaborative Classification over Multi-Agent Networks” Organization of the Supplementary Material This supplementary material provides detailed proofs, auxiliary results, and additional experimental information supporting the main paper. The theoretical appendices follow the order of the main development. Appendix A presents the network margin and training analysis and collects the auxiliary quantities used in the Theorems and subsequent proofs. Appendices B–D establish the sufficient-communication, finite-round, and finite-precision classification guarantees, respectively. Appendix E provides the Probably Approximately Correct (PAC)-style generalization analysis, while Appendix F develops the relation to social learning. Appendix G collects the corresponding raw score results when the empirical centering step () is omitted. Finally, Appendix H presents additional experimental details. Cross-references to theorem, equation, section, and figure numbers in the main paper are retained throughout. Appendix A Network Margin and Training Guarantee This appendix provides the technical details supporting the network margin and training guarantees in Section of the main paper. We first collect the complexity and margin quantities used in the finite-sample analysis. We then prove the effect of additive centering and the δ-margin training guarantee under approximate optimization. The final subsection compares the resulting bound with homogeneous centralized training. A.1 Definitions of Auxiliary Quantities This subsection collects the definitions and relevant properties of several auxiliary quantities used throughout the theoretical analysis. Rademacher complexity We begin with introducing the Rademacher complexity of a general function class G. Let ≜z1,z2,…,zmZ \z_1,z_2,…,z_m\ be a fixed sample set of size m. The empirical Rademacher complexity of G with respect to Z is defined as ℜ^()≜supg∈|1m∑n=1mng(zn)| R_Z(G) _ r _g | 1m _n=1^m r_ng(z_n) | (A.1) where 1,…,m r_1,…, r_m are independent Rademacher random variables. For an i.i.d. random sample set ′Z of size m, the Rademacher complexity of G is ℜm()≜′ℜ^′(). R_m(G) _Z R_Z (G). (A.2) For the local function class ℱkF_k and training set size NkN_k, we define the individual Rademacher complexity by ρk≜ℜNk(ℱk), _k R_N_k(F_k), (A.3) and the corresponding network Rademacher complexity by the weighted average involving the Perron vector π: ρ≜∑k=1Kπkρk.ρ _k=1^K _k _k. (A.4) Here, ℜNk(ℱk) R_N_k(F_k) is evaluated for NkN_k i.i.d. local feature samples drawn from the feature marginal induced by QkQ_k. Sample-size factor We define two quantities related to the size of local training sets: Nmax≜maxk⁡Nk,α≜∑k=1KπkNmaxNk.N_ _kN_k, α _k=1^K _k N_ N_k. (A.5) The factor α captures the effect of unequal local training set sizes. Since Nmax/Nk≥1N_ /N_k≥ 1 and ∑kπk=1 _k _k=1, we have α≥1α≥ 1. In particular, α=1α=1 when all local training sets have the same size. Function EΦ(r,δ) E_ (r,δ) Following Eqs. (A.16), (A.18), and (A.20) of [14], we use the same construction with a generic risk level r<Φ⁡(0)r< (0). For any prescribed margin δ≥0δ≥ 0, let dδ⋆(r)d_δ (r) denote the unique nonnegative solution of d−δ−Φ⁡(d)−r2LΦ=0.d-δ- (d)-r2L_ =0. (A.6) The corresponding auxiliary function is EΦ(r,δ)≜dδ⋆(r)−δ4=Φ⁡(dδ⋆(r))−r8LΦ. E_ (r,δ) d_δ (r)-δ4= (d_δ (r))-r8L_ . (A.7) For r=or= R^o, this reduces to the quantity EΦ(o,δ) E_ ( R^o,δ) introduced in [14]. An important property of the quantity dδ⋆(r)d_δ (r) is that, for fixed r, it increases with the prescribed margin δ. Since Φ is non-increasing under Assumption , EΦ(r,δ) E_ (r,δ) decreases as δ increases. For a fixed feasible δ, the same construction shows that EΦ(r,δ) E_ (r,δ) decreases as the risk level r increases. Maximum feasible margin δmax(r) _ (r) According to the definition in Eq. (A.21) of [14], we have dr≜infx≥0:Φ⁡(x)≤r,d_r \x≥ 0: (x)≤ r\, (A.8) with dr=+∞d_r=+∞ if the set is empty. The quantity δmax(r) _ (r) is defined as the largest prescribed margin for which the solution dδ⋆(r)d_δ (r) remains below drd_r, namely, δmax(r)≜supδ≥0:dδ⋆(r)<dr. _ (r) \δ≥ 0:d_δ (r)<d_r\. (A.9) From (A.7), dδ⋆(r)<drd_δ (r)<d_r is precisely the condition that ensures EΦ(r,δ)>0 E_ (r,δ)>0. Therefore, 0≤δ<δmax(r)0≤δ< _ (r) specifies the feasible range of prescribed margins. Since drd_r is non-increasing in r under Assumption , this feasible range becomes more restrictive as the risk level r increases. In Proposition , the generic risk level is r=o+εr= R^o+ _ opt. A.2 Proof of Proposition Proof. Since raw+=μmid+D μ_ raw^+= _ mid+D and raw−=μmid−D μ_ raw^-= _ mid-D, the two shifted expected margins in () are D+(μmid−χπ)andD−(μmid−χπ).D+( _ mid- _π) D-( _ mid- _π). (A.10) Their minimum is D−|μmid−χπ|D-| _ mid- _π|, which proves (). Since the absolute value term is minimized at zero, Δχ(f) _χ(f) is maximized when χπ=μmid _π= _ mid. Setting respectively χπ=0 _π=0 and χπ=(f) _π= μ_ emp(f) gives the two identities in (), and comparing them yields the condition (). Finally, k,(fk′)=k,(fk)+ok μ_k, emp(f _k)= μ_k, emp(f_k)+o_k, and hence () follows directly. Therefore, the centered local scores are identical before and after the additive offsets, and substituting them into () gives the same DeGroot iterates at every communication round. ∎ Remarks on centering. The achieved two-sided expected margin Δχ(f) _χ(f) should be distinguished from the prescribed target δ≥0δ≥ 0 used in the subsequent guarantees. In particular, Δχ(f) _χ(f) may be negative. An additive shift changes only the location of the two class means relative to threshold zero and leaves their separation 2D(f)2D(f) unchanged. Hence, if D⁡(f)≤0D(f)≤ 0, no additive center can produce a positive two-sided expected margin. The proposition concerns this expected margin criterion and does not imply that centering universally improves classification accuracy. For example, if the score is already the exact equal-prior log-likelihood ratio, zero is the Bayes threshold and a nonzero shift can worsen the resulting decision rule. A.3 Proof of Proposition First, we recall the δ-margin consistent training event associated with condition (): δ≜+(^)>δ,−(^)<−δ,C_δ \ μ^+( f)>δ,\, μ^-( f)<-δ \, (A.11) and Pc,δ≜ℙ⁡(δ)P_c,δ (C_δ). The proof extends the exact-ERM analysis of [14], which in turn uses uniform deviation bounds established in [13]. We retain the uniform concentration and margin-to-risk arguments from that analysis, while accounting explicitly for the nonzero empirical optimization gaps of the classifiers returned by the local training algorithms from (). The key new step is to show that these gaps replace the target risk o R^o in the exact-ERM argument by the effective risk level o+ε R^o+ _ opt. Proof. Uniform deviations. We introduce the empirical and population surrogate risks for the network as (f) R_ emp(f) ≜∑k=1Kπkk,(fk), _k=1^K _k R_k, emp(f_k), (A.12) ⁡(f) R(f) ≜∑k=1KπkΦ(fk(k)). _k=1^K _kE ( γf_k( h_k)). (A.13) Since ℱ=ℱ1×⋯×ℱKF=F_1×·s×F_K and the population risk separates across agents, inff∈ℱ⁡(f)=∑k=1Kπkinffk∈ℱk(k,)∼QkΦ(fk(k))=o _f R(f)= _k=1^K _k _f_k _kE_( h_k, γ) Q_k \! ( γf_k( h_k) )= R^o (A.14) where o R^o is defined in (). Let U U ≜supf∈ℱ|(f)−⁡(f)|, _f | R_ emp(f)- R(f)|, (A.15) Vμ V_μ ≜supf∈ℱ|(f)−μmid(f)|. _f | μ_ emp(f)- _ mid(f)|. (A.16) The following two uniform deviation bounds are the concentration results used in the exact-ERM analysis of [14], recalled there from Theorem 3 of [13]. For every x>4ρx>4ρ, ℙ⁡(Vμ≥x)≤exp⁡−Nmax(x−4ρ)22α2β2,P(V_μ≥ x)≤ \- N_ (x-4ρ)^22α^2β^2 \, (A.17) and, for every x>4LΦρx>4L_ ρ, ℙ⁡(U≥x)≤exp⁡−Nmax(x−4LΦρ)22α2β2LΦ2.P(U≥ x)≤ \- N_ (x-4L_ ρ)^22α^2β^2L_ ^2 \. (A.18) We use these established bounds directly. Effect of approximate empirical optimization. From the definition in (), condition () is equivalent to k,(^k)≤inffk∈ℱk,(fk)+εk, R_k, emp( f_k)≤ _f_k _k R_k, emp(f_k)+ _k, opt (A.19) for every agent k. Multiplying by πk _k and summing gives (^)≤∑k=1Kπkinffk∈ℱk,(fk)+ε. R_ emp( f)≤ _k=1^K _k _f_k _k R_k, emp(f_k)+ _ opt. (A.20) Because the coordinates fkf_k can be selected independently over the Cartesian product class ℱF, the weighted sum of the local infima is the infimum of the network empirical risk. Hence, (^)≤inff∈ℱ(f)+ε. R_ emp( f)≤ _f R_ emp(f)+ _ opt. (A.21) The infimum defining o R^o need not be attained. By the definition of U, for every f∈ℱf , (f)≤⁡(f)+U R_ emp(f)≤ R(f)+U. Hence, using (A.21), ⁡(^) R( f) ≤(^)+U ≤ R_ emp( f)+U ≤inff∈ℱ(f)+ε+U ≤ _f R_ emp(f)+ _ opt+U ≤inff∈ℱ⁡(f)+U+ε+U ≤ _f \ R(f)+U \+ _ opt+U =o+ε+2U = R^o+ _ opt+2U (A.22) where the last equality follows because U does not depend on f and o=inff∈ℱ⁡(f) R^o= _f R(f). Therefore, ⁡(^)≤o+ε+2U. R( f)≤ R^o+ _ opt+2U. (A.23) For convenience, define the effective risk level r≜o+ε.r R^o+ _ opt. (A.24) Thus, ⁡(^)≤r+2U. R( f)≤ r+2U. (A.25) This is the step that differs from the exact-ERM analysis, for which r=or= R^o. Reduction of margin failure to the deviation events. By Proposition , Δcen(^)=D⁡(^)−|(^)−μmid(^)|. _ cen( f)=D( f)- | μ_ emp( f)- _ mid( f) |. (A.26) Fix any constant d>δd>δ. If D⁡(^)>dD( f)>d and Vμ<d−δV_μ<d-δ hold simultaneously, then Δcen(^)>d−(d−δ)=δ. _ cen( f)>d-(d-δ)=δ. (A.27) Consequently, δ¯⊆Vμ≥d−δ∪D(^)≤d. C_δ \! \V_μ≥ d-δ \∪ \D( f)≤ d \. (A.28) We next use the same surrogate risk reduction as in the proof of the exact-ERM result in [14]. Under the uniform class prior, convexity of Φ and Jensen’s inequality give ⁡(f) R(f) =12∑k=1Kπk[(+1)Φ(fk(k))+(−1)Φ(−fk(k))] = 12 _k=1^K _k [E^(+1) (f_k( h_k))+E^(-1) (-f_k( h_k)) ] ≥12[Φ⁡(raw+(f))+Φ⁡(−raw−(f))] ≥ 12 [ \! ( μ_ raw^+(f) )+ \! (- μ_ raw^-(f) ) ] ≥Φ⁡(raw+(f)−raw−(f)2) ≥ \! ( μ_ raw^+(f)- μ_ raw^-(f)2 ) =Φ⁡(D⁡(f)). = (D(f)). (A.29) Thus, since Φ is non-increasing, D⁡(^)≤dD( f)≤ d implies ⁡(^)≥Φ⁡(d) R( f)≥ (d). Combining this implication with (A.23) yields D(^)≤d⊆U≥Φ⁡(d)−r2. \D( f)≤ d \ \U≥ (d)-r2 \. (A.30) Using (A.28) and (A.30), followed by the established deviation bounds in (A.17)–(A.18), gives ℙ⁡(δ¯) ( C_δ) ≤exp⁡−Nmax(d−δ−4ρ)22α2β2 ≤ \- N_ (d-δ-4ρ)^22α^2β^2 \ +exp⁡−Nmax2α2β2LΦ2(Φ⁡(d)−r2−4LΦρ)2, + \- N_ 2α^2β^2L_ ^2 ( (d)-r2-4L_ ρ )^2 \, (A.31) for every d>δd>δ satisfying d−δ>4ρd-δ>4ρ and (Φ⁡(d)−r)/2>4LΦρ( (d)-r)/2>4L_ ρ. Balancing the two deviations. We now use the same balancing construction as in [14], with the exact target risk o R^o replaced by the effective risk r defined in (A.24). Specifically, choose d=dδ⋆(r)d=d_δ (r) from Appendix A.1, which satisfies d−δ=Φ⁡(d)−r2LΦ.d-δ= (d)-r2L_ . (A.32) Under the conditions of Proposition , this choice is admissible. By (A.7), the two terms in (A.31) are then equal, and ℙ⁡(δ¯)≤2exp⁡−8Nmaxα2β2(EΦ(r,δ)−ρ)2.P( C_δ)≤ 2 \- 8N_ α^2β^2 ( E_ (r,δ)-ρ )^2 \. (A.33) Since Pc,δ=1−ℙ⁡(δ¯)P_c,δ=1-P( C_δ) and r=o+εr= R^o+ _ opt, we obtain (). When ε=0 _ opt=0, the effective risk reduces to r=or= R^o, which recovers the exact-ERM result of [14]. ∎ Interpretation of the optimization-gap condition. Proposition is conditional on the empirical optimality gap bounds in () for the score functions ^k f_k returned by local training. It does not provide an algorithm-specific guarantee that a prescribed εk, _k, opt is attained. In particular, when ℱkF_k is parameterized by a neural network, convexity of the scalar surrogate loss Φ does not imply convexity of the empirical training objective with respect to the network parameters. Any optimization guarantee for a particular training algorithm is therefore separate from the statistical statement of Proposition . Once such tolerances are available, their effect enters the bound through the quantity ε _ opt. A.4 Homogeneous Centralized-Training Comparison While the bound in Proposition is derived for independent local training, where each agent trains its own classifier without exchanging model updates, it is instructive to compare this setup against a centralized benchmark. For clarity, we focus on the homogeneous case where all agents share the same feature space, model class, and local population distribution. In the heterogeneous case, different agents may observe different feature distributions or spaces and use distinct architectures, so a single centralized model is not directly comparable without introducing additional modeling choices. For this comparison, suppose a total of N_ tot i.i.d. training samples from the common local population is available. A centralized learner uses all N_ tot samples to train one classifier, whereas independent training partitions the same sample budget across K agents. For a symmetric comparison, we consider an even split, Nk=NK,k=1,…,K.N_k= N_ totK, k=1,…,K. (A.34) We next examine the quantities that enter the bound in (). (i) Network complexity ρ: For many standard base models, such as feedforward neural networks with norm-constrained weights and kernel methods over bounded balls in reproducing kernel Hilbert spaces, the Rademacher complexity admits the standard O⁡(1/m)O(1/ m) dependence on the training set size m [13, 62, 63]. For the homogeneous setting with an even split, all local complexities are identical and therefore ρ=ℜN/K(ℱ),ρ=ℜN(ℱ)ρ ind= R_N_ tot/K(F), ρ cen= R_N_ tot(F) (A.35) where ℱF denotes the common function class. Hence, for model classes whose Rademacher complexity decreases with sample size as above, ρ≤ρ.ρ cen≤ρ ind. (A.36) (i) Function EΦ(o+ε,δ) E_ ( R^o+ _ opt,δ): Because the local population distribution and model class are common, the individual target risks are identical, and hence the target risk o R^o is the same for centralized and independent training. The loss function Φ is also unchanged. Let ε _ opt cen denote the empirical optimization tolerance of the centralized learner and let ε≜∑k=1Kπkεk, _ opt ind _k=1^K _k _k, opt (A.37) denote the corresponding π-weighted tolerance for independent training. If the two procedures are compared at matched optimization accuracy, ε=ε≜ε, _ opt cen= _ opt ind , (A.38) then EΦ(o+ε,δ) E_ ( R^o+ ,δ) is identical in the two bounds. If their optimization tolerances differ, the corresponding effective risk levels must instead be retained separately. The theory does not presume that either training procedure solves its empirical objective more accurately. (i) Admissible margin range: At matched optimization accuracy, the effective risk o+ε R^o+ and the loss Φ are the same under the two setups. Consequently, δmax(o+ε) _ ( R^o+ ), and hence the admissible range of δ, is also identical. The common model class also yields the same score bound β. Suppose now that δ lies in the common admissible range and that ρ<EΦ(o+ε,δ).ρ ind< E_ ( R^o+ ,δ). (A.39) Under the dependence of training set size described above, ρ≤ρ cen≤ρ ind, so Proposition applies to both training setups. For centralized training, taking K=1K=1, Nmax=N_ =N_ tot, and α=1α=1 gives Pc,δ≥1−2exp⁡−8Nβ2(EΦ(o+ε,δ)−ρ)2.P_c,δ cen≥ 1-2 \- 8N_ totβ^2 ( E_ ( R^o+ ,δ)-ρ cen )^2 \. (A.40) For independent training with the even split, Nmax=N/KN_ =N_ tot/K and α=1α=1, which gives Pc,δ≥1−2exp⁡−8NKβ2(EΦ(o+ε,δ)−ρ)2.P_c,δ ind≥ 1-2 \- 8N_ totKβ^2 ( E_ ( R^o+ ,δ)-ρ ind )^2 \. (A.41) Therefore, under this homogeneous comparison with matched optimization accuracy, the centralized benchmark benefits from both a larger effective size of training set in the exponent and, for the model classes considered above, a Rademacher complexity that is no larger. Consequently, it yields a tighter lower bound on Pc,δP_c,δ than that obtained by evenly splitting the same sample budget among independently trained models. This comparison makes explicit a statistical cost associated with independent training. Appendix B Sufficient Communication This appendix provides the prediction argument underlying the sufficient-communication guarantee in Section of the main paper. We first record the bounded-difference concentration inequality obtained by combining the class-conditional approximate tensorization of entropy (ATE) assumption with the entropy method. We then apply this inequality under each class-conditional testing law PγP_γ and combine the resulting prediction bound with the training guarantee from Proposition . The same ATE concentration tool will also be used in the subsequent prediction analyses of other appendices. We recall the σ-field generated by the training data and local training randomness in (): ≜σ⁡(1,…,K,1,…,K).T σ(D_1,…,D_K,S_1,…,S_K). (B.1) B.1 Bounded-Difference Inequality under ATE We first record the bounded-difference concentration consequence of ATE used throughout the prediction analysis. It follows by applying the standard entropy-method bounded-difference argument of [64] with the approximate tensorization inequality of [49] in place of exact tensorization. We use this established consequence without reproving it. Lemma B.1 (Bounded differences under ATE). Let X=(X1,…,XK)X=(X_1,…,X_K) have law P satisfying ATE with coefficient τ, in the sense of (). Suppose a bounded measurable function F satisfies |F⁡(x)−F⁡(x′)|≤dk|F(x)-F(x )|≤ d_k (B.2) whenever x and x′x differ only in coordinate k. Then, for every θ∈ℝθ , log⁡expθ⁡(F⁡(X)−F(X))≤τθ28∑k=1Kdk2. \θ(F(X)-EF(X))\≤ τθ^28 _k=1^Kd_k^2. (B.3) Consequently, if ∑k=1Kdk2>0 _k=1^Kd_k^2>0, then, for every u≥0u≥ 0, ℙ⁡(F⁡(X)−F(X)≤−u)≤exp⁡−2u2τ∑k=1Kdk2.P (F(X)-EF(X)≤-u )≤ \- 2u^2τ _k=1^Kd_k^2 \. (B.4) The same bound holds for the upper tail. If ∑k=1Kdk2=0 _k=1^Kd_k^2=0, then F is constant and the concentration statement is trivial. B.2 Proof of Theorem Fix a realization of the training phase that belongs to δC_δ. Conditional on T, the learned score functions and empirical training means are fixed, whereas the fresh testing pair remains independent of the training phase. Conditional on ∗=γ γ^*=γ, the joint testing observation ∗ h^* therefore has law PγP_γ. We bound the prediction error separately under the two class-conditional laws and then average over the uniform class prior. To lighten notation, throughout this proof we write f^k f_k, ckc_k, and c_ ave for the corresponding realized training-dependent quantities; the same convention will be used in the later prediction proofs. Proof. Suppose first that ∗=+1 γ^*=+1. From the definition of the aggregate classifier c_ ave in (), ∗(+1)c(∗) _ h^*^(+1)c_ ave( h^*) =∑k=1Kπkk∗(+1)ck(k∗) = _k=1^K _kE_ h_k^*^(+1)c_k( h_k^*) =μ+(f^)>δ =μ^+( f)>δ (B.5) where the last inequality follows because the fixed training realization belongs to δC_δ. Now consider another observation collection h that differs from ∗ h^* only in coordinate k. From () and Assumption , |c(∗)−c(^)| |c_ ave( h^*)-c_ ave( h) | =πk|ck(k∗)−ck(^k)| = _k |c_k( h_k^*)-c_k( h_k) | =πk|f^k(k∗)−f^k(^k)| = _k | f_k( h_k^*)- f_k( h_k) | ≤2βπk. ≤ 2β _k. (B.6) This reveals that the coordinate oscillations of c_ ave are dk=2βπkd_k=2β _k. Conditional on ∗=+1 γ^*=+1 and T, the testing observation has law P+1P_+1, which satisfies ATE with coefficient τ+1 _+1. Hence, following (B.5), Lemma B.1 gives ℙ⁡(c(∗)≤0|∗=+1,) \;P (c_ ave( h^*)≤ 0 2.0mu | 2.0mu γ^*=+1,T ) =ℙ⁡(c(∗)−∗(+1)c(∗)≤−μ+(f^)|∗=+1,) =P (c_ ave( h^*)-E_ h^*^(+1)c_ ave( h^*)≤-μ^+( f) 2.0mu | 2.0mu γ^*=+1,T ) ≤exp⁡−δ22τ+1β2∑k=1Kπk2. ≤ \- δ^22 _+1β^2 _k=1^K _k^2 \. (B.7) The final inequality follows from μ+(f^)>δμ^+( f)>δ on δC_δ. For ∗=−1 γ^*=-1, we similarly have ∗(−1)c(∗)=μ−(f^)<−δE_ h^*^(-1)c_ ave( h^*)=μ^-( f)<-δ (B.8) on δC_δ, and the same coordinate oscillations dk=2βπkd_k=2β _k apply. Accordingly, we have ℙ⁡(c(∗)≥0|∗=−1,)≤exp⁡−δ22τ−1β2∑k=1Kπk2.P (c_ ave( h^*)≥ 0 2.0mu | 2.0mu γ^*=-1,T )≤ \- δ^22 _-1β^2 _k=1^K _k^2 \. (B.9) Since ∗ γ^* is independent of T and has the uniform prior, the following holds ℙ⁡(ℳ|) (M 1.0mu| 1.0muT) =12ℙ(c(∗)≤0|∗=+1,) = 12P (c_ ave( h^*)≤ 0 2.0mu | 2.0mu γ^*=+1,T ) +12ℙ(c(∗)≥0|∗=−1,) + 12P (c_ ave( h^*)≥ 0 2.0mu | 2.0mu γ^*=-1,T ) ≤12∑γ∈Γexp⁡−δ22τγβ2∑k=1Kπk2 ≤ 12 _γ∈ \- δ^22 _γβ^2 _k=1^K _k^2 \ ≤exp⁡−δ22τmaxβ2∑k=1Kπk2 ≤ \- δ^22 _ β^2 _k=1^K _k^2 \ (B.10) almost surely on δC_δ. This proves the conditional bound () in Theorem . Since δC_δ is T-measurable and the preceding conditional bound holds almost surely on δC_δ, we have ⁡[[δ]ℙ(ℳ|)] \! [ 1[C_δ]P(M 1.0mu| 1.0muT) ] ≤exp⁡−δ22τmaxβ2∑k=1Kπk2ℙ(δ) ≤ \- δ^22 _ β^2 _k=1^K _k^2 \P(C_δ) ≤exp⁡−δ22τmaxβ2∑k=1Kπk2. ≤ \- δ^22 _ β^2 _k=1^K _k^2 \. (B.11) Combining (B.11) with the decomposition in () gives Pe≤exp⁡−δ22τmaxβ2∑k=1Kπk2+ℙ⁡(δ¯).P_e≤ \- δ^22 _ β^2 _k=1^K _k^2 \+P( C_δ). (B.12) If the assumptions of Proposition also hold, substituting the bound on ℙ⁡(δ¯)P( C_δ) from () into (B.12) gives Pe P_e ≤exp⁡−δ22τmaxβ2∑k=1Kπk2 ≤ \- δ^22 _ β^2 _k=1^K _k^2 \ +2exp⁡−8Nmaxα2β2(EΦ(o+ε,δ)−ρ)2. +2 \- 8N_ α^2β^2 ( E_ ( R^o+ _ opt,δ)-ρ )^2 \. (B.13) This is the explicit classification error bound under sufficient communication. ∎ Appendix C Finite-Round Communication Appendix B establishes the classification guarantee for the limiting case of sufficient communication. At a finite communication round, a similar concentration argument applies. The proof below reuses the conditioning convention and ATE bounded-difference inequality from Appendix B Proof. Fix a realization of the training phase that belongs to δC_δ. As in Appendix B.2, the learned functions, their empirical training means, and therefore the centered score functions are fixed conditional on T. We proceed first with the case ∗=+1 γ^*=+1. From (), we have ℙ⁡(ℳk,t|∗=+1,)=ℙ⁡(k,t(∗)≤0|∗=+1,)P (M_k,t 2.0mu | 2.0mu γ^*=+1,T )=P ( λ_k,t( h^*)≤ 0 2.0mu | 2.0mu γ^*=+1,T ) (C.1) where the argument of ∗ h^* in the notation k,t(∗) λ_k,t( h^*) emphasizes the dependence on the testing observation collection. Consider another observation collection h that differs from ∗ h^* only in coordinate ℓ , () gives |k,t(∗)−k,t(^)| | λ_k,t( h^*)- λ_k,t( h) | =|[At]ℓk(cℓ(ℓ∗)−cℓ(^ℓ))| = |[A^t]_ k (c_ ( h_ ^*)-c_ ( h_ ) ) | ≤2β[At]ℓk. ≤ 2β[A^t]_ k. (C.2) Thus, the coordinate oscillation associated with the ℓ testing view is dℓ=2β[At]ℓkd_ =2β[A^t]_ k. We next control the deviation in the class-conditional mean caused by incomplete mixing. Since |f^ℓ|≤β| f_ |≤β and its empirical training mean lies in [−β,β][-β,β], the realized centered class mean satisfies |μℓ+(f^ℓ)|≤2β| _ ^+( f_ )|≤ 2β. Hence, using (), |∗(+1)k,t(∗)−μ+(f^)| |E_ h^*^(+1) λ_k,t( h^*)-μ^+( f) | =|∑ℓ=1K([At]ℓk−πℓ)μℓ+(f^ℓ)| = | _ =1^K ([A^t]_ k- _ ) _ ^+( f_ ) | ≤2β∑ℓ=1K|[At]ℓk−πℓ| ≤ 2β _ =1^K |[A^t]_ k- _ | ≤2βKC(A,σ)σt. ≤ 2β KC(A,σ)σ^t. (C.3) Therefore, we have μ+(f^)>δμ^+( f)>δ when δC_δ is true. Consequently, for every t satisfying rt(δ)>0r_t(δ)>0, the following holds: ∗(+1)k,t(∗) _ h^*^(+1) λ_k,t( h^*) >δ−2βKC(A,σ)σt >δ-2β KC(A,σ)σ^t =rt(δ)>0. =r_t(δ)>0. (C.4) Conditional on ∗=+1 γ^*=+1 and T, the testing observation has law P+1P_+1. Therefore, using the similar arguments used in (B.7) yields ℙ⁡(ℳk,t|∗=+1,)≤exp⁡−rt2(δ)2τ+1β2∑ℓ=1K([At]ℓk)2.P (M_k,t 2.0mu | 2.0mu γ^*=+1,T )≤ \- r_t^2(δ)2 _+1β^2 _ =1^K ([A^t]_ k )^2 \. (C.5) Repeating the steps in (C.3)–(C.5) for the case ∗=−1 γ^*=-1, we can show that almost surely on δC_δ, ℙ⁡(ℳk,t|)≤exp⁡−rt2(δ)2τmaxβ2∑ℓ=1K([At]ℓk)2,P(M_k,t 1.0mu| 1.0muT)≤ \- r_t^2(δ)2 _ β^2 _ =1^K ([A^t]_ k )^2 \, (C.6) which proves the bound in (). If the conditions of Proposition also hold, substituting the bound on ℙ⁡(δ¯)P( C_δ) from () gives the following explicit upper bound for classification after t communication rounds: Pk,t P_k,t ≤exp⁡−rt2(δ)2τmaxβ2∑ℓ=1K([At]ℓk)2 ≤ \- r_t^2(δ)2 _ β^2 _ =1^K ([A^t]_ k )^2 \ +2exp⁡−8Nmaxα2β2(EΦ(o+ε,δ)−ρ)2. +2 \- 8N_ α^2β^2 ( E_ ( R^o+ _ opt,δ)-ρ )^2 \. (C.7) This completes the proof of Theorem . ∎ Appendix D Finite-Precision Communication Appendix C establishes the finite-round classification guarantee under exact real-valued message exchange. Theorem additionally accounts for stochastic finite-precision communication. Relative to the finite-round analysis, two new issues must be controlled: i) quantization errors accumulate as they propagate through the network, and i) their distributions are input-dependent, so that the resulting errors need not be independent across rounds or from the fresh testing views. We first derive a conditional sub-Gaussian bound for the accumulated quantization perturbation by unrolling the recursion and conditioning on the quantization history. We then combine this bound with the finite-round concentration bound established in Appendix C. Proof. For t=0t=0, no quantization has yet occurred: xk,0(b)=ck(k∗)=k,0x_k,0^(b)=c_k( h_k^*)= λ_k,0 and Vk,0q=0V_k,0 q=0. The claimed conditional bound therefore reduces to the corresponding real-valued finite-round bound. We henceforth consider t≥1t≥ 1. To isolate the effect of stochastic quantization, we condition on the training realization and the fresh testing example, so that only the rounding randomness remains. Let t(b)≜(x1,t(b),…,xK,t(b))⊤,tfp≜(x1,tfp,…,xK,tfp)⊤ x_t^(b) (x_1,t^(b),…,x_K,t^(b)) , x_t fp (x_1,t fp,…,x_K,t fp) (D.1) where tfp x_t fp corresponds to the matched full-precision recursion obtained from the same initialization by replacing b Q_b with the identity map. In particular, xk,tfp=k,tx_k,t fp= λ_k,t (D.2) where k,t λ_k,t is defined in (). Before analyzing the rounding errors, note that the prescribed quantization range is preserved by the recursion. From (), xk,0(b)=ck(k∗)∈[−2β,2β]x_k,0^(b)=c_k( h_k^*)∈[-2β,2β]. The outputs of the quantizer b Q_b remain in the same interval, and () forms each subsequent decision statistic as a convex combination of the transmitted values. Hence, by induction, xk,s(b)∈[−2β,2β]for every agent k and round s.x_k,s^(b)∈[-2β,2β] every agent $k$ and round $s$. (D.3) This indicates that the input to the stochastic quantizer remains within its prescribed range at every round. Define the sender-level rounding errors εℓ,sq≜qℓ,s(b)−xℓ,s(b),sq≜(ε1,sq,…,εK,sq)⊤. _ ,s q q_ ,s^(b)-x_ ,s^(b), _s q ( _1,s q,…, _K,s q) . (D.4) Let W=A⊤W=A . The finite-precision and matched full-precision recursions satisfy s+1(b)=Ws(b)+Wsq,s+1fp=Wsfp, x_s+1^(b)=W x_s^(b)+W _s q, x_s+1 fp=W x_s fp, (D.5) with the same initialization. Expanding the recursion therefore gives t(b)−tfp=∑s=0t−1Wt−ssq. x_t^(b)- x_t fp= _s=0^t-1W^t-s _s q. (D.6) Therefore, a rounding error generated at round s is propagated through Wt−sW^t-s before contributing to the decision statistic at round t. Let sQ_s denote the σ-field generated by T, the fresh testing pair (∗,∗)( h^*, γ^*), and all quantizer randomness through round s−1s-1. Conditional on sQ_s, the current decision statistics of each agent and therefore the active quantization bins are fixed. Following the stochastic rounding protocol (), the sender-level rounding errors at round s are conditionally independent across senders, satisfy ⁡[εℓ,sq|s]=0,E\! [ _ ,s q 2.0mu | 2.0muQ_s ]=0, (D.7) and each takes values in an interval of length at most Δb _b. Hence, for any sQ_s-measurable weights w1,…,wKw_1,…,w_K, Hoeffding’s lemma [44] gives, for every θ∈ℝθ , ⁡[exp⁡(θ∑ℓ=1Kwℓεℓ,sq)|s]≤exp⁡θ2Δb28∑ℓ=1Kwℓ2.E\! [ \! (θ _ =1^Kw_ _ ,s q ) |Q_s ]≤ \! \ θ^2 _b^28 _ =1^Kw_ ^2 \. (D.8) Given a fixed terminal pair (k,t)(k,t), the kkth coordinate of (D.6) can be written as xk,t(b)−xk,tfp=∑s=0t−1Us,Us≜∑j=1K[At−s]jkεj,sq.x_k,t^(b)-x_k,t fp= _s=0^t-1U_s, U_s _j=1^K[A^t-s]_jk _j,s q. (D.9) Applying (D.8) with wj=[At−s]jkw_j=[A^t-s]_jk gives ⁡[eθUs|s]≤exp⁡θ2Δb28∑j=1K([At−s]jk)2.E\! [e^θ U_s 2.0mu | 2.0muQ_s ]≤ \! \ θ^2 _b^28 _j=1^K([A^t-s]_jk)^2 \. (D.10) It is worth noting that the errors at different rounds need not be independent because the quantization bin at a later round depends on the preceding randomized updates. For this reason, in the following, we accumulate the bounds through repeated conditioning rather than by multiplying unconditional moment generating functions (MGFs). According to the law of total expectation, conditioning first on t−1Q_t-1 gives [exp(θ∑s=0t−1Us)|,∗,∗] \;E\! [ \! (θ _s=0^t-1U_s ) |T, h^*, γ^* ] =[exp(θ∑s=0t−2Us)[eθUt−1|t−1]|,∗,∗] =E\! [ \! (θ _s=0^t-2U_s )E\! [e^θ U_t-1 2.0mu | 2.0muQ_t-1 ] |T, h^*, γ^* ] ≤expθ2Δb28∑j=1K([A]jk)2[exp(θ∑s=0t−2Us)|,∗,∗]. ≤ \! \ θ^2 _b^28 _j=1^K([A]_jk)^2 \E\! [ \! (θ _s=0^t-2U_s ) |T, h^*, γ^* ]. (D.11) Repeating the same conditioning step for Ut−2,…,U0U_t-2,…,U_0 yields [exp(θ[xk,t(b)−xk,tfp])|,∗,∗] \;E\! [ \! (θ[x_k,t^(b)-x_k,t fp] ) |T, h^*, γ^* ] ≤exp⁡θ2Δb28∑r=1t∑j=1K([Ar]jk)2 ≤ \! \ θ^2 _b^28 _r=1^t _j=1^K([A^r]_jk)^2 \ =exp⁡θ2Vk,tq2 = \! \ θ^2V_k,t q2 \ (D.12) where Vk,tqV_k,t q is defined in (). This derivation requires conditional independence only among the sender-level rounding draws within each round; temporal independence of the rounding errors is not needed. We now combine the accumulated quantization perturbation with the fluctuation induced by the fresh testing views. Fix γ∈Γγ∈ and a training realization in δC_δ, and define μk,t(γ)≜[γxk,tfp|,∗=γ],Yk,t(γ)≜γxk,tfp−μk,t(γ). _k,t^(γ) \! [γ x_k,t fp 2.0mu | 2.0muT, γ^*=γ ], Y_k,t^(γ) γ x_k,t fp- _k,t^(γ). (D.13) Since xk,tfp=k,tx_k,t fp= λ_k,t, the finite-round mean argument leading to (C.4) in Appendix C, applied to either class, gives μk,t(γ)≥rt(δ). _k,t^(γ)≥ r_t(δ). (D.14) For fixed T and ∗=γ γ^*=γ, changing only the ℓ testing view changes γxk,tfpγ x_k,t fp by at most 2β[At]ℓk2β[A^t]_ k. Applying the MGF form of Lemma B.1, namely (B.3), under PγP_γ therefore gives [eθYk,t(γ)|,∗=γ]≤expθ22τγβ2∑ℓ=1K([At]ℓk)2E\! [e^θ Y_k,t^(γ) 2.0mu | 2.0muT, γ^*=γ ]≤ \! \ θ^22 _γβ^2 _ =1^K([A^t]_ k)^2 \ (D.15) for any θ∈ℝθ . Define the signed quantization perturbation: Zk,tq,γ≜γ⁡(xk,t(b)−xk,tfp).Z_k,t q,γ γ (x_k,t^(b)-x_k,t fp ). (D.16) Although the quantizer uses fresh auxiliary randomness, the distribution of Zk,tq,γZ_k,t q,γ depends on the realized testing example through the input-dependent quantization bins. Thus, Zk,tq,γZ_k,t q,γ and Yk,t(γ)Y_k,t^(γ) are not assumed to be independent. Instead, we condition first on the complete fresh testing example. Since Yk,t(γ)Y_k,t^(γ) is then fixed, the tower property and (D.12) give [eθ⁡(Yk,t(γ)+Zk,tq,γ)|,∗=γ] \;E\! [e^θ(Y_k,t^(γ)+Z_k,t q,γ) 2.0mu | 2.0muT, γ^*=γ ] =[eθYk,t(γ)[eθZk,tq,γ|,∗,∗=γ]|,∗=γ] =E\! [e^θ Y_k,t^(γ)E\! [e^θ Z_k,t q,γ 2.0mu | 2.0muT, h^*, γ^*=γ ] |T, γ^*=γ ] ≤expθ2Vk,tq2[eθYk,t(γ)|,∗=γ] ≤ \! \ θ^2V_k,t q2 \E\! [e^θ Y_k,t^(γ) 2.0mu | 2.0muT, γ^*=γ ] ≤exp⁡θ22[τγβ2∑ℓ=1K([At]ℓk)2+Vk,tq]. ≤ \! \ θ^22 [ _γβ^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \. (D.17) Therefore, the sub-Gaussian variance proxies associated with the testing views and quantization errors add in the conditional MGF bound. In particular, no unconditional independence between the two fluctuations is required. Suppose rt(δ)>0r_t(δ)>0. Since γxk,t(b)=μk,t(γ)+Yk,t(γ)+Zk,tq,γ,γ x_k,t^(b)= _k,t^(γ)+Y_k,t^(γ)+Z_k,t q,γ, (D.18) the event ℳk,t(b)=∗xk,t(b)≤0M_k,t^(b)=\ γ^*x_k,t^(b)≤ 0\ defined in (), conditional on ∗=γ γ^*=γ, together with (D.14), implies Yk,t(γ)+Zk,tq,γ≤−rt(δ).Y_k,t^(γ)+Z_k,t q,γ≤-r_t(δ). (D.19) Let Vk,t(γ)≜τγβ2∑ℓ=1K([At]ℓk)2+Vk,tq.V_k,t^(γ) _γβ^2 _ =1^K([A^t]_ k)^2+V_k,t q. (D.20) For any λ>0λ>0, Chernoff’s method and (D.17) give ℙ⁡(Yk,t(γ)+Zk,tq,γ≤−rt(δ)|,∗=γ) \;P (Y_k,t^(γ)+Z_k,t q,γ≤-r_t(δ) 2.0mu | 2.0muT, γ^*=γ ) ≤exp⁡−λrt(δ)+λ22Vk,t(γ). ≤ \! \-λ r_t(δ)+ λ^22V_k,t^(γ) \. (D.21) Optimizing at λ=rt(δ)/Vk,t(γ)λ=r_t(δ)/V_k,t^(γ) yields ℙ⁡(ℳk,t(b)|,∗=γ) \;P(M_k,t^(b) 2.0mu | 2.0muT, γ^*=γ) ≤exp⁡−rt2(δ)2[τγβ2∑ℓ=1K([At]ℓk)2+Vk,tq]. ≤ \! \- r_t^2(δ)2 [ _γβ^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \. (D.22) Since the fresh label is independent of T and has the uniform prior, averaging the two class-conditional bounds gives, almost surely on δC_δ, ℙ⁡(ℳk,t(b)|) (M_k,t^(b) 2.0mu | 2.0muT) ≤12∑γ∈Γexp⁡−rt2(δ)2[τγβ2∑ℓ=1K([At]ℓk)2+Vk,tq] ≤ 12 _γ∈ \! \- r_t^2(δ)2 [ _γβ^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \ ≤exp⁡−rt2(δ)2[τmaxβ2∑ℓ=1K([At]ℓk)2+Vk,tq], ≤ \! \- r_t^2(δ)2 [ _ β^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \, (D.23) which proves (). Using the decomposition in () with ℳM replaced by ℳk,t(b)M_k,t^(b), if the conditions of Proposition also hold, substituting its bound on ℙ⁡(δ¯)P( C_δ) gives Pk,t(b) P_k,t^(b) ≤exp⁡−rt2(δ)2[τmaxβ2∑ℓ=1K([At]ℓk)2+Vk,tq] ≤ \! \- r_t^2(δ)2 [ _ β^2 _ =1^K([A^t]_ k)^2+V_k,t q ] \ +2exp⁡−8Nmaxα2β2(EΦ(o+ε,δ)−ρ)2. +2 \- 8N_ α^2β^2 ( E_ ( R^o+ _ opt,δ)-ρ )^2 \. (D.24) This is the explicit unconditional finite-precision classification bound referred to in Theorem and completes the proof. ∎ Appendix E PAC-Style Generalization Guarantee This appendix proves Theorem of the main paper. Unlike the prediction guarantees in Appendices B–D, the PAC-style result does not rely on the expected margin event δC_δ or the class-conditional ATE assumption. Its sampling requirement is instead that the network examples (n,n)( h_n, γ_n) are i.i.d. across n, while the K local views within one example may remain statistically dependent. The main technical issue is the empirical centering operation in (), which leads the resulting classifier class to be sample-dependent. The proof handles this issue in three steps. We first bound the sensitivity of the uniform population–empirical risk gap to replacing one network example. We then embed every realized centered aggregate in a fixed offset-augmented function class and control its expected uniform deviation by Rademacher complexity. Finally, McDiarmid’s inequality yields a high-probability bound, which is specialized to the η-margin loss Φη _η in Section . Proof. We prove the collaborative bound in (), with the local bound following from the same argument by taking a single agent. Recall that =(n,n)n=1ND=\( h_n, γ_n)\_n=1^N (E.1) contains N i.i.d. complete network examples drawn from Q, and that k=(k,n,n)n=1ND_k=\( h_k,n, γ_n)\_n=1^N is its projection at agent k. The proof treats each complete network example as one independent sample coordinate and therefore never requires independence among the local views within an example. For fk∈ℱkf_k _k, we define its empirical training mean on kD_k and centered score by f¯k(k) f_k(D_k) ≜1N∑n=1Nfk(k,n), 1N _n=1^Nf_k( h_k,n), (E.2) fk,k(hk) c_f_k,D_k(h_k) ≜fk(hk)−f¯k(k). f_k(h_k)- f_k(D_k). (E.3) For f=(f1,…,fK)∈ℱf=(f_1,…,f_K) , let ,f(h)≜∑k=1Kπkfk,k(hk), c_ ave,f^D(h) _k=1^K _k c_f_k,D_k(h_k), (E.4) and, for a generic LΦL_ -Lipschitz loss Φ , define (f) R_ emp^D(f) ≜1N∑n=1NΦ⁡(n,f(n)), 1N _n=1^N \! ( γ_n c_ ave,f^D( h_n) ), (E.5) (f) R^D(f) ≜(,)∼QΦ(,f()). _( h, γ) Q \! ( γ c_ ave,f^D( h) ). (E.6) The superscript D emphasizes that, even for fixed f, the deployed centered aggregate score associated with c_ ave depends on the training sample. We therefore study Ω⁡()≜supf∈ℱ|(f)−(f)|. (D) _f | R^D(f)- R_ emp^D(f) |. (E.7) Bounded difference of Ω⁡() (D) Let ′D be obtained from D by replacing only the j-th complete network example (j,j)( h_j, γ_j) by (j′,j′)( h _j, γ _j). Using |supf|af|−supf|bf||≤supf|af−bf| | _f|a_f|- _f|b_f| |≤ _f|a_f-b_f| (E.8) and the triangle inequality, we obtain |Ω⁡()−Ω⁡(′)| | (D)- (D ) | ≤supf∈ℱ|(f)−′(f)| ≤ _f | R^D(f)- R^D (f) | +supf∈ℱ|(f)−′(f)|. + _f | R_ emp^D(f)- R_ emp^D (f) |. (E.9) Since only one observation changes in each projection kD_k and |fk|≤β|f_k|≤β by Assumption , one gets from (E.2) that |f¯k(k)−f¯k(k′)|≤2βN. | f_k(D_k)- f_k(D _k) |≤ 2βN. (E.10) For any f∈ℱf and any h, it follows that |,f(h)−,f′(h)| | c_ ave,f^D(h)- c_ ave,f^D (h) | =|∑k=1Kπk(f¯k(k′)−f¯k(k))| = | _k=1^K _k ( f_k(D _k)- f_k(D_k) ) | ≤∑k=1Kπk|f¯k(k′)−f¯k(k)| ≤ _k=1^K _k | f_k(D _k)- f_k(D_k) | ≤2βN ≤ 2βN (E.11) where we used ∑k=1Kπk=1 _k=1^K _k=1. By the triangle inequality and the Lipschitz property of Φ , supf∈ℱ|(f)−′(f)|≤2βLΦN. _f | R^D(f)- R^D (f) |≤ 2β L_ N. (E.12) For the empirical risk, we distinguish the unchanged samples from the replaced one. For an unchanged coordinate n≠jn≠ j, we have (n,n)=(n′,n′)( h_n, γ_n)=( h _n, γ _n), and therefore |n,f(n)−n′,f′(n′)| | γ_n c_ ave,f^D( h_n)- γ _n c_ ave,f^D ( h _n) | =|,f(n)−,f′(n)| = | c_ ave,f^D( h_n)- c_ ave,f^D ( h_n) | ≤2βN. ≤ 2βN. (E.13) At the replaced coordinate, |fk|≤β|f_k|≤β and |f¯k(k)|≤β| f_k(D_k)|≤β imply |,f(h)|≤2β.| c_ ave,f^D(h)|≤ 2β. (E.14) Hence, we have |j,f(j)−j′,f′(j′)| | γ_j c_ ave,f^D( h_j)- γ _j c_ ave,f^D ( h _j) | ≤|,f(j)|+|,f′(j′)| ≤ | c_ ave,f^D( h_j) |+ | c_ ave,f^D ( h _j) | ≤4β. ≤ 4β. (E.15) Applying the Lipschitz property of Φ to these two cases yields |(f)−′(f)| | R_ emp^D(f)- R_ emp^D (f) | ≤LΦN[(N−1)2βN+4β] ≤ L_ N [(N-1) 2βN+4β ] ≤6βLΦN, ≤ 6β L_ N, (E.16) for every f∈ℱf . Combining the two terms in (E.12) and (E.16) gives |Ω⁡()−Ω⁡(′)|≤8βLΦN. | (D)- (D ) |≤ 8β L_ N. (E.17) This replacement argument uses only bounded local scores and independence across complete network examples. No factorization of the joint law Q across agents is required, which implies that the conditional independence assumption of the local views across agents is not required. Expectation of Ω⁡() (D) A direct symmetrization of the function class consisting of ,f c_ ave,f^D, ∀f∈ℱ∀ f would incorrectly treat a sample-dependent class as fixed. To address this issue, we instead embed every realized centered classifier ,f c_ ave,f^D in a deterministic augmented class. Define af()≜∑k=1Kπkf¯k(k).a_f(D) _k=1^K _k f_k(D_k). (E.18) Since |af()|≤β|a_f(D)|≤β and ,f(h)=f(h)−af(),f(h)≜∑k=1Kπkfk(hk), c_ ave,f^D(h)=f_ ave(h)-a_f(D), f_ ave(h) _k=1^K _kf_k(h_k), (E.19) every realized centered classifier belongs to the fixed class off≜h↦f(h)−a:f∈ℱ,a∈[−β,β].G_off \h f_ ave(h)-a:f ,\ a∈[-β,β] \. (E.20) Define the corresponding fixed-class uniform deviation Δoff()≜supg∈off|Φ(g())−1N∑n=1NΦ⁡(ng(n))|. _off(D) _g _off |E ( γg( h))- 1N _n=1^N ( γ_ng( h_n)) |. (E.21) Then Ω⁡()≤Δoff(). (D)≤ _off(D). (E.22) To apply the contraction inequality in its standard form, define Φ~(u)≜Φ(u)−Φ(0). (u) (u)- (0). (E.23) Then Φ~(0)=0 (0)=0, Φ~ remains LΦL_ -Lipschitz, and replacing Φ by Φ~ does not change any population–empirical risk difference. Let g≜(ng,ng)n=1ND g \( h_n g, γ_n g) \_n=1^N (E.24) be an independent ghost sample drawn from the same joint distribution Q. Since offG_off is independent of both D and gD g, Jensen’s inequality gives Δoff() \;E_D _off(D) =supg∈off|g1N∑n=1N[Φ~(ngg(ng))−Φ~(ng(n))]| =E_D _g _off |E_D g 1N _n=1^N [ \! ( γ_n gg( h_n g) )- \! ( γ_ng( h_n) ) ] | ≤,gsupg∈off|1N∑n=1N[Φ~(ngg(ng))−Φ~(ng(n))]|. _D,D g _g _off | 1N _n=1^N [ \! ( γ_n gg( h_n g) )- \! ( γ_ng( h_n) ) ] |. (E.25) Introducing N independent Rademacher variables 1,…,N r_1,…, r_N and using the usual symmetrization argument, we obtain Ω() (D) ≤Δoff() _off(D) ≤2,supg∈off|1N∑n=1NnΦ~(ng(n))| ≤ 2E_D, r _g _off | 1N _n=1^N r_n \! ( γ_ng( h_n) ) | ≤4LΦ,supg∈off|1N∑n=1Nnng(n)| ≤ 4L_ E_D, r _g _off | 1N _n=1^N r_n γ_ng( h_n) | (E.26) where the last step follows from the contraction inequality for Rademacher averages [62, 44]. For g⁡(h)=f(h)−a∈offg(h)=f_ ave(h)-a _off, using the definition of f_ ave in (), we further have supg∈off|1N∑n=1Nnng(n)| _g _off | 1N _n=1^N r_n γ_ng( h_n) | ≤∑k=1Kπksupfk∈ℱk|1N∑n=1Nnnfk(k,n)| ≤ _k=1^K _k _f_k _k | 1N _n=1^N r_n γ_nf_k( h_k,n) | +β|1N∑n=1Nnn|. +β | 1N _n=1^N r_n γ_n |. (E.27) Conditional on the sample set D, the variables nn r_n γ_n are again independent Rademacher signs. Therefore, using the definition ρ=∑kπkρkρ= _k _k _k from Appendix A.1, ,supg∈off|1N∑n=1Nnng(n)| \;E_D, r _g _off | 1N _n=1^N r_n γ_ng( h_n) | ≤ρ+β|1N∑n=1Nn| ≤ρ+ _ r | 1N _n=1^N r_n | ≤ρ+βN ≤ρ+ β N (E.28) Thus, Ω()≤4LΦρ+4βLΦN.E (D)≤ 4L_ ρ+ 4β L_ N. (E.29) High-probability bound Applying McDiarmid’s inequality [65] to (E.17) gives, for every x>0x>0, ℙ⁡(Ω⁡()−Ω()≥x)≤exp⁡−Nx232β2LΦ2.P ( (D)-E (D)≥ x )≤ \- Nx^232β^2L_ ^2 \. (E.30) Taking x=4βLΦ2log⁡(1/ϵ)Nx=4β L_ 2 (1/ε)N (E.31) and combining with (E.29), we obtain, with probability at least 1−ϵ1-ε, uniformly for all f∈ℱf , |(f)−(f)| | R^D(f)- R_ emp^D(f) | ≤4LΦρ+4βLΦN ≤ 4L_ ρ+ 4β L_ N +4βLΦ2log⁡(1/ϵ)N. +4β L_ 2 (1/ε)N. (E.32) We now take Φ=Φη = _η, the η-margin loss in (), for which LΦ=1/ηL_ =1/η. Using the definition of ℛ⁡(ϵ,N)R(ε,N) in (), the preceding display becomes |(f)−(f)|≤4ρη+ℛ⁡(ϵ,N). | R^D(f)- R_ emp^D(f) |≤ 4ρη+R(ε,N). (E.33) For a given training set D, (f) R^D(f) is the expected η-margin loss of the deployed classifier ,f c_ ave,f^D, while (f) R_ emp^D(f) denotes its empirical value. Hence () gives Pe(,f)≤Pe,η(,f)+4ρη+ℛ⁡(ϵ,N).P_e( c_ ave,f^D)≤ P_e, emp^η( c_ ave,f^D)+ 4ρη+R(ε,N). (E.34) The high-probability event is uniform over f∈ℱf and depends only on the aligned network training set D. Therefore, given a fixed D, (E.34) may be evaluated at any data-dependent outputs ^k=k(k,k)∈ℱk f_k=A_k(D_k,S_k) _k, for every realization of the local training randomness, without any empirical optimality condition. This establishes (). Applying the same argument to the projected sample kD_k for a fixed agent k gives () and completes the proof. ∎ Remark on a refinement. The preceding proof is formulated for a generic LΦL_ -Lipschitz loss and controls the effect of replacing one complete network example through the bounded score range. This produces the factor βLΦβ L_ in the confidence term. For the η-margin loss Φη _η, a sharper concentration argument is available because 0≤Φη≤10≤ _η≤ 1. Specifically, after embedding the empirically centered classifiers in the fixed offset-augmented class offG_off, define the one-sided population–empirical deviation Δoff,+()≜supg∈offΦη(g())−1N∑n=1NΦη(ng(n)). _off,+(D) _g _off \E _η( γg( h))- 1N _n=1^N _η( γ_ng( h_n)) \. (E.35) Standard one-sided symmetrization and the non-absolute contraction inequality in [44] give Δoff,+()≤2η(ρ+βN).E _off,+(D)≤ 2η (ρ+ β N ). (E.36) Moreover, replacing one complete network example changes this fixed-class uniform deviation by at most 1/N1/N, since the η-margin loss takes values in [0,1][0,1]. Thus, the bounded-difference constant for Δoff,+() _off,+(D) is 1/N1/N, and McDiarmid’s inequality yields the alternative confidence term log⁡(1/ϵ)2N. (1/ε)2N. (E.37) Consequently, the collaborative bound associated with the η-margin loss can be refined to Pe()≤Pe,η()+2ρη+2βηN+log⁡(1/ϵ)2N.P_e( c_ ave)≤ P_e, emp^η( c_ ave)+ 2ρη+ 2βη N+ (1/ε)2N. (E.38) The analogous refinement for a local classifier is Pe(k)≤Pe,η(k)+2ρkη+2βηN+log⁡(1/ϵ)2N.P_e( c_k)≤ P_e, emp^η( c_k)+ 2 _kη+ 2βη N+ (1/ε)2N. (E.39) These refinements use the boundedness of Φη _η and a one-sided fixed-class argument. Theorem retains the bound obtained from the generic Lipschitz-loss argument above. Appendix F Relation to Social Learning This appendix returns to the prediction dynamics and relates the DeGroot collaboration rule () used in the main paper to social learning (SL) based on geometric pooling. We will first formulate the corresponding static-observation specialization. Then, we establish its finite-round classification guarantee, and compare the two update mechanisms. F.1 Static Observation Specialization A defining feature of the DeGroot model () is that agents update their opinions k,t λ_k,t solely by averaging those of their neighbors, without acquiring new observations over time. In contrast, SL rules are designed for inference in streaming environments, where agents continuously receive new observations and combine them with information aggregated from their neighbors. To connect these two inference mechanisms, we consider the geometric-pooling SL mechanism [35], specialized to the learned local decision statistics used in social machine learning [13, 14]: k,t=∑ℓ=1Kaℓk(ℓ,t−1+ℓ(ℓ,t)) λ_k,t SL= _ =1^Ka_ k ( λ_ ,t-1 SL+ c_ ( h_ ,t) ) (F.1) where ℓ,t h_ ,t denotes the private observation received by agent ℓ at round t. The linear recursion in (F.1) is the log-domain form of the corresponding geometric SL update. Unrolling it gives k,t=∑ℓ=1K[At]ℓkℓ,0+∑i=1t∑ℓ=1K[At+1−i]ℓkℓ(ℓ,i). λ SL_k,t= _ =1^K[A^t]_ k λ SL_ ,0+ _i=1^t _ =1^K[A^t+1-i]_ k c_ ( h_ ,i). (F.2) We assume no prior information about the class label, so that k,0=0 λ SL_k,0=0. In the static observation specialization, ℓ,t=ℓ∗ h_ ,t= h_ ^* for every t, and therefore k,t=∑ℓ=1K∑i=1t[Ai]ℓkℓ(ℓ∗). λ_k,t SL= _ =1^K _i=1^t[A^i]_ k c_ ( h_ ^*). (F.3) As the same testing example is reused at every round, the repeated terms in (F.3) do not represent new temporal evidence. Instead, they repeatedly accumulate the statistics associated with the same K local views. Since [Ai]ℓk→πℓ[A^i]_ k→ _ , its Cesàro averages satisfy limt→∞1tk,t _t→∞ 1t λ_k,t SL =∑ℓ=1Klimt→∞1t∑i=1t[Ai]ℓkℓ(ℓ∗) = _ =1^K _t→∞ 1t _i=1^t[A^i]_ k c_ ( h_ ^*) =∑ℓ=1Kπℓℓ(ℓ∗)=(∗). = _ =1^K _ c_ ( h_ ^*)= c_ ave( h^*). (F.4) Hence, whenever (∗)≠0 c_ ave( h^*)≠ 0, the prediction of every agent eventually agrees with the collective prediction based on the aggregate classifier c_ ave. The sufficient-communication guarantee of Theorem therefore applies to the limiting SL prediction as well. For the finite-round analysis, we define ^k,t=sgn(k,t),ℳk,t=∗k,t≤0. γ_k,t SL= ( λ_k,t SL), _k,t SL=\ γ^* λ_k,t SL≤ 0\. (F.5) Similar to Theorems –, we analyze the conditional probability ℙ⁡[ℳk,t|]P[M_k,t SL 1.0mu| 1.0muT]. To quantify the cumulative deviation of the finite-round weights from the Perron vector, define A≜maxk∈∑i=1∞∑ℓ=1K|[Ai]ℓk−πℓ|, B_A _k _i=1^∞ _ =1^K|[A^i]_ k- _ |, (F.6) which is finite under Assumption . For every σ∈(σA,1)σ∈( _A,1) and corresponding C⁡(A,σ)C(A,σ) in (), it holds that A≤KC(A,σ)σ1−σ. B_A≤ KC(A,σ)σ1-σ. (F.7) With this definition, we establish the classification guarantee for the SL rule. Proposition F.1 (Static-observation SL classification guarantee). Suppose Assumptions , , and hold, and let δ>0δ>0. Define κ≜2βA.κ 2β B_A. (F.8) Almost surely on δC_δ, for every agent k∈k and every integer t>κ/δt>κ/δ, ℙ⁡(ℳk,t|)≤exp⁡−(tδ−κ)22τmaxβ2∑ℓ=1K(∑i=1t[Ai]ℓk)2. (M_k,t SL 2.0mu| 2.0muT)≤ \- (tδ-κ)^22 _ β^2 _ =1^K ( _i=1^t[A^i]_ k )^2 \. (F.9) This bound converges to the sufficient-communication bound for c_ ave as t→∞t→∞. F.2 Proof of Proposition F.1 Proof. The proof reuses the conditioning convention and ATE bounded-difference inequality from Appendix B. Relative to the finite-round DeGroot analysis presented in Appendix C, the only change is that the SL statistic uses the cumulative weights ∑i=1t[Ai]ℓk _i=1^t[A^i]_ k rather than the single-round weights [At]ℓk[A^t]_ k. Fix a realization of the training phase in δC_δ. Conditional on T, the trained classifiers are fixed, while the fresh testing vector follows PγP_γ under ∗=γ γ^*=γ. Since the same testing view is reused at every round, its repeated appearances are not treated as independent coordinates. Instead, changing view ℓ changes all of its accumulated contributions simultaneously. From (F.3), changing only coordinate ℓ of the testing vector changes the statistic by at most |k,t(∗)−k,t(^)| | λ_k,t SL( h^*)- λ_k,t SL( h) | ≤∑i=1t[Ai]ℓk|cℓ(ℓ∗)−cℓ(^ℓ)| ≤ _i=1^t[A^i]_ k |c_ ( h_ ^*)-c_ ( h_ ) | ≤2β∑i=1t[Ai]ℓk. ≤ 2β _i=1^t[A^i]_ k. (F.10) Thus Lemma B.1 applies with coordinate oscillations dℓ,k,t=2β∑i=1t[Ai]ℓk.d_ ,k,t SL=2β _i=1^t[A^i]_ k. (F.11) We next control the class-conditional mean. For either γ∈Γγ∈ , |∗(γ)k,t(∗)−tμγ(f^)| \; |E_ h^*^(γ) λ_k,t SL( h^*)-tμ^γ( f) | =|∑i=1t∑ℓ=1K([Ai]ℓk−πℓ)μℓγ(f^ℓ)| = | _i=1^t _ =1^K ([A^i]_ k- _ )μ^γ_ ( f_ ) | ≤2β∑i=1t∑ℓ=1K|[Ai]ℓk−πℓ| ≤ 2β _i=1^t _ =1^K |[A^i]_ k- _ | ≤2βA=κ ≤ 2β B_A=κ (F.12) where |μℓγ(f^ℓ)|≤2β|μ^γ_ ( f_ )|≤ 2β follows from |cℓ|≤2β|c_ |≤ 2β. For ∗=+1 γ^*=+1, the event δC_δ implies μ+(f^)>δμ^+( f)>δ, so for t>κ/δt>κ/δ, ∗(+1)k,t(∗)>tδ−κ>0.E_ h^*^(+1) λ_k,t SL( h^*)>tδ-κ>0. (F.13) The lower-tail form of Lemma B.1 therefore gives ℙ⁡(ℳk,t|∗=+1,) \;P (M_k,t SL 2.0mu | 2.0mu γ^*=+1,T ) ≤exp⁡−(tδ−κ)22τ+1β2∑ℓ=1K(∑i=1t[Ai]ℓk)2. ≤ \- (tδ-κ)^22 _+1β^2 _ =1^K ( _i=1^t[A^i]_ k )^2 \. (F.14) A similar bound is obtained for the case ∗=−1 γ^*=-1 by replacing τ+1 _+1 with τ−1 _-1. The inequality in (F.9) is then established using the definition of τmax _ in (). Finally, limt→∞1t∑i=1t[Ai]ℓk=πℓ, _t→∞ 1t _i=1^t[A^i]_ k= _ , (F.15) which implies limt→∞1t2∑ℓ=1K(∑i=1t[Ai]ℓk)2=∑ℓ=1Kπℓ2. _t→∞ 1t^2 _ =1^K ( _i=1^t[A^i]_ k )^2= _ =1^K _ ^2. (F.16) Since (tδ−κ)2/t2→δ2(tδ-κ)^2/t^2→δ^2, we obtain limt→∞(tδ−κ)22τmaxβ2∑ℓ=1K(∑i=1t[Ai]ℓk)2=δ22τmaxβ2∑ℓ=1Kπℓ2, _t→∞ (tδ-κ)^22 _ β^2 _ =1^K ( _i=1^t[A^i]_ k )^2= δ^22 _ β^2 _ =1^K _ ^2, (F.17) which is the exponent that appears in the bound () for the aggregate classifier c_ ave under sufficient communication. This completes the proof. ∎ F.3 Comparison with the DeGroot Rule By Jensen’s inequality, the following holds: ∑ℓ=1K(∑i=1t[Ai]ℓk)2 _ =1^K ( _i=1^t[A^i]_ k )^2 ≤t∑ℓ=1K∑i=1t([Ai]ℓk)2 ≤ t _ =1^K _i=1^t ([A^i]_ k )^2 ≤t∑ℓ=1K∑i=1t[Ai]ℓk=t2 ≤ t _ =1^K _i=1^t[A^i]_ k=t^2 (F.18) where the last equality uses left-stochasticity of A. Hence, for every agent, ℙ⁡(ℳk,t|)≤exp⁡−(tδ−κ)22τmaxβ2t2P (M_k,t SL 2.0mu| 2.0muT )≤ \- (tδ-κ)^22 _ β^2t^2 \ (F.19) almost surely on δC_δ whenever t>κ/δt>κ/δ. The finite-round DeGroot and static observation SL statistics are generally different: the former uses the weights [At]ℓk[A^t]_ k in (), whereas the latter accumulates ∑i=1t[Ai]ℓk _i=1^t[A^i]_ k in (F.3). Accordingly, neither finite-round analytical bound is asserted to vary monotonically with t, because its denominator depends on the corresponding finite-round mixing weights. Nevertheless, () and (F.4) show that, whenever (∗)≠0 c_ ave( h^*)≠ 0, both mechanisms eventually yield the same collective prediction γ in (). Their error bounds likewise converge to the sufficient-communication bound. A particularly transparent connection arises for the fully connected consensus matrix A=π⊤A=π 1 . In this case, Am=A^m=A for every integer m≥1m≥ 1, and for every t≥1t≥ 1, k,t=∑ℓ=1Kπℓℓ(ℓ∗)=(∗), λ_k,t= _ =1^K _ c_ ( h_ ^*)= c_ ave( h^*), (F.20) whereas k,t=t∑ℓ=1Kπℓℓ(ℓ∗)=t(∗). λ_k,t SL=t _ =1^K _ c_ ( h_ ^*)=t c_ ave( h^*). (F.21) Therefore, the two statistics have the same sign and produce the same prediction after one communication round. Moreover, exact mixing is achieved after one communication round, since [At]ℓk=πℓ,t≥1.[A^t]_ k= _ , t≥ 1. (F.22) Hence, for t≥1t≥ 1, using the exact identity [At]ℓk=πℓ[A^t]_ k= _ directly in the finite-round DeGroot argument makes the transient mean error identically zero, and the resulting specialized concentration bound coincides with the sufficient-communication bound. For the SL rule, A=0 B_A=0 and hence κ=0κ=0, while ∑i=1t[Ai]ℓk=tπℓ. _i=1^t[A^i]_ k=t _ . (F.23) Its finite-round bound therefore reduces to the same sufficient-communication bound for every t≥1t≥ 1. Finally, the two recursions use different initial conditions because they represent different inference semantics. For SL, k,0=0 λ_k,0 SL=0 is the state before the first observation is incorporated. For the DeGroot rule, k,0=k(k∗) λ_k,0= c_k( h_k^*) is the local decision statistic available before communication. This discrepancy at t=0t=0 does not affect the preceding finite-round or limiting comparison. Appendix G Raw-Score Counterparts The main analysis uses the empirically centered local scores k c_k in (). As discussed in Proposition , empirical centering acts as a threshold alignment mechanism rather than a structural requirement for collaboration. This appendix provides the corresponding results when the learned scores ^k f_k are used directly, without empirical centering. The raw aggregate score f_ ave and its class-conditional means rawγ μ_ raw^γ are already defined in (). Moreover, the raw expected zero-threshold margin is Δraw(f) _ raw(f) in (). Since the arguments closely parallel those developed for the centered scores c_ ave in Appendices A–F, we state only the changes needed for the raw-score counterparts. G.1 Raw-Score Training Guarantee For 0≤δ<β0≤δ<β and a generic risk level r, we define EΦraw(r,δ)≜Φ⁡(δ)+Φ⁡(β)−2r16LΦ. E_ ^raw(r,δ) (δ)+ (β)-2r16L_ . (G.1) For the learned functions ^=^1,…,^K f=\ f_1,…, f_K\, we denote the raw-score δ-margin training event by δraw _δ^raw ≜Δraw(^)>δ \ _ raw( f)>δ \ =raw+(^)>δ,raw−(^)<−δ = \ μ_ raw^+( f)>δ,\, μ_ raw^-( f)<-δ \ (G.2) where the equality follows from (), () and (). This is the raw-score counterpart of the centered condition (). Similar to Proposition , we can establish a bound on the achievability of (G.2) as follows. Corollary G.1 (Raw-score δ-margin consistent training). Suppose Assumptions – hold and the approximate optimization condition () is satisfied. Let r≜o+εr R^o+ _ opt (G.3) where ε _ opt is defined in (). If 0≤δ<β,r<Φ⁡(δ)+Φ⁡(β)2,ρ<EΦraw(r,δ)0≤δ<β, r< (δ)+ (β)2, ρ< E_ ^raw(r,δ) (G.4) then ℙ⁡(δraw)≥1−exp⁡−8Nmaxα2β2(EΦraw(r,δ)−ρ)2.P(C_δ^raw)≥ 1- \- 8N_ α^2β^2 ( E_ ^raw(r,δ)-ρ )^2 \. (G.5) Proof. The analysis on the approximate optimization remains unchanged from the proof of Proposition . With the argument in Appendix A.3, we have ⁡(^)≤r+2U,r=o+ε R( f)≤ r+2U, r= R^o+ _ opt (G.6) where U is the uniform risk deviation in (A.15). It remains to relate failure of the δ-margin condition (G.2) to the population surrogate risk. We introduce the following two class-conditional risks: +(f) R^+(f) ≜∑k=1Kπk(+1)Φ(fk(k)), _k=1^K _kE^(+1) (f_k( h_k)), (G.7) −(f) R^-(f) ≜∑k=1Kπk(−1)Φ(−fk(k)). _k=1^K _kE^(-1) (-f_k( h_k)). (G.8) Under the uniform class prior, ⁡(f)=[+(f)+−(f)]/2 R(f)=[ R^+(f)+ R^-(f)]/2. By the same Jensen argument used in (A.29), +(f)≥Φ⁡(raw+(f)),−(f)≥Φ⁡(−raw−(f)). R^+(f)≥ ( μ_ raw^+(f) ), R^-(f)≥ (- μ_ raw^-(f) ). (G.9) Moreover, Assumption and the non-increasing property of Φ imply +(f)≥Φ⁡(β),−(f)≥Φ⁡(β). R^+(f)≥ (β), R^-(f)≥ (β). (G.10) If δrawC_δ^raw fails, one of the two class-conditional risks is at least Φ⁡(δ) (δ), while the other is at least Φ⁡(β) (β). Therefore, δraw¯⊆(^)≥Φ⁡(δ)+Φ⁡(β)2. C_δ^raw \ R( f)≥ (δ)+ (β)2 \. (G.11) Combining (G.11) with (G.6) gives, on δraw¯ C_δ^raw, U≥Φ⁡(δ)+Φ⁡(β)−2r4=4LΦEΦraw(r,δ). U≥ (δ)+ (β)-2r4=4L_ E_ ^raw(r,δ). (G.12) Applying the uniform deviation bound (A.18) at this threshold proves (G.5). ∎ Compared with Proposition , the change in the bound (G.5) occurs in the reduction from margin event failure to population risk. The centered analysis additionally controls the empirical training mean, whereas the raw margin in () is determined directly by the two raw class-conditional means. G.2 Sufficient-Communication Counterpart Set f=^f= f in the raw aggregate score (), and define ℳraw≜∗f(∗)≤0.M^raw \ γ^*f_ ave( h^*)≤ 0\. (G.13) This is the raw-score counterpart of the event ℳM in (). When δrawC_δ^raw is true, the class-conditional mean of f(∗)f_ ave( h^*) lies on the correct side of zero by more than δ. Moreover, changing only testing view k changes f(∗)f_ ave( h^*) by at most 2βπk2β _k, which is the same coordinate oscillation used for the centered aggregate in Appendix B.2. Applying Lemma B.1 under the two class-conditional laws PγP_γ and using the definition of τmax _ gives ℙ⁡(ℳraw|)≤exp⁡−δ22τmaxβ2∑k=1Kπk2P(M^raw 1.0mu| 1.0muT)≤ \- δ^22 _ β^2 _k=1^K _k^2 \ (G.14) almost surely on δrawC_δ^raw. Thus, for the same prescribed margin δ, the prediction-side bound (G.14) has exactly the same form as its centered counterpart in (). Applying the same tower-property decomposition as in (), with ℳM and δC_δ replaced by ℳrawM^raw and δrawC_δ^raw, respectively, gives ℙ⁡(ℳraw) (M^raw) =⁡[ℙ⁡(ℳraw|)] =E\! [P(M^raw 1.0mu| 1.0muT) ] ≤⁡[[δraw]ℙ(ℳraw|)]+ℙ⁡(δraw¯). \! [ 1[C_δ^raw]P(M^raw 1.0mu| 1.0muT) ]+P( C_δ^raw). (G.15) Combining (G.15) with (G.14) and the raw-score training guarantee (G.5) gives the corresponding unconditional guarantee. G.3 Finite-Round Counterpart Without empirical centering, the finite-round statistic corresponding to () becomes k,traw≜∑ℓ=1K[At]ℓk^ℓ(ℓ∗), λ_k,t^raw _ =1^K[A^t]_ k f_ ( h_ ^*), (G.16) with ℳk,traw≜∗k,traw≤0.M_k,t^raw \ γ^* λ_k,t^raw≤ 0\. (G.17) The finite-round mixing argument pertaining to k,traw λ_k,t^raw follows Appendix C. The coordinate oscillation remains 2β[At]ℓk2β[A^t]_ k. The only change in the expectation term follows from |(γ)^ℓ(ℓ)|≤β, |E^(γ) f_ ( h_ ) |≤β, (G.18) instead of the bound 2β2β for empirically centered scores ℓ(ℓ) c_ ( h_ ). Using the matrix-power estimate in (), |(γ)k,traw−rawγ(^)|≤βKC(A,σ)σt. |E^(γ) λ_k,t^raw- μ_ raw^γ( f) |≤β KC(A,σ)σ^t. (G.19) Accordingly, define the raw-score counterpart of the residual margin in () by rtraw(δ)≜δ−βKC(A,σ)σt.r_t^raw(δ) δ-β KC(A,σ)σ^t. (G.20) Repeating the argument used to establish Theorem gives, for every agent k and every t≥0t≥ 0 such that rtraw(δ)>0r_t^raw(δ)>0, ℙ⁡(ℳk,traw|)≤exp⁡−[rtraw(δ)]22τmaxβ2∑ℓ=1K([At]ℓk)2P(M_k,t^raw 1.0mu| 1.0muT)≤ \- [r_t^raw(δ)]^22 _ β^2 _ =1^K([A^t]_ k)^2 \ (G.21) almost surely on δrawC_δ^raw. Therefore, relative to the bound in (), only the residual margin changes: rt(δ)=δ−2βKC(A,σ)σt⟶rtraw(δ)=δ−βKC(A,σ)σt.r_t(δ)=δ-2β KC(A,σ)σ^t r_t^raw(δ)=δ-β KC(A,σ)σ^t. (G.22) The corresponding unconditional guarantee follows by combining (G.21) with (G.5), in the same manner as the centered finite-round decomposition (). G.4 Finite-Precision Counterpart We next consider the raw-score counterpart of the finite-precision recursion (). Since |^k(h)|≤β| f_k(h)|≤β by Assumption , the raw inputs can be quantized over [−β,β][-β,β] rather than the centered range [−2β,2β][-2β,2β] in (). Accordingly, we employ a narrower raw-score alphabet braw≜vjraw=−β+jΔbraw:j=0,…,Lb−1V_b^raw \v_j^raw=-β+j _b^raw:j=0,…,L_b-1 \ (G.23) where Δbraw≜2βLb−1=Δb2. _b^raw 2βL_b-1= _b2. (G.24) We apply the same adjacent-level stochastic rounding protocol as in () using this raw-score alphabet. The initialization and all quantizer outputs lie in [−β,β][-β,β], and the update in () is a convex combination. Therefore, this interval is preserved throughout the recursion. The accumulated perturbation term corresponding to () is Vk,tq,raw V_k,t^q,raw ≜(Δbraw)24∑r=1t∑j=1K([Ar]jk)2 ( _b^raw)^24 _r=1^t _j=1^K([A^r]_jk)^2 =β2(Lb−1)2∑r=1t∑j=1K([Ar]jk)2 = β^2(L_b-1)^2 _r=1^t _j=1^K([A^r]_jk)^2 =14Vk,tq. = 14V_k,t q. (G.25) Let xk,t(b),rawx_k,t^(b),raw denote the recursion () initialized by xk,0(b),raw=^k(k∗)x_k,0^(b),raw= f_k( h_k^*) (G.26) and using the raw quantization range above. Define ℳk,t(b),raw≜∗xk,t(b),raw≤0.M_k,t^(b),raw \ γ^*x_k,t^(b),raw≤ 0\. (G.27) The conditional-MGF argument in Appendix D is unchanged. Replacing rt(δ)r_t(δ) and Vk,tqV_k,t q in () respectively by rtraw(δ)r_t^raw(δ) and Vk,tq,rawV_k,t^q,raw therefore gives ℙ⁡(ℳk,t(b),raw|) \;P(M_k,t^(b),raw 2.0mu| 2.0muT) ≤exp⁡−[rtraw(δ)]22[τmaxβ2∑ℓ=1K([At]ℓk)2+Vk,tq,raw] ≤ \- [r_t^raw(δ)]^22 [ _ β^2 _ =1^K([A^t]_ k)^2+V_k,t^q,raw ] \ (G.28) almost surely on δrawC_δ^raw whenever rtraw(δ)>0r_t^raw(δ)>0. Thus, relative to the centered result (), the raw-score counterpart changes only the residual finite-round margin rtraw(δ)r_t^raw(δ) and the quantization proxy Vk,tq,rawV_k,t^q,raw. The unconditional guarantee follows by combining (G.28) with (G.5), as in Theorem . G.5 PAC-Style Generalization Counterpart We next consider the PAC-style result under the aligned network training set () and its local projections (). The raw aggregate f_ ave in () does not contain the sample-dependent empirical training mean that appears in the centered aggregate. Therefore, the fixed-class refinement technique described at the end of Appendix E applies directly without the additional offset variable. For the raw local and aggregate scores, respectively, define Pe(fk) P_e(f_k) ≜ℙ⁡(sgn(fk(k∗))≠∗|), ( (f_k( h_k^*))≠ γ^* 2.0mu| 2.0muT ), (G.29) Pe(f) P_e(f_ ave) ≜ℙ⁡(sgn(f(∗))≠∗|). ( (f_ ave( h^*))≠ γ^* 2.0mu| 2.0muT ). (G.30) We establish the analogous PAC-style bounds for f_ ave and fkf_k as follows. Corollary G.2 (Raw-score margin generalization bound). Under the sampling setup of Theorem , fix η>0η>0 and ϵ∈(0,1)ε∈(0,1). Then, with probability at least 1−ϵ1-ε, uniformly for all f∈ℱf , Pe(f)≤Pe,η(f)+2ρη+log⁡(1/ϵ)2N.P_e(f_ ave)≤ P_e, emp^η(f_ ave)+ 2ρη+ (1/ε)2N. (G.31) Moreover, for every fixed agent k, with probability at least 1−ϵ1-ε, uniformly for all fk∈ℱkf_k _k, Pe(fk)≤Pe,η(fk)+2ρkη+log⁡(1/ϵ)2N.P_e(f_k)≤ P_e, emp^η(f_k)+ 2 _kη+ (1/ε)2N. (G.32) Proof. Following (E.35), we define the one-sided deviation Δ+raw()≜supf∈ℱΦη(f())−Pe,η(f) _+^raw(D) _f \E _η( γf_ ave( h))-P_e, emp^η(f_ ave) \ (G.33) where Φη _η and Pe,ηP_e, emp^η are defined in () and (), respectively. Since the function class consisting of f_ ave, ∀f∈ℱ∀ f is fixed, the one-sided symmetrization and contraction argument used in the refinement of Appendix E gives Δ+raw()≤2η∑k=1Kπkρk=2ρη.E _+^raw(D)≤ 2η _k=1^K _k _k= 2ρη. (G.34) Moreover, 0≤Φη≤10≤ _η≤ 1, so replacing one complete network example changes Δ+raw() _+^raw(D) by at most 1/N1/N. McDiarmid’s inequality therefore gives, with probability at least 1−ϵ1-ε, Δ+raw()≤2ρη+log⁡(1/ϵ)2N. _+^raw(D)≤ 2ρη+ (1/ε)2N. (G.35) Using () proves (G.31). Since this high-probability event is uniform over f∈ℱf , it may be evaluated at f=^f= f for every realization of the local training randomness. Applying the same argument to the projected sample kD_k gives (G.32). ∎ Relative to the centered refinement in Appendix E, the offset-complexity term is absent in Corollary G.2 because the raw-score classifier does not contain an empirical training mean. As already characterized by () and (), this difference does not imply a universal ordering of the resulting classifiers, since empirical centering also changes their margins relative to the zero decision threshold. G.6 Static-Observation Social Learning Counterpart Replacing the centered decision statistic k,t λ_k,t SL in (F.3) by the learned raw scores gives k,t,≜∑ℓ=1K∑i=1t[Ai]ℓk^ℓ(ℓ∗). λ_k,t SL,raw _ =1^K _i=1^t[A^i]_ k f_ ( h_ ^*). (G.36) We define ℳk,t,≜∗k,t,≤0.M_k,t SL,raw \ γ^* λ_k,t SL,raw≤ 0\. (G.37) Using the cumulative mixing constant A B_A in (F.6), we denote κraw≜βA=κ2 _raw β B_A= κ2 (G.38) where κ=2βAκ=2β B_A is defined in (F.8). The proof of Proposition F.1 carries over directly. The cumulative coordinate oscillations are unchanged, while |(γ)^ℓ(ℓ)|≤β|E^(γ) f_ ( h_ )|≤β reduces the cumulative mean-transient term in (F.12) from κ to κraw _raw. Consequently, almost surely on δrawC_δ^raw, for every agent k and every integer t>κraw/δt> _raw/δ, ℙ⁡(ℳk,t,|)≤exp⁡−(tδ−κraw)22τmaxβ2∑ℓ=1K(∑i=1t[Ai]ℓk)2.P(M_k,t SL,raw 2.0mu| 2.0muT)≤ \- (tδ- _raw)^22 _ β^2 _ =1^K ( _i=1^t[A^i]_ k )^2 \. (G.39) This is the direct raw-score counterpart of (F.9). Since limt→∞1t∑i=1t[Ai]ℓk=πℓ, _t→∞ 1t _i=1^t[A^i]_ k= _ , (G.40) the bound (G.39) converges to the bound associated with raw scores under sufficient communication (G.14). Appendix H Additional Experimental Details This appendix provides additional implementation details and supplementary figures for the experiments conducted in Section . The presentation is grouped into the CIFAR-10 benchmark, the ModelNet40 benchmark, and the communication and robustness studies. H.1 CIFAR-10 Benchmark H.1.1 Experimental Protocol This subsection provides additional implementation details for the CIFAR-10 patch-partition benchmark introduced in Section . Each agent trains a convolutional neural network (CNN) independently on the patch associated with its grid location. The network consists of two convolutional layers followed by fully connected layers and produces a two-dimensional output zk(k)∈ℝ2z_k( h_k) ^2 for each input patch k h_k [14]. Its two entries, denoted by zk,+1(k)z_k,+1( h_k) and zk,−1(k)z_k,-1( h_k), are the logits associated with the two classes, which define the class-posterior probabilities through the softmax rule: p^k(γ|k)=exp⁡(zk,γ(k))exp⁡(zk,+1(k))+exp⁡(zk,−1(k)) p_k(γ| h_k)= (z_k,γ( h_k)) (z_k,+1( h_k))+ (z_k,-1( h_k)) (H.1) for γ∈+1,−1γ∈\+1,-1\. Here, these logits refer to the raw outputs of the neural network, and should not be confused with the theoretical logit function in (). In the binary case, their difference satisfies zk,+1(k)−zk,−1(k)=log⁡p^k(+1|k)p^k(−1|k).z_k,+1( h_k)-z_k,-1( h_k)= p_k(+1| h_k) p_k(-1| h_k). (H.2) Therefore, in this implementation, the scalar quantity used for collaboration is first formed by taking the difference between the two logits. This logit difference is then centered according to () to obtain the local decision statistic k(k) c_k( h_k). For each value of N0N_0, we first sample a balanced set of CIFAR-10 images containing equal numbers of cats and dogs. The same selected images are then used across all agents, with each agent retaining only the patch corresponding to its own grid location. In this way, N0N_0 denotes the size of the local labeled dataset at each agent before the train/validation split. We reserve 20%20\% of each local dataset for validation and use the remaining 80%80\% for training. Figure H.1 illustrates this construction: each row corresponds to the local dataset of one agent, and each column shows the local patches extracted from a common image across agents. Fig. H.1: Illustration of the local labeled datasets in the CIFAR-10 patch-partition benchmark. The highlighted columns illustrate two representative images from the two classes contributing different local patches to all agents. Each local CNN classifier is trained independently using the Adam optimizer with learning rate 10−410^-4, mini-batch size 256256, and cross-entropy loss for at most 100100 epochs. Early stopping is applied in all CIFAR-10 experiments: training is monitored on the validation set and terminated once the validation loss ceases to improve over several consecutive validation checks, and the model parameters corresponding to the best validation loss are retained. For each value of N0N_0, all reported quantities in Fig. are averaged over 200200 Monte Carlo repetitions. In each repetition, the balanced image set is resampled, the train/validation split is regenerated, and all local classifiers are retrained from scratch. In the main baseline comparison, the communication graph is generated once and then kept fixed across repetitions. Each agent is also assumed to have a self-loop in the communication graph, which is omitted from Fig. for clarity. The corresponding combination policy A is constructed using the uniform averaging rule: aℓk=1|k|,∀ℓ∈ka_ k= 1|N_k|, ∀ _k (H.3) where |k||N_k| denotes the cardinality of kN_k. H.1.2 Baselines Figure compares the following methods. Non-cooperative Each agent predicts independently from its own local decision statistic, without any communication or aggregation. Avg-stat The local decision statistics are averaged uniformly across agents, and the final prediction is obtained by thresholding the resulting aggregate. This provides a simple averaging reference, and coincides with the proposed method when the combination policy A is doubly-stochastic and the communication is sufficient. Vote Each agent first generates a hard class prediction based on its local decision statistic. The final output is then determined by majority vote across the agents. Learned fusion A centralized affine fusion rule is fitted on a held-out validation set. If ⁡(h)∈ℝK c(h) ^K denotes the vector of local decision statistics for sample h, the baseline predicts according to the sign of w⊤(h)+bfusw c(h)+b_ fus (H.4) where both the fusion weights w∈ℝKw ^K and the bias bfus∈ℝb_ fus are learned from validation data. Learned simplex fusion This baseline also learns a centralized linear fusion rule from validation data, but under the structural constraints wk≥0,∑k=1Kwk=1,w_k≥ 0, _k=1^Kw_k=1, (H.5) with no additive bias term. It may therefore be interpreted as a learned convex combination of the local decision statistics. AdaBoost A centralized boosting model is trained using the local predictors as constituent learners. Unlike the proposed method, this baseline introduces cooperation during training rather than only at test time. VFL-JT To provide a task-matched joint training comparator for the partitioned CIFAR-10 data, we implement a score-level vertical federated joint-training scheme inspired by the feature-distributed learning framework in [57]. For an aligned example, branch k receives the same local patch as the corresponding independently trained agent and outputs a scalar binary score sks_k. The coordinating server forms uVFL=bVFL+∑k=1Kwksk,u_ VFL=b_ VFL+ _k=1^Kw_ks_k, (H.6) and the branch parameters together with the fusion head are optimized jointly using the global binary logistic loss. During training, the local scores are sent to the server and the corresponding loss derivatives with respect to the local scores are returned to the branches. Raw patches and local parameter tensors are not exchanged, and no parameter averaging is performed. Because VFL-JT and the proposed method communicate at different stages, we report their training- and inference-time communication separately. Let EVFLE_ VFL denote the realized number of VFL-JT training epochs, and NtrN_ tr and NvalN_ val the numbers of aligned training and validation samples, respectively. Recall that Eoff(A)E_ off(A) defined in () denotes the number of directed non-self communication links. The resulting communication costs, measured by the number of transmissions, are summarized in Table H.1. TABLE H.1: Communication cost of VFL-JT and the proposed scheme. Method Training Inference per sample VFL-JT KEVFL(2Ntr+Nval)KE_ VFL(2N_ tr+N_ val) K Proposed 00 tEoff(A)tE_ off(A) For VFL-JT, each training sample involves the transmission of K local scores to the server and the return of K corresponding loss derivatives, while each validation sample requires the K forward score transmissions. For the proposed method, communication occurs only during test-time collaboration over the directed non-self links of the network. If each communicated scalar is represented using b bits, the corresponding bit costs are obtained by multiplying the entries in Table H.1 by b. Central oracle A centralized classifier from the same CNN family is trained on the full 32×3232× 32 CIFAR-10 image. In this implementation, the first convolutional layer is adapted to the full-image input, and the first fully connected layer is resized accordingly to match the resulting feature dimension. This baseline is included only as a full-information reference and is not constrained by the distributed observation model. Ours (no centering) This ablation implements the same collaboration rule as the proposed method, but removes the centering step in (). In other words, each agent uses the trained score ^k f_k rather than the centered decision statistic k c_k. H.1.3 Conditional Mutual Information In Section , the estimation of the mutual information I⁡(hk;hℓ|γ=y)I(h_k;h_ 1.0mu| 1.0muγ=y) appearing in () is carried out in two stages. For the reported results in Figs. and , the conditional mutual information is estimated on the training split using all 10,00010,000 selected training samples. We also repeated the same analysis on the testing split and obtained the same qualitative conclusions. First, for each agent k, the raw patch vectors are stacked into a data matrix whose rows correspond to samples and whose columns correspond to pixel values, and principal component analysis (PCA) is then applied to this matrix [45]. A reduced representation of dimension 2020 is retained. The PCA transformation is fitted once per agent on the chosen split, and the resulting reduced coordinates are then partitioned according to the class label. Second, for each class y∈−1,+1y∈\-1,+1\, the mutual information between the two reduced patch representations is estimated by a k-nearest-neighbor (kNN) estimator with neighborhood parameter kNN=5k_ N=5 [58]. The two class-conditional estimates are then combined according to (). All reported values are given in nats. We also evaluated PCA dimensions 1010, 2020, and 3030, together with neighborhood parameters kNN=3k_N=3, 55, and 1010. Across these choices, the same qualitative conclusions are found. H.1.4 Heterogeneous Models This subsection provides additional details for the heterogeneous version of the CIFAR-10 patch-partition benchmark in Section . We consider three local model families: 1. Family A: the patch-level CNN adopted in the main CIFAR-10 experiments; 2. Family B: logistic regression on the flattened raw patch pixels; 3. Family C: a small residual convolutional network with adaptive pooling [59]. All other aspects of the experiment remain unchanged from the homogeneous CIFAR-10 benchmark, including the local datasets, the train/validation splitting protocol, the communication graph, the combination policy, and the training hyperparameters. Because different patch locations are not equally informative, the placement of a model family on the 3×33× 3 grid can affect its apparent performance. To reduce this spatial bias, we use a balanced assignment protocol based on the base layout: (1)=ABCBCACABF^(1)= array[]|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em| 5.0pt @intercol A 5.0pt 5.0pt& B 5.0pt 5.0pt& C 5.0pt\\ 5.0pt @intercol B 5.0pt 5.0pt& C 5.0pt 5.0pt& A 5.0pt\\ 5.0pt @intercol C 5.0pt 5.0pt& A 5.0pt 5.0pt& B 5.0pt\\ array (H.7) where A, B, and C denote the three model families. A cyclic shift of this pattern yields two additional layouts: (2)=BCACABABC,(3)=CABABCBCA.F^(2)= array[]|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em| 5.0pt @intercol B 5.0pt 5.0pt& C 5.0pt 5.0pt& A 5.0pt\\ 5.0pt @intercol C 5.0pt 5.0pt& A 5.0pt 5.0pt& B 5.0pt\\ 5.0pt @intercol A 5.0pt 5.0pt& B 5.0pt 5.0pt& C 5.0pt\\ array\;\;, ^(3)= array[]|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em|@ 0.5emc@ 0.5em| 5.0pt @intercol C 5.0pt 5.0pt& A 5.0pt 5.0pt& B 5.0pt\\ 5.0pt @intercol A 5.0pt 5.0pt& B 5.0pt 5.0pt& C 5.0pt\\ 5.0pt @intercol B 5.0pt 5.0pt& C 5.0pt 5.0pt& A 5.0pt\\ array\;. (H.8) In the study of collaboration gain at a fixed N0N_0 illustrated by Fig. , the values are averaged over these three layouts so that each family appears equally often at each spatial location. In the N0N_0-sweep of Fig. , the base layout (1)F^(1) is kept fixed. For each agent k, we quantify the performance gain from collaboration at round t by Δk(t)≜Pk,0−Pk,t _k(t) P_k,0-P_k,t (H.9) where we recall that Pk,tP_k,t denotes the probability of error of agent k at communication round t. For the heatmap shown in Fig. , we report the corresponding relative error reduction at the representative communication round t=15t=15: Δk(%)(15)≜100⋅Δk(15)max⁡Pk,0,ϵsafe _k^(\%)(15) 100· _k(15) \P_k,0, _safe\ (H.10) where ϵsafe=10−6 _safe=10^-6 is a small numerical safeguard. H.2 ModelNet40 Benchmark The ModelNet40 experiments follow the same general pipeline as the CIFAR-10 benchmark in Appendix H.1, except for the differences described below. The dataset is constructed from CAD models by rendering multiple views of each object using Blender. For each object, we generate K=12K=12 RGB views from fixed camera positions whose azimuth angles are uniformly spaced around the object, with a common elevation angle of 30∘30 . All rendered images are generated at resolution 224×224224× 224 with transparent background. The local classifier is a lightweight convolutional network, referred to as MicroCNN. It consists of a single convolutional layer with kernel size 7×77× 7 and stride 44, followed by a ReLU nonlinearity, global average pooling, an optional dropout layer, and a final linear layer producing two logits. As in the CIFAR-10 experiments, the local score is formed from the difference between two logits and then centered according to (). For each value of N0N_0, we sample a balanced set of objects containing N0/2N_0/2 examples from each class and split it into training and validation subsets using the same 20%20\% validation fraction as in the CIFAR-10 study. The local models are trained independently using cross-entropy loss and validation-based early stopping, but with SGD optimizer and learning rate 0.050.05. For the VFL-JT comparison, we use the same score-level joint-training construction described in Appendix H.1.2, with K=12K=12 view-specific MicroCNN branches. Centralized full-observation reference In the ModelNet40 benchmark, the central oracle has access to all 1212 views of the same object. The model consists of K view-specific MicroCNN backbones, one for each rendered view, whose feature vectors are concatenated and passed to a centralized multilayer head for final prediction. The oracle is initialized from the independently trained local models and then trained in two stages. In the first stage, all backbone parameters are frozen and only the fusion head is optimized. In the second stage, all parameters are unfrozen and the full model is fine-tuned jointly using a smaller learning rate. Both training stages use cross-entropy loss, the Adam optimizer, and validation-based early stopping. H.2.1 Temperature Scaling For each agent k, we denote by zk(k)∈ℝ2z_k( h_k) ^2 the two-dimensional output of its local classifier for feature vector k h_k, whose entries are the logits associated with the two classes. A positive scalar temperature Tk>0T_k>0 is fitted on the validation set by minimizing the cross-entropy loss of the rescaled logits: minTk>0⁡1Nval∑n=1NvalℓCE(zk(k,n)Tk,γn) _T_k>0\; 1N_ val _n=1^N_ val _ CE\! ( z_k( h_k,n)T_k,\, _n ) (H.11) where ℓCE _ CE denotes the cross-entropy loss, and (k,n,γn)( h_k,n, _n) denotes the n-th labeled example in the validation set. In the implementation, we optimize over log⁡Tk T_k in order to enforce positivity. After fitting TkT_k, the classifier output is replaced by zk(k)/Tkz_k( h_k)/T_k. The local score used for collaboration is then formed by taking the difference between the two calibrated logits and is subsequently centered according to (). Additional temperature scaling diagnostics Figure H.2 summarizes the fitted temperature parameters on the ModelNet40 benchmark across training set sizes N0N_0. For each value of N0N_0, we report the average of TkT_k over the agents together with a 95%95\% confidence interval across 200200 Monte Carlo repetitions, as well as the average run-wise standard deviation of Tkk=1K\T_k\_k=1^K. A learned temperature Tk=1T_k=1 leaves the logits unchanged, Tk>1T_k>1 softens them by reducing their magnitude, and Tk<1T_k<1 sharpens them. The figure shows that the fitted temperatures are generally greater than one, and that both their average level and their variability across agents decrease as N0N_0 increases. This trend supports the conclusion that calibration is most critical in data-constrained regimes and becomes more stable as local classifier’s reliability improves with additional training data. Figure H.3 reports the corresponding temperature diagnostics for the CIFAR-10 benchmark considered in Fig. . Similar to ModelNet40, the fitted temperatures are larger and more variable at smaller training set sizes. As N0N_0 increases, however, they move closer to one, indicating that the logits require less rescaling in the higher-data regime. The corresponding results of decision margin and probability of error with temperature scaling are shown in Figs. H.4a and H.4b, respectively. Compared with Figs. and , temperature scaling tends to increase the achieved decision margin and slightly lowers the probability of error, with the effect being most visible at smaller N0N_0. Fig. H.2: Temperature scaling diagnostics for the ModelNet40 benchmark. The orange curve shows the average of TkT_k over agents, with error bars indicating 95%95\% confidence intervals across Monte Carlo repetitions. The blue curve shows the average run-wise standard deviation of Tkk=1K\T_k\_k=1^K, which quantifies the variability of the fitted temperatures across agents. Fig. H.3: Temperature scaling diagnostics for the CIFAR-10 benchmark. (a) Decision margin (b) Probability of error Fig. H.4: Performance of temperature scaling on CIFAR-10. H.2.2 Correlation Stress Test This subsection provides additional details for the controlled correlation stress test studied in Section . The perturbed decision statistics xi,k′\x _i,k\ are generated from the clean centered statistics xi,k\x_i,k\ using the perturbation model in () and (). After perturbation, the decision statistics over the test set are collected into a matrix X∈ℝNtest×K,X ^N_test× K, (H.12) whose (i,k)(i,k)-th entry is the perturbed decision statistic of agent k for the i-th testing sample, where NtestN_test denotes the number of testing samples. To quantify the induced dependence, we compute the unconditional average pairwise Pearson correlation across agents: corruncond(X)≜2K⁡(K−1)∑1≤k<ℓ≤Kr^kℓ(X)corr_ uncond(X) 2K(K-1) _1≤ k< ≤ K r_k (X) (H.13) where r^kℓ(X) r_k (X) denotes the sample Pearson correlation between columns k and ℓ of X. The label-conditional counterpart is obtained by restricting X to each class separately and then averaging the corresponding pairwise correlations over the two classes. In addition to these dependence measures, Fig. reports the collaboration gain after t communication rounds, measured by P0−Pt=1K∑k=1K(Pk,0−Pk,t)P_0-P_t= 1K _k=1^K(P_k,0-P_k,t) (H.14) where P0P_0 denotes the average probability of error across agents in the non-cooperative setting and PtP_t is the corresponding average probability of error after t rounds of collaboration at the same perturbation level. For completeness, Fig. H.5 reports the corresponding non-cooperative and collaborative probabilities of error as functions of the dependence strength r. For a fixed perturbation amplitude σpert _pert, varying r changes the balance between the shared and idiosyncratic perturbation components while preserving the marginal perturbation distribution at each agent. This explains why the non-cooperative error remains approximately constant as r varies. In contrast, the collaborative error generally increases with r, since a stronger shared component makes the perturbations across agents more correlated and reduces the benefit of combining their information. Consequently, the gap between the non-cooperative and collaborative curves narrows as r increases, consistent with the reduction in collaboration gain observed in Fig. . Increasing σpert _pert, on the other hand, increases the perturbation magnitude and raises the overall error levels. The clean reference curves show the corresponding performance without the injected perturbations. Fig. H.5: Probability of error under the controlled correlation stress test. For each perturbation amplitude σpert _pert, the figure compares the non-cooperative error at t=0t=0 and the collaborative error after t=20t=20 communication rounds as functions of the dependence strength r. The clean non-cooperative and collaborative error levels are also shown for reference. H.3 Communication and Robustness This subsection collects the implementation details for the communication design, adaptive stopping, and robustness studies in Section of the main paper. H.3.1 Topology and Quantization This subsection provides additional implementation details for the communication design study in Section . Figure H.6 illustrates the ring and grid communication topologies used in the experiments. (a) Ring (b) Grid Fig. H.6: Ring and grid communication topologies used in Fig. . Self-loops are omitted from the plots for visual clarity. For each topology-rule pair, the graph and the corresponding combination policy A are generated once and then kept fixed throughout the Monte Carlo study. In this study, we choose N0=400N_0=400, which corresponds to all available training samples for the selected class pair. The training and testing sets are therefore kept fixed across repetitions, while the train/validation split and the training randomness are varied. For each finite-bit condition, the results are additionally averaged over 3030 independent quantizer realizations within each of the 200200 training repetitions. In this way, the comparison isolates the effect of the communication design from variability due to changes in the underlying graph or data. For the Metropolis rule used in the simulations, the communication weights are defined by aℓk=1max⁡|k|,|ℓ|,if ℓ≠k,ℓ∈k,1−∑m∈k∖kamk,if ℓ=k,0,otherwise.a_ k= cases 1 \|N_k|,|N_ |\,&if ≠ k, _k,\\ 1- _m _k \k\a_mk,&if =k,\\ 0,&otherwise. cases (H.15) For the undirected ring and grid topologies, this is the standard Metropolis rule in the literature [10], which ensures that the resulting combination policy A is doubly-stochastic. In the case of the directed Erdös–Rényi topology, the inclusion of self-loops at all agents guarantees that A is well defined and left-stochastic by construction (H.15). However, because the graph is directed, the policy does not generally remain doubly-stochastic. For the finite-precision study, we use the bounded posterior-probability difference gk(h)g_k(h) defined in (). This statistic is obtained directly from the same trained classifier, and no retraining is performed. Its empirical training mean μk,(gk) _k, emp(g_k) is recomputed from the training statistics in this representation, so that the centered statistic lies in [−2,2][-2,2]. For b∈4,6,8b∈\4,6,8\, we use Lb=2bL_b=2^b uniformly spaced reconstruction levels on [−2,2][-2,2], with step size Δb=42b−1. _b= 42^b-1. (H.16) The adjacent-level stochastic rounding protocol b Q_b and the quantize-before-average recursion () are those defined in Section . Quantization is applied at each sender before averaging, including the first exchange. For each testing sample and communication round, every sender draws one fresh quantization outcome independently across senders, rounds, and samples, and broadcasts the same quantized value to all of its outgoing neighbors. Since both stochastic quantization and left-stochastic averaging preserve the interval [−2,2][-2,2], no overload clipping is required. The matched full-precision reference uses the same trained classifiers, bounded scores, empirical centers, graph, and combination policy, but replaces the stochastic quantizer by the identity map. Thus, the finite-bit branches differ from the full-precision reference only through stochastic quantization. For the finite-bit branches, the per-round communication cost is measured in transmitted bits per testing sample: Bround=bEoff(A)B_ round=bE_ off(A) (H.17) where Eoff(A)E_ off(A) is defined in () and excludes self-loops from the communication count. Additional discussion of Fig. Figures H.7–H.9 provide alternative views of the results in Fig. , organized according to graph family, the construction rule of A, and communication precision. These views help separate the role of each factor in collaborative classification. Across the graph families considered here, the uniform averaging rule and the Metropolis rule generally produce different combination matrices A. One exception is the ring graph, for which the two rules generate the same matrix A. Accordingly, Fig. H.7a presents coincident trajectories at all tested precisions. For the grid graph, Fig. H.7b shows that the uniform averaging rule performs better than the Metropolis rule, although the gap becomes small in the heavily quantized (4-bit) regime. For the tested directed Erdös–Rényi graph, Fig. H.7c shows that uniform averaging performs better than the Metropolis rule under the matched full-precision reference and all three finite-bit settings. Figure H.8 relates these performance differences to the spectral properties of the corresponding combination matrices. In our experiments, across both construction rules, the second-largest eigenvalue magnitude σA _A is largest for the ring graph, intermediate for the grid, and smallest for the directed Erdös–Rényi graph. Since σA _A governs the speed of linear mixing associated with the local decision statistics in (), this ordering helps explain the convergence behaviors observed in Figs. H.8a and H.8b: the ring trajectories approach their limiting regime more slowly than those of the grid or Erdös–Rényi graphs. This spectral comparison concerns the transient mixing behavior. The classification performance also depends on the Perron vector of A and the distribution of heterogeneous local scores. The precision-specific views in Fig. H.9 further clarify the role of quantization. Since the ring and grid graphs are undirected, the Metropolis rule yields doubly-stochastic matrices for both graph families, and hence the same uniform Perron vector. Therefore, in the matched bounded full-precision setting, their decision statistics converge to the same limit. This is reflected in Fig. H.9a, where the corresponding curves approach nearly the same long-run error level. However, the ring graph exhibits a slower transient due to its larger σA _A. As the communication precision decreases, this coincidence is no longer exact. The finite-bit results exhibit a gradual precision hierarchy: the 8-bit trajectories remain close to full precision, 6-bit communication produces an intermediate degradation, and 4-bit communication produces the largest loss. Finally, although the ring graph converges more slowly in the early rounds, it attains a lower long-run error than the directed Erdös–Rényi graph in these experiments. In particular, the two graphs have the same number of communication links, and hence the same per-round communication cost at a fixed bit-width following (H.17). This comparison further illustrates the importance of graph topology in collaborative inference. (a) Ring topology (b) Grid topology (c) Directed Erdös–Rényi topology Fig. H.7: Effect of the construction rule of A and quantization on classification performance under different graph topologies. (a) Uniform averaging rule (b) Metropolis rule Fig. H.8: Effect of graph topology and quantization on classification performance under different construction rules of A. (a) Full-precision communication (b) 8-bit communication (c) 6-bit communication (d) 4-bit communication Fig. H.9: Effect of graph topology and the construction rule of A under different communication precisions. H.3.2 Adaptive Stopping This subsection provides additional details for the adaptive stopping study in Section . Our implementation follows a local event-triggered communication protocol. For the i-th testing sample and each agent k, let xi,ksent(t)x_i,k^sent(t) denote the last value that agent k has transmitted up to round t. At the next round, agent k computes a candidate statistic given by xi,knew(t)=∑ℓ∈kaℓkxi,ℓsent(t−1).x_i,k^new(t)= _ _ka_ k\,x_i, ^sent(t-1). (H.18) At initialization, we set xi,ksent(0)=xi,k(0)x_i,k^sent(0)=x_i,k(0), where xi,k(0)x_i,k(0) is the local decision statistic before communication. The stopping decision is then made from xi,knew(t)x_i,k^new(t) together with the agent’s local memory. If the trigger fires, the stored communicated value is updated; otherwise the previous transmitted value is kept. Therefore, an agent that remains silent at one round still computes a fresh candidate statistic and may become active again later if the trigger condition is violated. We investigate two local stopping-rule families. The first is a Δ -trigger rule, which is based directly on the change in the communicated statistic. Under this rule, agent k transmits the candidate statistic xi,knew(t)x_i,k^new(t) to its neighbors at round t only if |xi,knew(t)−xi,ksent(t−1)|>εsend |x_i,k^new(t)-x_i,k^sent(t-1) |> _send (H.19) where εsend>0 _send>0 denotes the threshold parameter. The second is a label-stability with confidence rule. Let yi,knew(t)=sign⁡(xi,knew(t))y_i,k^new(t)=sign (x_i,k^new(t) ) (H.20) be the label induced by the candidate statistic, and let si,k(t)s_i,k(t) denote the number of consecutive rounds up to round t over which this induced label has remained unchanged. In the implementation, the counter is updated by comparing the induced labels at two consecutive rounds. Thus, L=1L=1 corresponds to one unchanged transition, L=2L=2 to two consecutive unchanged transitions, and so on. Agent k does not transmit sample i at round t when si,k(t)≥L,|xi,knew(t)|≥τconfs_i,k(t)≥ L, |x_i,k^new(t)|≥ _conf (H.21) where L is a patience parameter and τconf _conf is a confidence threshold. To evaluate the performance of the two stopping rules, we sweep εsend _send over a dense grid for the Δ -trigger rule, and sweep (L,τconf)(L, _conf) over dense grids for the label-stability rule. Since transmissions may stop at different times for different agents and different samples, the communication cost for each rule is measured by the total number of transmissions per testing sample, obtained by counting the realized transmissions over directed non-self links across the communication rounds. The Pareto-efficient operating points shown in Fig. are then extracted in the plane of probability of error versus the total number of transmissions. H.3.3 Robustness This subsection provides additional details for the robustness study in Section . For the i-th testing sample, let xi,k(0)x_i,k(0) denote the local decision statistic of agent k before communication. The collaborative recursion then evolves according to xi,k(t)=∑ℓ∈kaℓkxi,ℓ(t−1).x_i,k(t)= _ _ka_ kx_i, (t-1). (H.22) For each perturbation scenario, we replace the initial statistics xi,k(0)\x_i,k(0)\ by corrupted values xi,kcorr(0)\x_i,k corr(0)\ and compare the resulting performance with the corresponding clean trajectory. In each repetition, m∈1,…,Km∈\1,…,K\ agents are corrupted, chosen uniformly without replacement, and kept fixed throughout that repetition. We consider three perturbation families. Benign noisy agents For each corrupted agent k, independent Gaussian perturbations are injected into the initial statistics: xi,kcorr(0)=xi,k(0)+αnoiseζi,k,ζi,k∼⁡(0,1).x_i,k corr(0)=x_i,k(0)+ _ noise _i,k, _i,k (0,1). (H.23) To normalize the perturbation level αnoise _ noise, we first compute the sample standard deviation of xi,k(0)i=1Ntest\x_i,k(0)\_i=1^N_ test for each agent k, and then take the median of these values across agents. Denoting the resulting scale estimate by s s, we set αnoise=ηnoises^, _ noise= _ noise s, (H.24) with a dimensionless multiplier ηnoise _ noise. Faulty agents We consider both stuck-at-zero failures and constant-bias failures. In the stuck-at-zero model, xi,kcorr(0)=0,x_i,k corr(0)=0, (H.25) whereas in the constant-bias model, xi,kcorr(0)=xi,k(0)+bfault,bfault=ηbiass^.x_i,k corr(0)=x_i,k(0)+b_ fault, b_ fault= _ bias s. (H.26) Here, ηbias _ bias is the dimensionless bias multiplier varied in the experiments. Adversarial reports For each corrupted agent k, we apply a scaled sign flip, xi,kcorr(0)=−ηadvxi,k(0),ηadv≥0.x_i,k corr(0)=- _ advx_i,k(0), _ adv≥ 0. (H.27) This model includes several special cases: ηadv=0 _ adv=0 corresponds to silencing, ηadv=1 _ adv=1 to a pure sign flip, and ηadv>1 _ adv>1 to an amplified wrong-sign report. References [1] A. H. Sayed, S. Tu, J. Chen, X. Zhao, and Z. J. Towfic (2013) Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior. IEEE Signal Processing Magazine 30 (3), p. 155–171. Cited by: §I-B2, §I, §I. [2] S. Kar and J. M. F. Moura (2013) Consensus + innovations distributed inference over networks: cooperation and sensing in networked systems. IEEE Signal Processing Magazine 30 (3), p. 99–109. Cited by: §I-B2, §I, §I. [3] M. I. Jordan, J. D. Lee, and Y. Yang (2019) Communication-efficient distributed statistical inference. Journal of the American Statistical Association 114 (526), p. 668–681. Cited by: §I-B2, §I, §I. [4] J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and J. S. Rellermeyer (2020) A survey on distributed machine learning. ACM Computing Surveys 53 (2), p. 1–33. Cited by: §I. [5] D. Jin, N. Kannengießer, S. Rank, and A. Sunyaev (2024) Collaborative distributed machine learning. ACM Computing Surveys 57 (4), p. 95:1–95:36. External Links: Document Cited by: §I. [6] V. Matta, P. Braca, S. Marano, and A. H. Sayed (2016) Diffusion-based adaptive distributed detection: Steady-state performance in the slow adaptation regime. IEEE Transactions on Information Theory 62 (8), p. 4710–4732. Cited by: §I. [7] D. Bajović, J. M. F. Moura, J. Xavier, and B. Sinopoli (2016) Distributed inference over directed networks: Performance limits and optimal design. IEEE Transactions on Signal Processing 64 (13), p. 3308–3323. Cited by: §I. [8] S. Shahrampour, A. Rakhlin, and A. Jadbabaie (2016) Distributed detection: Finite-time analysis and impact of network topology. IEEE Transactions on Automatic Control 61 (11), p. 3256–3268. Cited by: §I. [9] V. Matta, V. Bordignon, and A. H. Sayed (2025) Social learning: opinion formation and decision-making over graphs. NOW Book Series on Information and Learning Sciences, Now Publishers. Cited by: §I-B2, §I. [10] A. H. Sayed (2014) Adaptation, learning, and optimization over networks. Foundations and Trends® in Machine Learning 7 (4-5), p. 311–801. Cited by: §I, §I-B2, Assumption 3, §H.3.1. [11] A. Nedic (2020) Distributed gradient methods for convex machine learning problems in networks: Distributed optimization. IEEE Signal Processing Magazine 37 (3), p. 92–101. Cited by: §I. [12] P. Braca, L. M. Millefiori, A. Aubry, S. Marano, A. De Maio, and P. Willett (2022) Statistical hypothesis testing based on machine learning: Large deviations analysis. IEEE Open Journal of Signal Processing 3, p. 464–495. Cited by: §I. [13] V. Bordignon, S. Vlaski, V. Matta, and A. H. Sayed (2023) Learning from heterogeneous data based on social interactions over graphs. IEEE Transactions on Information Theory 69 (5), p. 3347–3371. External Links: Document Cited by: §I-B2, §I, §I-A, §I-A, §A.3, §A.3, §A.4, §F.1. [14] P. Hu, V. Bordignon, M. Kayaalp, and A. H. Sayed (2025) Non-asymptotic performance of social machine learning under limited data. Signal Processing 230, p. 109849. Cited by: §I-B2, §I, §I-A, §I-A, §I-A, §A.1, §A.1, §A.1, §A.3, §A.3, §A.3, §A.3, §A.3, §F.1, §H.1.1. [15] S. F. Yilmaz, B. Hasircioğlu, L. Qiao, and D. Gündüz (2025) Private collaborative edge inference via over-the-air computation. IEEE Transactions on Machine Learning in Communications and Networking 3, p. 215–231. Cited by: §I-A. [16] P. Yadav, C. Raffel, M. Muqeeth, L. Caccia, H. Liu, T. Chen, M. Bansal, L. Choshen, and A. Sordoni (2025) A survey on model MoErging: recycling and routing among specialized experts for collaborative learning. Transactions on Machine Learning Research. Cited by: §I-A, §I-B3. [17] C. Mavromatis, P. Karypis, and G. Karypis (2024) Pack of LLMs: Model fusion at test-time via perplexity optimization. In Conference on Language Modeling, Cited by: §I-A. [18] M. H. DeGroot (1974) Reaching a consensus. Journal of the American Statistical Association 69 (345), p. 118–121. Cited by: §I-A, §I-B, §I-A. [19] R. Olfati-Saber, J. A. Fax, and R. M. Murray (2007) Consensus and cooperation in networked multi-agent systems. Proceedings of the IEEE 95 (1), p. 215–233. Cited by: §I-A. [20] D. Acemoglu and A. Ozdaglar (2011) Opinion dynamics and learning in social networks. Dynamic Games and Applications 1 (1), p. 3–49. Cited by: §I-A. [21] A. G. Chandrasekhar, H. Larreguy, and J. P. Xandri (2020) Testing models of social learning on networks: Evidence from two experiments. Econometrica 88 (1), p. 1–32. Cited by: §I-A. [22] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics, p. 1273–1282. Cited by: §I-B1. [23] J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon (2016) Federated learning: Strategies for improving communication efficiency. arXiv:1610.05492. Cited by: §I-B1. [24] X. Lian, C. Zhang, H. Zhang, C. Hsieh, W. Zhang, and J. Liu (2017) Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §I-B1. [25] Z. Jiang, A. Balu, C. Hegde, and S. Sarkar (2017) Collaborative deep learning in fixed topology networks. In Advances in Neural Information Processing Systems, Vol. 30. Cited by: §I-B1. [26] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra (2018) Federated learning with non-iid data. arXiv:1806.00582. Cited by: §I-B1. [27] T. Lin, L. Kong, S. U. Stich, and M. Jaggi (2020) Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems, Vol. 33, p. 2351–2363. Cited by: §I-B1. [28] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al. (2021) Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14 (1–2), p. 1–210. Cited by: §I-B1. [29] A. Fallah, A. Mokhtari, and A. Ozdaglar (2020) Personalized federated learning with theoretical guarantees: A model-agnostic meta-learning approach. In Advances in Neural Information Processing Systems, Vol. 33, p. 3557–3568. Cited by: §I-B1. [30] D. Wen, K. Jeon, and K. Huang (2022) Federated dropout–A simple approach for enabling federated learning on resource constrained devices. IEEE wireless communications letters 11 (5), p. 923–927. Cited by: §I-B1. [31] S. Xie, D. Wen, X. Liu, C. You, T. Ratnarajah, and K. Huang (2024) Convergence analysis for federated dropout. In GLOBECOM 2024 - 2024 IEEE Global Communications Conference, p. 5313–5318. Cited by: §I-B1. [32] C. Mendler-Dünner, W. Guo, S. Bates, and M. I. Jordan (2021) Test-time collective prediction. In Advances in Neural Information Processing Systems, Vol. 34, p. 13719–13731. Cited by: §I-B1. [33] O. A. Alzubi, J. A. A. Alzubi, S. Tedmori, H. Rashaideh, and O. Almomani (2018) Consensus-based combining method for classifier ensembles. The International Arab Journal of Information Technology 15 (1), p. 76–86. Cited by: §I-B1. [34] A. Jadbabaie, P. Molavi, A. Sandroni, and A. Tahbaz-Salehi (2012) Non-Bayesian social learning. Games and Economic Behavior 76 (1), p. 210–225. Cited by: §I-B2. [35] A. Lalitha, T. Javidi, and A. D. Sarwate (2018) Social learning and distributed hypothesis testing. IEEE Transactions on Information Theory 64 (9), p. 6161–6179. Cited by: §I-B2, §F.1. [36] A. Nedic, A. Olshevsky, and C. A. Uribe (2017) Fast convergence rates for distributed non-Bayesian learning. IEEE Transactions on Automatic Control 62 (11), p. 5538–5553. Cited by: §I-B2. [37] V. Bordignon, V. Matta, and A. H. Sayed (2021) Adaptive social learning. IEEE Transactions on Information Theory 67 (9), p. 6053–6081. Cited by: §I-B2. [38] M. Kayaalp, Y. İnan, E. Telatar, and A. H. Sayed (2024) On the arithmetic and geometric fusion of beliefs for distributed inference. IEEE Transactions on Automatic Control 69 (4), p. 2265–2280. Cited by: §I-B2. [39] R. Polikar (2006) Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 6 (3), p. 21–45. Cited by: §I-B3, §I-B2. [40] L. I. Kuncheva (2014) Combining pattern classifiers: methods and algorithms. John Wiley & Sons. Cited by: §I-B3, §I-B2. [41] P. L. Bartlett, Y. Freund, W. S. Lee, and R. E. Schapire (1998) Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics 26 (5), p. 1651–1686. Cited by: §I-B3. [42] Z. Zhou (2012) Ensemble methods: foundations and algorithms. 1 edition, Chapman and Hall/CRC. Cited by: §I-B3. [43] P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe (2006) Convexity, classification, and risk bounds. Journal of the American Statistical Association 101 (473), p. 138–156. Cited by: §I-A. [44] M. Mohri, A. Rostamizadeh, and A. Talwalkar (2018) Foundations of machine learning. 2 edition, MIT press. Cited by: §I-A, §I-B1, §I-E, §I-E, Appendix D, Appendix E, Appendix E. [45] A. H. Sayed (2022) Inference and learning from data. Cambridge University Press. Cited by: §I-B, §IV-A1, §H.1.3. [46] M. Newman (2018) Networks. Oxford University Press. Cited by: §I-B, §IV-A. [47] T. G. Lewis (2011) Network science: theory and applications. John Wiley & Sons. Cited by: §I-B. [48] L. I. Kuncheva (2002) A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2), p. 281–286. Cited by: §I-A, §I-B1. [49] P. Caputo, G. Menz, and P. Tetali (2015) Approximate tensorization of entropy at high temperature. Annales de la Faculté des Sciences de Toulouse: Mathématiques 24 (4), p. 691–716. External Links: Document Cited by: §I-B, §B.1. [50] G. Fumera and F. Roli (2005) A theoretical and experimental analysis of linear combiners for multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (6), p. 942–956. Cited by: §I-B2. [51] T. Heskes (1997) Selecting weighting factors in logarithmic opinion pools. In Advances in Neural Information Processing Systems, Vol. 10. Cited by: §I-B2. [52] R. A. Horn and C. R. Johnson (2012) Matrix analysis. 2 edition, Cambridge University Press. Cited by: §I-C. [53] L. Xiao and S. Boyd (2004) Fast linear iterations for distributed averaging. Systems & Control Letters 53 (1), p. 65–78. Cited by: §I-C. [54] T. C. Aysal, M. J. Coates, and M. G. Rabbat (2007) Distributed average consensus using probabilistic quantization. In 2007 IEEE/SP 14th Workshop on Statistical Signal Processing, Madison, WI, USA, p. 640–644. Cited by: §I-D. [55] G. Fumera, F. Roli, and A. Serrau (2008) A theoretical analysis of bagging as a linear combination of classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (7), p. 1293–1299. Cited by: §I-E. [56] A. Krizhevsky (2009) Learning multiple layers of features from tiny images. Technical report University of Toronto. Cited by: §IV-A. [57] Y. Hu, D. Niu, J. Yang, and S. Zhou (2019) FDML: a collaborative machine learning framework for distributed features. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, p. 2232–2240. External Links: Document Cited by: §IV-A, §H.1.2. [58] A. Kraskov, H. Stögbauer, and P. Grassberger (2004) Estimating mutual information. Physical Review E 69 (6), p. 066138. Cited by: §IV-A1, §H.1.3. [59] K. He, X. Zhang, S. Ren, and J. Sun (2016) Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 770–778. Cited by: §IV-A2, item 3. [60] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao (2015) 3D shapenets: A deep representation for volumetric shapes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 1912–1920. Cited by: §IV-B. [61] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger (2017) On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70, p. 1321–1330. Cited by: §IV-B1. [62] P. L. Bartlett and S. Mendelson (2002) Rademacher and Gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research 3 (Nov), p. 463–482. Cited by: §A.4, Appendix E. [63] B. Neyshabur, R. Tomioka, and N. Srebro (2015) Norm-based capacity control in neural networks. In Conference on Learning Theory, p. 1376–1401. Cited by: §A.4. [64] S. Boucheron, G. Lugosi, and P. Massart (2003) Concentration inequalities using the entropy method. The Annals of Probability 31 (3), p. 1583–1614. External Links: Document Cited by: §B.1. [65] C. McDiarmid (1989) On the method of bounded differences. In Surveys in Combinatorics, 1989, J. Siemons (Ed.), London Mathematical Society Lecture Note Series, Vol. 141, p. 148–188. External Links: Document Cited by: Appendix E.

Instant research discovery