Paper deep dive
A Secure and Private Distributed Bayesian Federated Learning Design
Nuocheng Yang, Sihua Wang, Zhaohui Yang, Mingzhe Chen, Changchuan Yin, Kaibin Huang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/20/2026, 3:24:02 PM
Summary
The paper proposes a novel Distributed Federated Learning (DFL) framework that integrates Byzantine robustness, privacy preservation, and convergence acceleration. It utilizes a Bayesian approach for local model training and employs a Graph Neural Network (GNN)-based Reinforcement Learning (RL) algorithm to autonomously select optimal neighbor subsets for posterior exchange, thereby mitigating privacy leakage from honest-but-curious neighbors and attacks from Byzantine adversaries while ensuring fast convergence.
Entities (8)
Relation Signals (7)
Proposed Framework → integrates → Privacy Preservation
confidence 95% · we propose a novel DFL framework that integrates Byzantine robustness, privacy preservation, and convergence acceleration.
Proposed Framework → integrates → Byzantine Robustness
confidence 95% · we propose a novel DFL framework that integrates Byzantine robustness, privacy preservation, and convergence acceleration.
Neighbor Selection → issolvedby → Graph Neural Network
confidence 92% · we develop a fully distributed Graph Neural Network (GNN)-based Reinforcement Learning (RL) algorithm. This approach enables devices to make autonomous connection decisions
Neighbor Selection → issolvedby → Reinforcement Learning
confidence 92% · we develop a fully distributed Graph Neural Network (GNN)-based Reinforcement Learning (RL) algorithm.
Distributed Federated Learning → suffersfrom → privacy leakage
confidence 90% · DFL faces three critical challenges: privacy leakage from honest-but-curious neighbors
Distributed Federated Learning → suffersfrom → Byzantine adversaries
confidence 90% · vulnerability to Byzantine adversaries aiming to degrade model accuracy
Local Model Training → uses → Bayesian Approach
confidence 88% · each device trains a local model using a Bayesian approach
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Distributed Federated Learning (DFL) enables decentralized model training across large-scale systems without a central parameter server. However, DFL faces three critical challenges: privacy leakage from honest-but-curious neighbors, slow convergence due to the lack of central coordination, and vulnerability to Byzantine adversaries aiming to degrade model accuracy. To address these issues, we propose a novel DFL framework that integrates Byzantine robustness, privacy preservation, and convergence acceleration. Within this framework, each device trains a local model using a Bayesian approach and independently selects an optimal subset of neighbors for posterior exchange. We formulate this neighbor selection as an optimization problem to minimize the global loss function under security and privacy constraints. Solving this problem is challenging because devices only possess partial network information, and the complex coupling between topology, security, and convergence remains unclear. To bridge this gap, we first analytically characterize the trade-offs between dynamic connectivity, Byzantine detection, privacy levels, and convergence speed. Leveraging these insights, we develop a fully distributed Graph Neural Network (GNN)-based Reinforcement Learning (RL) algorithm. This approach enables devices to make autonomous connection decisions based on local observations. Simulation results demonstrate that our method achieves superior robustness and efficiency with significantly lower overhead compared to traditional security and privacy schemes.
Tags
Links
- Source: https://arxiv.org/abs/2602.20003v1
- Canonical: https://arxiv.org/abs/2602.20003v1
Trouble viewing inline? Open PDF directly →
Full Text
93,548 characters extracted from source content.
Expand or collapse full text
A Secure and Private Distributed Bayesian Federated Learning Design Nuocheng Yang, Student Member, IEEE, Sihua Wang, Zhaohui Yang, Mingzhe Chen Member, IEEE, Changchuan Yin, Senior Member, IEEE, and Kaibin Huang, Fellow, IEEE N. Yang, S. Wang, and C. Yin are with the Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China (e-mails: yangnuocheng,sihuawang,ccyin@bupt.edu.cn).Z. Yang is with the College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou 310027, China, and Zhejiang Provincial Key Lab of Information Processing, Communication and Networking (IPCAN), Hangzhou 310007, China (e-mail: yang_zhaohui@zju.edu.cn).M. Chen is with the Department of Electrical and Computer Engineering and Institute for Data Science and Computing, University of Miami, Coral Gables, FL, 33146 USA (email: mingzhe.chen@miami.edu).K. Huang is with the Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong (e-mail: huangkb@hku.hk). Abstract Distributed Federated Learning (DFL) enables decentralized model training across large-scale systems without a central parameter server. However, DFL faces three critical challenges: privacy leakage from honest-but-curious neighbors, slow convergence due to the lack of central coordination, and vulnerability to Byzantine adversaries aiming to degrade model accuracy. To address these issues, we propose a novel DFL framework that integrates Byzantine robustness, privacy preservation, and convergence acceleration. Within this framework, each device trains a local model using a Bayesian approach and independently selects an optimal subset of neighbors for posterior exchange. We formulate this neighbor selection as an optimization problem to minimize the global loss function under security and privacy constraints. Solving this problem is challenging because devices only possess partial network information, and the complex coupling between topology, security, and convergence remains unclear. To bridge this gap, we first analytically characterize the trade-offs between dynamic connectivity, Byzantine detection, privacy levels, and convergence speed. Leveraging these insights, we develop a fully distributed Graph Neural Network (GNN)-based Reinforcement Learning (RL) algorithm. This approach enables devices to make autonomous connection decisions based on local observations. Simulation results demonstrate that our method achieves superior robustness and efficiency with significantly lower overhead compared to traditional security and privacy schemes. Index Terms: Distributed federated learning, data privacy, Byzantine robust, graph neural network, reinforcement learning. I Introduction Federated learning (FL) allows wireless devices to cooperatively train a machine learning (ML) model without local data transmission, which ensures data locally and reduces communication overhead [1, 2, 3, 4]. Existing FL frameworks can generally be categorized into two main paradigms: centralized FL (CFL) [5] and distributed FL (DFL) [6]. In CFL, each device needs to upload its local model to a parameter server for global model aggregation, which is not always feasible due to limited resources (i.e., energy and bandwidth) in practical wireless networks [7]. On the other hand, DFL enables devices to collaboratively train an ML model by exchanging local FL models with a subset of neighbors through spontaneous device-to-device (D2D) connections, making it suitable for deployment in large-scale systems. Moreover, frequent model transmission through D2D connections is also vulnerable to security issues [8, 9, 10], caused by poisoned model updates from Byzantine adversaries, and privacy issues [11, 12, 13], caused by private data reconstruction. Recently, a number of existing works such as [14, 15, 16, 17, 18, 19, 20, 21, 22, 23] have studied the security and privacy issues in CFL framework. For security issues, the authors in [14, 15, 17, 16] detected Byzantine adversaries by comparing Euclidean distance and cosine similarity between poisoned and normal model updates. Moreover, for privacy issues, the authors in [18, 19, 20, 21, 22, 23] utilized differential privacy, secure multiparty computation, and homomorphic encryption to ensure that the model updates are invisible to data reconstruction adversaries. However, jointly considering security and privacy issues is challenging since security-ensuring schemes need accurate model updates for comparison, while privacy-preserving schemes need invisible model updates for confidentiality. To tackle this problem, the authors in [24, 25] utilized multiple servers to collect different parts of accurate model updates from all devices while no single server can obtain complete model updates, thus ensuring that model updates are invisible. However, the methods in [24, 25] relied on additional centralized servers which are unsuitable for the DFL framework. In addition, the authors in [26, 27] utilized the clustering method to distinguish Byzantine adversaries from normal devices by comparing the distance between their encrypted model updates. Obviously, only the devices with independent and identically distributed (IID) training datasets can generate the same encrypted model update, which is unrealizable in non-IID scenarios. Some related works [28, 29] jointly considered the security and privacy issues in the DFL framework. The authors in [28] proposed a hierarchical DFL framework that assigns the devices to different clusters where the devices in each cluster need to transmit its model updates encrypted by CheonKim-Kim-Song (CKKS) fully homomorphic encryption technique to the same device (referred as parameter server) for intra-cluster model aggregation, thus ensuring privacy preserving. Then, the parameter servers exchange aggregated model updates after decryption with neighboring parameter servers for security inter-cluster model aggregation, and the model updates with poor validation performance would be removed. However, the parameter server in each cluster suffers from heavy computational and communication overhead, thus causing unfairness and instability in the DFL framework. Moreover, the parameter server cannot detect the poisoned updates as the model updates are encrypted during the intra-cluster model aggregation, which threatens security. In addition, DFL would collapse if poisoned updates accounted for the majority as in intra-cluster the models are average aggregated. Different from [28] where the devices transmit their encrypted model updates to the parameter server within the cluster, the devices in [29] exchange the encrypted model updates with all available neighbors and share security proof generated by zero-knowledge succinct non-interactive argument of knowledge (zk-SNARK) method to validate the security of the exchanged model updates. However, generating security proof leads to substantial computational and storage overheads [30]. Moreover, these prior works [28, 29] suffer from heavy cryptographic computation overhead while ignoring the acceleration of DFL learning and the device in DFL can only obtain partial information (e.g., device connection and model updates) from its neighbors. To tackle encryption burden, privacy-security conflict, and slow convergence rate, we propose a novel lightweight DFL framework to achieve Byzantine robustness, privacy preserving, and convergence acceleration. In particular, the proposed DFL framework enables each device first to train the local model using the Bayesian approach and then independently select a subset of its neighbors to exchange model updates. Compared to the traditional ML methods that treat model weights as deterministic variables, the Bayesian approach estimates the distribution of model weights by a Maximum a Posteriori (MAP) estimator based on the local prior and training dataset. In this way, the devices only need to exchange the updated model’s posterior with neighbors and integrate the received distribution into the local prior. Our key contributions are as follows: • We propose a novel DFL framework in which distributed devices independently select a subset of neighbors (i.e., device connection scheme) to exchange the updated posterior based on partial information (e.g., device connection and model updates). Particularly, to detect Byzantine adversaries, the proposed framework enables each device to compare the received models with the expected result formulated from the shared model updates in the last round. Moreover, to achieve privacy preserving, the proposed framework enables each device to hide critical information from its neighbors through careful device connection design. Compared to the traditional algorithms that detect Byzantine adversaries by comparing the differences between exchanged model updates directly or ensuring data privacy by encryption, the proposed framework offers a distinct perspective from the device connection scheme design which controls the exchange of model updates. • We formulate the selection of neighboring devices as an optimization problem whose goal is to minimize DFL training loss while accounting for privacy, security, and recourse constraints. To solve this problem, we first analyze how the device connection scheme impacts the accuracy of the local prior approximation (i.e. expected result). This, in turn, affects the private data leakage, and the detection of the Byzantine adversaries. Then, we analytically characterize the effect of the device connection scheme on the DFL convergence speed. Given these analyses, we can design the device connection based on the characteristics of the device connection matrix incorporating privacy, security, and convergence acceleration requirements. • To learn the optimal device connection based on partial information, we construct a novel distributed graph neural network (GNNs) based reinforcement learning (RL) method. By employing GNN-based RL method, each device can adapt to the dynamic dimension of its neighbors and generate an optimal device connection scheme independently. Numerical evaluation results show that our proposed DFL framework can achieve security, privacy, and convergence acceleration with lightweight and fair overhead compared to the baselines. Notations: Unless otherwise indicated, matrices are represented by bold capital letters (i.e. A), vectors are denoted by bold lowercase letters (i.e. v), and scalars are donated by plain font (i.e. d). The term jj|| w|| donates the L2-norm and jj| w| represents the L1-norm of w. I System Model and Problem Formulation Figure 1: Illustration of the considered DFL model. Consider a wireless network that consists of a set ℛR of R mobile devices which can be divided into three categories: 1) Conventional device set ℳ1M_1 with M1M_1 devices first train the local model using the local dataset and then exchange the model updates with a subset of neighbors in ℛR; 2) Honest-but-curious (HBC) device set ℳ2M_2 with M2M_2 devices not only train and exchange model updates (as the conventional devices do) but also attempt to reconstruct private data from other devices; 3) Byzantine adversary device set ℬB with B devices transmit randomized model to other devices in ℛR without local training. Let ℳ=ℳ1∪ℳ2M=M_1 _2 with M devices be the standard training device set which satisfies ℬ∩ℳ=∅B = and ℬ∪ℳ=ℛB =R. We assume that each device i in ℳM has a local dataset iD_i, containing NiN_i training data samples with N=∑i=1MNiN= _i=1^MN_i being the total number of training data samples across devices. Each training data sample n consists of an input feature vector i,n∈ℝNI×1 x_i,n ^N_ I× 1 and a corresponding label vector i,n∈ℝNO×1 y_i,n ^N_ O× 1. Then, the DFL problem can be expressed as min1,⋯,M1M∑i=1MF(i), _ w_1,·s, w_M 1MΣ _i=1^MF ( w_i ), (1) s.t.i=j,∀i,j∈ℳ, s.t.\,\, w_i= w_j,∀ i,j , (1a) where i∈ℝV×1 w_i ^V× 1 is a vector to capture the local FL model on device i. F(i)F ( w_i ) is given by F(i)=1Ni∑n=1Nif(i;ϕ(i,i,n),i,n),F ( w_i )= 1N_iΣ _n=1^N_if ( w_i;φ ( w_i, x_i,n ), y_i,n ), (2) where ϕ(i,i,n)φ ( w_i, x_i,n ) denotes the neural network function and f(i;ϕ(i,i,n),i,n)f ( w_i;φ ( w_i, x_i,n ), y_i,n ) is the loss function to measure the difference between the output of ϕ(i,i,n)φ ( w_i, x_i,n ) and the label i,n y_i,n. I-A Training Local Model with Bayesian Approach In contrast to the deterministic approaches that treat model weights as unknown deterministic variables, Bayesian learning treats model weights as uncertain distributions. Bayesian learning estimates the local model by employing the MAP estimator based on the local dataset which is expressed as argmaxiP(iji). _ w_iP ( w_i|D_i ). (3) Based on the Bayesian theorem, the posterior P(iji)P ( w_i|D_i ) is achieved by P(iji)=Pl(iji)P^(i)P(i), P ( w_i|D_i )= P_l (D_i| w_i ) P ( w_i )P (D_i ), (4) where Pl(iji)P_l (D_i| w_i ) is the data likelihood, P^(i) P ( w_i ) is the local prior, and P(i)P (D_i ) represents the evidence. For each data sample (i,n,i,n)∈i ( x_i,n, y_i,n ) _i, Pl(iji)P_l (D_i| w_i ) can be given by a parameterized Gaussian distribution: Pl(iji)=∏n=1NiPl(i,njϕ(i,i,n)). splitP_l (D_i| w_i )=Π _n=1^N_iP_l ( y_i,n|φ ( w_i, x_i,n ) ). split (5) The data likelihood related to the cross-entropy loss in classification problems can be modeled by categorical distribution: Pl(i,njϕ(i,i,n))=exp(ΓE(i,n;ϕ(i,i,n))), splitP_l ( y_i,n|φ ( w_i, x_i,n ) )\!=\! (-C_E ( y_i,n;φ ( w_i, x_i,n ) ) ), split (6) where E(⋅)C_E (· ) is the cross-entropy function. Then, we introduce the Bayesian FL process. I-A1 Local training process From a Bayesian perspective, the primary goal is to maximize the global data likelihood P(iji)P ( w_i|D_i ) given P^(i) P ( w_i ). However, directly calculating P(iji)P ( w_i|D_i ) according to (4) involves calculating the evidence P(i)P (D_i ), which contains an intractable multidimensional integral [31]. To tackle this issue, we use a tractable variational distribution q(i;i)q ( w_i; ψ_i ) to estimate P(iji)P ( w_i|D_i ), where q(i;i)∼(i;i,t,i,t)q ( w_i; ψ_i ) ( w_i; μ_i,t, σ_i,t ) with i=i,i ψ_i= \ μ_i, σ_i \ being the set of the mean and standard deviation of q(i;i)q ( w_i; ψ_i ). The optimal variational posterior q(i;i)q ( w_i; ψ_i ) can be obtained by minimizing the Kullback-Leibler (KL) divergence, which is (i,t,i,t)= ( μ_i,t, σ_i,t )= argmini[q(i;i)jP^(i)P(iji)], _ ψ_iD [q ( w_i; ψ_i )|| P ( w_i )P ( w_i|D_i ) ], (7) = = argmini[log(P(i))Γℬ(i)] _ ψ_i [ (P (D_i ) )-B ( ψ_i ) ] =(a) (a)= argmaxiℬ(i), _ ψ_iB ( ψ_i ), where (⋅)D (· ) denotes the KL divergence. (a) holds since log(P(i)) (P (D_i ) ) in (7) is independent to i ψ_i and can be regarded as a constant, and ℬ(i)=i∼ilogPl(iji)Γ(q(i;i)jP^(i)) ( ψ_i )=E_ w_i ψ_i P_l (D_i| w_i )-D (q ( w_i; ψ_i )|| P ( w_i ) ) (8) is the evidence lower bound (ELBO) [32]. The term logPl(iji) P_l (D_i| w_i ) is the cross entropy. Then, we adopt stochastic gradient descent (SGD) to update i μ_i and i σ_i, which can be expressed as i′=i+ηt∑n∈it∂ℬ(i)∂i,i′=i+ηt∑n∈it∂ℬ(i)∂i, aligned μ_i &= μ_i+ _tΣ _n _i^t ( ψ_i )∂ μ_i, σ_i = σ_i+ _tΣ _n _i^t ( ψ_i )∂ σ_i, aligned (9) where ηt _t is the learning rate, and itN_i^t is the subset of training data samples (i.e., minibatch) selected from the local dataset. I-A2 Model transmission process Then, each device i exchanges the updated local models P(iji)P ( w_i|D_i ) with a subset of its neighbors for model aggregation. We adopt an orthogonal frequency division multiple access (OFDMA) transmission scheme. Let W be the available bandwidth for transmission and t p_t is the transmit power matrix at time slot t with [t]i,j [ p_t ]_i,j being the model transmit power of device i to device j. The location of device i is captured by a vector i,t=[νi,t1,νi,t2] ν_i,t=[ _i,t^1, _i,t^2], and t=[1,t,⋯,R,t] ν_t= [ ν_1,t,·s, ν_R,t ] is the location matrix over all devices at iteration t. The data rate of device i transmitting the model updates to device j is li,jt([t]i,[t]i,j)=SW|[t]i|log2(1+[t]i,jhi,jt(t)W|[t]i|σN2), l_i,j^t( [ U_t ]_i, [ p_t ]_i,j)\!\!=\!\! S W| [ U_t ]_i| _2 (1+ [ p_t ]_i,jh_i,j^t ( ν_t ) W| [ U_t ]_i|σ^2_ N ), (10) where t∈0,1R×R U_t∈ \0,1 \^R× R is the FL transmission matrix with [t]i,j=1 [ U_t ]_i,j=1 implying that device i will exchange its local FL model with device j at iteration t, and [t]i,j=0 [ U_t ]_i,j=0, otherwise. [t]i [ U_t ]_i is the FL transmission vector of device i and j[t]ij=∑j∈ℳ[t]i,j| [ U_t ]_i|= _j [ U_t ]_i,j is the number of devices that will transmit the updated local models to device i. hi,jt(t)=ρi,j(di,jt(t))−2h_i,j^t ( ν_t )= _i,j (d_i,j^t ( ν_t ) )^-2 is the channel gain between device i and j, where ρi,j _i,j is the Rayleigh fading parameters. di,jt(t)d_i,j^t ( ν_t ) is the distance between device i and j. σN2σ^2_ N represents the variance of additive white Gaussian noise. S is the data size of the P(iji)P ( w_i|D_i ). I-A3 Model aggregation process Given P(jjj)P ( w_j|D_j ) that is received from neighbors, device i updates its prior as [33] P^(i)=exp(∑j∈ℛ[t]i,jlogP(jjj)), P ( w_i )= (Σ _j [ A_t ]_i,j P ( w_j|D_j ) ), (11) where t∈[0,1]R×R A_t∈ [0,1 ]^R× R is the model aggregation weight matrix with [t]i,j [ A_t ]_i,j denoting the aggregation weight of P(jjj)P ( w_j|D_j ) and satisfying ∑j∈ℛ[t]i,j=1 _j [ A_t ]_i,j=1. Note that [t]i,j>0 [ A_t ]_i,j>0 if and only if [t]i,j=1 [ U_t ]_i,j=1. Since the contribution of P(jjj)P ( w_j|D_j ) to P^(i) P ( w_i ) is determined by [t]i,j [ A_t ]_i,j, we can decrease the negative effects of Byzantine adversary j by decreasing [t]i,j [ A_t ]_i,j. To this end, we introduce a parameter ϱ to define the maximum tolerable negative impact, which is represented as the model aggregation weight for Byzantine adversaries and given by: Definition 1. (Security constraint) The maximum tolerable negative impact of Byzantine adversaries is defined as ϱ , and then, the security constraint is given by [t]i,j⩽ϱ,∀i∈ℳ,∀t∈,ifj∈ℬ. [ A_t]_i,j ,∀ i ,∀t ,if\ j . (12) Remark 1. Note that Definition 1 specifically restricts the negative impact of Byzantine adversaries by actively limiting their contributions once they are identified (i.e., j∈ℬj ). I-B Bayesian Based HBC Devices Definition To accurately reconstruct private data of device i, the HBC device j needs to acquire P(t−1)(iji)P (t-1 ) ( w_i|D_i ) and P(t)(iji)P (t ) ( w_i|D_i ) which are directly exchanged among devices, and P^(t)(i) P (t ) ( w_i ) which is locally accessible. Since the HBC device j cannot access P^(t)(i) P (t ) ( w_i ), it can only approximate it based on the received posterior from the neighbors shared with device i. In other words, to protect local data privacy, device i must avoid allowing any other device j to obtain P(t−1)(iji)P (t-1 ) ( w_i|D_i ), P(t)(iji)P (t ) ( w_i|D_i ), and P^(t)(i) P (t ) ( w_i ) simultaneously, which can be summarized as two conditions, as detailed below: • Condition 1: Device j can receive both P(t−1)(iji)P (t-1 ) ( w_i|D_i ) and P(t)(iji)P (t ) ( w_i|D_i ) simultaneously (i.e., [t−1]i,j⊙[t]i,j=1 [ U_t-1 ]_i,j [ U_t ]_i,j=1). • Condition 2: The device j’ approximated prior Pi,j,tIP_i,j,t^IP of device i approaches P^(t)(i) P (t ) ( w_i ). To describe Condition 1, we introduce a vector i,t(t−1,t)∈0,1R×1 v_i,t ( U_t-1, U_t )∈ \0,1 \^R× 1 to determine the subset of devices that receives P(t−1)(iji)P (t-1 ) ( w_i|D_i ) and P(t)(iji)P (t ) ( w_i|D_i ) simultaneously, which is i,t(t−1,t)=[t−1]i⊙[t]i, v_i,t ( U_t-1, U_t )= [ U_t-1 ]_i [ U_t ]_i, (13) where ⊙ is an element-wise multiplication operation, [i,t(t−1,t)]z=1 [ v_i,t ( U_t-1, U_t ) ]_z=1 implies Condition 1 is satisfied by device z, and [i,t(t−1,t)]z=0 [ v_i,t ( U_t-1, U_t ) ]_z=0, otherwise. Similarly, to describe Condition 2, we first introduce a vector i,j(t)∈0,1R×1 r_i,j ( U_t )∈ \0,1 \^R× 1 to determine the subset of devices that contribute to P^(t)(i) P (t ) ( w_i ) and also exchange the model updates with device j, which is given by i,j(t)=[t]i⊙[t]j, r_i,j ( U_t )= [ U_t ]_i [ U_t ]_j, (14) where the z-th element [i,j(t)]z=1 [ r_i,j ( U_t ) ]_z=1 implies that device z’ model can be used by device j to approximate Pi,j,tIP_i,j,t^IP, and [i,j(t)]z=0 [ r_i,j ( U_t ) ]_z=0, otherwise. Then, device j can approximate Pi,j,tIP_i,j,t^IP by Pi,j,tIP=exp(∑z∈ℛ[i,j(t−1)]zji,j(t−1)jlogP(t−1)(zjz)).P_i,j,t^IP= (Σ _z [ r_i,j ( U_t-1 ) ]_z| r_i,j ( U_t-1 )| P (t-1 ) ( w_z|D_z ) ). (15) The gap between Pi,j,tIP_i,j,t^IP and P^(t+1)(i) P (t+1 ) ( w_i ) is given by H(P^(t)(i),Pi,j,tIP) H ( P (t ) ( w_i ),P_i,j,t^IP ) (16) =∑z=1V([i,t]zΓ[i,j,tIP]z)2+([i,t]zΓ[i,j,tIP]z)2, 1.42271pt=Σ _z=1^V ( [ μ_i,t ]_z- [ μ_i,j,t^IP ]_z )^2+ ( [ σ_i,t ]_z- [ σ_i,j,t^IP ]_z )^2, where i,j,tIP μ_i,j,t^IP and i,j,tIP σ_i,j,t^IP are respectively the mean vector and standard deviation vector of Pi,j,tIP_i,j,t^IP 111According to [34], the reconstruction data error, represented by the L2 distance between the reconstructed data based on the Deep Leakage from Gradients (DLG) [11, 12, 13] method and the origin data, increases as H(P^(t)(i),Pi,j,tIP)H ( P (t ) ( w_i ),P_i,j,t^IP ) increases.. V is the dimension of model updates. Then, we introduce a parameter ψ to define the maximum tolerable H(P^(t)(i),Pi,j,tIP)H ( P (t ) ( w_i ),P_i,j,t^IP ) to protect data privacy when Condition 1 is satisfied, which can be summarized by the following definition. Definition 2. (Privacy constraint) The maximum tolerable privacy requirement is defined as ψ. Consequently, by combining the Condition 1 and Condition 2, the privacy constraint is given by H(P^(t)(i),Pi,j,tIP)≥ψ,if[i,t]j=1,∀i,j∈ℳ,H ( P (t ) ( w_i ),P_i,j,t^IP )≥ψ,if\ [ v_i,t ]_j=1,∀ i,j , (17) where i,t v_i,t is short for i,t(t−1,t) v_i,t ( U_t-1, U_t ). Remark 2. Note that Definition 2 establishes the permissible level of approximation of critical information of the local prior without compromising privacy. The specific value of ψ should be determined based on the requirements of each device. I-C Problem Formulation We formulate our optimization problem whose goal is to minimize the DFL training loss while jointly considering transmission delay, transmission power, security, and privacy constraints. The optimization problem is formulated as mint,t,t _ U_t, A_t, p_t 1M∑i=1MF(i) 1MΣ _i=1^MF ( w_i ) (18) s.t. s.t.\,\, [t]i,j⩽ϱ,∀i∈ℳ,∀j∈ℛ,∀t∈, if j∈ℬ, [ A_t]_i,j ,∀ i ,∀ j ,∀t , if j , (18a) H(P^(t)(i),Pi,j,tIP)≥ψ,∀i∈ℳ,∀j∈ℛ, H ( P (t ) ( w_i ),P_i,j,t^IP )≥ψ,∀ i ,∀ j , if [i,t]j=1,∀t∈, if [ v_i,t ]_j=1,∀t , (18b) li,jt([t]i,[t]i,j)⩽Γ,∀i,j∈ℛ,∀t∈, l_i,j^t ( [ U_t ]_i, [ p_t ]_i,j ) ,∀ i,j ,∀t , (18c) ∑j=1R[t]i,j⩽pmax,∀i∈ℛ,∀t∈, Σ _j=1^R [ p_t ]_i,j p_max,∀ i ,∀t , (18d) where (18a) is the security constraint, as shown in Definition 1. (18b) is the privacy constraint, as shown in Definition 2. (18c) is the DFL model transmission delay constraint where Γ is the maximum transmission delay per iteration. (18d) is the transmit power constraint where pmaxp_max is the maximum transmit power. The problem in (18) is challenging to solve by traditional algorithms due to the following reasons. First, each device must collect all devices’ local models so as to minimize the DFL training loss while satisfying the security, privacy, and transmission delay constraints. However, each device can only collect neighbors’ information in the DFL framework. Thus, we must collect global information and then find the optimal t,t U_t, A_t, and t p_t when using traditional iterative methods which introduce addition delay and computational overhead. Also, as the device’s location and channel information vary, each device must re-execute the iterative methods, which leads to additional overhead. Second, constraint (18a) needs to be satisfied when j∈ℬj is achieved first. Moreover, before satisfying constraint (18a), the devices need first to determine the j∈ℬj . Thus, each device i must identify the Byzantine adversaries among its neighbors based on partial information (e.g., device connection and model updates), which cannot be solved by traditional detection methods that collect and compare all local models in the network. I Security, Privacy, and Convergence Acceleration Analysis I-A Security Constraint Analysis To satisfy the security constraint in (18a), device i must first identify the Byzantine adversary j among its neighboring devices (i.e., j∈ℬj ). Note that, the key difference between Byzantine adversaries and standard training devices is that the Byzantine adversaries send random model updates without local training while the standard training devices send the model updates that are optimized according to (7) and (8). Hence, to distinguish Byzantine adversaries, device i needs to verify whether Pl(t)(jjj)P_l (t ) (D_j| w_j ) and (P(t)(jjj)jP^(t)(j))D (P (t )( w_j|D_j)|| P (t ) ( w_j ) ) in (8) are optimized. However, since Pl(t)(jjj)P_l (t ) (D_j| w_j ) is locally accessible, the only feasible approach is to verify whether (P(t)(jjj)jPj,i,tIP)D (P (t )( w_j|D_j)||P_j,i,t^IP ) is optimized. To this end, we define a threshold κ(t)κ ( U_t ) to identify Byzantine adversaries, which is Definition 3. (Identify Byzantine adversaries) The threshold of device i to identify Byzantine adversary j is defined as κ(t)κ ( U_t ), then, we introduce Byzantine identification matrix t∈0,1R×R B_t∈ \0,1 \^R× R which is given by [t]i,j=1,if(P(t)(jjj)jPj,i,tIP)≥κ(t),0,otherwise, [ B_t ]_i,j= \ aligned &1,if\ D (P (t ) ( w_j|D_j )||P_j,i,t^IP )≥κ( U_t),\\ &0,otherwise,\\ aligned . (19) where [t]i,j=1 [ B_t ]_i,j=1 implies that device i classifies device j as a Byzantine adversary, [t]i,j=0 [ B_t ]_i,j=0, otherwise. Note that, κ(t)κ ( U_t ) is affected by [t]j [ U_t ]_j and needs to be determined during the training process. Then (18a) can be rewritten as [t]i,j[t]i,j⩽ϱ,∀i∈ℳ,∀j∈ℛ,∀t∈, 1_ \ [ B_t ]_i,j \[ A_t]_i,j ,∀ i ,∀ j ,∀t , (20) where x=x 1_ \x \=x if x>0x>0, x=0 1_ \x \=0, otherwise. I-B DFL Convergence Analysis Next, we will analyze how t A_t affects the performance of DFL in (18). To find the relationship between t A_t and the DFL performance, we must first analyze the convergence rate of DFL. To this end, we first make the following assumptions, as done in [35, 33] • Assumption1~1: For each device i∈ℳi , there exists a i ψ_i such that q(i;i)=P^(i)P(iji)q ( w_i; ψ_i )= P ( w_i )P ( w_i|D_i ). In this case, as shown in (7), it is possible to derive the expected loss to zero. • Assumption2~2: For each device i∈ℳi , the gap between Pl(iji)P_l (D_i| w_i ) and P(i)P (D_i ) is bounded, we have ‖logPl(iji)P(i)‖≤L,∀i∈ℳ. \| P_l (D_i| w_i )P (D_i ) \|≤ L,∀ i . (21) These assumptions are natural where Assumption 11 assumes the learning problem is realizable for all devices, and Assumption 22 assumes an upper bound of ‖logPl(i|i)P(i)‖ \| P_l (D_i| w_i )P (D_i ) \| exists. To analyze the convergence rate of DFL, we can analyze the convergence of posterior P(t)(iji)P (t ) ( w_i|D_i ). However, since the devices’ posterior depends on a subset of its neighbors, the posterior of each device is inconsistent. Then, we introduce P⋆(t)()P (t ) ( w ) as the global prior which is the average aggregation of all standard training devices’ posterior P(t)(iji)P (t ) ( w_i|D_i ), which satisfies: logP⋆(t)()=∑j∈ℳ1MlogP(jjj). P (t ) ( w )=Σ _j 1M P ( w_j|D_j ). (22) Moreover, due to the Assumption 11, the prior P⋆(t)()P (t ) ( w ) is the optimal prior at iteration t. To this end, we can analyze the convergence rate of DFL by analyzing how t A_t affects the upper bound of the gap between P(t)(iji)P (t ) ( w_i|D_i ) and the optimal prior P⋆(t)()P (t ) ( w ), which is shown in Theorem 1. Theorem 1. Given the model aggregation matrix t A_t at each iteration, the upper bound of the gap between each local posterior P(t+1)(iji)P (t+1 ) ( w_i|D_i ) and the average posterior P⋆(t+1)()P (t+1 ) ( w ) is given by 1M∑i=1MjjlogP(t+1)(iji)Γ 1MΣ _i=1^M| P (t+1 ) ( w_i|D_i )\!- logP⋆(t+1)()j2 P (t+1 ) ( w )|^2 (23) ≤(LM∑τ=1tξτ)2, ≤ (L MΣ _τ=1^t _τ )^2, where ξτ∈[0,1] _τ∈ [0,1 ] is the second largest eigenvalue of ∏z=τtz⊤(∏z=τtz⊤)⊤Π _z=τ^t A_z (Π _z=τ^t A_z ) . Proof. See Appendix -A. ∎ From Theorem 1, we can see that the gap between P(t+1)(iji)P (t+1 ) ( w_i|D_i ) and P⋆(t+1)()P (t+1 ) ( w ) decreases as the second largest eigenvalues of ∏z=τtz⊤(∏z=τtz⊤)⊤Π _z=τ^t A_z (Π _z=τ^t A_z ) decreases. Thus, we can minimize ξτ _τ to decrease the gap between the DFL posterior P(t+1)(iji)P (t+1 ) ( w_i|D_i ) and the optimal DFL prior P⋆(t+1)()P (t+1 ) ( w ). Then, problem (18) can be rewritten as mint,t,t _ U_t, A_t, p_t ∑τ=1tξτ Σ _τ=1^t _τ (24) s.t. s.t.\,\, [t]i,j[t]i,j⩽ϱ,∀i∈ℳ,∀j∈ℛ,∀t∈, 1_ \ [ B_t ]_i,j \[ A_t]_i,j ,∀ i ,∀ j ,∀t , (24a) (18b)Γ(18d) ( eq:max1b)-( eq:max1d) where (24a) is due to the (20). To solve (24), one promising solution is to use an RL based algorithm that can adapt to the dynamic environment. However, the traditional RL algorithms that rely on fixed input dimensions may be invalid for addressing the problem with varying input dimensions caused by the varying number of neighbors in DFL. Moreover, since the security, privacy, and convergence acceleration as shown in (24) are affected by t A_t at previous iterations, devices need to jointly optimize t A_t and t U_t by considering the correlation between consecutive iterations. To tackle these issues, one promising solution is to replace traditional fixed-input neural networks in RL with the neural networks that combine GNNs [36] with recurrent neural networks (RNN) [37] so as to adapt dynamic input dimensions and jointly handling the spatial and temporal information. Hence, we propose a distributed RL algorithm that combines GNN and RNN to solve the problem (24). IV Proposed GNN Based RL Method In this section, our goal is to propose a novel GNN based RL algorithm that enables each device to determine the t U_t, t A_t, and t p_t independently based on the partial information (e.g., received posterior, approximated prior, and device connection) from neighbors to achieve security, privacy, and convergence acceleration which satisfy the transmission constraint. Since t A_t and t p_t can only be optimized after t U_t is determined, a two stages distributed approach is proposed. In the first stage, we propose a GNN based RL algorithm which enables each device i to optimize [t]i [ U_t ]_i by extracting the spatial and temporal features based on partial information. In the second stage, t p_t is determined by convex optimization, and [t]i [ A_t ]_i is determined based on the received model updates. Compared to standard RL algorithms [38], our proposed GNN based RL approach applies GNN and RNN serves as the backbone to capture dependencies in the spatial dimension including wireless channel and neighbor’ features and applies RNN to capture correlations in the time series dimension, then the RL method is used to infer the optimal action (i.e., t U_t and t A_t). Figure 2: Illustration of our proposed GNN-based algorithm. Next, we will first introduce the use of GNN-RNN based models to estimate t U_t. Then, devices exchange the model updates and allocate transmission power t p_t based on t U_t and determine t A_t based on the received model updates. Finally, we will show the RL algorithm to update the proposed GNN-RNN based models as shown in Fig. 2. IV-A GNN-RNN Based Model Design Figure 3: Exchange information and FL model with neighbors. IV-A1 Input To define the input of the GNN model at each device i, we first denote the set of k nearest neighbors of device i in iteration t by ℒ1(i,k,t)L^1 (i,k,t ), i.e. first-hop devices. Let ℒ2(i,k,t)L^2 (i,k,t ) be the set of k nearest neighbors of the devices in ℒ1(i,k,t)L^1 (i,k,t ), i.e. second-hop devices. For example, in Fig. 3, ℒ1(i,k,t)=l1,1,⋯,l1,nL^1 (i,k,t )=\l_1,1,·s,l_1,n\ while ℒ2(i,k,t)=l2,1,⋯,l2,mL^2 (i,k,t )=\l_2,1,·s,l_2,m\. Let ℒ(i,k,t)=ℒ1(i,k,t)∪ℒ2(i,k,t)∪iL (i,k,t )=L^1 (i,k,t ) ^2 (i,k,t )∪\i\ and jℒ(i,k,t)j|L (i,k,t )| are the number of devices in ℒ(i,k,t)L (i,k,t ). Then, the features of the connection between device i and device j are defined as i,j,t=[H(P^(t)(i),Pi,j,tIP),[t−1]i,j] E_i,j,t= [H ( P (t ) ( w_i ),P_i,j,t^IP ), [ A_t-1 ]_i,j ] being the vector containing H(P^(t)(i),Pi,j,tIP)H ( P (t ) ( w_i ),P_i,j,t^IP ) and [t−1]i,j [ A_t-1 ]_i,j. Then, the edge features i,j,t E_i,j,t need to be normalized which is given by ~i,j,t=i,j,t∑k=0|ℒ1(i,k,t)|i,k,t. E_i,j,t= E_i,j,t _k=0^|L^1 (i,k,t )| E_i,k,t. (25) The inputs to the GNN of each device i at iteration t are denoted by ~i,j,t E_i,j,t, t ν_t, and neighbor relationship ℒ1(i,k,t)L^1 (i,k,t ). IV-A2 Spatial features extraction The GNN hidden layers to extract spatial features consist of a) graph attention (GAT) layer I, b) fully connected (FC) layer I, c) GAT layer I, and d) FC layer IV, which are expressed as i,t=ψ(1K1∑k1=1K1∑k∈ℒ1(i,j)αi,k1k1[kt,~i,k,t]), q_i,t=ψ ( 1K_1Σ _k_1=1^K_1Σ _k ^1 (i,j ) _i,k^k_1 W^k_1 [ ν_k^t, E_i,k,t ] ), (26) i,t=ψ(K2i,t), z_i,t=ψ ( W^K_2 q_i,t ), (27) i,t′=ψ(1K3∑k3=1K3∑k∈ℒ1(i,j)αi,k3k3k,t), q _i,t=ψ ( 1K_3Σ _k_3=1^K_3Σ _k ^1 (i,j ) _i,k^k_3 W^k_3 z_k,t ), (28) i,t′=ψ(K4i,t′), z _i,t=ψ ( W^K_4 q _i,t ), (29) where i,t q_i,t and i,t′ q _i,t are respectively the output of the GAT layer I and I, i,t z_i,t and i,t′ z _i,t are respectively the output of the FC layers I and FC layers IV. K1K_1 and K3K_3 are respectively the numbers of attention heads in the first GAT layer and the second GAT layer. k1∈ℝ4×V1 W^k_1 ^4× V_1 and k3∈ℝV1×V2 W^k_3 ^V_1× V_2 are the GAT model parameters used to transform the input features into higher-level features. k2∈ℝV1×V1 W^k_2 ^V_1× V_1 and k4∈ℝV2×V2 W^k_4 ^V_2× V_2 are the two FC layers. Here, we need to note that i,t z_i,t updated by device i would be shared with devices in ℒ(i,k)L (i,k ) for computation in (27). αi,jk1 _i,j^k_1 and αi,jk3 _i,j^k_3 are the self-attention coefficients of the first and second GAT layers, and ψ(⋅)ψ (· ) is the leaky rectified linear unit function. To define the attention coefficients αi,jk1 _i,j^k_1, we need to define the FC function [39] that can compute the correlation between two neighbor devices as follows: si,j=ψ(α1×(k1[it,~i,j,t],k1[jt,~j,i,t])).s_i,j=ψ ( W^α_1× ( W^k_1 [ ν_i^t, E_i,j,t ], W^k_1 [ ν_j^t, E_j,i,t ] ) ). (30) Subsequently, we normalize the correlation across each device in ℒ1(i,k,t)L^1 (i,k,t ) as the attention coefficient using the softmax function to calculate the self-attention coefficient of device i and device j [40]: αi,jk1=esi,j∑l=1|ℒ1(i,k,t)|esi,l,∀j∈ℒ1(i,k,t). _i,j^k_1= e^s_i,j _l=1^|L^1 (i,k,t )|e^s_i,l,∀ j ^1 (i,k,t ). (31) IV-A3 Temporal features extraction The hidden RNN layers to extract temporal features between i,t′ z _i,t in the time series iterations consist of a) Update gate layer, b) Reset gate layer, c) Candidate representation layer, and d) Final update layer, which are expressed as i,t=ψ(e1i,t′+e2i,t−1), e_i,t=ψ ( W^e_1 z _i,t+ W^e_2 h_i,t-1 ), (32) i,t=ψ(r1i,t′+r2i,t−1), r_i,t=ψ ( W^r_1 z _i,t+ W^r_2 h_i,t-1 ), (33) ~i,t=δ(ai,t′+U(i,t⊙i,t−1)), h_i,t=δ ( W^a z _i,t+U ( r_i,t h_i,t-1 ) ), (34) i,t=(1Γi,t)⊙i,t−1+i,t⊙~i,t, h_i,t= (1- e_i,t ) h_i,t-1+ e_i,t h_i,t, (35) where i,t e_i,t is the update gate and i,t r_i,t being the forget gate. i,t−1 h_i,t-1 and i,t h_i,t are the extracted temporal features of iteration tΓ1t-1 and t, respectively. ~i,t h_i,t is the candidate representation which aggregated the information of iteration tΓ1t-1 and t. δ(⋅)δ (· ) is the hyperbolic tangent function. e1,e2,r1 W^e_1, W^e_2, W^r_1, r2 W^r_2, and a W^a are trainable parameters. IV-A4 Output Given the extracted features i,t′ z _i,t and i,t h_i,t, devices need to determine the t U_t first, and then exchange the local model updates based on the device connection scheme t U_t. Then, devices determine t A_t based on (P(t)(jjj)jPj,i,tIP)D (P (t ) ( w_j|D_j )||P_j,i,t^IP ). To determine t U_t, we calculate the device connection probability μi,jt _i,j^t by applying the softmax function based on i,t′ z _i,t which is given by μi,jt=ei,t′×j,t⊤′∑l=1|ℒ1(i,k,t)|ei,t′×l,t⊤′,∀j∈ℒ1(i,k,t). _i,j^t= e z _i,t× z^ _j,t _l=1^|L^1 (i,k,t )|e z _i,t× z^ _l,t,∀ j ^1 (i,k,t ). (36) Given the estimated connection probabilities it=[μi,1t,⋯,μi,|ℒ1(i,k,t)|t] μ_i^t= [ _i,1^t,·s, _i,|L^1 (i,k,t )|^t ], we employ an applying scheme that enables each device to determine device connections independently which is summarized as follows: 1. Each device i sends a connection request to its neighbor j that has the highest connection probability μi,jt∈it _i,j^t∈ μ_i^t when its remaining transmission power is sufficient to transmit its models, and receives the request from its neighbors. 2. Each device builds a connection with the neighbors that has the highest connection probability request. Steps 1)-2) are repeated until the remaining transmit power of each device i is not enough to satisfy the communication requirements. The optimal transmit power allocation of pi,j,tp_i,j,t for device i for transmitting its model updates to device j is given by [t]i,j=WσN2j[t]ijhi,jt(t)(2SWΓ|[t]i|Γ1). [ p_t ]_i,j= Wσ^2_ N| [ U_t ]_i|h_i,j^t ( ν_t ) (2 SW | [ U_t ]_i|-1 ). (37) Then, devices exchange the model updates based on [t]i [ U_t ]_i, and subsequently, (P(t)(jjj)jPj,i,tIP)D (P (t ) ( w_j|D_j )||P_j,i,t^IP ) can be calculated. Given model (P(t)(jjj)jPj,i,tIP)D (P (t ) ( w_j|D_j )||P_j,i,t^IP ), a GNN layer is employed to determine the model aggregation weight, which is given by hi,j,t=ψ(K5×[i,t,(P(t)(jjj)jPj,i,tIP)]), h_i,j,t=ψ ( W^K_5× [ h_i,t,D (P (t ) ( w_j|D_j )||P_j,i,t^IP ) ] ), (38) where K5 W^K_5 is a trainable parameter. Then, t A_t is determined by a softmax function based on hi,j,th_i,j,t, which is given by [t]i,j=e[t]i,jhi,j,t∑l=1|[t]i|e[t]i,lhi,l,t,∀j∈ℒ1(i,k,t). [ A_t ]_i,j= e 1_ \ [ U_t ]_i,j \h_i,j,t _l=1^| [ U_t ]_i|e 1_ \ [ U_t ]_i,l \h_i,l,t,∀ j ^1 (i,k,t ). (39) IV-B PPO Based Algorithm Design Then, we propose a PPO-based RL method for GNN-RNN based model update. The proposed PPO method consists of six components: a) agent, b) action, c) state, d) policy, e) critic, and f) reward, which are specified as follows: • Agent: Our agent is the mobile devices in ℛR that determines t U_t and t A_t independently. • Action: We define the action of agent i at iteration t as the i,jt∈[0,1] μ_i,j^t∈ [0,1 ] and [t]i,j∈[0,1] [ A_t ]_i,j∈ [0,1 ] that represents the device connection probability and the model aggregation weight. • State: The state observed by device i at iteration t, defined as set i,tS_i,t that consists of: 1) the neighboring devices’ location information [1,t,⋯,|ℒ(i,k,t)|,t] [ ν_1,t,·s, ν_|L (i,k,t )|,t ], 2) the historical model aggregation weight matrix [t]i [ A_t ]_i, 3) the KL divergence between the approximated prior and the local prior H(P^(t)(i),Pi,j,tIP)H ( P (t ) ( w_i ),P_i,j,t^IP ), 4) the distance between the approximated prior and the received posterior (P(t)(jjj)jPj,i,tIP)D (P (t ) ( w_j|D_j )||P_j,i,t^IP ) if [t]i,j=1 [ U_t ]_i,j=1. • Policy: The policy is the probability of the agent i for choosing μi,jt∈[0,1] _i,j^t∈ [0,1 ] and [t]i,j∈[0,1] [ A_t ]_i,j∈ [0,1 ] given the state i,tS_i,t. The PPO algorithm uses a GNN-RNN based model parameterized by t θ_t to build the relationship between the input state i,tS_i,t and the output policy that can achieve the minimum ξτ _τ and satisfy the constraints in (24), which is also called the actor. Then, the policy can be expressed as t(i,t,it,[t]i)=P(it,[t]iji,t) π_ θ_t (S_i,t, μ_i^t, [ A_t ]_i )=P ( μ_i^t, [ A_t ]_i|S_i,t ). • Critic: The critic t′(i,t) V_ θ_t^ (S_i,t ) in the proposed method is a function to estimate the value-function of the current policy for a given state i,tS_i,t, which is the GNN-RNN based model which parameterized by t′ θ_t^ . • Reward: The reward of choosing action it,t μ_i^t, A_t based on state i,tS_i,t is given by r(it,tji,t)= r ( μ_i^t, A_t|S_i,t )= ΓξtΓ∑j=1R[t]i,j[t]i,j−ϱ - _t-Σ _j=1^R 1_ \ 1_\[ B_t]_i,j\[ A_t]_i,j- \ (40) Γ∑j=1R[i,t]j(ψ−H(P^(t)(i),Pi,j,tIP)), -Σ _j=1^R 1_ \[ v_i,t]_j (ψ-H ( P^(t)( w_i),P_i,j,t^IP ) ) \, Since the reward of the proposed method is equivalent to the objective function of the problem (24) and the constraint, the proposed PPO algorithm can solve the minimization problem (24) by maximizing the reward. IV-C PPO Algorithm for Total Reward Maximization Next, we introduce the entire procedure of training the proposed PPO algorithm for solving the problem (24). The proposed PPO algorithm used to update GNN-RNN based model is trained offline, which means that the device connection scheme and model aggregation matrix are trained using the historical data of the DFL system. In the proposed PPO algorithm, the objective function of the actor is (i,t,it,tt)=[ptA(i,t,it,t)], L (S_i,t, μ_i^t, A_t θ_t )=E [p_ θ_tA (S_i,t, μ_i^t, A_t ) ], (41) where A(i,t,it,t)=r(i,t,it,t)+γt′(i,t)Γt′(i,t), A (S_i,t, μ_i^t, A_t )=r (S_i,t, μ_i^t, A_t )+γ V_ θ_t^ (S_i,t )- V_ θ_t^ (S_i,t ), (42) and pt=t/t−1p_ θ_t= π_ θ_t/ π_ θ_t-1 is the advantage function and the probability ratio of the current policy and the old policy function, respectively. To satisfy the trust region constraint in PPO, the proposed PPO-based approach maximizes a clipping surrogate objective function, which is expressed as c(i,t,it,tjt) L^c (S_i,t, μ_i^t, A_t| θ_t ) (43) =[minpt,ζ(pt,1Γϵ,1+ϵ)A(i,t,it,t)], 11.38092pt=E\! [ \! \p_ θ_t,\!ζ (p_ θ_t,\!1\!-\!ε,\!1\!+\!ε ) \A (S_i,t, μ_i^t, A_t ) ], where ϵε is a hyper-parameter that adjusts the clipping fraction of the clipping range, and ζ(⋅)ζ (· ) is the clip function. Then, we can update the actor model t θ_t by mini-batch SGD method which can be represented as t=t−1Γ1B∑(i,t,it,t,rt,i,t+1)∇t−1c(i,t,it,tjt−1). θ_t= θ_t-1- 1B\!\!\!Σ _ (S_i,t, μ_i^t, A_t,r_t,S_i,t+1 )\!\!\! _ θ_t-1 L^c (S_i,t, μ_i^t, A_t| θ_t-1 ). (44) After conducting n iterations of policy function optimization, we utilize regression on the mean-squared error to adjust the value function ′ V_ θ based on the actual rewards, which is given by t′=t−1′Γ1B∑(i,t,it,t,rt,i,t+1)∇t−1′(t−1′(i,t)Γri,t)2. θ_t^ = θ_t-1^ \!-\! 1B\!\!Σ _ (S_i,t, μ_i^t, A_t,r_t,S_i,t+1 )\!\!\!\!\!\!\!\!\!\! _ θ_t-1^ ( V_ θ_t-1^ (S_i,t )-r_i,t )^2. (45) By iteratively running the policy updating step (44) and the state-value updating step (45), the parameters t θ_t and t′ θ_t of the policy and state-value can find the relation between the device connections and the total reward, jointly considering the DFL performance, privacy protection, and robust aggregation. The specific training process of the proposed PPO algorithm is summarized in Algorithm 1. Algorithm 1 GNN based PPO Algorithm for FL model transmission optimization 1: Initialize policy parameters t θ_t, initial state-value function parameters t′ θ_t^ . 2: for t=1,2,…,Tt=1,2,…,T do 3: Collect (i,t,it,t,rt,i,t+1) (S_i,t, μ_i^t, A_t,r_t,S_i,t+1 ) by running policy t π_ θ_t in the environment; 4: Compute advantage estimates, A(i,t,it,t)A (S_i,t, μ_i^t, A_t ) based on the current state-value function Vt′V_ θ_t^ ; 5: Update the policy by maximizing the (43); 6: Fit state-value function by regression on mean-squared error based on (45); 7: end for V Simulation Results For our simulations, we consider a network with a circular area having a radius r=1000r=1000 m, consisting M standard training devices and B Byzantine adversaries that upload random posterior [28]. The other simulation parameters are shown in Tabel I. We perform simulations using LeNet on MNIST [41], EMNIST [42] datasets, and ShuffleNet [43] on CIFAR-10 [44] dataset for the performance comparison. To simulate data distribution skew across devices, we assume that the proportion of class labels for each device i over N classes is parameterized by a vector i=[i,1,⋯,i,N]∼DirN(1) β_i= [ β_i,1,·s, β_i,N ] Dir_N (1 ), which follows a Dirichlet distribution [45] with classes prior 11. The experiments are conducted on a computer with 3.43.4GHz IntelCore i7Γ137007-13700KF processor, 6464GB of RAM, GeForce RTX 40904090 GPU running Linux. TABLE I: Simulation Parameters Parameters Values Parameters Values W 1 MHz f 3.33.3 GHz σN2σ^2_N -174 dB Γ 1010 ms For comparison, we utilize three baselines: • A hierarchical DFL method that assigns devices into 33 clusters in which a randomly selected device in each cluster acts as the parameter server to collect the model updates encrypted by the CKKS technique and aggregate with the federated average algorithm. Then, the parameter servers decrypt and exchange the aggregated model updates with each other for inter-cluster model aggregation, in which the received model with verified performance less than 50%50\% would be removed, as shown in [28] (labeled "THDFL" in plots). • A DFL method that enables devices to exchange model updates encrypted by the CKKS technique and a security proof generated by the zk-SNARK method with all available neighbors and aggregate with the federated average algorithm, as shown in [29] (labeled "PT-DFL" in plots). • A deterministic-based DFL approach that enables devices to build a random connection scheme and aggregate with the federated average algorithm (labeled "Random connection" in plots). V-A DFL Performance Analysis Figure 4: Identification accuracy vs. number of iterations. Fig. 4 shows how the average identification accuracy of all considered algorithms changes as the number of iterations varies on the MNIST dataset for R=6R=6 and B=1B=1. From Fig. 4, we can see that the proposed method achieves similar accuracy performance with the PT-DFL method. This is due to the fact that the proposed method can detect Byzantine adversaries as accurately as PT-DFL which is based on the zk-SNARK method. From Fig. 4, we can also see that the proposed method achieves up to 13%13\% and 30%30\% improvement compared to the THDFL method and random connection scheme, respectively The 13%13\% and 30%30\% gain stem from the fact that the proposed method enables each device to distinguish the Byzantine adversaries and reduce their model aggregation weight, whereas the THDFL method and the random connection method permit poisoned model updates to participate in the federated averaging aggregation process, thereby degrading identification accuracy. Fig. 4 shows that the proposed method achieves stable convergence compared with the THDFL method and the random connection method. This implies that our proposed method utilizes a dynamic device connection that minimizes ξt _t to achieve yields significant gains in convergence speed compared to the THDFL method and the random connection method, which adopt a completely random clustering and connection strategy. Figure 5: Identification accuracy vs. the number of Byzantine adversaries. Fig. 5 shows how the average DFL identification accuracy changes as the number of Byzantine adversaries varies on the MNIST dataset for R=12R=12 case. From Fig. 5, we can see that as the proportion of Byzantine adversaries increases, all considered methods have lower average identification accuracy. This is due to the fact that the number of standard training devices decreased and the destructiveness of the Byzantine adversaries increased. From Fig. 5, we can also observe that the PT-DFL and the proposed method can improve the identification accuracy by up to 35%35\% and 25%25\% compared with THDFL and random connection scheme, respectively. The 35%35\% gain stems from the fact the PT-DFL method can detect and exclude all malicious devices by zk-SNARK based method. The 25%25\% gain stems from the fact that the proposed method can detect the Byzantine adversaries based on partial information (e.g., received posterior, approximated prior, and device connection) and decrease their model aggregation weight thus ensuring Byzantine robustness. Fig. 5 also shows that the accuracy of THDFL and the random connection scheme collapses when the proportion of Byzantine adversaries reaches 50%50\%. This is due to the fact that the THDFL and the random connection scheme enable all Byzantine adversaries to participate in the model aggregation with the same aggregation weight as the standard training devices, and hence they are impacted by Byzantine adversaries directly and eventually crash. (a) R=6R=6 with B=2B=2 (b) R=6R=6 with B=3B=3. Figure 6: The proposed method defends results against Byzantine adversaries. The model aggregation weights across standard training devices are surrounded by blue boxes. Fig. 6 presents a visualization of the averaged model aggregation weight matrix t∈[0,1]6×6 A_t∈ [0,1 ]^6× 6 generated by the proposed method trained in R=6R=6 and B=2B=2 case. Fig. 6 (a) and Fig. 6 (b) are in B=2B=2 and B=3B=3 cases, respectively. From Fig. 6 we can see that the model aggregation weight corresponding to the Byzantine adversaries exhibits significant deviations compared to those of the standard training devices (surrounded by blue boxes). This is due to the fact that the proposed method can decrease the model aggregation weight of Byzantine adversaries, thus ensuring Byzantine robustness. From Fig. 6 we can also see that the model aggregation weights of each device’s local model updates increase as the proportion of Byzantine adversaries increases. This is due to the fact that the Byzantine robustness of the proposed method relies on distinguishing the model updates that do not reflect the approximated prior which is considered as an averaged aggregation of all received model updates. Consequently, an increasing proportion of Byzantine adversaries also interferes with the credibility of other standard training devices and increases the model aggregation weights of each device’s local model updates. V-B Sensitivity Analysis Figure 7: Cumulative Distribution Function (CDF) of H(P^(t)(i),Pj,i,tIP)H ( P (t ) ( w_i ),P_j,i,t^IP ) Fig. 7 assesses the gap between the local prior and the approximated prior for the considered methods without encryption with the EMNIST dataset in R=12R=12 case. The considered baselines consist of the proposed method, the traditional DFL (deterministic based) with a random connection scheme, and the Bayesian DFL with a random connection scheme. The gap considered in the traditional DFL is measured based on the gap between the local model weight and the approximated model weight by other devices. The results are shown as the cumulative distribution function (CDF) of the value of the gap obtained over training rounds. We do not consider encryption-based methods (e.g., PT-DFL and THDFL) as their model updates are exchanged in ciphertext, which requires access to decryption keys. From Fig. 7 we can see that the proposed method yields significant gains in the gap between the local model and the approximated model compared to other baselines. This is because the proposed method keeps the local prior invisible to neighbors by utilizing Bayesian training method and optimizes the gap between the true prior and the approximated prior by adjusting the device connection scheme based on the proposed GNN method. Besides, although the Bayesian based DFL with random connection method can protect the local posterior through model aggregation in a Bayesian fashion compared to the traditional DFL, it ignores the optimization of neighbor selection for model exchange, which also causes privacy leakage. From Fig. 7 we can also see that traditional DFL with random connection has about 30%30\% of complete model exposure (where the gap between the local model weight and the approximated model weight equals zero), which implies its high risks of privacy leakage. This is due to the fact that the adversaries in the traditional DFL with random connection can fully reconstruct their neighbors’ updated local models easily when they receive all model updates transmitted during the same communication round. From Fig. 7, we can demonstrate that the proposed Bayesian DFL and GNN based method can significantly enhance privacy preservation compared with the traditional methods. (a) EMNIS./Sample. (b) EMNIS./Random. (c) EMNIS./Proposed. (d) MNIST/Sample. (e) MNIST/Random. (f) MNIST/Proposed. (g) CIFAR./Sample. (h) CIFAR./Random. (i) CIFAR./Proposed. Figure 8: Defending results against DLG attacks. V-C Privacy Analysis Fig. 8 is a visualization of reconstruction results at the average H(P^(t)(i),Pi,j,tIP)H ( P (t ) ( w_i ),P_i,j,t^IP ) of the random connection scheme and the proposed method for EMNIST, MNIST, and the CIFAR-10 dataset corresponding to the CDF plot in Fig. 7. From Fig. 8, we can see that the proposed algorithm effectively prevents the leakage of private data compared to the random connection scheme. This is due to the fact that the proposed method keeps the local prior invisible and prevents other devices from accurate approximations of local prior through active device connection optimization. V-D Complexity Analysis TABLE I: Delay of each operation Index Operation Delay 1 CKSS encryption [46, 47] 11.4μ11.4\ /byte 2 CKSS addition 9.54μ9.54\ /byte 3 CKSS decryption 3.82μ3.82\ /byte 4 zk-SNARK proof generation1 77 s 5 zk-SNARK proof verify 35m35\ ms 6 Proposed method 30m30\ ms 1 Containing delay of setup and the proof generation with input depth 500500 and kernel size 55 [29, 30]. In Table I we provide a computation delay comparison associated with the CKSS homomorphic encryption algorithm employed in both the PT-DFL and THDFL methods, the zk-SNARK algorithm utilized in the PT-DFL method, and our proposed method. From Table I, we can see that the overhead of the CKSS homomorphic encryption operation increases linearly with the size of the model updates increases, and the zk-SNARK proof generation operation to validate the security is unbearable compared to other considered methods. This is due to the fact that the proposed method satisfies privacy and security requirements only by controlling the information difference (the accuracy of approximated prior) between devices without cryptographic operations. Moreover, the devices selected as parameter servers in the THDFL scheme need to perform CKKS encryption, addition and decryption, while other devices only need to perform CKSS encryption, which causes unfairness. Figure 9: Identification accuracy vs. convergence time. Fig. 9 shows how the average DFL identification accuracy changes as the convergence time varies with CIFAR-10 dataset in R=6R=6 and B=1B=1 case. In this figure, we can see that, the proposed method increase the identification accuracy by up to 5%5\% and 20%20\% compared to THDFL method and PT-DFL method, respectively. This is due to the fact that PT-DFL and THDFL suffer from a heavy computational delay introduced by encryption and proof generation operations. In contrast, the proposed method allows devices to exchange model updates without cryptographic operations, enabling more training rounds within the same time consumption. VI Conclusion In this paper, we proposed a novel DFL framework in which mobile devices collaboratively train an ML model via exchanging their model updates with an appropriate subset of neighbors (i.e., device connection scheme) to achieve Byzantine robustness, privacy preserving, and convergence acceleration. We first analytically characterized the relationship between the dynamic device connection and Byzantine adversary’s detection, privacy protection requirement, and DFL convergence acceleration. Through mathematical analysis, we found that: 1) hiding critical information from neighbors can effectively prevent private data from leakage; 2) devices can verify whether the models received from neighbors have been updated as expected to identify Byzantine adversaries; 3) the convergence speed of DFL can be modeled as the characteristics of the device connection matrix which are affected by device connection scheme design. Then, to jointly achieve these goals by using partial information (e.g., device connection and model updates) in the DFL framework, we proposed a GNN-based RL method that allows each device to independently infer the device connections and model aggregation matrix. Numerical evaluation on real-world machine learning tasks demonstrates that the proposed method yields significant gains in Byzantine robustness, privacy-preserving, and convergence acceleration with lightweight overhead compared to conventional approaches. For future work, we are going to consider a more general case that mobile devices may depart dynamically from DFL. In this case, the devices’ mobility patterns will play an important role in the problem formulation. Moreover, in this case, DFL should enable each departed device to selectively eliminate identifiable information through machine unlearning which would be very interesting and technically challenging due to the distributed characteristic of DFL. -A Proof of Theorem 1 Proof. To prove Theorem 1, we first introduce data likelihood distribution matrix Pl(t)(j)P (t )_l (D| w ), prior matrix P^(t)() P (t ) ( w ), posterior matrix P(t)(j)P (t ) ( w|D ), and evidence matrix P_D as follows: Pl(t)(j)=[Pl(1j1),⋯,Pl(MjM)], P (t )_l (D| w )= [P_l (D_1| w_1 ),·s,P_l (D_M| w_M ) ], (46) P^(t)()=[P^(t)(1),⋯,P^(t)(M)], P (t ) ( w )= [ P (t ) ( w_1 ),·s, P (t ) ( w_M ) ], P(t)(j)=[P(1j1),⋯,P(MjM)], P (t ) ( w|D )= [P ( w_1|D_1 ),·s,P ( w_M|D_M ) ], P=[P(1),⋯,P(M)]. P_D= [P (D_1 ),·s,P (D_M ) ]. The local model updates process (4) can be rewritten as P(t)(j)=Pl(t)(j)P^(t)()P(). P (t ) ( w|D )= P_l (t ) (D| w ) P (t ) ( w )P (D ). (47) The model aggregation process (11) can be rewritten as logP^(t)()=logP(t−1)(j)t−1. P (t ) ( w )= P (t-1 ) ( w|D ) A_t-1. (48) Moreover, the matrix of the average aggregation of all devices’ posterior P⋆(t)()P (t ) ( w ) is given by logP⋆(t)()=log(P(t)(j)). P (t ) ( w )= (P (t ) ( w|D ) ) J. (49) where =1MM×M J= \ 1M \^M× M. The gap between logP(t)(j) P (t ) ( w|D ) and logP⋆(t)() P (t ) ( w ) can be given by E(t)()=logP(t)(j)ΓlogP⋆(t)() E (t ) ( w )= P (t ) ( w|D )- P (t ) ( w ) (50) =log(Pl(t−1)(j)P(t−1))t−1⊤+logP^(t−1)()t−1⊤ = ( P (t-1 )_l (D| w )P (t-1 )_D ) A_t-1 + P (t-1 ) ( w ) A_t-1 Γ(log(Pl(t−1)(j)P(t−1))t−1⊤+logP^(t−1)()t−1⊤) - ( ( P (t-1 )_l (D| w )P (t-1 )_D ) A_t-1 + P (t-1 ) ( w ) A_t-1 ) J =(logP(t−1)(j)ΓlogP⋆(t−1)())t−1⊤ = ( P (t-1 ) ( w|D )- P (t-1 ) ( w ) ) A_t-1 +log(Pl(t−1)(j)P(t−1))t−1⊤(Γ) + ( P (t-1 )_l (D| w )P (t-1 )_D ) A_t-1 ( I- J ) =E(t−1)()t−1⊤+log(Pl(t−1)P(t−1))t−1⊤(Γ), =E (t-1 ) ( w ) A_t-1 + ( P (t-1 )_lP (t-1 )_D ) A_t-1 ( I- J ), where the first equality holds due to (47). Therefore, we can rewrite E(t)()E (t ) ( w ) in (50) as E(t)() E (t ) ( w ) (51) =E(t−1)()t−1⊤+log(Pl(t−1)(j)P(t−1))t−1⊤(Γ) =E (t-1 ) ( w ) A_t-1 + ( P (t-1 )_l (D| w )P (t-1 )_D ) A_t-1 ( I- J ) =E(t−2)()t−2⊤t−1⊤+log(Pl(t−1)(j)P(t−1))t−1⊤(Γ) =E (t-2 ) ( w ) A_t-2 A_t-1 \!+\! ( P (t-1 )_l (D| w )P (t-1 )_D ) A_t-1 ( I\!-\! J ) +log(Pl(t−2)(j)P(t−2))t−2⊤(Γ)t−1⊤ 11.38092pt+ ( P (t-2 )_l (D| w )P (t-2 )_D ) A_t-2 ( I- J ) A_t-1 =E(1)()∏τ=1t−1τ+∑τ=1t−1log(Pl(τ)(j)P(τ))τ⊤(Γ)∏z=τ+1t−1z⊤ =E (1 ) ( w )Π _τ=1^t-1 A_τ\!\!+\!\!Σ _τ=1^t-1 \! ( P (τ )_l (D| w )P (τ )_D ) A_τ ( I\!-\! J )\!\!\!\!Π _z=τ+1^t-1\!\! A_z =∑τ=1t−1log(Pl(τ)(j)P(τ))τ⊤(Γ)∏z=τ+1t−1z⊤. =Σ _τ=1^t-1 ( P (τ )_l (D| w )P (τ )_D ) A_τ ( I- J )Π _z=τ+1^t-1 A_z . The last equality is due to the assumption that all local models have the same initial parameters, thus, E(1)=0E (1 )=0. Next, we derive the Frobenius norm of E(t)()E (t ) ( w ) to measure the upper bound of the gap between the average posterior P⋆(t)(j)P (t ) ( w|D ) and each local posterior P(t)(j)P (t ) ( w|D ) which is given by jE(t+1)()jjF |E (t+1 ) ( w )|_ F (52) ⩽j∑τ=1tlog(Pl(τ)(j)P(τ))τ⊤(Γ)∏z=τ+1t−1z⊤jjF |Σ _τ=1^t ( P (τ )_l (D| w )P (τ )_D ) A_τ ( I- J )Π _z=τ+1^t-1 A_z |_ F ⩽∑τ=1tjjτ⊤(Γ)∏z=τ+1tz⊤jjopjlog(Pl(τ)P(τ))jjF Σ _τ=1^t| A_τ ( I- J )Π _z=τ+1^t A_z |_ op| ( P (τ )_lP (τ )_D )|_ F ⩽LM∑τ=1tjjτ⊤(Γ)∏z=τ+1tz⊤jjop. L MΣ _τ=1^t| A_τ ( I- J )Π _z=τ+1^t A_z |_ op. The second and the third inequalities are due to [Lemma 5, [48]] and Assumption 2, respectively. Then, we need to analyze the upper bound of jτ⊤(Γ)∏z=τ+1tz⊤jjop|| A_τ ( I- J )Π _z=τ+1^t A_z ||_ op according to the definition of the matrix operator norm, which is given by jτ⊤(Γ)∏z=τ+1tz⊤jjop \!\!| A_τ ( I- J )Π _z=τ+1^t A_z |_ op (53) =λmax((τ⊤(Γ)∏z=τ+1tz⊤)⊤(τ⊤(Γ)∏z=τ+1tz⊤)) =\!\! _max\!\! (\! ( A_τ ( I- J )\!\!\!Π _z=τ+1^t A_z ) \!\!\! ( A_τ ( I- J )\!\!\!Π _z=τ+1^t A_z )\! ) =△λmax((τ⊤(Γ)τ+1)(τ⊤(Γ)τ+1)⊤) =\!\! _max\!\! ( ( A_τ ( I- J ) Q_τ+1 ) ( A_τ ( I- J ) Q_τ+1 ) ) =λmax((τ⊤(Γ)τ+1)((τ+1)⊤(Γ)⊤τ)) =\!\! _max\!\! ( ( A_τ ( I- J ) Q_τ+1 ) ( ( Q_τ+1 ) ( I- J ) A_τ ) ) =λmax((Γ)τ⊤(τ+1(τ+1)⊤)τ(Γ)) =\!\! _max\!\! ( ( I- J ) A_τ ( Q_τ+1 ( Q_τ+1 ) ) A_τ ( I- J ) ) =λmax((Γ)ττ⊤(Γ))=λmax((Γ)ττ⊤), =\!\! _max\!\! ( ( I- J ) Q_τ Q_τ ( I- J ) )= _max\!\! ( ( I- J ) Q_τ Q_τ ), where τ=∏z=τtz⊤ Q_τ=Π _z=τ^t A_z with ττ⊤ Q_τ Q_τ being a real symmetric matrix since every z A_z is row random matrix. The term λmax() _max ( C ) represent the the maximum eigenvector of matrix C. Then, we assume τ=∏z=τtz⊤ Q_τ=Π _z=τ^t A_z can be decomposed into orthogonal matrix of the eigenvectors τ⊤ P_τ and diagonal matrix of eigenvalues τ _τ which is sort in descending order. Then, we have ττ⊤=τ⊤ττ Q_τ Q_τ = P_τ _τ P_τ. Thus, (53) can be rewritten by λmax((Γ)ττ⊤)=max(τΓdiag1,0,⋯,0)=△ξτ, _max ( ( I- J ) Q_τ Q_τ )\!\!=\!\! \!max\!\! (\! _τ\!\!-\!\!diag \1,0,·s\!,0 \ )\!\! =\!\! _τ, (54) where ξτ∈[0,1] _τ∈ [0,1 ] is the second largest eigenvalue of ∏z=τtz⊤(∏z=τtz⊤)⊤Π _z=τ^t A_z (Π _z=τ^t A_z ) . The first equality is due to the fact that every row of ττ⊤ J Q_τ Q_τ is identical, hence, the rank of the matrix is 11. Consequently, ττ⊤ J Q_τ Q_τ has precisely one eigenvalue equal to 11, with all other eigenvalues being 0. Then, (52) can be rewritten as 1M∑i=1MjjlogP(t+1)(iji)Γ 1MΣ _i=1^M| P (t+1 ) ( w_i|D_i )\!- logP⋆(t+1)()j2 P (t+1 ) ( w )|^2 (55) ≤(LM∑τ=1tξτ)2. ≤ (L MΣ _τ=1^t _τ )^2. This completes the proof. ∎ References [1] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Guest editorial special issue on distributed learning over wireless edge networks-part I,” IEEE J. Sel. Areas Commun., vol. 40, no. 2, p. 445–448, Jan. 2022. [2] M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Trans. Wireless Commun., vol. 20, no. 1, p. 269–283, Oct., 2021. [3] Z. Chen, W. Yi, Y. Liu, and A. Nallanathan, “Robust federated learning for unreliable and resource-limited wireless networks,” IEEE Trans. Wireless Commun., vol. 23, no. 8, p. 9793–9809, Feb. 2024. [4] M. Aristodemou, X. Liu, Y. Wang, K. G. Kyriakopoulos, S. Lambotharan, and Q. Wei, “Maximizing uncertainty for federated learning via bayesian optimization-based model poisoning,” IEEE Trans. Inf. Forensic Secur., vol. 20, Jan. 2025. [5] D. Wei, X. Xu, Y. Liu, H. V. Poor, and M. Chen, “Optimizing model splitting and device task assignment for deceptive signal-assisted private multi-hop split learning,” IEEE Journal on Selected Areas in Communications, vol. 44, p. 1512–1528, Nov. 2026. [6] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Distributed learning in wireless networks: Recent progress and future challenges,” IEEE J. Sel. Areas Commun., vol. 39, no. 12, p. 3579–3605, Dec. 2021. [7] D. Wei, X. Xu, S.Mao, and M. Chen, “Optimizing Communication and Device Clustering for Clustered Federated Learning with Differential Privacy,” ArXiv, vol. 2507.07320, Jul. 2025. [8] S. Guo, T. Zhang, H. Yu, X. Xie, L. Ma, T. Xiang, and Y. Liu, “Byzantine-resilient decentralized stochastic gradient descent,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 6, p. 4096–4106, June 2022. [9] C. Chen, J. Liu, H. Tan, X. Li, K. I-K. Wang, P. Li, K. Sakurai, and D. Dou, “Trustworthy federated learning: privacy, security, and beyond,” Knowl. Inf. Syst., Nov. 2024. [10] Y. Wan, Y. Qu, W. Ni, Wei, X. Yong, L. Gao, and E. Hossain, “Data and model poisoning backdoor attacks on wireless federated learning, and the defense mechanisms: A comprehensive survey,” IEEE Commun. Surv. Tutor, vol. 26, no. 3, p. 1861–1897, Feb. 2024. [11] L. Zhu, Z. Liu, and S. Han, “Deep leakage from gradients,” in Proc. International Conference on Neural Information Processing Systems (NeurIPS), Vancouver BC Canada, Dec. 2019. [12] H. Yin, A. Mallya, A. Vahdat, J. Kautz J. M. Alvarez, and P. Molchanov, “See through gradients: Image batch recovery via gradinversion,” in Proc. Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, Nov. 2021. [13] T. Hideaki, “Aijack: Security and privacy risk simulator for machine learning,” arXiv preprint arXiv:2312.17667, Dec. 2023. [14] C. Fung, C. J. M. Yoon, and I. Beschastnikh, “The limitations of federated learning in Sybil settings,” in Proc. International Symposium on Recent Advances in Intrusion Detection (RAID), San Sebastian, Spain, Oct. 2020. [15] B. Luo S. Awan and F. Li, “Contra: Defending against poisoning attacks in federated learning,” in Proc. European Symposium on Research in Computer Security (ESORICS), Darmstadt, Germany, Oct. 2021. [16] T.-A. Nguyen, L. T. Le, T. D. Nguyen, W. Bao, S. Seneviratne, C.S. Hong, and N. H. Tran., “Federated pca on grassmann manifold for iot anomaly detection,” IEEE ACM Trans. Netw., vol. 32, no. 5, p. 4456–4471, Oct. 2024. [17] A. R. Ghavamipour, B. Z. H. Z. Zhao, O. Ersoy, and F. Turkmen, “Privacy-preserving aggregation for decentralized learning with Byzantine-robustness,” arXiv preprint arXiv:2404.17970, Apr. 2024. [18] S. Kalra, J. Wen, J. C. Cresswell, M. Volkovs, and H. R. Tizhoosh, “Decentralized federated learning through proxy model sharing,” Nat. Commun., vol. 14, p. 2899, Mar. 2023. [19] Lvjun Chen, Di Xiao, Zhuyang Yu, and Maolan Zhang, “Secure and efficient federated learning via novel multi-party computation and compressed sensing,” Inf. Sci., vol. 667, p. 120481, July 2024. [20] M. Hao, H. Li, G. Xu, S. Liu, and H. Yang, “Towards efficient and privacy-preserving federated deep learning,” in Proc. IEEE International Conference on Communications (ICC), Shanghai, China, May 2019. [21] J. Hui, Z. Wang, Y. Shen, B. Lin, P. Sun, X. Pang, J. Liu, and K. Ren, “Shield against gradient leakage attacks: Adaptive privacy-preserving federated learning,” IEEE ACM Trans. Netw., vol. 32, no. 2, p. 1407–1422, Apr. 2024. [22] T. Nguyen and M. T. Thai, “Preserving privacy and security in federated learning,” IEEE ACM Trans. Netw., vol. 32, no. 1, p. 833–843, Feb. 2024. [23] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. S. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Trans. Inf. Forensic Secur., vol. 15, p. 3454–3469, Apr. 2020. [24] Y. Dong, X. Chen, K. Li, D. Wang, and S. Zeng, “Flod: Oblivious defender for private Byzantine-robust federated learning with dishonestmajority,” in Proc. Network and Distributed System Security (NDSS) Symposium, Berlin, Heidelberg, Oct. 2021. [25] Z. Lu, S. Lu, X. Tang Y. Cui, and J. Wu, “Split aggregation: Lightweight privacy-preserving federated learning resistant to Byzantine attacks,” IEEE Trans. Inf. Forensics Secur., vol. 19, p. 5575–5590, May 2024. [26] X. Liu, H. Li, G. Xu, Z. Chen, X. Huang, and R. Lu, “Privacy-enhanced federated learning against poisoning adversaries,” IEEE Trans. Inf. Forensics Secur., vol. 16, p. 4574–4588, Aug. 2021. [27] Y. Miao, Z. Liu, H. Li, K.-K. R. Choo, and R. H. Deng, “Privacy-preserving Byzantine-robust federated learning via blockchain systems,” IEEE Trans. Inf. Forensics Secur., vol. 17, p. 2848–2861, Aug. 2022. [28] Y. Li, X. Wang, R. Sun, X. Xie, S. Ying, and S. Ren, “Trustiness-based hierarchical decentralized federated learning,” Knowledge-Based Syst., vol. 276, p. 110763, Sept. 2023. [29] L. Wang, X. Zhao, Z. Lu, W. Lin, and S. Zhang, “Enhancing privacy preservation and trustworthiness for decentralized federated learning,” Inf. Sci., vol. 628, p. 449–468, May 2023. [30] J. Ernstberger, S. Chaliasos, G. Kadianakis, S. Steinhorst, P. Jovanovic, A. Gervais, B. Livshits, and M. Orrù, “zk-Bench: A toolset for comparative evaluation and performance benchmarking of snarks,” in Proc. Security and Cryptography for Networks (SCN), Amalfi, Italy, Sep. 2024, p. 46–72. [31] C. Xia, D. H. K. Tsang, and V. K. N. Lau, “Structured Bayesian federated learning for green AI: A decentralized model compression using Turbo-VBI-Based approach,” IEEE Internet Things J., vol. 11, no. 7, p. 12783–12798, Apr. 2024. [32] M. D. David, K. Alp, and D. M. Jon, “Variational inference: A review for statisticians,” J. Am. Stat. Assoc., vol. 112, no. 518, p. 859–877, July. 2017. [33] X. Wang, A. Lalitha, T. Javidi, and F. Koushanfar, “Peer-to-peer variational federated learning over arbitrary graphs,” IEEE J. Sel. Areas Inf. Theory, vol. 3, no. 2, p. 172–182, June 2022. [34] H. Zhang, J. Hong, Y. Deng, M. Mahdavi, and J. Zhou, “Understanding deep gradient leakage via inversion influence functions,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), New Orleans, LA, USA, Dec. 2023. [35] A. Lalitha, T. Javidi, and A. D. Sarwate, “Social learning and distributed hypothesis testing,” IEEE Trans. Inf. Theory, vol. 64, no. 9, p. 6161–6179, May 2018. [36] Z. Luo, Y. Bao, and C. Wu, “Optimizing task placement and online scheduling for distributed gnn training acceleration in heterogeneous systems,” IEEE ACM Trans. Netw., vol. 32, no. 5, p. 3715–3729, Oct. 2024. [37] P. Hu, Y. Chen, L. Pan, Z. Fang, F. Xiao, and L. Huang, “Multi-user delay-constrained scheduling with deep recurrent reinforcement learning,” IEEE ACM Trans. Netw., vol. 32, no. 3, p. 2344–2359, June 2024. [38] Y. Wang, M. Chen, T. Luo, W. Saad, D. Niyato, H. V. Poor, and S. Cui, “Performance optimization for semantic communications: An attention-based reinforcement learning approach,” IEEE J. Sel. Areas Commun., vol. 40, no. 9, p. 2598–2613, July. 2022. [39] V. Petar, C. Guillem, C. Arantxa, R. Adriana, L. Pietro, and B. Yoshua, “Graph attention networks,” in Proc. International Conference on Learning Representations (ICLR), Vancouver, BC, Canada, Apr. 2018. [40] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. International Conference on Neural Information Processing Systems (NIPS), Dec. 2017. [41] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, p. 2278–2324, Nov. 1998. [42] C. Gregory, A. Saeed, T. Jonathan, and v. S. André, “Emnist: Extending mnist to handwritten letters,” in Proc. International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, July 2017. [43] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” in Proc. European Conference on Computer Vision (ECCV), Munich, Germany, Sep. 2018. [44] A. Krizhevsky, “Learning multiple layers of features from tiny images,” [Online], Apr. 2009. [45] T.-M. H. Hsu, H. Qi, and M. Brown, “Measuring the effects of non-identical data distribution for federated visual classification,” arXiv preprint arXiv:1909.06335, 2019. [46] J. H. Cheon, A. Kim, M. Kim, and Y. Song, “Homomorphic encryption for arithmetic of approximate numbers,” in Proc. Advances in Cryptology, Hong Kong, China, Dec. 2017. [47] A. Benaissa, B. Retiat, B. Cebere, and A. E. Belfedhal, “Tenseal: A library for encrypted tensor operations using homomorphic encryption,” in Proc. ICLR Workshop on Distributed and Private Machine Learning, May 2021. [48] W. Liu, L. Chen, and W. Zhang, “Decentralized federated learning: Balancing communication and computing costs,” IEEE Trans. Signal Inf. Process. Networks, vol. 8, p. 131–143, Feb. 2022.