Paper deep dive
ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning
Chuiyang Meng, Ming Tang, Vincent W. S. Wong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/21/2026, 12:28:51 AM
Summary
The paper proposes ASFL, an Adaptive Split Federated Learning framework for wireless networks that optimizes convergence rate, delay, and energy consumption. It introduces an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to jointly solve adaptive model splitting and resource allocation (resource blocks and transmit power) under long-term constraints, achieving significant reductions in delay and energy compared to baselines like FedAvg and ACC-SFL.
Entities (8)
Relation Signals (7)
ASFL → evaluatedon → CIFAR-10
confidence 95% · We conduct experiments on CIFAR-10 and CIFAR-100 datasets
ASFL → uses → OOE-BCD
confidence 95% · We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively.
ASFL → optimizes → Convergence Rate
confidence 90% · To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL
ASFL → reduces → Energy Consumption
confidence 90% · reduces the total delay and energy consumption by up to 75% and 80%, respectively.
ASFL → reduces → Delay
confidence 90% · reduces the total delay and energy consumption by up to 75%
ASFL → usesmodel → VGG-19
confidence 90% · We conduct experiments on CIFAR-10 and CIFAR-100 datasets using VGG-19 and ResNet-50.
ASFL → comparedwith → FedAvg
confidence 85% · We compare our proposed ASFL framework with federated averaging (FedAvg)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively.
Tags
Links
- Source: https://arxiv.org/abs/2603.04437v1
- Canonical: https://arxiv.org/abs/2603.04437v1
Trouble viewing inline? Open PDF directly →
Full Text
102,150 characters extracted from source content.
Expand or collapse full text
ASFL: An Adaptive Model Splitting and Resource Allocation Framework for Split Federated Learning Chuiyang Meng1, Graduate Student Member, IEEE, Ming Tang2, Member, IEEE, and Vincent W.S. Wong1, Fellow, IEEE E-mail: chuiyangmeng, vincentw@ece.ubc.ca, tangm3@sustech.edu.cn Abstract Federated learning (FL) enables multiple clients to collaboratively train a machine learning model without sharing their raw data. However, the limited computation resources of the clients may result in a high delay and energy consumption on training. In this paper, we propose an adaptive split federated learning (ASFL) framework over wireless networks. ASFL exploits the computation resources of the central server to train part of the model and enables adaptive model splitting as well as resource allocation during training. To optimize the learning performance (i.e., convergence rate) and efficiency (i.e., delay and energy consumption) of ASFL, we theoretically analyze the convergence rate and formulate a joint learning performance and resource allocation optimization problem. Solving this problem is challenging due to the long-term delay and energy consumption constraints as well as the coupling of the model splitting and resource allocation decisions. We propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm to solve the problem iteratively. Experimental results show that when compared with five baseline schemes, our proposed ASFL framework converges faster and reduces the total delay and energy consumption by up to 75% and 80%, respectively. I Introduction The emergence of federated learning (FL) [mcmahan2017communication] enables distributed model training across clients (e.g., smartphones, sensors, and embedded systems) while preserving data privacy. However, in practical systems, conventional FL suffers from several limitations. First, FL requires clients to train the entire model locally, which imposes substantial computation and energy burden on resource-constrained clients. As the model and datasets grow in size and complexity, training full models on these clients becomes prohibitively expensive in terms of the delay and energy consumption on training. This degrades the learning efficiency significantly. Second, the central server remains under-utilized in FL, as it only performs simple parameter aggregation with a small computation load. This can lead to system-wide inefficiency, especially in scenarios where the central server is equipped with powerful computation resources (e.g., graphics processing units (GPUs)) and an uninterrupted power supply. Recently, split learning (SL) [gupta2018distributed] has been proposed to address the aforementioned issues. During SL training, the model is split into two parts. Each client maintains one part of the model (called the client-side model), while the central server stores the remaining part (called the server-side model). The client performs forward propagation (FP) and determines the intermediate output using its client-side model and local data. The intermediate output is sent to the central server. The central server continues the training with backpropagation (BP) using the server-side model and determines the gradient of the model. It sends the gradient back to the client to complete the BP. When a client has completed its BP with its client-side model, it then transfers the model to the next client for further training. In [vepakomma2018split], the authors applied SL to the use case of eHealth. In [poirot2019split], the authors utilized SL for medical image classification. In [li2023convergence], the authors provided theoretical analysis and derived the convergence bound of sequential SL. In [zhang2025split], the authors proposed a hierarchical SL scheme for fine‑tuning large language models over wireless networks. In [lin2025leo], the authors adopt SL for satellite networks with intermittent connectivity and resource constraints. Although SL can leverage the computation resources of the central server and has attracted significant attention in the research community [vepakomma2018split, poirot2019split, li2023convergence, zhang2025split, lin2025leo], it has two limitations. First, this sequential learning approach can cause the model to forget previously learned information when new information is available, which can lead to catastrophic forgetting [kirkpatrick2017overcoming]. Second, a client may experience a long idle time while waiting for other clients to finish their training. This can reduce the learning efficiency. To address the aforementioned limitations, split federated learning (SFL) has been proposed [thapa2022splitfed, hong2022efficient, 10234718, 10378869, han2024convergence, dachille2024impact]. In SFL, in addition to model splitting as in SL, clients can train their client-side models in parallel. In [thapa2022splitfed], the authors proposed an integrated learning framework of SL and FL. In [hong2022efficient], the authors proposed a split-mix strategy for FL to achieve model customization. In [10234718], the authors proposed an SFL framework for a ring topology. In [10378869], the authors proposed a knowledge distillation approach tailored for personalized federated split learning. In [han2024convergence], the authors analyzed the convergence rate of SFL under strongly convex, convex, and non-convex settings. In [dachille2024impact], the authors theoretically and empirically analyzed the effect of model splitting points on the learning performance of SFL. However, the aforementioned works [thapa2022splitfed, hong2022efficient, 10234718, 10378869, han2024convergence, dachille2024impact] did not consider the physical layer characteristics (e.g., wireless channel conditions) when implementing SFL in practical wireless systems. Unlike the conventional FL approaches where model exchange occurs after several model updates, SFL requires transmitting the intermediate output and gradient in each model update. It is crucial to allocate network resources efficiently in order to improve the learning efficiency. Some recent works have proposed resource allocation algorithms for SFL [10274134, 10314792, 10301639, 10040976, 10304624, lin2023efficient, tirana2024workflow, lin2024adaptsfl, 10740645, 10910050, 10700751, 10855336, lin2025hasfl, 10980018]. In [10274134], the authors proposed a clustering-based SFL framework for model splitting and resource allocation. In [10314792], the authors proposed a personalized SFL framework in wireless networks. The authors in [10301639] proposed a scalable SFL framework which balances the learning performance and delay on training. The authors in [10040976] proposed a cluster-based parallel SL framework and a resource allocation strategy. In [10304624], the authors proposed an SFL framework to jointly determine the cut layer and bandwidth allocation for clients. We denote this framework as ACC-SFL. In [lin2023efficient], the authors proposed an efficient parallel SL (EPSL) framework which minimizes the training latency. The authors in [tirana2024workflow] explored the integration of multiple helpers in parallel SL and proposed a workflow scheduling algorithm. The authors in [lin2024adaptsfl] proposed an SFL framework which adaptively controls the model splitting and client-side model aggregation in SFL. The authors in [10740645] proposed a fine-grained parallelization framework to accelerate SFL on heterogeneous clients. The authors in [10910050] proposed a wireless SFL framework which jointly tackles the data heterogeneity and client heterogeneity. The authors in [10700751] optimized the model splitting points to minimize the overall training latency in SFL. The authors in [10855336] proposed a split federated low-rank adaptation framework to fine-tune large language models in wireless networks. The authors in [lin2025hasfl] adaptively controlled the batch sizes and model splitting points for edge devices in wireless networks to tackle the resource heterogeneity issue. The authors in [10980018] proposed a two-tier hierarchical SFL framework in wireless networks and optimized its resource allocation. The aforementioned works [10274134, 10314792, 10301639, 10040976, 10304624, lin2023efficient, tirana2024workflow, lin2024adaptsfl, 10740645, 10910050, 10700751, 10855336, 10980018, lin2025hasfl] assumed that there are no errors in the received data. In practical wireless systems, channel fading can lead to packet errors during communications, which can degrade the learning performance. Moreover, these works assumed that the models are pre-split before training and require periodic client-side model aggregation by the central server, resulting in significant communication overhead. They do not consider adaptive model splitting, where the models can be split at different layers in each training round and only several middle layers are transmitted between the clients and the central server, which can potentially reduce the communication overhead. In this paper, we address the following question: How to jointly improve the learning performance and reduce the delay and energy consumption of SFL training over wireless networks? We aim to develop an SFL framework which enables adaptive model splitting and resource allocation over wireless networks. Achieving this goal is challenging due to the following reasons. First, the dynamic wireless channel conditions may introduce packet errors and thus degrade the learning performance. Second, it is challenging to characterize the effect of model splitting and resource allocation decisions (i.e., resource block (RB) and transmit power allocation decisions) on the convergence rate. Third, the coupling of those decisions further complicates the solution. To overcome these challenges, our work makes the following contributions: • We propose an adaptive SFL (ASFL) framework to determine the model splitting and resource allocation decisions in each training round. The proposed ASFL framework can improve the convergence rate and adapt to dynamic channel conditions in wireless networks. In addition, ASFL takes into account packet errors during the intermediate output transmission and their impact on the learning performance. • We analyze the convergence rate of ASFL. In particular, we quantify the impact of those decisions (i.e., model splitting, RB allocation, and transmit power allocation decisions) on the convergence rate. Based on the analytical results, we formulate a joint problem which optimizes the learning performance subject to the long-term delay and energy consumption constraints of ASFL. • To solve the formulated problem, we propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm by decoupling the problem into a model splitting subproblem, an RB allocation subproblem, and a transmit power allocation subproblem. To solve the model splitting subproblem, we propose an online optimization algorithm to determine the model splitting decision by considering the long-term delay and energy consumption constraints. The stability of our proposed online optimization algorithm is guaranteed by showing that the decisions satisfy the long-term delay and energy consumption constraints. Then, we determine the RB allocation decision by solving an integer programming problem. Finally, we propose an iterative algorithm to solve the transmit power allocation subproblem. We solve these subproblems alternately in each training round. • We conduct experiments on CIFAR-10 and CIFAR-100 datasets using VGG-19 and ResNet-50. We compare our proposed ASFL framework with federated averaging (FedAvg) [mcmahan2017communication], SL [vepakomma2018split], SFL [thapa2022splitfed], ACC-SFL [10304624], and EPSL [lin2023efficient]. Results show that in a wireless network setting with 10 clients, our proposed ASFL framework converges faster than the baseline schemes and can reduce the total delay and energy consumption on training by up to 75% and 80%, respectively. The rest of this paper is organized as follows. The system model is introduced in Section I. The theoretical analysis and problem formulation are presented in Section I. In Section IV, we present our proposed OOE-BCD algorithm. Experimental results are given in Section V. Conclusions are drawn in Section VI. Notations: We use italic upper case letters, boldface upper case letters, and boldface lower case letters to denote sets, matrices, and vectors, respectively. ℝM×NR^M× N denotes the set of M×NM× N real-valued matrices. N∈ℝN1_N ^N and N∈ℝN0_N ^N denote the all-ones and all-zeros column vectors with dimension N, respectively. Mathematical operators [⋅]E[·], (⋅)⊺(·) , [⋅;⋅][·;·], ⟨⋅⟩ · , |⋅| · , ∥⋅∥0\|·\|_0, and ∥⋅∥\|·\| denote the expectation, transpose, column-wise concatenation, inner product, absolute value, 0-norm, and 2-norm, respectively. ∼ denotes “distributed as”. [⋅]U[·] and (⋅)CN(·) denote the uniform distribution and complex normal distribution, respectively. The key notations used in this work are summarized in Table I. Table I: List of key notations Notation Definition Notation Definition B Bandwidth of a resource block qmq_m Output size of the m-th layer (in bits) BDNB^DN Downlink bandwidth ℛR Set of all training rounds cnDN,rc_n^DN,r Downlink transmission rate for client n in the r-th training round R Number of training rounds cnUP,rc_n^UP,r Uplink transmission rate for client n in the r-th training round snrs_n^r Packet error rate of the intermediate output transmission of client n in the r-th training round DnD_n Number of training samples of client n unru_n^r Allocation decision of the k-th RB for client n in the r-th training round fnf_n CPU frequency of client n vmFPv_m^FP Computation workload for FP of the m-th layer (in FLOPs) fsf^s CPU frequency of the central server vmBPv_m^BP Computation workload for BP of the m-th layer (in FLOPs) |hnr| h_n^r Channel gain for client n in the r-th training round nrw_n^r Client n’s model in the r-th training round ℳM Set of all model layers nrz_n^r Intermediate output of client n in the r-th training round K Set of all RBs α Waterfall threshold K Number of RBs βnr _n^r Binary indicator of packet error experienced by client n in the r-th training round M Number of model layers γ Upper bound of average delay on training N Set of all clients δ Upper bound of average energy consumption on training N Number of clients λmr _m^r Model splitting decision of the m-th layer in the r-th training round N0N_0 Received noise power spectral density n ξ_n A mini-batch of training samples of client n pnrp_n^r Transmit power for client n in the r-th training round ψm _m Size of the m-th layer (in bits) psp^s Transmit power for the central server κcκ^c Number of CPU cycles required for a client to complete one FLOP pmaxp^max Maximum transmit power ϕφ Energy consumption coefficient I System Model We consider SFL over a wireless network. Let ℛ=1,2,…,RR=\1,2,…,R\ and =1,2,…,NN=\1,2,…,N\ denote the set of R training rounds and the set of N clients, respectively. Each client n∈n has a local dataset nD_n with DnD_n training samples. Let nrw_n^r denote client n’s model in the r-th training round. We consider the model has M layers. The set of layers is denoted by ℳ=1,…,MM=\1,…,M\. In the r-th training round, nrw_n^r is split into a client-side model nc,rw_n^c,r and a server-side model ns,rw_n^s,r, which are trained by client n and the central server, respectively. We have nr=[nc,r;ns,r]w_n^r=[w_n^c,r;w_n^s,r]. For client n∈n , let n=(n,n)∼n ξ_n=(x_n,y_n) _n denote a mini-batch of training samples of nD_n, where nx_n and ny_n are the training data and the corresponding labels, respectively. Let Fn(nr;n)F_n(w_n^r; ξ_n) denote the local loss function of client n on n ξ_n with model nrw_n^r. We denote the expected loss of client n with model nrw_n^r as Fn(nr)=n∼nFn(nr;n)F_n(w_n^r)=E_ ξ_n _nF_n(w_n^r; ξ_n). Let ¯r w^r denote the average model of all clients at the beginning of the r-th training round, which is given by ¯r=∑n=1NDnnr/∑j=1NDj w^r= _n=1^ND_nw_n^r/ _j=1^ND_j. The corresponding global loss function is denoted as F(¯r)=∑n∈DnFn(nr)/∑j∈Dj.F( w^r)= _n D_nF_n(w_n^r)/ _j D_j. We aim to minimize F(¯R+1)F( w^R+1) after R training rounds. The optimal model is denoted as ⋆w . I-A Learning Model Figure 1: Illustration of our proposed ASFL framework. Figure 2: An example of the adaptive model splitting process in stage 1 for a model with 7 layers. A client sends its fourth layer to the central server at the beginning of the r-th training round. An illustration of our proposed ASFL framework is shown in Fig. 1. The proposed framework has three stages. Stage 1: An adaptive model splitting process is invoked. Let r=(λ1r,λ2r,…,λMr)∈0,1M λ^r=( _1^r, _2^r,…, _M^r)∈\0,1\^M denote the model splitting decision vector in the r-th training round. If λmr=1 _m^r=1, then the m-th layer is for the client-side model. Otherwise, it is for the server-side model. Each client has at least one layer in its client-side model for privacy protection, i.e., λ1r=1 _1^r=1. Note that ‖r‖0\| λ^r\|_0 denotes the number of layers of the client-side model. If ‖r‖0<‖r−1‖0\| λ^r\|_0<\| λ^r-1\|_0, then the clients have fewer layers in the r-th training round than in the (r−1)(r-1)-th training round. Hence, the clients transmit the model parameters of middle layers (i.e., from the (‖r‖0+1)(\| λ^r\|_0+1)-th to the ‖r−1‖0\| λ^r-1\|_0-th layer) to the central server. Otherwise, the central server transmits the model parameters of middle layers to the clients. In addition, we require that all layers up to and including the ‖r‖0\| λ^r\|_0-th layer are for the client-side model, while the subsequent layers are for the server-side model. Therefore, we have λmr≤λm−1r _m^r≤ _m-1^r, for m∈ℳ\1m \1\. An example of adaptive model splitting is shown in Fig. 2. Based on the model splitting decision r λ^r, we denote the client-side model and server-side model of client n∈n at the beginning of the r-th training round as n,rc,rw_n, λ^r^c,r and n,rs,rw_n, λ^r^s,r, respectively. Stage 2: Each client performs FP using its client-side model and local dataset. For client n∈n , it generates the intermediate output, which is denoted as nrz_n^r. Let Φnc,r(⋅) _n^c,r(·) denote the function to generate the intermediate output by using client n’s client-side model in the r-th training round. We have nr=Φnc,r(n,rc,r,n)z_n^r= _n^c,r(w_n, λ^r^c,r,D_n). When a client has finished its client-side model FP, it transmits its intermediate output and the corresponding labels to the central server via wireless channels. Let =1,2,…,KK=\1,2,…,K\ denote the set of K RBs. We denote un,kr∈0,1u_n,k^r∈\0,1\ as the allocation decision of the k-th RB for client n∈n in the r-th training round. If un,kr=1u_n,k^r=1, then the k-th RB is allocated to client n. Otherwise, un,kr=0u_n,k^r=0. Each RB is allocated to only one client. That is, ∑n=1Nun,kr≤1,k∈,r∈ℛ _n=1^Nu_n,k^r≤ 1,k ,r . Let r∈ℝN×KU^r ^N× K denote the RB allocation decision matrix in the r-th training round, with nr=(un,1r,un,2r,…,un,Kr)⊺∈ℝ1×Nu_n^r=(u_n,1^r,u_n,2^r,…,u_n,K^r) ^1× N denoting the n-th row vector. We denote pnr∈[0,Pmax]p_n^r∈[0,P ] as client n’s uplink transmit power in the r-th training round, where PmaxP denotes the maximum transmit power. Let r=(p1r,p2r,…,pNr)∈ℝNp^r=(p_1^r,p_2^r,…,p_N^r) ^N denote the transmit power decision vector for all clients in the r-th training round. We consider that packet errors can occur in the intermediate output transmission. Each client transmits the intermediate output in a single packet. The packet error rate of the intermediate output transmission of client n∈n in the r-th training round is given by [9210812]: snr(nr,pnr)=|hnr|[1−exp(−αBN0∑k=1Kun,krpnr|hnr|2)], \!\!\!s_n^r(u_n^r,p_n^r)=E_ h_n^r [1-exp (- α BN_0 _k=1^Ku_n,k^rp_n^r h_n^r ^2 ) ], (1) where α, B, and N0N_0 denote the waterfall threshold [5703199], the bandwidth of an RB, and the received noise power spectral density, respectively. |hnr| h_n^r denotes the channel gain between client n and the central server in the r-th training round. Let ^nr z_n^r denote client n’s intermediate output, which will be used by the central server for further training with client n’s server-side model in the r-th training round. Let βnr _n^r denote the binary indicator of packet error experienced by client n in the r-th training round. If the received intermediate output has no packet error (with probability 1−snr(nr,pnr)1-s_n^r(u_n^r,p_n^r)), then βnr _n^r is equal to 1. Otherwise, βnr _n^r is equal to 0. That is, we have ℙ(βnr=1)=1−snr(nr,pnr)P( _n^r=1)=1-s_n^r(u_n^r,p_n^r) and ℙ(βnr=0)=snr(nr,pnr)P( _n^r=0)=s_n^r(u_n^r,p_n^r), respectively. We denote r=(β1r,β2r,…,βNr)∈ℝN β^r=( _1^r, _2^r,…, _N^r) ^N as the indicator vector of all clients. Similar to [9210812], the central server discards packets with errors. Thus, ^nr z_n^r is given by ^nr=βnrnr z_n^r= _n^rz_n^r. Stage 3: The central server performs FP. Let Φns,r(⋅) _n^s,r(·) denote the function to generate the FP output by using client n’s server-side model in the r-th training round. The generated output is given by ^nr=Φns,r(n,rs,r,^nr) y_n^r= _n^s,r(w_n, λ^r^s,r, z_n^r). The central server performs BP over client n’s server-side model to minimize the loss Fnr(nr;n)=LF(^nr,n)F_n^r(w_n^r; ξ_n)=L_F( y_n^r,y_n), where LF(⋅)L_F(·) denotes the loss function (e.g., cross-entropy loss). Let ~r N^r and ∇Fns,r(n,rs,r;n)∇ F_n^s,r(w_n, λ^r^s,r; ξ_n) denote the set of clients with no packet errors in the received intermediate output and the gradient of client n’s server-side model in the r-th training round, respectively. In the r-th training round, client n’s server-side model is updated as ns,r+1=n,rs,r−η∇Fns,r(n,rs,r;n),n∈~r,w_n^s,r+1=w_n, λ^r^s,r-η∇ F_n^s,r(w_n, λ^r^s,r; ξ_n), n∈ N^r, (2) where η denotes the learning rate. When all server-side models have been updated, the central server performs aggregation as ¯s,r+1=∑n∈~rDnns,r+1/∑j∈~rDj w^s,r+1= _n∈ N^rD_nw_n^s,r+1/ _j∈ N^rD_j, The central server then sends the first layer of gradient ∇Fns,r(n,rs,r;n)∇ F_n^s,r(w_n, λ^r^s,r; ξ_n) (i.e., the gradient of the (‖r‖0+1)(\| λ^r\|_0+1)-th layer of nrw_n^r) back to client n. Client n performs BP by using the received gradient. Let ∇Fnc,r(nr;n)∇ F_n^c,r(w_n^r; ξ_n) denote the gradient of client n’s client-side model in the r-th training round. In the r-th training round, client n updates its client-side model as nc,r+1=n,rc,r−η∇Fnc,r(n,rc,r;n),n∈~r.w_n^c,r+1=w_n, λ^r^c,r-η∇ F_n^c,r(w_n, λ^r^c,r; ξ_n), n∈ N^r. (3) Since the model update of ASFL depends on decision variables r,r,rr∈ℛ\U^r,p^r, λ^r\_r , we characterize their impact on the convergence rate in Section I. I-B Delay and Energy Consumption Model In this subsection, we present the delay and energy consumption in each training round as functions of decision variables r,r,rr∈ℛ\U^r,p^r, λ^r\_r . In the r-th training round, since r−1 λ^r-1 has been determined, it is not a decision variable. I-B1 Delay Similar to other works on SFL (e.g., [lin2023efficient]), we consider synchronous learning where all clients begin each stage simultaneously. An illustration of the delay in one ASFL training round is shown in Fig. 3. Figure 3: An illustration of the delay in one ASFL training round. We now present the delay incurred in each stage. In stage 1, the delay is incurred due to adaptive model splitting. That is, the clients or the central server may need to send several middle layers to the other based on the model splitting decisions. We use orthogonal frequency division multiple access (OFDMA) for transmission. In the r-th training round, for client n∈n , the downlink transmission rate cnDN,rc_n^DN,r and the uplink transmission rate cnUP,rc_n^UP,r are as follows: cnDN,r=BDNlog2(1+ps|hnr|2BDNN0), c_n^DN,r=B^DN _2 (1+ p^s h_n^r ^2B^DNN_0 ), (4) cnUP,r(nr,pnr)=∑k=1Kun,krBlog2(1+pnr|hnr|2BN0), c_n^UP,r(u_n^r,p_n^r)= _k=1^Ku_n,k^rB _2 (1+ p_n^r h_n^r ^2BN_0 ), (5) where BDNB^DN and psp^s denote the downlink bandwidth and the transmit power of the central server, respectively. Let TnS1,rT_n^S1,r denote the delay of stage 1 of client n in the r-th training round. Let =(ψ1,ψ2,…,ψM)∈ℝM ψ=( _1, _2,…, _M) ^M denote the model size vector, with ψm _m representing the size of the m-th layer (in bits). If ‖r‖0≥‖r−1‖0\| λ^r\|_0≥\| λ^r-1\|_0, then the size of the transmitted model parameters of the middle layers is equal to (r−r−1)⊺( λ^r- λ^r-1) ψ. Otherwise, the size of the transmitted model parameters of the middle layers is (r−1−r)⊺( λ^r-1- λ^r) ψ. Hence, TnS1,rT_n^S1,r satisfies TnS1,r(nr,pnr,r)=(r−r−1)⊺cnDN,r,if ‖r‖0≥‖r−1‖0,(r−1−r)⊺cnUP,r(nr,pnr),otherwise. T_n^S1,r(u_n^r,p_n^r, λ^r)= cases ( λ^r- λ^r-1) ψc_n^DN,r,&if $\| λ^r\|_0≥\| λ^r-1\|_0$,\\ ( λ^r-1- λ^r) ψc_n^UP,r(u_n^r,p_n^r),&otherwise. cases (6) The delay of stage 1 in the r-th training round is given by TS1,r(r,r,r)=maxn∈TnS1,r(nr,pnr,r). T^S1,r(U^r,p^r, λ^r)= _n \T_n^S1,r(u_n^r,p_n^r, λ^r)\. (7) Stage 2 has two processes: the client-side model FP and the intermediate output transmission. Let FP=(v1FP,…,vMFP)∈ℝMv^FP=(v_1^FP,…,v_M^FP) ^M denote the vector of computation workload for FP, with vmFPv_m^FP denoting the computation workload for FP of the m-th layer (in floating point operations (FLOPs)). The delay of the client-side model FP of client n in the r-th training round is as follows: TnCFP,r(r)=κc(r)⊺FPDnfn, T_n^CFP,r( λ^r)= κ^c( λ^r) v^FPD_nf_n, (8) where fnf_n and κcκ^c denote client n’s central processing unit (CPU) frequency and the number of CPU cycles required for a client to complete one FLOP, respectively. Let qmq_m denote the output size of the m-th layer (in bits). Since r λ^r is a binary vector and λmr≤λm−1r,m∈ℳ\1 _m^r≤ _m-1^r,m \1\, the size of the intermediate output at the ‖r‖\| λ^r\|-th layer can be represented by ∑m=1M−1(λmr−λm+1r)qm _m=1^M-1( _m^r- _m+1^r)q_m. The delay of the intermediate output transmission of client n in the r-th training round is TnUP,r(nr,pnr,r)=Dn∑m=1M−1(λmr−λm+1r)qmcnUP,r(nr,pnr). T_n^UP,r(u_n^r,p_n^r, λ^r)= D_n _m=1^M-1( _m^r- _m+1^r)q_mc_n^UP,r(u_n^r,p_n^r). (9) The total delay of stage 2 in the r-th training round is TS2,r(r,r,r) T^S2,r(U^r,p^r, λ^r) = = maxn∈TnCFP,r(r)+TnUP,r(nr,pnr,r). \> _n \T_n^CFP,r( λ^r)+T_n^UP,r(u_n^r,p_n^r, λ^r)\. (10) Stage 3 has four processes: the server-side model FP, the server-side model BP, the gradient transmission, and the client-side model BP. Since βnr _n^r is a random variable, we define the expected delay of the server-side model FP in the r-th training round as the time required for the central server to complete the FLOPs of all server-side model FP, which is given by r[TSFP,r(r,r,r)] _ β^r[T^SFP,r(U^r,p^r, λ^r)] = = κs(M−r)⊺FP∑n=1N(1−snr(nr,pnr))Dnfs, \> κ^s(1_M- λ^r) v^FP _n=1^N (1-s_n^r(u_n^r,p_n^r) )D_nf^s, (11) where fsf^s and κsκ^s denote the CPU frequency of the central server and the number of CPU cycles required for the central server to complete one FLOP, respectively. The expected delay of the server-side model BP in the r-th training round is r[TSBP,r(r,r,r)] _ β^r[T^SBP,r(U^r,p^r, λ^r)] = = κs(M−r)⊺BP∑n=1N(1−snr(nr,pnr))Dnfs, \> κ^s(1_M- λ^r) v^BP _n=1^N (1-s_n^r(u_n^r,p_n^r) )D_nf^s, (12) where BP=(v1BP,…,vMBP)∈ℝMv^BP=(v_1^BP,…,v_M^BP) ^M denotes the vector of computation workload for BP, with vmBPv_m^BP denoting the computation workload for BP of the m-th layer (in FLOPs). The size of the gradient of the (‖r‖0+1)(\| λ^r\|_0+1)-th layer can be represented by ∑m=1M−1(λmr−λm+1r)ψm+1 _m=1^M-1( _m^r- _m+1^r) _m+1. The expected delay for the gradient transmission to client n∈n in the r-th training round is βnr[TnDN,r(nr,pnr,r)] _ _n^r[T_n^DN,r(u_n^r,p_n^r, λ^r)] = = (1−snr(nr,pnr))Dn∑m=1M−1(λmr−λm+1r)ψm+1cnDN,r. \> (1\!-\!s_n^r(u_n^r,p_n^r) )D_n _m=1^M-1( _m^r- _m+1^r) _m+1c_n^DN,r. (13) The expected delay of the client-side model BP of client n∈n in the r-th training round is obtained as βnr[TnCBP,r(nr,pnr,r)]=κc(1−snr(nr,pnr))(r)⊺BPDnfn. \!E_ _n^r[T_n^CBP,r(u_n^r,p_n^r, λ^r)]\!=\! κ^c (1\!-\!s_n^r(u_n^r,p_n^r) )( λ^r) v^BPD_nf_n. (14) The expected total delay of all clients in stage 3 of the r-th training round is given by r[TS3,r(r,r,r)] _ β^r[T^S3,r(U^r,p^r, λ^r)] = = maxn∈r[TSFP,r(r,r,r)+TSBP,r(r,r,r) \> _n \E_ β^r[T^SFP,r(U^r,p^r, λ^r)+T^SBP,r(U^r,p^r, λ^r) +TnDN,r(nr,pnr,r)+TnCBP,r(nr,pnr,r)]. +T_n^DN,r(u_n^r,p_n^r, λ^r)+T_n^CBP,r(u_n^r,p_n^r, λ^r)]\. (15) Thus, the expected total delay in the r-th training round is r[Tr(r,r,r)]=TS1,r(r,r,r) _ β^r[T^r(U^r,p^r, λ^r)]=T^S1,r(U^r,p^r, λ^r) +TS2,r(r,r,r)+r[TS3,r(r,r,r)]. +T^S2,r(U^r,p^r, λ^r)+E_ β^r[T^S3,r(U^r,p^r, λ^r)]. (16) I-B2 Energy Consumption Each client incurs energy consumption through four processes: model splitting, client-side model FP, intermediate output transmission, and client-side model BP. We do not consider the energy consumption at the central server due to its continuous power supply. The energy consumption of client n∈n for model splitting in the r-th training round is given by EnMS,r(nr,pnr,r)=(r−1−r)⊺pnrcnUP,r(nr,pnr). E_n^MS,r(u_n^r,p_n^r, λ^r)= ( λ^r-1- λ^r) ψp_n^rc_n^UP,r(u_n^r,p_n^r). (17) The energy consumption of client n∈n for client-side FP in the r-th training round is the energy required for client n to process all training samples, which is given by EnFP,r(r)=Dn(r)⊺FPϕκc(fn)2, E_n^FP,r( λ^r)=D_n( λ^r) v^FPφκ^c(f_n)^2, (18) where ϕφ is the energy consumption coefficient. The energy consumption of client n∈n for the intermediate output transmission in the r-th training round can be expressed as EnUP,r(nr,pnr,r)=pnrTnUP,r(nr,pnr,r). E_n^UP,r(u_n^r,p_n^r, λ^r)=p_n^rT_n^UP,r(u_n^r,p_n^r, λ^r). (19) The expected energy consumption of client n∈n for client-side model BP in the r-th training round is given by βnr[EnCBP,r(nr,pnr,r)] _ _n^r[E_n^CBP,r(u_n^r,p_n^r, λ^r)] = = (1−snr(pnr,nr))Dn(r)⊺BPϕκc(fn)2. \> (1-s_n^r(p_n^r,u_n^r) )D_n( λ^r) v^BPφκ^c(f_n)^2. (20) The expected total energy consumption of client n in the r-th training round is obtained as βnr[Enr(nr,pnr,r)]=EnMS,r(nr,pnr,r)+EnFP,r(r) _ _n^r[E_n^r(u_n^r,p_n^r, λ^r)]=E_n^MS,r(u_n^r,p_n^r, λ^r)+E_n^FP,r( λ^r) +EnUP,r(nr,pnr,r)+βnr[EnCBP,r(nr,pnr,r)]. +E_n^UP,r(u_n^r,p_n^r, λ^r)+E_ _n^r[E_n^CBP,r(u_n^r,p_n^r, λ^r)]. (21) I Theoretical Analysis and Problem Formulation In this section, we first characterize the impact of decision variables on the convergence rate of our proposed ASFL framework. Then, we present the problem formulation. I-A Theoretical Analysis Without loss of generality, we conduct analysis under non-convex loss functions. We first present the following assumptions which are widely used in the literature (e.g., [li2019convergence, WangLLJP20, 9261995]). Assumption 1. The loss function of each client n∈n is continuously differentiable and L-smooth. That is, for arbitrary two vectors nrw_n^r and ~nr w_n^r, we have Fn(nr)≤Fn(~nr)+⟨∇Fn(~nr),nr−~nr⟩+L2‖nr−~nr‖2.F_n(w_n^r)≤ F_n( w_n^r)+ ∇ F_n( w_n^r),w_n^r- w_n^r + L2\|w_n^r- w_n^r\|^2. Assumption 2. The variance of the local stochastic gradient of each client n∈n is upper-bounded, i.e., n∼n[‖∇Fn(nr;n)−∇Fn(nr)‖2]≤σF2E_ ξ_n _n [\|∇ F_n(w_n^r; ξ_n)-∇ F_n(w_n^r)\|^2 ]≤ _F^2. Assumption 3. The expected square norm of the gradient of each client n∈n is upper-bounded, i.e., n∼n[‖∇Fn(nr;n)‖2]≤ψF2E_ ξ_n _n[\|∇ F_n(w_n^r; ξ_n)\|^2]≤ _F^2. We introduce two lemmas which are widely used in the literature (e.g., [li2019convergence, WangLLJP20]) to facilitate our proof. Lemma 1. For arbitrary vector nrw_n^r, n∈n , we have ‖∑n=1Nnr‖2≤N∑n=1N‖nr‖2\| _n=1^Nw_n^r\|^2≤ N _n=1^N\|w_n^r\|^2. Lemma 2. For arbitrary two vectors nrw_n^r and ~nr w_n^r, n∈n , we have ⟨nr,~nr⟩=12(‖nr‖2+‖~nr‖2−‖nr−~nr‖2) _n^r, w_n^r = 12(\|w_n^r\|^2+\| w_n^r\|^2-\|w_n^r- w_n^r\|^2). We denote the average client-side model and server-side model after model splitting at the beginning of the r-th training round as ¯rc,r w_ λ^r^c,r and ¯rs,r w_ λ^r^s,r, respectively. Now, we present the convergence rate of our proposed ASFL in Theorem 1. Theorem 1. Under Assumptions 1 −- 3 and Lemmas 1 −- 2, the convergence rate of our proposed ASFL is bounded by 1R∑r=1R‖∇F(¯r)‖2≤2R(F(¯1)−F(⋆))+3(σF2+ψF2) 1R _r=1^R\|∇ F( w^r)\|^2≤ 2R (F( w^1)-F(w ) )+3( _F^2+ _F^2) + + 2(L+2)NR∑r=1R∑n=1N(‖¯rc,r−n,rc,r‖2+(1−βnr)2‖¯rs,r‖2)⏟Average long-term model discrepancies \> 2(L+2)NR _r=1^R _n=1^N(\| w_ λ^r^c,r-w_n, λ^r^c,r\|^2+(1- _n^r)^2\| w_ λ^r^s,r\|^2)_Average long-term model discrepancies + + 2(L+2)η2ψF2. \>2(L+2)η^2 _F^2. (22) Proof. The model of client n at the beginning of the (r+1)(r+1)-th training round is given by nr+1=nr−βnrη∇Fn(nr;nr)w_n^r+1=w_n^r- _n^rη∇ F_n(w_n^r; ξ_n^r). We can derive ¯r+1 w^r+1 as follows: ¯r+1= w^r+1=\> [1N∑n=1N(n,rc,r−βnrη∇Fnc(nr;nr)); [ 1N _n=1^N (w_n, λ^r^c,r- _n^rη∇ F_n^c(w_n^r; ξ_n^r) ); 1N∑n=1Nβnr(n,rs,r−η∇Fns(nr;nr))] 1N _n=1^N _n^r (w_n, λ^r^s,r-η∇ F_n^s(w_n^r; ξ_n^r) ) ] = =\> [1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r] [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ] −1N∑n=1Nβnrη∇Fn(nr;nr). - 1N _n=1^N _n^rη∇ F_n(w_n^r; ξ_n^r). (23) Based on eqn. (I-A), we expand [F(¯r+1)]E[F( w^r+1)] as follows: [F(¯r+1)] [F( w^r+1)] = = [F(¯r−¯r+[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r] \>E [F ( w^r- w^r+ [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ] −ηN∑n=1Nβnr∇Fn(nr;nr))] - ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) ) ] ≤(a) (a)≤ [F(¯r)]−[⟨∇F(¯r),¯r−[1N∑n=1Nn,rc,r; \>E[F( w^r)]-E [ ∇ F( w^r), w^r- [ 1N _n=1^Nw_n, λ^r^c,r; 1N∑n=1Nβnrn,rs,r]+ηN∑n=1Nβnr∇Fn(nr;nr)⟩] 1N _n=1^N _n^rw_n, λ^r^s,r ]+ ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) ] +L2[∥[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r]−¯r + L2E [ \| [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ]- w^r −ηN∑n=1Nβnr∇Fn(nr;nr)∥2] - ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ] =(b) (b)= [F(¯r)]−12([∥∇F(¯r)∥2+∥¯r−[1N∑n=1Nn,rc,r; \>E[F( w^r)]- 12 (E [\|∇ F( w^r)\|^2+ \| w^r- [ 1N _n=1^Nw_n, λ^r^c,r; 1N∑n=1Nβnrn,rs,r]+ηN∑n=1Nβnr∇Fn(nr;nr)∥2] 1N _n=1^N _n^rw_n, λ^r^s,r ]+ ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ] −[∥∇F(¯r)−¯r+[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r] -E [ \|∇ F( w^r)- w^r+ [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ] −ηN∑n=1Nβnr∇Fn(nr;nr)∥2])+L2[∥[1N∑n=1Nn,rc,r; - ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ] )+ L2E [ \| [ 1N _n=1^Nw_n, λ^r^c,r; 1N∑n=1Nβnrn,rs,r]−¯r−ηN∑n=1Nβnr∇Fn(nr;nr)∥2], 1N _n=1^N _n^rw_n, λ^r^s,r ]- w^r- ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ], (24) where inequality (a) results from Assumption 1. Equality (b) is obtained by using Lemma 2. For illustration simplicity, we define A1=[‖∇F(¯r)−¯r+[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r]−ηN∑n=1Nβnr∇Fn(nr;nr)‖2]A_1=E [ \|∇ F( w^r)- w^r+ [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ]- ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ] and A2=[‖[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r]−¯r−ηN∑n=1Nβnr∇Fn(nr;nr)‖2]A_2=E [ \| [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ]- w^r- ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ]. A1A_1 satisfies A1= A_1= [∥∇F(¯r)−1N∑n=1N∇Fn(¯r;ξnr) \>E [ \|∇ F( w^r)- 1N _n=1^N∇ F_n( w^r; _n^r) +1N∑n=1N∇Fn(¯r;ξnr)−¯r+[1N∑n=1Nn,rc,r; + 1N _n=1^N∇ F_n( w^r; _n^r)- w^r+ [ 1N _n=1^Nw_n, λ^r^c,r; 1N∑n=1Nβnrn,rs,r]−ηN∑n=1Nβnr∇Fn(nr;nr)∥2] 1N _n=1^N _n^rw_n, λ^r^s,r ]- ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ] ≤(a) (a)≤ 3[‖1N∑n=1N∇Fn(¯r)−1N∑n=1N∇Fn(¯r;ξnr)‖2] \>3E [ \| 1N _n=1^N∇ F_n( w^r)- 1N _n=1^N∇ F_n( w^r; _n^r) \|^2 ] +3[‖1N∑n=1N∇Fn(¯r;ξnr)‖2]+3A2 +3E [ \| 1N _n=1^N∇ F_n( w^r; _n^r) \|^2 ]+3A_2 ≤(b) (b)≤ 3N∑n=1N[‖∇Fn(¯r)−∇Fn(¯r;ξnr)‖2] \> 3N _n=1^NE [ \|∇ F_n( w^r)-∇ F_n( w^r; _n^r) \|^2 ] +3N∑n=1N[‖∇Fn(¯r;ξnr)‖2]+3A2 + 3N _n=1^NE [ \|∇ F_n( w^r; _n^r) \|^2 ]+3A_2 ≤(c) (c)≤ 3(σF2+ψF2)+3A2, \>3( _F^2+ _F^2)+3A_2, (25) where inequalities (a) and (b) result from Lemma 1. Inequality (c) results from Assumptions 2 and 3. Therefore, [F(¯r+1)]E[F( w^r+1)] satisfies [F(¯r+1)]≤ [F( w^r+1)]≤ [F(¯r)]−12‖∇F(¯r)‖2+32(σF2+ψF2) \>E[F( w^r)]- 12\|∇ F( w^r)\|^2+ 32( _F^2+ _F^2) +L+22A2. + L+22A_2. (26) Then, we analyze A2A_2. In particular, it satisfies A2= A_2= [∥ηN∑n=1Nβnr∇Fn(nr;nr)+[1N∑n=1N¯rc,r; \>E [ \| ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r)+ [ 1N _n=1^N w_ λ^r^c,r; 1N∑n=1N¯rs,r]−[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r]∥2] 1N _n=1^N w_ λ^r^s,r ]- [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ] \|^2 ] ≤(a) (a)≤ 2[∥ηN∑n=1Nβnr∇Fn(nr;nr)∥2]+2[∥[1N∑n=1N¯rc,r; \>2E [ \| ηN _n=1^N _n^r∇ F_n(w_n^r; ξ_n^r) \|^2 ]+2E [ \| [ 1N _n=1^N w_ λ^r^c,r; 1N∑n=1N¯rs,r]−[1N∑n=1Nn,rc,r;1N∑n=1Nβnrn,rs,r]∥2] 1N _n=1^N w_ λ^r^s,r ]- [ 1N _n=1^Nw_n, λ^r^c,r; 1N _n=1^N _n^rw_n, λ^r^s,r ] \|^2 ] ≤(b) (b)≤ 2η2ψF2N∑n=1N(βnr)2+2N∑n=1N‖¯rc,r−n,rc,r‖2 \> 2η^2 _F^2N _n=1^N( _n^r)^2+ 2N _n=1^N \| w_ λ^r^c,r-w_n, λ^r^c,r \|^2 +2N∑n=1N‖¯rs,r−βnrn,rs,r‖2, + 2N _n=1^N \| w_ λ^r^s,r- _n^rw_n, λ^r^s,r \|^2, (27) where inequality (a) is derived using Lemma 1. Inequality (b) results from Lemma 1 and Assumption 3. Therefore, [F(¯r+1)]E[F( w^r+1)] satisfies [F(¯r+1)] [F( w^r+1)] ≤ ≤ [F(¯r)]−12‖∇F(¯r)‖2+32(σF2+ψF2) \>E[F( w^r)]- 12\|∇ F( w^r)\|^2+ 32( _F^2+ _F^2) +(L+2)η2ψF2N∑n=1N(βnr)2+L+2N∑n=1N‖¯rc,r−n,rc,r‖2 + (L+2)η^2 _F^2N _n=1^N( _n^r)^2+ L+2N _n=1^N \| w_ λ^r^c,r-w_n, λ^r^c,r \|^2 +L+2N∑n=1N‖¯rs,r−βnrn,rs,r‖2. + L+2N _n=1^N \| w_ λ^r^s,r- _n^rw_n, λ^r^s,r \|^2. (28) To determine the convergence rate, we sum up both sides of inequality (I-A) for all R training rounds and multiply both sides by 2R 2R. Then, we rearrange the inequality above and obtain the convergence rate of our proposed ASFL as 1R∑r=1R‖∇F(¯r)‖2 1R _r=1^R\|∇ F( w^r)\|^2 ≤(a) (a)≤ 2R(F(¯1)−F(⋆))+2(L+2)η2ψF2NR∑r=1R∑n=1N(βnr)2 \> 2R (F( w^1)-F(w ) )+ 2(L+2)η^2 _F^2NR _r=1^R _n=1^N ( _n^r )^2 +2(L+2)NR∑r=1R∑n=1N‖¯rc,r−n,rc,r‖2+3(σF2+ψF2) + 2(L+2)NR _r=1^R _n=1^N \| w_ λ^r^c,r-w_n, λ^r^c,r \|^2+3( _F^2+ _F^2) +2(L+2)NR∑r=1R∑n=1N‖¯rs,r−βnrn,rs,r‖2 + 2(L+2)NR _r=1^R _n=1^N \| w_ λ^r^s,r- _n^rw_n, λ^r^s,r \|^2 ≤(b) (b)≤ 2R(F(¯1)−F(⋆))+2(L+2)η2ψF2+3(σF2+ψF2) \> 2R (F( w^1)-F(w ) )+2(L+2)η^2 _F^2+3( _F^2+ _F^2) +2(L+2)NR∑r=1R∑n=1N‖¯rc,r−n,rc,r‖2 + 2(L+2)NR _r=1^R _n=1^N \| w_ λ^r^c,r-w_n, λ^r^c,r \|^2 +2(L+2)NR∑r=1R∑n=1N(1−βnr)2‖¯rs,r‖2, + 2(L+2)NR _r=1^R _n=1^N(1- _n^r)^2 \| w_ λ^r^s,r \|^2, (29) where inequality (a) is obtained by using F(¯R+1)≥F(⋆)F( w^R+1)≥ F(w ). Inequality (b) results from the fact that βnr≤1 _n^r≤ 1 and ¯rs,r=n,rs,r w_ λ^r^s,r=w_n, λ^r^s,r. This completes the proof of Theorem 1. ∎ Remark 1. Theorem 1 suggests that the first two terms and the fourth term on the right-hand side of inequality (I-A) are independent of the decision variables r,r,rr∈ℛ\U^r,p^r, λ^r\_r . Thus, we should optimize the decision variables r,r,rr∈ℛ\U^r,p^r, λ^r\_r in order to minimize the average long-term model discrepancies (i.e., the third term on the right-hand side of inequality (I-A)). Recall from Section I-A that βnr _n^r is a binary random variable, which is equal to 1 with probability 1−snr(nr,pnr)1-s_n^r(u_n^r,p_n^r). Hence, we aim to minimize the expected average long-term model discrepancies by considering the randomness of βnr _n^r. On the other hand, the decision variables r,r,rr∈ℛ\U^r,p^r, λ^r\_r can affect the delay and energy consumption significantly. Thus, we need to determine them properly to jointly improve the learning performance and efficiency of our proposed ASFL. I-B Problem Formulation We aim to minimize the expected average long-term model discrepancies while guaranteeing the delay and energy consumption constraints. We formulate the problem as follows: minimizer,r,rr∈ℛ *minimize_ subarrayc\U^r,p^r,\\ λ^r\_r subarray 1NR∑r=1R∑n=1N(∥¯rc,r−n,rc,r∥2 1NR _r=1^R _n=1^N ( \| w_ λ^r^c,r-w_n, λ^r^c,r \|^2 +(1−snr(nr,pnr))2∥¯rs,r∥2) + (1-s_n^r(u_n^r,p_n^r) )^2 \| w_ λ^r^s,r \|^2 ) (30a) subjectto \>to 1R∑r=1Rr[Tr(r,r,r)]≤γ, 1RΣ _r=1^RE_ β^r[T^r(U^r,p^r, λ^r)]≤γ, (30b) 1R∑r=1Rβnr[Enr(nr,pnr,r)]≤δ,n∈, 1RΣ _r=1^RE_ _n^r[E_n^r(u_n^r,p_n^r, λ^r)]≤δ, n , (30c) ∑n=1Nun,kr≤1,k∈,r∈ℛ, Σ _n=1^Nu_n,k^r≤ 1, k ,r , (30d) 0≤pnr≤Pmax,n∈,r∈ℛ, 0≤ p_n^r≤ P^max, n ,r , (30e) λ1r=1,r∈ℛ, _1^r=1, r , (30f) λmr≤λm−1r,m∈ℳ\1,r∈ℛ, _m^r≤ _m-1^r, m \1\,r , (30g) λmr∈0,1,m∈ℳ\1,r∈ℛ, _m^r∈\0,1\, m \1\,r , (30h) un,kr∈0,1,n∈,k∈,r∈ℛ. u_n,k^r∈\0,1\, n ,k ,r . (30i) Constraints (30b) and (30c) ensure that the average delay and energy consumption on training are bounded. Constraints (30d) −- (30i) are constraints for the RB allocation, transmit power allocation, and model splitting decisions. The challenges of solving problem (30) are twofold. First, due to the dynamic wireless channel conditions and time-varying model parameters, it is difficult to guarantee constraints (30b) and (30c) while minimizing the nonconvex objective function (30a). Second, the coupling of model splitting and resource allocation decisions complicates the solution. To address these challenges, we propose an online optimization enhanced block coordinate descent (OOE-BCD) algorithm. IV OOE-BCD Algorithm To address the aforementioned challenges, we decompose problem (30) into three subproblems and solve them iteratively in each training round. In each subproblem, we optimize a decision variable by fixing the other two decision variables. Our proposed OOE-BCD algorithm is presented at the end of this section. To avoid the communication overhead of sending the entire client-side model to the central server for determining the objective value (30a), each client n randomly samples a subset of its client-side model parameters with a sampling ratio ι 111As in [10971879], a small value of sampling ratio is sufficient to characterize the average model discrepancies. The corresponding communication cost can be considered to be negligible. as ~n,rc,r w_n, λ^r^c,r and transmits the sampled client-side model to the central server. The average sampled client-side model is given by ~ravg,r=1N∑n=1N~n,rc,r w_ λ^r^avg,r= 1N _n=1^N w_n, λ^r^c,r. The objective function (30a) can be written as 1R∑r=1Rgobjr(r,r,r)≈1NR∑r=1R∑n=1N(1ι‖~ravg,r−~n,rc,r‖2+(1−snr(nr,pnr))2‖¯rs,r‖2) 1R _r=1^Rg_obj^r(U^r,p^r, λ^r)≈ 1NR _r=1^R _n=1^N ( 1 \| w_ λ^r^avg,r- w_n, λ^r^c,r\|^2+(1-s_n^r(u_n^r,p_n^r))^2\| w_ λ^r^s,r\|^2 ), where gobjr(r,r,r)g_obj^r(U^r,p^r, λ^r) denotes the objective function in the r-th training round. IV-A Model Splitting Subproblem We first fix the decision variables r,rr∈ℛ\U^r,p^r\_r and optimize the model splitting decisions rr∈ℛ\ λ^r\_r . The objective is denoted as gobjr(r)g_obj^r( λ^r). The challenges of solving this subproblem are twofold. First, the model splitting decisions are coupled between two consecutive training rounds. When we determine r+1 λ^r+1, the objective function (30a) and constraints (30b) and (30c) depend on the previous decision r λ^r. Second, we need to balance the long-term objective and constraints. Hence, to address these two challenges, we propose an online optimization algorithm. IV-A1 Online Optimization Algorithm By fixing the decision variables r,rr∈ℛ\U^r,p^r\_r and rearranging constraints (30b) and (30c), we have g0r(r)=1R∑r=1Rr[Tr(r)]−γ≤0g_0^r( λ^r)= 1RΣ _r=1^RE_ β^r[T^r( λ^r)]-γ≤ 0 and gnr(r)=1R∑r=1Rβnr[Enr(r)]−δ≤0g_n^r( λ^r)= 1RΣ _r=1^RE_ _n^r[E_n^r( λ^r)]-δ≤ 0, n∈n . Given r,rr∈ℛ\U^r,p^r\_r , problem (30) can be transformed into the following form: minimizerr∈ℛ *minimize_ subarrayc\ λ^r\_r subarray 1R∑r=1Rgobjr(r) 1RΣ _r=1^Rg_obj^r( λ^r) (31a) subjectto \>to 1R∑r=1Rg0r(r)≤0, 1RΣ _r=1^Rg_0^r( λ^r)≤ 0, (31b) 1R∑r=1Rgnr(r)≤0,n≤, 1RΣ _r=1^Rg_n^r( λ^r)≤ 0,\>\>\>n , (31c) constraints(30f),(30g),and(30h). \>\> ( c8),\> ( c7),\>and\> ( c2). By using Lyapunov optimization [Neelybook], we introduce N+1N+1 virtual queues QnrQ_n^r, 0≤n≤N0≤ n≤ N in the r-th training round to account for constraints (31b) and (31c). Let r=(Q0r,Q1r,…,QNr)∈ℝN+1Q^r=(Q_0^r,Q_1^r,…,Q_N^r) ^N+1 denote the virtual queue vector. We initialize rQ^r as 0=N+1Q^0=0_N+1. In the r-th training round, for 0≤n≤N0≤ n≤ N, we update each virtual queue as Qnr+1=maxμQnr+(1−μ)gnr(r),0, Q_n^r+1= \μ Q_n^r+(1-μ)g_n^r( λ^r),0\, (32) where μ∈[0,1]μ∈[0,1] is a tunable parameter that controls the impact of constraints (31b) and (31c). We characterize the virtual queue backlog as L(r)=12‖r‖2L(Q^r)= 12\|Q^r\|^2, r∈ℛr . We then define a Lyapunov drift to characterize the stability of the virtual queue as Δr=L(r+1)−L(r)=12(‖r+1‖2−‖r‖2),r∈ℛ. ^r=L(Q^r+1)-L(Q^r)= 12(\|Q^r+1\|^2-\|Q^r\|^2), r . (33) To jointly optimize the constraints and the objective, we define a drift-plus-penalty term as Δr+Vgobjr(r) ^r+Vg_obj^r( λ^r), where V is a nonnegative coefficient. To jointly guarantee the stability of the virtual queue vector and optimize our objective, instead of solving problem (31), we solve the following optimization problem in each training round r∈ℛr : minimizer *minimize_ subarrayc λ^r subarray Δr+Vgobjr(r) ^r+Vg_obj^r( λ^r) (34a) subjectto \>to λ1r=1, _1^r=1, (34b) λmr≤λm−1r,m∈ℳ\1, _m^r≤ _m-1^r,\>\>m \1\, (34c) λmr∈0,1,m∈ℳ\1. _m^r∈\0,1\,\>\>m \1\. (34d) Due to constraints (34b), (34c), and (34d), there are M feasible solutions. Hence, we can use an exhaustive search approach to solve this problem efficiently. IV-A2 Stability Analysis We provide stability analysis of our proposed online optimization algorithm and characterize the impact of the tunable parameter μ. We introduce an assumption which is widely used in the literature (e.g., [9687317, he2024online]). Assumption 4. There exist two positive constants TmaxT and EmaxE such that for arbitrary decision vector r λ^r, we have r[Tr(r)]≤TmaxE_ β^r[T^r( λ^r)]≤ T and βnr[Enr(r)]≤EmaxE_ _n^r[E_n^r( λ^r)]≤ E , n∈n , r∈ℛr . We present three lemmas which bound the constraints, value of the virtual queue, and drift-plus-penalty term, respectively. Lemma 3. The values of the square of constraints satisfy (g0r(r))2=maxγ2,(Tmax−γ)2=G1(g_0^r( λ^r))^2= \γ^2,(T -γ)^2\=G_1 and (gnr(r))2=maxδ2,(Emax−δ)2=G2(g_n^r( λ^r))^2= \δ^2,(E -δ)^2\=G_2, n∈n , r∈ℛr . Proof. Based on Assumption 4, we have |g0r(r)|=maxγ,Tmax−γ. g_0^r( λ^r) = \γ,T -γ\. (35) Thus, the following equality always hold: (g0r(r))2=maxγ2,(Tmax−γ)2. (g_0^r( λ^r) )^2= \γ^2,(T -γ)^2\. (36) We use similar steps to obtain (gnr(r))2=maxδ2,(Emax−δ)2 (g_n^r( λ^r) )^2= \δ^2,(E -δ)^2\, n∈n , r∈ℛr . ∎ Lemma 4. The values of virtual queues are bounded by Q0r≤G1Q_0^r≤ G_1 and Qnr≤G2Q_n^r≤ G_2, n∈n , r∈ℛr . Proof. We use induction for our proof. First, we have Q11=0<G1Q_1^1=0< G_1. Then, we suppose Q0r≤G1Q_0^r≤ G_1. Q1r+1Q_1^r+1 satisfies Q1r+1≤(a) Q_1^r+1 (a)≤ |μQ0r+(1−μ)g0r(r)| \> μ Q_0^r+(1-μ)g_0^r( λ^r) ≤(b) (b)≤ μQ0r+(1−μ)|g0r(r)| \>μ Q_0^r+(1-μ) g_0^r( λ^r) ≤(c) (c)≤ μG1+(1−μ)G1 \>μ G_1+(1-μ) G_1 = = G1, \> G_1, (37) where inequalities (a) and (b) hold due to eqn. (32) and the triangle inequality, respectively. Inequality (c) results from Assumption 4 and Lemma 3. We use similar steps to derive Qnr≤G2Q_n^r≤ G_2, n∈n , r∈ℛr . ∎ Lemma 5. The drift-plus-penalty term is bounded by Δr+Vgobjr(r) ^r+Vg_obj^r( λ^r) ≤ ≤ 12(1−μ)2(G1+NG2)+μ(1−μ)(Q0rg0r(r) \> 12(1-μ)^2(G_1+NG_2)+μ(1-μ)(Q_0^rg_0^r( λ^r) +∑n=1NQnrgnr(r))+Vgobjr(r),r∈ℛ. \>+ _n=1^NQ_n^rg_n^r( λ^r))+Vg_obj^r( λ^r), r . (38) Proof. The Lyapunov drift satisfies Δr=12((Q1r+1)2−(Q0r)2)+12∑n=1N((Qnr+1)2−(Qnr)2). ^r= 12 ((Q_1^r+1)^2-(Q_0^r)^2 )+ 12 _n=1^N ((Q_n^r+1)^2-(Q_n^r)^2 ). (39) In particular, (Q1r+1)2−(Q0r)2 (Q_1^r+1)^2-(Q_0^r)^2 =(a) (a)= 12(μ2−1)(Q0r)2+μ(1−μ)Q0rg0r(r) \> 12(μ^2-1)(Q_0^r)^2+μ(1-μ)Q_0^rg_0^r( λ^r) +12(1−μ)2(g0r(r))2 + 12(1-μ)^2(g_0^r( λ^r))^2 ≤(b) (b)≤ μ(1−μ)Q0rg0r(r)+12(1−μ)2G1, \>μ(1-μ)Q_0^rg_0^r( λ^r)+ 12(1-μ)^2G_1, (40) where equality (a) results from eqn. (32). Inequality (b) results from Lemma 3 and the fact that 12(μ2−1)(Q0r)2≤0 12(μ^2-1)(Q_0^r)^2≤ 0. Then, we use similar steps to bound (Qnr+1)2−(Qnr)2(Q_n^r+1)^2-(Q_n^r)^2, n∈n . Finally, we substitute these two terms into the drift-plus-penalty term and obtain the result. ∎ Based on Lemma 5, we present an additional assumption which is widely used in the literature (e.g. [Neelybook, 9687317]). Assumption 5. For any r∈ℛr , let gobj⋆g_obj denote the optimal objective achieved by the decision vector r λ^r. The optimal value of g0rg_0^r is denoted by g0⋆g_0 . For n∈n , the optimal value of gnrg_n^r is denoted by gn⋆g_n . There exists a nonnegative constant C such that for an arbitrary decision vector r λ^r, we have Δr+Vgobjr(r) ^r+Vg_obj^r( λ^r) ≤ ≤ 12(1−μ)2(G1+NG2)+μ(1−μ)(Q0rg0⋆+∑n=1NQnrgn⋆) \> 12(1-μ)^2(G_1+NG_2)+μ(1-μ)(Q_0^rg_0 + _n=1^NQ_n^rg_n ) +Vgobj⋆+C. +Vg_obj +C. (41) Then, we show the constraint violation and performance gap in the following theorems. Theorem 2. Based on Lemma 4, the constraint violation is bounded by 1R∑r=1Rgnr(r)≤(1+1(1−μ)R)G1,if n=0,(1+1(1−μ)R)G2,if n∈. \!\!\! 1R _r=1^Rg_n^r( λ^r)≤ cases (1+ 1(1-μ)R ) G_1,&if $n=0$,\\ (1+ 1(1-μ)R ) G_2,&if $n∈ N$. cases (42) Proof. We first analyze g0r(r)g_0^r( λ^r). Based on eqn. (32), we have (1−μ)g0r(r)≤Q1r+1−Q0r+(1−μ)Q0r. (1-μ)g_0^r( λ^r)≤ Q_1^r+1-Q_0^r+(1-μ)Q_0^r. (43) By performing telescoping sum on both sides and using the fact that Q11=0Q_1^1=0, we have (1−μ)∑r=1Rg0r(r)≤Q1R+1+(1−μ)∑r=1RQ0r. (1-μ) _r=1^Rg_0^r( λ^r)≤ Q_1^R+1+(1-μ) _r=1^RQ_0^r. (44) By dividing (1−μ)R(1-μ)R on both sides and using Lemma 4, we have 1R∑r=1Rg0r(r)≤(1+1(1−μ)R)G1. 1R _r=1^Rg_0^r( λ^r)≤ (1+ 1(1-μ)R ) G_1. (45) We can use similar steps to obtain the result when n∈n . ∎ Theorem 3. Based on Assumption 5 and Lemmas 3 −- 4, the performance gap is bounded by 1R∑r=1R(gobjr(r)−gobj⋆)≤W+CV, 1RΣ _r=1^R (g_obj^r( λ^r)-g_obj )≤ W+CV, (46) where W=12(1−μ)2(G1+NG2)+μ(1−μ)(G1(Tmax−γ)+G2(Emax−δ)N)W= 12(1-μ)^2(G_1+NG_2)+μ(1-μ) ( G_1(T -γ)+ G_2(E -δ)N ). Proof. Based on Assumption 5, we have L(r+1)−L(r)+Vgobjr(r) L(Q^r+1)-L(Q^r)+Vg_obj^r( λ^r) ≤ ≤ 12(1−μ)2(G1+NG2)+μ(1−μ)(Q0rg1⋆+∑n=1NQnrgn⋆) \> 12(1-μ)^2(G_1+NG_2)+μ(1-μ)(Q_0^rg_1 + _n=1^NQ_n^rg_n ) +Vgobj⋆+C +Vg_obj +C ≤(a) (a)≤ W+Vgobj⋆+C, \>W+Vg_obj +C, (47) where inequality (a) results from Lemmas 3 −- 4. Then, we perform telescoping sum on both sides and obtain L(R+1)−L(1)+V∑r=1Rgobjr(r) L(Q^R+1)-L(Q^1)+V _r=1^Rg_obj^r( λ^r) ≤ ≤ R(W+C)+V∑r=1Rgobj⋆. \>R(W+C)+V _r=1^Rg_obj . (48) Since L(R+1)≥0L(Q^R+1)≥ 0 and L(1)=0L(Q^1)=0, we rearrange the inequality and divide both sides by RVRV to obtain the result. ∎ Remark 2. Theorems 2 and 3 suggest that as R approaches infinity, the constraint violation is bounded by constant values, which indicates the stability of our proposed algorithm. The performance gap is also bounded by a constant value. In addition, there is a trade-off between the performance gap and constraint violation. That is, as μ increases, the performance gap decreases (i.e., the convergence rate is improved) while the constraint violation increases (i.e., the total delay and energy consumption on training increases) and vice versa. We will show in Section V-B that setting μ properly can balance the learning performance and efficiency of our proposed ASFL. IV-B RB Allocation Subproblem Note that the RB allocation decision matrix rU^r is independent between training rounds. Hence, given rp^r and r λ^r, we can optimize rU^r in each training round r∈ℛr independently by solving the following optimization problem: minimizer *minimize_ subarraycU^r subarray gobjr(r) g_obj^r(U^r) (49a) subjectto \>to r[Tr(r)]≤γ, _ β^r[T^r(U^r)]≤γ, (49b) βnr[Enr(r)]≤δ,n∈, _ _n^r[E_n^r(U^r)]≤δ, n , (49c) ∑n=1Nun,kr≤1,k∈, Σ _n=1^Nu_n,k^r≤ 1, k , (49d) un,kr∈0,1,n∈,k∈. u_n,k^r∈\0,1\, n ,k . (49e) Constraints (49b) and (49c) are convex with respect to (w.r.t.) rU^r. Problem (49) is an integer programming problem, which can be efficiently solved via a convex optimization tool (e.g., CVXPY [diamond2016cvxpy]). IV-C Transmit Power Allocation Subproblem The transmit power decision vector rp^r is independent between training rounds. Hence, given rU^r and r λ^r, we can optimize rp^r in each training round r∈ℛr independently. Tackling the delay and energy consumption constraints is challenging since they are nonconvex w.r.t. rp^r. Thus, to linearize these constraints, we first introduce auxiliary variables and formulate an equivalent problem. We then derive the solution. We introduce three non-negative auxiliary variables (i.e., γ1r _1^r, γ2r _2^r, and γ3r _3^r) to bound the delay in those three stages. Similarly, we introduce three non-negative auxiliary variables (i.e., δ1r _1^r, δ2r _2^r, and δ3r _3^r) to bound EnMS,rE_n^MS,r, EnUP,rE_n^UP,r, and EnUP,rE_n^UP,r, respectively. Thus, given rU^r and r λ^r, the transmit power allocation subproblem is as follows: minimizer,γ1r,γ2r,γ3r,δ1r,δ2r,δ3r *minimize_ subarraycp^r, _1^r, _2^r, _3^r,\\ _1^r, _2^r, _3^r subarray gobjr(r) g_obj^r(p^r) (50a) subjectto \>to TnS1,r(pnr)≤γ1r,n∈, T_n^S1,r(p_n^r)≤ _1^r, n , (50b) TnS2,r(pnr)≤γ2r,n∈, T_n^S2,r(p_n^r)≤ _2^r, n , (50c) βnr[TnS3,r(pnr)]≤γ3r,n∈, _ _n^r[T_n^S3,r(p_n^r)]≤ _3^r, n , (50d) EnMS,r(pnr)≤δ1r,n∈, E_n^MS,r(p_n^r)≤ _1^r, n , (50e) EnUP,r(pnr)≤δ2r,n∈, E_n^UP,r(p_n^r)≤ _2^r, n , (50f) βnr[EnCBP,r(pnr)]≤δ3r,n∈, _ _n^r[E_n^CBP,r(p_n^r)]≤ _3^r, n , (50g) γ1r+γ2r+γ3r≤γ, _1^r+ _2^r+ _3^r≤γ, (50h) δ1r+δ2r+δ3r≤δ−maxn∈EnFP,r, _1^r+ _2^r+ _3^r≤δ- _n \E_n^FP,r\, (50i) 0≤pnr≤Pmax,n∈. 0≤ p_n^r≤ P^max, n . (50j) To solve problem (50), we introduce an iterative algorithm to optimize rp^r and γ1r,γ2r,γ3r,δ1r,δ2r,δ3r\ _1^r, _2^r, _3^r, _1^r, _2^r, _3^r\ alternately. First, given γ1r,γ2r,γ3r,δ1r,δ2r,δ3r\ _1^r, _2^r, _3^r, _1^r, _2^r, _3^r\, we determine the feasible domain w.r.t. rp^r based on constraints (50b) −- (50g). From constraint (50b), pnrp_n^r satisfies pnr≥BN0(2(r−1−r)⊺γ1rB∑k=1Kun,kr−1)|hnr|2. p_n^r≥ BN_0 (2 ( λ^r-1- λ^r) ψ _1^rB _k=1^Ku_n,k^r-1 ) h_n^r ^2. (51) From constraint (50c), pnrp_n^r satisfies pnr≥BN0(2fn∑m=1M−1(λmr−λm+1r)qm(γ2rfn−κcDnB(r)⊺FP)∑k=1Kun,kr−1)|hnr|2. \!p_n^r≥ BN_0 (2 f_n _m=1^M-1( _m^r- _m+1^r)q_m( _2^rf_n-κ^cD_nB( λ^r) v^FP) _k=1^Ku_n,k^r\!-\!1 ) h_n^r ^2. (52) From constraint (50d), pnrp_n^r satisfies pnr≤1ln((ω1,nr−γ3rω1,ir|hnr|[exp(−αBN0∑k=1Kun,kr|hnr|2)])), p_n^r≤ 1 ( _1,n^r- _3^r _1,i^rE_ h_n^r [ (- α BN_0 _k=1^Ku_n,k^r h_n^r ^2 .) ] ), (53) where ω1,nr= _1,n^r= κsDn(M−r)⊺(FP+BP)fs+κcDn(r)⊺BPfn \> κ^sD_n( 1_M- λ^r) (v^FP+v^BP )f^s+ κ^cD_n( λ^r) v^BPf_n +∑m=1M−1(λmr−λm+1r)ψm+1BDNlog2(1+ps|hnr|2BDNN0). + _m=1^M-1( _m^r- _m+1^r) _m+1B^DN _2(1+ p^s h_n^r ^2B^DNN_0). (54) For constraint (50e), it is equivalent to ln(1+pnr|hnr|2BN0)pnr≥(r−1−r)⊺ln(2)δ1rB∑k=1Kun,kr (1+ p_n^r h_n^r ^2BN_0 .)p_n^r≥ ( λ^r-1- λ^r) ψ 2 _1^rB _k=1^Ku_n,k^r. We use the Taylor series to expand the numerator on the left-hand side of this inequality. In particular, pnrp_n^r satisfies pnr≤2BN0|hnr|2(1−(r−1−r)⊺N0ln(2)δ1r∑k=1Kun,kr|hnr|2). p_n^r≤ 2BN_0 h_n^r ^2 (1- ( λ^r-1- λ^r) ψN_0 2 _1^r _k=1^Ku_n,k^r h_n^r ^2 ). (55) Similarly, from constraint (50f), pnrp_n^r satisfies pnr≤2BN0|hnr|2(1−DnN0ln(2)∑m=1M−1(λm−λm+1)qmδ2r∑k=1Kun,kr|hnr|2). \!\!\!p_n^r≤ 2BN_0 h_n^r ^2 (1\!-\! D_nN_0 2 _m=1^M-1( _m\!-\! _m+1)q_m _2^r _k=1^Ku_n,k^r h_n^r ^2 ). (56) From constraint (50g), pnrp_n^r satisfies pnr≤1ln(Dn(r)⊺BPϕκc(fn)2∑kun,kr−δ3r|hnr|[exp(−αBN0∑kun,kr|hnr|2)]Dn(r)⊺BPϕκc(fn)2). p_n^r≤ 1 ( D_n( λ^r) v^BPφκ^c(f_n)^2 _ku_n,k^r\>-\> _3^rE_ h_n^r [exp (- α BN_0 _ku_n,k^r h_n^r ^2 ) ]D_n( λ^r) v^BPφκ^c(f_n)^2 ). (57) Let pnc1,r,pnc2,r,pnc3,r,pnc4,r,pnc5,r,pnc6,rn∈\p_n^c1,r,p_n^c2,r,p_n^c3,r,p_n^c4,r,p_n^c5,r,p_n^c6,r\_n denote the right-hand side of inequalities (51) −- (57), respectively. Given γ1r,γ2r,γ3r,δ1r,δ2r,δ3r\ _1^r, _2^r, _3^r, _1^r, _2^r, _3^r\, problem (50) can be reformulated as minimizer *minimize_ subarraycp^r subarray gobjr(r) g_obj^r(p^r) (58a) subjectto \>to maxpnc1,r,pnc2,r≤pnr≤minPmax,pnc3,r, \p_n^c1,r,p_n^c2,r\≤ p_n^r≤ \P^max,p_n^c3,r, pnc4,r,pnc5,r,pnc6,r,n∈. p_n^c4,r,p_n^c5,r,p_n^c6,r\, n . (58b) The solution to problem (58) is given by Theorem 4. Theorem 4. Given γ1r,γ2r,γ3r,δ1r,δ2r,δ3r\ _1^r, _2^r, _3^r, _1^r, _2^r, _3^r\, for n∈n , the solution to problem (58) is obtained as pnr=maxpnc1,r,pnc2,r,C1,1ln((−ω3,nr2ω2,nr)),C2,minPmax,pnc3,r,pnc4,r,pnc5,r,pnc6,r,otherwise,\!\!\!p_n^r= cases \p_n^c1,r,p_n^c2,r\,&\!C1,\\ 1 (- _3,n^r2 _2,n^r),&\!C2,\\ \P^max,p_n^c3,r,p_n^c4,r,p_n^c5,r,p_n^c6,r\,&\!otherwise, cases (59) where ω2,nr=|hnr|[exp(−αBN0∑k=1Kun,kr|hnr|2)]2‖n,rs,r‖2 _2,n^r=E_ h_n^r [ (- α BN_0 _k=1^Ku_n,k^r h_n^r ^2 .) ]^2\|w_n, λ^r^s,r\|^2, ω3,nr=−2|hnr|[exp(−αBN0∑k=1Kun,kr|hnr|2)]⟨¯rs,r,n,rs,r⟩ _3,n^r=-2E_ h_n^r [ (- α BN_0 _k=1^Ku_n,k^r h_n^r ^2 .) ] w_ λ^r^s,r,w_n, λ^r^s,r , n∈n , C1 is the condition where ω3,nr<0 _3,n^r<0 and 1/ln((−ω3,nr2ω2,nr))<maxpnc1,r,pnc2,r1/ (- _3,n^r2 _2,n^r)< \p_n^c1,r,p_n^c2,r\. C2 is the condition where ω3,nr<0 _3,n^r<0 and maxpnc1,r,pnc2,r≤1/ln((−ω3,nr2ω2,nr))≤minPmax,pnc3,r,pnc4,r,pnc5,r,pnc6,r \p_n^c1,r,p_n^c2,r\≤ 1/ (- _3,n^r2 _2,n^r)≤ \P^max,p_n^c3,r,p_n^c4,r,p_n^c5,r,p_n^c6,r\. Proof. We first determine the critical point of gobjr(r)g_obj^r(p^r) as 1/ln((−ω3,nr2ω2,nr))1/ (- _3,n^r2 _2,n^r). Under C1, gobjr(r)g_obj^r(p^r) is increasing and is minimized at the lower bound of the domain. Under C2, gobjr(r)g_obj^r(p^r) is minimized at the critical point. Otherwise, gobjr(r)g_obj^r(p^r) is decreasing and is minimized at the upper bound of the domain. ∎ Algorithm 1 Iterative Algorithm for Problem (58) 1: Input: Iteration index τ1:=0 _1:=0, power objective tolerance εp _p. 2: Randomly initialize γ1r,0,γ2r,0,γ3r,0,δ1r,0,δ2r,0,δ3r,0\ _1^r,0, _2^r,0, _3^r,0, _1^r,0, _2^r,0, _3^r,0\. 3: While τ1=0or|gp(r,τ1)−gp(r,τ1−1)|>εp _1=0\>or\> g_p(p^r, _1)-g_p(p^r, _1-1) > _p do 4: τ1:=τ1+1 _1:= _1+1. 5: Update r,τ1p^r, _1 based on eqn. (59). 6: Update γ1r,τ1,γ2r,τ1,γ3r,τ1,δ1r,τ1,δ2r,τ1,δ3r,τ1\ _1^r, _1, _2^r, _1, _3^r, _1, _1^r, _1, _2^r, _1, _3^r, _1\. 7: End while 8: Output: r,τ1p^r, _1. Second, given rp^r, we update γ1r,γ2r,γ3r,δ1r,δ2r,δ3r\ _1^r, _2^r, _3^r, _1^r, _2^r, _3^r\ as γ1r=maxn∈TnS1,r(pnr) _1^r= _n \T_n^S1,r(p_n^r)\, γ2r=maxn∈TnS2,r(pnr) _2^r= _n \T_n^S2,r(p_n^r)\, γ3r=maxn∈βnr[TnS3,r(pnr)] _3^r= _n \E_ _n^r[T_n^S3,r(p_n^r)]\, δ1r=maxn∈EnMS,r(pnr) _1^r= _n \E_n^MS,r(p_n^r)\, δ2r=maxn∈EnUP,r(pnr) _2^r= _n \E_n^UP,r(p_n^r)\, and δ3r=maxn∈βnr[EnCBP,r(pnr)] _3^r= _n \E_ _n^r[E_n^CBP,r(p_n^r)]\, respectively. This iterative algorithm is presented in Algorithm 1. In summary, to solve problem (30), we update each decision variable by solving problems (34), (49), and (58) alternately in each training round. Our proposed OOE-BCD algorithm is shown in Algorithm 2. IV-D Computation Complexity Analysis To determine the computation complexity of our proposed OOE-BCD algorithm in each training round, we analyze each of the three subproblems. For the model splitting subproblem, since there are at most M feasible solutions, the computation complexity is linear in the number of model splitting decisions, which is (M)O(M). For the RB allocation subproblem, the computation complexity is (NK)O(N^K). For the transmit power allocation subproblem, since we need to determine the transmit power for each client, the computation complexity is (Nlog2(1εp))O(N _2( 1 _p)). Therefore, the overall computation complexity of our proposed OOE-BCD algorithm in one training round is (log2(1εo)(M+NK+Nlog2(1εp))O( _2( 1 _o)(M+N^K+N _2( 1 _p)). Algorithm 2 OOE-BCD Algorithm for Problem (30) 1: Input: Iteration index τ2:=0 _2:=0, objective tolerance εo _o. 2: Randomly initialize r,0,r,0\U^r,0,p^r,0\. 3: While τ2=0or|gobjr(r,τ2,r,τ2,r,τ2)−gobjr(r,τ2−1,r,τ2−1,r,τ2−1)|>εo _2=0\>or\> g_obj^r(U^r, _2,p^r, _2, λ^r, _2)-g_obj^r(U^r, _2-1,p^r, _2-1, λ^r, _2-1) > _o do 4: τ2:=τ2+1 _2:= _2+1. 5: Update r,τ2 λ^r, _2 by solving problem (34). 6: Update r,τ2U^r, _2 by solving problem (49). 7: Update r,τ2:=r,τ1p^r, _2:=p^r, _1 based on Alg. 1. 8: End while 9: Output: r,⋆,r,⋆,r,⋆:=r,τ2,r,τ2,r,τ2\U^r, ,p^r, , λ^r, \:=\U^r, _2,p^r, _2, λ^r, _2\. Table I: List of key simulation parameters Para. Value Para. Value Para. Value B 1 MHz PmaxP 1.5 W K 8 BDNB^DN 8 MHz κsκ^s 132 132 cycles/FLOP psp^s 5 W N0N_0 −-173 dBm/Hz κcκ^c 116 116 cycles/FLOP ι 0.05 α 1 fsf^s 101010^10 cycles/s V 10 γ 20 s δ 0.50.5 J ϕφ 10−2810^-28 V Performance Evaluation V-A Simulation Setup We consider a scenario where clients are randomly located in a circular coverage area with a radius of 500 m. The central server is located at the center of the area. The channel coefficient hnrh_n^r is obtained as hnr=θnrχnrh_n^r= _n^r _n^r, where θnr=−30−40log10(dnr) _n^r=-30-40 _10(d_n^r) denotes the path loss coefficient (in dB) [10274134] and dnrd_n^r denotes the distance between client n∈n and the central server in the r-th training round. χnr∼(0,1) _n^r (0,1) denotes the Rayleigh fading coefficient. The CPU frequency of each client n∈n follows fn∼[1GHz, 1.6GHz]f_n [1\,GHz,\,1.6\,GHz] [lin2023efficient]. We set εo=εp=0.01 _o= _p=0.01. Unless stated otherwise, we set μ=0.5μ=0.5. A list of key simulation parameters is presented in Table I. We use CIFAR-10 and CIFAR-100 [krizhevsky2009learning] as the datasets. We adopt VGG-19 [simonyan2014very] and ResNet-50 [he2016deep] as the models with 19 and 50 layers, respectively. The model is allowed to be split only before the convolutional layer or the fully-connected layer. The number of training rounds R, the learning rate η, and the batch size are set to 200, 0.0001, and 64, respectively. To study the effect of non-independent and identically distributed (non-IID) data partitioning, similar to other works in FL [chen2020fedbe, 9835537], we use the Dirichlet distribution Dir(ρ)Dir(ρ) with parameter ρ to distribute the data with different labels across clients. Let NcN_c denote the number of classes of a dataset. The Dirichlet distribution Dir(ρ)Dir(ρ) generates the probability vector n=(qn1,qn2,…,qnNc)q_n=(q^1_n,q^2_n,…,q^N_c_n), where qniq^i_n represents the fraction of data samples from class i assigned to client n, with ∑n=1Ncqni=1 _n=1^N_cq^i_n=1. The probability density function of the Dirichlet distribution is given by f(n;ρ)=Γ(Ncρ)(Γ(ρ))Nc∏n=1Nc(qni)ρ−1f(q_n;ρ)= (N_cρ) ( (ρ) )^N_c _n=1^N_c(q^i_n)^ρ-1, where Γ(⋅) (·) is the gamma function. When ρ decreases, the data heterogeneity across clients increases and vice versa. Unless stated otherwise, we set N=10N=10 and ρ=10ρ=10. To ensure fairness, we allocate the same number of training samples to each client’s local dataset. We compare the performance of our proposed ASFL with the following five baseline schemes: • FedAvg [mcmahan2017communication]: Clients train their local models simultaneously. We use the same RB and transmit power allocation decisions as ASFL. • SL [vepakomma2018split]: Clients perform training sequentially. We use the same RB and transmit power allocation decisions as ASFL and randomly make the model splitting decision. • SFL [thapa2022splitfed]: Clients train their client-side models in parallel and send the updated client-side models to the central server to perform model aggregation. We randomly make the model splitting decision. Then, we use the same RB and transmit power allocation decisions as ASFL. • ACC-SFL [10304624]: The model splitting and RB allocation decisions are determined using its proposed optimization algorithm. We randomly choose each client’s transmit power. • EPSL [lin2023efficient]: The model splitting and resource allocation decisions are made using its proposed optimization algorithm. We introduce evaluation metrics for our experiments as follows: • Average testing accuracy: This is the ratio of the total number of correct predictions to the number of testing samples. It quantifies how well the trained model performs on downstream tasks. • Total delay on training: This is the end-to-end model training latency. • Total energy consumption on training: This metric captures the total communication and computation energy consumption of all clients. It corresponds to the total battery usage of end devices. • Average packet error rate: This is the ratio of the number of packets with errors to the total number of transmitted packets. It captures the communication reliability over wireless links, which strongly affects the success rate of model update. V-B Experiments Table I: Comparison of the average testing accuracy between different baseline schemes. Base Model Baseline CIFAR-10 (%) CIFAR-100 (%) VGG-19 ASFL 89.20 62.10 FedAvg [mcmahan2017communication] 89.69 61.35 SL [vepakomma2018split] 83.41 58.30 SFL [thapa2022splitfed] 88.77 62.43 ACC-SFL [10304624] 88.02 61.96 EPSL [lin2023efficient] 87.59 61.58 ResNet-50 ASFL 92.52 66.19 FedAvg [mcmahan2017communication] 90.91 65.72 SL [vepakomma2018split] 88.81 62.27 SFL [thapa2022splitfed] 91.07 65.92 ACC-SFL [10304624] 90.94 64.08 EPSL [lin2023efficient] 89.47 63.46 (a) Comparison of the average testing accuracy versus (a) total delay and (b) total energy consumption on training. Figure 5: Comparison of the average packet error rate and average long-term model discrepancies. Figure 6: Effect of the degree of data heterogeneity ρ on the average testing accuracy. Figure 7: Effect of the number of clients on the average testing accuracy. Figure 8: Comparison of the total delay on training and model splitting. V-B1 Comparison of the Learning Performance and Efficiency In Table I, we compare the average testing accuracy on CIFAR-10 and CIFAR-100 datasets. Results show that our proposed ASFL outperforms the baseline schemes in most cases. In particular, when testing under ResNet-50 with CIFAR-100 dataset, our proposed ASFL outperforms FedAvg, SL, SFL, ACC-SFL, and EPSL in terms of the average testing accuracy by 1.61%, 3.71%, 1.45%, 1.58%, and 3.05%, respectively. This validates the effectiveness of ASFL in improving the learning performance. To compare the learning efficiency, we compare the average testing accuracy using VGG-19 and CIFAR-100 under the first 1500 s and 200 J in Fig. 5(a). Results show that our proposed ASFL converges faster and achieves a higher average testing accuracy and a lower total delay and energy consumption when compared with the baseline schemes. In particular, our proposed ASFL outperforms FedAvg and SL in both the total delay and energy consumption on training. In addition, to achieve an average testing accuracy of 0.5, our proposed ASFL achieves a total delay that is 51%, 69%, and 75% lower than ACC-SFL, SFL, and EPSL, respectively. Our proposed ASFL also reduces the energy consumption by 80%, 56%, and 74% when compared with ACC-SFL, SFL, and EPSL, respectively. It shows that our proposed ASFL improves the learning efficiency. V-B2 Comparison of the Average Packet Error Rate and Average Long-Term Model Discrepancies To further evaluate the learning performance, we compare the average packet error rate of all clients of our proposed ASFL with the baseline schemes using VGG-19 and CIFAR-100 in Fig. 6. Results show that our proposed ASFL achieves a lower average packet error rate when compared with ACC-SFL and EPSL. Note that our proposed ASFL achieves the same packet error rate as FedAvg, SL, and SFL since they use the same RB and transmit power allocation decisions. In addition, our proposed algorithm achieves a lower average long-term model discrepancies when compared with the baseline schemes. The lower average packet error rates and reduced average long-term model discrepancies contribute to the better learning performance of our proposed ASFL. V-B3 Effect of Data Heterogeneity We evaluate the robustness of the proposed ASFL algorithm under different levels of data heterogeneity across clients by comparing the average testing accuracy across different values of ρ using VGG-19 and CIFAR-100. A smaller ρ corresponds to a higher degree of non-IID data distribution. To control the degree of data heterogeneity across clients, we set ρ to be 0.1, 1, 10, and 100. As shown in Fig. 6, ASFL outperforms the baseline schemes in terms of the average testing accuracy in most cases. These results highlight the robustness of ASFL to different degrees of non-IID client data distributions. V-B4 Scalability Analysis We investigate the scalability of our proposed ASFL by setting the number of clients to 10, 20, 30, 40, and 50, respectively. We compare the average testing accuracy achieved by our proposed ASFL framework with the baseline schemes using VGG-19 and CIFAR-100 in Fig. 8. It can be observed that our proposed AFSL framework constantly outperforms the baseline schemes, validating the scalability of our proposed ASFL. V-B5 Overhead of Adaptive Model Splitting In Fig. 8, we present the comparison between the total delay on training and the delay incurred by adaptive model splitting using VGG-19 and CIFAR-100. It can be observed that the overhead introduced by switching the model splitting points contributes only a small fraction of the overall training delay. In particular, at the end of the 200th training round, the total delay of adaptive model splitting accounts for 20.86% of the total delay on training. Furthermore, the model splitting points are switched almost every round in the first 150 training rounds. This is necessary to maintain the convergence and adapt to the dynamic wireless channel conditions. Thus, the corresponding delay of model splitting increases. Then, as training progresses and the model begins to converge, the need for splitting point switching diminishes, resulting in a significantly reduced frequency of model splitting switching in the subsequent stages. Therefore, the delay of model splitting gradually stabilizes. (a) Comparison of (a) average testing accuracy and (b) total delay and energy consumption on training. V-B6 Experiment on the Real-world Environment To show the performance of our proposed ASFL on real-world scenarios, we use RENEW/FDD Massive MIMO dataset [8368089], which measured channel state information (CSI) collected in outdoor environments to conduct the experiments. Since these measurements are taken over real radio links, they capture the impact of fading and interference on the channel quality. For each client in our experiments, we calculate the channel gain by selecting a specific transmit–receive antenna pair from the measured CSI tensors in the dataset. In Fig. 10(a), we present the comparison of the average testing accuracy, total delay on training, and total energy consumption on training using VGG-19 and CIFAR-100. We can observe that our proposed ASFL outperforms the baseline schemes, showcasing the effectiveness of ASFL in improving the learning performance. In addition, our proposed ASFL achieves the lowest total delay and energy consumption on training when compared with the baseline schemes, indicating the effectiveness of ASFL in improving the learning efficiency. (b) Effect of the adaptive model splitting on the average testing accuracy versus (a) total delay and (b) total energy consumption on training. (c) Effect of the RB and transmit power allocation on the average testing accuracy versus (a) total delay and (b) total energy consumption on training. V-B7 Effect of Adaptive Model Splitting We also consider a baseline scheme (SFL-λm _m) by pre-splitting the model at the m-th layer before training and optimizing other decision variables. We choose m=2m=2 and 33 since they can achieve higher average testing accuracies. Fig. 10(b) shows that using VGG-19 and CIFAR-100, our proposed ASFL converges faster than SFL-λ3 _3 and achieves an average testing accuracy that is 0.076 higher than SFL-λ3 _3. Our proposed ASFL also reduces the total energy consumption in early training rounds. For example, to achieve an average testing accuracy of 0.5, our proposed ASFL achieves a total energy consumption on training that is 17% and 23% lower than SFL-λ2 _2 and SFL-λ3 _3, respectively. It indicates the effectiveness of adaptive model splitting. V-B8 Effect of the RB Allocation and Transmit Power Allocation We consider three other baseline schemes: (a) ASFL-pmax: we set clients’ transmit power to the maximum value and optimize other decision variables, (b) ASFL-prd: we randomly set clients’ transmit power and optimize other decision variables, and (c) ASFL-RBrd: we randomly allocate RBs to clients and optimize other decision variables. Fig. 10(c) shows that using VGG-19 and CIFAR-100, our proposed ASFL outperforms ASFL-RBrd in both the total delay and energy consumption on training. In addition, to achieve an average testing accuracy of 0.55, ASFL achieves a total delay on training that is 50% and 61% lower than ASFL-pmax and ASFL-prd, respectively. ASFL also reduces the energy consumption by 63% and 41% when compared with ASFL-pmax and ASFL-prd, respectively. (d) Effect of μ on the (a) average testing accuracy, (b) total delay, and (c) total energy consumption on training. V-B9 Effect of the Tunable Parameter μ Fig. 10(d) shows the effect of μ on the average testing accuracy, total delay and energy consumption of 100 training rounds using VGG-19 and CIFAR-100. When μ varies from 0.1 to 0.9, the average testing accuracy increases with an increase in the total delay and energy consumption. This is because as μ increases, the corresponding weights on constraints (31b) and (31c) become smaller. Constraints (31b) and (31c) have less impact and become larger when solving problem (34). It indicates the trade-off of tuning μ on the learning performance and efficiency. VI Conclusion In this paper, we proposed an ASFL framework over wireless networks. It enables adaptive model splitting and provides efficient resource allocation during training. We theoretically analyzed the convergence rate of our proposed ASFL framework. We designed an OOE-BCD algorithm to adaptively determine the model splitting and resource allocation decisions. Experimental results showed that when compared with five baseline schemes, our proposed ASFL framework converged faster and reduced the total delay and energy consumption on training by up to 75% and 80%, respectively. We also showed the robustness of our proposed algorithm to different degrees of data heterogeneity across clients’ local datasets. For future work, we plan to design ASFL with heterogeneous model splitting to further improve the learning efficiency under heterogeneous hardware configurations across clients. References