Paper deep dive
CTP-FL: Common-Trajectory Gradient Prediction for Federated Learning
Junkang Liu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Communication-efficient federated optimization commonly spends several gradient evaluations between server updates. Existing local-update methods use this computation to advance an independent model on each client. Under heterogeneous data, however, these models evaluate gradients at different locations, making the aggregated update difficult to interpret as a gradient of the global objective. We study an alternative use of the same computation budget: \emph{evaluate the global objective along a shared, predicted path}. We propose Common-Trajectory Predictive Federated Learning (\texttt{CTP-FL}). At each round, all clients construct the same sequence of query points from the current global model and the previous aggregated direction, evaluate $K$ stochastic gradients along this sequence, and upload their average. The server then performs a single global update. Thus, \texttt{CTP-FL} uses $K$ mini-batch gradients per client and one model-sized vector in each communication direction, matching the per-round computation and communication of full-participation FedAvg-M. Shared query points make the aggregated direction an unbiased estimator of the average \emph{global} gradient along the predicted path. The remaining discrepancy from the gradient at the current model is controlled by the path length, without assuming bounded client-gradient dissimilarity or bounded gradients. For smooth non-convex objectives, we establish an $\mathcal{O}\!\left( \sqrt{L\Delta\sigma^2/(NKR)}+L\Delta/R \right)$ average-stationarity bound under full participation. The analysis isolates a testable trade-off: extending the prediction path provides more forward-looking gradient information but increases its displacement bias.
Tags
Links
- Source: https://arxiv.org/abs/2609.35130v1
- Canonical: https://arxiv.org/abs/2609.35130v1
Trouble viewing inline? Open PDF directly →
Full Text
51,793 characters extracted from source content.
Expand or collapse full text
CTP-FL: Common-Trajectory Gradient Prediction for Federated Learning Junkang LiuTianjin University Email: junkangliukk@gmail.com Abstract Communication-efficient federated optimization commonly spends several gradient evaluations between server updates. Existing local-update methods use this computation to advance an independent model on each client. Under heterogeneous data, however, these models evaluate gradients at different locations, making the aggregated update difficult to interpret as a gradient of the global objective. We study an alternative use of the same computation budget: evaluate the global objective along a shared, predicted path. We propose Common-Trajectory Predictive Federated Learning (CTP-FL). At each round, all clients construct the same sequence of query points from the current global model and the previous aggregated direction, evaluate K stochastic gradients along this sequence, and upload their average. The server then performs a single global update. Thus, CTP-FL uses K mini-batch gradients per client and one model-sized vector in each communication direction, matching the per-round computation and communication of full-participation FedAvg-M. Shared query points make the aggregated direction an unbiased estimator of the average global gradient along the predicted path. The remaining discrepancy from the gradient at the current model is controlled by the path length, without assuming bounded client-gradient dissimilarity or bounded gradients. For smooth non-convex objectives, we establish an (LΔσ2/(NKR)+LΔ/R)O\! ( L σ^2/(NKR)+L /R ) average-stationarity bound under full participation. The analysis isolates a testable trade-off: extending the prediction path provides more forward-looking gradient information but increases its displacement bias. 1 Introduction Federated learning trains a shared model from decentralized data while limiting communication between clients and a server. Federated Averaging (FedAvg) addresses communication cost by performing multiple client updates before each aggregation (McMahan et al., 2017). This structure has shaped a large body of work on local optimization (Stich, 2019). Yet the role of the computation performed between communications deserves closer examination: K local gradient evaluations need not be used to produce K independent client trajectories. Where does the difficulty arise? Let F()=N−1∑i=1NFi()F( x)=N^-1 _i=1^NF_i( x). In a local-update method, every client begins a round at the same r x^r, but its later gradients are evaluated at client-specific points ir,k x_i^r,k. Even when all clients participate, the average gradient at these different points is generally not a gradient of F at any single common point. This distinction is particularly relevant under heterogeneous client objectives: differences in local gradients alter the trajectories, which in turn alter where subsequent gradients are measured. SCAFFOLD addresses this client drift with control variates (Karimireddy et al., 2020). FedAvg-M instead introduces a global momentum direction into local updates and proves full-participation convergence without a bounded data-heterogeneity assumption (Cheng et al., 2024). These results establish that heterogeneity can be handled without assuming similar client gradients. They also motivate a different question: can the gradient locations themselves be coordinated without increasing per-round communication? A shared path in place of independent trajectories. We propose Common-Trajectory Predictive Federated Learning (CTP-FL). At round r, the server model r x^r and previous aggregate direction r v^r define K shared query points kr=r−tkr,tk∈[0,h],k=0,…,K−1. z_k^r= x^r-t_k v^r, t_k∈[0,h], k=0,…,K-1. (1) Every client evaluates one mini-batch gradient at each kr z_k^r and uploads only their average. The server averages these client vectors to obtain r+1 v^r+1 and updates r+1=r−γr+1 x^r+1= x^r-γ v^r+1. The previous direction can be recovered from consecutive global models, so it need not be transmitted as an additional vector. The distinction from local SGD and FedAvg-M is structural. CTP-FL does not execute K local model updates: the K computations probe a common predicted path before one server update. It therefore gives up the adaptive local trajectories that can make local training effective, while ensuring that clients evaluate the same sequence of model parameters. The prediction path is also different from Lookahead’s interpolation between fast and slow weights (Zhang et al., 2019): here it specifies where distributed gradients are measured, rather than combining locally updated weights. What does the shared path buy? Conditioned on the history of round r, independent unbiased mini-batch gradients give [r+1∣ℱr]=1K∑k=0K−1∇F(kr).E[ v^r+1 _r]= 1K _k=0^K-1∇ F( z_k^r). (2) This identity holds for arbitrarily different client objectives under full participation: client-gradient differences cancel at every common query point. The direction is not, in general, an unbiased estimate of ∇F(r)∇ F( x^r). If each FiF_i is L-smooth, its displacement bias satisfies ‖1K∑k=0K−1∇F(kr)−∇F(r)‖≤LK∑k=0K−1tk‖r‖. \| 1K _k=0^K-1∇ F( z_k^r)-∇ F( x^r) \|≤ LK _k=0^K-1t_k\| v^r\|. (3) For the uniform path from 00 to h, the right-hand side equals Lh‖r‖/2Lh\| v^r\|/2 as an upper bound. Equations equation 2 and equation 3 expose the method’s central trade-off: h determines how far ahead the clients measure the global gradient field, while also controlling the bias relative to the current iterate. Guarantee and scope. Under L-smoothness, unbiased stochastic gradients with variance at most σ2σ^2, and full client participation, we prove that the choice h=γh=γ with an appropriate server stepsize satisfies 1R∑r=0R−1‖∇F(r)‖2=(LΔσ2NKR+LΔR), 1R _r=0^R-1E\|∇ F( x^r)\|^2=O\! ( L σ^2NKR+ L R ), (4) where Δ=F(0)−infF() =F( x^0)- _ xF( x). The proof requires neither bounded gradients nor bounded client-gradient dissimilarity. This rate is of the same order as the full-participation FedAvg-M guarantee; it does not establish a theoretical acceleration over FedAvg-M. Importantly, the cancellation in equation 2 relies on full participation. With client sampling, an additional sampling-variance term appears, and the present algorithm does not provide a heterogeneity-independent partial-participation guarantee. Our contributions are: • We introduce a communication-neutral allocation of K client gradient evaluations to a shared predicted trajectory, with one model-sized uplink and downlink vector per client per round. • We characterize the resulting direction as the average global gradient along the shared trajectory and identify its displacement bias separately from stochastic-gradient noise. • We prove the non-convex full-participation guarantee in equation 4 without bounded gradients or a bounded data-heterogeneity assumption. 2 Related Work Local updates and communication efficiency. FedAvg reduces communication by allowing clients to take multiple optimization steps before model averaging (McMahan et al., 2017). Local SGD provides a broader distributed-optimization perspective on this strategy (Stich, 2019). Its benefit, however, depends on how the saved communication interacts with the errors introduced by independent local trajectories. Woodworth et al. (Woodworth et al., 2020) show that local SGD and mini-batch SGD do not uniformly dominate one another across objective classes. This comparison is especially relevant to our setting: CTP-FL allocates K gradient evaluations per client to K shared query points, rather than to K successive local model updates. When its prediction span is zero, CTP-FL reduces to a synchronized mini-batch-gradient baseline. Thus, any benefit of prediction must be measured against both this baseline and strong local-update methods at matched computation and communication budgets. Data heterogeneity and client drift. Under heterogeneous client objectives, local models can follow different trajectories even when they start from the same global model. FedProx regularizes local optimization around the current server model (Li et al., 2020), while SCAFFOLD uses client and server control variates to correct client drift (Karimireddy et al., 2020). Mime combines control variates with server-side optimizer statistics to approximate the behavior of centralized optimizers during local training (Karimireddy et al., 2021). FedAvg-M injects a global momentum direction into local updates and establishes full-participation convergence without assuming bounded client-gradient dissimilarity; it also develops momentum variants of SCAFFOLD for partial participation (Cheng et al., 2024). Hence, the absence of a bounded heterogeneity assumption is not, by itself, a distinguishing claim for CTP-FL. Our distinction is where gradients are evaluated: at local, client-dependent iterates in these methods, and at the same predicted positions on every client in CTP-FL. Under full participation, averaging the latter gradients gives an estimator of the global gradient averaged along the shared path. The remaining bias is governed by the path’s displacement from the current model. Prediction and shared gradient evaluation. Prediction has also been used to organize optimization across multiple steps. Lookahead maintains fast and slow weights and interpolates between them after several optimizer updates (Zhang et al., 2019). CTP-FL uses the previous aggregated direction for a different purpose: it specifies the locations of the current round’s gradient evaluations, without advancing separate client models along that path. The resulting direction is generally biased relative to ∇F(r)∇ F( x^r), even though it is an unbiased estimate of the average global gradient at the queried positions. Our analysis makes this displacement bias explicit and bounds it using smoothness and the prediction span. The present guarantee applies to full participation; subsampling clients introduces additional sampling variance that the basic CTP-FL update does not remove. 3 The Proposed Algorithm 3.1 Problem Setup FL seeks to learn a global model collaboratively over clients by minimizing the population risk: fi():=ξi∼i[Fi(,ξi)],f():=1N∑i=1Nfi().f_i( x):=E_ _i _i [F_i( x; _i) ], f( x):= 1N _i=1^Nf_i( x). (5) The function FiF_i is the loss function on client i. ξi∼i[⋅]E_ _i _i[·] denotes conditional expectation with respect to the sample ξi _i. N is the number of clients, and x is global model. 3.2 From Local Trajectories to a Shared Gradient Path Consider the federated objective min∈ℝdF()≔1N∑i=1NFi(), _ x ^dF( x) 1N _i=1^NF_i( x), (6) where FiF_i is the objective of client i. A communication round typically allows each client to compute K stochastic gradients before uploading one model-sized vector. In local-update methods, these gradients are evaluated along client-specific iterates ir,k x_i^r,k. Consequently, even under full participation, their aggregate takes the form 1N∑i=1N∇Fi(ir,k), 1N _i=1^N∇ F_i( x_i^r,k), (7) which generally differs from ∇F()∇ F( x) at any common model x. The mismatch arises from the locations of gradient evaluation: heterogeneity first separates the client trajectories, after which subsequent gradients are measured at different locations. CTP-FL uses the same K gradient evaluations to query a shared predictive path. Instead of constructing N local trajectories, all clients evaluate their objectives at the same sequence of points. This makes the client average at each point an estimate of the corresponding global gradient. The remaining approximation is explicit: the queried global gradients are displaced from the current server model. 3.3 Common-Trajectory Predictive Federated Learning At the start of communication round r, let r x^r be the global model and r v^r the direction aggregated in the previous round. Set 0= v^0= 0. Given a prediction span h≥0h≥ 0, define tk=khK−1,K>1,0,K=1,kr=r−tkr,k=0,…,K−1.t_k= cases khK-1,&K>1,\\[4.0pt] 0,&K=1, cases z_k^r= x^r-t_k v^r, k=0,…,K-1. (8) The points kr z_k^r are identical across clients and are fixed before the current round’s mini-batches are sampled. Client i computes ir=1K∑k=0K−1gi(kr,ξi,kr), u_i^r= 1K _k=0^K-1g_i( z_k^r; _i,k^r), (9) where gi(kr,ξi,kr)g_i( z_k^r; _i,k^r) is a stochastic gradient of FiF_i at the shared point kr z_k^r. The server aggregates r+1=1N∑i=1Nir,r+1=r−γr+1. v^r+1= 1N _i=1^N u_i^r, x^r+1= x^r-γ v^r+1. (10) Algorithm 1 summarizes the procedure. The K query points are gradient-evaluation locations; they are not successive local model updates. The previous direction requires no separate downlink vector under full participation. A client retaining r−1 x^r-1 recovers it from r=r−1−rγ,r≥1. v^r= x^r-1- x^rγ, r≥ 1. (11) Thus, each round uses K mini-batch gradient evaluations per client, one d-dimensional uplink vector ir u_i^r, and one d-dimensional downlink model r x^r. The protocol has the same per-round gradient count and vector communication as full-participation FedAvg-M, although its computation does not perform local SGD steps. 3.4 Why Shared Queries Change the Error Structure Let ℱrF_r contain the history before the current mini-batches are drawn. Under unbiased stochastic gradients, [r+1∣ℱr]=1K∑k=0K−1∇F(kr)⏟r.E[ v^r+1 _r]= 1K _k=0^K-1∇ F( z_k^r)_ H^r. (12) The equality follows by averaging across all clients at each common point: 1N∑i=1N∇Fi(kr)=∇F(kr). 1N _i=1^N∇ F_i( z_k^r)=∇ F( z_k^r). It holds regardless of the magnitude of ∇Fi(kr)−∇F(kr)∇ F_i( z_k^r)-∇ F( z_k^r). Accordingly, heterogeneity does not create an additional client-trajectory term in the conditional mean under full participation. The path-averaged direction r H^r is generally distinct from ∇F(r)∇ F( x^r). For L-smooth F, ‖r−∇F(r)‖≤LK∑k=0K−1tk‖r‖≤Lh2‖r‖.\| H^r-∇ F( x^r)\|≤ LK _k=0^K-1t_k\| v^r\|≤ Lh2\| v^r\|. (13) The prediction span therefore has a precise role: it governs how far the algorithm probes along its previous direction and how much displacement bias that probe can introduce. When h=0h=0, all queries coincide at r x^r, giving synchronized mini-batch SGD with NKNK gradient evaluations per round. A curvature interpretation. The shared path also provides a useful interpretation of what the prediction changes. If F is a quadratic with constant Hessian H, then the uniform schedule in equation 8 yields the exact identity r=∇F(r)−h2Hr. H^r=∇ F( x^r)- h2H v^r. (14) Thus, relative to the h=0h=0 direction, the prediction incorporates a Hessian–direction term through gradient evaluations, without constructing a Hessian or communicating extra state. When r v^r approximates the current global gradient, positive-curvature directions are attenuated by this term; for general non-convex objectives, its effect depends on the local curvature and the accuracy of r v^r. Equation equation 14 is an interpretation of the update, not a claim of unconditional acceleration. Our convergence analysis takes h=γh=γ and controls the resulting predictive bias jointly with stochastic gradient noise. It establishes a full-participation stationarity guarantee without a bounded-gradient or bounded-client-dissimilarity assumption. Under partial participation, the identity at each shared point holds only after expectation over client sampling; the sampled direction has an additional client-sampling variance. The basic CTP-FL algorithm does not remove that term. Algorithm 1 Common-Trajectory Predictive Federated Learning (CTP-FL) 1: Server stepsize γ, prediction span h, communication rounds R, gradients per round K, number of clients N 2: Initialize global model 0 x^0 and prediction direction 0= v^0= 0 3: for r=0,…,R−1r=0,…,R-1 do 4: Server broadcasts r x^r 5: for each client i=1,…,Ni=1,…,N in parallel do 6: Recover r←(r−1−r)/γ v^r←( x^r-1- x^r)/γ if r>0r>0; otherwise use 0= v^0= 0 7: ir← u_i^r← 0 8: for k=0,…,K−1k=0,…,K-1 do 9: tk←kh/(K−1)t_k← kh/(K-1) if K>1K>1; otherwise t0←0t_0← 0 10: kr←r−tkr z_k^r← x^r-t_k v^r 11: Sample mini-batch Bir,kB_i^r,k; ir,k←∇Fi(kr,Bir,k) g_i^r,k←∇ F_i( z_k^r;B_i^r,k) 12: ir←ir+ir,k/K u_i^r← u_i^r+ g_i^r,k/K 13: end for 14: Client i sends ir u_i^r to the server 15: end for 16: r+1←1N∑i=1Nir v^r+1← 1N _i=1^N u_i^r 17: r+1←r−γr+1 x^r+1← x^r-γ v^r+1 18: end for 19: Global model R x^R 4 Theoretical Analysis We analyze the non-convex objective F()=1N∑i=1NFi(),F∗=inf∈ℝdF()>−∞.F( x)= 1N _i=1^NF_i( x), F_*= _ x ^dF( x)>-∞. (15) Our analysis separates two sources of error in the server direction: stochastic-gradient noise and the displacement between the current model and the shared predictive trajectory. Under partial participation, client sampling introduces a third source of error. We use the following standard assumptions (Karimireddy et al., 2020; Cheng et al., 2024). Assumption 4.1 (Smoothness). Each local objective FiF_i is L-smooth. That is, for every i∈[N]i∈[N] and ,∈ℝd x, y ^d, ‖∇Fi()−∇Fi()‖≤L‖−‖.\|∇ F_i( x)-∇ F_i( y)\|≤ L\| x- y\|. (16) Assumption 4.2 (Unbiased stochastic gradients). For every client i and every x, its mini-batch gradient gi(,ξ)g_i( x;ξ) satisfies ξ[gi(,ξ)]=∇Fi(),ξ‖gi(,ξ)−∇Fi()‖2≤σ2.E_ξ[g_i( x;ξ)]=∇ F_i( x), _ξ\|g_i( x;ξ)-∇ F_i( x)\|^2≤σ^2. (17) The mini-batches used at different query points, clients, and communication rounds are conditionally independent. Neither assumption bounds individual gradients or the differences ∇Fi()−∇F()∇ F_i( x)-∇ F( x). Predictive direction. Let ℱrF_r denote the history before the mini-batches of round r are sampled. For the shared query points kr=r−tkr,tk=kh/(K−1),K>1,0,K=1, z_k^r= x^r-t_k v^r, t_k= caseskh/(K-1),&K>1,\\ 0,&K=1, cases define r=1K∑k=0K−1∇F(kr). H^r= 1K _k=0^K-1∇ F( z_k^r). (18) Under full participation, the server direction in Algorithm 1 obeys [r+1∣ℱr]=r,[‖r+1−r‖2∣ℱr]≤σ2NK.E[ v^r+1 _r]= H^r, \! [\| v^r+1- H^r\|^2 _r ]≤ σ^2NK. (19) In particular, heterogeneous client gradients cancel at each shared query point. The resulting direction estimates the average gradient along the predicted path, rather than the gradient at r x^r. Smoothness bounds this predictive bias by ‖r−∇F(r)‖≤LK∑k=0K−1tk‖r‖≤Lh2‖r‖.\| H^r-∇ F( x^r)\|≤ LK _k=0^K-1t_k\| v^r\|≤ Lh2\| v^r\|. (20) Thus, the path length h controls a bias–variance trade-off without introducing a client-dissimilarity constant. Theorem 4.3 (Full-participation convergence of CTP-FL). Suppose Assumptions 4.1 and 4.2 hold, and all N clients participate in every round. Initialize 0= v^0= 0, set h=γh=γ, and let Δ=F(0)−F∗,0<γ≤14L. =F( x^0)-F_*, 0<γ≤ 14L. Then Algorithm 1 satisfies 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2NK. 1R _r=0^R-1E\|∇ F( x^r)\|^2≤ 6 γ R+ 5Lγσ^2NK. (21) If Δ>0 >0 and γ=(4L+5Lσ2R6ΔNK)−1,γ= (4L+ 5Lσ^2R6 NK )^-1, (22) then 1R∑r=0R−1‖∇F(r)‖2≤230LΔσ2NKR+24LΔR. 1R _r=0^R-1E\|∇ F( x^r)\|^2≤ 2 30L σ^2NKR+ 24L R. (23) Theorem 4.3 establishes the same (LΔσ2/(NKR)+LΔ/R)O\! ( L σ^2/(NKR)+L /R ) order as the full-participation guarantee of FedAvg-M (Cheng et al., 2024). It does not imply a strictly faster convergence rate. The methodological difference lies in how the K gradient evaluations are allocated: FedAvg-M advances client-specific local trajectories, whereas CTP-FL evaluates gradients along a common trajectory. The proof is given in Appendix A. Scope under partial participation. For completeness, suppose a subset rS_r of size S<NS<N is sampled uniformly without replacement and the server averages only the selected clients’ directions. Define ir=1K∑k=0K−1∇Fi(kr),Dr=1N∑i=1N‖ir−r‖2, H_i^r= 1K _k=0^K-1∇ F_i( z_k^r), D_r= 1N _i=1^N\| H_i^r- H^r\|^2, and ρS=N−S(N−1). _S= N-SS(N-1). Here DrD_r is the client dispersion along the observed predictive trajectory; the algorithm neither computes it nor assumes that it is bounded. Theorem 4.4 (Partial-participation trajectory bound). Under Assumptions 4.1 and 4.2, uniform sampling without replacement, h=γh=γ, and 0<γ≤1/(4L)0<γ≤ 1/(4L), 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2SK+5LγρSR∑r=0R−1[Dr]. aligned 1R _r=0^R-1E\|∇ F( x^r)\|^2≤\;& 6 γ R+ 5Lγσ^2SK\\ &+ 5Lγ _SR _r=0^R-1E[D_r]. aligned (24) The last term is the finite-population variance caused by client sampling. Therefore, Theorem 4.4 does not give a uniform heterogeneity-independent rate when S<NS<N. This limitation is intrinsic to the uncorrected sampled direction: even at a minimizer of F, individual client gradients may be arbitrarily large and cancel only after full aggregation. The proof and an explicit two-client counterexample are given in Appendix A. Table 1: Test accuracy (%) after 300 communication rounds on CIFAR-100 and Tiny-ImageNet. Each experiment uses N=100N=100 clients, S=10S=10 participating clients per round, mini-batch size 50, and K=50K=50 gradient evaluations per participating client. Data are partitioned with Dirichlet concentrations α∈0.1,0.05α∈\0.1,0.05\. Bold denotes the highest accuracy and underlining denotes the second-highest accuracy among the reported methods in each column. Method ResNet-18 ViT-Tiny CIFAR-100 Tiny-ImageNet CIFAR-100 Tiny-ImageNet Dir-0.1 Dir-0.05 Dir-0.1 Dir-0.05 Dir-0.1 Dir-0.05 Dir-0.1 Dir-0.05 FedAvg 60.17 56.75 47.48 43.80 27.24 23.42 15.68 14.05 SCAFFOLD 60.69 56.43 47.76 43.92 26.86 23.23 15.70 14.21 FedCM 66.61 62.65 41.16 36.00 28.23 25.74 18.88 18.15 CTP-FL 69.25 64.16 55.62 51.81 51.16 47.55 34.32 31.33 5 Experiments Experimental setup. We evaluate CTP-FL on CIFAR-100 (Krizhevsky, 2009) and Tiny-ImageNet (Le and Yang, 2015), using ResNet-18 (He et al., 2016) and ViT-Tiny (Dosovitskiy et al., 2020). To examine statistical heterogeneity, we partition the training data among N=100N=100 clients using a Dirichlet distribution with concentration α∈0.1,0.05α∈\0.1,0.05\; the smaller value represents a more concentrated allocation of classes across clients. At each of 300 communication rounds, S=10S=10 clients participate, and each participating client computes K=50K=50 mini-batch gradients with batch size 50. For CTP-FL, these gradients are evaluated at the K shared predictive points in Eq. equation 8; they are not K successive local model updates. Baselines and comparison budget. We compare against FedAvg (McMahan et al., 2017), SCAFFOLD (Karimireddy et al., 2020), and FedCM (Xu et al., 2021). All reported methods use the same number of communication rounds and participating clients. The value K=50K=50 fixes the number of mini-batch gradient evaluations for CTP-FL; for local-update baselines it corresponds to the number of local gradient steps. We report test accuracy at the end of round 300 in Table 1. Main results. CTP-FL achieves the highest reported accuracy in all eight settings of Table 1. On CIFAR-100 with ResNet-18, it improves over the strongest listed baseline by 2.64 and 1.51 percentage points under Dir-0.1 and Dir-0.05, respectively. On Tiny-ImageNet with ResNet-18, the corresponding gains are 7.86 and 7.89 points. The differences are larger for ViT-Tiny: 22.93 and 21.81 points on CIFAR-100, and 15.44 and 13.18 points on Tiny-ImageNet. These comparisons describe the results at round 300; they do not, on their own, establish faster convergence or identify the cause of the gains. Theory–experiment scope. These experiments use partial client participation, whereas Theorem 4.3 establishes a heterogeneity-independent rate under full participation. Under partial participation, Theorem 4.4 includes a trajectory-dependent client-sampling term. Consequently, the empirical performance in Table 1 should not be presented as verification of the full-participation rate. 6 Conclusion We introduced CTP-FL, which uses the gradient evaluations available between communication rounds to probe a shared predictive trajectory. Because all clients evaluate gradients at the same points, their full-participation average estimates the global gradient along that trajectory, rather than combining gradients measured at divergent local models. The prediction span makes the resulting displacement bias explicit and controllable. Under smoothness and bounded stochastic-gradient variance, we proved a non-convex full-participation stationarity rate of (LΔσ2/(NKR)+LΔ/R)O\! ( L σ^2/(NKR)+L /R ) without assuming bounded gradients or bounded client-gradient dissimilarity. Each client computes K mini-batch gradients and uploads one model-sized vector per round. In the reported partial-participation experiments, CTP-FL achieves the highest final accuracy among the listed methods across CIFAR-100 and Tiny-ImageNet with ResNet-18 and ViT-Tiny. Our theory and experiments have distinct scopes. The heterogeneity-independent rate applies to full participation; sampling clients introduces an additional variance term that the current algorithm does not eliminate. Further work should examine control mechanisms for partial participation and determine when probing a predictive trajectory improves on querying the current model alone under matched computation and communication budgets. References Cheng et al. (2024) Z. Cheng, X. Huang, P. Wu, and K. Yuan Momentum benefits non-IID federated learning simply and provably. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §4, §4. Dosovitskiy et al. (2020) A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. An image is worth 16x16 words: transformers for image recognition at scale. In International Conference on Learning Representations, Cited by: §5. He et al. (2016) K. He, X. Zhang, S. Ren, and J. Sun Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 770–778. Cited by: §5. Karimireddy et al. (2021) S. P. Karimireddy, M. Jaggi, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh Mime: mimicking centralized stochastic algorithms in federated learning. External Links: 2008.03606, Link Cited by: §2. Karimireddy et al. (2020) S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh SCAFFOLD: stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, p. 5132–5143. External Links: Link Cited by: §1, §2, §4, §5. Krizhevsky (2009) A. Krizhevsky Learning multiple layers of features from tiny images. Technical report University of Toronto. Cited by: §5. Le and Yang (2015) Y. Le and X. Yang Tiny imagenet visual recognition challenge. CS 231N 7 (7), p. 3. Cited by: §5. Li et al. (2020) T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, Vol. 2. External Links: Link Cited by: §2. Liu et al. (2025a) J. Liu, Y. Liu, F. Shang, H. Liu, J. Liu, and W. Feng Improving generalization in federated learning with highly heterogeneous data via momentum-based stochastic controlled weight averaging. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 38894–38939. External Links: Link Cited by: Appendix B. Liu et al. (2026a) J. Liu, F. Shang, H. Liu, J. Liu, W. An, and Y. Liu Taming preconditioner drift: unlocking the potential of second-order optimizers for federated learning on non-iid data. External Links: 2602.19271, Link Cited by: Appendix B. Liu et al. (2026b) J. Liu, F. Shang, H. Liu, Y. Tian, Y. Liu, J. Liu, K. Zhu, and Z. Lin FedAdamW: a communication-efficient optimizer with convergence and generalization guarantees for federated large models. Proceedings of the AAAI Conference on Artificial Intelligence 40, p. 23748–23756. External Links: Document Cited by: Appendix B. Liu et al. (2024) J. Liu, F. Shang, Y. Liu, H. Liu, Y. Li, and Y. Gong Fedbcgd: communication-efficient accelerated block coordinate gradient descent for federated learning. In Proceedings of the 32nd ACM International Conference on Multimedia, p. 2955–2963. Cited by: Appendix B. Liu et al. (2025b) J. Liu, F. Shang, Y. Tian, H. Liu, and Y. Liu Consistency of local and global flatness for federated learning. In Proceedings of the 33rd ACM International Conference on Multimedia, p. 3875–3883. External Links: Document Cited by: Appendix B. Liu et al. (2025c) J. Liu, F. Shang, J. Zhou, H. Liu, Y. Liu, and J. Liu FedMuon: accelerating federated learning with matrix orthogonalization. arXiv preprint arXiv:2510.27403. External Links: Document, Link Cited by: Appendix B. Liu et al. (2025d) J. Liu, Y. Tian, F. Shang, Y. Liu, H. Liu, J. Zhou, and D. Ding DP-FedPGN: finding global flat minima for differentially private federated learning via penalizing gradient norm. External Links: 2510.27504, Link Cited by: Appendix B. McMahan et al. (2017) B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, Vol. 54, p. 1273–1282. External Links: Link Cited by: §1, §2, §5. Stich (2019) S. U. Stich Local SGD converges fast and communicates little. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2. Woodworth et al. (2020) B. Woodworth, K. K. Patel, S. Stich, Z. Dai, B. Bullins, B. McMahan, O. Shamir, and N. Srebro Is local SGD better than minibatch SGD?. In Proceedings of the 37th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 119, p. 10334–10343. External Links: Link Cited by: §2. Xu et al. (2021) J. Xu, S. Wang, L. Wang, and A. C. Yao Fedcm: federated learning with client-level momentum. arXiv preprint arXiv:2106.10874. Cited by: §5. Zhang et al. (2019) M. Zhang, J. Lucas, J. Ba, and G. E. Hinton Lookahead optimizer: k steps forward, 1 step back. In Advances in Neural Information Processing Systems, Vol. 32. External Links: Link Cited by: §1, §2. Appendix A Convergence Analysis of CTP-FL We consider min∈ℝdF()≔1N∑i=1NFi(),F∗≔infF()>−∞. _ x ^dF( x) 1N _i=1^NF_i( x), F_* _ xF( x)>-∞. (25) The following assumptions match the standard smoothness and stochastic gradient assumptions used in FedAvg-M. Assumption A.1 (Smoothness). For each client i, FiF_i is L-smooth: ‖∇Fi()−∇Fi()‖≤L‖−‖for all ,∈ℝd.\|∇ F_i( x)-∇ F_i( y)\|≤ L\| x- y\| all x, y ^d. Assumption A.2 (Stochastic gradients). For each client i and every x, ξ[gi(,ξ)]=∇Fi(),ξ‖gi(,ξ)−∇Fi()‖2≤σ2.E_ξ[g_i( x;ξ)]=∇ F_i( x), _ξ\|g_i( x;ξ)-∇ F_i( x)\|^2≤σ^2. Fresh mini-batches are sampled independently across clients, local gradient evaluations, and communication rounds. CTP-FL update. Initialize 0= v^0= 0. At round r, let rS_r contain S clients, sampled uniformly without replacement, independently of the mini-batches. Set tk=kγK−1,K≥2,0,K=1,kr=r−tkr,k=0,…,K−1.t_k= cases kγK-1,&K≥ 2,\\[4.0pt] 0,&K=1, cases z_k^r= x^r-t_k v^r, k=0,…,K-1. (26) Each selected client computes ir=1K∑k=0K−1gi(kr,ξi,kr). u_i^r= 1K _k=0^K-1g_i( z_k^r; _i,k^r). (27) The server updates r+1=1S∑i∈rir,r+1=r−γr+1. v^r+1= 1S _i _r u_i^r, x^r+1= x^r-γ v^r+1. (28) Full participation corresponds to S=NS=N. Define ir≔1K∑k=0K−1∇Fi(kr),r≔1N∑i=1Nir=1K∑k=0K−1∇F(kr). h_i^r 1K _k=0^K-1∇ F_i( z_k^r), h^r 1N _i=1^N h_i^r= 1K _k=0^K-1∇ F( z_k^r). (29) The predictive bias is r≔r−∇F(r). b^r h^r-∇ F( x^r). For partial participation, define the observed trajectory dispersion Dr≔1N∑i=1N‖ir−r‖2.D_r 1N _i=1^N\| h_i^r- h^r\|^2. (30) This is an analysis quantity, not an assumption or an input to CTP-FL. Lemma A.3 (Bias and conditional variance). Under Assumptions A.1–A.2, conditional on the history before client sampling at round r, ‖r‖2≤L2γ24‖r‖2.\| b^r\|^2≤ L^2γ^24\| v^r\|^2. (31) Moreover, r[r+1]=r,E_r[ v^r+1]= h^r, (32) and r‖r+1−r‖2≤σ2SK+N−S(N−1)Dr,E_r\| v^r+1- h^r\|^2≤ σ^2SK+ N-SS(N-1)D_r, (33) where the second term is defined to be zero when S=NS=N. Proof. Since F is L-smooth and K−1∑k=0K−1tk≤γ/2K^-1 _k=0^K-1t_k≤γ/2, ‖r‖ \| b^r\| ≤1K∑k=0K−1‖∇F(kr)−∇F(r)‖ ≤ 1K _k=0^K-1\|∇ F( z_k^r)-∇ F( x^r)\| ≤LK∑k=0K−1tk‖r‖≤Lγ2‖r‖, ≤ LK _k=0^K-1t_k\| v^r\|≤ Lγ2\| v^r\|, (34) which proves equation 31. Unbiasedness of the stochastic gradients and uniform client sampling give equation 32. For equation 33, first condition on rS_r. The independent stochastic gradient errors have mean zero and their average has second moment at most σ2/(SK)σ^2/(SK). The remaining variance is that of sampling without replacement from the finite population iri=1N\ h_i^r\_i=1^N: r‖1S∑i∈rir−r‖2=N−S(N−1)1N∑i=1N‖ir−r‖2.E_r \| 1S _i _r h_i^r- h^r \|^2= N-SS(N-1) 1N _i=1^N\| h_i^r- h^r\|^2. The two errors have zero cross term, completing the proof. ∎ Theorem A.4 (Full-participation convergence). Suppose Assumptions A.1–A.2 hold, all N clients participate, and 0<γ≤1/(4L)0<γ≤ 1/(4L). Let Δ≔F(0)−F∗ F( x^0)-F_*. Then CTP-FL satisfies 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2NK. 1R _r=0^R-1E\|∇ F( x^r)\|^2≤ 6 γ R+ 5Lγσ^2NK. (35) In particular, no bounded-gradient or bounded-data-heterogeneity assumption is needed. Proof. Write a≔L2γ24,Zr≔r‖r+1−r‖2.a L^2γ^24, Z_r _r\| v^r+1- h^r\|^2. Here a≤1/64a≤ 1/64 and, under full participation, Zr≤σ2/(NK)Z_r≤σ^2/(NK). Smoothness of F, the server update, and r[r+1]=∇F(r)+rE_r[ v^r+1]=∇ F( x^r)+ b^r imply r[F(r+1)]≤ _r[F( x^r+1)]≤\; F(r)−γ⟨∇F(r),∇F(r)+r⟩ F( x^r)-γ ∇ F( x^r),∇ F( x^r)+ b^r +Lγ22(‖∇F(r)+r‖2+Zr). + Lγ^22 (\|∇ F( x^r)+ b^r\|^2+Z_r ). (36) Using −⟨,⟩≤‖2/4+‖2- p, q ≤\| p\|^2/4+\| q\|^2, ‖+‖2≤2‖2+2‖2\| p+ q\|^2≤ 2\| p\|^2+2\| q\|^2, and Lγ≤1/4Lγ≤ 1/4 gives the conservative bound r[F(r+1)]≤F(r)−γ4‖∇F(r)‖2+3γ2‖r‖2+Lγ22Zr.E_r[F( x^r+1)]≤ F( x^r)- γ4\|∇ F( x^r)\|^2+ 3γ2\| b^r\|^2+ Lγ^22Z_r. (37) For compactness, set AR≔∑r=0R−1‖∇F(r)‖2,VR≔∑r=0R−1‖r‖2,R≔∑r=0R−1[Zr].A_R _r=0^R-1E\|∇ F( x^r)\|^2, V_R _r=0^R-1E\| v^r\|^2, _R _r=0^R-1E[Z_r]. Summing equation 37, applying F(R)≥F∗F( x^R)≥ F_*, and using Lemma A.3, we obtain AR≤4Δγ+6aVR+2LγR.A_R≤ 4 γ+6aV_R+2L _R. (38) Next, 0= v^0= 0 and, for r≥1r≥ 1, r=∇F(r−1)+r−1+(r−r−1). v^r=∇ F( x^r-1)+ b^r-1+ ( v^r- h^r-1 ). The final parenthesized term has conditional mean zero. Consequently, ‖r‖2 \| v^r\|^2 ≤2‖∇F(r−1)‖2+2‖r−1‖2+2[Zr−1] ≤ 2E\|∇ F( x^r-1)\|^2+2E\| b^r-1\|^2+2E[Z_r-1] ≤2‖∇F(r−1)‖2+2a‖r−1‖2+2[Zr−1]. ≤ 2E\|∇ F( x^r-1)\|^2+2aE\| v^r-1\|^2+2E[Z_r-1]. (39) Summing over r and extending nonnegative sums yields VR≤2AR+2aVR+2R.V_R≤ 2A_R+2aV_R+2Z_R. (40) Since a≤1/64a≤ 1/64, VR≤21−2a(AR+R)≤3(AR+R).V_R≤ 21-2a(A_R+Z_R)≤ 3(A_R+Z_R). Substituting this into equation 38 gives (1−18a)AR≤4Δγ+(18a+2Lγ)R.(1-18a)A_R≤ 4 γ+(18a+2Lγ)Z_R. Because a≤1/64a≤ 1/64, AR≤6Δγ+(26a+3Lγ)R.A_R≤ 6 γ+(26a+3Lγ)Z_R. (41) Finally, R≤Rσ2/(NK)Z_R≤ Rσ^2/(NK) and a=(Lγ)2/4≤Lγ/16a=(Lγ)^2/4≤ Lγ/16, so that 26a+3Lγ<5Lγ26a+3Lγ<5Lγ. Substitution into equation 41 proves equation 35. ∎ Theorem A.5 (Partial-participation trajectory bound). Suppose Assumptions A.1–A.2 hold, 1≤S<N1≤ S<N, and 0<γ≤1/(4L)0<γ≤ 1/(4L). With DrD_r defined in equation 30, 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2SK+5Lγ(N−S)S(N−1)R∑r=0R−1[Dr]. aligned 1R _r=0^R-1E\|∇ F( x^r)\|^2≤\;& 6 γ R+ 5Lγσ^2SK\\ &+ 5Lγ(N-S)S(N-1)R _r=0^R-1E[D_r]. aligned (42) This theorem imposes no bound on DrD_r. Accordingly, equation 42 is a valid trajectory-dependent inequality, but does not by itself guarantee a uniform heterogeneity-independent convergence rate. Proof. The proof of Theorem A.4 up to equation 41 applies unchanged. By Lemma A.3, R≤Rσ2SK+N−S(N−1)∑r=0R−1[Dr].Z_R≤ Rσ^2SK+ N-SS(N-1) _r=0^R-1E[D_r]. Substitute this bound into equation 41 and use 26a+3Lγ<5Lγ26a+3Lγ<5Lγ. ∎ Proposition A.6 (Why client sampling cannot be ignored). For the CTP-FL update equation 28 with S<NS<N, Assumptions A.1–A.2 alone cannot yield a uniform stationarity bound depending only on Δ,L,σ,N,S,K,R ,L,σ,N,S,K,R, and γ that vanishes when Δ=σ=0 =σ=0. Proof. Take N=2N=2, S=1S=1, d=1d=1, and exact gradients (σ=0σ=0). For any M>0M>0, define F1(x)=L2x2+Mx,F2(x)=L2x2−Mx.F_1(x)= L2x^2+Mx, F_2(x)= L2x^2-Mx. Then F(x)=Lx2/2F(x)=Lx^2/2. Initialize x0=0x^0=0 and v0=0v^0=0. Thus x0x^0 is the global minimizer and Δ=0 =0. At round zero, every predictive location equals zero, regardless of K. The single sampled client returns v1=Mv^1=M or v1=−Mv^1=-M, each with probability 1/21/2. Hence x1=−γv1,‖∇F(x1)‖2=L2γ2M2.x^1=-γ v^1, \|∇ F(x^1)\|^2=L^2γ^2M^2. For any R≥2R≥ 2, the average stationarity measure therefore contains the strictly positive term L2γ2M2/RL^2γ^2M^2/R, which can be made arbitrarily large by increasing M, although Δ=σ=0 =σ=0. ∎ Theorem A.7 (Convergence of CTP-FL under full participation). Let F()=1N∑i=1NFi(),Δ=F(0)−F∗,F∗=infF()>−∞.F( x)= 1N _i=1^NF_i( x), =F( x^0)-F_*, F_*= _ xF( x)>-∞. Suppose that every FiF_i is L-smooth and that the stochastic gradients are unbiased, independent across clients and gradient evaluations, and have variance at most σ2σ^2. All N clients participate in every round. Run CTP-FL with 0= v^0= 0 and predictive locations kr=r−tkr,tk=kγ/(K−1),K≥2,0,K=1. z_k^r= x^r-t_k v^r, t_k= caseskγ/(K-1),&K≥ 2,\\ 0,&K=1. cases If 0<γ≤1/(4L)0<γ≤ 1/(4L), then 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2NK. 1R _r=0^R-1E\|∇ F( x^r)\|^2≤ 6 γ R+ 5Lγσ^2NK. (43) In particular, for Δ>0 >0, choose γ=(4L+5Lσ2R6ΔNK)−1. γ= (4L+ 5Lσ^2R6 NK )^-1. (44) Then 1R∑r=0R−1‖∇F(r)‖2≤230LΔσ2NKR+24LΔR=(LΔσ2NKR+LΔR). 1R _r=0^R-1E\|∇ F( x^r)\|^2≤ 2 30L σ^2NKR+ 24L R=O\! ( L σ^2NKR+ L R ). (45) No bounded-gradient or bounded-data-heterogeneity assumption is imposed. Proof. Let ℱrF_r denote the history before the fresh mini-batches of round r are sampled. Define r=1K∑k=0K−1∇F(kr),r=r−∇F(r),a=L2γ24. h^r= 1K _k=0^K-1∇ F( z_k^r), b^r= h^r-∇ F( x^r), a= L^2γ^24. Because all clients evaluate their gradients at the same kr z_k^r, unbiasedness and independence give [r+1∣ℱr]=r,[‖r+1−r‖2∣ℱr]≤σ2NK.E[ v^r+1 _r]= h^r, \! [\| v^r+1- h^r\|^2 _r ]≤ σ^2NK. (46) Smoothness and K−1∑k=0K−1tk≤γ/2K^-1 _k=0^K-1t_k≤γ/2 yield ‖r‖≤LK∑k=0K−1tk‖r‖≤Lγ2‖r‖,‖r‖2≤a‖r‖2.\| b^r\|≤ LK _k=0^K-1t_k\| v^r\|≤ Lγ2\| v^r\|, \| b^r\|^2≤ a\| v^r\|^2. (47) Since r+1=r−γr+1 x^r+1= x^r-γ v^r+1, the descent lemma and equation 46 imply [F(r+1)∣ℱr]≤ [F( x^r+1) _r]≤\; F(r)−γ⟨∇F(r),∇F(r)+r⟩ F( x^r)-γ ∇ F( x^r),∇ F( x^r)+ b^r +Lγ22(‖∇F(r)+r‖2+σ2NK). + Lγ^22 (\|∇ F( x^r)+ b^r\|^2+ σ^2NK ). (48) Apply −⟨,⟩≤14‖2+‖2,‖+‖2≤2‖2+2‖2.- p, q ≤ 14\| p\|^2+\| q\|^2, \| p+ q\|^2≤ 2\| p\|^2+2\| q\|^2. Because Lγ≤1/4Lγ≤ 1/4, we obtain [F(r+1)∣ℱr]≤F(r)−γ4‖∇F(r)‖2+3γ2‖r‖2+Lγ2σ22NK.E[F( x^r+1) _r]≤ F( x^r)- γ4\|∇ F( x^r)\|^2+ 3γ2\| b^r\|^2+ Lγ^2σ^22NK. (49) Set AR=∑r=0R−1‖∇F(r)‖2,VR=∑r=0R−1‖r‖2,QR=Rσ2NK.A_R= _r=0^R-1E\|∇ F( x^r)\|^2, V_R= _r=0^R-1E\| v^r\|^2, Q_R= Rσ^2NK. Summing equation 49, using F(R)≥F∗F( x^R)≥ F_*, and applying equation 47 give AR≤4Δγ+6aVR+2LγQR.A_R≤ 4 γ+6aV_R+2Lγ Q_R. (50) Furthermore, 0= v^0= 0 and r=∇F(r−1)+r−1+(r−r−1)(r≥1). v^r=∇ F( x^r-1)+ b^r-1+( v^r- h^r-1) (r≥ 1). The last term has conditional mean zero. Thus VR≤2AR+2aVR+2QR.V_R≤ 2A_R+2aV_R+2Q_R. (51) Since γ≤1/(4L)γ≤ 1/(4L), we have a≤1/64a≤ 1/64. Therefore equation 51 implies VR≤21−2a(AR+QR)≤3(AR+QR).V_R≤ 21-2a(A_R+Q_R)≤ 3(A_R+Q_R). Substituting into equation 50 gives (1−18a)AR≤4Δγ+(18a+2Lγ)QR.(1-18a)A_R≤ 4 γ+(18a+2Lγ)Q_R. As a≤1/64a≤ 1/64, AR≤6Δγ+(26a+3Lγ)QR.A_R≤ 6 γ+(26a+3Lγ)Q_R. Finally, a=(Lγ)2/4≤Lγ/16a=(Lγ)^2/4≤ Lγ/16, so 26a+3Lγ<5Lγ26a+3Lγ<5Lγ. This proves equation 43. For the parameter choice equation 44, γ≤1/(4L)γ≤ 1/(4L) and 6ΔγR=24LΔR+30LΔσ2NKR. 6 γ R= 24L R+ 30L σ^2NKR. Also, 5Lγσ2NK≤30LΔσ2NKR, 5Lγσ^2NK≤ 30L σ^2NKR, which proves equation 45. ∎ Theorem A.8 (CTP-FL under partial participation). Under the assumptions of Theorem A.7, suppose each round samples S<NS<N clients uniformly without replacement. Define ir=1K∑k=0K−1∇Fi(kr),r=1N∑i=1Nir, h_i^r= 1K _k=0^K-1∇ F_i( z_k^r), h^r= 1N _i=1^N h_i^r, and Dr=1N∑i=1N‖ir−r‖2.D_r= 1N _i=1^N\| h_i^r- h^r\|^2. For 0<γ≤1/(4L)0<γ≤ 1/(4L), 1R∑r=0R−1‖∇F(r)‖2≤6ΔγR+5Lγσ2SK+5Lγ(N−S)S(N−1)R∑r=0R−1[Dr]. aligned 1R _r=0^R-1E\|∇ F( x^r)\|^2≤\;& 6 γ R+ 5Lγσ^2SK\\ &+ 5Lγ(N-S)S(N-1)R _r=0^R-1E[D_r]. aligned (52) No uniform upper bound on DrD_r is assumed. Proof. Conditional on the history before client sampling, uniform sampling gives r[r+1]=rE_r[ v^r+1]= h^r. The finite-population variance identity and independent mini-batches give r‖r+1−r‖2≤σ2SK+N−S(N−1)Dr.E_r\| v^r+1- h^r\|^2≤ σ^2SK+ N-SS(N-1)D_r. Repeat the proof of Theorem A.7, replacing Rσ2/(NK)Rσ^2/(NK) throughout by Rσ2SK+N−S(N−1)∑r=0R−1[Dr]. Rσ^2SK+ N-SS(N-1) _r=0^R-1E[D_r]. The same bias and descent bounds yield equation 52. ∎ Appendix B Related Work Communication-efficient federated optimization. Communication-efficient federated learning commonly allows clients to perform multiple local updates between aggregation rounds. FedBCGD reduces the amount of information transmitted in each round by updating and communicating parameter blocks, and further incorporates drift control and variance reduction in its accelerated variant (Liu et al., 2024). This approach addresses the size of each message. In contrast, CTP-FL communicates one model-sized vector per participating client and changes where local gradients are evaluated: clients query the same server-defined predictive trajectory, so their updates can be averaged without first combining models that have followed different local trajectories. Alignment under heterogeneous data. Several recent methods study different forms of local–global misalignment. FedSWA and FedMoSWA use stochastic weight averaging and momentum-based control to improve generalization under highly heterogeneous data (Liu et al., 2025a). FedNSAM examines the mismatch between local and global flatness and uses a global Nesterov direction to improve their consistency (Liu et al., 2025b). These methods primarily target the properties of the resulting solution, including flatness and generalization. Our focus is the geometry of gradient evaluation during a communication round: when all clients evaluate at common points, averaging their gradients estimates the gradient of the global objective at those points, irrespective of how different the individual client gradients are. Federated adaptive and structured optimizers. FedAdamW combines local correction, decoupled weight decay, and aggregation of second-moment estimates for federated large-model training (Liu et al., 2026b). FedMuon exploits matrix orthogonalization and local–global alignment to improve federated optimization of matrix-structured parameters (Liu et al., 2025c). FedPAC identifies preconditioner drift as a source of instability when local second-order optimizers induce incompatible client geometries, and proposes preconditioner alignment and update correction (Liu et al., 2026a). Unlike these optimizer-specific mechanisms, CTP-FL applies to stochastic-gradient evaluations without transmitting moments or preconditioners. Its common trajectory aligns the locations of gradient evaluation rather than optimizer states. Global flatness and privacy. DP-FedPGN encourages globally flat solutions in client-level differentially private federated learning through a global gradient-norm penalty (Liu et al., 2025d). Its objective and privacy accounting are different from ours. We cite it because it likewise illustrates that a quantity defined by the global objective need not be faithfully represented by independently optimized local objectives.