Paper deep dive
How Much Regularization Survives Averaging? Update Masking in Federated Learning
Wenhao Yan, Fu Kuroda, Yucheng Jin, Zhenke Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/26/2026, 4:17:13 AM
Summary
This paper investigates why implicit regularization from update masking fails to survive Federated Averaging (FedAvg). The authors prove that independent per-client masks attenuate the curvature penalty by the cohort size, while synchronized masks recover it by a factor equal to the inverse gradient diversity. Experiments on CIFAR-10 reveal that minibatch sampling noise, rather than data heterogeneity, is the primary cause of low recovery factors, making effective regularization via masking impractical in standard federated settings.
Entities (7)
Relation Signals (5)
Update Masking → induces → Curvature Penalty
confidence 96% · Masking charges the optimizer for moving in sharp directions... R(Δ) denote the curvature penalty induced by masking
Federated Averaging → attenuates → Curvature Penalty
confidence 95% · when each client draws its own mask, federated averaging weakens that charge by exactly the cohort size
Gradient Diversity → determines → Recovery Factor
confidence 94% · giving every client the same mask brings it back by a factor equal to the inverse gradient diversity of the cohort
Minibatch Sampling → causes → Low Recovery Factor
confidence 93% · Minibatch sampling noise is what actually holds it there... changing data heterogeneity a thousandfold leaves it between 1.17 and 1.50
Data Heterogeneity → hasminimalimpacton → Recovery Factor
confidence 92% · changing data heterogeneity a thousandfold leaves it between 1.17 and 1.50
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Federated learning on non-IID data seeks flat minima to generalize across clients, and existing methods borrow sharpness-aware minimization from centralized training. There is a second way to reach flat minima, in which the regularization comes for free from noise added to the parameter updates, and it has never been carried over to the federated setting as an implicit regularizer. We show the reason. Masking charges the optimizer for moving in sharp directions. We prove that when each client draws its own mask, federated averaging weakens that charge by exactly the cohort size, and that giving every client the same mask brings it back by a factor equal to the inverse gradient diversity of the cohort. In our experiment setting on CIFAR-10, that factor is 1.19 out of a possible 10. Turning off minibatch sampling raises it to 8.96, while changing data heterogeneity a thousandfold leaves it between 1.17 and 1.50. The configurations keeping the regularization train far too poorly to use.
Tags
Links
- Source: https://arxiv.org/abs/2608.23286v2
- Canonical: https://arxiv.org/abs/2608.23286v2
Trouble viewing inline? Open PDF directly →
Full Text
27,925 characters extracted from source content.
Expand or collapse full text
How Much Regularization Survives Averaging? Update Masking in Federated Learning Thanks: *Corresponding author: Zhenke Chen (chanzzkk@gmail.com). 1st Wenhao Yan Affiliation: Faculty of Science & Technology Sophia University Tokyo, Japan wh_yan@eagle.sophia.ac.jp 2nd Fu Kuroda Affiliation: Faculty of Science & Technology Sophia University Tokyo, Japan f-kuroda-5a8@eagle.sophia.ac.jp 3rd Yucheng Jin Affiliation: Technical R&D Department Shendian Energy Co., Ltd. Lishui, China 15010330370@163.com 4th Zhenke Chen* Affiliation: Technical R&D Department Shendian Energy Co., Ltd. Lishui, China chanzzkk@gmail.com Abstract Federated learning on non-IID data seeks flat minima to generalize across clients, and existing methods borrow sharpness-aware minimization from centralized training. There is a second way to reach flat minima, in which the regularization comes for free from noise added to the parameter updates, and it has never been carried over to the federated setting as an implicit regularizer. We show the reason. Masking charges the optimizer for moving in sharp directions. We prove that when each client draws its own mask, federated averaging weakens that charge by exactly the cohort size, and that giving every client the same mask brings it back by a factor equal to the inverse gradient diversity of the cohort. In our experiment setting on CIFAR-10, that factor is 1.191.19 out of a possible 1010. Turning off minibatch sampling raises it to 8.968.96, while changing data heterogeneity a thousandfold leaves it between 1.171.17 and 1.501.50. The configurations keeping the regularization train far too poorly to use. Index Terms: federated learning, gradient diversity, implicit regularization, update masking I Introduction Federated learning (FL) trains a shared model across clients that cannot pool their data [1]. When those clients hold non-identically distributed data, the aggregated model is required to exhibit generalization performance across distributions it never sees jointly. Consequently, extensive research has pursued flat minima as the way to facilitate that generalization. Notable works along this direction include FedSAM [2], FedSMOO [3], and FedGAMMA [4], importing sharpness-aware minimization from centralized training. However, centralized training also offers a second route to flat minima, which has not been taken in FL as an implicit regularizer so far. Rather than explicitly perturbing the loss function, this kind of approaches operates directly on the update dynamics, including dropout, additive gradient noise, and most recently block-wise update masking, which discards entire parameter blocks randomly and rescales the survivors to stay unbiased, all of which induce an implicit curvature penalty through the variance of the perturbation. Masking is the sharpest instance, since discarding half of each update vector consistently outperforms most advanced dense optimizers in large language model (LLM) pre-training [5]. Dropout and additive noise do appear in FL, as a means of reducing client compute [6] and as a privacy mechanism [7] respectively, but not for the curvature penalty they induce. In this paper, we find that this phenomenon is not an oversight. Our contributions are summarized as follows. 1. We show that under FedAvg with independent per-client masks, the curvature penalty induced by masking is attenuated by exactly the cohort size, and we derive the resulting penalty. 2. Synchronizing the mask across clients restores the penalty, by a factor equal to the reciprocal of the cohort’s gradient diversity. That factor reaches the cohort size at peak, and drops below one when client updates cancel, then synchronizing hurts the model performance. 3. Measuring that factor on CIFAR-10 with FedAvg, we find it near its lower bound. Minibatch sampling noise is what actually holds it there. Data heterogeneity does not. I Preliminaries I-A Notation and Federated Averaging The model parameters θ are partitioned into B disjoint blocks θ(b)b=1B \θ^(b) \_b=1^B, typically one per parameter tensor. HbbH_b denotes the corresponding diagonal block of the Hessian, and H¯ H denotes the Hessian of the global objective. FedAvg proceeds in rounds. In each round, the server broadcasts θ to a cohort C of KcK_c clients randomly sampled from the whole population, each client runs τ local epochs, then the server computes the average of the resulting local updates Δk≜θ−θk,Δ¯=1Kc∑k∈CΔk, _k θ- _k, = 1K_c _k∈ C _k, (1) where θk _k denotes the parameters of the client k after local training. We use τ instead of E for the number of local epochs since E denotes the expectation throughout. Note that τ counts local epochs here, not local SGD steps as in some related prior work. I-B Update Masking Given a block-partitioned update Δ from any dense optimizer, block-wise masking replaces it with Δ~(b)=1pm(b)Δ(b),m(b)∼Bernoulli(p), ^(b)= 1pm^(b) ^(b),m^(b) Bernoulli(p), (2) which is drawn independently across blocks. The factor 1/p1/p is the key to make the masked update unbiased, and [Δ~(b)]=Δ(b)E [ ^(b) ]= ^(b). It is that variance which produces the curvature penalty. The masking granularity fundamentally shapes the surviving curvature structure, as independent block‑wise sampling decouples coordinates and a finer partitioning consequently retains fewer cross-coordinate terms. We follow prior work in masking whole parameter tensors. I-C Connection with Gradient Diversity For a cohort of updates Δkk∈C \ _k \_k∈ C, we use ΔS _S to denote the gradient diversity [8] as ΔS≜∑k‖Δk‖2‖∑kΔk‖2. _S _k \| _k \|^2 \| _k _k \|^2. (3) Throughout we work with its reciprocal 1/ΔS1/ _S, which measures how much individual effort of clients survives aggregation. It equals KcK_c when the cohort moves as one, and falls below one as the updates conflict. By the Cauchy-Schwarz inequality, we get ΔS≥1/Kc _S≥ 1/K_c, with equality if and only if all Δk _k are equal. Equivalently, 1/ΔS≤Kc1/ _S≤ K_c. Neither quantity is bounded on the other side. We have ΔS>1 _S>1 and 1/ΔS<11/ _S<1 whenever ∑i<j⟨Δi,Δj⟩<0 _i<j _i, _j <0, which stands for whenever the cohort’s updates point in conflicting directions and largely cancel in the average. FedExP explicitly connects its adaptive server step size to gradient diversity, though only in the special case of a single full-batch local step. The connection is in fact exact for any number of local steps, their step size rearranges to (Kc/2)⋅ΔS(K_c/2)· _S, so the server takes larger steps precisely when the cohort disagrees [9]. I Analysis The masking penalty was established in the centralized setting, for a single worker masking an individual optimizer step. We restate it here, then derive what becomes of it when many clients mask their round-level updates and the server averages the result. Throughout, we let R(Δ)≜1−p2p∑b=1B(Δ(b))⊤HbbΔ(b)R( ) 1-p2p _b=1^B( ^(b)) H_b ^(b) (4) denote the curvature penalty induced by masking an update Δ , where H is the Hessian at the point being evaluated. Proposition 1. Let Δt=(Δt(1),…,Δt(B)) _t=( _t^(1),…, _t^(B)) be a block-partitioned update from a base optimizer that is masked as in (2), then conditioned on the state at step t, [l(θt−Δ~t)]=l(θt−Δt)+R(Δt)+O(‖Δt‖3)E [l( _t- _t) ]=l( _t- _t)+R( _t)+O( \| _t \|^3) (5) Since the masked update is unbiased, the second term originates entirely in its variance. The quadratic form (Δt(b))⊤HbbΔt(b)( _t^(b)) H_b _t^(b) measures how sharply the loss curves along the update direction within block b. It becomes large when the step moves into a narrow valley, small when it moves along a flat one. Only the diagonal blocks of the Hessian appear, since independent masks across blocks cancel the cross terms in expectation. Masking therefore charges the optimizer for moving in sharp directions, and the coefficient 1−p2p 1-p2p sets the price, which vanishes as p→1p→ 1 and grows as more of the update is discarded. I-A Independent Masks At the beginning, the natural attempt is to let each client draw its own mask. Proposition 2. If the masks are drawn independently across clients as well as blocks, the curvature penalty at the aggregated model, denoted RindR_ind, is Rind=1Kc2∑k∈CR(Δk).R_ind= 1K_c^2 _k∈ CR( _k). (6) Proof: By the same expansion as in Proposition 1, and since masking leaves the mean update intact, the excess over the unmasked step is 12tr(H¯Cov(Δ¯~)) 12tr( HCov( )). Independence across clients gives Cov(Δ¯~)=1Kc2∑kCov(Δ~k)Cov( )= 1K_c^2 _kCov( _k), and each term contributes R(Δk)R( _k). ∎ Aggregation is linear in the updates but the penalty is quadratic, so the averaging coefficient enters squared while the sum contributes only KcK_c terms. The result is the cohort average of the single-worker penalty, attenuated by a further factor KcK_c. Each client still incurs the full penalty along its own trajectory, only the penalty at the aggregated model is diluted. I-B Synchronized Masks Dilution occurs because of the masks averaging out. Broadcasting a single mask seed per round, so that every client applies the same block mask, removes the necessity to average. Proposition 3. If a single mask is drawn per round and then shared with all clients, the corresponding curvature penalty RshrR_shr is Rshr=R(Δ¯).R_shr=R( ). (7) Proof: A shared mask gives Δ¯~(b)=1pm(b)Δ¯(b) ^(b)= 1pm^(b) ^(b), therefore the aggregated inherits the covariance of a single masked update, with Δ¯ in place of Δ . ∎ The penalty returns at full strength, and it now acts on the aggregated direction rather than on each client’s local trajectory, which is the one that actually needs to be regularized. Whether it is worth anything depends on how large Δ¯ is relative to the individual Δk _k. I-C Recovery Factor Propositions 2 and 3 differ only in the object that the penalty is applied to. Taking their ratio, and treating H¯bb H_b as isotropic within blocks so that the quadratic forms reduce to squared norms, RshrRind=‖Δ¯‖21Kc2∑k‖Δk‖2=1ΔS. R_shrR_ind= \| \|^2 1K_c^2 _k \| _k \|^2= 1 _S. (8) Corollary 1. Synchronizing the mask recovers the curvature penalty by a factor of 1/ΔS1/ _S, the reciprocal gradient diversity of the cohort. Here we find that the fraction of the masking regularization that survives federated averaging is governed by a quantity that is already measured in the aggregation of federated optimization. It is available at the server from the updates it receives. The only additional work is KcK_c squared norms of Δk _k with no communication cost. By the previous content, synchronization recovers at most a factor KcK_c, and recovers nothing once the cohort’s updates are mutually orthogonal. In the cancellation regime, the penalty is smaller than the independent one when 1/ΔS<11/ _S<1. That means it does even worse than doing nothing. Therefore broadcasting such a shared mask is strictly counterproductive. I-D Signal-Noise Decomposition Corollary 1 explains how much of the regularization survives, but offers no insight into the factors that determine the observed particular values. Here we divide each client’s gradient update into a useful component that all clients agree on and a client-specific noise Δk=μ+εk, _k=μ+ _k, (9) where μ≜[Δk]μ [ _k], so that εk _k has zero mean by construction. We assume the εk _k are independent across clients with common variance σ2=‖εk‖2σ^2=E \| _k \|^2. Then we substitute it into the two terms of ΔS _S and drop the cross terms, which vanish in expectation, ∑k‖Δk‖2≈Kc(‖μ‖2+σ2), _k \| _k \|^2≈ K_c( \|μ \|^2+σ^2), (10) ‖∑kΔk‖2=‖Kcμ+∑kεk‖2≈Kc2‖μ‖2+Kcσ2. \| _k _k \|^2= \|K_cμ+ _k _k \|^2≈K_c^2 \|μ \|^2+K_cσ^2. (11) The shared component adds coherently across the cohort and grows as Kc2K_c^2 while the noise adds incoherently and grows only as KcK_c. Then we set r=‖μ‖2σ2r= \|μ \|^2σ^2, 1ΔS=Kc‖μ‖2+σ2‖μ‖2+σ2=Kcr+1r+1, 1 _S= K_c \|μ \|^2+σ^2 \|μ \|^2+σ^2= K_cr+1r+1, (12) which recovers KcK_c as r→∞r→∞ and approaches 11 as r→0r→ 0. Finally, the recovery factor is a signal-to-noise ratio in disguise, and the gap between coherent and incoherent addition is its entire content. Meanwhile, both components are identifiable from statistics the server already forms. Let A=1Kc∑k‖Δk‖2A= 1K_c _k \| _k \|^2 and B=‖Δ¯‖2B= \| \|^2. The averaging of federated aggregation suppresses εk _k but not μ, so A≈‖μ‖2+σ2A≈ \|μ \|^2+σ^2 while B≈‖μ‖2+σ2/KcB≈ \|μ \|^2+σ^2/K_c. σ2σ^2 and ‖μ‖2 \|μ \|^2 can be separated by subtraction: σ^2=A−B1−1/Kc, σ^2= A-B1-1/K_c, (13) ‖μ^‖2=A−σ^2. \| μ \|^2=A- σ^2. (14) The reparameterization carries no information beyond ΔS _S itself. But the two estimators separate a low recovery factor caused by a weak shared direction from one caused by large client noise. The estimators assume the noise is independent across clients with comparable magnitude, which is reasonable when clients hold similar amounts of data. IV Federated Measurement IV-A Setup We train ResNet-18 with GroupNorm on CIFAR-10 [10], partitioned across K=100K=100 clients by a per-class Dirichlet distribution with concentration α. Each round the server samples a cohort of Kc=10K_c=10 clients uniformly, each round the selected clients run τ local epochs of SGD at learning rate 0.010.01 with batch size bsbs. The server applies the nkn_k-weighted mean of the updates where nkn_k denotes the quantity of samples stored in client k. All runs are 200 rounds with three seeds, and we report medians over the last 50 rounds. We vary bs∈32,128,fullbs∈ \32,128,full \ and τ∈1,2,5τ∈ \1,2,5 \, where fullfull means a single gradient step over the client’s entire shard, and separately α∈100,1,0.1α∈ \100,1,0.1 \. GroupNorm replaces BatchNorm because batch statistics are a known confound under non-IID partitioning. Since H¯ H is not available, ΔS _S is computed in the Euclidean norm rather than the H¯ H-weighted one. Correctness is established by a control in which all clients receive identical shards and identical seeds, so that every Δk _k coincides and 1/ΔS=Kc1/ _S=K_c to machine precision. We report ΔS _S under the uniform convention before, and give the nkn_k-weighted value alongside it in this section. The two agree to within 0.04%0.04\% at α=100α=100, where shard sizes vary by under 3%3\%. They diverge as the partition becomes unbalanced, we report the gap at α=0.1α=0.1. IV-B Effect of Local Training Configuration Two quantities distinguish the configurations in Table I. The local batch size controls whether a client’s gradient is exact or sampled. The number of local steps per round denoted as T, controls how far a client travels before reporting, which equals τ times the number of minibatches per epoch for a client holding nkn_k samples. With about 500500 samples per client, the grid spans T=1T=1 to T=80T=80, from a single full-batch step at one extreme to eighty minibatch steps at the other. The two axes are not independent, since halving the batch size also doubles both the sampling noise per step and T, and separating their effects is the purpose of this sweep. The pattern that emerges is not a gradient but a cliff, and it does not fall where we would expect. The three full-batch configurations give 8.968.96, 4.134.13, 3.963.96. Every configuration using a minibatch (3232 or 128128) lies between 1.101.10 and 1.361.36. The gap spans a factor of nearly three, and nothing occupies it. A natural explanation is that clients drift apart the longer they train alone, which would make the recovery factor fall with T. However, it does not hold across the following two groups: five steps at full batch gives 3.963.96, while four steps at a batch of 128128 gives 1.361.36. What distinguishes them is not the number of steps but whether those steps use exact local gradients or sampled ones. The two also reach almost the same accuracy, 0.4930.493 and 0.4490.449, so the gap is not a matter of how far training has progressed. It is obvious that within each group the ordering is monotone in T, so the number of local steps is the thing that does matter. But it simply matters far less than whether the gradients are stochastic. Raising τ from 11 to 55 moves the recovery factor from 1.241.24 to 1.101.10 at a batch of 3232, and from 8.968.96 to 3.963.96 at full size batch. The mechanism is most sensitive where the noise is smallest. Corollary 1 allows 1/ΔS1/ _S to fall below one, in which case a synchronized mask does worse than independent masks. It happens at a batch of 128128 with τ=1τ=1 where 4.5%4.5\% of rounds fall in that regime. The fraction declines as the noise grows and reaches zero in the three noisiest cells. It is never large, but it shows the regime is real rather than a formal possibility. The two aggregation conventions agree to within 1%1\% in every cell, as expected at α=100α=100 where shard sizes vary little. TABLE I: FedAvg on a near-IID split (α=100α=100), Kc=10K_c=10 of 100 clients per round, 200 rounds, 3 seeds, sweeping local batch size bsbs and epochs τ over T=⌈n¯k/bs⌉τT= n_k/bs \,τ local steps per round (n¯k≈500 n_k≈ 500). Entries are medians across seeds of each run’s median over the last 50 rounds; Kc=10K_c=10 is the ceiling on 1/ΔS1/ _S. 1/ΔS1/ _S bsbs τ T unwtd. nkn_k-wtd. μ^2 μ^2 σ^2 σ^2 acc rds.<1% subarraycrds.\\ <1\,\% subarray full 1 1 8.96 8.95 3.6×10−33.6× 10^-3 4.9×10−44.9× 10^-4 0.314 0.0 full 2 2 4.13 4.13 6.2×10−46.2× 10^-4 1.2×10−31.2× 10^-3 0.383 0.0 full 5 5 3.96 3.96 3.7×10−33.7× 10^-3 7.6×10−37.6× 10^-3 0.493 0.0 128 1 4 1.36 1.35 8.4×10−48.4× 10^-4 1.9×10−21.9× 10^-2 0.449 4.5 128 2 8 1.32 1.31 1.2×10−31.2× 10^-3 3.4×10−23.4× 10^-2 0.534 1.7 32 1 16 1.24 1.24 2.1×10−32.1× 10^-3 7.4×10−27.4× 10^-2 0.553 1.0 128 5 20 1.18 1.18 2.8×10−32.8× 10^-3 1.2×10−11.2× 10^-1 0.683 0.0 32 2 32 1.19 1.19 4.1×10−34.1× 10^-3 1.9×10−11.9× 10^-1 0.680 0.0 32 5 80 1.10 1.10 7.3×10−37.3× 10^-3 6.0×10−16.0× 10^-1 0.814 0.0 IV-C Decomposition of the Recovery Factor Splitting each client’s update into the part the cohort agrees on and the part it does not lets us ask which of the two is responsible. Across the nine configurations the noise variance spans three orders of magnitude, from 4.9×10−44.9× 10^-4 at full batch with τ=1τ=1 to 6.0×10−16.0× 10^-1 at a batch size of 3232 with τ=5τ=5. The shared component spans far less from 6.2×10−46.2× 10^-4 to 7.3×10−37.3× 10^-3, and it moves in the opposite direction. Clients that take more local steps travel further along the shared direction as well as accumulating more noise. The recovery factor falls because the noise outgrows the signal by over two orders of magnitude, instead of the signal disappearing. Configurations reach different points in training after 200200 rounds, with test accuracy ranging from 0.3140.314 to 0.8140.814, so a direct comparison risks confounding sampling noise with training progress. The grid contains a case that separates the two. The full batch run with τ=5τ=5 reaches 0.4930.493 accuracy, between the 0.4490.449 from the batch size of 128128 with τ=1τ=1 and the 0.5340.534 of the same batch with τ=2τ=2. Its noise variance is lower than both by more than a factor of two, and its recovery factor is three times higher. Training progress does not place it between its neighbors. It is the sampling regime that places it apart from them. IV-D Trade-off with Model Accuracy Recovering the regularization is possible, but the configurations that recover it are not ones anybody would train with. Moving from a batch size of 3232 with τ=2τ=2 to full batch with τ=5τ=5 raises the recovery factor from 1.191.19 to 3.963.96, a gain of 3.33.3 times but still reaches only 40%40\% of the ceiling KcK_c, and costs 0.1870.187 in test accuracy from 0.6800.680 to 0.4930.493. Pushing further to full batch with τ=1τ=1, recovering buys 8.968.96 at an accuracy of 0.3140.314. Fig. 1: Recovery factor against final test accuracy for the nine configurations of Table I. Marker shape denotes local batch size, annotations denote τ. Fig.1 plots the two quantities against each other. Only one cell is dominated, whose batch size is 128 with τ=1τ=1. But the gap in the middle means there is nothing to trade along. Moving toward better recovery costs a large amount of accuracy, and even then most of the regularization is still gone. IV-E Effect of Data Heterogeneity Since the motivation is non-IID data, it is worth asking whether heterogeneity is what holds the recovery factor down. Table I varies the Dirichlet concentration with the local protocol fixed. As is shown in the unweighted column, α=100α=100 and α=1α=1 are indistinguishable and α=0.1α=0.1 rises to 1.501.50. Under the nkn_k-weighted convention all three are flat at about 1.181.18. The disagreement appears only at α=0.1α=0.1, where a few clients hold a hundred times more data than others, and it is the same in every seed, so it reflects the partition rather than run-to-run variation. The α=0.1α=0.1 runs also reach lower accuracy, so part of the difference reflects a different point in training rather than heterogeneity as such. Neither changes the conclusion. Over three orders of magnitude in α the recovery factor moves between 1.171.17 and 1.501.50, while changing the local training protocol at a single α moves it from 1.191.19 to 8.968.96. Heterogeneity is not the main reason that holds it down. TABLE I: Varying the Dirichlet concentration α with the local protocol fixed at bs=32bs=32, τ=2τ=2. Shard sizes are far more uneven at α=0.1α=0.1, so the two aggregation conventions differ there. 1/ΔS1/ _S α unwtd. nkn_k-wtd. μ^2 μ^2 σ^2 σ^2 acc rds.<1% subarraycrds.\\ <1\,\% subarray 100 1.19 1.19 4.1×10−34.1× 10^-3 1.9×10−11.9× 10^-1 0.680 0.0 1 1.19 1.17 3.9×10−33.9× 10^-3 1.7×10−11.7× 10^-1 0.626 0.2 0.1 1.50 1.18 6.5×10−36.5× 10^-3 1.1×10−11.1× 10^-1 0.439 3.0 IV-F Discussion and Limitations Three limitations bound what these measurements support. The analysis rests on a second-order expansion that assumes small updates, which multi-step minibatch training violates. Those are precisely the configurations where the recovery factor sits near one, so the penalty there should be read as indicative rather than exact. We measure ΔS _S in the Euclidean norm because the global Hessian is unavailable at the server, and Corollary 1 is exact only under the H¯ H-weighted norm. And we measure how much of the penalty survives aggregation, not what it is worth. A recovery factor of 1.191.19 does not by itself say masking would have helped had it survived. All results are for one architecture and one dataset. The attenuation of Proposition 2 uses only that the perturbation is zero-mean and independent across clients, so it applies equally to update-level dropout and to additive gradient noise. Perturbations that shift the mean rather than the variance, as sharpness-aware minimization does, pass through averaging undiminished, which may be why the flatness-seeking methods successfully transferred to federated learning have been of that kind. V Conclusion We asked how much of the curvature penalty induced by update masking survives FedAvg. With independent per-client masks it is attenuated by the cohort size. Synchronizing the masks restores it by the reciprocal gradient diversity of the cohort, a factor bounded above by the cohort size but not below by one. On CIFAR-10 it sits at 1.191.19 against a ceiling of 1010, and what holds it there is minibatch sampling noise rather than data heterogeneity. The configurations that recover it are not ones anybody would train with. References [1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas (2017) Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, p. 1273–1282. Cited by: §I. [2] Z. Qu, X. Li, R. Duan, Y. Liu, B. Tang, and Z. Lu (2022) Generalized federated learning via sharpness aware minimization. In International conference on machine learning, p. 18250–18280. Cited by: §I. [3] Y. Sun, L. Shen, S. Chen, L. Ding, and D. Tao (2023) Dynamic regularized sharpness aware minimization in federated learning: approaching global consistency and smooth landscape. In International conference on machine learning, p. 32991–33013. Cited by: §I. [4] R. Dai, X. Yang, Y. Sun, L. Shen, X. Tian, M. Wang, and Y. Zhang (2023) Fedgamma: federated learning with global sharpness-aware minimization. IEEE Transactions on Neural Networks and Learning Systems 35 (12), p. 17479–17492. Cited by: §I. [5] T. Joo, W. Xia, C. Kim, M. Zhang, and E. Ie (2026) On surprising effectiveness of masking updates in adaptive optimizers. arXiv preprint arXiv:2602.15322. Cited by: §I. [6] D. Wen, K. Jeon, and K. Huang (2022) Federated dropout—a simple approach for enabling federated learning on resource constrained devices. IEEE wireless communications letters 11 (5), p. 923–927. Cited by: §I. [7] A. Cheng, P. Wang, X. S. Zhang, and J. Cheng (2022) Differentially private federated learning with local regularization and sparsification. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10112–10121. Cited by: §I. [8] D. Yin, A. Pananjady, M. Lam, D. Papailiopoulos, K. Ramchandran, and P. Bartlett (2018) Gradient diversity: a key ingredient for scalable distributed learning. In International Conference on Artificial Intelligence and Statistics, p. 1998–2007. Cited by: §I-C. [9] D. Jhunjhunwala, S. Wang, and G. Joshi (2023) Fedexp: speeding up federated averaging via extrapolation. arXiv preprint arXiv:2301.09604. Cited by: §I-C. [10] A. Krizhevsky (2009) Learning multiple layers of features from tiny images. Technical Report University of Toronto, Toronto, Ontario, Canada. Cited by: §IV-A.