Paper deep dive
FedA2L: Adaptive layer-wise learning rate adjustment in decentralized federated learning
Van Truong Vo, Khoa Nguyen, Taehong Kim
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94 times faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.09208v1
- Canonical: https://arxiv.org/abs/2608.09208v1
Trouble viewing inline? Open PDF directly →
Full Text
95,177 characters extracted from source content.
Expand or collapse full text
FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Van Truong Vo , Khoa Nguyenand Taehong Kim ∗ School of Information and Communication Engineering, Chungbuk National University, Cheongju, South Korea A R T I C L E I N F O Keywords: Decentralized federated learning Layer-wise learning rate adaptation Data heterogeneity Network consensus Convergence acceleration Communication efficiency Internet of Things (IoT) Edge Computing A B S T R A C T Decentralized intelligence systems with heterogeneous devices and limited coordination increasingly rely on decentralized federated learning (DFL). However, DFL suffers from convergence inefficiency under data heterogeneity due to the use of a uniform learning rate (LR) that ignores layer-specific optimization needs. Foundational layers are responsible for maintaining network consensus, while specialized layers adapt to local data characteristics, leading to conflicting gradients and degraded performance under non-IID conditions. To address this fundamental tension, this work introduces FedA2L, a method that dynamically adjusts layer-wise LRs based on model divergence signals. By leveraging local update intensity and network consensus constraints, FedA2L seamlessly integrates into existing DFL protocols without additional communication or coordination. Extensive evaluations across DFL algorithms, various model architectures, and datasets demonstrate that FedA2L achieves up to 4.94×faster convergence than vanilla DFL and reduces communication rounds by up to 59% compared to scheduler-based baselines. Furthermore, FedA2L exhibits resilience to severe data heterogeneity, larger network sizes, and sparse topologies, reducing communication overhead and establishing it as a versatile optimization tool for resource-constrained or large-scale distributed learning in edge and IoT deployments. The code is released at https://github.com/nclabteam/FedA2L. 1. Introduction Recent advances in Industrial IoT (IIoT) and Cyber- Physical Systems (CPS) require robust decentralized intel- ligence mechanisms to maintain real-time synchronization between physical assets and their Digital Twins (DT) [1]. In such settings, centralized cloud-based learning introduces un- acceptable latency, privacy risks, and single points of failure [2]. In this context, federated learning (FL) has emerged as a distributed computing paradigm that enables collaborative model training on decentralized data, preventing direct access to sensitive information at individual nodes [3]. In standard FL, a central server coordinates training by collecting local model updates from participating clients, aggregating them into a global model, and broadcasting updated parameters back to clients for the next round. However, this central- ization introduces vulnerabilities, such as communication bottlenecks [4], and scalability issues [5, 6]. DFL mitigates these issues by removing the central server, enabling direct peer-to-peer (P2P) communication among nodes according to a network topology [7]. In DFL, each node independently performs local model training on its private data and exchanges model parameters only with direct neigh- bors, then aggregates received models to achieve network consensus without central coordination [8]. This architecture enhances robustness, alleviates communication bottlenecks by distributing aggregation load across the network, and im- proves scalability. These properties make DFL well suited for © 2026. This manuscript version is made available under the C-BY- NC-ND 4.0 license (https://creativecommons.org/licenses/by-nc-nd/4.0/). This is the accepted manuscript version of an article published in Future Generation Computer Systems. The final published version is available at https://doi.org/10.1016/j.future.2026.108743. ∗ Corresponding author ORCID(s): large-scale networked systems, including edge intelligence systems [9,10], Internet of Things (IoT) networks [11,12], and traffic-aware communication environments [13], where centralized coordination is often impractical. Despite these advantages, DFL systems confront a critical challenge of statistical heterogeneity in distributed data. In practice, data on each node is non-identically and inde- pendently distributed (non-IID). This heterogeneity causes each node’s local learning objectives to drift away from the consensus objectives enforced by neighbor-wise aggregation, a phenomenon known as client drift [14,15]. This drift manifests as inconsistency between local gradients and the global optimization direction, leading to conflicting parameter updates. These conflicts slow convergence and reduce model quality [16,17]. This problem intensifies as data heterogeneity becomes more severe, creating a critical obstacle to achieving efficient and accurate distributed learning. This problem is further compounded in deep neural networks (DNNs), where client drift emerges unevenly across model layers owing to their hierarchical structure in feature extraction. Foundational layers capture general and shared features to support network consensus, while specialized layers adapt to task-specific features in local data [18,19]. Within DFL, this architectural hierarchy creates fundamentally different optimization challenges for each layer. Specialized layers are highly adapted to local data characteristics to achieve good performance on local objectives, causing significant dispersion between nodes. Conversely, foundational layers that over-homogenize shared representations limit their capacity to maintain task-general features to support local performance [20]. Recent work on layer-wise personalized FL [21] confirms this asymmetry, demonstrating that optimal layer-wise treatment requires Truong et al.: Preprint submitted to ElsevierPage 1 of 18 arXiv:2608.09208v1 [cs.LG] 10 Aug 2026 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning balancing local update intensity in specialized layers with network consensus constraints in foundational layers. This tension is exacerbated by the widespread use of a uniform LR across all model layers in standard DFL algorithms. Treating the LR as a single scalar hyperparameter forces all layers to update at the same rate, ignoring the distinct optimization landscapes, gradient magnitudes, and update sensitivities exhibited by different layers. As a result, uniform LRs in DFL systems suffer from training instability, prolonged convergence time, and suboptimal model quality, becoming more significant in the case of severe non-IID conditions. To address this critical gap in DFL systems, this study proposes the “Adaptive Layer-wise Learning Rate Adjust- ment in Decentralized Federated Learning” (FedA2L), a serverless method for dynamically tuning layer-wise LRs based on model divergence signals. The core innovation is formalizing model states at distinct transitions of each DFL round: the base state (before local training), the trained state (after local gradient updates), and the aggregated state (after network consensus). Through analyzing transitions between these three states, FedA2L derives two layer-wise metrics capturing distinct aspects of the optimization dynamics: weight divergence (휎) measuring local update intensity across nodes and aggregation stability (휁) measuring network consensus constraints. By leveraging these signals, FedA2L dynamically assigns a unique LR to each model layer, enabling simultaneous optimization of local learning and network consensus without requiring central coordination or additional communication overhead. The primary contribu- tions can be summarized as follows: •FedA2L introduces a state-based formalization that characterizes model dynamics across three states of each DFL round, enabling precise analysis of layer- wise divergence and aggregation behavior. • Adaptation of two layer-wise metrics (weight diver- gence휎and aggregation stability휁) enables layer-wise dynamic LR adjustment to quantify and balance local update intensity and network consensus constraints. •Layer-wise adaptive LRs based on local model di- vergence metrics can be seamlessly integrated into existing DFL algorithms without modifying the core aggregation logic or introducing additional communi- cation overhead. •Extensive empirical validation demonstrates that FedA2L significantly accelerates convergence across six DFL algorithms and DFedHPO, spanning multiple neural network architectures and diverse datasets, achieving up to 4.94×faster convergence than vanilla DFL and up to59%fewer communication rounds than the best-performing scheduler-based baselines. These contributions are particularly valuable for decentral- ized intelligence systems operating under dynamic conditions, where layer-wise adaptive mechanisms balance localized learning and stable network consensus across distributed entities. The remainder of this paper is organized as follows. Section 2 reviews related work on DFL and adaptive hy- perparameter optimization methods. Section 3 presents the problem formulation, the three-state model of DFL node procedures, and the proposed dual-metric layer-wise LR adaptation mechanism with its algorithmic integration. To formally validate these properties, Section 4 establishes convergence guarantees for both non-convex and strongly convex settings. Section 5 evaluates convergence speed, model accuracy, robustness under severe data heterogeneity, node scalability, and sparse network topologies, together with ablation and hyperparameter sensitivity analyses of FedA2L. Section 6 discusses the advantages and limitations of FedA2L and outlines directions for future work. Section 7 concludes the paper. 2. Related work 2.1. Decentralized federated learning DFL eliminates the reliance on a central server, enabling robust and scalable P2P collaboration. Foundational work such as D-PSGD [28] provided convergence guarantees using P2P model averaging. Recent surveys [29,6] system- atically categorize DFL advances, with research focusing on enhancing communication efficiency, convergence speed under non-IID conditions, and robustness to security and privacy threats [30,31]. This includes analyzing the impact of network topology [32,33] and developing advanced protocols to enhance communication efficiency, such as using sparsifica- tion and quantization to compress model updates [34,35], re- ducing transmission overhead in bandwidth-constrained edge environments. Advanced protocols, such as FedAWA [36] and FedAW [37], enhance aggregation by reweighing client contributions based on local data characteristics or model similarity, improving alignment with the overall learning dynamics. However, these methods assume uniform LRs during local training or apply only basic decay strategies across all layers, limiting adaptability to heterogeneous data and hierarchical model dynamics. While promising for general scenarios, such limitations hinder the deployment of robust decentralized intelligence in complex cyber-physical infrastructures that require coordinated learning across di- verse and heterogeneous devices. 2.2. Hyperparameter optimization in FL and DFL The LR is a critical hyperparameter in distributed opti- mization, affecting both convergence speed and model quality. In centralized FL, server-side adaptive optimization methods have been developed to enhance performance. FedYogi [22] employs a server-side adaptive optimizer that stabi- lizes convergence by maintaining coordinate-wise second moment estimates. FLARE [23] dynamically adjusts LRs for selected devices based on their individual computational capabilities. Recent work on federated hyperparameter op- timization [38,39] highlights the challenge of balancing Truong et al.: Preprint submitted to ElsevierPage 2 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Table 1 Comparison of adaptive learning-rate methods for federated learning and decentralized federated learning. MethodLayer-wise LRDynamic per RoundServer-freeNo Extra Comm.Signal Source FedYogi [22]×✓×✓Server-side second moment FLARE [23] ×✓×✓Device capability DFedHPO [24]×✓Pre-training search AutoLR [25] ✓×Gradient norms Fed-LAMB [26]✓×Gradient statistics FLAYER [21] ✓×Global gradient/loss signals FedLWS [27]✓×Weight shrinking FedA2L✓Local state transitions efficiency with adaptive tuning. However, all these methods are fundamentally server-dependent and incompatible with DFL. In DFL settings, DFedHPO [24] performs a one-time, decentralized hyperparameter search prior to model training and outputs a consensus hyperparameter configuration that re- mains fixed during subsequent communication rounds. While these methods eliminate the need for a server, both server- dependent adaptive methods and one-shot static search are unable to accommodate evolving layer-specific heterogeneity or continual drift during training. Layer-wise adaptive methods, motivated by hierarchical feature learning [18,19], have been explored in centralized FL. AutoLR [25] adjusts per-layer rates based on gradient norms, while Fed-LAMB [26] incorporates layer-wise and dimension-wise adaptivity. More recently, FLAYER [21] adjusts layer-specific LRs using gradient or loss signals, and FedLWS [27] proposes adaptive layer-wise weight shrinking. However, these methods face fundamental limitations in DFL: (1) they require centralized coordination for global layer-wise statistics, (2) incur significant computational overhead, and (3) treat aggregation as a post-processing step rather than an optimization signal source. Table 1 summarizes these limita- tions and positions FedA2L against existing hyperparameter optimization methods across the four properties that define an effective DFL solution. Current DFL methods often overlook the internal dy- namics of local training, relying on static LRs that fail to account for node-specific data heterogeneity and layer- specific learning dynamics. Existing adaptive LR methods are either server-dependent, operate at a coarse model level, or perform static hyperparameter searches. To the best of our knowledge, no prior work has developed a dynamic, layer-wise adaptive LR method tailored to the serverless nature of DFL. This highlights the need for decentralized mechanisms that can respond to layer-specific optimization behavior without a centralized coordinator. To address this gap, an effective solution should rely solely on locally available information, incur no additional communi- cation overhead, and support layer-wise adaptation to capture heterogeneous optimization roles across model layers while preserving training stability under severe non-IID conditions. Based on these requirements, we design a serverless, communication-efficient, and layer-wise adaptive LR mech- anism for DFL. The key insight is that the standard DFL training loop already contains locally observable signals: the parameter transitions during local training and network consensus directly encode both local update intensity and network consensus constraints at the layer level. By extracting and combining these signals without any additional coordina- tion, the proposed method achieves lightweight yet effective per-layer rate control that integrates seamlessly into existing DFL protocols, as detailed in the following section. 3. Methodology 3.1. Decentralized federated learning setup We consider a DFL system comprising a set of nodes interconnected in a P2P topology. Each node푖 ∈ holds a private local dataset 푖 , which remains strictly local and unshared. Instead of relying on a centralized server, each node푖collaborates by exchanging model parameters only with its direct neighbors 푖 . The objective is to learn model parameters that minimize the average of local loss functions across the network: min 휃 1 ,...,휃 || (휃 1 ,...,휃 || ) ≜ 1 || || ∑ 푖=1 푖 (휃 푖 ).(1) where 푖 (휃 푖 ) = 피 (푥,푦)∼ 푖 [퓁(휃 푖 ;푥,푦)], and퓁(⋅)denotes a standard loss function. Solving this optimization problem requires iterative training through communication rounds, alternating between local training on heterogeneous data and network consensus aggregation. 3.2. Problem definition Statistical heterogeneity in distributed federated systems creates uneven, layer-specific optimization challenges with significant implications for system convergence efficiency and practical deployment. During local training, special- ized layers adapt rapidly to node-specific data, whereas foundational layers preserve task-general representations. Conversely, during network consensus, parameter averag- ing applies uniform pressure across all layers, stabilizing Truong et al.: Preprint submitted to ElsevierPage 3 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning foundational representations but potentially suppressing necessary adaptation in specialized layers. This layer-wise disparity creates competing optimization objectives: local learning drives specialized layers toward node-specific op- tima, while network consensus pushes foundational layers toward network-wide agreement. Despite the disparity observed across layers, existing DFL methods typically apply a single scalar LR to all model layers, treating휂as a global hyperparameter that modulates updates uniformly regardless of layer position or role. This uniform approach creates a fundamental optimization conflict. Low LRs help maintain consensus in foundational layers and stabilize the network, but they limit the ability of specialized layers to adapt to local data. In contrast, high LRs accelerate adaptation in specialized layers but destabilize foundational layers, causing parameter oscillations and degraded conver- gence. This dilemma becomes more pronounced under severe non-IID conditions, where conflicting local optima across heterogeneous nodes further widen the gap between local update intensity and network consensus constraints. As a result, uniform LRs produce unstable parameter trajectories, prolonged convergence measured in communi- cation rounds, and suboptimal model quality. In bandwidth- constrained and latency-sensitive edge deployments, resolv- ing this conflict efficiently is critical: every communication round saved through improved convergence translates directly to reduced bandwidth consumption, lower energy expendi- ture, and faster system response time. Such efficiency gains are especially critical for large-scale IoT and mobile edge network scenarios where communication and energy costs directly impact system viability. Addressing this limitation requires dynamic, layer-wise LR adjustment derived from layer-specific signals that are extracted locally, without global coordination. 3.3. Node-level procedure and layer-wise architecture Standard DFL methods apply a single scalar LR휂uni- formly to all model layers, ignoring their distinct optimization needs. In contrast, the proposed approach assigns a layer- wise LR vector휂 푟 푖 = [휂 푟 푖,1 ,...,휂 푟 푖,퐿 ] at each node푖and round 푟, where each component휂 푟 푖,푙 represents the LR for layer 푙. This vectorized design allows different layers to adopt LRs that better match their roles: foundational layers can use conservative rates to preserve network consensus, while specialized layers can use larger rates to accelerate adaptation to local training. The core challenge is to compute these per- layer LRs adaptively using only information available locally within each DFL round. To enable such computation, the standard DFL node procedure is formalized in terms of three intra-round model states, as illustrated in Fig. 1(a). Within each communication round푟, node푖performs three sequential operations: (1) local training on its private dataset 푖 for퐸epochs, (2) exchange of updated parameters with neighbors푗 ∈ 푖 , and (3) aggre- gation of neighbor models to maintain network consensus. This iterative process repeats over푅communication rounds. FedA2L characterizes the node procedure using three model states within each round. The base state휃 퐵,푟 푖,푙 represents the parameters of layer푙at node푖before local training in round푟. After local training on 푖 for퐸epochs, node푖 reaches the trained state휃 푇,푟 푖,푙 , reflecting local update intensity from node-specific data distributions. This local training transition,휃 퐵,푟 푖,푙 → 휃 푇,푟 푖,푙 , captures how local optimization pushes the layer toward node-specific optima. After model exchange and aggregation with neighbors푗 ∈ 푖 , node푖 obtains the aggregated state휃 퐴,푟 푖,푙 , incorporating neighbor information. This consensus transition,휃 푇,푟 푖,푙 → 휃 퐴,푟 푖,푙 , captures network consensus constraints at the layer level. The aggre- gated state then becomes the base state for the next round (휃 퐵,푟+1 푖,푙 ∶ = 휃 퐴,푟 푖,푙 ), ensuring continuity across communication rounds. 3.4. FedA2L methodology FedA2L dynamically computes and assigns layer-wise LRs from real-time model divergence signals extracted from the three-state model of DFL node procedures. The approach operates entirely locally at each node, requiring no centralized coordination or additional communication, thereby preserving the decentralized architecture of DFL. Fig. 1(b) illustrates the integrated FedA2L method procedure within the standard DFL workflow shown in Fig. 1(a). To provide a formal abstraction of the overall process, FedA2L can be viewed as a per-node mapping from intra- round model states to layer-wise LRs. Specifically, at each node푖and round푟, the method defines the following mapping: 푀 푟 푖 ( 휃 퐵,푟 푖 ,휃 푇,푟 푖 ,휃 퐴,푟 푖 ) → 휂 푟+1 푖 = [ 휂 푖,1 ,휂 푖,2 ,...,휂 푖,퐿 ] 푟+1 (2) This mapping is realized through three sequential steps. First, dual layer-wise metrics are extracted from the intra- round state transitions, capturing both local update intensity and network consensus constraints. Next, these metrics are normalized using recent temporal statistics, filtering transient fluctuations to ensure that LR adjustments respond to persistent optimization trends. Finally, the normalized signals are fused into per-layer adaptive LRs through a bounded mechanism. The following subsections detail each step in turn. 3.4.1. Step 1: Dual metrics for characterizing layer dynamics Two complementary metrics are extracted from the three- state model to capture distinct aspects of layer-wise optimiza- tion dynamics. Weight divergence quantifies local update intensity during the local training, while aggregation stability quantifies network consensus constraints during the network consensus. These metrics provide layer-specific diagnostic signals encoding the competing forces of local learning and network consensus. FedA2L leverages these signals to compute per-layer LRs balancing both requirements. Local update intensity via weight divergence. To quantify how strongly each layer responds to node-specific Truong et al.: Preprint submitted to ElsevierPage 4 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Aggregat ing M odels Training M odel Exc hanging M odels Node Proc edure (a) DFL Framew or k (b) FedA2L M et hod Update lay er- w ise learning rate vec tor Neighbors Net w or k Consensus Transit ion Loc al Training Transit ion Figure 1: (a) Standard DFL framework with three intra-round model states per layer (휃 퐵,푟 푖,푙 ,휃 푇,푟 푖,푙 ,휃 퐴,푟 푖,푙 ). (b) FedA2L integrated into the DFL framework. During each round, the three states of layer푙are used to compute two layer-wise metrics,휎 푟 푖,푙 (Eq. 3) and휁 푟 푖,푙 (Eq. 4), which in turn determine the adaptive layer-wise learning rate vector 휂 푟+1 푖 = [휂 푖,1 ,...,휂 푖,퐿 ] 푟+1 for the next round. data, we examine the parameter change during the local training transition. This transition reveals the degree to which local optimization on node-specific data modifies each layer’s parameters. After node푖completes퐸epochs of local training, transitioning from the base state휃 퐵,푟 푖,푙 to the trained state휃 푇,푟 푖,푙 , the per-layer local update intensity is defined as: 휎 푟 푖,푙 = ‖휃 푇,푟 푖,푙 − 휃 퐵,푟 푖,푙 ‖ 2 ‖휃 퐵,푟 푖,푙 ‖ 2 + 휖 (3) where‖ ⋅ ‖ 2 is the L2-norm and휖is a small constant for numerical stability. Large values of휎 푟 푖,푙 reflects substantial parameter modifications during local training, indicating strong adaptation to node-specific data characteristics. Con- versely, small values indicate stable layer parameters with minimal changes, suggesting the layer is already well-adapted to local data. This normalized ratio design is scale-invariant, preventing bias from layers with larger absolute parameter magnitudes, and provides a principled measure of relative update magnitude across heterogeneous layer architectures. This per-layer perspective on weight divergence adapts concepts from prior FL work [40] into a diagnostic signal that operates at layer granularity rather than at the model level. Network consensus constraints via aggregation stabil- ity. To quantify how well each layer aligns with neighbors during the network consensus transition, we examine the magnitude of parameter changes that occur when node푖ag- gregates trained parameters with its neighbors, transitioning from the trained state휃 푇,푟 푖,푙 to the aggregated state휃 퐴,푟 푖,푙 . The aggregation stability metric measures the proportion of layer parameters that remain stable after aggregation, indicating consensus alignment. Motivated by prior work on consensus mechanisms [41], the aggregation stability for layer푙is defined as: 휁 푟 푖,푙 = 1 |휃 푟 푖,푙 | |휃 푟 푖,푙 | ∑ 푘=1 핀 [ | | | 휃 퐴,푟 푖,푙,푘 − 휃 푇,푟 푖,푙,푘 | | | < 휏 ] (4) where|휃 푟 푖,푙 |denotes the total number of parameters in layer 푙,휃 푇,푟 푖,푙,푘 and휃 퐴,푟 푖,푙,푘 represent the푘-th parameter of layer푙at node푖after local training and after aggregation in round푟, respectively,휏is a stability threshold determining parameter stability tolerance, and핀[⋅]is the indicator function. The metric counts parameters with change magnitude less than휏, normalized by total parameter count. A high휁 푟 푖,푙 value (approaching 1) indicates most param- eters remained stable during aggregation, reflecting strong consensus alignment and minimal disagreement among neigh- bors. Conversely, a low value indicates substantial parameter Truong et al.: Preprint submitted to ElsevierPage 5 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning corrections during aggregation, suggesting weak consensus alignment that may require stabilization in subsequent rounds. 3.4.2.Step 2: Z-score-based normalization for anomaly detection These raw metrics (휎 푟 푖,푙 and휁 푟 푖,푙 ) can fluctuate significantly due to stochastic gradient variations and transient network dynamics, particularly during early training when limited historical information is available. To prevent LR adjustments from reacting to noise rather than genuine optimization dynamics, FedA2L employs two mechanisms that stabilize metric estimation: a warm-up initialization period and tem- poral normalization. For a warm-up initialization, the base LR is applied uniformly across all layers during the initial푅 warm rounds, allowing sufficient metric history to accumulate and enabling early consensus to stabilize. This warm-up period provides a stable foundation for reliable metric estimation, after which FedA2L transitions to temporal normalization to refine LR modulation. After the warm-up period, FedA2L transitions to tem- poral normalization to ensure that LR adjustments respond to persistent trends rather than transient fluctuations. For rounds푟 > 푅 warm , raw metrics are standardized using Z-score normalization computed over a sliding temporal window of the last휌rounds. This process produces the normalized intensity휔 푟 푖,푙 (derived from휎 푖,푙 ) and훿 푟 푖,푙 (derived from휁 푖,푙 ) signals, defined as: 휔 푟 푖,푙 = 휎 푟 푖,푙 − 1 휌 ∑ 푟 푘=푟−휌 휎 푘 푖,푙 STD(휎 푘 푖,푙 푟 푘=푟−휌 ) + 휖 (5) 훿 푟 푖,푙 = 휁 푟 푖,푙 − 1 휌 ∑ 푟 푘=푟−휌 휁 푘 푖,푙 STD(휁 푘 푖,푙 푟 푘=푟−휌 ) + 휖 (6) whereSTD(⋅)computes the sample standard deviation over the last휌rounds. These Z-scores quantify deviation from recent historical patterns in standardized units: large positive values indicate abnormal behavior compared to recent history, while values near zero reflect typical and stable behavior. This normalization ensures that LR adjustments respond to meaningful changes in optimization dynamics rather than absolute metric magnitudes, improving robustness across varying network heterogeneity and data distributions. By expressing each metric as a deviation from its own recent history, this step also places휎 푟 푖,푙 and휁 푟 푖,푙 on a comparable scale across layers, which is examined further in the ablation study of Section 5.6. 3.4.3. Step 3: Adaptive layer-wise learning rate To reflect local update intensity and network consensus constraints simultaneously, FedA2L synthesizes the normal- ized signals into a unified fusion score. This fusion process leverages an exponential transformation to assign higher weights to larger deviations while ensuring that all weights remain positive. The fusion score is computed as: 휆 푟 푖,푙 = 훽 ⋅ 푒 휔 푟 푖,푙 + (1 − 훽) ⋅ 푒 훿 푟 푖,푙 (7) where훽 ∈ [0,1]governs the relative emphasis between local update intensity (first term) and network consensus con- straints (second term). This weighted combination produces a positive-valued fusion score that integrates both optimization signals into a single, interpretable measure. Then, this fusion score is applied to modulate the base LR 휂 0 by combining with a global decay factor훾 푟 = (1+휉⋅푟) −0.5 to produce layer-wise adaptive LRs. The LR for layer푙is defined as: 휂 푟 푖,푙 = 휂 0 ⋅ ( 1 + tanh(log(휆 푟 푖,푙 )) ) ⋅ 훾 푟 (8) where휉controls the convergence speed across rounds, and the logarithmic transformation stabilizes the range of휆 푟 푖,푙 by compressing large variations that could lead to unstable updates. Thetanhfunction provides smooth, bounded mod- ulation, and the decay factor훾 푟 applies polynomial decay, gradually reducing LRs across rounds while still allowing rapid adaptation in the early stages of training. Since휆 푟 푖,푙 > 0,log(휆 푟 푖,푙 ) ∈ ℝandtanh(log(휆 푟 푖,푙 )) ∈ (−1,1) , which implies1 + tanh(log(휆 푟 푖,푙 )) ∈ (0,2). Together with훾 푟 ≤ 1, the effective layer-wise LRs satisfy0 < 휂 푟 푖,푙 < 2휂 0 훾 푟 , so FedA2L cannot arbitrarily amplify the step size. Moreover, the Z-score normalization in Eqs. 5 and 6 is computed over a sliding window of휌rounds; persistent divergence shifts the running mean and variance, driving the normalized scores back toward zero and pulling휆 푟 푖,푙 (and thus the LR multiplier) toward 1 over time. When the fusion score휆 푟 푖,푙 = 1(typical behavior), the modulation factor equals 1, giving휂 푟 푖,푙 = 휂 0 ⋅ 훾 푟 , which aligns with the global decay schedule and represents a steady- state condition. For휆 푟 푖,푙 > 1(high update intensity or weak consensus), the modulation factor exceeds 1, yielding 휂 푟 푖,푙 > 휂 0 ⋅ 훾 푟 and allowing faster adaptation to node-specific patterns. In contrast, when휆 푟 푖,푙 < 1(low update intensity or strong consensus), the LR decreases below the base rate, 휂 푟 푖,푙 < 휂 0 ⋅ 훾 푟 , preserving stable shared representations and network consensus. FedA2L interprets large deviations in the normalized metrics휔 푟 푖,푙 (derived from휎 푟 푖,푙 ) and훿 푟 푖,푙 (derived from휁 푟 푖,푙 ) as evidence that a layer is under-adapting to its local objective in non-IID settings. Intuitively, reducing the LR under large deviations may suppress the magnitude of these normalized metrics; however, it also suppresses necessary adaptation in non-IID settings and limits layer-wise specialization. Therefore, when these normalized metrics indicate unusually strong local updates or weak consensus, FedA2L assigns a temporarily higher LR to that layer to accelerate adaptation under non-IID data. Importantly, the bounded multiplier1 + tanh(log휆 푟 푖,푙 ) ∈ (0,2) , the global decay factor훾 푟 , and the sliding-window Truong et al.: Preprint submitted to ElsevierPage 6 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Z-score normalization ensure that persistent divergence is re-centered and the effective LR is gradually pulled back toward the global schedule. This fine-grained modulation directly addresses the asymmetric layer-wise dynamics by enabling specialized layers to adapt when local signals are strong, while allowing foundational layers to stabilize when consensus signals are strong. 3.5. FedA2L algorithm Algorithm 1: FedA2L method integrated into a DFL framework. 1Initialize: For each node푖 ∈ , initialize base model휃 퐵,푟 푖 at round 푟, local dataset 푖 , neighbor list 푖 , and history lists 휎 푖 ,휁 푖 ← ∅. 2 for 푟 = 1 to 푅 do 3for each node 푖 ∈ do 4휃 푇,푟 푖 ← LocalTraining(휃 퐵,푟 푖 , 푖 ,휂 푟 푖 ); 5Exchange 휃 푇,푟 푖 with neighbors 푖 ; 6휃 퐴,푟 푖 ← Aggregation(휃 푇,푟 푗 푗∈ 푖 ∪푖 ); 7휃 퐵,푟+1 푖 ← 휃 퐴,푟 푖 // FedA2L: Adaptive LR Computation 8for each layer 푙 = 1 to 퐿 do 9휎 푟 푖,푙 ← ‖휃 푇,푟 푖,푙 −휃 퐵,푟 푖,푙 ‖ 2 ‖휃 퐵,푟 푖,푙 ‖ 2 +휖 10휁 푟 푖,푙 ← 1 |휃 푟 푖,푙 | ∑ |휃 푟 푖,푙 | 푘=1 핀 [ | | | 휃 퐴,푟 푖,푙,푘 − 휃 푇,푟 푖,푙,푘 | | | < 휏 ] 11Update metric histories 휎 푖 ,휁 푖 ; 12if round 푟 > 푅 warm then 13Compute Z-scores 휔 푟 푖,푙 using Eq. 5; 14Compute Z-scores 훿 푟 푖,푙 using Eq. 6; 15휆 푟 푖,푙 ← 훽푒 휔 푟 푖,푙 + (1 − 훽)푒 훿 푟 푖,푙 16휂 푟+1 푖,푙 ← 휂 0 (1 + tanh(log(휆 푟 푖,푙 )))훾 푟 17end if 18end for 19end for 20 end for Algorithm 1 presents the complete FedA2L method integrated modularly into a standard DFL training workflow over푅communication rounds. At each round푟, after local training and network consensus, FedA2L executes locally at each node. It computes layer-wise metrics휎 푟 푖,푙 and휁 푟 푖,푙 from locally available model states (휃 퐵,푟 푖 ,휃 푇,푟 푖 ,휃 퐴,푟 푖 ) and then produces the adaptive LR vector휂 푟+1 푖,푙 for the next round. This local-only execution preserves decentralization and does not introduce additional data exposure beyond the standard DFL model-sharing protocol. FedA2L achieves high computational efficiency, with the per-round and per-node computation having complexity (|휃|⋅| 푖 |⋅퐸), where|휃|denotes the model size,| 푖 |is the dataset size, and퐸is the number of local steps. In addition, adaptive computation involves per-layer metrics (휎 푟 푖,푙 ,휁 푟 푖,푙 ) and historical statistics, with complexity(|휃|+퐿휌), where퐿is the number of layers and휌is the normalization window size. As the adaptive computation relies solely on model states rather than training data, this overhead scales only with model depth and normalization history, independent of the local dataset size and training workload. Since| 푖 | ⋅ 퐸 ≫ 퐿휌in practical DFL configurations, the resulting overhead remains limited compared with the dominant cost of local training. Critically, FedA2L introduces zero additional communication overhead, requiring no extra messages, proxy datasets, or centralized coordination, and preserving the decentralized nature and communication efficiency of DFL. 4. Convergence analysis We now analyze FedA2L in the DFL setup of Section 3.1. Recall that the network optimizes min 휃 1 ,...,휃 || (휃 1 ,...,휃 || ) ≜ 1 || ∑ 푖∈ 푖 (휃 푖 ), (9) where 푖 (휃 푖 ) = 피 (푥,푦)∼ 푖 [퓁(휃 푖 ;푥,푦)] is the local loss at node 푖. For the analysis, we also consider the centralized surrogate 푓(휃) ≜ 1 || ∑ 푖∈ 푖 (휃),(10) and track the average model ̄ 휃 푟 ≜ 1 || ∑ 푖∈ 휃 퐵,푟 푖 ,(11) where휃 퐵,푟 푖 denotes the base state of node푖at round푟 (Section 3.3). Within each round, FedA2L modifies only the local step sizes. If the model is decomposed into퐿layers,휃 퐵,푟 푖 = [휃 퐵,푟 푖,1 ;...;휃 퐵,푟 푖,퐿 ], the effective LR for layer푙at node푖in round 푟 is given by Eq. 8: 휂 푟 푖,푙 = 휂 0 훾 푟 ( 1+tanh ( log휆 푟 푖,푙 ) ) , 훾 푟 = (1+휉푟) −1∕2 , (12) where휆 푟 푖,푙 > 0is the fusion score computed from the dual metrics (휎 푟 푖,푙 and휁 푟 푖,푙 ) in Eqs. 3-7. Sincetanh ∶ ℝ→ (−1,1) and 휆 푟 푖,푙 > 0, we always have 0 < 휂 푟 푖,푙 < 2휂 0 훾 푟 for all 푖,푙,푟,(13) i.e., FedA2L keeps all layer-wise LRs strictly positive and uniformly bounded. Assumptions. We adopt standard assumptions from decen- tralized optimization and DFL: A1 (Smoothness). Each 푖 is핃-smooth:‖∇ 푖 (푥) − ∇ 푖 (푦)‖ ≤ 핃‖푥 − 푦‖ for all 푥,푦. Hence 푓 is also 핃-smooth. A2 (Unbiased gradients & bounded variance). Stochas- tic gradients푔 푖 (휃;푏)(where푏denotes a random mini-batch drawn from 푖 ) satisfy 피[푔 푖 (휃;푏)] = ∇ 푖 (휃),피 [ ‖푔 푖 (휃;푏)−∇ 푖 (휃)‖ 2 ] ≤ 휚 2 . A3 (Mixing matrix & spectral gap). Each mixing matrix 푊 푟 = [푤 푟 푖푗 ] 푖,푗 used in the aggregation step (Algorithm 1, Truong et al.: Preprint submitted to ElsevierPage 7 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning line 6) is symmetric and doubly stochastic,푊 푟 ퟏ = ퟏand (푊 푟 ) ⊤ ퟏ = ퟏ, and there exists a spectral gap휙 ∈ (0,1]such that ‖ ‖ ‖ 푊 푟 − 1 || ퟏ ⊤ ‖ ‖ ‖ 2 ≤ 1 − 휙 for all 푟. A4 (Data heterogeneity). There exists퐵 ≥ 0such that for all 휃, 1 || ∑ 푖∈ ‖ ‖ ‖ ∇ 푖 (휃) − ∇푓(휃) ‖ ‖ ‖ 2 ≤ 퐵 2 . A5 (Bounded and slowly varying multipliers). Define layer-wise multipliers푚 푟 푖,푙 via휂 푟 푖,푙 = 푚 푟 푖,푙 (휂 0 훾 푟 ), where휂 0 = 푐∕ √ 푅. There exist constants0 < 푚 min ≤ 푚 max < ∞and Δ 푚 < ∞ such that 푚 min ≤ 푚 푟 푖,푙 ≤ 푚 max ,|푚 푟+1 푖,푙 −푚 푟 푖,푙 | ≤ Δ 푚 , ∀푖,푙,푟. In FedA2L,푚 푟 푖,푙 = 1 + tanh(log휆 푟 푖,푙 ), so0 < 푚 푟 푖,푙 < 2by design, and the Z-score based update over a finite window (Eqs. 5-6) bounds metric volatility, which, when coupled with the boundedtanhtransformation, ensures that|푚 푟+1 푖,푙 −푚 푟 푖,푙 | ≤ Δ푚 for some finite Δ푚. We also measure consensus by the mean-squared dis- agreement 퐷 푟 ≜ 1 || ∑ 푖∈ 퐿 ∑ 푙=1 ‖ ‖ ‖ 휃 퐵,푟 푖,푙 − ̄ 휃 푟 푙 ‖ ‖ ‖ 2 , ̄ 휃 푟 푙 = 1 || ∑ 푖∈ 휃 퐵,푟 푖,푙 . (14) 4.1. Non-convex convergence We first consider the general non-convex setting. Let 휂 푟 min ≤ 휂 푟 푖,푙 ≤ 휂 푟 max denote lower/upper bounds on the effective step sizes in round 푟, induced by (13) and A5. Theorem 1 (Non-convex FedA2L convergence). Assume A1-A5 and one local SGD epoch per round at each node (Algorithm 1 with퐸 = 1). Suppose the effective step sizes satisfy0 < 휂 푟 min ≤ 휂 푟 푖,푙 ≤ 휂 푟 max with휂 푟 max 퐿 sufficiently small for all푟, which is the standard small-step condition ensuring that higher-order perturbation terms remain controlled under Lipschitz smoothness. Then, for any 푅 ≥ 1, ∑ 푅−1 푟=0 휂 푟 min 피 [ ‖∇푓( ̄ 휃 푟 )‖ 2 ] ∑ 푅−1 푟=0 휂 푟 min ≤ 2 ( 푓( ̄ 휃 0 ) − 푓 ⋆ ) ∑ 푅−1 푟=0 휂 푟 min + 퐶 1 퐿휂 max 휚 2 || + 퐶 2 퐿 2 휂 2 max 퐵 2 휙 2 + 퐶 3 퐿휂 2 max Δ 2 푚 휙 2 , (15) where푓 ⋆ = inf 휃 푓(휃),휂 max = max 0≤푟<푅 휂 푟 max , and 퐶 1 ,퐶 2 ,퐶 3 > 0 are constants independent of 푅. Sketch of proof. Using핃-smoothness, one shows that the average model satisfies 피[푓( ̄ 휃 푟+1 )] ≤ 피[푓( ̄ 휃 푟 )] − 휂 푟 min 2 피‖∇푓( ̄ 휃 푟 )‖ 2 + 푂 ( (휂 푟 max ) 2 ( 휚 2 + 퐵 2 + 피퐷 푟 )) . A separate recursion for the disagreement퐷 푟 follows from the spectral gap휙of푊 푟 : consensus contracts by a factor (1 − 휙)each round, but is driven by stochastic gradient noise휚 2 , heterogeneity퐵 2 , and the per-round driftΔ 푚 of the multipliers, which remains bounded due to the multiplier construction in A5 and the bounded tanh transformation in Eq. 8. Summing the descent inequality over푟 = 0,...,푅−1, bounding피퐷 푟 by the steady state of this consensus recursion, and dividing by ∑ 푅−1 푟=0 휂 푟 min yields (15). Diminishing step sizes. For the theoretical analysis, we set훾 푟 = 1and use the budget-dependent base LR휂 0 = 푐∕ √ 푅from A5. Combined with bounded multipliers푚 푟 푖,푙 ∈ [푚 min ,푚 max ](A5), this gives round-independent effective step sizes 휂 푟 푖,푙 = 푚 푟 푖,푙 휂 0 , so 휂 max = 푚 max 푐 √ 푅 = 푂 ( 1 √ 푅 ) , 푅−1 ∑ 푟=0 휂 푟 min ≥ 푅 ⋅ 푚 min 푐 √ 푅 = Ω( √ 푅). The three error terms in(15)then satisfy퐶 1 퐿휂 max = 푂(1∕ √ 푅)and퐶 2 ,퐶 3 ∝휂 2 max = 푂(1∕푅), so all terms vanish as 푅→ ∞. Substituting into Theorem 1 yields ∑ 푅−1 푟=0 휂 푟 min 피‖∇푓( ̄ 휃 푟 )‖ 2 ∑ 푅−1 푟=0 휂 푟 min = 푂 ( 1 √ 푅 ) , matching the standard푂(1∕ √ 푅)non-convex rate of decen- tralized SGD. FedA2L does not worsen the convergence order; the bounded multipliers푚 푟 푖,푙 only modify the constants 퐶 1 ,퐶 2 ,퐶 3 . In practice we employ the diminishing schedule 훾 푟 = (1 + 휉푟) −1∕2 to improve empirical stability. 4.2. Strongly convex case If푓is additionally휇-strongly convex and the effective step sizes are kept constant over rounds,휂 푟 푖,푙 ∈ [휂 min ,휂 max ] with휂 max sufficiently small, the same arguments with a Lyapunov function푉 푟 = 피[푓( ̄ 휃 푟 )−푓 ⋆ ]+훾 피[퐷 푟 ]show that FedA2L preserves the linear convergence of decentralized SGD: for suitable 훾 > 0, 피 [ 푓( ̄ 휃 푅 ) − 푓 ⋆ ] ≤ 휌 푅 0 ( 푓( ̄ 휃 0 ) − 푓 ⋆ ) + 푂 ( 퐿 휂 max 휚 2 || + 퐿 2 휂 2 max 퐵 2 휙 2 + 퐿휂 2 max Δ 2 푚 휙 2 ) , (16) with contraction factor휌 0 ∈ (0,1)determined jointly by휇 and the spectral gap휙. Thus, in the constant step-size regime, FedA2L does not degrade the qualitative linear rate of the underlying DFL algorithm; it only changes the constants through bounded, layer-wise multipliers. This strongly convex result is provided for completeness in the constant step-size regime. Since our experiments adopt the diminishing schedule in Eq. 12, the main theoretical implication for the practical setup is the preservation of the standard non-convex convergence rate established above. Truong et al.: Preprint submitted to ElsevierPage 8 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning 5. Experiments This section presents the evaluation setup and perfor- mance analysis of FedA2L across image classification and time-series forecasting (TSF) tasks, covering multiple bench- mark datasets, model architectures, DFL algorithms, and network configurations. The evaluation considers five com- plementary dimensions. Convergence speed is measured by communication rounds required to reach a target accuracy. Computational efficiency is measured by total time to reach the same target. Model quality is assessed by best test accuracy on classification tasks and MSE on regression tasks, capturing whether convergence gains carry through to model performance. Robustness is examined under varying data heterogeneity, network scales, and sparse topologies. Finally, ablation and sensitivity studies isolate the contribution of each methodological component and examine how key design choices affect convergence behavior. The corresponding analyses are presented in Sections 5.2-5.6. 5.1. Experimental setup 5.1.1. Datasets Experimental evaluation was conducted on five bench- mark datasets spanning image classification and TSF. For image classification, three standard benchmarks are used: CIFAR-10, CIFAR-100, and TinyImageNet. CIFAR-10 and CIFAR-100 [42] contain 60,000 32×32 color images across 10 and 100 classes, respectively, enabling assessment of both coarse and fine-grained classification tasks. TinyImageNet [43] comprises 100,000 64×64 color images across 200 classes, providing a more challenging and realistic benchmark for evaluating scalability on larger-scale vision tasks. To emulate realistic heterogeneous data distributions (non-IID data distribution), a Dirichlet distribution with parameter훼was used to partition datasets across nodes. Each node푖receives a subset of samples, where훼controls heterogeneity level. Lower훼values correspond to severe non-IID conditions where each node receives highly skewed class distributions, while higher values yield more balanced distributions approaching IID. Default experiments used 훼 = 0.1, with robustness analysis varying 훼 ∈ 0.01,0.5. For TSF, two real-world benchmarks are used: Exchang- eRate [44] and BeijingAirQuality [45]. ExchangeRate con- tains 7,588 daily exchange rate records across 8 currencies, where each currency corresponds to a node. BeijingAirQual- ity comprises hourly air quality measurements from 12 monitoring stations across Beijing, with 11 variates per node. Both use an input horizon of 96 and a prediction horizon of 96 with naturally defined node partitions (i.e., no Dirichlet-based partitioning). These datasets reflect real-world distributed time-series data with temporal dependencies across nodes. 5.1.2. Models Model architectures cover varying complexity levels. For image classification, a 4-layer convolutional neural network (CNN) for lightweight evaluation, ResNet-18 for moderate-scale tasks, and ResNet-34 for deeper network eval- uation. For TSF, we use Timer [46], an 18-layer transformer- based forecasting model. This diversity of models enables a comprehensive evaluation of FedA2L’s effectiveness across classification and regression settings with architectures of different depth and complexity. 5.1.3. DFL configuration All experiments were conducted with|| = 10nodes ar- ranged in a fully connected P2P topology, using synchronous communication unless otherwise specified. Each node per- forms local training for퐸 = 1epoch per communication round, with initial LR휂 0 = 0.01. Classification experiments train for푅 = 500communication rounds, and convergence is evaluated by measuring the rounds required to reach predefined target test accuracy. The target accuracies are selected as stable and practically meaningful convergence levels that the majority of baseline methods can reach and sustain, reflecting genuine learning progress in the distributed setting. TSF experiments train for푅 = 100communication rounds, and convergence is evaluated by the rounds required to reach a target MSE threshold. All experiments are con- ducted on a server equipped with an NVIDIA GeForce RTX 4090 GPU and 126 GB of RAM, running PyTorch 2.5.0 under Python 3.12. All results represent mean values across 5 independent runs with different random seeds to ensure statistical robustness. 5.1.4. Baseline methods FedAvg [3] (vanilla averaging baseline), FedProx [14] (proximal term for heterogeneity), FedYogi [22] (adaptive second-moment optimization), FedNTD [47] (knowledge distillation approach), FedAWA [36] (adaptive aggregation method), DFedSAM [48] (sharpness-aware minimization for flat minima under non-IID conditions), and DFedHPO [24] (decentralized hyperparameter optimization). DFedHPO con- ducts a one-time decentralized search to provide a single optimal fixed LR configuration before training begins, while FedA2L continuously adapts layer-wise LRs during training. As a result, the fixed LR from DFedHPO serves as an initial- ization, and subsequent optimization is governed by FedA2L. Accordingly, DFedHPO is evaluated as a standalone reference baseline, using its searched LR as a fixed configuration with- out additional tuning procedures or schedulers. This enables a direct comparison between static pre-training hyperparameter search and dynamic layer-wise adaptation during training. These baselines represent classical, regularized, adaptive, network-aware, geometric, and hyperparameter optimization strategies in DFL, providing a solid foundation for compari- son with FedA2L, a layer-wise adaptive LR method. For each base algorithm, three distinct LR strategies are evaluated, with the exception of DFedHPO, which is assessed only under its searched LR configuration as described above: •Vanilla (Unchanged LR): Single constant LR main- tained throughout training, serving as the reference for comparison. Truong et al.: Preprint submitted to ElsevierPage 9 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning •Scheduler-based: Global LR adjusted per round ac- cording to established schedulers: StepLR (exponential decay), CosineAnnealingWarmRestarts (CAWR) with periodic warm restarts [49], OneCycleLR (OCLR) [50] with triangular scheduling, and Hyperbolic scheduler [51] providing efficient decay. •FedA2L (Proposed): Layer-wise adaptive LRs dynam- ically computed per round. These three strategy categories encompass the spectrum from static (vanilla) through globally adaptive (schedulers) to locally adaptive layer-wise (FedA2L), providing a com- prehensive evaluation framework. This evaluation structure enables fair assessment: unchanged and scheduler baselines provide performance bounds for global (non-adaptive) strate- gies, while FedA2L demonstrates the advantage of per-layer adaptation. 5.2. How fast does FedA2L converge? 5.2.1. Comparison across models and datasets FedA2L demonstrates consistent acceleration across diverse models and datasets, with the performance gain increasing as model complexity and dataset diversity in- crease. Across the 36 scheduler-strategy settings in Table 2, FedA2L achieves the fastest convergence in 32 cases and the second-fastest in 3, confirming robust and generalizable performance. FedA2L also converges faster than DFedHPO across all evaluated configurations, confirming the advantage of dynamic layer-wise adaptation over static hyperparameter search. The Improv. rows quantify the percentage reduction in communication rounds relative to the best-performing baseline per setting, providing a direct measure of practical efficiency gain. Across ResNet-18 and ResNet-34 architectures, FedA2L consistently reduces convergence rounds across all seven algorithms, with the strongest gains on TinyImageNet ranging from 2.77% under DFedSAM to 59.34% under FedYogi. Against the best scheduler, FedA2L achieves further reduc- tions on all ResNet settings. Additionally, FedA2L exhibits notably low variance across independent runs: on TinyIma- geNet with ResNet-34 under FedAvg, FedA2L converges in71±1.25rounds compared to148±38.81for CAWR, reflecting stable and reproducible behavior that is important for deployment in resource-constrained edge environments. On CNN architectures, FedA2L achieves positive gains on CIFAR-10 across most algorithms, reaching up to 28.75% improvement under DFedSAM, while ranking second in the remaining settings where the base algorithm already partially captures layer-level variance. In all such cases, FedA2L still outperforms the vanilla baseline, confirming that the layer- wise mechanism does not destabilize training. The reasons for these interactions are discussed in the following subsection. 5.2.2. Comparison across DFL algorithms A key property of FedA2L is its algorithmic orthog- onality, enabling seamless integration with existing DFL protocols without modifying the core aggregation logic. This is particularly important for practitioners deploying mature, established DFL systems. Table 2 shows consistent acceleration across six DFL algorithms, confirming that this property holds across fundamentally different DFL design philosophies under diverse update mechanisms. Under FedProx, which applies proximal regularization to reduce client drift, FedA2L still achieves 2.24×speedup on CNN CIFAR-10 (127 vs. 285 rounds), confirming that layer-wise rate adaptation targets an orthogonal optimization dimen- sion to loss-based regularization. Under FedAWA, which reweighs aggregation contributions based on model similarity, FedA2L reaches 72 rounds versus 356 rounds on ResNet-34 TinyImageNet (4.94×speedup), demonstrating that adaptive aggregation and adaptive layer-wise LRs are complementary rather than competing mechanisms. The only exceptions arise in two CNN configurations under FedNTD and two CIFAR-100 settings under FedYogi. In these cases, the base algorithm already partially captures layer-level variance through knowledge distillation in Fed- NTD and per-parameter moment estimation in FedYogi, re- ducing the available margin for further layer-wise modulation. This limits, but does not eliminate, the benefit of additional layer-wise adaptation. Nonetheless, FedA2L still outperforms the vanilla baseline in all four cases, confirming that the layer- wise mechanism remains compatible with the base protocol even when divergence signals are partially attenuated. 5.2.3. Comparison across learning rate strategies Global LR schedulers address temporal heterogeneity by adjusting rates over training rounds. However, they cannot address spatial heterogeneity, which refers to the differing optimization needs across model layers. FedA2L consistently outperforms these scheduler-based baselines (Table 2) by leveraging local layer signals to detect each layer’s specific optimization needs during each communication round. This distinction becomes critical in deep heterogeneous mod- els. With ResNet-18 on CIFAR-100 (45% target accuracy), FedA2L converges in 177 rounds compared to 491 rounds for the vanilla baseline under FedAvg (2.77×speedup) and 197 rounds for the best scheduler, CAWR (10.15% further improvement). On TinyImageNet with ResNet-18, the advantage becomes more pronounced. Under FedNTD, all schedulers fail to converge, whereas FedA2L reaches the target in 175 rounds, demonstrating that global scheduling is insufficient under the combined effects of deep layer heterogeneity and severe non-IID conditions. 5.2.4. Comparison across model architectures The effectiveness of FedA2L scales consistently with architectural depth, following a pattern visible across all algorithm rows in Table 2. On ResNet-34, the deepest architecture evaluated, FedA2L delivers the strongest gains across all algorithms. On TinyImageNet, improvements range from 2.77% under DFedSAM to 59.34% under FedYogi, while the vanilla baseline fails to converge in several settings where FedA2L successfully reaches the target accuracy. This pattern confirms that deep feature hierarchies combined with Truong et al.: Preprint submitted to ElsevierPage 10 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Table 2 Communication rounds to target accuracy in DFL (FedA2L vs. baselines). Results use a sliding average (window = 10). Bold: fastest; underline: second-fastest; “_”: no convergence. ‘1 푠푡 count’ = number of fastest cases. DFedHPO is reported only under its searched LR, as it does not incorporate additional LR scheduling. ModelCNNResNet-18ResNet-341 푠푡 Count DatasetCIFAR-10 CIFAR-100CIFAR-100TINYCIFAR-100TINY AlgorithmMethod64%29%45%35%41%30% FedAvg Vanilla374±38.93 29±22.07491±53.13_406±50.26_0 StepLR 190±23.2527±0.4309±33.86 314±44.85308±8.52 214±29.170 CAWR191±40.63 23±0.64197±46.2_199±23.84 148±38.811 OCLR165±17.8158±1.09266±11.23 450±9.29142±9.94225±12.530 Hyperbolic221±17.8227±36.8432±37.5__256±32.930 FedA2L 136±8.2123±0.49177±6.97 187±15.83120±12.171±1.256 Improv.17.57%0.0%10.15%40.44%15.49%52.03% FedProx Vanilla285±51.0425±0.8__389±75.5_0 StepLR 144±14.625±0.8__406±66.09_0 CAWR148±64.78 23±0.63297±23.63_244±41.62 194±22.351 OCLR 149±29.9748±0.8247±11.38447±4.72132±10182±11.920 Hyperbolic 213±27.625±0.75__266±38.96 251±52.890 FedA2L127±24.89 23±0.49175±15.14 185±7.88115±3.2482±0.826 Improv.11.81%0.0%29.15%58.61%12.88%54.95% FedYogi Vanilla311±34.9827±1.79__367±51.4308±5.630 StepLR163±29.9227±1.79__306±51.39 207±27.810 CAWR189±35.88 23±0.63198±48.88 248±35.39198±67.97 195±21.822 OCLR161±22.0149±0.98273±8.45 454±11.23152±17.69182±12.030 Hyperbolic 214±31.2627±1.09361±27.6_324±42.15 251±72.240 FedA2L141±25.8225±0.49210±8.7240±6.98134±7.67874±1.674 Improv.12.42%-8.69%-6.06%3.22%11.84%59.34% FedNTD Vanilla225±68.6524±0.89242±29.17_239±45.26_0 StepLR141±27.9124±0.89133±40.57_209±51_1 CAWR 190±72.5221±0.4143±3.06_136±20.44149±1.51 OCLR150±17.7447±1.02161±4.21_99±1.94124±3.290 Hyperbolic218±24.6824±1.2175±16.02___0 FedA2L167±17.29 23±0.49122±7.81 175±17.1588±1.0272±1.694 Improv. -18.44%-9.5%8.3%100%11.11%41.94% FedAWA Vanilla387±51.94 30±29.67__264±75356±74.960 StepLR195±14.03 27±53.83409±49.50_202±41.41 201±15.920 CAWR243±77.9823±0.8247±68.22 248±41.42197±24.31 146±1.251 OCLR 174±18.1148±1.02244±17.68440±29.56139±3.76209±0.820 Hyperbolic 218±13.9526±1.74372±86.86_286±70.81 495±113.50 FedA2L152±14.6623±0.8177±7.49195±8.24112±3.2672±0.946 Improv.12.64%0.0%27.45%21.37%19.42%50.68% DFedSAM Vanilla346±55.8736±5.01167±17.59 254±17.59117±10.2172±2.160 StepLR160±24.0736±60.81142±22.01 207±48.56101±7.7673±4.240 CAWR197±24.09 27±0.4797±22.2384±1.2592±2.6286±1.251 OCLR170±29.5853±0.8295±1.794±2.4585±1.8973±0.470 Hyperbolic196±19.1331±0.47161±0.47175±55.595±27.82 175±55.750 FedA2L114±2.3627±0.4988±1.2581±2.8580±2.6270±0.826 Improv.28.75%0.0%7.37%3.57%5.88%2.77% DFedHPOVanilla170±10.2760±0.82__243±10.34 141±45.9 large class diversity produce strong layer-wise divergence signals, precisely the condition under which FedA2L’s per- layer rate control is most effective under non-IID settings. On ResNet-18, the reduced depth and lower class diversity yield consistent but more modest improvements across all base algorithms on both CIFAR-100 and TinyImageNet. On CNN, FedA2L acts primarily as a training stabilizer, achieving a2.75×speedup over vanilla under FedAvg on CIFAR-10 Truong et al.: Preprint submitted to ElsevierPage 11 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Table 3: Comparing time to reach 45% target accuracy (total time in minutes) on CIFAR-100 using ResNet-18 (Dirichlet훼 = 0.1). AlgorithmFedAvgFedProxFedYogiFedNTDFedAWADFedSAM Method Time per round (s) Total Time (min) Time per round (s) Total Time (min) Time per round (s) Total Time (min) Time per round (s) Total Time (min) Time per round (s) Total Time (min) Time per round (s) Total Time (min) Vanilla28.43232.6330.02_28.62_32.69131.8528.18_46.61129.73 StepLR 28.23145.3830.02_28.60_32.4671.9528.38193.4646.2367.68 CAWR28.1592.4129.68146.9228.8295.0932.8278.2228.48117.2346.4746.59 OCLR 28.15124.7829.78122.6028.62130.2232.3586.8128.15114.4846.5145.31 Hyperbolic28.15202.7130.29_28.70172.6532.9496.0828.48176.5946.7477.00 FedA2L 29.2886.3831.4791.7930.73107.5534.6470.4430.4189.7047.0545.07 50100150200250 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (a) FedAvg 50100150200250 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (b) FedProx 50100150200250 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (c) FedYogi 306090120150 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (d) FedNTD 50100150200250 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (e) FedAWA 306090120150 Time (mins) 0 10 20 30 40 50 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (f) DFedSAM Figure 2: Time to reach 45% target accuracy on CIFAR-100 using ResNet-18 (Dirichlet 훼 = 0.1) across six DFL algorithms. while narrowing scheduler gaps rather than delivering the strongest speedups in every configuration. These results indi- cate that FedA2L’s benefits increase with feature hierarchy depth and representation complexity, rather than with network depth alone. 5.2.5. Comparison of convergence time FedA2L consistently ranks within the top two in total convergence time across all six evaluated algorithms, achiev- ing the lowest total time in five cases. As shown in Table 3, FedA2L incurs a modest increase in time per round relative to vanilla, for example 29.28s versus 28.43s under FedAvg, due to layer-wise metric computation. However, this overhead is consistently outweighed by the reduction in total training time, since fewer rounds are needed to reach the target accuracy. To reach the 45% target accuracy, FedA2L converges in 177 rounds (86.38 minutes), whereas the vanilla baseline requires 491 rounds (232.63 minutes), yielding a 2.69×reduction in total time despite the higher per-round cost. Fig. 2 further illustrates this trend across all six DFL algorithms, showing that FedA2L consistently achieves the lowest total time, with the most pronounced gains under FedProx and FedAWA, where FedA2L reduces total training time by 25.1% (91.79 vs. 122.60 minutes) and 21.65% (89.70 vs. 114.48 minutes) relative to the best competing base- line, respectively (Table 3). These results demonstrate that FedA2L’s layer-wise adaptation provides measurable wall- clock efficiency across diverse DFL protocols, supporting its practicality for time-sensitive industrial IoT, cyber-physical systems, and decentralized intelligence deployments where both communication and computational costs are critical. 5.3. How well does FedA2L perform in accuracy? Beyond accelerating convergence, FedA2L achieves the highest test accuracy across diverse settings, maintaining both model quality and training stability under non-IID conditions. Fig. 3 compares the highest achieved accuracy as model depth and dataset complexity increase. FedA2L reaches the Truong et al.: Preprint submitted to ElsevierPage 12 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning FedAvgFedProxFedYogiFedNTDFedAWADFedSAM 40 45 50 55 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (a) ResNet-18 on CIFAR-100 FedAvgFedProxFedYogiFedNTDFedAWADFedSAM 40 45 50 55 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (b) ResNet-34 on CIFAR-100 FedAvgFedProxFedYogiFedNTDFedAWADFedSAM 25 30 35 40 45 Accuracy (%) Vanilla StepLR CAWR OCLR Hyperbolic FedA2L (c) ResNet-34 on TinyImageNet Figure 3: Highest test accuracy achieved across algorithms, models, and datasets under non-IID partitioning (Dirichlet 훼 = 0.1). Table 4 Convergence performance of FedA2L vs. baselines in DFL frameworks. Rounds to target accuracies varying data heterogeneity, scalability, and network topologies. Results are smoothed using a sliding average (window size= 15). Formatting: Bold = fastest; Underline = second-fastest; Dash = failure to converge. SettingHeterogeneityScalabilityTopology Target Accuracy40%45%42%40%33%45% Algorithm MethodDir(0.01) Dir(0.5)20 nodes 30 nodesRing K-Connected FedAvg Vanilla300354____ StepLR206109____ CAWR194136398348__ OCLR298132341366493296 Hyperbolic315248____ FedA2L 181104278339318216 FedAWA Vanilla380406____ StepLR209125____ CAWR193140_451_299 OCLR316135352379497296 Hyperbolic314226____ FedA2L187106273334319211 highest accuracy in the majority of configurations across all six algorithms. The accuracy improvement stems from the same underlying mechanism as convergence improvement, by dynamically balancing layer-specific roles based on local divergence metrics. FedA2L avoids the trade-off inherent in uniform LRs where high rates destabilize and low rates under-adapt, maintaining both leading accuracy and stable convergence across model architectures and datasets. Overall, fine-grained per-layer control contributes to consistently higher accuracy without compromising training stability in heterogeneous decentralized environments. 5.4. How robust is FedA2L in practical DFL settings? To validate practical applicability beyond baseline con- ditions, this section evaluates FedA2L’s robustness under severe non-IID data distributions, increased node scalability, and challenging network topologies (Table 4). Across these demanding conditions, FedA2L consistently reduces commu- nication rounds required to reach target accuracy, demonstrat- ing that adaptive layer-wise LRs remain effective and stable as DFL training conditions become more challenging. 5.4.1. Comparison under heterogeneous data distributions The convergence results under heterogeneous data dis- tributions (Table 4, heterogeneity columns) underscore the robust capability of FedA2L to mitigate the severe impact of non-IID data. Under the highly skewed훼 = 0.01conditions that create extreme optimization challenges, baseline perfor- mance degrades sharply due to client drift where inconsistent local gradients pull the model away from the optimum. In con- trast, FedA2L achieves the fastest convergence (181 rounds for FedAvg, 187 for FedAWA) and maintains a substantial 1.66×speedup over the vanilla baseline. This result validates that the layer-wise adaptive mechanism remains effective, enabling a stable and accelerated convergence that is crucial in practical non-IID settings. 5.4.2. Comparison across node scalability The evaluation of node scalability (Table 4, scalability columns), from 20 to 30 clients, confirms that the perfor- mance benefits of FedA2L are consistent in increasingly com- plex DFL environments, with more distributed interactions, even as the consensus challenge grows. Larger networks Truong et al.: Preprint submitted to ElsevierPage 13 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning introduce more gradients to aggregate, amplifying noise and slowing consensus propagation. FedA2L sustains its performance advantage over all baselines, requiring 278 and 273 rounds under FedAvg and FedAWA, respectively, for 20 nodes, and 339 and 334 rounds for 30 nodes. These results demonstrate that FedA2L effectively mitigates this increased noise and the ensuing consensus delays inherent in larger peer-to-peer networks, allowing layer-wise adaptive LRs to remain effective without proportional degradation as network size increases. 5.4.3. Comparison across network topologies The evaluation across network topologies (Table 4, topol- ogy columns), particularly the challenging sparse Ring topol- ogy, provides compelling evidence of FedA2L’s resilience to communication bottlenecks. The Ring topology is structurally demanding due to its high diameter and severely restricted neighbor interactions, causing most baseline methods to fail to converge entirely. Despite these severe structural constraints, FedA2L successfully converges, achieving a remarkable1.55×speedup (318 rounds for FedAvg, 319 for FedAWA) compared to the best-performing converging baseline (OCLR). This demonstrates that layer-wise adaptive LRs effectively compensate for structural communication limitations, enabling faster convergence even under severely constrained neighbor interactions. 5.5. Results on TSF Table 5 reports communication rounds and total time to reach target MSE on both datasets under FedAvg and FedAWA. FedA2L achieves the fastest convergence across all settings. On ExchangeRate, FedA2L reaches target MSE in 11 rounds versus 16 rounds for the best scheduler (OCLR) under FedAvg, a1.45×speedup that translates to 31.25% reduction in communication rounds and 26.37% reduction in total time. Under FedAWA, the gap widens: 11 versus 17 rounds (1.55×speedup, 35.29% fewer rounds, 31.67% less time). On BeijingAirQuality, the improvement is more modest: 18 versus 22 rounds under FedAvg (13.64% fewer rounds, 9.26% less time) and 20 versus 22 rounds under FedAWA (9.1% fewer rounds, 5.91% less time). The smaller gains indicate that highly multivariate, high-frequency data (11 variates, hourly) in BeijingAirQuality exhibits less layer- wise optimization conflict than univariate, low-frequency data (1 variate, daily) in ExchangeRate, where per-layer LR tuning has more room to improve. In all cases, FedA2L’s modest per-round overhead (28-31 s versus 28-29 s for the baselines) is outweighed by the reduction in total rounds required, confirming the practical benefit of adaptive layer- wise LRs across both temporal forecasting settings. 5.6. Hyperparameter sensitivity analysis and ablation study We conduct a two-part analysis to examine the design choices and configuration parameters of FedA2L. First, we analyze the dynamics and stability of the layer-wise LR feed- back mechanism to verify that the adaptive behavior is driven by meaningful optimization signals rather than noise. Second, Table 5 Communication rounds and total time to reach target MSE (0.0575 for ExchangeRate, 4.3582 for BeijingAirQuality) for FedA2L and baseline learning rate strategies. SettingExchangeRateBeijingAirQuality Algorithm MethodRound Total time (s) Round Total time (s) FedAvg Vanilla471170.46481195.36 StepLR551365.63__ CAWR892180.64381376.40 OCLR 16393.8622806.05 Hyperbolic882158.42421519.98 FedA2L11289.9818731.38 Improv.31.25% 26.37%13.64%9.26% FedAWA Vanilla581391.18401547.33 StepLR_2121.43__ CAWR982360.4341307.71 OCLR17423.3922852.10 Hyperbolic_2379.8381470.10 FedA2L11307.3120801.69 Improv.35.29% 31.67%9.1%5.91% we conduct a hyperparameter sensitivity and component ablation analysis to evaluate how the design parameters and the dual-metric structure affect both convergence speed and model performance. 5.6.1. LR feedback dynamics and stability To inspect the behavior of FedA2L, we analyze the joint evolution of the layer-wise LR and the local divergence metrics. Fig. 4 shows the top three layers ranked by their highest local divergence metrics in a representative config- uration (ResNet-18, CIFAR-100, FedAvg at훼 = 0.1). For each selected layer, we plot across rounds the effective LR 휂 푟 푖,푙 together with the normalized weight divergence휎 푟 푖,푙 and aggregation instability 1 − 휁 푟 푖,푙 1 . Across all three layers, we observe large spikes in both휎 푟 푖,푙 and1−휁 푟 푖,푙 during early rounds, indicating strong local updates and non-negligible corrections by neighbors. In response, the corresponding LRs temporarily increase but remain within a narrow range and then decay smoothly over time. As training progresses, the divergence and aggregation-instability signals rapidly diminish toward zero, and휂 푟 푖,푙 converges toward a stable plateau determined by the global decay factor훾 푟 . Importantly, even when a layer exhibits large divergence or strong correction, the LR does not grow without bound; transient boosts are followed by reductions, consistent with the bounded1 + tanh(log(휆 푟 푖,푙 ))modulation and the self- correcting Z-score normalization described in Section 3.4.3. These trajectories show that FedA2L briefly increases LRs when local signals are strong, enabling short-term beneficial divergence in specialized layers under non-IID data, while the normalization and decay mechanisms gradually return the rates toward the global schedule and prevent runaway behavior. 1 We use1−휁 푟 푖,푙 instead of aggregation stability (휁 푟 푖,푙 ) so that higher values consistently indicate stronger neighbor corrections, aligning the direction of both instability signals. All metric curves are linearly rescaled to[0,1]for visualization. Truong et al.: Preprint submitted to ElsevierPage 14 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning 100200300400500 No. of Round 0.00 0.25 0.50 0.75 1.00 1.25 Learning Rate ( ) 1e2 LR () Weight divergence () Aggregation instability (1 - ) 0.0 0.2 0.4 0.6 0.8 1.0 Normalized metrics (a) Highest divergence layer 100200300400500 No. of Round 0.00 0.25 0.50 0.75 1.00 1.25 Learning Rate ( ) 1e2 LR () Weight divergence () Aggregation instability (1 - ) 0.0 0.2 0.4 0.6 0.8 1.0 Normalized metrics (b) Second highest divergence layer 100200300400500 No. of Round 0.00 0.25 0.50 0.75 1.00 1.25 Learning Rate ( ) 1e2 LR () Weight divergence () Aggregation instability (1 - ) 0.0 0.2 0.4 0.6 0.8 1.0 Normalized metrics (c) Third highest divergence layer Figure 4: LR feedback dynamics for the three layers with the highest local divergence (ResNet-18, CIFAR-100, FedAvg, Dirichlet 훼 = 0.1). The plots show the layer-wise learning rate휂 푟 푖,푙 (left axis), together with the normalized weight divergence휎 푟 푖,푙 and aggregation instability 1 − 휁 푟 푖,푙 (right axis), over communication rounds. Table 6 Hyperparameter sensitivity and ablation analysis of FedA2L on CIFAR-100 (훼 = 0.1). Results are reported in terms of communication rounds to reach the target accuracy and best test accuracy (%). ModelMetric 휏푅 warm 휌휉훽 10 −2 10 −3 10 −4 102040510200.050.10.30.00.61.0 CNN Rounds@29% 222123243829232825302924232123 Best accuracy33.39 33.39 33.3133.11 32.59 31.1433.29 32.97 32.8832.33 32.87 33.3332.58 33.63 32.46 ResNet-18 Rounds@45%173171192462207175197176179179177_176175180 Best accuracy48.46 48.81 48.5345.46 47.78 48.6346.94 48.78 48.6948.73 48.86 44.9049.23 49.50 49.10 ResNet-34 Rounds@45%148146153249167156160156156168157_158150159 Best accuracy48.67 48.67 48.6146.52 48.63 48.8748.27 48.78 48.8648.73 48.86 44.2947.22 48.63 47.22 5.6.2. Hyperparameter sensitivity and configuration To assess robustness and reproducibility, we conduct a sensitivity study on CIFAR-100 across CNN (29% target), ResNet-18 and ResNet-34 (45% target), varying each of the five FedA2L design parameters (푅 warm ,휌,휉,휏, and 훽) independently while holding the others at their defaults. Table 6 reports both convergence efficiency and best test accuracy for each parameter, providing a joint view of how each parameter affects both dimensions of performance. Fig. 5 presents the full accuracy trajectories under varying훽, where the extreme settings훽 = 0and훽 = 1serve as direct ablations of the individual휁and휎components, respectively, enabling assessment of their contribution to both convergence behavior and final model accuracy. Influence of stability threshold (휏). The stability thresh- old휏in Eq. 4 sets the tolerance boundary for counting a parameter as stable during aggregation, directly controlling the resolution of the consensus signal휁. Too small a value makes휁uninformative by classifying nearly all parameter changes as unstable; too large a value loses discrimination between genuine consensus and client drift. As shown in Table 6, CNN is negligibly affected across all tested values, while휏 = 10 −4 degrades convergence on ResNet-18 and ResNet-34 from 171 to 192 rounds and from 146 to 153 rounds respectively, confirming that휁loses its discriminative power at excessively fine thresholds. A stable operating region exists between10 −2 and10 −3 , and휏 = 10 −3 achieves competitive accuracy across all architectures, confirming that convergence efficiency and model quality are jointly optimized. We adopt휏 = 10 −3 across all model architectures. Influence of warm-up rounds (푅 warm ). The warm-up period푅 warm defines the number of initial rounds during which FedA2L applies the base LR uniformly across all layers before adaptive modulation begins. Its sensitivity scales with model depth, where shallow architectures accumulate reliable divergence statistics within a few rounds, whereas deeper models require a longer history for per-layer signals to stabilize. As shown in Table 6, reducing푅 warm to 10 rounds degrades convergence from 175 to 462 rounds on ResNet- 18 and from 156 to 249 rounds on ResNet-34, while all CNN settings yield comparable results. Critically, very short warm-up periods also reduce best accuracy, as insufficient metric history produces unreliable early LR modulation that degrades the full training trajectory. We adopt푅 warm = 10 for CNN and 푅 warm = 40 for ResNet architectures. Influence of temporal window size (휌). The window size휌controls the number of recent rounds used in Z-score normalization, directly governing the temporal horizon of the adaptive signal. FedA2L is largely insensitive to this parameter because Z-score normalization inherently reduces short-term fluctuations regardless of window length, making the adaptive signal robust to the exact history size. Table 6 confirms this robustness: convergence varies within a narrow range across휌 ∈ 5,10,20on all architectures, ranging from 23 to 25 rounds on CNN and from 176 to 197 rounds on ResNet-18, without a consistent directional trend, and best accuracy follows the same pattern across all architectures. ResNet architectures show marginally better convergence with휌 = 10, consistent with their greater need for stable statistical history. We adopt휌 = 5for CNN and휌 = 10for Truong et al.: Preprint submitted to ElsevierPage 15 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning 100200300400500 No. of Rounds 0 5 10 15 20 25 30 35 Accuracy (%) 300350400450500 30 35 = 0.0 = 0.2 = 0.4 = 0.6 = 0.8 = 1.0 (a) CNN 100200300400500 No. of Rounds 0 10 20 30 40 50 Accuracy (%) 300350400450500 45 50 = 0.0 = 0.2 = 0.4 = 0.6 = 0.8 = 1.0 (b) ResNet-18 100200300400500 No. of Rounds 0 10 20 30 40 50 Accuracy (%) 300350400450500 45 50 = 0.0 = 0.2 = 0.4 = 0.6 = 0.8 = 1.0 (c) ResNet-34 Figure 5: Effect of fusion balance훽on convergence speed and model performance under FedAvg on CIFAR-100. The extreme values훽 = 0(휁-only) and훽 = 1(휎-only) constitute direct component ablations of휁and휎, respectively, in Eq. 7, while intermediate values reflect the full dual-metric fusion. ResNet architectures as minimal window sizes that ensure sufficient statistical stability. Influence of global decay constant (휉). The global decay factor(1 + 휉 ⋅ 푟) −0.5 in Eq. 8 controls how aggressively the base LR decreases across rounds, setting the pace at which global optimization slows relative to layer-wise adaptation. The sensitivity of휉is architecture-dependent: deeper models require a gentler decay to allow layer-wise signals to fully differentiate per-layer rates during the critical mid-training phase, while shallower models converge faster and tolerate earlier decay. As shown in Table 6,휉 = 0.3leads to convergence failure on ResNet-18 and ResNet-34, with best accuracy collapsing to 44.90% and 44.29% respec- tively, confirming that aggressive decay suppresses layer-wise adaptation before the per-layer signals become effective. In contrast, reducing휉to 0.1 restores stable convergence at 177 and 157 rounds on ResNet-18 and ResNet-34. Meanwhile, CNN achieves its lowest convergence in 24 rounds under the same aggressive setting, as its simpler feature hierarchy reaches the target earlier under faster decay. We adopt휉 = 0.1 for ResNet architectures and 휉 = 0.3 for CNN. Influence of fusion balance (훽). The fusion balance 훽in Eq. 7 is a key operational parameter, as it controls how FedA2L balances local update intensity휎and network consensus constraints 휁 when constructing the per-layer LR signal. The extreme cases훽 = 0, which relies solely on network consensus constraints휁, and훽 = 1, which relies solely on local update intensity휎, serve as direct ablations of each individual component. Both single-signal configurations exhibit slower convergence and lower best accuracy than any balanced setting, as evidenced by Table 6 and the accuracy trajectories in Fig. 5. The performance degradation observed in both settings indicates that neither signal alone is sufficient for effective layer-wise adaptation. For ResNet-18 on CIFAR-100, the best test accuracy drops from 49.50% at훽 = 0.6to 49.23% at훽 = 0.0and 49.10% at훽 = 1.0, demonstrating that휎and휁provide complementary and non- redundant information whose combination is necessary for full performance. For deeper architectures such as ResNet- 34, performance degradation at훽 = 1.0is more pronounced than at훽 = 0.0, with the required rounds increasing from 150 to 159 at훽 = 1.0, compared to 158 at훽 = 0.0. Table 7 FedA2L default configuration. ParameterTested rangeCNN ResNet-18 ResNet-34 푅 warm 10,20,401040 휌5,10,20510 휏1푒 − 2,1푒 − 3,1푒 − 41e-31e-3 훽 0.0,0.2,0.4,0.6,0.8,1.0 0.60.6 휉0.05,0.1,0.30.30.1 † † 휉 = 0.05 is applied for TinyImageNet due to higher class diversity. This suggests that network consensus constraints offer more informative layer-wise signals in complex feature hierarchies. FedA2L remains stable across훽 ∈ [0.4,0.8], with훽 = 0.6 consistently achieving the strongest convergence speed and accuracy across all evaluated architectures. We therefore adopt훽 = 0.6as the default, reflecting a moderate emphasis on local update intensity while retaining sufficient weight on network consensus constraints. Overall robustness and recommended configuration. The default configuration is summarized in Table 7. Among the five parameters,휌and휏show limited sensitivity across all tested architectures, while훽remains stable within[0.4,0.8], confirming that the dual-signal fusion is robust to reasonable variation in weighting. The primary configuration parameters are푅 warm and휉, both of which interact directly with model depth and should be adjusted when deploying on architectures outside the configurations evaluated here. 6. Discussion FedA2L dynamically adjusts LRs at the layer level using only locally available model statistics, integrating seamlessly into any DFL protocol without modifying the core aggregation logic or introducing additional communication overhead. This distinguishes it from server-dependent adap- tive methods such as FedYogi, layer-wise adaptive approaches that rely on centralized gradient statistics, and divergence- based approaches [52,53,54] that utilize deviation signals for representation alignment or aggregation reweighting. Such a mechanism is particularly relevant under non-IID data Truong et al.: Preprint submitted to ElsevierPage 16 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning and dynamic network conditions where consistent global statistics are difficult to obtain. The layer-wise LR signals in FedA2L may also be extended to personalized DFL, where client-specific adaptation is incorporated into decentralized optimization. While FedA2L offers flexibility, it involves several practi- cal considerations. First, some hyperparameters (e.g.,푅 warm and휉) depend on model depth and may require additional tuning across different architectures. Second, the performance gains are less pronounced when the base DFL algorithm already incorporates strong drift correction or adaptive regularization (e.g., FedNTD and FedYogi), where scheduler- based baselines remain competitive. Overall, despite these considerations, FedA2L provides a robust and effective foun- dation for layer-wise adaptation in decentralized learning. 7. Conclusion This paper presents FedA2L, a layer-wise adaptive LR method designed specifically for DFL. By exploiting dual state-transition signals, FedA2L enables per-layer LR modu- lation that captures layer-specific heterogeneity while stabi- lizing network consensus, without requiring modification to the underlying DFL protocol. Extensive experiments demon- strate that FedA2L consistently accelerates convergence and improves model accuracy across diverse models, datasets, and DFL algorithms, without additional communication overhead. The reduction in communication rounds further improves communication efficiency, which is beneficial in bandwidth- constrained and latency-sensitive environments. These results indicate that layer-wise adaptation provides an effective mechanism for addressing heterogeneity in decentralized learning, establishing FedA2L as a practical and scalable optimization approach for DFL systems. Funding This research was supported by the Basic Science Re- search Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. NRF-2022R1I1A3072355). References [1]Wen Sun, Shiyu Lei, Lu Wang, Zhiqiang Liu, and Yan Zhang. Adaptive federated learning and digital twin for industrial internet of things. IEEE Transactions on Industrial Informatics, 17(8):5605–5614, 2021. doi: 10.1109/TII.2020.3034674. [2]V Padmavathi, R Kanimozhi, and R Saminathan. Digital twin driven smart factories: real time physics based co-simulation using edge ai and federated learning. Scientific Reports, 15(1):43373, 2025. [3] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017. [4]Tian Wang, Yan Liu, Xi Zheng, Hong-Ning Dai, Weijia Jia, and Mande Xie. Edge-based communication optimization for distributed federated learning. IEEE Transactions on Network Science and Engineering, 9 (4):2015–2024, 2021. [5] Shuai Wang, Youliang Tian, Jinbo Xiong, Jianfeng Ma, and Yan Zhang. VerifyDFL: Secure aggregation for decentralized federated learning with input validation in mobile edge intelligence. IEEE Transactions on Cognitive Communications and Networking, 12:2526–2541, 2026. doi: 10.1109/TCCN.2025.3587771. [6]Liangqi Yuan, Ziran Wang, Lichao Sun, Philip S. Yu, and Christo- pher G. Brinton. Decentralized federated learning: A survey and perspective. IEEE Internet of Things Journal, 11(21):34617–34638, 2024. doi: 10.1109/JIOT.2024.3407584. [7]Anusha Lalitha, Shubhanshu Shekhar, Tara Javidi, and Farinaz Koushanfar. Fully decentralized federated learning. In Third workshop on bayesian deep learning (NeurIPS), volume 12, 2018. [8]Peng Wang, Wen Sun, Haibin Zhang, Wenqiang Ma, and Yan Zhang. Distributed and secure federated learning for wireless computing power networks. IEEE Transactions on Vehicular Technology, 72(7):9381– 9393, 2023. [9]Zheyi Chen, Qingnan Jiang, Lixian Chen, Xing Chen, Jie Li, and Geyong Min. MC-2PF: A multi-edge cooperative universal framework for load prediction with personalized federated deep learning. IEEE Transactions on Mobile Computing, 24(6):5138–5154, 2025. doi: 10.1109/TMC.2025.3528404. [10]Zheyi Chen, Jie Liang, Zhengxin Yu, Hongju Cheng, Geyong Min, and Jie Li. Resilient collaborative caching for multi-edge systems with robust federated deep learning. IEEE Transactions on Networking, 33 (2):654–669, 2024. [11]Jianchun Liu, Jiaming Yan, Hongli Xu, Lun Wang, Zhiyuan Wang, Jinyang Huang, and Chunming Qiao. Accelerating decentralized federated learning with probabilistic communication in heterogeneous edge computing. IEEE Transactions on Networking, 34:486–501, 2026. doi: 10.1109/TON.2025.3600015. [12]Mbasa Joaquim Molo, Lucia Vadicamo, Claudio Gennaro, and Emanuele Carlini. Decentralized edge learning: A comparative study of distillation strategies and dissimilarity measures. Future Generation Computer Systems, 176:108171, 2026. ISSN 0167-739X. doi: https://doi.org/10.1016/j.future.2025.108171. [13]Zheyi Chen, Junjie Zhang, Geyong Min, Zhaolong Ning, and Jie Li. Traffic-Aware Lightweight Hierarchical Offloading Toward Adaptive Slicing-Enabled SAGIN. IEEE Journal on Selected Areas in Communi- cations, 42(12):3536–3550, 2024. doi: 10.1109/JSAC.2024.3459020. [14]Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Tal- walkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429–450, 2020. [15] Radwan Selo, Majid Kundroo, and Taehong Kim. FedTVD: Balancing data quality and quantity for robust federated learning. Future Generation Computer Systems, page 108177, 2025. [16]Liang Gao, Huazhu Fu, Li Li, Yingwen Chen, Ming Xu, and Cheng- Zhong Xu. FedDC: Federated learning with non-IID data via local drift decoupling and correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10112– 10121, 2022. [17]Junyoung Park, Sungpil Woo, and Joohyung Lee. Def-Ag: An energy- efficient decentralized federated learning framework via aggregator clients. Future Generation Computer Systems, 175:108114, 2026. ISSN 0167-739X. doi: https://doi.org/10.1016/j.future.2025.108114. [18]Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, 27, 2014. [19]Matthew D Zeiler and Rob Fergus. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I 13, pages 818–833. Springer, 2014. [20]Ahmed Elhussein and Gamze Gürsoy. PLayer-FL: A principled approach to personalized layer-wise cross-silo federated learning, 2025. URL https://arxiv.org/abs/2502.08829. [21]Weihang Chen, Cheng Yang, Jie Ren, Zhiqiang Li, and Zheng Wang. Optimizing personalized federated learning through adaptive layer- wise learning. In James Kwok, editor, Proceedings of the Thirty- Fourth International Joint Conference on Artificial Intelligence, IJCAI- 25, pages 4860–4868. International Joint Conferences on Artificial Truong et al.: Preprint submitted to ElsevierPage 17 of 18 FedA2L: Adaptive Layer-wise Learning Rate Adjustment in Decentralized Federated Learning Intelligence Organization, August 2025. doi: 10.24963/ijcai.2025/541. Main Track. [22]Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečn ` y, Sanjiv Kumar, and H Brendan McMahan. Adaptive federated optimization. arXiv preprint arXiv:2003.00295, 2021. [23]Bingnan Xiao, Jingjing Zhang, Wei Ni, and Xin Wang. FLARE: A new federated learning framework with adjustable learning rates over resource-constrained wireless networks. IEEE Transactions on Wireless Communications, 2025. [24] Anam Nawaz Khan, Qazi Waqas Khan, Atif Rizwan, Rashid Ahmad, and Do Hyeun Kim. Consensus-driven hyperparameter optimization for accelerated model convergence in decentralized federated learning. Internet of Things, 30:101476, 2025. ISSN 2542-6605. doi: https: //doi.org/10.1016/j.iot.2024.101476. [25]Youngmin Ro and Jin Young Choi. AutoLR: Layer-wise pruning and auto-tuning of learning rates in fine-tuning of deep networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2486–2494, 2021. [26]Belhal Karimi, Ping Li, and Xiaoyun Li. Fed-LAMB: layer-wise and dimension-wise locally adaptive federated learning. In Uncertainty in Artificial Intelligence, pages 1037–1046. PMLR, 2023. [27] Changlong Shi, Jinmeng Li, He Zhao, Dandan Guo, and Yi Chang. FedLWS: Federated learning with adaptive layer-wise weight shrinking. arXiv preprint arXiv:2503.15111, 2025. [28]Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. Advances in neural information processing systems, 30, 2017. [29]Enrique Tomás Martínez Beltrán, Mario Quiles Pérez, Pedro Miguel Sánchez Sánchez, Sergio López Bernal, Gérôme Bovet, Manuel Gil Pérez, Gregorio Martínez Pérez, and Alberto Huertas Celdrán. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials, 25(4):2983–3013, 2023. [30]Lingling Wang, Xueqin Zhao, Zhongkai Lu, Lin Wang, and Shouxun Zhang. Enhancing privacy preservation and trustworthiness for decentralized federated learning. Information Sciences, 628:449–468, 2023. ISSN 0020-0255. doi: https://doi.org/10.1016/j.ins.2023.01.130. [31]Yuhao Zhou, Minjia Shi, Yuxin Tian, Qing Ye, and Jiancheng Lv. DeFTA: A plug-and-play peer-to-peer decentralized federated learning framework. Information Sciences, 670:120582, 2024. ISSN 0020-0255. doi: https://doi.org/10.1016/j.ins.2024.120582. [32] Angelia Nedić, Alex Olshevsky, and Michael G Rabbat. Network topology and communication-computation tradeoffs in decentralized optimization. Proceedings of the IEEE, 106(5):953–976, 2018. [33]Vo van Truong, Pham Khanh Quan, Dong-Hwan Park, and Taehong Kim. Performance evaluation of decentralized federated learning: Impact of fully and k-connected topologies, heterogeneous computing resources, and communication bandwidth. IEEE Access, 13:32741– 32755, 2025. doi: 10.1109/ACCESS.2025.3542772. [34]Anastasia Koloskova, Sebastian U. Stich, and Martin Jaggi. Decentral- ized stochastic optimization and gossip algorithms with compressed communication. In Proceedings of the 36th International Conference on Machine Learning (ICML), volume 97 of Proceedings of Machine Learning Research, pages 3478–3487, 2019. [35]Zhenheng Tang, Shaohuai Shi, Bo Li, and Xiaowen Chu. GossipFL: A decentralized federated learning framework with sparsified and adap- tive communication. IEEE Transactions on Parallel and Distributed Systems, 34(3):909–922, 2022. [36]Changlong Shi, He Zhao, Bingjie Zhang, Mingyuan Zhou, Dandan Guo, and Yi Chang. FedAWA: Adaptive optimization of aggregation weights in federated learning using client vectors. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 30651–30660, 2025. [37]Yitong Tang. Adapted weighted aggregation in federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23763–23765, 2024. [38]Majid Kundroo and Taehong Kim. Federated learning with hyper- parameter optimization. Journal of King Saud University-Computer and Information Sciences, 35(9):101740, 2023. [39]Krishna Kanth Nakka, Ahmed Frikha, Ricardo Mendis, Xue Jiang, and Xuebing Zhou. Federated hyperparameter optimization through reward-based strategies: Challenges and insights. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4236–4244, 2024. [40] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-IID data. arXiv preprint arXiv:1806.00582, 2018. [41]Shu Zheng, Tiandi Ye, Xiang Li, and Ming Gao. Federated learning via consensus mechanism on heterogeneous data: A new perspective on convergence. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7595–7599. IEEE, 2024. [42] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. [43]Yann Le and Xuan Yang. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. [44] Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pages 95–104, 2018. [45]Shuyi Zhang, Bin Guo, Anlan Dong, Jing He, Ziping Xu, and Song Xi Chen. Cautionary tales on air-quality improvement in beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2205), 2017. [46]Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: generative pre-trained trans- formers are large time series models. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024. [47]Gihun Lee, Yongjin Shin, Minchan Jeong, and Se-Young Yun. Preser- vation of the global knowledge by not-true self knowledge distillation in federated learning. CoRR, abs/2106.03097, 2021. [48] Yifan Shi, Li Shen, Kang Wei, Yan Sun, Bo Yuan, Xueqian Wang, and Dacheng Tao. Improving the model consistency of decentralized federated learning. In Proceedings of the 40th International Confer- ence on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 31269–31291. PMLR, 23–29 Jul 2023. [49] Ilya Loshchilov and Frank Hutter. SGDR: Stochastic gradient descent with warm restarts, 2017. URL https://arxiv.org/abs/1608.03983. [50]Leslie N. Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates, 2018. URL https://arxiv.org/abs/1708.07120. [51] Tae-Geun Kim. HyperbolicLR: Epoch insensitive learning rate scheduler, 2025. URL https://arxiv.org/abs/2407.15200. [52]Kai Hu, Yaogen Li, Shuai Zhang, Jiasheng Wu, Sheng Gong, Shanshan Jiang, and Liguo Weng. FedMMD: A Federated weighting algorithm considering Non-IID and Local Model Deviation. Expert Systems with Applications, 237:121463, 2024. [53]Jaewon Jang and Bong Jun Choi. Personalized federated learning via deviation tracking representation learning. In 2024 International Conference on Information Networking (ICOIN), pages 762–766. IEEE, 2024. [54]Wenjie Yao, Guanglu Sun, Suxia Zhu, Ruidong Wang, Xinzhong Zhu, HuiYing Xu, and Xiguang Wei. FedRDA: Representation Deviation Alignment in Heterogeneous Federated Learning. IEEE Transactions on Industrial Informatics, 2025. Truong et al.: Preprint submitted to ElsevierPage 18 of 18