Paper deep dive
FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning
Radwan Selo, Majid Kundroo, Taehong Kim
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client's local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced datasets with imbalances from disproportionately influencing the global model. At the same time, dataset size is incorporated to ensure scalability and fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.
Tags
Links
- Source: https://arxiv.org/abs/2608.09221v1
- Canonical: https://arxiv.org/abs/2608.09221v1
Trouble viewing inline? Open PDF directly →
Full Text
68,332 characters extracted from source content.
Expand or collapse full text
FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning Radwan Selo, Majid Kundroo and Taehong Kim ∗ School of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, Republic of Korea A R T I C L E I N F O Keywords: Federated Learning Total Variation Distance Independent and Identically Distributed A B S T R A C T Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client’s local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced datasets with imbalances from disproportionately influencing the global model. At the same time, dataset size is incorporated to ensure scalability and fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings. 1. Introduction FL [1] has emerged as a revolutionary paradigm in distributed machine learning, enabling collaborative model training across multiple decentralized clients without requir- ing them to share their raw data. This design inherently enhances data privacy and security, addressing significant concerns in sensitive domains such as healthcare, finance, and personalized devices [2, 3, 4]. In contrast to traditional centralized learning, where data is aggregated in a single location, FL leverages local training on client devices, trans- mitting only model updates to a central server for aggrega- tion. This decentralized approach aligns with stringent data privacy regulations, such as the General Data Protection Regulation (GDPR), and mitigates the risks associated with transferring large volumes of sensitive data over networks [5, 6]. Moreover, FL significantly reduces communication overhead and computational burden on centralized systems, making it well-suited for large-scale, heterogeneous environ- ments. Despite its advantages, FL faces several challenges that hinder its practical deployment in real-world scenarios. These challenges can be broadly categorized into commu- nication efficiency, system heterogeneity, and data hetero- geneity. Communication efficiency is critical as FL systems often operate in resource-constrained environments, such as © 2026. This manuscript version is made available under the C- BY-NC-ND 4.0 license (http://creativecommons.org/licenses/by-nc-nd/4. 0/). This is the accepted manuscript version of an article published in Fu- ture Generation Computer Systems. The final published version is avail- able at https://doi.org/10.1016/j.future.2025.108177. ∗ Corresponding author radwan-selo@cbnu.ac.kr (R. Selo); kundroomajid@cbnu.ac.kr (M. Kundroo); taehongkim@cbnu.ac.kr (T. Kim) ORCID(s): edge devices or Internet of Things (IoT) networks, where bandwidth and latency [7, 8] are limiting factors [9, 10, 11, 12]. System heterogeneity, on the other hand, arises from the diverse computational capacities of client devices, which can lead to unequal participation and delays in model updates [13]. Among these, data heterogeneity, where the data across clients is non-Independent and Identically Dis- tributed (non-IID) exhibit significant variations in size and class distributions, hence. This issue is inherently tied to the decentralized nature of the system. Clients typically generate data based on their local environments and user behaviors [14]. For example, healthcare organizations contributing data to a federated medical imaging model may have patient populations that differ in demographics, disease prevalence, and imaging protocols, while mobile devices participating in FL-based recommendation systems may exhibit distinct usage patterns. This heterogeneity becomes particularly problematic during model aggregation [15, 16], as some clients possess large but skewed datasets, while others contribute smaller yet more balanced datasets. Standard aggregation methods, such as FedAvg [17], tend to prioritize clients with larger datasets, often resulting in global models that overfit dominant distributions [18] and struggle with fairness and generalization. Addressing this imbalance is crucial for developing robust and equitable FL models that generalize well across diverse client populations. A key barrier to effective FL is ensuring that the global model remains both fair and robust despite these challenges. This challenge is amplified by the fact that clients often learn from skewed local data, while the global model must perform well on more balanced data. This mismatch high- lights the importance of a client’s data quality, particularly its label distribution. However, considering only quality may : Preprint submitted to ElsevierPage 1 of 14 arXiv:2608.09221v1 [cs.LG] 10 Aug 2026 FedTVD undervalue clients with large but imbalanced data. Existing aggregation strategies often lead to models that overfit to dominant data distributions, causing model drift [19, 20], a phenomenon where the model’s performance deteriorates as local data deviates from the global model over time. This drift is particularly prevalent with non-IID dataset, where clients hold data with varying class distributions. Thus, it is essential to consider both the distributional quality and quantity of each client’s data, penalizing highly skewed distributions while still leveraging useful large-scale data, to support stable convergence and better generalization. To address these challenges, this study introduces Federated Learning with Total Variation Distance (FedTVD), a novel algorithm designed to mitigate the impact of heterogeneous label distributions. By leveraging the TVD [21, Chapter 4, p. 47], a robust and interpretable metric for quantifying the divergence between probability distributions, FedTVD employs a dual weighting mechanism that balances label distribution skewness with dataset size, thereby promoting better convergence, improved generalization, and enhanced fairness. By intelligently adjusting each client’s influence based on the nature of its data, our approach aims to ensure that the final model is both inclusive and robust. Ultimately, by addressing the inherent data heterogeneity among clients, we aim to create a system that achieves a balanced integra- tion of diverse data sources, leading to a global model that is more accurate, generalizable, and fair. In light of the challenges posed by data heterogeneity and the limitations of traditional aggregation methods, this work makes several significant contributions to advancing FL. In particular, our work provides a comprehensive al- gorithm that not only addresses fairness and robustness in global model training but also enhances overall performance through practical, scalable solutions. Our key contributions are as follows: 1. TVD-based Weighting: We introduce a novel weight- ing mechanism that assigns adaptive weights to clients based on both label distribution skewness and dataset size. This ensures that clients with heavily skewed data have a moderated influence on the global model, while those with substantial and balanced data are appropriately emphasized, thereby promoting better convergence, improved generalization, and enhanced fairness. 2. Empirical Validation: Through extensive empirical evaluations across diverse datasets and application scenarios, we demonstrate that FedTVD consistently outperforms existing methods, particularly in envi- ronments with highly heterogeneous data. By dynam- ically adjusting client contributions based on data quality and quantity, FedTVD ensures stable conver- gence and enhanced performance, making it particu- larly suitable for large-scale, non-IID environments. The remainder of this paper is structured as follows: Section 2 reviews related work in FL, exploring various methods used to handle challenges like non-IID data and model drift. Section 3 describes the methodology of this study, outlining the problem formulation, including model drift and data quality/quantity, and detailing the innovative TVD-based weighting mechanism used to improve global model convergence. Section 4 explains the experimental setup, providing information on the environment, dataset distribution strategies, hardware configuration, model archi- tecture and hyperparameters employed in the study. Section 5 presents the results and a comprehensive discussion, high- lighting the performance of the FedTVD approach under both IID and non-IID data conditions. Additionally, experi- ments were conducted to evaluate the effect of increasing the number of clients and the Client Participation Rate (CPR) on model accuracy. Finally, Section 6 concludes the paper, summarizing the key findings and suggesting directions for future research. 2. Related Works FL has been the focus of extensive research, with numer- ous algorithms proposed to address its inherent challenges. A key area of exploration is mitigating the effects of non-IID data across clients, which significantly impact the efficiency and fairness of collaborative training. This section reviews prominent FL algorithms and their strategies for handling data heterogeneity, highlighting their contributions and lim- itations in the context of developing more robust aggregation methods, with a summarized overview provided in Table 1. 2.1. Data Distribution Heterogeneity Balancing the contribution of clients with diverse dataset sizes and data distributions remains a fundamental challenge in FL. Classical methods such as FedAvg aggregate updates using dataset-size-weighted averaging, which is effective for IID data but can result in biased models under non- IID scenarios. FedProx [22] augments this by penalizing local updates that deviate from the global model, leading to greater algorithmic stability. However, neither FedAvg nor FedProx fully mitigates issues arising from non-uniform label or feature distributions. FedNova [23] further adjusts for system heterogeneity by normalizing updates based on each client’s local progress. More recently, FedDkw [27] has introduced KL divergence to weight the similarity between client and server label distributions for aggregation, yet this approach can suffer from instability due to the unbounded nature of KL and neglects the size of each client’s dataset. Overall, while these algorithms address parts of the problem, a unified solution that robustly balances both data quantity and distributional diversity is still lacking. 2.2. Feature Heterogeneity Distinct feature distributions among clients are espe- cially prominent in domains such as medical imaging and multi-institutional datasets. FedBN [25] provides an effec- tive remedy by maintaining client-specific batch normal- ization parameters while synchronizing only the remaining : Preprint submitted to ElsevierPage 2 of 14 FedTVD Table 1 Summary of Key FL Methods, Addressed Challenges and Limitations MethodYear Key IdeaAddressed Challenges Limitations FedAvg [17]2017 Weighted average of local models by dataset size Data quantity imbal- ance (assumes IID data) Poor performance with non-IID data bias toward majority distri- butions FedProx [22]2020 Adds proximal term to local objectives for stability System heterogene- ity Insufficient for distributional (la- bel/feature) heterogeneity FedNova [23]2020 Normalizes client updates by training progress Computational (sys- tem) imbalance Assumes data distribution unifor- mity poor with skewed data FedMA [24]2020 Aligns and averages layer-wise model representations across clients Feature representa- tion heterogeneity Does not explicitly account for variations in class distribution skew or disparities in client dataset sizes during aggregation FedBN [25]2021 Local batch normalization pa- rameters global weight aggre- gation Feature distribution shift Does not address label skew or data quantity imbalance Scaffold [26]2021 Control variates to reduce client drift and update diver- gence Client drift and training stability Requires careful tuning limited handling of label and feature im- balance FedDkw [27]2023 Weights updates by KL di- vergence between client and global data distributions Data distribution heterogeneity Ignores dataset size and un- bounded KL may destabilize ag- gregation BN-Scaffold [28]2024 Incorporates local batch nor- malization into Scaffold to ad- dress drift and feature shift Client drift and fea- ture shift May depend on hyperparameter tuning does not fully resolve label skew or dataset size imbalance Clustered FL [29, 30, 31] 2021- 2024 Client clustering to train subgroup-specific models Highly diverse client data distributions Relies on accurate and stable clustering struggles with dynamic client populations Personalized FL [32, 33] 2023 Client-specific model adapta- tion or fine-tuning of global model Local data hetero- geneity and person- alization Adds complexity focuses on local optimization not global solutions FedOpt [34]2021 Adaptive optimizers applied to global aggregation Convergence under data and system het- erogeneity Depend on hyperparameter tuning and does not fully resolve data skew model weights. This preserves locality in feature statis- tics, improving global performance where input distribu- tions vary. Another approach, FedMA [24], aligns and av- erages layer-wise model representations across clients to reduce feature representation mismatch during aggregation. These strategies enhance model robustness under heteroge- neous input spaces, but they still largely overlook skew in class distributions and disparities in dataset sizes. 2.3. Client Drift Local model updates often diverge from the global tra- jectory in the presence of data heterogeneity, resulting in diminished convergence rates. Scaffold [26] directly ad- dresses this challenge by utilizing control variates, which help align local updates with the direction of global opti- mization and significantly improve training stability under non-IID data. A recent variant, BN-Scaffold [28], enhances this by incorporating local batch normalization into the Scaf- fold framework, thereby addressing both update divergence and feature shift in heterogeneous environments. Although these approaches demonstrate improved convergence, they do not fully resolve underlying label skew or dataset size imbalance, and their effectiveness may depend on careful hyperparameter tuning. 2.4. Clustered Federated Learning Clustered FL addresses the challenge of highly diverse data distributions by partitioning clients into subgroups with similar local data characteristics, enabling each cluster to train its own specialized model [30, 31, 29, 35]. These clusters are typically formed using gradient-based statistics, distributional similarity, or auxiliary metadata. By aligning models more closely with subgroup distributions, clustered FL improves personalization and convergence in non-IID settings. However, the effectiveness of this approach relies : Preprint submitted to ElsevierPage 3 of 14 FedTVD heavily on accurate and stable clustering, which can be compromised in dynamic environments with shifting client populations or evolving data characteristics, where com- putational complexity requires multi-objective optimization [36, 37]. 2.5. Personalized Federated Learning Personalized FL (PFL) addresses the challenge of local data heterogeneity by allowing each client to fine-tune a shared global model or maintain personalized components [32, 33]. This improves local performance by adapting to individual client distributions. However, it differs from stan- dard FL by focusing on client-specific optimization, which resolves local heterogeneity but does not offer a unified global solution. 2.6. Optimization and Aggregation Enhancements This line of research aims to improve training efficiency and convergence under heterogeneous conditions by refining global optimization and aggregation strategies. FedOpt and its variants (FedAdam, FedYogi) [34] apply adaptive learn- ing rate techniques based on aggregated gradients, resulting in more stable updates across diverse clients. Additionally, hyperparameter optimization methods such as FedAdap [38, 39] fine-tune training parameters to improve performance in non-IID scenarios. While these techniques enhance op- timization dynamics and support convergence, they do not fully resolve deeper issues related to data distribution skew, system variability, or long-term generalization in complex federated settings. In summary, while existing federated learning methods have advanced the handling of non-IID data, they often struggle to effectively balance the influence of data quantity and distribution heterogeneity during aggregation. Many approaches address individual aspects like system hetero- geneity, feature shifts, or client drift, yet a unified, principled solution that dynamically accounts for both dataset size and label distribution similarity remains lacking. To address this, we propose FedTVD, a novel aggregation method that leverages robust metrics to fairly and efficiently weight client contributions, leading to more stable, equitable, and gener- alized global models. 3. Methodology This section introduces the FedTVD algorithm and ex- plains how it addresses the challenges of model aggregation in non-IID FL environments. We first discuss how the imbal- ance between data quantity and data quality in client updates can negatively affect model convergence and performance. We then present the FedTVD strategy, which corrects for this imbalance by jointly considering both factors during aggregation. Finally, we provide a detailed breakdown of the algorithm, including the client-server workflow and how TVD is incorporated into the aggregation process. 3.1. Overview of FedTVD FedTVD is an aggregation strategy that considers both the size of each client’s dataset (data quantity) and how well that dataset represents the global data distribution (data quality). This dual perspective enables more balanced and effective model aggregation unlike FedAvg, which relies solely on data quantity. Data quantity refers to the total number of local samples available to each client. In conventional FL frameworks such as FedAvg, client aggregation weights are determined solely based on dataset size, under the assumption that larger datasets produce more reliable updates. However, in non-IID scenarios, larger datasets may be significantly biased toward specific classes and thus may not contribute proportionally to the generalization of the global model. In contrast, data quality measures how closely a client’s local data distribution aligns with the global model dis- tribution, which is typically assumed to be IID and class- balanced. For example, in a classification problem with three classes, the ideal global distribution would allocate one-third of the data to each class. In practical FL settings, clients typically exhibit varying degrees of deviation from this ideal distribution due to user-specific or environmental factors. To quantify this deviation, we employ the TVD, a statistical metric that measures the disparity between a client’s local label distribution and the global balanced distribution. A TVD of zero indicates a perfectly balanced client dataset, while higher TVD values signal more severe distributional skewness. Fig. 1 illustrates the difference between FedAvg and FedTVD across three clients. Each client holds samples from three classes. Client 1 has a balanced dataset with (30, 30, 30) samples per class, yielding a TVD of zero. Client 2 dis- plays moderate imbalance with (20, 40, 40), and Client 3 has a notably skewed dataset of (10, 30, 70). In FedAvg, Client 3 receives the highest aggregation weight, driven solely by its larger dataset size. Conversely, FedTVD mitigates this issue by adjusting weights based on both data quantity and TVD. As a result, Client 1 receives a higher aggregation weight un- der FedTVD, reflecting its higher data quality, despite hav- ing fewer samples. By dynamically balancing data quantity and quality, FedTVD promotes equitable and robust model aggregation, steering updates away from clients with skewed datasets and toward those with representative distributions. This fosters improved convergence and generalization in heterogeneous FL environments. At the system level, FedTVD introduces minor yet im- pactful modifications to the standard FL workflow. Clients locally compute their label distributions and associated TVD values relative to the global distribution. These metrics, along with model updates and dataset sizes, are commu- nicated to the server. The server then performs a refined aggregation by jointly incorporating data quantity and TVD, replacing the simplistic quantity-only weighting used in FedAvg. The subsequent subsection provides a detailed de- scription of the FedTVD workflow, outlining its client- server interactions and aggregation mechanism. : Preprint submitted to ElsevierPage 4 of 14 FedTVD Client 1 Client 3Client 1Client 2 where is the number of samples at client Data Quantity Aggregation P = (, , ) Q = (, , ) Client 2 P = (, , ) Q = (, , ) Client 3 P = (, , ) Q = (, , ) Data Quality Data Quantity Client 1 contributes more to the global model due to its more balanced local dataset Aggregation Data Quantity (FedAvg) Data Quantity and Quality (FedTVD) Client 1 receives a lower weighting in the aggregation process because data quality is not considered. Client dataset Size of Distribution of Balanced Distribution Client model Global model Total number of samples across clients Client scaling factor Fig. 1: Comparison of FedAvg and FedTVD aggregation strategies. FedAvg relies only on data quantity (number of samples), while FedTVD incorporates both data quantity and quality, ensuring more balanced aggregation and mitigating model drift in non-IID settings. 3.2. Detailed FedTVD Algorithmic Steps The FedTVD algorithm follows the typical client-server paradigm in FL. In each round 푡, the server distributes the current global model 푤 푡 to participating clients. Clients then update these parameters using their local datasets and send back the updated parameters, along with additional statistics. Below we provide a step-by-step outline of our approach, culminating in Algorithm 1. 3.2.1. Client-Side Operations (1) Local Training: Each client 푘 begins with the global parameters 푤 and proceeds to train on its private dataset 퐷 푘 using an appropriate optimizer, such as stochastic gradient descent (SGD). Multiple local epochs can be executed if network constraints allow. (2) Label Distribution Computation: Following train- ing, the client computes its label distribution 푃 by counting how many examples fall into each class. Formally, if the total number of classes is 퐶, the label distribution 푃 is given by: 푃 = ( |퐷 푘 1 | |퐷 푘 | , |퐷 푘 2 | |퐷 푘 | , ..., |퐷 푘 퐶 | |퐷 푘 | ) (1) where |퐷 푘 푖 | denotes the number of local samples of class 푖, and |퐷 푘 | = ∑ 퐶 푖=1 |퐷 푘 푖 | is the total number of local samples for client 푘. To evaluate how this local distribution deviates from a balanced target, the client constructs a reference distribution 푄, in which all classes are assumed to be equally likely: 푄 = ( 1 퐶 , 1 퐶 , ..., 1 퐶 ) (2) This balanced distribution serves as a baseline for assessing the level of skew in the client’s local data. (3) TVD Calculation: The client calculates the TVD to measure the deviation between its local data distribution 푃 and the balanced reference distribution 푄. The TVD is defined as: TVD = 1 2 퐶 ∑ 푖=1 | | 푃 푖 − 푄 푖 | | (3) TVD quantifies the total difference between the two distri- butions by summing the absolute differences of their cor- responding probabilities. A higher TVD indicates a larger imbalance in the local data distribution, which is useful for identifying clients with more skewed data. This helps : Preprint submitted to ElsevierPage 5 of 14 FedTVD the server adjust the model aggregation process, giving less weight to clients with higher TVD values to ensure a more balanced global model. (4) Return of Results: Finally, the client 푘 sends its updated model parameters 푤 푘 , the computed TVD 푘 , and its local sample size |퐷 푘 | back to the server. 3.2.2. Server-Side Operations (1) Model Broadcast: The server holds a global model parameter vector 푤, which is initialized at the start. At each round 푡, the server broadcasts 푤 푡 to all participating clients. (2) Reception of Client Updates: Once clients finish their local training, the server collects the updated model parameters 푤 푘 , TVD 푘 , and local sample sizes |퐷 푘 | from each client 푘 in the current round. (3) Softmax-Based TVD Weighting: The server applies the softmax function to the set of TVD values received from all clients to transform them into a normalized weighting distribution: 푎 푘 = 푒 −TVD 푘 ∑ 퐾 푗=1 푒 −TVD 푗 (4) This transformation adjusts each client’s weight based on the degree of balance in their local label distribution. Clients with more balanced distributions (i.e., lower TVD values) are assigned higher weights, thereby contributing more sig- nificantly to the global model update. In contrast, clients with skewed distributions (higher TVD) are down-weighted. The softmax function also ensures a smooth and differentiable weighting scheme, preventing any single client from domi- nating the aggregation process. (4) Composite Scaling Factors: Following the softmax- based weighting described above, the server integrates each client’s weight푎 푘 , which reflects the balance of its local label distribution, with its relative data volume. First, it computes the total number of samples across all clients: |퐷| = 퐾 ∑ 푘=1 |퐷 푘 |(5) This allows the computation of the proportion of data held by each client 푘, expressed as |퐷 푘 | |퐷| , which captures the contribution of the client in terms of dataset size. The final scaling factor 푠 푘 for client 푘 is then computed by combining the softmax-derived weight 푎 푘 , which reflects the balance of its local label distribution, with its relative data volume. This combination is controlled via a tunable parameter 휆 ∈ [0, 1] as follows: 푠 푘 = 휆⋅ 푎 푘 + (1 − 휆)⋅ |퐷 푘 | |퐷| (6) Here, 휆 enables flexible control over the influence of data quality versus quantity in the aggregation process. A higher 휆 emphasizes data quality (i.e., label balance), while a lower 휆 emphasizes dataset size. Notably, setting 휆 = 1 results in weighting purely based on data quality, and setting 휆 = 0 Algorithm 1: Federated Learning with TVD (FedTVD) 1 Function CLIENT_SIDE_OPERATIONS(푤): 2 Train local model 푤 on local data 퐷 푘 // Calculate local label distribution 3 푃 = ( |퐷 푘 1 | |퐷 푘 | , |퐷 푘 2 | |퐷 푘 | , ..., |퐷 푘 퐶 | |퐷 푘 | ) Eq. (1) // Initialize global balanced distribution 4 푄 = ( 1 퐶 , 1 퐶 , ..., 1 퐶 ) Eq. (2) // Calculate TVD 5 TVD = 1 2 퐶 ∑ 푖=1 | | 푃 푖 − 푄 푖 | | Eq. (3) 6 return 푤, TVD, and |퐷 푘 | 7 End Function 8 Function SERVER_SIDE_OPERATIONS(): 9 Initialize global model parameters 푤 10 for each round 푡 = 1, 2, ...,푇 do 11Distribute global model 푤 푡 to all participating clients 12for each client 푘 = 1, 2, ...,퐾 do 13푤 푘 , TVD 푘 , and |퐷 푘 |← CLIENT_SIDE_OPERATIONS(푤 푡 ) 14end // Calculate softmax of TVD values 15 푎 푘 = 푒 −TVD 푘 ∑ 퐾 푗=1 푒 −TVD 푗 Eq. (4) // Calculate total number of samples across clients 16 |퐷| = 퐾 ∑ 푘=1 |퐷 푘 |Eq. (5) // Compute scaling factor 17 푠 푘 = 휆⋅ 푎 푘 + (1 − 휆)⋅ |퐷 푘 | |퐷| Eq. (6) // Aggregate using 18 푤 푡+1 = 퐾 ∑ 푘=1 푠 푘 푤 푘 Eq. (7) 19 end 20 End Function reduces the algorithm to FedAvg, where only dataset size is considered. This composite scaling factor therefore accounts for both the quality of the client’s data (via 푎 푘 , which penalizes distributional skew) and its quantity (via |퐷 푘 | |퐷| ). By jointly considering these two dimensions, the algorithm ensures that clients with both large and well-balanced datasets exert greater influence on the global model, while clients with lim- ited or highly skewed data are appropriately down-weighted. : Preprint submitted to ElsevierPage 6 of 14 FedTVD This strategy enables a principled trade-off between fairness and statistical representativeness in federated model aggre- gation. (5) Model Aggregation: Finally, the server aggregates the local models into the new global model 푤 푡+1 using: 푤 푡+1 = 퐾 ∑ 푘=1 푠 푘 푤 푘 (7) This updated model 푤 푡+1 is then broadcast to the clients in the next round. 3.3. Complexity Analysis To evaluate the scalability of FedTVD, we analyze its computational complexity and compare it to that of FedAvg. We consider the dominant operations per communication round on both client and server sides. Client-Side Complexity: The primary computational cost on the client side is local training. Assuming each client performs 퐸 epochs of SGD over their local dataset 퐷 푘 , the training complexity is: (퐸⋅ |퐷 푘 |⋅ |푤|) where |푤| is the number of model parameters. In addition to training, FedTVD computes the local label distribution and TVD. Label counting has linear complexity in the dataset size and class count,(|퐷 푘 | + 퐶), while TVD computation is(퐶). Given that 퐶 is typically much smaller than |퐷 푘 |, the total client-side complexity remains dominated by the training phase. Server-Side Complexity: At the server, FedAvg ag- gregates 퐾 client models per round, costing(퐾⋅ |푤|). FedTVD introduces lightweight additional operations: • Softmax over 퐾 TVD values:(퐾) • Total sample count:(퐾) • Composite scaling factor computation:(퐾) • Weighted model aggregation:(퐾⋅ |푤|) Although FedTVD adds four extra steps at the server, all but one are lightweight(퐾) scalar operations. Only the final step—weighted model aggregation—shares the same dominant complexity(퐾⋅|푤|) as FedAvg. Thus, FedTVD preserves the same overall order of server-side complexity as FedAvg, with only minor constant-factor overhead. 3.4. Convergence Analysis We analyze the convergence behavior of FedTVD un- der standard assumptions in federated optimization. Our approach builds upon the convergence theory of FedAvg, which we briefly recall here to provide a foundation for comparison. In FedAvg, the global objective function is defined as: 퐹(푤) = 푁 ∑ 푖=1 푝 푖 퐹 푖 (푤), where 푝 푖 = |퐷 푖 | ∑ 푁 푗=1 |퐷 푗 | , and each local objective function 퐹 푖 (푤) is assumed to be 퐿-smooth and 휇-strongly convex. Under these assumptions, FedAvg achieves the following convergence bound after 푇 communication rounds: 피[퐹(푤 푇 ) − 퐹(푤 ∗ )]≤ 푂 ( 1 푇 ) + 푂(훿 2 FedAvg ), where all gradients are evaluated at the current global model 푤 푡 , and the client drift term is defined as: 훿 2 FedAvg = 1 퐾 퐾 ∑ 푘=1 ‖ ‖ ∇퐹 푘 (푤 푡 ) − ∇퐹(푤 푡 ) ‖ ‖ 2 . FedTVD modifies only the aggregation step of FedAvg. Instead of relying solely on the sample proportion |퐷 푖 |, it in- corporates both data quantity and data quality. In particular, each client is assigned a weight푠 푘 that depends on its dataset size and the TVD of its label distribution. The updated global model is then computed using a normalized weighted average: 푤 푡+1 = 퐾 ∑ 푘=1 푠 푘 푤 푡 푘 , with 퐾 ∑ 푘=1 푠 푘 = 1, 푠 푘 ≥ 0. This structure remains consistent with the class of algo- rithms analyzed in FedAvg, and thus FedTVD inherits the same theoretical convergence guarantees under convexity and smoothness. Formally, the expected optimality gap for FedTVD sat- isfies: 피[퐹(푤 푇 ) − 퐹(푤 ∗ )]≤ 푂 ( 1 푇 ) + 푂(훿 2 FedTVD ), where the modified client drift term is given by: 훿 2 FedTVD = 퐾 ∑ 푘=1 푠 푘 ‖ ‖ ∇퐹 푘 (푤 푡 ) − ∇퐹(푤 푡 ) ‖ ‖ 2 . Since TVD captures the degree of label imbalance, which often leads to biased local gradients, FedTVD nat- urally reduces drift by assigning lower weights to clients with highly skewed distributions. This weighting effect suppresses the contribution of highly biased updates, which implies that 훿 2 FedTVD ≤ 훿 2 FedAvg in typical non-IID scenarios. In conclusion, FedTVD maintains the same convergence rate as FedAvg under standard convexity and smoothness assumptions. By incorporating distributional information into the aggregation process, it reduces the impact of client drift, which is often the dominant source of error in non-IID FL. As a result, FedTVD is theoretically as sound as FedAvg while offering improved robustness in practical scenarios. 3.5. Advantages and Novelty (A) Mitigating non-IID Effects: FedTVD explicitly mod- erates clients exhibiting large distributional discrepancies from a balanced reference distribution. By computing the TVD of each client’s local label distribution with respect to : Preprint submitted to ElsevierPage 7 of 14 FedTVD an ideal (uniform) distribution, our method identifies those with skewed or biased data. This mechanism prevents the global model from becoming dominated by a few skewed distributions, ensuring a more equitable update that remains resilient in heterogeneous settings. (B) Robustness to Label Skew Outliers: Clients with highly imbalanced label distributions can act as statisti- cal outliers and introduce bias during model aggregation. FedTVD addresses this issue by assigning lower aggregation weights to such clients using an exponential penalty based on their TVD. This reduces their influence on the global model and helps improve robustness against distributional skew. We note that our notion of robustness is limited to mitigating the effect of label imbalance, and does not address other types of outliers such as clients with corrupted data labels or adversarial clients [40]. (C) Balanced Data-Driven Weighting: While addressing skewness is vital, large datasets may hold significant infor- mation. FedTVD reconciles these needs by incorporating both the TVD-based weight (푎 푘 ) and the sample-size frac- tion ( |퐷 푘 | |퐷| ). This hybrid scheme strikes a balance between distributional alignment and data volume, preventing large but skewed datasets from overwhelming the global model while still leveraging their information-rich contribution. (D) Light Overhead: Compared to traditional FL, the main overhead of FedTVD is the computation and transmission of TVD for each client. However, this overhead remains mini- mal compared to raw data exchange or large-scale gradient sharing. The primary computational overhead in FedTVD arises from computing the TVD metric, which involves sum- ming over class distributions, This complexity is negligible compared to the overall cost of FL, making FedTVD scalable to large deployments. (E) Extensibility: Though introduced in the context of a standard FL approach, the TVD modification is designed as a lightweight, drop-in mechanism that can be integrated into diverse federated algorithms. Because it involves mini- mal additional computation and does not require substantial changes to communication or optimization routines, it can be readily adopted in any FL framework to handle non-IID data. (F) Real-World Use Cases: FedTVD is particularly well- suited to FL environments where client data is skewed due to user-specific, demographic, or operational factors. In healthcare applications, for instance, different hospitals or medical centers may collect patient data that naturally varies in disease prevalence or age distribution, which leads to highly imbalanced local datasets. FedTVD can mitigate the influence of such skewed clients during model aggrega- tion while preserving meaningful contributions. Similarly, in large-scale sensor networks or environmental monitoring, data collected from different regions or sensor types may reflect localized patterns that deviate from the overall dis- tribution. While FedTVD computes TVD using a uniform reference distribution to avoid domination by any particular class, this reference can be adapted in practice to reflect domain-specific priors if a more accurate global distribution is known. This flexibility ensures that FedTVD remains applicable across both standard and application-aware FL scenarios, enhancing fairness and robustness in aggregation. Summary: In conclusion, FedTVD offers a balanced and adaptive aggregation strategy by coupling data quantity with a principled TVD-based penalty. This approach effectively addresses heterogeneous data distributions and outlier risks while imposing minimal overhead. The key operations and line-by-line equations are outlined in Algorithm 1, providing a clear blueprint for practical deployment in a wide range of FL scenarios. 4. Experimental Setup This section outlines the experimental setup, which in- cludes the datasets, data partitioning techniques, compu- tational environment, and configurations used to evaluate the proposed approach. In order to evaluate the model’s effectiveness under various client participation rates and data heterogeneity levels, we provide a comprehensive un- derstanding of its performance in diverse scenarios. 4.1. Implementation Details and System Configuration All experiments were conducted using Python 3.11, with PyTorch 2.2.2 serving as the core deep learning frame- work. FL simulations were implemented using the FedEasy framework [41], enabling efficient orchestration of client- server interactions. To ensure consistency across experi- mental runs, all computations were performed on a high- performance computing server equipped with an Intel Core i9-10900X CPU running at 3.70 GHz, 192 GB of RAM, and four NVIDIA GeForce RTX 4090 GPUs. This compu- tational setup facilitated large-scale federated training while maintaining reproducibility and efficiency. 4.2. Datasets and Model Architectures The evaluation was conducted on three standard bench- mark datasets: FashionMNIST (FMNIST) [42], CIFAR- 10 [43], and CIFAR-100, each presenting a different level of complexity. FMNIST, a grayscale image classification dataset, was used with a standard Convolutional Neural Net- work (CNN) architecture [44]. CIFAR-10 was evaluated us- ing a ResNet-18 model [45], while CIFAR-100 was trained on a deeper ResNet-34 architecture. These datasets provided a diverse range of classification challenges, allowing for an in-depth analysis of the proposed method’s robustness across both simple and complex tasks. 4.3. Data Partitioning and Experimental Variants To simulate realistic non-IID conditions, data partition- ing was performed using a Dirichlet allocation strategy [46], controlling the extent of statistical heterogeneity across clients. The concentration parameter 훼 was varied to reflect different levels of non-IID data distributions, with lower values 훼 = 0.1 indicating highly skewed distributions and : Preprint submitted to ElsevierPage 8 of 14 FedTVD Table 2 Mean test accuracy for the last 10 rounds across different datasets (FMNIST, CIFAR-10, and CIFAR-100) under varying Dirichlet non-IID distributions (훼 = 0.1, 0.5, 1.0) and IID settings. DatasetModel Dir (훼)FedAvgFedProxFedNovaFedDkwFedTVD (ours) FMNISTCNN 0.182.64± 0.71 80.69± 1.68 83.63± 0.22 83.99± 0.61 85.26± 0.65 0.586.11± 0.56 86.08± 0.39 86.65± 0.14 87.20± 0.19 87.21± 0.21 1.087.69± 0.36 87.62± 0.41 88.01± 0.10 88.03± 0.21 88.39± 0.14 IID88.95± 0.16 88.83± 0.25 88.89± 0.15 88.89± 0.25 88.74± 0.23 CIFAR10 Resnet-18 0.148.24± 3.35 49.44± 2.69 53.52± 2.20 56.30± 1.61 58.82± 1.75 0.573.33± 0.42 72.61± 1.00 74.26± 0.19 74.51± 0.37 74.69± 0.29 1.076.17± 0.32 75.22± 0.62 75.37± 0.43 75.13± 0.41 76.22± 0.45 IID77.56± 0.37 77.34± 0.45 77.23± 0.32 77.12± 0.40 78.04± 0.25 CIFAR100 Resnet-34 0.135.31± 1.12 36.12± 0.44 36.26± 0.57 36.16± 0.87 37.68± 0.38 0.538.91± 0.36 38.36± 0.75 38.89± 0.44 38.21± 1.08 39.58± 0.71 1.039.30± 0.55 39.30± 0.92 39.09± 0.27 39.06± 0.69 39.79± 0.51 IID39.24± 0.68 39.33± 0.29 39.76± 0.59 39.95± 0.20 40.51± 0.45 higher values 훼 = 1 representing more balanced partitions. For reference, a fully IID scenario was also included in the analysis by setting 훼 to a sufficiently large value. Two major experimental settings were designed to an- alyze the impact of data heterogeneity and CPR. The first set of experiments investigated the effect of varying non- IID levels on model convergence and performance across all three datasets. In this setting, different values of 훼 = 0.1, 0.5,푎푛푑1 were considered, allowing an in-depth eval- uation of the impact of statistical heterogeneity. Each ex- periment involved 100 clients, with 10% of them randomly selected in each round to participate in training. The second set of experiments specifically focused on assessing how the number of clients and CPR influence learning dynamics. These trials were conducted using the CIFAR-10 dataset under a highly non-IID setting 훼 = 0.1. The number of clients was varied between 100 and 200, while the CPR was systematically adjusted to 10%, 20%, and 50%, enabling an analysis of scalability and participation effects on model convergence. 4.4. Hyperparameters and Training Process To ensure fair comparisons, all experiments followed a standardized training protocol. Model updates were op- timized using SGD with a learning rate of 0.01 and a momentum coefficient of 0.9. Each client performed four local epochs per training round with a batch size of 32 before transmitting model updates to the central server. Training was conducted over 300 communication rounds to thoroughly evaluate model convergence and stability across different settings. For FedTVD, the hyperparameter 휆 was fixed at 0.50, reflecting an equal balance between client label distribution quality and dataset size during aggregation. To ensure robustness and reproducibility, each experiment was repeated five times using different random seeds, with data resampling performed independently in each experiment for training only. The testing dataset was fixed on the server and remained unchanged throughout all runs. The reported results represent the mean ± standard deviation over these runs. 5. Results & Discussion In this section, we present the results from compre- hensive comparative experiments conducted to verify the effectiveness of the proposed FedTVD algorithm. These ex- periments benchmark FedTVD against several state-of-the- art FL algorithms, including FedAvg, FedProx, FedNova, and FedDkw. The results, presented in the following subsec- tions, demonstrate the superior performance of FedTVD in terms of test accuracy and convergence speed, particularly in extreme non-IID data settings. 5.1. Performance Analysis Across Varying Data Distributions and Datasets In scenarios with extreme data heterogeneity, where the Dirichlet parameter 훼 = 0.1, FedTVD demonstrates a clear advantage over other algorithms. As shown in Fig. 2a, 3a and Fig. 4a, FedTVD achieves faster convergence and higher accuracy compared to FedAvg, FedProx, and FedNova, par- ticularly on the CIFAR-10 dataset. The results in Table 2 also reinforce this finding, where FedTVD reaches 58.82% test accuracy on CIFAR-10, surpassing FedAvg (48.24%) and FedDkw (56.30%). This performance is especially sig- nificant in environments where the clients’ data are highly skewed, resulting in significant heterogeneity between client datasets. In such settings, FedTVD’s adaptive aggregation mechanism prioritizes the updates that contribute most ef- fectively to improving the global model, which helps miti- gate the negative impact of extreme data imbalances. Specif- ically, FedTVD is able to focus on the more informative updates from clients with data distributions that are more : Preprint submitted to ElsevierPage 9 of 14 FedTVD 050100150200250300 Round 10 20 30 40 50 60 70 80 90 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (a) 훼 = 0.1 050100150200250300 Round 10 20 30 40 50 60 70 80 90 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (b) 훼 = 0.5 050100150200250300 Round 10 20 30 40 50 60 70 80 90 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (c) 훼 = 1.0 Fig. 2: Accuracy trends over rounds for the FMNIST dataset using a CNN model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 훼. 050100150200250300 Round 10 20 30 40 50 60 70 80 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (a) 훼 = 0.1 050100150200250300 Round 10 20 30 40 50 60 70 80 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (b) 훼 = 0.5 050100150200250300 Round 10 20 30 40 50 60 70 80 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (c) 훼 = 1.0 Fig. 3: Accuracy trends over rounds for the CIFAR-10 dataset using a ResNet-18 model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 훼. representative of the global model, thus enabling faster and more stable convergence. The benefit of FedTVD be- comes more pronounced as the data heterogeneity increases, demonstrating its ability to adapt to highly non-IID settings and outperform other methods that struggle to converge effi- ciently under such conditions. In conclusion, FedTVD excels in highly heterogeneous environments, where its adaptive aggregation scheme allows it to outperform other FL al- gorithms by better handling the skewed data distributions across clients. As data heterogeneity becomes more moderate, repre- sented by 훼 = 0.5 and 훼 = 1.0, FedTVD maintains its advantage in both accuracy and convergence speed. The trends in Fig. 3b, Fig. 3c, Fig. 4b, and Fig. 4c, along with Table 2, demonstrate that FedTVD consistently outperforms FedAvg, FedProx, and FedNova. On CIFAR-100, FedTVD achieves 39.58% accuracy for 훼 = 0.5, slightly surpassing FedAvg (38.91%) and FedDkw (38.21%). For 훼 = 1.0, it reaches 39.79%, outperforming FedAvg (39.30%) and FedDkw (39.06%). This suggests that even in less extreme non-IID scenarios, FedTVD effectively balances client con- tributions and mitigates data imbalance, leading to stable and efficient convergence. Unlike FedAvg and FedNova, which struggle with slower adaptation in moderately heterogeneous settings, FedTVD dynamically adjusts client weights to pri- oritize more informative updates. This adaptability results in consistent performance improvements across both CIFAR- 10 and CIFAR-100. Even in IID settings, where client data distributions are uniform, FedTVD maintains competitive accuracy and 050100150200250300 Round 10 15 20 25 30 35 40 45 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (a) 훼 = 0.1 050100150200250300 Round 10 15 20 25 30 35 40 45 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (b) 훼 = 0.5 050100150200250300 Round 10 15 20 25 30 35 40 45 Accuracy (%) FedAvg FedProx FedNova FedDkw FedTVD (c) 훼 = 1.0 Fig. 4: Accuracy trends over rounds for the CIFAR-100 dataset using a ResNet-34 model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 훼. : Preprint submitted to ElsevierPage 10 of 14 FedTVD Table 3 Mean test accuracy for the last 10 rounds on the CIFAR-10 dataset under extreme non-IID settings (훼 = 0.1) with varying numbers of clients and CPR. #Clients CPRFedAvgFedProxFedNovaFedDkwFedTVD (ours) 100 0.1 48.24± 3.35 49.44± 2.69 53.52± 2.20 56.30± 1.61 58.82± 1.75 0.2 56.45± 0.82 56.55± 1.58 58.37± 0.77 59.98± 1.14 61.68± 1.57 0.5 60.60± 0.94 61.18± 0.82 62.00± 0.65 62.35± 0.89 62.93± 0.86 200 0.1 46.71± 1.04 45.28± 2.42 52.91± 1.84 55.11± 1.83 57.39± 3.30 0.2 52.05± 2.49 50.99± 2.22 56.24± 1.48 57.57± 1.61 58.78± 1.88 0.5 55.18± 2.26 56.41± 2.17 58.62± 1.49 59.19± 1.18 61.12± 0.97 stable convergence, as shown in Table 2 and Figs. 2c and 3c. While the performance gap with existing methods like FedAvg and FedDkw is marginal, FedTVD’s adaptive aggre- gation remains effective, ensuring consistent updates across clients. Its ability to perform well across both IID and non-IID environments highlights its versatility in FL. This minor gain may arise because, despite IID sampling, the Dirichlet-based partitioning with finite 훼 can still lead to slight class imbalances, allowing the TVD term to act as a weak regularizer. 5.2. Influence of Total Number of Clients and CPR This section analyzes the performance according to CPR with 100 clients. For a low CPR of 0.1, where only 10% of clients participate in each communication round, FedTVD achieves an accuracy of 58.82%, outperforming FedDkw (56.30%) and FedAvg (48.24%), as shown in Table 3. As CPR increases, FedTVD’s performance improves, reaching 62.93% at a CPR of 0.5, maintaining its lead over Fed- Dkw (62.35%). This improvement can be attributed to the increased diversity and quantity of updates as more clients contribute to the aggregation process, allowing FedTVD to better capture the global data distribution and enhance test set accuracy. FedTVD benefits from broader data access, further boosting its performance. As the client count increases to 200, the overall perfor- mance of all algorithms slightly decreases due to greater data distribution heterogeneity and reduced data volume per client. At a CPR of 0.1, FedTVD achieves 57.39%, slightly outperforming FedDkw (55.11%) and significantly surpassing FedAvg (46.71%). At a higher CPR of 0.5, FedTVD achieves 61.12%, maintaining its lead over Fed- Dkw (59.19%). The reduced accuracy compared to the 100- client scenario is expected, as the broader distribution of data increases variability in local updates, and the reduced data per client limits the representativeness of individual models. Despite these challenges, FedTVD’s robust adaptive mechanism minimizes the performance drop by aggregating the most relevant updates, ensuring that the global model quality remains high even with a larger number of clients. 5.3. Impact of Data Quality In FL, both data quantity and data quality influence model performance. This section explores the impact of quality-aware weighting in FL aggregation by comparing FedAvg (which prioritizes dataset size) and FedTVD (which dynamically adjusts weights based on both label distribution and dataset size) on the CIFAR-10 dataset under a Dirichlet distribution with 훼 = 0.1, using 100 clients (CPR of 0.1). Fig. 5 illustrates the difference between traditional quantity- based aggregation (Fig. 5a) and FedTVD’s quality-aware approach (Fig. 5b). In Fig. 5a, larger datasets dominate aggregation, leading to an imbalance where a few clients contribute excessively while others have minimal influence. In contrast, Fig. 5b shows a more even distribution of con- tributions, as FedTVD dynamically adjusts weights based on both dataset size and label distribution. This ensures that clients with highly skewed data do not disproportionately impact the global model while still leveraging large, high- quality datasets. As a result, FedTVD achieves a 10.58% accuracy improvement over FedAvg in extreme non-IID settings (훼 = 0.1, Table 2), demonstrating the importance of incorporating data quality into FL aggregation. Table 4 Mean test accuracy over the last 10 rounds on CIFAR- 10 for different 휆 values under two non-IID settings: extreme (훼 = 0.1) and moderate (훼 = 1.0). Each setting uses 100 clients (CPR of 0.1). 휆 Accuracy (mean± std) 훼 = 0.1훼 = 1.0 0.0048.24± 3.3576.17± 0.32 0.25 55.88± 0.7176.25± 0.22 0.5058.82± 1.7576.22± 0.45 0.7558.16± 0.7976.05± 0.55 1.0060.18± 0.9676.03± 0.52 To further examine the impact of balancing data quantity and quality, we provide an ablation study on the weighting parameter 휆, which controls the trade-off between the two components during aggregation. A value of 휆 = 0.00 corre- sponds to using data quantity only (i.e., FedAvg), while 휆 = : Preprint submitted to ElsevierPage 11 of 14 FedTVD 12345678910 Rounds Client 1 Client 2 Client 3 Client 4 Client 5 Client 6 Client 7 Client 8 Client 9 Client 10 Clients (a) Data Quantity only 12345678910 Rounds Client 1 Client 2 Client 3 Client 4 Client 5 Client 6 Client 7 Client 8 Client 9 Client 10 Clients (b) Data Quantity and Quality Fig. 5: Conceptual illustration of different weighting strategies in FL on the CIFAR-10 dataset under a Dirichlet distribution with 훼 = 0.1, using 100 clients (CPR of 0.1). The bubble size represents the relative contribution of each client during aggregation. (a) When only data quantity is considered, clients with larger datasets dominate the updates. (b) A combined approach balances both factors, ensuring that high-quality contributions are prioritized alongside larger datasets. 1.00 corresponds to FedTVD’s full reliance on data quality. Intermediate values interpolate between these extremes. As shown in Table 4, under extreme non-IID conditions (훼 = 0.1), even small increases in 휆 lead to notable accuracy gains—for example,휆 = 0.25 improves performance by over 7% compared to 휆 = 0.00. Although 휆 = 1.00 achieves the highest accuracy in this extreme setting, it performs slightly worse in the moderate skewed setting (훼 = 1.0). Relying solely on data quality, as with 휆 = 1.00, may not generalize well across all FL scenarios since, in less skewed environ- ments, data quantity becomes more important. In contrast, 휆 = 0.50 delivers consistently strong results across both scenarios—ranking second-best for 훼 = 0.1 and matching the top accuracy for 훼 = 1.0. This balanced approach, which combines contributions from both data quality and quantity, makes 휆 = 0.50 a robust default choice for diverse FL environments. These results highlight the critical role of balancing data quality and quantity in FL aggregation. Unlike tradi- tional methods that overweight large but potentially biased datasets, FedTVD with 휆 = 0.50 ensures a fairer contribu- tion from all clients, leading to better generalization across diverse non-IID scenarios and more stable convergence. 6. Conclusion FedTVD advances the state of the art in FL by incorpo- rating a lightweight yet effective metric, the TVD, into the aggregation procedure. By combining TVD-derived weights with sample-based proportions, it captures both distribu- tional alignment and data volume, helping to mitigate the common challenges of non-IID data. This leads to more robust convergence, particularly in heterogeneous settings. FedTVD’s ability to handle outliers and imbalanced data makes it well-suited for real-world applications where client data is naturally diverse. While our approach specifically targets label distribu- tion skew, it does not directly address feature distribution skew, concept drift, or fairness across subgroups defined by non-label features. Tackling these more complex forms of heterogeneity would require significant methodological extensions and richer datasets. We view this as an important future direction, with potential to further improve fairness, robustness, and adaptability in large-scale federated systems. Funding This research was supported by Basic Science Research Program through the National Research Foundation of Ko- rea (NRF) funded by the Ministry of Education (No. NRF- 2022R1I1A3072355). Data Availability The data used in this study are publicly available. Declaration of Competing Interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. : Preprint submitted to ElsevierPage 12 of 14 FedTVD CRediT authorship contribution statement Radwan Selo: Conceptualization, Methodology, Visu- alization, and Writing – original draft. Majid Kundroo: Formal analysis, Validation, and Writing – review and edit- ing. Taehong Kim: Resources, Validation, Supervision and Writing – review and editing. References [1] Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency, 2017. URL https://arxiv.org/abs/1610.05492. [2] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol., 10(2), January 2019. ISSN 2157-6904. doi: 10.1145/ 3298981. URL https://doi.org/10.1145/3298981. [3] Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. A review of applications in federated learning. Computers & Industrial Engineering, 149: 106854, 2020. ISSN 0360-8352. doi: https://doi.org/10.1016/j.cie. 2020.106854. URL https://w.sciencedirect.com/science/article/ pii/S0360835220305532. [4] Chen Zhang, Yu Xie, Hang Bai, Bin Yu, Weihong Li, and Yuan Gao. A survey on federated learning. Knowledge-Based Systems, 216:106775, 2021. ISSN 0950-7051. doi: https://doi.org/10.1016/j. knosys.2021.106775. URL https://w.sciencedirect.com/science/ article/pii/S0950705121000381. [5] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and trends® in machine learning, 14(1–2):1–210, 2021. [6] Han Xu, Priyadarsi Nanda, and Jie Liang. Reciprocal federated learn- ing framework: Balancing incentives for model and data owners. Fu- ture Generation Computer Systems, 161:146–161, 2024. ISSN 0167- 739X. doi: https://doi.org/10.1016/j.future.2024.06.055. URL https: //w.sciencedirect.com/science/article/pii/S0167739X24003558. [7] Xuechen Chen, Aixiang Wang, Xiaoheng Deng, and Jinsong Gui. Latency-efficient wireless federated learning with sparsification and quantization for heterogeneous devices. IEEE Internet of Things Journal, 12(1):488–501, 2025. doi: 10.1109/JIOT.2024.3462722. [8] Abdul Wahab Mamond, Majid Kundroo, and Taehong Kim. Autoencoder-based decentralized federated learning for efficient communication. Computer Networks, 272:111676, 2025. ISSN 1389-1286. doi: https://doi.org/10.1016/j.comnet.2025.111676. URL https://w.sciencedirect.com/science/article/pii/ S1389128625006437. [9] Pham Khanh Quan, Majid Kundroo, and Taehong Kim. Experimental evaluation and analysis of federated learning in edge computing environments. IEEE Access, 11:33628–33639, 2023. doi: 10.1109/ ACCESS.2023.3262945. [10] Bernardo Barreto, Carlos Senna, Pedro Rito, and Susana Sargento. Mobfedls: A framework to provide federated learning for mobile nodes in v2x environments. Future Generation Computer Systems, 163:107514, 2025. ISSN 0167-739X. doi: https://doi.org/10.1016/ j.future.2024.107514. URL https://w.sciencedirect.com/science/ article/pii/S0167739X24004783. [11] Hemant Kumar Apat, Rashmiranjan Nayak, and Bibhudatta Sahoo. A comprehensive review on internet of things application placement in fog computing environment. Internet of Things, 23:100866, 2023. ISSN 2542-6605. doi: https://doi.org/10.1016/j.iot.2023. 100866. URL https://w.sciencedirect.com/science/article/pii/ S2542660523001890. [12] Hemant Kumar Apat, Veena Goswami, Bibhudatta Sahoo, Ra- bindra K. Barik, and Manob Jyoti Saikia. Fog service place- ment optimization: A survey of state-of-the-art strategies and tech- niques. Computers, 14(3), 2025. ISSN 2073-431X. doi: 10.3390/ computers14030099. URL https://w.mdpi.com/2073-431X/14/3/99. [13] Fatih Ilhan, Gong Su, Qingyang Wang, and Ling Liu. Scalable federated learning with system heterogeneity. In 2023 IEEE 43rd In- ternational Conference on Distributed Computing Systems (ICDCS), pages 1037–1040, 2023. doi: 10.1109/ICDCS57875.2023.00113. [14] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60, 2020. doi: 10.1109/MSP. 2020.2975749. [15] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. NA, 2018. doi: 10.48550/ARXIV.1806.00582. URL https://arxiv.org/ abs/1806.00582. [16] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. Federated learning on non-iid data: A survey. Neurocomputing, 465:371–390, 2021. ISSN 0925-2312. doi: https://doi.org/10.1016/j.neucom.2021. 07.098. URL https://w.sciencedirect.com/science/article/pii/ S0925231221013254. [17] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017. [18] Yi Hu, Hanchi Ren, Chen Hu, Jingjing Deng, and Xianghua Xie. An element-wise weights aggregation method for federated learn- ing. In 2023 IEEE International Conference on Data Mining Workshops (ICDMW), page 188–196. IEEE, December 2023. doi: 10.1109/icdmw60847.2023.00031. URL http://dx.doi.org/10.1109/ ICDMW60847.2023.00031. [19] Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi, and Phillip B Gibbons. Federated learning under distributed concept drift. In International Conference on Artificial Intelligence and Statistics, pages 5834–5853. PMLR, 2023. [20] Leyla Rahimli, Feras M. Awaysheh, Sawsan Al Zubi, and Sadi Alawadi. Federated learning drift detection : An empirical study on the impact of concept and data drift. In 2024 2nd International Conference on Federated Learning Technologies and Applications, FLTA 2024 :, pages 241–250, 2024. ISBN 9798350354812. doi: 10.1109/FLTA63145.2024.10839814. [21] David A. Levin, Yuval Peres, and Elizabeth L. Wilmer. Markov Chains and Mixing Times. American Mathematical Society, Provi- dence, RI, 2nd edition, 2017. ISBN 978-1-4704-2962-1. [22] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heteroge- neous networks. Proceedings of Machine learning and systems, 2: 429–450, 2020. [23] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611–7623, 2020. [24] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papail- iopoulos, and Yasaman Khazaeni. Federated learning with matched averaging, 2020. URL https://arxiv.org/abs/2002.06440. [25] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. Fedbn: Federated learning on non-iid features via local batch normalization, 2021. URL https://arxiv.org/abs/2102.07623. [26] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In Interna- tional conference on machine learning, pages 5132–5143. PMLR, 2020. [27] Boyuan Li, Shengbo Chen, and Keping Yu. Feddkw – federated learning with dynamic kullback–leibler-divergence weight. ACM Trans. Asian Low-Resour. Lang. Inf. Process., April 2023. ISSN 2375-4699. doi: 10.1145/3594779. URL https://doi.org/10.1145/ 3594779. Just Accepted. [28] Gonzalo Iñaki Quintana, Laurence Vancamberg, Vincent Jugnon, Mathilde Mougeot, and Agnès Desolneux. Bn-scaffold: controlling the drift of batch normalization statistics in federated learning, 2024. : Preprint submitted to ElsevierPage 13 of 14 FedTVD URL https://arxiv.org/abs/2410.03281. [29] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3710–3722, 2021. doi: 10.1109/TNNLS. 2020.3015958. [30] Minghao Li, Dmitrii Avdiukhin, Rana Shahout, Nikita Ivkin, Vladimir Braverman, and Minlan Yu. Federated learning clients clustering with adaptation to data drifts, 2024. URL https://arxiv. org/abs/2411.01580. [31] Yuxiao Deng, Anqi Wang, Lei Zhang, Ying Lei, Beibei Li, and Yizhou Li. Fedrfc: Federated learning with recursive fuzzy clustering for improved non-iid data training. Future Generation Computer Systems, 160:835–843, 2024. ISSN 0167-739X. doi: https://doi.org/ 10.1016/j.future.2024.06.049. URL https://w.sciencedirect.com/ science/article/pii/S0167739X24003509. [32] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Net- works and Learning Systems, 34(12):9587–9603, 2023. doi: 10.1109/ TNNLS.2022.3160699. [33] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(9):11237–11244, Jun. 2023. doi: 10. 1609/aaai.v37i9.26330. URL https://ojs.aaai.org/index.php/AAAI/ article/view/26330. [34] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H. Brendan McMa- han. Adaptive federated optimization, 2021. URL https://arxiv.org/ abs/2003.00295. [35] Abdul Wahab Mamond, Majid Kundroo, Seong-eun Yoo, Seonghoon Kim, and Taehong Kim. Fldqn: Cooperative multi-agent federated reinforcement learning for solving travel time minimization problems in dynamic environments using sumo simulation. Sensors, 25(3), 2025. ISSN 1424-8220. doi: 10.3390/s25030911. URL https: //w.mdpi.com/1424-8220/25/3/911. [36] Hemant Kumar Apat and Bibhudatta Sahoo. Jagw: A hybrid meta- heuristic algorithm for iot workflow placement in fog computing en- vironment. Simulation Modelling Practice and Theory, 144:103163, 2025. ISSN 1569-190X. doi: https://doi.org/10.1016/j.simpat.2025. 103163. URL https://w.sciencedirect.com/science/article/pii/ S1569190X2500098X. [37] Hemant Kumar Apat, Pranati Sattarapu, Rudra Narayan Dash, Veena Goswami, Sagarika Mohanty, and Rabindra K. Barik. Leveraging towards multi-objective iot application placement in fog computing environment. In 2023 IEEE 3rd International Conference on Smart Technologies for Power, Energy and Control (STPEC), pages 1–6, Dec 2023. doi: 10.1109/STPEC59253.2023.10430628. [38] Majid Kundroo and Taehong Kim. Efficient federated learning with adaptive client-side hyper-parameter optimization. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS), pages 973–974, 2023. doi: 10.1109/ICDCS57875.2023. 00103. [39] Majid Kundroo and Taehong Kim. Demystifying impact of key hyper-parameters in federated learning: A case study on cifar-10 and fashionmnist. IEEE Access, 12:120570–120583, 2024. doi: 10.1109/ACCESS.2024.3450894. [40] Kummari Naveen Kumar, Chalavadi Krishna Mohan, and Linga Reddy Cenkeramaddi. The impact of adversarial attacks on federated learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2672–2691, 2024. doi: 10.1109/TPAMI.2023.3322785. [41] Majid Kundroo, Ghani Haider, Nguyen Khoa, Abdul Wahab Ma- mond, and Taehong Kim. Fedeasy : Federated learning with ease. SoftwareX, 31:102276, 2025. ISSN 2352-7110. doi: https://doi.org/ 10.1016/j.softx.2025.102276. URL https://w.sciencedirect.com/ science/article/pii/S2352711025002432. [42] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017. [43] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. [44] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceed- ings of the IEEE, 86(11):2278–2324, 1998. [45] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778, 2016. [46] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification, 2019. URL https://arxiv.org/abs/1909.06335. : Preprint submitted to ElsevierPage 14 of 14