Paper deep dive

Hierarchical Federated Unlearning for Large Language Models

Yisheng Zhong, Zhengbang Yang, Zhuangdi Zhu

Year: 2025Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 43

Models: Zephyr-7B-beta

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/11/2026, 1:10:21 AM

Summary

The paper introduces 'Federated UnLearning Merge' (FULM), a framework for decentralized, privacy-preserving unlearning in Large Language Models (LLMs). FULM addresses the challenges of heterogeneous unlearning requests and asymmetric data access by decoupling unlearning and retention into task-specific adapters. It employs a hierarchical merging strategy that clusters adapters based on parameter distribution similarity and uses voting-based aggregation to mitigate inter-domain and intra-domain interference, effectively balancing unlearning performance with model utility.

Entities (6)

LLM · technology · 99%FULM · framework · 98%LoRA · technique · 98%MUSE · benchmark · 95%TOFU · benchmark · 95%WMDP · benchmark · 95%

Relation Signals (3)

FULM → evaluatedon → WMDP

confidence 98% · We evaluate our approach on three LLM unlearning benchmarks: WMDP, TOFU, and MUSE.

FULM → utilizes → LoRA

confidence 95% · each adaptation is encapsulated via LoRA fine-tuning

FULM → mitigates → Inter-domain interference

confidence 92% · FULM identifies disparities within task adapters... and applies hierarchical aggregation strategies tailored to these patterns.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large Language Models (LLMs) are increasingly integrated into real-world applications, raising concerns about privacy, security and the need to remove undesirable knowledge. Machine Unlearning has emerged as a promising solution, yet faces two key challenges: (1) practical unlearning needs are often continuous and heterogeneous, and (2) they involve decentralized, sensitive data with asymmetric access. These factors result in inter-domain and intra-domain interference, which further amplifies the dilemma of unbalanced forgetting and retaining performance. In response, we propose a federated unlearning approach for LLMs that is scalable and privacy preserving. Our method decouples unlearning and retention via task-specific adapter learning and employs a hierarchical merging strategy to mitigate conflicting objectives and enables robust, adaptable unlearning updates. Comprehensive experiments on benchmarks of WMDP, MUSE, and TOFU showed that our approach effectively handles heterogeneous unlearning requests while maintaining strong LLM utility compared with baseline methods.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

42,574 characters extracted from source content.

Expand or collapse full text

Hierarchical Federated Unlearning for Large Language Models Yisheng Zhong yzhong7@gmu.edu George Mason UniversityFairfaxVAUSA , Zhengbang Yang zyang30@gmu.edu George Mason UniversityFairfaxVAUSA and Zhuangdi Zhu zzhu24@gmu.edu George Mason UniversityFairfaxVAUSA (2018) Abstract. Large Language Models (LLMs) are increasingly integrated into real-world applications, raising concerns about privacy, security and the need to remove undesirable knowledge. Machine Unlearning has emerged as a promising solution, yet faces two key challenges: (1) practical unlearning needs are often continuous and heterogeneous, and (2) they involve decentralized, sensitive data with asymmetric access. These factors result in inter-domain and intra-domain interference, which further amplifies the dilemma of unbalanced forgetting and retaining performance. In response, we propose a federated unlearning approach for LLMs that is scalable and privacy preserving. Our method decouples unlearning and retention via task-specific adapter learning and employs a hierarchical merging strategy to mitigate conflicting objectives and enables robust, adaptable unlearning updates. Comprehensive experiments on benchmarks of WMDP, MUSE, and TOFU showed that our approach effectively handles heterogeneous unlearning requests while maintaining strong LLM utility compared with baseline methods.111Code will be published at: https://anonymous.4open.science/r/Unlearning-B493 LLM unlearning, Federated unlearning, Hierarchical model merging, Task heterogeneity, Unlearning-utility trade-off †copyright: acmlicensed†journalyear: 2025†doi: X.X†conference: Federated Learning for Data Mining and Graph Analytics ; August 4, 2025; Toronto, ON, Canada†isbn: 978-1-4503-X-X/2018/06†ccs: Computing methodologies Learning paradigms 1. Introduction Large Language Models (LLMs) have shown remarkable capabilities in generating content that resonates with human knowledge. While LLMs are increasingly integrated into real-world applications, growing concerns about privacy and copyright protection (Voigt and Bussche, 2017; Pardau, 2018) have called for mechanisms to actively remove undesirable knowledge previously learned by these models. Accordingly, Machine Unlearning has emerged as a promising technique to tackle this need without retraining the model from scratch (Yao et al., 2024a; Nguyen et al., 2024; Xu et al., 2023). Prior LLM unlearning methods assume access to both the data to be forgotten and a representative subset of the original training data to retain general knowledge (Liu et al., 2025). They commonly optimize on a dual objective with competing components: one optimizing on the unlearning data, and the other the retaining data (Maini et al., 2024), yet still face trade-offs including incomplete unlearning of the target content and over-forgetting of retention knowledge. Moreover, in realistic scenarios, unlearning requests can arise externally, continuously, from decentralized sources, ranging from white-box model users and intellectual property owners to red-teamers, who often have individual unlearning needs but are reluctant to reveal the sensitive unlearning data. In response, we investigate a practical paradigm of Federated Unlearning for LLMs, which is built upon a federated learning mechanisms (McMahan et al., 2017) to tackle LLM unlearning with asymmetric data access: the party initiating the unlearning request (client) may lack access to retaining data, while the LLM provider (server) is restricted from directly accessing the unlearning data due to privacy or security regulations. This scheme enables decentralized unlearning requests without requiring centralized data collection. Despite its innovative framework, our empirical study shows that a naive federated mechanism alone often results in performance degradation, either degrading unlearning performance or at the sacrifice of retention utility drop, which reflects the persistent challenge in existing unlearning and becomes more pronounced given decentralized and heterogeneous unlearning requests. To address these entangled challenges, we propose a systematic algorithm and framework, named Federated UnLearning Merge (FULM), to synergize the decentralized unlearning requests continuously directed to an LLM server, as shown in Figure 1. FULM decouples the unlearning and retention objectives and instead integrates retention signals during federated aggregation to preserve useful knowledge in the LLM. By analyzing client updates, FULM identifies disparities within task adapters regarding parameter distributions, magnitudes, and directions, and applies hierarchical aggregation strategies tailored to these patterns. We evaluate our approach on three LLM unlearning benchmarks: WMDP, TOFU, and MUSE. Results show that given dynamic unlearning tasks, FULM outperforms existing baselines of unlearning or FL merging, and can effectively remove target knowledge while preserving model utility, thus offering an accountable and privacy-preserving approach to LLM knowledge governance. Figure 1. Our method tackles heterogeneous, decentralized LLM unlearning requests without transmitting the sensitive unlearning or retention data. 2. Related Work Machine unlearning has recently been applied to LLMs to address security and privacy concerns (Yao et al., 2024b; Cao and Yang, 2015; Shastri et al., 2019; Mantelero, 2013). Early approaches primarily involve gradient ascent that fine-tunes the model to increase prediction error on the forget set (Jang et al., 2022). Different efforts have been proposed to improve unlearning stability. Negative Preference Optimization (NPO) (Zhang et al., 2024) reframes unlearning as a preference alignment task by treating the forget set as negative examples, requiring curation of paired response data regarding forgetting knowledge. Task vector methods (Ilharco et al., 2023) subtract the influence of forgetting knowledge from the model by negating the parameters of task adapters related to forgetting knowledge, while interpolation-based methods like WHP (Eldan and Russinovich, 2023) softly blend target and reinforced models. To preserve model utility during unlearning, recent work proposed regularization based strategies to maintain performance on a retention set. Gradient Descent on the Retain Set (GDR) (Liu et al., 2022a; Maini et al., 2024; Zhang et al., 2024) adds a standard cross-entropy loss over the retain data, while KL Divergence Minimization (KLR) (Maini et al., 2024; Zhang et al., 2024) aligns the output distributions of the unlearned and original models. These methods improved stability but assume simultaneous access to both the unlearn and the retain dataset. A few works explored retaining data-free LLM unlearning at the cost of non-negligible utility degradation (Wang et al., 2025). Federated Learning and Unlearning: Federated Learning (FL) is a decentralized machine learning technique that enables clients to collaboratively train models without sharing raw data (Li et al., 2020; McMahan et al., 2017). FL was initially proposed for uniform client distributions and evolved to tackle data or systematic heterogeneity (McMahan et al., 2017; Lu et al., 2024; Li et al., 2022). (Liu et al., 2021, 2022b). A few pioneering works explored federated unlearning on traditional, smaller-scale neural networks for class-wise prediction tasks. Methods include client-side retraining (Liu et al., 2021), knowledge distillation (Wu et al., 2022), or model parameter pruning (Wang et al., 2022), which presume access to class labels (Wang et al., 2022), extensive retraining rounds (Liu et al., 2022b). Adapter Merging for LLMs aims to integrate multiple independently-trained adapters into a single model for multi-task purposes without further fine-tuning. While Low-Rank Adaptation (LoRA) (Hu et al., 2021) has become dominant for parameter-efficient fine-tuning, merging these adapters presents significant challenges due to weight entanglement and interference between task-specific updates (Tang et al., 2024). Task Arithmetic (TA) (Ilharco et al., 2023) is a straightforward paradigm for LLM adapter composition by linearly combining fine-tuned LLM adapters, called task vectors. This approach enables merging without retraining, but has been shown to suffer from interference between task-specific updates (Yadav et al., 2023; Daheim et al., 2024). To address this, recent methods include pruning low-magnitude weights (Yadav et al., 2023), rescaling sparse vectors (He et al., 2025), or aligning models in tangent space to reduce conflict (Ortiz-Jimenez et al., 2023). Specifically, TIES (Yadav et al., 2023) mitigates this through a three-step process: (1) trimming negligible parameter changes to reduce noise, (2) selecting dominant signs across adapters for each parameter, and (3) averaging updates matching the selected signs. Our work connects to prior work by efficiently merging multiple unlearning and retention adapters for balanced model utility, without assuming access to relevant data or post-hoc finetuning. Our empirical study (Section 5) demonstrated that neither task arithmetic nor the TISE method alone perfectly addresses the heterogeneous unlearning task merging and thus motivated a more tailored merging approach. 3. Preliminaries Machine Unlearning. Machine unlearning is a mechanism to selectively remove undesirable knowledge from a trained model without the need to retrain the model from scratch. Given a model θ, a forget set uD_u containing undesirable knowledge, and a retention set rD_r representing useful knowledge, a representative unlearning objective optimizes the following: (1) minθ′⁡ℒunlearn(u;θ′)+λℒretain(r;θ′), _θ L_unlearn(D_u;θ )+ _retain(D_r;θ ), where λ balances the trade-off between forgetting and retention. Conventional unlearning methods usually implement gradient ascent on ℒunlearnL_unlearn, and optionally apply regularization techniques such as KL-divergence minimization to constrain the model parameter divergence from θ to θ′θ (Maini et al., 2024). Optimizing the above goal requires simultaneous access to both datasets of uD_u and rD_r. Federated Learning (FL). In a standard FL framework, a client k∈[K]k∈[K] holds a local dataset kD_k, which computes a local gradient update Δθk _k based on the global model θ: Δθk(t)=−η∇ℒ(k;θ(t)), _k^(t)=-η (D_k;θ^(t)), where ∇θk(t)∇ _k^(t) is the local model update after training with the global model copy θ(t)θ^(t) on local data kD_k at communication round t, and η is the learning rate. The server aggregates gradient updates from different clients to update the global model: θ(t+1)←θ(t)+1K∑k=1KΔθk(t).θ^(t+1)←θ^(t)+ 1KΣ _k=1^K _k^(t). FL typically involves multiple communication rounds t to iteratively improve the global model. In contrast, we employ a one-shot FL setting and omit frequent parameter exchanges. Low Rank Adaptation (LoRA). Full model fine-tuning on LLMs is computationally expensive and memory-intensive. To address this, LoRA provides parameter-efficient fine-tuning of pretrained LLMs by introducing low-rank matrices. Instead of directly modifying the pretrained weight matrix W∈ℝd×dW ^d× d, LoRA defeines the adapted weight matrix W′W as: W′=W+BA,W =W+BA, where W denotes the original frozen pretrained weights, and B∈ℝd×r,A∈ℝr×dB ^d× r,A ^r× d are low-rank matrices with rank r≪dr d. LoRA is particularly well-suited for machine unlearning, as the knowledge designated for removal usually occupies a small subspace of the model’s overall knowledge and thus can be effectively captured by low-rank adaptations. 4. Method Problem Setting: We consider a federated unlearning framework where a central server hosts a foundation model (e.g., an open-sourced LLM on the Hugging Face platform) and receives multiple unlearning requests from decentralized clients. Each client k∈[K]k∈[K] has white-box access to LLM parameters θ and a private unlearning dataset DukD^k_u containing knowledge to be forgotten. Optionally, the client may obtain a retention dataset DrkD^k_r of information to preserve, although it is often absent with Drk=∅D^k_r= . The server maintains access to pretraining data, which partially overlaps with client data but cannot be shared due to privacy constraints. 4.1. Split-and-Merge: Decoupled Unlearning and Retenion via Dual Task adapters To address asymmetric data access while maintaining balanced forgetting and preservation, we first propose to decouple unlearning and retention into independent adaptation tasks. Our framework encompasses two phases: a split phase where unlearning and retention are handled independently, and merge phase where the resulting adapters are aggregated at the server. Specifically, clients first perform unlearning locally on their private datasets DukD^k_u using standard unlearning objectives, such as gradient ascent (Neel et al., 2021). Whereas, retention adapters can be generated either by clients with available retention data DrkD^k_r, or by the server using a lightweight subset of pretraining data. Without losing generality, we assume that each adaptation is encapsulated via LoRA fine-tuning, although our methods can be extended to varying adapter architectures that capture task-specific parameter updates. Empirical results (Section 5) show that this decoupled approach outperforms traditional unlearning with a dual objective as in Eq 1, especially when the unlearning and retention datasets are resemblant and thus conflicting, such as in structured entry-wise element unlearning scenarios (Maini et al., 2024). 4.2. Multi-Task Unlearning Transfer via Hierarchical Federated Merging Once the LLM server collects multiple task adapters for unlearning and retention, the core challenge resides in an effective merging approach to balance two complementary tradeoffs: the inter-domain unlearning interference, and the intra-domain interference between potentially conflicting unlearning and retaining task objectives. Let ∇θu(r)i∇ _u(r)^i denote the unlearning (or retaining) task adapter trained on dataset iD_i. The above interferences mainly stem from two aspects: (1) the underlying data distribution manifested by iD_i, and (2) the task objective ℒ(Di;θ)L(D_i;θ) of either forgetting or preserving knowledge from such dataset. We take a hierarchical, data-free merging approach to tackle task vectors’ heterogeneity induced by both their data and learning objectives, without presuming the LLM server to access proxy data for merging. Similarity-aware Adapter Clustering: The LLM server first clusters task vectors according to their parameter distribution similarity. Our rationale is that task adapters trained with similar objectives ℒ(⋅)L(·) and data distributions iD_i produce parameter updates induced by ∇ℒ(Di;θ) (D_i;θ), that adapt the model toward similar directions, resulting in positively correlated perturbations, and vice versa. Figure 2. Cosine similarity across task vectors, where vectors from near-iid sources exhibit high similarity, and those from heterogeneous domains are nearly orthogonal. Task vectors trained for retention (TOFUrTOFU_r) show negative correlation to their unlearning counterparts (TOFUTOFU). Without losing generality, we use cosine similarity as the task adapter correlation metric. The results of our empirical study in Figure 2 confirm our design, in that both the data distribution and the learning objectives influence the similarity patterns of the task vector. Specifically, (i) unlearning task adapters trained on orthogonal datasets DiD_i and DjD_j produce orthogonal parameter shifts with near-zero cosine similarity. Meanwhile, (i) unlearning adapters trained on near-iid datasets exhibit a strong positive correlation in their parameter updates. Interestingly, (i) unlearning and retention adapters θui _u^i and θrj _r^j trained on disjoint but near-iid data DiD_i and DjD_j demonstrate negatively correlated similarity that reflects their contrastive objectives. Hierarchical Merging Strategies: Following task vector clustering via cosine similarity, we propose a two-step hierarchical merging strategy: Step 1: Intra-Cluster Merging. We first define a similarity threshold ξ>0ξ>0 to group adapters with high cosine similarity, which correspond to near-iid data and aligned objectives. Within each cluster kC_k, we apply a voting-based merging strategy such as TIES (Yadav et al., 2023), which selects dominant parameter directions while minimizing destructive interference. Compared to task arithmetic methods (e.g. addition), voting-based merging avoids over-amplifying similar updates and lead to more stable representations for tightly aligned tasks (see Section 5.2). For parameter-efficient adapters like LoRA, similarity is computed after recovering low-rank representations in the original parameter space. Thus, our method is agnostic to specific adapter architectures, so long as they are compatible with numerical operations on the same LLM backbone. For instance, adapters can be learned with LoRA that target different linear layer combinations (e.g. selective vs. all attention layers). Step 2: Inter-Cluster Merging. We then treat the merged cluster-level adapters as a set =∇Θkk=1||C=\∇ _k\_k=1^|C|, where each ∇Θk∇ _k represents a merged vector from one cluster. Since these vectors capture distinct information domains, we use task arithmetic addition to preserve all unique update information without dilution. The complete hierarchical merging process is formalized as: ∇Θ←sum(∇Θk∈|Θk←vote(∇θj∈kj=1|k|)k=1||), ∇ _sum ( \∇ _k | _k _vote(\∇ _j _k\_j=1^|C_k|) \_k=1^|C| ), where voteA_vote represents voting-based intra-cluster aggregation and sumA_sum denotes summation across clusters. This approach eliminates the need to manage multiple task-specific adapters and supports scalability for continual and dynamic unlearning requests. The overall process is summarized in Algorithm 1 of Appendix. 5. Experiments 5.1. Experimental Setup Datasets: We conducted experiments on three representative unlearning benchmarks: (1) WMDP (Yao et al., 2024a), for which we employed two domain-specific forgetting sets, Biosecurity and Cybersecurity, that contain sensitive content sourced from PubMed and GitHub; (2) TOFU (Maini et al., 2024), which is a QA benchmark featuring fictional authors as the unlearning target and real-world factual QA pairs as the retention set; and (3) MUSE (Nguyen et al., 2024), from which we selected a subset of the Harry Potter (HP) series, using contents from Harry Potter and the Goblet of Fire as the forgetting set. Evaluation Metrics. We primarily measured the Forget↓ performance using the forgetting set to assess how well target content is removed, and the model Utility↑ as the retention performance, which is evaluated on MMLU (Hendrycks et al., 2021) benchmark that contains 14,079 questions across 57 tasks. Additionally, for TOUFU-related tasks, we added three more metrics: Retain ↑ , Real Authors↑ , and Real World↑ , all measured using ROUGE, a standard metric that computes the textual overlap between model outputs and reference texts. Specifically, Retain↑ on a held-out retaining set measures the preservation of useful knowledge; and Real Authors↑ / Real World↑ refer to factual QA subsets in the TOFU dataset not intended for removal. We eventually report an averaged, balanced performance metric: Overall=1||[∑imri+∑j(1−muj)]Overall= 1|M| [ _im^i_r+ _j(1-m^j_u) ], where mrim^i_r measures retention (e.g. utility on desirable knowledge), and mujm^j_u measures unlearning, reversed to reflect forgetting effectiveness. Baselines. For evaluating merging performance, we compared our method against the following baselines: (1) the Avg that computes the parameter-wise average of all task vectors, (2) SUM which performs direct arithmetic addition, (3) TIES (Yadav et al., 2023), a voting-based gradient merging strategy, and (4) KNOT (Stoica et al., 2024) that aligns LoRA adapters on a low-dimensional parameter subspace via joint SVD-based transformations to improve merge quality. To compare decoupled versus joint unlearning, we adopt Gradient Difference (GD) from (Maini et al., 2024) as a baseline, which is a joint optimization approach by minimizing the composite loss of Equation 1. Decentralized Unlearning Scenarios: We adopted the Zephyr-7B model as our base LLM. All evaluated task adapters were learned with LoRA with a rank r=16r=16 and a scaling factor of 3232. For the TOFU and MUSE datasets, we applied LoRA to all linear layers, while for WMDP, we applied LoRA only to the down-projection linear layers of the fifth to seventh layers. We evaluated two FL settings: (1) a near-iid setup using TOFU datasets, where the unlearning dataset was evenly split across five clients, along with a retention adapter trained on a separate dataset that was unavailable to clients; and (2) a heterogeneous setup with seven unlearning adapters trained on diverse domains, with three on WMDP-Bio, three on WMDP-Cyber, where the WMDP-Cyber and WMDP-Bio forgetting sets were randomly partitioned into three subsets each, and one on HP. For both settings, all unlearning adapters were learned with only unlearning data. See Appendix B.1 for details. 5.2. Performance Evaluation 5.2.1. Effects of Hierarchical Merging: Table 1 and Table 2 summarize the performance of FULM and baseline merging methods under near-iid and heterogeneous setups, respectively. Across both scenarios, our hierarchical merging strategy consistently achieves the most balanced trade-off between unlearning and retention. In the near-iid setting (Table 1), where merging was performed on 5 client-provided unlearning adapter ∇θui∇ _u^i and 1-server provided retention adapter θr _r, non-hierarchical baselines struggled to maintain balance. For example, the SUM merge amplifies unlearning but severely degrades utility (56.92%↑ ), compared to FULM (70.62%↑ ). Meanwhile, TIES and AVG disrupt retention effects and lead to impaired performance on desirable knowledge categories like Real Authors/Real World. The KNOT baseline that leverages SVD prior to adapter merging shows negligible difference from other baselines. For the heterogeneous setting (Table 2), non-hierarchical merging lead to severe cross-domain interference and ineffective unlearning. While SUM merging achieves comparable unlearning to FULM, it dramatically sacrifices model utility. FULM maintains strong forgetting while effectively preserving critical knowledge. KNOT merging is excluded from evaluation of heterogeneous setting, as it requires unified LoRA architectures for SVD decomposition. Table 1. Merging Near-iid Unlearning Domains. Method Real Authors↑ Real World↑ Retain↑ Forget↓ Utility↑ Overall↑ Ties 50.38% 59.05% 48.94% 50.67% 57.42% 53.02% SUM 87.58% 78.95% 40.38% 39.17% 56.92% 64.93% AVG 62.71% 61.79% 49.66% 49.90% 63.29% 57.51% KnOT 52.37% 61.84% 45.59% 48.12% 56.25% 66.21% FULM (proposed) 99.06% 78.35% 64.48% 51.84% 70.62% 72.13% Table 2. Merging Heterogeneous Unlearning Domains. Unlearning Tasks Biology↓ Cyber Security↓ Harry Potter↓ Model Utility↑ Overall↑ Avg All Vectors 59.85% 38.42% 41.17% 56.79% 54.34% TIES All Vectors 62.45% 41.87% 46.71% 58.28% 51.81% SUM All Vectors 27.46% 34.47% 29.31% 30.42% 59.80% FULM (proposed) 39.05% 25.82% 34.48% 57.25% 64.48% 5.2.2. Effects of Decoupled Unlearning and Retention : As shown in Table 3, our split-and-merge strategy achieves optimal balance when merging one unlearning and one retention adapter. Notably, our decoupled approach outperforms GD (joint training on combined datasets) and demonstrate that separate optimization followed by task-arithmetic merging is more effective than joint optimization especially when data conflicts. This also makes our method suitable for asymmetric data access where different parties hold unlearning and retention data. Table 3. Effects of merging two unlearning and retention adapters on the TOFU dataset. GD represents joint training. Method Real Authors↑ Real World↑ Retain↑ Forget↓ Utility↑ Overall↑ GD (centralized) 55.27% 74.93% 89.62% 53.57% 67.16% 66.68% KnOT 86.83% 75.78% 48.94% 50.19% 69.97% 66.27% FULM (SUM) 86.13% 75.85% 48.37% 20.79% 70.62% 72.04% 5.2.3. Breaking Down Analysis of Hierarchical Unlearning Merging Step1: Intra-Cluster Merging on Near-iid Task Vectors: We investigated different unlearning merging strategies for adapters within a cluster. As shown in Table 5.2.3, TIES achieves stronger unlearning (e.g., Cyber 26.02%26.02\% and Bio 41.32%41.32\%) while preserving utility (58.91%58.91\% and 57.95%57.95\%), especially outperforming SUM, which severely degrades retention utility, as it accumulates update magnitudes across unlearning tasks, leading to overly large activations that compromise model general usability. These indicate that voting-based merging (TIES) is suitable for near-iid task vectors by mitigating mutual interference.while promoting generalization. See Table B.2 in Appendix B.2 for full results. Table 4. Intra-Cluster Merging (Step 1) (a) WMDP-Cyber Method Bio↑ Cyber↓ Util.↑ Overall↑ Pretrained 63.71% 44.00% 58.19% 59.30% !50 AVG 65.44% 28.52% 58.74% 65.22% TIES 65.44% 26.02% 58.91% 66.11% SUM 24.98% 26.62% 23.90% 40.75% (b) WMDP-Bio Method Bio↓ Cyber↑ Util.↑ Overall↑ Pretrained 63.71% 44.00% 58.19% 46.16% !50 AVG 47.36% 44.22% 58.02% 51.63% TIES 41.32% 44.29% 57.95% 53.64% SUM 24.98% 32.51% 52.07% 53.20% Step2: Merging non-IID Task Vectors: After applying TIES within each cluster, we obtain three centroid adaptors: WMDP-Cyber, WMDP-Bio, and MUSE-HP. We then perform the second-stage merging across these non-iid domains. As shown in Table 5.2.3, the SUM method delivers the strongest forgetting across all domains while maintaining competitive utility (57.25%). This indicates that, in heterogeneous scenarios, direct summation preserves domain-specific forgetting with minimal interference as a simple yet effective aggregation strategy. Full results are shown in Table LABEL:tab:wmdp_noniid_full in Appendix B.2. Table 5. Inter-Cluster Merging (Step 2) Unlearning Tasks Biology↓ Cyber Security↓ Harry Potter↓ Model Utility↑ Overall↑ Pretrained Model 63.71% 44.00% 49.87% 58.19% 50.15% !50 Avg 63.63% 43.03% 46.36% 58.39% 51.34% TIES 62.45% 41.87% 46.71% 58.28% 51.81% SUM 39.05% 25.82% 34.48% 57.25% 64.48% 6. Conclusion We present FULM, a federated unlearning framework for LLMs, which enables continual and decentralized knowledge removal without centralized data aggregation. To address the challenge of asymmetric data access, where separate parties hold unlearning and retention data, FULM decouples unlearning and retention objectives and performs hierarchical task vector merging, which adapts to both near-iid and heterogeneous unlearning requests while preserving critical knowledge. Comprehensive experiments on WMDP, TOFU, and MUSE benchmarks show that FULM effectively removes undesirable content, maintains utility, and scales to heterogeneous real-world unlearning scenarios, offering a practical and privacy-preserving solution for LLM unlearning. References (1) Cao and Yang (2015) Yinzhi Cao and Junfeng Yang. 2015. Towards Making Systems Forget with Machine Unlearning. In 2015 IEEE Symposium on Security and Privacy. 463–480. https://doi.org/10.1109/SP.2015.35 Daheim et al. (2024) Nico Daheim, Thomas Möllenhoff, Edoardo Maria Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan. 2024. Model Merging by Uncertainty-Based Gradient Matching. arXiv:2310.12808 [cs.LG] https://arxiv.org/abs/2310.12808 Eldan and Russinovich (2023) Ronen Eldan and Mark Russinovich. 2023. Who’s Harry Potter? Approximate Unlearning in LLMs. arXiv:2310.02238 [cs.CL] https://arxiv.org/abs/2310.02238 He et al. (2025) Yifei He, Yuzheng Hu, Yong Lin, Tong Zhang, and Han Zhao. 2025. Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic. arXiv:2408.13656 [cs.LG] https://arxiv.org/abs/2408.13656 Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Understanding. arXiv:2009.03300 [cs.CY] https://arxiv.org/abs/2009.03300 Hu et al. (2021) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL] https://arxiv.org/abs/2106.09685 Ilharco et al. (2023) Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing Models with Task Arithmetic. arXiv:2212.04089 [cs.LG] https://arxiv.org/abs/2212.04089 Jang et al. (2022) Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022. Knowledge Unlearning for Mitigating Privacy Risks in Language Models. arXiv:2210.01504 [cs.CL] https://arxiv.org/abs/2210.01504 Li et al. (2022) Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. 2022. Federated learning on non-iid data silos: An experimental study. In 2022 IEEE 38th international conference on data engineering (ICDE). IEEE, 965–978. Li et al. (2020) Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated Learning: Challenges, Methods, and Future Directions. IEEE Signal Processing Magazine 37, 3 (May 2020), 50–60. https://doi.org/10.1109/msp.2020.2975749 Liu et al. (2022a) Bo Liu, Qiang Liu, and Peter Stone. 2022a. Continual Learning and Private Unlearning. arXiv:2203.12817 [cs.AI] https://arxiv.org/abs/2203.12817 Liu et al. (2021) Gaoyang Liu, Xiaoqiang Ma, Yang Yang, Chen Wang, and Jiangchuan Liu. 2021. FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models. In 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS). 1–10. https://doi.org/10.1109/IWQOS52092.2021.9521274 Liu et al. (2025) Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence (2025), 1–14. Liu et al. (2022b) Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. 2022b. The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retraining. In IEEE INFOCOM 2022 - IEEE Conference on Computer Communications. IEEE, 1749–1758. https://doi.org/10.1109/infocom48880.2022.9796721 Lu et al. (2024) Zili Lu, Heng Pan, Yueyue Dai, Xueming Si, and Yan Zhang. 2024. Federated Learning With Non-IID Data: A Survey. IEEE Internet of Things Journal 11, 11 (2024), 19188–19209. https://doi.org/10.1109/JIOT.2024.3376548 Maini et al. (2024) Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C. Lipton, and J. Zico Kolter. 2024. TOFU: A Task of Fictitious Unlearning for LLMs. First Conference on Language Modeling (2024). Mantelero (2013) Alessandro Mantelero. 2013. The EU Proposal for a General Data Protection Regulation and the Roots of the ‘Right to Be Forgotten’. Computer Law & Security Review 29, 3 (June 2013), 229–235. Available at SSRN: https://ssrn.com/abstract=2473151. McMahan et al. (2017) Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273–1282. Neel et al. (2021) Seth Neel, Aaron Roth, and Saeed Sharifi-Malvajerdi. 2021. Descent-to-Delete: Gradient-Based Methods for Machine Unlearning. In Proceedings of the 32nd International Conference on Algorithmic Learning Theory (Proceedings of Machine Learning Research, Vol. 132), Vitaly Feldman, Katrina Ligett, and Sivan Sabato (Eds.). PMLR, 931–962. https://proceedings.mlr.press/v132/neel21a.html Nguyen et al. (2024) Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2024. A Survey of Machine Unlearning. arXiv:2209.02299 [cs.LG] https://arxiv.org/abs/2209.02299 Ortiz-Jimenez et al. (2023) Guillermo Ortiz-Jimenez, Alessandro Favero, and Pascal Frossard. 2023. Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models. arXiv:2305.12827 [cs.LG] https://arxiv.org/abs/2305.12827 Pardau (2018) Stuart L. Pardau. 2018. The California Consumer Privacy Act: Towards a European-Style Privacy Regime in the United States? Journal of Technology Law & Policy 23, 1 (2018). https://scholarship.law.ufl.edu/jtlp/vol23/iss1/2 Shastri et al. (2019) Supreeth Shastri, Melissa Wasserman, and Vijay Chidambaram. 2019. The Seven Sins of Personal-Data Processing Systems under GDPR. In 11th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 19). USENIX Association, Renton, WA. https://w.usenix.org/conference/hotcloud19/presentation/shastri Stoica et al. (2024) George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. 2024. Model merging with SVD to tie the Knots. arXiv:2410.19735 [cs.CV] https://arxiv.org/abs/2410.19735 Tang et al. (2024) Anke Tang, Li Shen, Yong Luo, Yibing Zhan, Han Hu, Bo Du, Yixin Chen, and Dacheng Tao. 2024. Parameter Efficient Multi-task Model Fusion with Partial Linearization. arXiv:2310.04742 [cs.LG] https://arxiv.org/abs/2310.04742 Voigt and Bussche (2017) Paul Voigt and Axel Bussche. 2017. The EU General Data Protection Regulation (GDPR): A Practical Guide. https://doi.org/10.1007/978-3-319-57959-7 Wang et al. (2022) Junxiao Wang, Song Guo, Xin Xie, and Heng Qi. 2022. Federated Unlearning via Class-Discriminative Pruning. arXiv:2110.11794 [cs.CV] https://arxiv.org/abs/2110.11794 Wang et al. (2025) Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Parag Shah, Yujia Bao, Yang Liu, and Wei Wei. 2025. LLM Unlearning via Loss Adjustment with Only Forget Data. The Thirteenth International Conference on Learning Representations (2025). Wu et al. (2022) Chen Wu, Sencun Zhu, and Prasenjit Mitra. 2022. Federated Unlearning with Knowledge Distillation. arXiv:2201.09441 [cs.LG] https://arxiv.org/abs/2201.09441 Xu et al. (2023) Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, and Philip S. Yu. 2023. Machine Unlearning: A Survey. arXiv:2306.03558 [cs.CR] https://arxiv.org/abs/2306.03558 Yadav et al. (2023) Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. TIES-Merging: Resolving Interference When Merging Models. arXiv:2306.01708 [cs.LG] https://arxiv.org/abs/2306.01708 Yao et al. (2024a) Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024a. Large language model unlearning. Advances in Neural Information Processing Systems 37 (2024), 105425–105475. Yao et al. (2024b) Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024b. Large Language Model Unlearning. arXiv:2310.10683 [cs.CL] https://arxiv.org/abs/2310.10683 Zhang et al. (2024) Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. 2024. Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning. arXiv:2404.05868 [cs.LG] https://arxiv.org/abs/2404.05868 Appendix A Algorithm Overview Algorithm 1 One-Round Hierarchical Federated Unlearning Merging 1: 2:- Clients 1,2,…,K\1,2,...,K\ with private unlearning datasets Duk\D_u^k\ and optional retention datasets Drk\D_r^k\ 3:- Server with global model parameters θ and access to pretraining data. 4:Broadcast: Server transmits current foundation model θ to all clients 5:for each client k∈[K]k∈[K] in parallel do 6: Perform LoRA-based unlearning on DukD_u^k to obtain adapter ∇θuk∇ _u^k 7: if Drk≠∅D_r^k≠ then 8: Optionally perform retention fine-tuning to obtain ∇θrk∇ _r^k 9: end if 10: Return task adapters ∇θuk∇ _u^k (and optionally ∇θrk∇ _r^k) to the server 11:end for 12:Similarity-aware Clustering: Cluster all received task adapters ∇θu(r)k\∇ _u(r)^k\ into sets 1,…,||\C_1,...,C_|C|\ based on cosine similarity 13:for each cluster kC_k do 14: Intra-cluster merging via voting: ∇Θk←vote(∇θj∈k)∇ _k _vote (\∇ _j _k\ ) 15:end for 16:Inter-cluster Aggregation: Merge cluster-level vectors: ∇Θ←sum(∇Θkk=1||)∇ _sum (\∇ _k\_k=1^|C| ) 17:Server Retention (Optional): If applicable, server performs retention fine-tuning using pretraining subset to obtain ∇θrserver∇ _r^server 18:Model Update: Apply combined updates: θ′=θ+∇Θ+∇θrserverθ =θ+∇ +∇ _r^server 19:return Updated model parameters θ′θ Appendix B Additional Experimental Details As shown in Table 6, continued unlearning over epochs progressively degrades model performance on the retained set, which shares distributional characteristics with the forget set (i.e., fabricated character biographies in TOFU). In contrast, general knowledge domains such as Real Authors and Real World remain relatively stable, indicating that unlearning primarily affects semantically proximate content. Table 6. Impact of Continued Unlearning on Retention and Utility over Epochs (TOFU) Epoch Real Authors↑ Real World↑ Retain↑ Forget↓ Utility↑ 1 71.20% 81.94% 89.93% 91.54% 58.89% 2 69.70% 82.51% 89.30% 83.15% 58.77% 3 63.85% 81.65% 83.33% 71.03% 57.57% 4 59.35% 82.68% 69.92% 58.53% 55.49% 5 58.35% 79.94% 66.08% 55.10% 54.44% As shown in Table 7, increasing the size of the forgetting set leads to broader degradation in both model utility and retention performance. This highlights a trade-off between unlearning strength and knowledge preservation: larger forget sets impose greater disruption on the model’s internal representations, thereby weakening its ability to retain unrelated or general knowledge. Table 7. Impact of Forgetting Set Size on TOFU Dataset Forget Set Size Model Utility↑ Real Authors↑ Real World↑ Retain↑ Forget↓ 25% of dataset 55.97% 72.25% 84.62% 72.34% 71.23% 5% of dataset 57.16% 69.42% 80.80% 82.06% 69.95% B.1. Adapter Training Objectives We apply different adapter finetuning objectives for unlearning across three benchmarks: WMDP, TOFU, and MUSE, each targeting distinct types of sensitive knowledge. WMDP (Biosecurity and Cybersecurity). To unlearn hazardous knowledge in biosafety and cybersecurity domains (1,273 biology and 1,987 cybersecurity questions), we employ the RMU (Representation Misdirection for Unlearning) method. The training objective consists of a forget loss that perturbs model activations on hazardous inputs xf∼Dforgetx_f D_forget at an intermediate layer ℓ : (2) ℒforget=xf∼Dforget[1Lf∑token t∈xf‖Mupdated(t)−c⋅‖22]L_forget=E_x_f D_forget [ 1L_f _token t∈ x_f \|M_updated(t)-c·u \|_2^2 ] where ∼Uniform([0,1]d)u ([0,1]^d) is a random unit vector, c is a hyperparameter, and Mupdated(⋅)M_updated(·) denotes the model activations at layer ℓ . Training Procedure. During adapter training, we interleave gradient updates from biosecurity and cybersecurity examples in DforgetD_forget, and minimize ℒforgetL_forget. We update only the adapter parameters in the down-projection linear layers of Transformer layers 5 through 7, while keeping the backbone model frozen to ensure efficient and localized unlearning. TOFU and MUSE (Gradient Ascent Unlearning). For the TOFU and MUSE benchmarks, we apply a gradient ascent approach to encourage the model to forget specific content by maximizing the standard training loss on the forget dataset xf∼Dforgetx_f D_forget: (3) ℒforget=xf∼Dforget[ℓ(xf,w)]L_forget=E_x_f D_forget [ (x_f,w) ] where ℓ(xf,w) (x_f,w) is the loss function of the model (e.g., cross-entropy) on input xfx_f with weights w. Although both benchmarks share this objective, they differ in data format: • TOFU contains fabricated personal information presented in the form of question-answer (Q&A) pairs, such as “What is Alice Zhang’s phone number?”. • MUSE consists of long-form text passages extracted from copyrighted books, such as Harry Potter and the Goblet of Fire. In both cases, we train the adapter to disrupt the model’s ability to reproduce the target content. B.2. Full experimental results of Intra- and Inter-Cluster Merging. Table B.2 and Table LABEL:tab:wmdp_noniid_full present the full results for both stages of our hierarchical unlearning merging process. Step 1: Intra-Cluster Merging. We include individual unlearning vectors (CyberiCyber_i, BioiBio_i) alongside the aggregated results (AVG, TIES, SUM). Table 8. Intra-Cluster Merging (Step 1) (a) WMDP-Cyber

Instant research discovery