Paper deep dive
Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning
Praveen Bushipaka, Lucia Passaro, Tommaso Cucinotta
Models: Llama-3.1-8B-Instruct
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 92%
Last extracted: 3/11/2026, 1:10:37 AM
Summary
The paper evaluates standard practices in LLM unlearning, specifically focusing on dataset construction (neighbor sets) and sampling strategies. It identifies that relying on single neighbor sets is suboptimal and that standard 1:1 sampling is inefficient. The authors propose 'Modular Entity-Level Unlearning' (MELU), a strategy that pairs forget targets with their specific retain samples, demonstrating improved stability and effectiveness compared to cyclic sampling.
Entities (5)
Relation Signals (3)
WPU â contains â Direct Neighbor set
confidence 95% ¡ WPU dataset... consists of 100 forget targets, Direct neighbor set and general retain set.
Llama-3.1-8B-Instruct â evaluatedusing â MELU
confidence 90% ¡ Before unlearning, we fine-tuned LLaMA 3.1 8B Instruct model... We also introduce Modular Entity-Level Unlearning (MELU) strategy.
MELU â improves â Unlearning Stability
confidence 90% ¡ We demonstrate that this modular approach, combined with robust algorithms, provides a clear and stable path towards effective unlearning.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A conventional LLM Unlearning setting consists of two subsets -"forget" and "retain", with the objectives of removing the undesired knowledge from the forget set while preserving the remaining knowledge from the retain. In privacy-focused unlearning research, a retain set is often further divided into neighbor sets, containing either directly or indirectly connected to the forget targets; and augmented by a general-knowledge set. A common practice in existing benchmarks is to employ only a single neighbor set, with general knowledge which fails to reflect the real-world data complexities and relationships. LLM Unlearning typically involves 1:1 sampling or cyclic iteration sampling. However, the efficacy and stability of these de facto standards have not been critically examined. In this study, we systematically evaluate these common practices. Our findings reveal that relying on a single neighbor set is suboptimal and that a standard sampling approach can obscure performance trade-offs. Based on this analysis, we propose and validate an initial set of best practices: (1) Incorporation of diverse neighbor sets to balance forget efficacy and model utility, (2) Standard 1:1 sampling methods are inefficient and yield poor results, (3) Our proposed Modular Entity-Level Unlearning (MELU) strategy as an alternative to cyclic sampling. We demonstrate that this modular approach, combined with robust algorithms, provides a clear and stable path towards effective unlearning.
Tags
Links
- Source: https://arxiv.org/abs/2509.05316
- Canonical: https://arxiv.org/abs/2509.05316
- Code: https://github.com/praveensonu/MELU
Trouble viewing inline? Open PDF directly â
Full Text
60,690 characters extracted from source content.
Expand or collapse full text
Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning â Praveen Bushipaka 1,2 (), Lucia Passaro 2 , and Tommaso Cucinotta 1 1 Scuola Superiore SantâAnnaname.surname@santannapisa.it, 2 University of Pisalucia.passaro@unipi.it,praveen.bushipaka@phd.unipi.it Abstract.A conventional LLM Unlearning setting consists of two sub- sets -"forget" and "retain", with the objectives of removing the un- desired knowledge from the forget set while preserving the remaining knowledge from the retain. In privacy-focused unlearning research, a re- tain set is often further divided into neighbor sets, containing either directly or indirectly connected to the forget targets; and augmented by a general-knowledge set. A common practice in existing benchmarks is to employ only a single neighbor set, with general knowledge which fails to reflect the real-world data complexities and relationships. LLM Unlearning typically involves 1:1 sampling or cyclic iteration sampling. However, the efficacy and stability of these de facto standards have not been critically examined. In this study, we systematically evaluate these common practices. Our findings reveal that relying on a single neigh- bor set is suboptimal and that a standard sampling approach can ob- scure performance trade-offs. Based on this analysis, we propose and validate an initial set of best practices:(1)Incorporation of diverse neighbor sets to balance forget efficacy and model utility,(2)Stan- dard 1:1 sampling methods are inefficient and yield poor results,(3) Our proposedModular Entity-Level Unlearning (MELU)strat- egy as an alternative to cyclic sampling. We demonstrate that this mod- ular approach, combined with robust algorithms, provides a clear and stable path towards effective unlearning. Our code can be found at https://github.com/praveensonu/MELU. Keywords:Best practices¡Selective Sampling¡Forget-Retain Sam- pling¡Batch & Sequential Unlearning¡Machine Unlearning in LLMs. 1 Introduction Large Language Models (LLMs)[32,1] are trained on vast amounts of data scraped from the web, enabling them to process billions of learnable param- eters. This extensive scaling enables them to address a wide array of complex linguistic tasks, exhibiting performance that approaches human-level proficiency â Accepted for presentation at the Workshop on Innovations, Privacy-preservation, and Evaluations Of machine Unlearning Techniques (WIPE-OUT 2025), ECML PKDD 2025, Porto, Portugal. arXiv:2509.05316v1 [cs.LG] 29 Aug 2025 2P.Bushipaka et al. in both language understanding and generation. However, this scale introduces a significant challenge: the models can memorize sensitive information such as personal data, copyrighted content, harmful data and output this information [3,9], raising concerns over their potential misuse [29]. To address this problem,LLM Unlearning[35,24] has emerged as a promising technique, aiming to remove specific knowledge and abilities while preserving the overall integrity and performance of the model. The conventional approach to LLM Unlearning involves two primary goals: (1) the unlearning process should remove the specified target knowledge and its associated abilities; (2) the un- learning must respect the model integrity and must not affect the non-target model abilities even if they are directly or indirectly related to the target [20]. For instance, if the target knowledge includes information about an authorBenedetto Varchiwho was born inFlorence, Italythen the unlearning process should suc- cessfully remove the Benedetto Varchiâs association to Florence, while retaining all the other knowledge about Florence (such as its connection of a city in Italy). To achieve these objectives, unlearning usually involves two datasets: aForget set, containing the knowledge that needs to be erased and aRetain set, contain- ing the knowledge that needs to be preserved. The process generally maximizes loss on the forget set and minimizes it on the retain set, helping to avoid issues likeDegeneration BehaviorandCatastrophic Forgetting. As LLM Unlearning transitions from a theoretical concept to a practical tool, best practices for adopting unlearning need to be looked into. Researchers often construct a retain set using a 1:1 ratio of forget-to-retain samples [22,25], drawing from a single type of neighbor set and a general knowledge pool [22]. When the retain set is larger, a cyclic sampling approach is employed to pair the forget and retain data during the unlearning process [25,12]. While these practices offer a straightforward path, their impact on the goal of unlearning is poorly understood. Do these simple heuristics represent an optimal strategy, or do they introduce hidden risks and performance ceilings? In this work, we make a step towards establishing a set of evidence-based best practices for LLM Unlearning. First, we look into the dataset creation practice, by extending the Wikipedia Person Unlearning (WPU) [21] dataset by incorpo- rating multiple neighbor sets into it. We analyze common data configurations (such as only direct or indirect neighbors) and sampling methods (1:1 sampling, cyclic), identify their strengths and weaknesses, and propose our strategy as an alternative. More precisely,our contributionsare as follows: 1.A Critical Analysis of Common Data Practices: A systematic evalua- tion of how retain set composition impacts unlearning outcomes. We demon- strate how a diverse retain set is crucial for balancing Forget Efficacy and Model Utility. Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning3 2.Comparison of common sampling strategies of LLM Unlearning: A comparative analysis of common sampling methods for LLM Unlearning. We discover that the common practice of 1:1 sampling is ineffective. 3.Proposal of MELU as a sampling technique: We introduceModu- lar Entity-Level Unlearning (MELU), a simple structured sampling strategy, that demonstrates more stable and effective Unlearning than con- ventional cyclic sampling. 2 Preliminaries 2.1 Unlearning in Large Language Models Prior works of unlearning in Large Language Models focus on classification tasks [12], but due to the increase in adoption of generative AI in industries and everyday life, especially chatbots and instruction-tuned LLMs, the research focus has moved to question-and-answer (Q&A) tasks. Given the Large Language modelMwith its parametersθ, the forget set D f =x f ,y f contains the samples that need to be forgotten byM(;θ)and the retain setD r =x r ,y r thoseM(;θ)needs to preserve, wherex,yare questions and answers (inputs and their labels) in the LLM Unlearning task. Our goal is to provide an updated/forgotten LLM with parametersθ â satisfying the objectives mentioned previously. Although designs vary, most fine-tuning based unlearning algorithms objec- tives can be mathematically written as [13] min θ â L(θ â ) = min θ â (âL f (θ â ) +ÎťL r (θ â )) The equation provides the objectives, the first loss term - forget lossL f (θ â ) maximizes the loss on forget set, and the second loss term - retain lossL R (θ â ) minimizes the loss on the retain set.Îťis a hyper-parameter controlling the retain strength. Entity UnlearningThere are two types of unlearning:Instance-Level Unlearn- ingandEntity-Level Unlearning[5,22,21]: the former erases specific knowledge about a forget-target, whereas the latter removes all knowledge of that entity (e.g. a person, institution, book series etc). Formally, given entitiesÎľ=e 1 ,e 2 ..e n to forget, eache i is represented by Q&A pairse i =(x i1 ,y i1 )...(x in ,y in ). The modelM(θ), is trained on datasetD, is split into forget setD f and a disjoint retain setD r =D f . In this work, we focus on Entity-level unlearning. 2.2 Datasets SinceD r is disjoint fromD f , it could contain potentially all the pretrained data excluding theD f . This is impractical to implement, and prior works address this challenge by assessing performance on general knowledge benchmarks such 4P.Bushipaka et al. as MMLU [34] or creating an entirely new general knowledge dataset [17,22,21] and creating neighbor sets, which are subsets ofD r expected to be influenced by the unlearning process. These neighbor sets are constructed based on the assumption that data points similar toD f or involved in unlearning are more likely to be impacted during unlearning. From the literature, we identify three types of neighbor sets: Direct Neighbor set(N d )-Direct Neighbor sets contain the entities that are closely associated and directly connected toD f [21,14]. These include but are not limited to place of birth, family tree, education, personal achievements, and everything that is directly linked to the forget target. For instance, for a forget target - âBenedetto Varchiwas born inFlorenceâ, and information on Flo- rence is considered as a part of its corresponding Direct Neighbor set, assuming this is directly influenced due to the forgetting of Benedetto Varchiâs birthplace knowledge. Indirect Neighbor set(N ind )-First introduced in TOFU [22], an indirect set consists of entities sharing a semantic or contextual relationship with the forget target, without being directly linked. These connections may be based on historical period, domain, ideology, or thematic relevance-not necessarily profes- sion. For example, if the entity isBenedetto Varchiâs, an Italian humanist and historian of the fifteenth century, the corresponding Indirect neighbor set con- sists of data on Leonardo Bruni, Francesco Petrarca etc. who were also Italian historians of the similar period. It is difficult to derive the indirect connections without looking at the model activations and pre-trained dataset. So, for our study, we use the already proposed approach of profession to be the indirect connection. Syntactic similarity(N s )-Introduced by [4], they expand the present neigh- bor sets to syntactic similarity neighbor set, showing that syntactic similarity is the most influenced due to the nature of question-answering unlearning task. For example, âWhen was Benedetto Varchi born?â can have influence on a question with similar syntax such as âWhen was Donald Trump born?â. To avoid this, they propose an entirely new neighbor set. 2.3 LLM Unlearning Practices We do not discuss LLM Unlearning algorithms, rather we discuss their imple- mentations. For unlearning algorithms please refer Appendix:A.3. Batch and Sequential Unlearning Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning5 Batch Unlearning, commonly used refers to unlearning the model on all the forget targets at once. While straightforward, this approach has been observed to suffer from instability and leads to catastrophic collapse [12].Sequential Unlearning, proposed by [12] and further extended by [25], divides the forget set into chunks, processing each chunk independently and simultaneously processing the retain data, making it ideal for a realistic setting. 1:1 and Cyclic Sampling Given the simultaneous maximization of loss on forget sample and minimiza- tion on retain sample, a sampling method usually contains how these samples are arranged and how many samples are used in an epoch for the unlearning algorithm. A common sampling practice is1:1 Sampling, i.e., in an epoch, the number of retain samples is not higher than the number of forget samples. There are two ways to do this:(Method - a)creating the dataset with forget and re- tain samples of the same length â recent datasets such as the SemEVAL Task-4 competition 3 follow this structure;(Method - b)randomly choosing the same number of forget and retain samples for every epoch â initially implemented by [22] and followed by [21,26,36,8,23] and many more by reproducing their code, this practice has become common for baselines in LLM Unlearning. Another common sampling practice isCyclic Sampling[25,12], in which all the retain samples are utilized by cycling forget samples. As in figure 2, a cyclic setting might have a retain sample unrelated to the forget sample. A drawback of this approach is the loss calculation of forget sample with unrelated retain sample. In this study, we introduceModular Entity-Level Unlearning (MELU)strategy, in which during the unlearning process, each forget target is paired only with its respective retain samples. 3 Related Work Current unlearning datasets include either direct(N d )or indirect neighbor(N ind ) sets, but never both.TOFU[22] uses 200 synthetic authors as indirect neigh- bors(N ind ), plus 100 real authors and 117 facts, but omits interconnectivity [26] and ignores direct neighbors((N d )).RWKU[14] andWPU[21] include only a Direct Neighbor set plus a general knowledge set.RWKUâs focus on 200 high profile figures makes unlearning impractical because their large online footprints means pre-trained LLMs almost certainly have absorbed vast amounts of their data, making it difficult to unlearn; realistic unlearning requests involve individ- uals with moderate online presence. Datasets from [4,5] attempt to include both (N d )and(N ind )but they rely on bi-directional relationships for(N d ), requiring mutual links in their respective Wiki pages, creating a blind spot: e.g.,"Varchi was born in Florence", Varchiâs page links to Florence, but Florenceâs page does 3 https://llmunlearningsemeval2025.github.io/ 6P.Bushipaka et al. born VarchiFlorenceItaly is in ForgetDirect Neighbor Study in Petrarca of Bologna Indirect Neighbor F) Where was Varchi Born? R) Where was Tom Hanks Born? Syntactic Neighbor Fig. 1: Types of Neighbor sets and their connections to the forget sample not link back to Varchi. Although Florence is directly connected and would influ- ence the unlearning process, the bi-directional approach would exclude this from the retain set. Even though these benchmarks exist, LLM unlearning still lacks a standard protocol or methodology for building forget and retain sets, even as unlearning requests become common across applications. [30] provides a new direction on backdrops of the unlearning benchmark datasets. Post-Unlearning, they combine forget and retain queries and ask the model, only to find the model either recognizes them both as forget samples(outputs IDK)or retain samples (outputs correct answers). In the1:1 SamplingMethod-alimits the retain set during the data con- struction, andMethod-blimits the retain set during unlearning, especially experiments conducted on [22] benchmark use various splits, fail to leverage the full retain set. For example, TOFU benchmark (4000 samples) has three splits of various forget set sizes (40, 200, 400), leading various retain set sizes (3.96k, 3.8k, 3.6k). TOFU authors unlearn for 5 epochs on these splits. For the largest split of 400 forget samples, at the maximum can attend only 2000 retain sam- ples (if retain samples are sequentially chosen). Albeit, SemEVAL dataset uses Method-a, the winner of the competition [25] doesnât follow this approach, in- stead they follow 1:n-forget:retain approach (in a cyclic sequential unlearning process), making for each batch only for 1 forget sample and n retain samples are present. In this paper, we extend the WPU [21] neighbor by adding indirect neigh- bors with syntactically similar Q&Aâs, and a dedicated test set. We further look into the sampling strategies provided by [22], 1:1 sampling, cyclic sampling in a batch unlearning scenario. We also introduceModular Entity-Level Un- learning (MELU)strategy, in which each forget target is paired only with its respective retain samples. Our work neither aims to present a benchmark dataset nor an unlearning method, rather improve current best practices in creating LLM Unlearning Benchmark datasets and sampling. 4 Experimental Setup 4.1 Dataset Construction In this work, we chose the Wikipedia Person Unlearning (WPU) [21] dataset, which consists of 100 forget targets, Direct neighbor set and general retain set. Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning7 The dataset is divided into forget_2, forget_20 and forget_100 parts, splitting the # of target entities to 2, 20 and 100. We chose forget_20 for the extension of Neighbor sets to(N ind ),(N s )and addition of a test set. The choice of WPU dataset comes from their approach in dataset construction. It was constructed by selecting the least popular Wikipedia people based on their views, these are the forget entities. They construct theD r through incorporating(N d )by scraping the hyperlinks that are connected to the Wikipedia page of the person (N d )and General knowledge set by scraping the Wiki pages of popular people on Wikipedia (General Knowledge set). Other datasets use well known figures information, which might be difficult to unlearn, or a bi-directional approach for (N d ), or synthetically created datasets. WPU stands as an ideal choice for our experiments, as the entities are not well known and has limited online presence providing a realistic unlearning situation. Indirect Neighbor set(N ind )creation -For Indirect connection we follow [22], and chose to find entities of similar profession. Finding similar profession entities for lesser known people was challenging as it requires scraping Google search suggestions, which were often unavailable or inconsistent. To overcome this, we used an LLM to generate similar profession names. Specifically, we chose LLaMA 3.3 70B model [10] for this task, since our unlearning experiments were conducted on the LLaMA 3.1 8B Instruct model [10]. We assumed that the models of the same family would likely share similar pre-training knowledge. We prompted (Appendix:A.1) the model to generate six names for each forget target, ending up with 120 indirect connections. Once we had the Indirect connection entities, we scraped their Wikipedia data and used LLaMA 3.3 70B model to generate the Q&Aâs from it. We aimed for at least two questions per section, and instructed the LLM to follow an Interrogative syntactic structure. In total, we ended up with 1409 Q&A pairs. Then, for each forget target, we randomly picked five indirect connection entities to build the(N ind )- giving us a total of 1144 Q&A pairs (Appendix:4). Test set(D t )creation -We construct a test with mix of multiple neighbors for evaluation. We use the remaining samples of(N ind )for indirect connections and 200 random samples from the general knowledge set for the test set. To create the samples for(N d ), we prompted LLaMA 3.3 70B to provide three new basic Q&A for every answer from forget set. If we are forgetting the link "Adrienne Monnier -> Paris", we made three Q&Aâs about Paris (Appendix:5). Finally, we created a test set with 738 Q&A pairs. Due to the pre-structure of the WPU dataset [21], which includes(N d )and general knowledge set, we were unable to create a standalone(N s )dataset. In- stead we generated the neighbor sets in a similar syntactic manner. To do this, in the Q&A generation prompt A.1, we provided the model to follow interroga- tive syntactic structure. We verified the syntactic similarity between the forget and retain Q&A pairs with edit distance algorithm [39], achieved a mean of 40% similarity. 8P.Bushipaka et al. ... ... Cyclic Forget SamplesRetain Samples ... Forget SamplesRetain Samples Forget EntitiesRetain connectionsRetain connectionsForget Entities MELU e 1 e 2 e 1 e 1 e 2 e 1 e 1 e 2 e 1 e 2 Fig. 2: MELU setting. In cyclic, each entities (e 1 ,e 2 ..e n ) forget samples are cycled on to unrelated (e 3 âe 1 ) connections, In MELU entities (e 1 ,e 2 ..e n ) are cycled only onto their respective target connections (e 1 âe 1 ). 4.2 Unlearning Methods Before unlearning, wefine-tunedLLaMA 3.1 8B Instruct [10] model on all the datasets. We experiment with three unlearning algorithms- Gradient Difference [19] and Negative Preference Optimization [38] for Un-targeted unlearning and Direct Preference Optimization [27] for Targeted Unlearning. An Un-targeted unlearning does not contain a replacement sample for the forgetting sample, whereas targeted unlearning does and typically has phrases such as "I donât know". We compare seven different settings. The initial three are on common data practices, and next four are common implementation practices. We have pool of |D f |= 98, and|D r |= 1801 samples. Given that all these experiments revolve around manipulating retain set, we select retain subsetsD r according to: Data Practices 4 1.Direct-Neighbor(N d ). RestrictD r to only samples with the(N d ). 2.Indirect-Neighbor(N ind ). RestrictD r to only samples with the(N ind ). 3.Balanced. We combine(N d )and(N ind )for ourD r . Since our(N ind )is over sampled, we balance the dataset by selecting an equal number of samples as the(N d )samples for each entity. Sampling Practices 1.1:1 seq. i.e., We draw|D r |=|D f |samples by selecting the top 98D r items, matching one retain for each forget. This is the methoda)from section 2.3. 2.1:1 random. i.e., We draw|D r |=|D f |samples by randomly selecting 98 D r items for each epoch, matching one retain for each forget. This is the methodb)from section 2.3. 3.Cyclic. Rotate through the fullD r pool sequentially until we collect 1801 samples - i.e. cycle the 98 sized|D f |across all the 1801 samples. 4 Given 1:1 sampling (both Sequence and Random) will not utilize the completeD r during Unlearning, all the Data Practices experiments are conducted in Cyclic Im- plementation. Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning9 4.Modular Entity-Level Unlearning (MELU). For each entity or forget target appearing theD f , we include only retain samples that share that same entity. To do this, we cycle the forget samples of a forget target only over the retain samples of the same target (figure 2). We will be left with general knowledge set to which we randomly assign a sample from theD f . We adopt LoRA[11] for all our experiments. For finetuning, rank = 64,Îą= 128, batch size = 32 for 10 epochs. For unlearning, rank = 8,Îą= 16, batch size = 8 5 for 4 epochs. All the experiments were conducted on 2 x 40GB A100 GPUs. Full algorithmic details are in Appendix:A.3. 4.3 Assessment Unlearning behavior is best assessed with the use of multiple metrics [22]. We employ three distinct metrics and aggregate them to compute two scores:For- get EfficacyandModel Utility. In line with prior works [22,36,38], we em- ployROUGE-L(verbatim memorization with word-level match),Conditional Probability(ground truth likelihood) andCosine Similarity(Semantic Sim- ilarity). To calculateForget Efficacy, we calculate1 - Arithmetic meanof these metric onD f and forModel Utilitywe calculate a harmonic mean of these met- rics onD r . 5 Results and Discussion BaselinesWe computed Forget Efficacy (on forget set) and Model Utility (on test set) on the base model before unlearning. We use these results as baselines. After applying the unlearning algorithms, for a fair comparison, we use Forget Efficacy and Model Utility on Test set (MU-T), given that the|D r |changes based on the setting. The test set is balanced and will be used to understand the Model Utility. The base model has a low Forget Efficacy (FE = 0.30) and high model utility (ME-T = 0.73). A good unlearned model should have a higher FE (up to 1) and MU-T closer to the baselineÂą5. We also compute MMLU [34] scores to understand the general Model utility, per target FE and MU-T for granular understanding and token diversity withDistinct-N[16] to understand the diversity of the generated outputs. 5.1 Evaluation of Unlearning Data Practices Direct vs Indirect:Interestingly, both GD and NPO show (figure 3 and table 1) a drop in FE when moving from Direct(N d )to Indirect(N ind )neighbor sets, contrary to our expectations. Since,(N d )is smaller in size (# 364 samples + # 293 general knowledge) compared to(N ind )(#1144 + #293), we anticipated forget set would be revised more frequently during unlearning. However,(N d ) 5 Batch size of 8 is maintained for every experiment by the aggregation of gradients over 8 samples, even when the hardware limitations prohibit batch size of 8. 10P.Bushipaka et al. neighbor with such a small neighbor set outperforms(N ind )in terms of FE. In contrast, DPO follows the expected trend, showing higher FE with the larger (N ind ). However,MU-T is consistently higherfor(N ind ), indicates, larger and more diverse sets preserve general model performance. On the other hand, Balanced fails to achieve better FE and MU-T. From theToken Diversity9a on the forget and test sets, we find that(N d ) is lower than the(N ind )neighbors on the test set across all the unlearning al- gorithms. For GD, we see an exponential drop in forget set diversity in indirect and balanced. For DPO and NPO, we see an increment in token diversity from indirect to balanced. But both GD and DPO fail to maintain token diversity (even in implementation settings), this is because of GDâs âDegeneration Behav- iorâ and DPOs âI donât knowâ phrases. NPO exhibits more favorable behavior with high token diversity. This is likely due to its bounded objective, preventing model collapse. Per target FE and MU-Tshow that GD, performs really well at forgetting with direct connections, but fails significantly at MU-T (10 targets are below 0.20 for MU-T). We find a similar situation with balanced, where the FE is higher and MU-T is lower. Although, indirect doesnât achieve same level of forgetting as direct, it always maintains>0.85FE on all targets and maintains MU-T up to 0.65 (0.08 shy from baseline). With preference based methods, we find a gradual increment in MU-T from direct to indirect to balanced. A strange phenomenon was observed that some targets such as "Ted Kooser" was harder to forget for both DPO and NPO (except direct case for NPO). With memorization scores (Appendix:Fig11), we find that these targets are highly memorized than others. MMLUscores on GD are inconsistent. Although they are not exponential drops or highs, but we find an increase in MMLU accuracy (Appendix:Fig8) on direct and balanced dataset experiments (â+1.3%). This is an unusual behavior. Where as, preference based methods show a stable accuracy. Cyclic and MELU, provide stable MMLU scores across all the models, showing with proper implementation unlearning can be stable. 5.2 Evaluation of Unlearning Sampling Practices Standard1:1 sampling(sequential and random) fails to produce meaningful forgetting (low FE) yet preserve MU-T across all the unlearning algorithms. Al- though, increase in number of epochs might improve the forgetting 6 , we already achieve better stability (FE and MU-T) with cyclic and MELU with the same number of epochs. 6 To test this we conducted a run of DPO with 1:1 random sampling by continuously increasing the epochs. At epoch 100 we achieved 0.79 FE and 0.78 MU-T. Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning11 Stability with MELUBoth Cyclic and MELU perform significantly better than standard 1:1 sampling. They maintain stable performance across all the unlearning algorithms and maintain accuracy on MMLU and token diversity. MELU, in particular, outperforms cyclic under DPO, boosting FE by 12% while maintaining MU-T. In NPO, MELU provides a small improvement from cyclic. But at theper-targetperformance, MELU holds a better FE and MU-T on all the targets for preference based methods. Under DPO, MELU increases the number of targets with FE > 0.9 (from 1 in cyclic to 3), while maintaining high MU-TâĽ0.8for the majority. In case of Amy Clampitt, FE improves byâ20%. For GD, MELU setup achieves stable FE (â0.9) across most entities and pro- vides higher MU-T across targets. Even in NPO, where overall FE grows slowly, MELU maintains MU-T while achieving reasonable FE. In cases of harder tar- gets such as "Ann Brashares" andTed Kooser, both cyclic and MELU perform well and forget (>0.50) better than 1:1 sampling (<0.10). This improved sta- bility of MELU can be attributed to a more consistent learning signal. In cyclic sampling, the model is subjected to high-variance gradients due to unrelated forget-retain pairs. MELU, by maintaining a relevancy between forget retain with lower variance per batch, could be leading to a stable performance. Overall MELU provides 1.High and Stable FE:approaching or exceeding 0.85 for DPO and GD, and maintaining competitive scores in NPO. 2.Minimal degradation in MU-T:consistently close to the baseline (0.73), even slightly exceeding it for some algorithms (e.g., NPO). 12P.Bushipaka et al. Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 FE MU-T Baseline FE Baseline MU-T Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 Gradient-based Unlearning 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 DPO-based Unlearning 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 NPO-based Unlearning Fig. 3: FE and MU-T comparison results. Top row provides the data practices results and bottom row provides implementation practices results. 6 Conclusion Our study demonstrates that the composition of the retain set is a critical, yet often under-looked factor in LLM Unlearning. Our findings show that relying solely on any single neighbor set is suboptimal and not the best practice. By including a diverse range of neighbors - we can improve the balance between forget efficacy and model utility. Furthermore, we show that the standard 1:1 sampling is an inefficient approach and when implementing unlearning, cyclic and Modular Entity-Level Unlearning (MELU) provides more stability. Albeit, we couldnât answer why these setups provides more stability, a research that could be looked into is if this caused due to the repetition of forget samples in the unlearning process (It is well-known that during the pretraining, memoriza- tion of a sample is correlated with its repetition in the corpus [2]). Our work also proves some of the already proposed problems in the unlearning literature, such as Gradient based approachâs instability [33,35], and how some targets are harder to forget because of the frequency of their knowledge and memorization (Appendix:11) in pre-trained or downstream task data [15]. We hope our work inspires researchers to look into methods to construct more diverse and realistic unlearning benchmarks and unlearning algorithm implementation techniques. For future works, We aim to conduct a rigorous comparative analysis against other model families, more unlearning algorithms and conduct a deeper evalua- tion, especially with consideration of sample memorization. Additionally, we aim Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning13 to do a comparative analysis with Sequential Unlearning and also incorporating MELU into sequential unlearning. 7 Limitations A key limitation of our study is the indirect neighbors and test sets generation with sister model. This limits the generalizability of our findings and requires expanding our experiments to other model families. Our MELU setup assumes that forget and retain samples are sufficiently distinct to be reliably separated. In real-world scenarios, however, such clear boundaries may not always exist, especially when entities share overlapping attributes or contexts, albeit one can enforce Knowledge graphs to define these clear connections[26]. Additionally, we do not deep dive into instability and stability issues such as GD and NPOâs bet- ter forgetting on direct neighbors and not with indirect neighbors. In contrast DPO acts opposite, this can be further looked into especially through the lens of explainability approaches on pre and post unlearning. Same with MELU and cyclic settings stable performances, a rigorous work needs to be done towards ad- dressing it. Another limitation is extension of general utility with HellaSwag [37], ARC [6] etc. Our experiments cover only a few unlearning algorithms with batch unlearning. While we propose MELU, we lack a direct comparative analysis to sequential unlearning [12] setups. Finally, because WPU [21] already includes direct and general neighbors, we could not construct a full syntactic neighbor set, leaving it unexplored. Acknowledgments This work has been partially supported by the EU EIC project EMERGE (Grant No. 101070918). References 1. Brown, T.B., Mann, B., Ryder, e.a.: Language models are few-shot learners. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. NIPS â20, Curran Associates Inc., Red Hook, NY, USA (2020) 2. Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., Zhang, C.: Quantifying memorization across neural language models. In: The Eleventh International Con- ference on Learning Representations (2023),https://openreview.net/forum?id= TatRHT_1cK 3. Carlini, N., Tramèr, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, Ă., Oprea, A., Raffel, C.: Ex- tracting training data from large language models. In: 30th USENIX Security Symposium (USENIX Security 21). p. 2633â2650. USENIX Association (Aug 2021),https://w.usenix.org/conference/usenixsecurity21/presentation/ carlini-extracting 14P.Bushipaka et al. 4. Chang, H., Lee, H.: Which retain set matters for LLM unlearning? a case study on entity unlearning. In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T. (eds.) Findings of the Association for Computational Linguistics: ACL 2025. p. 5966â5982. Association for Computational Linguistics, Vienna, Aus- tria (Jul 2025).https://doi.org/10.18653/v1/2025.findings-acl.310,https: //aclanthology.org/2025.findings-acl.310/ 5. Choi, M., Rim, D., Lee, D., Choo, J.: Opt-out: Investigating entity-level un- learning for large language models via optimal transport. In: Che, W., Nabende, J., Shutova, E., Pilehvar, M.T. (eds.) Proceedings of the 63rd Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pa- pers). p. 28280â28297. Association for Computational Linguistics, Vienna, Aus- tria (Jul 2025).https://doi.org/10.18653/v1/2025.acl-long.1371,https:// aclanthology.org/2025.acl-long.1371/ 6. Chollet, F., Knoop, M., Kamradt, G., Landers, B.: Arc prize 2024: Technical report (2025),https://arxiv.org/abs/2412.04604 7. Dorna, V., Mekala, A., Zhao, W., McCallum, A., Kolter, J.Z., Maini, P.: OpenUn- learning: A unified framework for llm unlearning benchmarks.https://github. com/locuslab/open-unlearning(2025), accessed: February 27, 2025 8. Fan, C., Liu, J., Lin, L., Jia, J., Zhang, R., Mei, S., Liu, S.: Simplicity prevails: Rethinking negative preference optimization for LLM unlearning (2025),https: //openreview.net/forum?id=Pd3jVGTacT 9. Golatkar, A., Achille, A., Soatto, S.: Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks . In: 2020 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR). p. 9301â9309. IEEE Computer Society, Los Alamitos, CA, USA (Jun 2020).https://doi. org/10.1109/CVPR42600.2020.00932,https://doi.ieeecomputersociety.org/ 10.1109/CVPR42600.2020.00932 10. Grattafiori, A., Dubey, A., et al, A.J.: The llama 3 herd of models (2024),https: //arxiv.org/abs/2407.21783 11. Hu, E.J., yelong shen, Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Con- ference on Learning Representations (2022),https://openreview.net/forum?id= nZeVKeeFYf9 12. Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., Seo, M.: Knowl- edge unlearning for mitigating privacy risks in language models. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pa- pers). p. 14389â14408. Association for Computational Linguistics, Toronto, Canada (Jul 2023).https://doi.org/10.18653/v1/2023.acl-long.805,https: //aclanthology.org/2023.acl-long.805/ 13. Ji, J., Liu, Y., Zhang, Y., Liu, G., Kompella, R.R., Liu, S., Chang, S.: Reversing the forget-retain objectives: An efficient LLM unlearning framework from logit differ- ence. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024),https://openreview.net/forum?id=tYdR1lTWqh 14. Jin, Z., Cao, P., Wang, C., He, Z., Yuan, H., Li, J., Chen, Y., Liu, K., Zhao, J.: RWKU: Benchmarking real-world knowledge unlearning for large language models. In: The Thirty-eight Conference on Neural Information Processing Sys- tems Datasets and Benchmarks Track (2024),https://openreview.net/forum? id=wOmtZ5FgMH Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning15 15. Krishnan, A., Reddy, S., Mosbach, M.: Not all data are unlearned equally. In: Sec- ond Conference on Language Modeling (2025),https://openreview.net/forum? id=Kd97lfFfTu 16. Li, J., Galley, M., Brockett, C., Gao, J., Dolan, B.: A diversity-promoting objective function for neural conversation models. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies. p. 110â119. Association for Computational Linguis- tics, San Diego, California (Jun 2016).https://doi.org/10.18653/v1/N16-1014, https://aclanthology.org/N16-1014/ 17. Li, N., Pan, A., Gopal, A., Yue, S., Berrios, e.a.: The WMDP benchmark: Measur- ing and reducing malicious use with unlearning. In: Proceedings of the 41st Inter- national Conference on Machine Learning. Proceedings of Machine Learning Re- search, vol. 235, p. 28525â28550. PMLR (21â27 Jul 2024),https://proceedings. mlr.press/v235/li24bc.html 18. Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text Summarization Branches Out. p. 74â81. Association for Computational Linguis- tics, Barcelona, Spain (Jul 2004),https://aclanthology.org/W04-1013/ 19. Liu, B., Liu, Q., Stone, P.: Continual learning and private unlearning (2022), https://arxiv.org/abs/2203.12817 20. Liu, S., Yao, Y., Jia, J., et al.: Rethinking machine unlearning for large language models. Nature Machine Intelligence7, 181â194 (2025).https://doi.org/10. 1038/s42256-025-00985-0,https://doi.org/10.1038/s42256-025-00985-0 21. Liu, Y., Zhang, Y., Jaakkola, T., Chang, S.: Revisiting whoâs harry potter: To- wards targeted unlearning from a causal intervention perspective. In: Proceed- ings of the 2024 Conference on Empirical Methods in Natural Language Process- ing. p. 8708â8731. Association for Computational Linguistics, Miami, Florida, USA (Nov 2024).https://doi.org/10.18653/v1/2024.emnlp-main.495,https: //aclanthology.org/2024.emnlp-main.495/ 22. Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z.C., Kolter, J.Z.: TOFU: A task of fictitious unlearning for LLMs. In: First Conference on Language Modeling (2024), https://openreview.net/forum?id=B41hNBoWLo 23. Mekala, A., Dorna, V., Dubey, S., Lalwani, A., Koleczek, D., Rungta, M., Hasan, S., Lobo, E.: Alternate preference optimization for unlearning factual knowl- edge in large language models. In: Proceedings of the 31st International Con- ference on Computational Linguistics. p. 3732â3752. Association for Compu- tational Linguistics, Abu Dhabi, UAE (Jan 2025),https://aclanthology.org/ 2025.coling-main.252/ 24. Miranda, M., Ruzzetti, E.S., Santilli, A., Zanzotto, F.M., Bratières, S., RodolĂ , E.: Preserving privacy in large language models: A survey on current threats and solu- tions. Transactions on Machine Learning Research (2025),https://openreview. net/forum?id=Ss9MTTN7OL 25. Premptis, I., Lymperaiou, M., Filandrianos, G., Menis Mastromichalakis, O., Voulodimos, A., Stamou, G.: AILS-NTUA at SemEval-2025 task 4: Parameter- efficient unlearning for large language models using data chunking. In: Rosen- thal, S., RosĂĄ, A., Ghosh, D., Zampieri, M. (eds.) Proceedings of the 19th In- ternational Workshop on Semantic Evaluation (SemEval-2025). p. 1383â1405. Association for Computational Linguistics, Vienna, Austria (Jul 2025),https: //aclanthology.org/2025.semeval-1.184/ 26. Qiu, X., Shen, W.F., Chen, Y., Kurmanji, M., Cancedda, N., Stenetorp, P., Lane, N.D.: How data inter-connectivity shapes llms unlearning: A structural unlearning perspective (2025),https://arxiv.org/abs/2406.16810 16P.Bushipaka et al. 27. Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C.D., Finn, C.: Direct preference optimization: your language model is secretly a reward model. In: Pro- ceedings of the 37th International Conference on Neural Information Processing Systems. NIPS â23, Curran Associates Inc., Red Hook, NY, USA (2023) 28. Reimers, N., Gurevych, I.: Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). p. 3982â3992. Association for Computational Linguistics, Hong Kong, China (Nov 2019).https://doi.org/10. 18653/v1/D19-1410,https://aclanthology.org/D19-1410/ 29. Staab, R., Vero, M., Balunovic, M., Vechev, M.: Beyond memorization: Violating privacy via inference with large language models. In: The Twelfth International Conference on Learning Representations (2024),https://openreview.net/forum? id=kmn0BhQk7p 30. Thaker, P., Hu, S., Kale, N., Maurya, Y., Wu, Z.S., Smith, V.: Position: Llm unlearning benchmarks are weak measures of progress (2025),https://arxiv. org/abs/2410.02879 31. Tirumala, K., Markosyan, A.H., Zettlemoyer, L., Aghajanyan, A.: Memorization without overfitting: Analyzing the training dynamics of large language models. In: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (eds.) Advances in Neural Information Processing Systems (2022),https://openreview.net/forum?id=u3vEuRr08MT 32. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. p. 6000â6010. NIPSâ17, Curran Associates Inc., Red Hook, NY, USA (2017) 33. Wang, Q., Zhou, J.P., Zhou, Z., Shin, S., Han, B., Weinberger, K.Q.: Rethink- ing LLM unlearning objectives: A gradient perspective and go beyond. In: The Thirteenth International Conference on Learning Representations (2025),https: //openreview.net/forum?id=huo8MqVH6t 34. Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., Li, T., Ku, M., Wang, K., Zhuang, A., Fan, R., Yue, X., Chen, W.: Mmlu-pro: A more robust and challenging multi-task language understanding benchmark (2024),https://arxiv.org/abs/2406.01574 35. Yao, Y., Xu, X., Liu, Y.: Large language model unlearning. In: The Thirty-eighth Annual Conference on Neural Information Processing Systems (2024),https:// openreview.net/forum?id=8Dy42ThoNe 36. Yuan, X., Pang, T., Du, C., Chen, K., Zhang, W., Lin, M.: A closer look at ma- chine unlearning for large language models. In: The Thirteenth International Con- ference on Learning Representations (2025),https://openreview.net/forum?id= Q1MHvGmhyT 37. Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., Choi, Y.: HellaSwag: Can a machine really finish your sentence? In: Korhonen, A., Traum, D., MĂ rquez, L. (eds.) Proceedings of the 57th Annual Meeting of the Association for Com- putational Linguistics. p. 4791â4800. Association for Computational Linguis- tics, Florence, Italy (Jul 2019).https://doi.org/10.18653/v1/P19-1472,https: //aclanthology.org/P19-1472/ 38. Zhang, R., Lin, L., Bai, Y., Mei, S.: Negative preference optimization: From catas- trophic collapse to effective unlearning. In: First Conference on Language Modeling (2024),https://openreview.net/forum?id=MXLBXjQkmb Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning17 39. Zhang, S., Hu, Y., Bian, G.: Research on string similarity algorithm based on levenshtein distance. In: 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). p. 2247â2251 (2017). https://doi.org/10.1109/IAEAC.2017.8054419 A Appendix A.1 Prompts Q&A GenerationThis prompt was used to extract Q&As from the wiki pages. LLaMA 3.3-70B #system_prompt\\ You are an expert teacher, who can create questions and answers from a given context. Given the user wikipedia page context about # domain_person_name, please provide as many questions and answers possible from it. For each section, provide at least 2 questions and answers. The question and answers should follow the Interrogative syntactic structure, The questions should be on their birth, family background, education, career, achievements and other relevant topics. The output should be in JSON format with the following keys: \ "name": name of the person, "question1": question1, "answer1": answer1, "section" : part of the wikipedia section, "difficulty" : difficulty of the question, "question2": question2, ⌠\ Please be precise with the question and answer. Do not generate any other text. #prompt\\ # content 18P.Bushipaka et al. Indirect Connection generationThis prompt was used to generate six Indi- rect connections for a target. LLaMA 3.3-70B #prompt\\ For each name in the list, provide me 6 names that belong to the same domain as them (for example, if they are authors please provide authors similar as them). The output should be in a dictionary to make it into a dataframe. [âBenedetto Varchiâ, âWilhelm Wattenbachâ, âElsa Trioletâ, âTheopompusâ, âHeinrich Ritterâ, âAdrienne Monnierâ, âAnn Brasharesâ, âHartmann von Aueâ, âJorge SemprĂşnâ, âGiovanni Battista Castiâ, âNajaf Daryabandariâ, âHeinz Erhardtâ, âRudolf Christoph Euckenâ, âPaul Gerhardtâ, âMoshe Greenbergâ, âAmy Clampittâ, âTed Kooserâ, âAlfred Vogelâ, âSiegfried Lenzâ, âPhilip Stanhope, 5th Earl Stanhopeâ] A.2 Dataset A detailed pipeline in creating the indirect connections and its relevant test set samples are provided in the Figure 4. First, we use the prompt A.1 to generate 6 entities for each target. Followed by we scrape their wiki pages and generate Q&As with the LLaMA 3.3 70B model A.1 in an Interrogative syntactic structure manner to maintain the(N s )neighbor dataset. . . . Forget TargetLLaMA 3.3 70B Generate 6 entities Scrape wiki Generate Interrogative QA 5 entites Indirect retain set Test Set Fig. 4: Pipeline in creating Indirect connections for retain and test set Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning19 Q: Where was Adrienne Monnier born? A: Paris Llama 3.3 70B Q1: What is the name of the iconic tower in Paris, built for the 1889 World's Fair? A: The Eiffel Tower Q2: What is the name of the famous art musuem in Paris that houses the Mona Lisa? A: The Louvre Q3: Which river runs through the city of Paris? A: The Siene Fig. 5: Generation of Test set samples for the Direct Neighbors 020406080100 Total Questions Najaf Daryabandari Ann Brashares Ted Kooser Jorge SemprĂşn Alfred Vogel Paul Gerhardt Giovanni Battista Casti Heinrich Ritter Adrienne Monnier Theopompus Philip Stanhope, 5th Earl Stanhope Moshe Greenberg Wilhelm Wattenbach Elsa Triolet Heinz Erhardt Hartmann von Aue Siegfried Lenz Rudolf Christoph Eucken Amy Clampitt Benedetto Varchi 432 347 453 663 564 571 474 475 478 479 678 483 384 583 584 584 888 792 597 799 Distribution of Forget and Retain Questions Unique Forget Questions Unique Retain Questions Fig. 6: Composition of Forget-Retain samples per target 020406080100 Total Connections Najaf Daryabandari Ann Brashares Ted Kooser Jorge SemprĂşn Alfred Vogel Paul Gerhardt Giovanni Battista Casti Heinrich Ritter Philip Stanhope, 5th Earl Stanhope Adrienne Monnier Theopompus Moshe Greenberg Elsa Triolet Wilhelm Wattenbach Heinz Erhardt Hartmann von Aue Siegfried Lenz Rudolf Christoph Eucken Amy Clampitt Benedetto Varchi 527 1928 1340 1845 1945 2051 1856 2055 1761 2058 1861 2063 1766 2064 2064 2064 2068 2072 2077 2079 Distribution of Direct and Indirect Connections with Data Labels Direct Connections Indirect Connections Fig. 7: Composition of Direct and In- direct retain samples per target A.3 Experimental setup Model Finetuning: Before the application of unlearning, we initially finetune the model on the all the datasets we have i.e.,D f ,D r ,D t . We use the questions as prompts and conduct a Supervised Fine-tuning on the datasets. GivenD ft = D f +D r +D t , and its samples(x,y),xis question andyis the answer. A pair p i =p(x i ,y i )âD ft andy 1 ,..y T are the answer tokens, we calculate Negative- Log-Likelihood (NLL) loss forp i L(y|x;θ) = NLL(y|x;θ) =â T X t=1 logp y t |x,y <t ;θ (1) Unlearning Methods: Gradient Ascent (GA)is the most straightforward unlearning technique pro- posed for the Un-targeted unlearning. Itâs main idea is to maximize the loss as opposed to the training objective of minimization by negating the loss. In our work, we do not implement this. Due to its nature of negation, the maximization 20P.Bushipaka et al. becomes unbounded leading to catastrophic collapse. The predicted lossl(y|x;θ) on forget set can be written as: L GA (D f ;θ) =âL(y f |x f ;θ)(2) Gradient Difference (GD)proposed by [19] to mitigate the issues of Gradi- ent ascent. It builds on the concept of Gradient Ascent, but not only aims to maximize the loss on forget setD f , simultaneously minimizes the loss on the retain setD r . This maintains the balance of forgetting and retaining. The loss function can be written as: L GD (θ) =âL(D f ;θ) +L(D r ;θ)(3) Direct Preference Optimization (DPO)is proposed by [27] and was first used by [22], treats unlearning as a preference optimization problem by applying the standard DPO loss. This technique usesTargeted Unlearning, making a ne- cessity of replacement responses like "I donât know". Alike the standard DPO ap- proach, we use "I donât know" responses as positive samples and forget set as neg- ative samples to guide the modelâs response. For the implementation of DPO, we convert the forget set to a preference dataset containing winning responses and refusal responses. Preference datasetD p = (x i ,y i,win ,y i,lose ),iâ |D f |, where y i,win are randomly chosen from a subset of "I donât know" phrases, andy i,lose are the forget targets. The DPO loss can be calculated as: L DPO,β (θ) =âE D p " logĎ Î˛log p(y win |x;θ) p(y win |x;θ ref ) âβlog p(y lose |x;θ) p(y lose |x;θ ref ) # (4) whereĎis the sigmoid function andβis the inverse temperature controlling the preference strength. We useβ= 0.1for all our experiments. Provided, we have a retain set, we utilize the code implementation provided by [7], can be calculated as follows: L DPO+retain =ÎąL DPO,β (θ) +ÎłL(D r ;θ)(5) WhereÎąandÎłare hyperparameter to control the strength of DPO loss and NLL. For our experiments, bothÎąandÎłis always 1. Negative Preference Optimization (NPO) -proposed by [38], is an in- spiration of DPO, a variant that uses only the negative responses from theD f , disregarding they win making it an Un-targeted Unlearning. For the implementa- tion, we ignore the "I donât know" responses and provide only(x f ,y f ). Followed by, we calculate retain loss similarly as in DPO + retain with the same hyper- parameters values(β,Îą,Îł). L NPO,β (θ) =â 2 β E D p " logĎ âβlog p(y lose |x;θ) p(y lose |x;θ ref ) # (6) Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning21 L NPO+retain =ÎąL NPO,β (θ) +ÎłL(D r ;θ)(7) Evaluation Metrics: ROUGE (R)quantifies the world-level overlap between the modelâs output and the ground-truth answer. We compute the ROUGE-L [18] score between the generated responseg(x;θ â )and the ground-truth answer y, written asROUGEâL(g(x;θ â ),y). ROUGE-L provides the longest sequence overlap and the verbatim memory of the Unlearned Model(M;θ â ). Cosine Similarity (CS)measures the semantic similarity of the modelâs output against the ground-truth. We follow [36] setup, embed both with Sentence-BERT [28], calculate the cosine similarity and truncate the values less than 0. max cos g(x;θ â ), y ,0 Probability (P)defines the average likelihood assigned to each token given a question and its ground truth answer i.e.,(x,y). Following [22], we compute normalized conditional probability as P(y|x) = 1 T T X t=1 p y t |xâŚy <t ;θ â 22P.Bushipaka et al. A.4 Results Table 1: Experimental results. Dataset Practices MethodFEâMU-TâPPL-FâPPL-TâMMLU % Pre-Unlearning 0.300.7338.763710512.42 Gradient-based (Un-Targeted) GA0.440.67657294.87242062.3412.47 Direct0.960.343.09Ă10 82 3.28Ă10 80 13.60 Indirect0.890.651.27Ă10 90 1.79Ă10 82 8.40 Balanced0.970.552.24Ă10 85 1.79Ă10 82 13.29 DPO-based (Targeted) DPO0.700.4716643309812.21 Direct0.570.761.82Ă10 4 158.7212.27 Indirect0.710.775.84Ă10 7 180.3012.61 Balanced0.570.791.71Ă10 5 142.5812.35 NPO-based (Un-Targeted) NPO0.300.7338.763710512.37 Direct0.710.694.68Ă10 22 1.002Ă10 17 12.57 Indirect0.600.783.17Ă10 18 126.4712.37 Balanced0.610.742.6Ă10 18 3153.1912.63 Sampling Practices MethodFEâMU-TâPPL-FâPPL-TâMMLU % Gradient-based (Un-Targeted) 1:1 seq0.360.743.7Ă10 4 6.11Ă10 4 12.46 1:1 random 0.350.73225213275112.31 Cyclic0.970.651.80Ă10 86 3.0813.26 MELU0.890.703.18Ă10 90 15.9813.42 DPO-based (Targeted) 1:1 seq0.320.73124.553.0Ă10 3 12.33 1:1 random 0.360.7165.902539.5212.28 Cyclic0.700.802.57Ă10 7 118.5412.36 MELU0.820.795.8Ă10 14 87.7112.38 NPO-based (Un-Targeted) 1:1 seq0.370.711655.975.33Ă10 5 12.43 1:1 random 0.350.7214545.837854412.36 Cyclic0.630.781.60Ă10 18 35.1512.24 MELU0.650.797.86Ă10 21 54.0312.41 Direct Indirect Balanced 1:1 seq 1:1 random Cyclic MELU 7 8 9 10 11 12 13 14 MMLU % Gradient-based Direct Indirect Balanced 1:1 seq 1:1 random Cyclic MELU MMLU % DPO-based Direct Indirect Balanced 1:1 seq 1:1 random Cyclic MELU MMLU % NPO-based Fig. 8: General Model Utility (MMLU) across all experiments. Baseline accuracy is 12.42%. Standard vs. Modular Sampling: Best Practices for Reliable LLM Unlearning23 Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 Forget Test Baseline Forget Baseline Test Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 Direct Indirect Balanced 0.2 0.4 0.6 0.8 1.0 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 Gradient-based Unlearning 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 DPO-based Unlearning 1:1 seq 1:1 random Cyclic MELU 0.2 0.4 0.6 0.8 1.0 NPO-based Unlearning (a) Token diversity of the unlearned models. Top row: data practices; bottom row: implementation practices. Heat maps of per-entity FE and MU-T direct indirect balanced 1_1seq 1_1random cyclic melu Adrienne Monnier Alfred Vogel Amy Clampitt Ann Brashares Benedetto Varchi Elsa Triolet Giovanni Battista Casti Hartmann von Aue Heinrich Ritter Heinz Erhardt Jorge SemprĂşn Moshe Greenberg Najaf Daryabandari Paul Gerhardt Philip Stanhope, 5th Earl Stanhope Rudolf Christoph Eucken Siegfried Lenz Ted Kooser Theopompus Wilhelm Wattenbach 0.590.600.590.410.480.600.83 0.630.760.690.410.450.790.80 0.610.680.590.420.430.660.88 0.070.730.480.160.160.720.72 0.520.720.630.330.420.730.74 0.810.870.840.370.460.830.92 0.560.800.570.340.390.810.83 0.820.640.480.310.310.640.84 0.790.780.680.500.500.750.80 0.760.930.700.400.390.830.94 0.410.710.440.140.210.680.84 0.640.800.550.460.460.720.87 0.370.730.580.190.340.740.75 0.650.580.580.440.460.550.78 0.430.790.530.320.320.640.78 0.690.800.730.330.390.810.88 0.670.670.610.250.370.670.84 0.140.280.100.070.070.500.68 0.370.510.540.300.310.590.81 0.730.920.270.230.230.910.91 DPO direct indirect balanced 1_1seq 1_1random cyclic melu 0.960.890.950.520.510.950.89 0.970.890.980.420.500.980.89 0.980.870.990.410.400.990.87 0.990.900.990.160.160.990.90 0.960.900.960.410.360.960.90 0.980.900.990.440.360.990.90 0.980.940.980.390.390.980.94 0.980.890.990.460.450.990.89 0.980.900.980.510.510.980.90 0.970.880.990.380.380.990.88 0.890.880.960.250.250.980.88 0.970.890.980.440.430.980.89 0.960.900.960.450.450.960.90 0.970.910.970.570.510.970.91 0.960.910.960.210.210.960.91 0.970.900.980.320.280.980.90 0.980.890.990.340.310.990.89 0.990.900.990.070.070.990.90 0.900.900.950.360.350.950.90 0.970.860.980.250.250.980.86 GD direct indirect balanced 1_1seq 1_1random cyclic melu 0.780.570.610.520.500.590.55 0.770.730.750.530.410.810.80 0.780.720.710.420.500.680.70 0.540.490.550.170.160.460.61 0.750.610.730.440.440.660.60 0.760.570.840.380.340.880.87 0.750.760.690.510.380.650.69 0.660.660.700.310.340.700.74 0.670.630.570.460.510.660.66 0.820.780.740.380.380.770.82 0.740.440.480.260.160.540.56 0.740.600.640.440.430.620.64 0.740.550.330.450.450.690.70 0.700.650.710.560.560.630.68 0.600.450.460.330.200.380.53 0.690.640.600.320.310.630.68 0.730.680.680.310.300.700.69 0.530.160.170.070.070.100.13 0.710.550.680.350.330.700.69 0.850.710.560.260.250.670.80 NPO 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 FE Score 0.2 0.4 0.6 0.8 FE Score 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 FE Score FE Scores by Algorithm and Experiment Type (Overall Baseline FE: 0.30) (b) Forget Efficacy (FE). direct indirect balanced 1_1seq 1_1random cyclic melu Adrienne Monnier Alfred Vogel Amy Clampitt Ann Brashares Benedetto Varchi Elsa Triolet Giovanni Battista Casti Hartmann von Aue Heinrich Ritter Heinz Erhardt Jorge SemprĂşn Moshe Greenberg Najaf Daryabandari Paul Gerhardt Philip Stanhope, 5th Earl Stanhope Rudolf Christoph Eucken Siegfried Lenz Ted Kooser Theopompus Wilhelm Wattenbach 0.640.650.670.540.530.690.66 0.430.410.520.420.400.470.44 0.570.580.620.530.530.660.65 0.650.690.770.730.720.720.68 0.800.780.820.670.640.820.83 0.640.600.640.570.580.660.68 0.640.650.670.610.600.690.66 0.500.580.680.480.470.610.66 0.810.880.850.680.680.850.84 0.670.590.760.710.650.640.67 0.770.730.770.750.700.840.85 0.610.670.670.570.570.660.69 0.500.430.520.450.450.600.56 0.560.590.600.560.560.600.60 0.640.610.670.630.620.680.69 0.770.720.770.710.640.800.77 0.740.710.720.700.650.760.73 0.700.680.740.560.540.760.72 0.890.920.910.810.810.950.94 0.500.570.580.470.450.580.59 DPO direct indirect balanced 1_1seq 1_1random cyclic melu 0.210.510.440.590.520.530.56 0.120.400.300.450.450.420.42 0.120.320.330.540.520.450.51 0.210.450.440.640.680.550.53 0.330.620.480.750.740.610.66 0.170.440.360.500.510.470.56 0.090.340.240.590.660.400.47 0.160.520.320.500.500.550.52 0.250.550.450.760.760.540.65 0.320.560.420.680.680.570.63 0.320.630.560.740.730.600.64 0.200.580.410.610.630.560.52 0.170.440.360.510.480.460.52 0.200.460.440.600.570.540.59 0.200.390.310.690.700.470.52 0.330.630.630.720.730.590.76 0.280.530.540.710.700.540.62 0.220.520.400.620.630.570.54 0.290.660.670.780.780.760.77 0.130.590.330.530.530.490.50 GD direct indirect balanced 1_1seq 1_1random cyclic melu 0.480.690.560.570.500.600.62 0.400.490.470.450.450.470.51 0.480.570.530.510.510.580.65 0.580.690.640.660.680.660.66 0.690.750.740.660.680.810.79 0.480.590.550.530.500.680.61 0.400.650.590.590.540.530.60 0.440.570.570.460.520.590.66 0.710.800.750.690.720.840.79 0.600.660.620.660.680.690.67 0.620.780.720.720.720.790.78 0.500.650.660.590.580.710.70 0.510.560.510.450.420.500.60 0.480.640.530.580.580.620.67 0.450.680.600.640.640.660.67 0.640.700.710.680.700.800.75 0.640.700.660.660.690.720.73 0.550.720.640.530.550.680.68 0.830.870.830.770.740.830.86 0.460.620.560.510.510.630.57 NPO 0.4 0.5 0.6 0.7 0.8 0.9 MU_T Score 0.1 0.2 0.3 0.4 0.5 0.6 0.7 MU_T Score 0.4 0.5 0.6 0.7 0.8 MU_T Score MU_T Scores by Algorithm and Experiment Type (Overall Baseline MU_T: 0.73) (c) Model Utility Test (MU-T). Fig. 9: Token diversity and per-entity metrics. 24P.Bushipaka et al. DirectIndirectGeneral 0.0 0.2 0.4 0.6 0.8 1.0 MU T Direct DirectIndirectGeneral Indirect DirectIndirectGeneral Balanced MUT across Direct | Indirect | Balanced Settings BaselineGDDPONPO Fig. 10: MU-T across various data practices. We find Gradient Based method consistently performs bad in all the Data settings, where as NPO and DPO holds the MU-T well. A.5 Memorization We calculated each entities memorization with Exact Memorization (EM) score following [31], which is often used in Unlearning research to define/compute the success of forgetting in LLMs. Given we have multiple samples for each entity, we compute their Average EM score. We find that few samples such asTed Kooser, Philip Stanhope, Ann Brashes etcare highly memorized and were harder to forget in our experiments. 0.00.20.40.60.81.0 Average Memorization Score Adrienne Monnier Alfred Vogel Amy Clampitt Ann Brashares Benedetto Varchi Elsa Triolet Giovanni B. Casti Hartmann v. Aue Heinrich Ritter Heinz Erhardt Jorge SemprĂşn Moshe Greenberg Najaf Daryabandari Paul Gerhardt Philip Stanhope Rudolf C. Eucken Siegfried Lenz Ted Kooser Theopompus Wilhelm Wattenbach Memorization Scores of Entities Fig. 11: Memorization Scores of Each Entity