Paper deep dive

DUSK: Do Not Unlearn Shared Knowledge

Wonje Jeung, Sangyeon Yoon, Hyesoo Hong, Soeun Kim, Seungju Han, Youngjae Yu, Albert No

Year: 2025Venue: arXiv preprintArea: Model EditingType: BenchmarkEmbeddings: 95

Models: LLaMA-3-8B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 6:32:45 PM

Summary

DUSK is a new benchmark for machine unlearning in large language models that addresses the realistic scenario of overlapping data between 'forget' and 'retain' sets. Unlike previous benchmarks that assume disjoint sets, DUSK evaluates whether models can selectively remove unique information from a forget set while preserving shared factual knowledge supported by the retain set, using seven specific evaluation metrics.

Entities (5)

DUSK · benchmark · 100%Large Language Models · technology · 100%Machine Unlearning · field-of-study · 100%MUSE · benchmark · 95%TOFU · benchmark · 95%

Relation Signals (3)

Machine Unlearning → appliedto → Large Language Models

confidence 100% · Recent efforts have extended unlearning techniques to large language models (LLMs)

DUSK → evaluates → Machine Unlearning

confidence 100% · DUSK, a benchmark designed to evaluate unlearning methods under realistic data overlap.

DUSK → addresseslimitationof → TOFU

confidence 90% · we construct a synthetic dataset inspired by TOFU... existing benchmarks largely assume that the forget set contains only information to be removed

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) are increasingly deployed in real-world applications, raising concerns about the unauthorized use of copyrighted or sensitive data. Machine unlearning aims to remove such 'forget' data while preserving utility and information from the 'retain' set. However, existing evaluations typically assume that forget and retain sets are fully disjoint, overlooking realistic scenarios where they share overlapping content. For instance, a news article may need to be unlearned, even though the same event, such as an earthquake in Japan, is also described factually on Wikipedia. Effective unlearning should remove the specific phrasing of the news article while preserving publicly supported facts. In this paper, we introduce DUSK, a benchmark designed to evaluate unlearning methods under realistic data overlap. DUSK constructs document sets that describe the same factual content in different styles, with some shared information appearing across all sets and other content remaining unique to each. When one set is designated for unlearning, an ideal method should remove its unique content while preserving shared facts. We define seven evaluation metrics to assess whether unlearning methods can achieve this selective removal. Our evaluation of nine recent unlearning methods reveals a key limitation: while most can remove surface-level text, they often fail to erase deeper, context-specific knowledge without damaging shared content. We release DUSK as a public benchmark to support the development of more precise and reliable unlearning techniques for real-world applications.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

94,826 characters extracted from source content.

Expand or collapse full text

DUSK: Do Not Unlearn Shared Knowledge Wonje Jeung1 Sangyeon Yoon2∗ Hyesoo Hong1 Soeun Kim1 Seungju Han3 Youngjae Yu1 Albert No1 1Yonsei University 2Hongik University 3Standford University Equal contribution.Corresponding author Abstract Large language models (LLMs) are increasingly deployed in real-world applications, raising concerns about the unauthorized use of copyrighted or sensitive data. Machine unlearning aims to remove such “forget” data while preserving utility and information from the “retain” set. However, existing evaluations typically assume that forget and retain sets are fully disjoint, overlooking realistic scenarios where they share overlapping content. For instance, a news article may need to be unlearned, even though the same event—such as an earthquake in Japan—is also described factually on Wikipedia. Effective unlearning should remove the specific phrasing of the news article while preserving publicly supported facts. In this paper, we introduce DUSK, a benchmark designed to evaluate unlearning methods under realistic data overlap. DUSK constructs document sets that describe the same factual content in different styles, with some shared information appearing across all sets and other content remaining unique to each. When one set is designated for unlearning, an ideal method should remove its unique content while preserving shared facts. We define seven evaluation metrics to assess whether unlearning methods can achieve this selective removal. Our evaluation of nine recent unlearning methods reveals a key limitation: while most can remove surface-level text, they often fail to erase deeper, context-specific knowledge without damaging shared content. We release DUSK and code at https://ai-isl.github.io/dusk to support precise, reliable unlearning for real-world use. 1 Introduction Large language models (LLMs) are typically trained on web-scale corpora that include copyrighted materials, personal data, and user-generated content (carlini2021extracting, ; nasr2023scalable, ). As these models are deployed in real-world applications, individuals and organizations increasingly demand the removal of specific training examples due to legal and ethical concerns. These demands are driven by privacy regulations such as the GDPR (voigt2017eu, ) and reinforced by recent lawsuits (grynbaum2023times, ; openailawsuit2, ; githublitigation, ) over the unauthorized use of proprietary content. This has led to growing interest in machine unlearning (nguyen2022survey, ; liu2025rethinking, ), which focuses on removing the influence of forget data (e.g., copyrighted documents) from a trained model without retraining from scratch, while preserving information from retain data. To evaluate unlearning algorithms, several benchmarks have recently been proposed (maini2024tofu, ; shi2024muse, ; jin2024rwku, ). For example, MUSE (shi2024muse, ) targets copyright-related scenarios by focusing on the removal of entire documents. In such cases, unlearning algorithms are expected to erase both verbatim text and the underlying knowledge from the forget set, while preserving information learned from the retain set. However, existing evaluations often assume that the forget set and retain set are disjoint, overlooking the complexity of real-world data. In practice, documents frequently contain overlapping information. For instance, a New York Times report might state, "A 6.2 magnitude earthquake struck Tokyo on Monday," while Wikipedia might describe the same event as "A strong tremor shook the Japanese capital at the start of the week." Despite the different phrasing, both convey the same core facts—location, magnitude, and timing. This highlights a critical challenge: effectively unlearning only the unique information from a forget set without disrupting widely supported facts in the retain set. To address this gap, we introduce DUSK, a benchmark for evaluating unlearning in realistic multi-source settings where the same information can appear across both forget and retain data. DUSK constructs document sets that describe identical factual content in different styles, enabling controlled attribution: some information is present only in the forget set, while other content remains supported by the retain set. Specifically, DUSK is constructed using 120 synthetic professor profiles, organized into five documents. Each document contains 12 unique profiles that appear only in that document, along with 60 shared profiles that appear across multiple documents. This structure allows systematic evaluation of whether an algorithm can effectively remove content uniquely attributable to the forget set without erasing widely supported, shared knowledge. An overview of the DUSK framework is presented in Figure 1. DUSK defines seven evaluation metrics: (1) Verbatim Memorization, which checks whether exact text from the forget set has been fully removed; (2) Unique Forget Knowledge, which measures whether forget-only content is effectively erased; (3) Shared Knowledge, which ensures that overlapping information is preserved; (4) Unique Retain Knowledge, which verifies that retain-only facts remain intact; (5) Downstream Capability, which evaluates whether the model maintains general utility after unlearning; (6) Privacy Leakage, which checks for residual leakage of forget set information; and (7) Retain Deviation, which assesses whether the model’s behavior on the retain set is preserved. We evaluate nine unlearning methods on the DUSK benchmark. Our results reveal a critical limitation of existing methods: while these methods aim to remove content uniquely attributable to the forget set, they often struggle to fully separate this content from information still supported by the retain set. This incomplete disentanglement can degrade shared knowledge, compromising the model’s overall utility. These findings highlight a fundamental challenge in current unlearning approaches: precisely distinguishing forget-specific information from retained knowledge. To support further research, we release DUSK as a public benchmark for evaluating unlearning in real-world scenarios. Figure 1: DUSK provides a realistic unlearning evaluation scenario where forget documents (fsubscriptD_fDitalic_f) contain both unique information to be forgotten and shared knowledge that must be preserved. Unlike conventional setups that naively erase entire forget sets, DUSK evaluates whether unlearning methods can selectively remove sensitive information while retaining shared knowledge supported by other documents in the retain set (rsubscriptD_rDitalic_r), which is not subject to forgetting. 2 Related Work Machine Unlearning in LLMs: Methods and Applications. Machine unlearning aims to selectively remove the influence of forget data from a trained model while preserving its performance on retain data (cao2015towards, ; brophy2021machine, ; guo2019certified, ; jeon2024information, ). Recent efforts have extended unlearning techniques to large language models (LLMs) (liu2025rethinking, ), enabling their use in a range of applications such as removing copyrighted content (kassem2023preserving, ; wei2024evaluating, ), eliminating sensitive or harmful knowledge (maini2024tofu, ; yousefpour2025representation, ; zhang2024safe, ), mitigating bias (dige2024can, ; jeung2024large, ), and performing model editing (guo2024mechanistic, ). Most methods achieve unlearning by fine-tuning on forget data (chen2023unlearn, ; jia2024soul, ; cao2015towards, ; barbulescu2024each, ), commonly using gradient ascent (jang2022knowledge, ) or preference optimization (zhang2024negative, ). To scale these methods to large models, recent work has explored approaches such as guardrail-based techniques (thaker2024guardrail, ; gaolarge, ), and in-context unlearning (pawelczyk2023context, ). Despite this progress, recent studies have highlighted the fragility of current unlearning techniques (hu2024jogging, ; lynch2024eight, ; thaker2024position, ; zhang2024does, ; joshi2024towards, ; jeung2025seps, ), revealing fundamental challenges in achieving robust and reliable unlearning in practice. Machine Unlearning in LLMs: Benchmarks. As machine unlearning methods for LLMs evolve, the need for comprehensive evaluation benchmarks has become increasingly important. Early work introduced the “Who is Harry Potter” (WHP) task (eldan2023s, ), which targets entity-specific forgetting by fine-tuning models on fictional corpora and evaluating unlearning through related prompts while monitoring retention on unrelated tasks. Subsequent efforts expand the scope to hazardous knowledge, with the WMDP (li2024wmdp, ) focusing on removing information related to biosecurity and cybersecurity while preserving general model capabilities. To enable controlled evaluation of unlearning, TOFU (maini2024tofu, ) constructs synthetic author profiles with associated question-answer pairs generated by GPT-4 (achiam2023gpt, ). This synthetic setup ensures that the model’s knowledge of these authors originates solely from the fine-tuning process, allowing for precise assessment of unlearning effectiveness. MUSE (shi2024muse, ) advances evaluation by systematically assessing unlearning algorithms across six dimensions, including forgetting effectiveness, privacy leakage, utility retention, scalability, and sustainability. It aims to remove both verbatim content and underlying knowledge within the forget set. CoTAEval (wei2024evaluating, ) instead focuses on a narrower goal—removing only verbatim memorization while explicitly preserving the associated knowledge. Building on this line of work, RWKU (jin2024rwku, ) proposes a more practical and challenging setting where neither the forget nor retain corpus is accessible. It targets the removal of widely known real-world knowledge, such as facts about 200 famous individuals, and evaluates performance via membership inference attacks, adversarial probes, and tasks assessing reasoning, truthfulness, and fluency. However, existing benchmarks largely assume that the forget set contains only information to be removed, overlooking the realistic scenario where forget documents often contain both information that should be forgotten and information that should be retained. We address this gap by introducing DUSK, a benchmark for multi-source unlearning where forget-specific and retained knowledge coexist within each document. 3 The DUSK Benchmark 3.1 Problem Setting DUSK is designed to evaluate unlearning under more realistic conditions, where the retain and forget sets may share overlapping information. Specifically, our benchmark simulates a training corpus in which each document consists of two types of content: (1) shared knowledge, which refers to factual content that appears in multiple documents and should be retained even if one document is deleted, and (2) unique knowledge (i.e., document-specific), which is unique to a single document and should be forgotten when that document is removed. Given a forget request for a particular document, we assess whether an unlearning algorithm can: 1. Preserve shared knowledge that remains supported by the retain set. 2. Remove content uniquely attributable to the forgotten document. 3. Preserve information that is unique to other (non-forgotten) documents. This task formulation captures the challenge of unlearning in multi-source environments, where the boundary between what should be forgotten and what should be retained is not cleanly separable. 3.2 Problem Formulation and Notations Let fθsubscriptf_θfitalic_θ be a model trained on a dataset DD, and let f⊂subscriptD_f _f ⊂ D denote the subset of training data targeted for removal (i.e., forget set). The goal is to produce an updated model fθ′subscriptsuperscript′f_θ fitalic_θ′ that no longer exposes or relies on the information contained in fsubscriptD_fDitalic_f, while maintaining utility on the remaining data, r=∖fsubscriptsubscriptD_r=D _fDitalic_r = D ∖ Ditalic_f. We define fsubscriptK_fKitalic_f as the knowledge contained in the forget set fsubscriptD_fDitalic_f, and rsubscriptK_rKitalic_r as the knowledge contained in the retain set rsubscriptD_rDitalic_r. Prior works often assume that f∩r=∅subscriptsubscriptK_f _r= _f ∩ Kitalic_r = ∅, meaning the knowledge from the forget and retain sets does not overlap. However, this assumption rarely holds in practice. In many real-world cases, the same information appears across both sets in different phrasings or styles. As a result, an effective unlearning method must identify and remove only the portion of knowledge that is uniquely attributable to fsubscriptD_fDitalic_f, while preserving content that is also supported by rsubscriptD_rDitalic_r. 3.3 The DUSK Dataset Construction To ensure precise control over the origin and overlap of information, we construct a synthetic dataset inspired by TOFU (maini2024tofu, ). The dataset comprises 120 fictitious professor profiles, each built using structured attributes, such as academic department and institutional affiliation. Since these profiles have never appeared in any pretrained corpus, the dataset provides a clean experimental environment with clearly defined forget and retain sets. We generate five documents, each representing an independent data source. These documents collectively cover all 120 profiles, including both shared profiles that appear in multiple documents and unique profiles that are present in only one, allowing for fine-grained control over content attribution across sources. The profiles are partitioned into two disjoint subsets: • Shared Knowledge: 60 profiles appear in all five documents, each presented in a different style. These profiles represent redundantly supported knowledge that should be preserved regardless of which document is unlearned. • Unique Knowledge: The remaining 60 profiles are evenly distributed across the five documents, with each document containing 12 unique profiles that do not appear in any other. These profiles represent document-specific information that should be forgotten when the corresponding document is unlearned. We also create a holdout set (hsubscriptℎD_hDitalic_h) consisting of 120 professors that do not overlap with rsubscriptD_rDitalic_r or fsubscriptD_fDitalic_f, following the same process used to construct rsubscriptD_rDitalic_r and fsubscriptD_fDitalic_f. The holdout set has never been included in the training data for either the Retrain model or the Target model. 3.3.1 Data Generation Pipeline Figure 2: Distributions of country of nationality (left) and graduate year (right) for the seven most common attributes in GPT-4 outputs. This reveals mode collapse with default prompts, disproportionately favoring frequent values like “Canada” and “2010.” After prompt refinement, distributions become more balanced, reflecting a more diverse attribute range. Knowledge Source. We begin by generating a knowledge base of 120 fictitious professor profiles, each represented as a set of 20 question–answer (QA) pairs covering attributes such as birth year, nationality, department, and academic history. These QA pairs are synthesized using GPT-4 to ensure fluent and diverse natural language. However, we observe that GPT-4 exhibits strong biases in generating certain attribute values. For example, as shown in Figure 2, GPT-4 disproportionately favors the year 2010 when asked about graduation years, with this value appearing in nearly 40% of cases. Similarly, nationality values are skewed, with a strong preference for “Canadian,” which subsequently influences correlated fields such as birthplace and affiliated universities. To mitigate these biases, we iteratively refine the prompt design. When we observe skewed distributions in attributes such as nationality or religion in the generated data, we adjust the prompts accordingly by explicitly specifying underrepresented categories or by sampling values like graduation year uniformly within a reasonable range, and then regenerate the data. Through four iterations of prompt-based refinement, we obtain more balanced and realistic outputs from GPT-4. As illustrated in Figure 2, the final set of profiles exhibits significantly improved attribute diversity compared to the initial generations. Additional implementation details are described in Section A.1. Document Construction. Using the processed QA profiles as source knowledge, we construct five distinct documents, denoted as ii=15superscriptsubscriptsubscript15\D_i\_i=1^5 Ditalic_i i = 15, each expressing the underlying content through a different narrative style. We designate 1subscript1D_1D1 as the forget set and define the retain set as the union of the remaining documents, r=⋃i=25isubscriptsuperscriptsubscript25subscriptD_r= _i=2^5D_iDitalic_r = ⋃i = 25 Ditalic_i. To simulate stylistic diversity reflective of real-world corpora, we format each document using a distinct narrative genre: Chronological, which presents profiles as career timelines ordered by milestones; Feature Story, which uses narrative-driven descriptions akin to editorial articles; Interview, which formats profiles as fictional Q&A sessions with conversational tone; Inverted Pyramid, which follows journalistic convention by placing key facts first; and Listicle, which presents profiles in ranked or grouped lists using bullet-point highlights. Each document includes the same 60 shared profiles and 12 unique profiles, expressed in style-specific templates. This construction allows us to evaluate whether unlearning methods can selectively remove isolated information while maintaining general knowledge, when presented under stylistic or structural variation. Corresponding text examples for the shared knowledge in each document style are provided in Section A.2. 3.4 The DUSK Evaluation The DUSK evaluation framework characterizes unlearning behavior in three dimensions: (1) what should be forgotten, (2) what should be retained, and (3) whether the model behaves as if trained only on the retain set. An effective unlearning method should eliminate not only verbatim content from the forget set but also knowledge uniquely attributable to it; retain shared and exclusive information in the retain set while preserving downstream capabilities; and ensure that the model’s behavior becomes indistinguishable from that of a model trained without access to the forget set. 3.4.1 Forget Assessment Verbatim Memorization (VM). We assess whether the unlearned model can still reproduce exact phrasings from the forget set, even when the underlying knowledge is shared across both forget and retain sets. While such shared knowledge should be preserved, any specific wording originating from the forget document must be removed. This is particularly critical because the forget set often contains copyright-protected material and regenerating such text would indicate incomplete unlearning. To comprehensively evaluate memorization, we prompt the model with partial sequences from fsubscriptD_fDitalic_f, denoted as d[:ℓ]subscriptdelimited-[]:absentℓd_[: ]d[ : ℓ ], and compare the model’s continuations to the original text d[ℓ+1:]d_[ +1:]d[ ℓ + 1 : ] across multiple metrics. Specifically, we use ROUGE-1 and ROUGE-L (F1 scores) to measure overall lexical and structural overlap, and their Recall variants to emphasize ground-truth coverage. We further include Levenshtein Distance (levenshtein1966binary, ) to quantify the minimum number of edits required for alignment, Longest Common Subsequence (LCS) for sequential token overlap, and Cosine Similarity (cer2017semeval, ) for embedding-level semantic similarity. Higher scores indicate that the model remains capable of reproducing text that should have been forgotten. Unique Forget Knowledge (UFK). We evaluate whether the model retains knowledge f∖rsubscriptsubscriptK_f _rKitalic_f ∖ Kitalic_r that is uniquely attributable to the forget set fsubscriptD_fDitalic_f by prompting it with targeted questions. Overlap between the model’s responses and the correct answers is measured using ROUGE-L scores (lin2004rouge, ), where lower scores indicate more effective unlearning of forget source-specific information. 3.4.2 Retain Assessment Shared Knowledge (SK). Unlike prior benchmarks that aim to remove all knowledge about the forget set, multi-source scenarios, where training data originates from diverse and overlapping sources, often involve shared knowledge appearing in both fsubscriptD_fDitalic_f and rsubscriptD_rDitalic_r. In such cases, indiscriminately unlearning the entire forget set risks discarding overlapping content that should remain accessible. To assess the preservation of shared knowledge, we construct queries targeting f∩rsubscriptsubscriptK_f _rKitalic_f ∩ Kitalic_r, i.e., information present in both the forget and retain sets, and evaluate the model’s responses using ROUGE-L scores against ground truth answers. High scores indicate successful preservation, while low scores indicate unintended forgetting caused by overly aggressive unlearning. Unique Retain Knowledge (URK). To assess the preservation of retain-exclusive knowledge r∖fsubscriptsubscriptK_r _fKitalic_r ∖ Kitalic_f, we construct queries answerable only from rsubscriptD_rDitalic_r and not from fsubscriptD_fDitalic_f. Model responses are compared against ground truth answers using ROUGE-L scores, where higher scores indicate successful retention without unintended removal of unique retain content. Downstream Capability (DC). We verify that the model’s fundamental capabilities, such as reasoning, factual consistency, and fairness are preserved after unlearning. We assess performance across six downstream tasks: MMLU for broad knowledge and ability (hendrycks2020measuring, ), ARC-c for challenging reasoning (clark2018think, ), GSM8K for arithmetic and problem-solving (cobbe2021training, ), TriviaQA for factual recall (joshi2017triviaqa, ), TruthfulQA (MC1) for truthfulness evaluation (lin2021truthfulqa, ), and BBQ for social bias probing under ambiguity (parrish2021bbq, ). A successful unlearning method should remove only the targeted information while maintaining strong performance across these core competencies and downstream tasks. 3.4.3 Distributional Assessment (a) Target Model (trained on DD) (b) Retrain Model (trained on rsubscriptD_rDitalic_r) (c) RMU at 50 epochs Figure 3: Min-K++ Probability Distributions over fsubscriptD_fDitalic_f, rsubscriptD_rDitalic_r, and hsubscriptℎD_hDitalic_h. (a) Target model trained on both fsubscriptD_fDitalic_f and rsubscriptD_rDitalic_r show higher probabilities, reflecting retained knowledge, while hsubscriptℎD_hDitalic_h exhibits lower probabilities. (b) Retrain model reduces probabilities on the fsubscriptD_fDitalic_f, as they are not trained on fsubscriptD_fDitalic_f, representing ideal unlearning. (c) Some unlearned models achieve ideal low probabilities on fsubscriptD_fDitalic_f but risk collapsing rsubscriptD_rDitalic_r, which can be detected using Retain Deviation. Privacy Leakage. We assess privacy leakage by evaluating whether any behavioral traces from the forget set remain in the unlearned model. Following the MUSE benchmark (shi2024muse, ), we adopt a membership inference attack (MIA) framework and apply Min-K%++ (zhang2024min, ) to capture subtle distributional differences. Specifically, as shown in Figure 3, we measure the model’s ability to distinguish samples from forget set (fsubscriptD_fDitalic_f) and a holdout set (hsubscriptℎD_hDitalic_h), which consists of unseen data. We report the AUC-ROC of this discrimination task and normalize it relative to a Retrain model that excludes fsubscriptD_fDitalic_f for training. Privacy Leakage score is defined as: PrivacyLeak:=AUCunlearn⁢(f,h)−AUCretrain⁢(f,h)AUCretrain⁢(f,h).assignPrivacyLeaksubscriptAUCunlearnsubscriptsubscriptℎsubscriptAUCretrainsubscriptsubscriptℎsubscriptAUCretrainsubscriptsubscriptℎPrivacyLeak:= AUC_unlearn(D_f, % D_h)-AUC_retrain(D_f,D_h)% AUC_retrain(D_f,D_h).PrivacyLeak := divide start_ARG AUCunlearn ( Ditalic_f , Ditalic_h ) - AUCretrain ( Ditalic_f , Ditalic_h ) end_ARG start_ARG AUCretrain ( Ditalic_f , Ditalic_h ) end_ARG . A Privacy Leakage value close to zero indicates that the unlearned model treats fsubscriptD_fDitalic_f similarly to hsubscriptℎD_hDitalic_h, suggesting successful unlearning of fsubscriptD_fDitalic_f. Values below zero indicate under-unlearning, where the model continues to assign high probability to forget data. Conversely, values above zero reflect over-unlearning, where the model suppresses forget set too aggressively, leading to excessive forgetting. As shown in Figure 3(b), it is important to note that fsubscriptD_fDitalic_f and hsubscriptℎD_hDitalic_h are not expected to follow identical distributions even after ideal unlearning. This is because fsubscriptD_fDitalic_f may contain shared knowledge that overlaps with the rsubscriptD_rDitalic_r, while hsubscriptℎD_hDitalic_h consists entirely of unseen content. Retain Deviation. Unlearning often disrupts the model’s ability to distinguish rsubscriptD_rDitalic_r, causing both fsubscriptD_fDitalic_f and rsubscriptD_rDitalic_r to collapse toward hsubscriptℎD_hDitalic_h, as illustrated in Figure 3(c). This issue becomes more pronounced in multi-source settings, where overlapping information between fsubscriptD_fDitalic_f and rsubscriptD_rDitalic_r makes rsubscriptD_rDitalic_r more vulnerable to unintended forgetting. To quantify this side effect, we introduce a supplementary metric, Retain Deviation, which applies the same MIA framework to rsubscriptD_rDitalic_r and is defined as: RetainDeviation:=|AUCunlearn⁢(r,h)−AUCretrain⁢(r,h)AUCretrain⁢(r,h)|.assignRetainDeviationsubscriptAUCunlearnsubscriptsubscriptℎsubscriptAUCretrainsubscriptsubscriptℎsubscriptAUCretrainsubscriptsubscriptℎRetainDeviation:= | AUC_unlearn(D_r% ,D_h)-AUC_retrain(D_r,D_h% )AUC_retrain(D_r,D_h) |.RetainDeviation := | divide start_ARG AUCunlearn ( Ditalic_r , Ditalic_h ) - AUCretrain ( Ditalic_r , Ditalic_h ) end_ARG start_ARG AUCretrain ( Ditalic_r , Ditalic_h ) end_ARG | . A low Retain Deviation score indicates that the model retains its original capabilities on rsubscriptD_rDitalic_r after unlearning. As Retain Deviation increases, the model’s behavior on rsubscriptD_rDitalic_r diverges from its original state, suggesting that important retained knowledge may not have been properly preserved. (a) Forget Verbatim vs. Forget Knowledge. (b) Forget knowledge vs. Shared knowledge. Figure 4: Two-dimensional analysis of unlearning dynamics. We visualize model trajectories over multiple epochs to illustrate key trade-offs in DUSK. (a) shows the trade-off between verbatim and knowledge forgetting, while (b) shows the trade-off between shared knowledge and unique forget knowledge. 4 Experiments 4.1 Unlearning Methods Removing Forget Set. We introduce five unlearning methods designed to effectively remove the influence of forget data. Gradient Ascent (GA) (jang2022knowledge, ) maximizes the loss on the forget set fsubscriptD_fDitalic_f, reducing the model’s ability to reproduce its content. Negative Preference Optimization (NPO) (zhang2024negative, ) extends DPO (rafailov2024direct, ) for unlearning by treating samples in fsubscriptD_fDitalic_f as negative preferences relative to a Target model. RMU (li2024wmdp, ) modifies intermediate representations by pushing activations of the forget set toward random directions, while aligning retain set activations with those of a frozen Target model. Task Vector (TV) (ilharco2022editing, ) removes the influence of fsubscriptD_fDitalic_f by computing the parameter changes caused by fine-tuning on fsubscriptD_fDitalic_f and subtracting them from the original model weights. Lastly, Task Arithmetic for Unlearning (TAU) (barbulescu2024each, ) performs two steps: it first applies gradient ascent selectively to samples with high memorization scores, and then conducts task vector subtraction as described above. Preserving Retain Set. To maintain model utility during unlearning, we incorporate two regularization losses. Gradient Descent (GD) preserves performance on the retain set rsubscriptD_rDitalic_r by applying prediction loss. This ensures that removing fsubscriptD_fDitalic_f does not excessively degrade the model’s behavior on unrelated data. KL Divergence (KL) (hinton2015distilling, ) encourages consistency between the unlearned model’s predictions on rsubscriptD_rDitalic_r and those of a Target model. By minimizing this divergence, the model retains useful information while softly constraining deviation from its original output distribution. Consequently, we evaluate nine total configurations: GAGAGAGA, GAGDsubscriptGAGDGA_GDGAroman_GD, GAKLsubscriptGAKLGA_KLGAroman_KL, NPONPONPONPO, NPOGDsubscriptNPOGDNPO_GDNPOroman_GD, NPOKLsubscriptNPOKLNPO_KLNPOroman_KL, RMURMURMURMU, TVTVTVTV, and TAUTAUTAUTAU, where the suffix indicates an added utility-preserving objective. Additional details about the methods can be found in Section B.1. 4.2 Experimental Setup We begin with a pretrained base model (LLaMA-3-8B (grattafiori2024llama, )). Target model is obtained by fine-tuning on the full corpus (r∪fsubscriptsubscriptD_r _fDitalic_r ∪ Ditalic_f) for 5 epochs with a learning rate of 1×10−51superscript1051× 10^-51 × 10- 5, following prior benchmarks (maini2024tofu, ; shi2024muse, ). Retrain model is trained only on the retain set rsubscriptD_rDitalic_r under the same setup. For all unlearning methods, we adopt the AdamW optimizer with a learning rate of 1×10−51superscript1051× 10^-51 × 10- 5 and a batch size of 32, using the first epoch as a warm-up phase, consistent with prior work (maini2024tofu, ). Since unlearning performance is sensitive to the number of training epochs, we standardize the stopping criterion across methods: We terminate unlearning at the first epoch where the Unique Retain Knowledge (URK) score falls below 70. This ensures comparable utility levels, enabling fair and consistent comparisons across methods. Further implementation details are provided in Appendix B. 4.3 Unlearning Results 4.3.1 Forget Assessment Results. Table 1: Impact of Unlearning on UFK, SK, URK, and DC. We report both raw values and their differences relative to the Retrain model. Red indicates higher values, and Blue indicates lower values, with darker shades indicating greater magnitude. DC is calculated by averaging of six benchmarks explained in Section 3.4.2. Unique Forget Knowledge Shared Knowledge Unique Retain Knowledge Downstream Capability UFK (↓ ↓) SK (↑ ↑) URK (↑ ↑) DC (↑ ↑) Target 83.883.883.883.8 98.698.698.698.6 88.788.788.788.7 40.340.340.340.3 Retrain 5.25.25.25.2 98.398.398.398.3 84.884.884.884.8 40.640.640.640.6 GAGAGAGA 24.324.324.324.3 (+367%)percent367(+367\%)( + 367 % ) 50.750.750.750.7 (−48%)percent48(-48\%)( - 48 % ) 52.552.552.552.5 (−38%)percent38(-38\%)( - 38 % ) 37.537.537.537.5 (−8%)percent8(-8\%)( - 8 % ) GAGDsubscriptGAGDGA_GDGAroman_GD 38.238.238.238.2 (+635%)percent635(+635\%)( + 635 % ) 63.763.763.763.7 (−35%)percent35(-35\%)( - 35 % ) 63.663.663.663.6 (−25%)percent25(-25\%)( - 25 % ) 39.039.039.039.0 (−4%)percent4(-4\%)( - 4 % ) GAKLsubscriptGAKLGA_KLGAroman_KL 24.924.924.924.9 (+379%)percent379(+379\%)( + 379 % ) 56.256.256.256.2 (−43%)percent43(-43\%)( - 43 % ) 57.457.457.457.4 (−32%)percent32(-32\%)( - 32 % ) 38.638.638.638.6 (−5%)percent5(-5\%)( - 5 % ) NPONPONPONPO 43.043.043.043.0 (+727%)percent727(+727\%)( + 727 % ) 73.473.473.473.4 (−25%)percent25(-25\%)( - 25 % ) 69.669.669.669.6 (−18%)percent18(-18\%)( - 18 % ) 39.539.539.539.5 (−3%)percent3(-3\%)( - 3 % ) NPOGDsubscriptNPOGDNPO_GDNPOroman_GD 27.127.127.127.1 (+421%)percent421(+421\%)( + 421 % ) 54.454.454.454.4 (−45%)percent45(-45\%)( - 45 % ) 51.151.151.151.1 (−40%)percent40(-40\%)( - 40 % ) 37.837.837.837.8 (−7%)percent7(-7\%)( - 7 % ) NPOKLsubscriptNPOKLNPO_KLNPOroman_KL 30.430.430.430.4 (+485%)percent485(+485\%)( + 485 % ) 52.752.752.752.7 (−46%)percent46(-46\%)( - 46 % ) 52.852.852.852.8 (−38%)percent38(-38\%)( - 38 % ) 37.737.737.737.7 (−7%)percent7(-7\%)( - 7 % ) RMURMURMURMU 55.155.155.155.1 (+960%)percent960(+960\%)( + 960 % ) 74.074.074.074.0 (−25%)percent25(-25\%)( - 25 % ) 64.064.064.064.0 (−25%)percent25(-25\%)( - 25 % ) 39.139.139.139.1 (−4%)percent4(-4\%)( - 4 % ) TVTVTVTV 35.335.335.335.3 (+579%)percent579(+579\%)( + 579 % ) 62.462.462.462.4 (−37%)percent37(-37\%)( - 37 % ) 69.169.169.169.1 (−19%)percent19(-19\%)( - 19 % ) 40.340.340.340.3 (−1%)percent1(-1\%)( - 1 % ) TAUTAUTAUTAU 27.527.527.527.5 (+429%)percent429(+429\%)( + 429 % ) 33.533.533.533.5 (−66%)percent66(-66\%)( - 66 % ) 50.750.750.750.7 (−40%)percent40(-40\%)( - 40 % ) 35.835.835.835.8 (−12%)percent12(-12\%)( - 12 % ) Verbatim Memorization (VM). We first evaluate whether unlearning methods can suppress verbatim memorization of the forget set. Given prefix excerpts from fsubscriptD_fDitalic_f, we prompt the model to continue the text and measure similarity with the original using ROUGE, Levenshtein distance, LCS, and Cosine similarity. As shown in Figure 4(a), most methods reduce overlap with the original text, with TAU achieving the largest reductions, indicating strong suppression of verbatim memorization. Full results are provided in Section C.1. Unique Forget Knowledge (UFK). To assess whether unlearning removes not just surface expressions but deeper factual knowledge, we evaluate models on Unique Forget Knowledge (UFK), consisting of questions that rely exclusively on information from fsubscriptD_fDitalic_f. As shown in Figure 4(a), which plots average ROUGE scores against UFK accuracy, most methods shift into the Verbatim Forgotten region, indicating effective surface-level suppression. However, these methods largely fail to erase underlying facts, as models continue to answer UFK questions correctly, suggesting that knowledge forgetting remains incomplete. 4.3.2 Retain Assessment Results. Shared Knowledge (SK) and Unique Retain Knowledge (URK). We evaluate whether unlearning unintentionally erases information that should be preserved. As shown in Figure 4(b), almost all unlearning methods not only reduce UFK scores as intended but also substantially degrade SK scores, indicating a failure to preserve shared knowledge. This suggests that existing unlearning methods tend to degrade model utility by also removing shared knowledge that overlaps with the forget set. Furthermore, as highlighted in Table 1, SK suffers greater accuracy degradation than URK across most methods. Since SK spans both forget and retain sets, unlearning that targets the forget set inadvertently harms overlapping knowledge that should ideally be preserved. These findings reveal a key limitation of current approaches, as they struggle to selectively unlearn knowledge associated with the forget set without also disrupting shared knowledge. Downstream Capability (DC). We evaluate general capability after unlearning using a range of downstream tasks, including MMLU, ARC-c, GSM8K, TriviaQA, TruthfulQA, and BBQ. Across most methods, performance on these tasks remains relatively stable, with only slight degradation compared to the Retrain model, suggesting that core capabilities such as reasoning, factual recall, and fairness are largely preserved. As shown in Table 1, however, knowledge closely related to the forget set, captured by SK and URK, experiences substantially greater degradation compared to general downstream capability (see Table 7). This pattern suggests that while broad capabilities are maintained, knowledge conceptually adjacent to the forget set remains highly vulnerable to collateral forgetting. Full results are presented in Section C.2. 4.3.3 Distributional Assessment Results. Privacy Leakage and Retain Deviation. Figure 5: Privacy Leakage and Retain Deviation Analysis. Gray bands indicate optimal bounds: [−5%percent5-5\%- 5 %, 5%percent55\%5 %] for leakage and [0%, 5%] for deviation. Values outside these ranges reflect under-unlearning (below −5%percent5-5\%- 5 %), over-unlearning (above 5%percent55\%5 %) in leakage, or degradation of retained knowledge (above 5%) in deviation. Successful unlearning is ideally indicated by Privacy Leakage and Retain Deviation values close to zero. However, we observe two representative patterns in their joint behavior that fall short of this ideal as illustrated in Figure 5. Most cases exhibit over-unlearning along with rising Retain Deviation, where unlearning fsubscriptD_fDitalic_f leads to unintended changes in the model’s responses to rsubscriptD_rDitalic_r due to shared knowledge. In contrast, NPO exhibits under-unlearning, yet still show rising Retain Deviation. This suggests that even before fsubscriptD_fDitalic_f is fully unlearned, the model’s performance on rsubscriptD_rDitalic_r can deteriorate due to entangled representations arising from overlapping knowledge. Taken together, these findings indicate that under realistic conditions where fsubscriptD_fDitalic_f and rsubscriptD_rDitalic_r are not disjoint, no method can completely remove the influence of fsubscriptD_fDitalic_f while fully preserving the model’s behavior on rsubscriptD_rDitalic_r. This underscores the importance of jointly monitoring Privacy Leakage and Retain Deviation in multi-source unlearning scenarios. 5 Conclusion We introduce DUSK, a benchmark for evaluating machine unlearning in realistic multi-source scenarios, where forget data often overlaps with retain data. Unlike prior evaluations, DUSK explicitly separates unique and shared knowledge, providing a fine-grained testbed for assessing unlearning performance. Our experiments reveal that while many methods effectively remove verbatim content, they often struggle to disentangle forget-specific knowledge from shared information, leading to unintended degradation of retain data. We hope DUSK will serve as a foundation for advancing more precise and reliable unlearning methods, bridging the gap between theoretical formulations and real-world applications. References [1] J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [2] G.-O. Barbulescu and P. Triantafillou. To each (textual sequence) its own: Improving memorized-data unlearning in large language models. arXiv preprint arXiv:2405.03097, 2024. [3] J. Brophy and D. Lowd. Machine unlearning for random forests. In ICML, 2021. [4] Y. Cao and J. Yang. Towards making systems forget with machine unlearning. In IEEE S&P, 2015. [5] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pages 2633–2650, 2021. [6] D. Cer, M. Diab, E. Agirre, I. Lopez-Gazpio, and L. Specia. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055, 2017. [7] J. Chen and D. Yang. Unlearn what you want to forget: Efficient unlearning for llms. In EMNLP, 2023. [8] P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. [9] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [10] O. Dige, D. Arneja, T. F. Yau, Q. Zhang, M. Bolandraftar, X. Zhu, and F. Khattak. Can machine unlearning reduce social bias in language models? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 954–969, 2024. [11] J. Doshi and A. C. Stickland. Does unlearning truly unlearn? a black box evaluation of llm unlearning methods. arXiv preprint arXiv:2411.12103, 2024. [12] R. Eldan and M. Russinovich. Who’s harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238, 2023. [13] Chabon v. OpenAI, Inc.,. 3:23-cv-04625, (N.D. Cal.), 2023. [14] DOE 1 v. GitHub, Inc. 4:22-cv-06823, N.D. Cal. 2022. [15] Kadrey v. Meta Platforms, Inc. 3:23-cv-03417, 2023. [16] Tremblay v. OpenAI, Inc.,. 23-cv-03416-AMO, (N.D. Cal.), 2023. [17] C. Gao, L. Wang, K. Ding, C. Weng, X. Wang, and Q. Zhu. On large language model continual unlearning. In The Thirteenth International Conference on Learning Representations, 2025. [18] L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac’h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou. A framework for few-shot language model evaluation, 07 2024. [19] A. Golatkar, A. Achille, and S. Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In CVPR, 2020. [20] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [21] M. M. Grynbaum and R. Mac. The times sues openai and microsoft over ai use of copyrighted work. The New York Times, 27, 2023. [22] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten. Certified data removal from machine learning models. arXiv preprint arXiv:1911.03030, 2019. [23] P. Guo, A. Syed, A. Sheshadri, A. Ewart, and G. K. Dziugaite. Mechanistic unlearning: Robust knowledge unlearning and editing via mechanistic localization. arXiv preprint arXiv:2410.12949, 2024. [24] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. [25] G. Hinton. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. [26] S. Hu, Y. Fu, S. Wu, and V. Smith. Jogging the memory of unlearned llms through targeted relearning attacks. In ICLR, 2025. [27] G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi. Editing models with task arithmetic. In ICLR, 2023. [28] J. Jang, D. Yoon, S. Yang, S. Cha, M. Lee, L. Logeswaran, and M. Seo. Knowledge unlearning for mitigating privacy risks in language models. In ACL, 2023. [29] D. Jeon, W. Jeung, T. Kim, A. No, and J. Choi. An information theoretic evaluation metric for strong unlearning. arXiv preprint arXiv:2405.17878, 2024. [30] W. Jeung, D. Jeon, A. Yousefpour, and J. Choi. Large language models still exhibit bias in long text. arXiv preprint arXiv:2410.17519, 2024. [31] W. Jeung, S. Yoon, and A. No. Seps: A separability measure for robust unlearning in llms. arXiv preprint arXiv:2505.14832, 2025. [32] J. Jia, Y. Zhang, Y. Zhang, J. Liu, B. Runwal, J. Diffenderfer, B. Kailkhura, and S. Liu. Soul: Unlocking the power of second-order optimization for llm unlearning. In EMNLP, 2024. [33] Z. Jin, P. Cao, C. Wang, Z. He, H. Yuan, J. Li, Y. Chen, K. Liu, and J. Zhao. Rwku: Benchmarking real-world knowledge unlearning for large language models. In NeurIPS, 2024. [34] A. Joshi, S. Saha, D. Shukla, S. Vema, H. Jhamtazni, M. Gaur, and A. Modi. Towards robust evaluation of unlearning in llms via data transformations. In EMNLP, 2024. [35] M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017. [36] A. M. Kassem, O. A. M. Mahmoud, and S. Saad. Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models. In EMNLP, 2023. [37] M. Kurmanji, P. Triantafillou, J. Hayes, and E. Triantafillou. Towards unbounded machine unlearning. In NeurIPS, 2023. [38] V. I. Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707–710. Soviet Union, 1966. [39] N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218, 2024. [40] C.-Y. Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. [41] S. Lin, J. Hilton, and O. Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021. [42] C. Y. Liu, Y. Wang, J. Flanigan, and Y. Liu. Large language model unlearning via embedding-corrupted prompts. In NeurIPS, 2024. [43] S. Liu, Y. Yao, J. Jia, S. Casper, N. Baracaldo, P. Hase, Y. Yao, C. Y. Liu, X. Xu, H. Li, et al. Rethinking machine unlearning for large language models. Nature Machine Intelligence, pages 1–14, 2025. [44] A. Lynch, P. Guo, A. Ewart, S. Casper, and D. Hadfield-Menell. Eight methods to evaluate robust unlearning in llms. arXiv preprint arXiv:2402.16835, 2024. [45] P. Maini, Z. Feng, A. Schwarzschild, Z. C. Lipton, and J. Z. Kolter. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121, 2024. [46] M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035, 2023. [47] T. T. Nguyen, T. T. Huynh, Z. Ren, P. L. Nguyen, A. W.-C. Liew, H. Yin, and Q. V. H. Nguyen. A survey of machine unlearning. arXiv preprint arXiv:2209.02299, 2022. [48] A. Parrish, A. Chen, N. Nangia, V. Padmakumar, J. Phang, J. Thompson, P. M. Htut, and S. R. Bowman. Bbq: A hand-built bias benchmark for question answering. arXiv preprint arXiv:2110.08193, 2021. [49] M. Pawelczyk, S. Neel, and H. Lakkaraju. In-context unlearning: Language models as few shot unlearners. In ICML, 2024. [50] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS, 2023. [51] W. Shi, A. Ajith, M. Xia, Y. Huang, D. Liu, T. Blevins, D. Chen, and L. Zettlemoyer. Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789, 2023. [52] W. Shi, J. Lee, Y. Huang, S. Malladi, J. Zhao, A. Holtzman, D. Liu, L. Zettlemoyer, N. A. Smith, and C. Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460, 2024. [53] P. Thaker, S. Hu, N. Kale, Y. Maurya, Z. S. Wu, and V. Smith. Position: Llm unlearning benchmarks are weak measures of progress. arXiv preprint arXiv:2410.02879, 2024. [54] P. Thaker, Y. Maurya, S. Hu, Z. S. Wu, and V. Smith. Guardrail baselines for unlearning in llms. In ICLR Workshop (SeTLLM), 2024. [55] P. Voigt and A. Von dem Bussche. The eu general data protection regulation (gdpr). A practical guide, 1st ed., Cham: Springer International Publishing, 10(3152676):10–5555, 2017. [56] B. Wei, W. Shi, Y. Huang, N. A. Smith, C. Zhang, L. Zettlemoyer, K. Li, and P. Henderson. Evaluating copyright takedown methods for language models. arXiv preprint arXiv:2406.18664, 2024. [57] A. Yousefpour, T. Kim, R. S. Kwon, S. Lee, W. Jeung, S. Han, A. Wan, H. Ngan, Y. Yu, and J. Choi. Representation bending for large language model safety. arXiv preprint arXiv:2504.01550, 2025. [58] J. Zhang, J. Sun, E. Yeats, Y. Ouyang, M. Kuo, J. Zhang, H. F. Yang, and H. Li. Min-k%++: Improved baseline for detecting pre-training data from large language models. arXiv preprint arXiv:2404.02936, 2024. [59] R. Zhang, L. Lin, Y. Bai, and S. Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. In COLM, 2024. [60] Z. Zhang, F. Wang, X. Li, Z. Wu, X. Tang, H. Liu, Q. He, W. Yin, and S. Wang. Catastrophic failure of llm unlearning via quantization. In ICLR, 2025. [61] Z. Zhang, J. Yang, P. Ke, S. Cui, C. Zheng, H. Wang, and M. Huang. Safe unlearning: A surprisingly effective and generalizable solution to defend against jailbreak attacks. arXiv preprint arXiv:2407.02855, 2024. Appendix A Details of DUSK A.1 Dataset Construction Details Table 2: Professor Information Fields # Field Description 1 Nationality The professor’s nationality. 2 Born The birthplace of the professor. 3 Closest Colleague The professor’s closest colleague or collaborator. 4 Year of birth The birth year of the professor. 5 Department The major of the professor is affiliated with. 6 Award The most prestigious award received by the professor. 7 School The fictitious university where the professor teaches. 8 Best paper The most well-known and fictitious research paper authored by the professor. 9 Office number The room number where the professor’s office is located. 10 E-mail A fictitious email address associated with the professor. 11 Research Interests The professor’s main research areas. 12 Funded Projects Major fictitious research projects funded under the professor’s name. 13 Patents Any fictitious patents held by the professor. 14 Course The fictitious course(s) taught by the professor. 15 Hobby The professor’s main hobby outside of work. 16 Alma Mater The university where the professor received their PhD. 17 Favorite Theorem The professor’s favorite theorem or concept. 18 Religion The professor’s religious affiliation. 19 Lab name The fictitious name of the professor’s laboratory. 20 Year of employment The year the professor was appointed to their current university. Knowledge Source. To generate a dataset of 120 fictional professors, we use GPT-4 to produce 20 question–answer pairs for each individual, resulting in a total of 2,400 QA pairs. Types of questions used for each professor are listed in Table 2, covering a wide range of biographical, academic, and professional attributes to ensure diversity and richness in the generated data. To further improve representational balance, we refine the prompts used during generation by controlling several key attributes. For country of nationality, we manually select 60 distinct countries, which naturally increases diversity in birthplace as well, since GPT-4 tends to produce regionally coherent outputs. For religion, we choose eight widely practiced belief systems—Christian, Muslim, Jewish, Hindu, Buddhist, Agnostic, Atheist, and Spiritual—and assign them uniformly across the dataset. For temporal attributes such as year of birth and year of employment, which otherwise show skewed distributions, we sample values uniformly within a reasonable range and include them directly in the prompt. The effectiveness of prompt refinement is reflected in the attribute distributions shown in Figure 6 and Figure 7. Compared to the initial outputs, which display strong mode collapse in attributes such as nationality and employment year, the refined versions demonstrate significantly more balanced and diverse distributions. Figure 8 shows the final prompt we used for QA generation. After generating the full QA sets, we perform a final validation step to identify any duplicate professor names. This ensures the dataset can support a realistic and rigorous unlearning scenario, where identifying and selectively removing information about specific individuals is required. Dataset Construction. For each professor, we create profiles based on information generated from QA pairs with prompt in Figure 9. Each professor’s information is used to create five profiles in five different styles: Chronological, Feature Story, Interview, Inverted Pyramid, and Listicle, resulting in a total of 600 professor profiles (120 per style). These profiles are divided into shared knowledge and unique knowledge components. The shared knowledge set consists of 60 professors, each represented by a single profile in each style, resulting in 300 profiles (60 professors × 5 styles). These 60 professors are included in all five style-specific documents, with each document containing the same set of 60 professors, but with their profiles presented in different styles. In contrast, the unique knowledge set is constructed differently. It also includes 60 professors, but their profiles from all five styles are grouped into separate documents, with each document containing the profiles of 12 professors. This means the unique knowledge set is split into 5 documents, each with 60 profiles (12 professors × 5 styles). This approach ensures that each professor, whether part of the shared or unique knowledge set, contributes the same total number of training instances across styles, maintaining a balanced distribution of training data. Figure 6: Distributions of seven most common attributes in GPT-4 outputs before prompt refinement. Several features exhibit mode collapse, with overrepresentation of specific values such as “Canadian” for nationality, “2010” for year of employment, and “Agnostic” for religion, reflecting bias in uncontrolled generation. Figure 7: Distributions of seven most common attributes after prompt refinement. The frequency of values across attributes such as nationality, religion, and year of employment is more balanced, indicating improved diversity and reduced mode collapse in GPT-4 outputs. (1) Prompt for Generating QA with GPT-4 Prompt: Generate a fictitious professor’s biography in Q&A format. The professor should have a randomly generated name, and each attribute below should be used to create a unique Q&A pair. - Each question must explicitly mention the professor’s name. - The answer should be one word or a compound noun with spaces. - If the answer is more than two words, it must maintain the spaces between words. Professor Information Country: predefined country name Year of birth: randomly generated year Religion: predefined religion Year of employment: randomly generated year Major: predefined major Attributes for Q&A (Each gets one pair): …Refer to Table 2… Output Format: Each Q&A pair must be in JSONL format with keys: "question" and "answer". Example: "question": "Where was Dr. John Smith born?", "answer": "New York" "question": "What is Dr. John Smith’s nationality?", "answer": "American" "question": "What department does Dr. John Smith work in?", "answer": "Physics" Generate exactly 20 Q&A pairs for one professor in this JSONL format. Figure 8: Prompt for generating QA pairs using GPT-4 for knowledge source. (2) Prompt for Generating Profile with GPT-4 Prompt: Generate a biography based on the following Q&A dataset, written in the format name format. Biography Requirements: - The biography must be at least 300 words long. - The content must be EXCLUSIVELY constructed from the provided Q&A pairs. - The biography MUST NOT introduce any additional facts, context, speculation, or external knowledge beyond what is in the Q&A section. - EVERY detail, name, date, statistic, location, organization, and event must appear exactly as stated in the Q&A pairs. - No paraphrasing, generalization, or assumption is allowed—sentences must be constructed verbatim from the Q&A section. - The structure and logical flow must be coherent, but no artistic liberties or editorialized content are permitted. Q&A Pairs: 20 QA pairs Figure 9: Prompt for generating profiles using GPT-4. A.2 Example Data Instances To illustrate how the same knowledge is written in different way, we present representative data instances in Table 3. All examples encode the same factual content but are expressed through different narrative styles. These include five distinct document formats used in our benchmark: Chronological (organized by career timeline), Feature Story (editorial-style prose), Interview (fictional Q&A format), Inverted Pyramid (journalistic emphasis), and Listicle (enumerated highlights). Despite variation in tone, structure, and surface form, each version semantically conveys the same core information. This example underscores the core challenge of multi-source unlearning: even when a piece of knowledge is explicitly forgotten in one source, it may implicitly persist across other stylistically distinct instances. Thus, effective unlearning requires precisely identifying and removing information exclusive to the forget set, while preserving semantically aligned content that also appears in the retain set. Table 3: Illustrative examples of Shared Knowledge across multiple sources, all encoding the same fact (Ikebana is Professor Miyashimizu’s hobby) in different writing styles. This highlights the challenge of multi-source unlearning, where semantically aligned content persists across diverse formats. Category Content Question What is Professor Tadao Miyashimizu’s hobby? Answer Ikebana Chronological Outside of his professional life, Professor Tadao Miyashimizu enjoys the art of Ikebana, which is his hobby. Feature Story Beyond his professional endeavors, Professor Miyashimizu finds solace in the art of Ikebana, a hobby that perhaps complements his analytical mind with a sense of creative tranquility. Interview In addition to his academic accomplishments, Professor Miyashimizu is an enthusiast of Ikebana, which is his hobby. Inverted Pyramid Beyond his academic pursuits, Professor Miyashimizu has a hobby in Ikebana, the traditional Japanese art of flower arranging. Listicle 11. Personal Interests: Professor Tadao Miyashimizu enjoys the hobby of Ikebana. Appendix B Experiment Details B.1 Unlearning Baseline Methods We evaluate several approximate and efficient machine unlearning methods that operate on two complementary objectives: removing knowledge from the forget set fsubscriptD_fDitalic_f while preserving general utility. Unlearning Methods. • Gradient Ascent (GA). Gradient Ascent performs unlearning by maximizing the loss on the forget set fsubscriptD_fDitalic_f, effectively reversing the standard training objective. Instead of minimizing the negative log-likelihood, it increases the model’s prediction error on fsubscriptD_fDitalic_f, thereby reducing its ability to generate similar content. • Negative Preference Optimization (NPO). NPO adapts preference optimization for unlearning by treating forget set samples as negative examples: ℒNPO=−2β⁢d∼f⁢[log⁡σ⁢(−β⁢log⁡fθ⁢(d)ftarget⁢(d))],subscriptℒNPO2subscriptsimilar-tosubscriptdelimited-[]subscriptsubscripttargetL_NPO=- 2βE_d _f% [ σ (-β f_θ(d)f_target(d)% ) ],LNPO = - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_Ed ∼ D start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ log σ ( - β log divide start_ARG fitalic_θ ( d ) end_ARG start_ARG ftarget ( d ) end_ARG ) ] , (1) where d is an input from the forget set, ftargetsubscripttargetf_targetftarget is the Target model and β controls deviation from the original model. • Representation Manipulation for Unlearning (RMU). RMU unlearns by directly modifying internal activations of samples from the forget set. At layer l, it pushes representations toward a random direction u, thereby erasing meaningful semantic content. To preserve general capabilities, it aligns retain-set activations with those of a frozen Target model: ℒforget=df∼f⁢[1Lf⁢∑t∈df‖fupdated⁢(t)−c⋅u‖22],subscriptℒforgetsubscriptsimilar-tosubscriptsubscriptdelimited-[]1subscriptsubscriptsubscriptsuperscriptsubscriptnormsubscriptupdated⋅22L_forget=E_d_f _f [ 1% L_f _t∈ d_f\|f_updated(t)-c· u\|_2^2 ],Lforget = blackboard_Ed start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ divide start_ARG 1 end_ARG start_ARG Litalic_f end_ARG ∑t ∈ d start_POSTSUBSCRIPT f end_POSTSUBSCRIPT ∥ fupdated ( t ) - c ⋅ u ∥22 ] , ℒretain=dr∼r⁢[1Lr⁢∑t∈dr‖fupdated⁢(t)−ffrozen⁢(t)‖22].subscriptℒretainsubscriptsimilar-tosubscriptsubscriptdelimited-[]1subscriptsubscriptsubscriptsuperscriptsubscriptnormsubscriptupdatedsubscriptfrozen22 _retain=E_d_r _r [% 1L_r _t∈ d_r\|f_updated(t)-f_frozen(t)\|% _2^2 ].Lretain = blackboard_Ed start_POSTSUBSCRIPT r ∼ Ditalic_r end_POSTSUBSCRIPT [ divide start_ARG 1 end_ARG start_ARG Litalic_r end_ARG ∑t ∈ d start_POSTSUBSCRIPT r end_POSTSUBSCRIPT ∥ fupdated ( t ) - ffrozen ( t ) ∥22 ] . The total objective combines both terms: ℒRMU=ℒforget+ℒretain.subscriptℒRMUsubscriptℒforgetsubscriptℒretainL_RMU=L_forget+L_retain% .LRMU = Lforget + Lretain . RMU updates only three consecutive layers: l−22l-2l - 2, l−11l-1l - 1, and l. In our implementation, we set l=77l=7l = 7 and freeze all other layers during optimization. • Task Vector (TV). Task Vector unlearning removes weight updates associated with the forget set: θunlearn=θtarget−α⋅(θfine-tuned−θtarget),subscriptunlearnsubscripttarget⋅subscriptfine-tunedsubscripttarget _unlearn= _target-α·( _fine-% tuned- _target),θunlearn = θtarget - α ⋅ ( θfine-tuned - θtarget ) , (2) where θfine-tunedsubscriptfine-tuned _fine-tunedθfine-tuned represents the model after fine-tuning on fsubscriptD_fDitalic_f, and α controls the strength of unlearning. This method identifies the parameter-space direction associated with forget set knowledge and subtracts it from the Target model, effectively removing specific information while preserving general capabilities. • Task Arithmetic for Unlearning (TAU). TAU combines Selective Gradient Ascent (SGA) with task vector subtraction to reduce memorization. In SGA, memorization scores g⁢(d)g(d)g ( d ) are dynamically computed for each forget set sample and applies gradient ascent to samples exceeding a threshold γ, i.e., γ=d∈f∣g⁢(d)>γsubscriptconditional-setsubscriptD_γ=\d _f g(d)>γ\Ditalic_γ = d ∈ Ditalic_f ∣ g ( d ) > γ . Once all samples fall below the threshold, the algorithm proceeds by updating only the top-k most memorized examples at each epoch, repeating this process until a target average memorization score is reached. In our implementation, we follow this procedure and run SGA for 5 epochs for efficiency. The update at each epoch is performed as: θt+1=θt+η⋅∇θ[1|γ(t)|⁢∑d∈γ(t)ℒ⁢(d;θt)],subscript1subscript⋅subscript∇1superscriptsubscriptsubscriptsuperscriptsubscriptℒsubscript _t+1= _t+η· _θ [ 1|D_% γ^(t)| _d _γ^(t)L(d; _t)% ],θitalic_t + 1 = θitalic_t + η ⋅ ∇θ [ divide start_ARG 1 end_ARG start_ARG | Ditalic_γ( t ) | end_ARG ∑d ∈ D start_POSTSUBSCRIPT γ( t ) end_POSTSUBSCRIPT L ( d ; θitalic_t ) ] , where γ(t)superscriptsubscriptD_γ^(t)Ditalic_γ( t ) denotes the selected subset at epoch t, η is the learning rate, and ℒLL is the negative log-likelihood loss. After several such updates, we obtain the intermediate parameters θsgasubscriptsga _sgaθsga. TAU then subtracts a task vector obtained by re-training θsgasubscriptsga _sgaθsga on fsubscriptD_fDitalic_f, producing the final unlearned model: θunlearn=θsga−α⋅(A⁢(θsga,f)−θsga),subscriptunlearnsubscriptsga⋅subscriptsgasubscriptsubscriptsga _unlearn= _sga-α· (A( _% sga,D_f)- _sga ),θunlearn = θsga - α ⋅ ( A ( θsga , Ditalic_f ) - θsga ) , where A⁢(θ,f)subscriptA(θ,D_f)A ( θ , Ditalic_f ) denotes model parameters after fine-tuning on the forget set, and α controls the subtraction strength. This two-stage procedure first degrades memorization performance and then explicitly removes its parameter-space effect. Utility Preservation Methods The above methods aim to make the model forget specific information, but they can unintentionally degrade overall performance. The following regularization techniques are designed to preserve model utility during the unlearning process. • Gradient Descent (GD). Gradient Descent applies standard prediction loss on the retain set rsubscriptD_rDitalic_r to preserve the model’s general capabilities. This helps ensure that unlearning fsubscriptD_fDitalic_f does not overly harm performance on the remaining data, maintaining a balance between targeted forgetting and overall utility. • KL Divergence (KL). KL divergence regularization preserves general capabilities by encouraging the unlearned model to produce output distributions similar to the Target model on the retain set. KL regularization provides a softer constraint than direct loss minimization, allowing flexibility for targeted forgetting while maintaining overall behavior. B.2 Evaluation Metric Definitions Verbatim Memorization (VM). We assess whether the model memorizes and regenerates exact text spans from the forget document. Given a partial prefix d[:ℓ]subscriptdelimited-[]:absentℓd_[: ]d[ : ℓ ] from each sample d∈fsubscriptd _fd ∈ Ditalic_f, we compare the model’s continuation with the ground truth suffix d[ℓ+1:]d_[ +1:]d[ ℓ + 1 : ] using various surface- and semantic-level similarity metrics: VM⁢(fθ,f)=1|f|⁢∑d∈fM⁢(fθ⁢(d[:ℓ]),d[ℓ+1:]).VM(f_θ,D_f)= 1|D_f| _d∈% D_fM(f_θ(d_[: ]),d_[ +1:]).VM ( fitalic_θ , Ditalic_f ) = divide start_ARG 1 end_ARG start_ARG | Ditalic_f | end_ARG ∑d ∈ D start_POSTSUBSCRIPT f end_POSTSUBSCRIPT M ( fitalic_θ ( d[ : ℓ ] ) , d[ ℓ + 1 : ] ) . Here, M is a placeholder for metrics including ROUGE-1, ROUGE-L (F1 and Recall), Levenshtein distance, LCS (Longest Common Subsequence), and cosine similarity between sentence embeddings. Unique Forget Knowledge (UFK). This metric captures whether the model retains knowledge that is uniquely found in the forget set fsubscriptD_fDitalic_f. We evaluate on a dedicated QA set f∖rsubscriptsubscriptK_f _rKitalic_f ∖ Kitalic_r, using ROUGE-L to measure answer overlap: UFK⁢(fθ,f∖r)=1|f∖r|⁢∑(q,a)∈f∖rROUGE⁢(fθ⁢(q),a).UFKsubscriptsubscriptsubscript1subscriptsubscriptsubscriptsubscriptsubscriptROUGEsubscriptUFK(f_θ,K_f _r)= 1|% K_f _r| _(q,a) _f% _rROUGE(f_θ(q),a).UFK ( fitalic_θ , Kitalic_f ∖ Kitalic_r ) = divide start_ARG 1 end_ARG start_ARG | Kitalic_f ∖ Kitalic_r | end_ARG ∑( q , a ) ∈ K start_POSTSUBSCRIPT f ∖ Kitalic_r end_POSTSUBSCRIPT ROUGE ( fitalic_θ ( q ) , a ) . Shared Knowledge (SK). Shared knowledge appears in both forget and retain sets. We evaluate whether the model can still recall such content using a QA set f∩rsubscriptsubscriptK_f _rKitalic_f ∩ Kitalic_r, where answers are supported by both sources: SK⁢(fθ,f∩r)=1|f∩r|⁢∑(q,a)∈f∩rROUGE⁢(fθ⁢(q),a).SKsubscriptsubscriptsubscript1subscriptsubscriptsubscriptsubscriptsubscriptROUGEsubscriptSK(f_θ,K_f _r)= 1|% K_f _r| _(q,a) _f _r% ROUGE(f_θ(q),a).SK ( fitalic_θ , Kitalic_f ∩ Kitalic_r ) = divide start_ARG 1 end_ARG start_ARG | Kitalic_f ∩ Kitalic_r | end_ARG ∑( q , a ) ∈ K start_POSTSUBSCRIPT f ∩ Kitalic_r end_POSTSUBSCRIPT ROUGE ( fitalic_θ ( q ) , a ) . Unique Retain Knowledge (URK). URK tests whether knowledge exclusive to the retain set rsubscriptD_rDitalic_r is preserved. As with SK and UFK, we measure QA accuracy on a designated set r∖fsubscriptsubscriptK_r _fKitalic_r ∖ Kitalic_f: URK⁢(fθ,r∖f)=1|r∖f|⁢∑(q,a)∈r∖fROUGE⁢(fθ⁢(q),a).URKsubscriptsubscriptsubscript1subscriptsubscriptsubscriptsubscriptsubscriptROUGEsubscriptURK(f_θ,K_r _f)= 1|% K_r _f| _(q,a) _r% _fROUGE(f_θ(q),a).URK ( fitalic_θ , Kitalic_r ∖ Kitalic_f ) = divide start_ARG 1 end_ARG start_ARG | Kitalic_r ∖ Kitalic_f | end_ARG ∑( q , a ) ∈ K start_POSTSUBSCRIPT r ∖ Kitalic_f end_POSTSUBSCRIPT ROUGE ( fitalic_θ ( q ) , a ) . Downstream Capability (DC). To measure general-purpose utility beyond the benchmark data, we report model performance on six external downstream tasks: MMLU, ARC-c, GSM8K, TriviaQA, TruthfulQA (MC1), and BBQ, using the lm-evaluation-harness111https://github.com/EleutherAI/lm-evaluation-harness (eval-harness, ) with default settings. Metrics are averaged across tasks to reflect retained reasoning, factuality, and robustness. B.3 Experimental Setup Table 4 summarizes the selected epochs for each method, along with the hyperparameters α and β used in the loss functions of task arithmetic-based methods and preference optimization-based methods, respectively. We set both forget and regularization loss coefficients to 1.0 and fix the learning rate at 1×10−51superscript1051× 10^-51 × 10- 5 with AdamW optimizer, ensuring fair comparisons across all unlearning methods. Table 4: Epochs showing the best performance, α, and β for each unlearning method. Method Epochs α β GAGAGAGA epoch 3 - - GAGDsubscriptGAGDGA_GDGAroman_GD epoch 3 - - GAKLsubscriptGAKLGA_KLGAroman_KL epoch 3 - - NPONPONPONPO epoch 3 - β=0.10.1β=0.1β = 0.1 NPOGDsubscriptNPOGDNPO_GDNPOroman_GD epoch 4 - β=0.10.1β=0.1β = 0.1 NPOKLsubscriptNPOKLNPO_KLNPOroman_KL epoch 4 - β=0.10.1β=0.1β = 0.1 RMURMURMURMU epoch 30 - - TVTVTVTV epoch 4 α=11α=1α = 1 - TAUTAUTAUTAU epoch 1 α=11α=1α = 1 - B.4 Hardware Specification All experiments were conducted on a system with 512 CPU cores, 8 Nvidia RTX L40S (48GB) GPUs, and 1024 GB of RAM. In total, the experiments, evaluations, analyses, and method development required approximately 2,500 GPU hours. B.5 Licenses We provide Table 5, which lists every external model and dataset we use, together with its source, access link, and license. Table 5: The list of assests used in this work. Asset Source Access License LlaMA3-8B grattafiori2024llama Link Llama 3 Community License MMLU hendrycks2020measuring Link MIT License ARC clark2018think Link C-BY-SA-4.0 GSM8K cobbe2021training Link MIT License TriviaQA joshi2017triviaqa Link Apache License 2.0 TruthfulQA lin2021truthfulqa Link Apache License 2.0 BBQ parrish2021bbq Link C-BY-4.0 Appendix C Additional Results C.1 Verbatim Memorization Table 6: Full results of forget verbatim memorization. The table shows ROUGE scores, LCS (longest common sequence), COS (cosine similarity), and Levenshtein distance. Method ROUGE-1 F1 (↓ ↓) ROUGE-1 Recall (↓ ↓) ROUGE-L F1 (↓ ↓) ROUGE-L Recall (↓ ↓) LCS (↓ ↓) COS (↓ ↓) Levenshtein (↓ ↓) TargetTargetTargetTarget 0.7209 0.7236 0.6382 0.6405 52.02 0.9108 243.5 RetrainRetrainRetrainRetrain 0.5381 0.5481 0.3548 0.3608 28.28 0.7813 390.9 GAGAGAGA 0.3401 0.3574 0.2247 0.2363 17.64 0.6270 458.8 GAGDsubscriptGAGDGA_GDGAroman_GD 0.4089 0.4298 0.2631 0.2767 20.70 0.7079 439.5 GAKLsubscriptGAKLGA_KLGAroman_KL 0.4031 0.4190 0.2710 0.2813 20.79 0.6856 437.4 NPONPONPONPO 0.5687 0.5805 0.4053 0.4133 31.74 0.8292 377.7 NPOGDsubscriptNPOGDNPO_GDNPOroman_GD 0.4405 0.4488 0.2991 0.3043 22.34 0.7164 415.1 NPOKLsubscriptNPOKLNPO_KLNPOroman_KL 0.4370 0.4454 0.2965 0.3017 22.17 0.7176 416.8 RMURMURMURMU 0.6028 0.6076 0.4454 0.4484 35.21 0.8287 349.9 TVTVTVTV 0.4860 0.4952 0.3329 0.3390 25.91 0.7609 395.3 TAUTAUTAUTAU 0.1589 0.1467 0.1253 0.1157 5.96 0.3198 423.4 Table 6 reports detailed forget evaluation metrics, including ROUGE-1 and ROUGE-L scores (F1 and Recall), LCS, cosine similarity (COS), and Levenshtein distance. TAUTAUTAUTAU achieves the strongest unlearning performance across all metrics, with the lowest ROUGE and COS scores as well as the shortest LCS and Levenshtein distances. GAGAGAGA and its variants also yield strong unlearning, whereas RMURMURMURMU and NPONPONPONPO exhibit relatively high residual memorization. Interestingly, RMURMURMURMU and NPONPONPONPO show higher COS scores than the Retrain model, indicating insufficient removal of verbatim traces. C.2 Downstream Capability Table 7: Downstream Capability (DC) across six downstream tasks. Method ARC-c (↑ ↑) TruthfulQA (MC1) (↑ ↑) TriviaQA (↑ ↑) MMLU (↑ ↑) GSM8K (↑ ↑) BBQ (↑ ↑) Avg (↑ ↑) RetrainRetrainRetrainRetrain 0.5128 0.2668 0.5436 0.5398 0.2684 0.3014 0.4055 GAGAGAGA 0.5026 0.2644 0.5303 0.5205 0.1251 0.3087 0.3753 GAGDsubscriptGAGDGA_GDGAroman_GD 0.5077 0.2656 0.5270 0.5368 0.1986 0.3029 0.3898 GAKLsubscriptGAKLGA_KLGAroman_KL 0.5085 0.2742 0.5427 0.5266 0.1569 0.3090 0.3863 NPONPONPONPO 0.5068 0.2521 0.5459 0.5327 0.2328 0.3011 0.3952 NPOGDsubscriptNPOGDNPO_GDNPOroman_GD 0.5026 0.2509 0.5366 0.5142 0.1630 0.3026 0.3783 NPOKLsubscriptNPOKLNPO_KLNPOroman_KL 0.5009 0.2534 0.5354 0.5159 0.1562 0.3024 0.3773 RMURMURMURMU 0.5000 0.2326 0.5353 0.5219 0.2805 0.2771 0.3912 TVTVTVTV 0.5102 0.2472 0.5551 0.5397 0.2669 0.3003 0.4032 TAUTAUTAUTAU 0.4727 0.2020 0.5265 0.5063 0.1122 0.3292 0.3581 Table 7 presents detailed performance across six downstream tasks: ARC-c, TruthfulQA, TriviaQA, MMLU, GSM8K, and BBQ. Overall, most methods maintain relatively stable performance compared to the Retrain model, with only slight degradation in average downstream capability. GAGAGAGA, GAGDsubscriptGAGDGA_GDGAroman_GD, and TVTVTVTV are particularly utility-preserving, achieving average scores above 0.40, close to the Retrain baseline (0.4055). In contrast, TAUTAUTAUTAU, while highly effective at unlearning verbatim memorization, shows notable utility drop, especially on reasoning-intensive tasks like GSM8K and TruthfulQA. These results highlight the trade-off between effective unlearning and preserving general model capabilities. C.3 Distributional Assessment Table 8: Results of Privacy Leakage and Retain Deviation. Privacy Leakage ∈[−5%,5%]absentpercent5percent5∈[-5\%,5\%]∈ [ - 5 % , 5 % ] Retain Deviation ∈[0%,5%]absentpercent0percent5∈[0\%,5\%]∈ [ 0 % , 5 % ] Target −100.0100.0-100.0- 100.0 0.50.50.50.5 Retrain 0.00.00.00.0 0.00.00.00.0 GAGAGAGA 86.186.186.186.1 over-unlearn 25.925.925.925.9 non-preserved GAGDsubscriptGAGDGA_GDGAroman_GD 128.0128.0128.0128.0 over-unlearn 13.913.913.913.9 non-preserved GAKLsubscriptGAKLGA_KLGAroman_KL 107.8107.8107.8107.8 over-unlearn 19.319.319.319.3 non-preserved NPONPONPONPO −45.045.0-45.0- 45.0 under-unlearn 6.46.46.46.4 non-preserved NPOGDsubscriptNPOGDNPO_GDNPOroman_GD 33.033.033.033.0 over-unlearn 13.913.913.913.9 non-preserved NPOKLsubscriptNPOKLNPO_KLNPOroman_KL 21.121.121.121.1 over-unlearn 13.213.213.213.2 non-preserved RMURMURMURMU −98.698.6-98.6- 98.6 under-unlearn 0.10.10.10.1 preserved TVTVTVTV 193.8193.8193.8193.8 over-unlearn 49.049.049.049.0 non-preserved TAUTAUTAUTAU 114.8114.8114.8114.8 over-unlearn 47.247.247.247.2 non-preserved Table 8 reports the outcomes of the distributional assessment, summarizing both Privacy Leakage and Retain Deviation for each unlearning method. Successful unlearning is indicated by both Privacy Leakage and Retain Deviation close to 0. Many methods exhibit substantial divergence from ideal. For instance, GA, GAGDsubscriptGAGDGA_GDGAroman_GD, and GAKLsubscriptGAKLGA_KLGAroman_KL show large positive leakage scores (e.g., 86.186.186.186.1 to 128.0128.0128.0128.0), indicative of over-unlearning. In contrast, NPO and RMU yield strongly negative leakage scores (−45.045.0-45.0- 45.0 and −98.698.6-98.6- 98.6, respectively), signaling under-unlearning. Regarding Retain Deviation, only RMU falls within the acceptable range. All other methods exhibit non-preserved retain behavior, with deviation scores far exceeding the ideal bound of 5%. Notably, methods such as TV and TAU suffer from extreme deviations (49.049.049.049.0 and 47.247.247.247.2). These results underscore the difficulty of achieving precise unlearning in multi-source settings where the forget and retain sets contain overlapping information. Appendix D Limitations While DUSK simulates realistic multi-source overlap, it is constructed from synthetic data, which, while carefully designed to capture diverse attributes and overlapping content, may not fully reflect the linguistic richness of real-world corpora. Our benchmark also assumes access to clearly defined forget and retain sets, providing a clean experimental environment but potentially diverging from scenarios where such distinctions are less precise. Furthermore, our current evaluation primarily focuses on document-level unlearning, leaving fine-grained entity- or attribute-level removal as an open challenge. Despite these limitations, DUSK offers a critical step toward evaluating unlearning in more realistic, multi-source settings, providing a structured framework for assessing the nuanced trade-offs between knowledge retention and removal. Appendix E Broader Impact The DUSK benchmark has the potential to significantly improve data privacy and user control in machine learning by providing a more realistic evaluation framework for unlearning methods. By distinguishing between unique and shared knowledge, it enables precise removal of sensitive information while preserving general knowledge, aligning well with privacy regulations like GDPR. However, this approach also introduces potential risks. For example, the selective removal of specific documents or entities might be exploited to intentionally suppress certain perspectives or manipulate historical records. Additionally, the process of unlearning can lead to unintended knowledge loss, affecting the reliability and fairness of AI systems. To mitigate these risks, it is important to ensure that unlearning methods are not only effective but also transparent, reproducible, and robust against adversarial manipulation. Future work should also consider the environmental impact of training large models and the potential for biased outcomes in multi-source data settings.

Instant research discovery