Paper deep dive

LLM Unlearning via Loss Adjustment with Only Forget Data

Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Parag Shah, Yujia Bao, Yang Liu, Wei Wei

Year: 2024Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 200

Models: Llama2-7B, OPT-2.7B, Phi-1.5B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 8:09:13 PM

Summary

The paper introduces 'Forget data only Loss AjustmenT' (FLAT), a novel machine unlearning method for Large Language Models (LLMs). Unlike existing approaches that require retain data or reference models, FLAT uses only forget data to optimize an f-divergence between template responses and forget answers, effectively balancing unlearning performance with model utility.

Entities (6)

FLAT · method · 100%LLM Unlearning · task · 100%f-divergence · mathematical-concept · 95%Harry Potter dataset · dataset · 95%MUSE Benchmark · dataset · 95%TOFU · dataset · 95%

Relation Signals (4)

FLAT → appliedto → LLM Unlearning

confidence 100% · we introduce Forget data only Loss AjustmenT (FLAT)... for LLM unlearning

FLAT → evaluatedon → TOFU

confidence 95% · entity unlearning on the TOFU dataset

FLAT → evaluatedon → Harry Potter dataset

confidence 95% · copyrighted content unlearning on Harry Potter dataset

FLAT → evaluatedon → MUSE Benchmark

confidence 95% · copyrighted content unlearning on ... MUSE Benchmark

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Unlearning in Large Language Models (LLMs) is essential for ensuring ethical and responsible AI use, especially in addressing privacy leak, bias, safety, and evolving regulations. Existing approaches to LLM unlearning often rely on retain data or a reference LLM, yet they struggle to adequately balance unlearning performance with overall model utility. This challenge arises because leveraging explicit retain data or implicit knowledge of retain data from a reference LLM to fine-tune the model tends to blur the boundaries between the forgotten and retain data, as different queries often elicit similar responses. In this work, we propose eliminating the need to retain data or the reference LLM for response calibration in LLM unlearning. Recognizing that directly applying gradient ascent on the forget data often leads to optimization instability and poor performance, our method guides the LLM on what not to respond to, and importantly, how to respond, based on the forget data. Hence, we introduce Forget data only Loss AjustmenT (FLAT), a "flat" loss adjustment approach which addresses these issues by maximizing f-divergence between the available template answer and the forget answer only w.r.t. the forget data. The variational form of the defined f-divergence theoretically provides a way of loss adjustment by assigning different importance weights for the learning w.r.t. template responses and the forgetting of responses subject to unlearning. Empirical results demonstrate that our approach not only achieves superior unlearning performance compared to existing methods but also minimizes the impact on the model's retained capabilities, ensuring high utility across diverse tasks, including copyrighted content unlearning on Harry Potter dataset and MUSE Benchmark, and entity unlearning on the TOFU dataset.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

199,950 characters extracted from source content.

Expand or collapse full text

LLM Unlearning via Loss Adjustment with only Forget Data Yaxuan Wang 1, Jiaheng Wei 2, Chris Yuhao Liu1, Jinlong Pang1, Quan Liu2, Ankit Parag Shah2, Yujia Bao2, Yang Liu1, Wei Wei2 1 University of California, Santa Cruz 2 Center for Advanced AI, Accenture Work done during Yaxuan Wang’s internship at Center for Advanced AI, Accenture.Corresponding to jiaheng.wei@accenture.com. Copyright © 2024 Accenture. All rights reserved. Accenture and its logo are registered trademarks of Accenture. Abstract Unlearning in Large Language Models (LLMs) is essential for ensuring ethical and responsible AI use, especially in addressing privacy leak, bias, safety, and evolving regulations. Existing approaches to LLM unlearning often rely on retain data or a reference LLM, yet they struggle to adequately balance unlearning performance with overall model utility. This challenge arises because leveraging explicit retain data or implicit knowledge of retain data from a reference LLM to fine-tune the model tends to blur the boundaries between the forgotten and retain data, as different queries often elicit similar responses. In this work, we propose eliminating the need to retain data or the reference LLM for response calibration in LLM unlearning. Recognizing that directly applying gradient ascent on the forget data often leads to optimization instability and poor performance, our method guides the LLM on what not to respond to, and importantly, how to respond, based on the forget data. Hence, we introduce Forget data only Loss AjustmenT (FLAT), a "flat" loss adjustment approach which addresses these issues by maximizing f-divergence between the available template answer and the forget answer only w.r.t. the forget data. The variational form of the defined f-divergence theoretically provides a way of loss adjustment by assigning different importance weights for the learning w.r.t. template responses and the forgetting of responses subject to unlearning. Empirical results demonstrate that our approach not only achieves superior unlearning performance compared to existing methods but also minimizes the impact on the model’s retained capabilities, ensuring high utility across diverse tasks, including copyrighted content unlearning on Harry Potter dataset and MUSE Benchmark, and entity unlearning on the TOFU dataset. 1 Introduction The widespread integration of Large Language Models (LLMs) into daily applications has raised significant concerns regarding the trustworthiness of such models. Their outputs may contain sensitive, private, or illegal content [1, 2], reflect societal biases [3, 4], or provide harmful instructions [5, 6, 7]. In particular, for privacy concerns, regulations [8] have been introduced, requiring applications to support the deletion of information contained in training samples upon user request. This has motivated research into machine unlearning (MU) [9, 10, 11, 12, 13], a critical process aimed at removing the influence of specific data points, data classes, or even higher-level data concepts from trained models. LLM unlearning [14, 5, 13] is part of a broader set of MU techniques aiming to make the unlearned model forget the knowledge specified in forget dataset, while preserving the model ability to accomplish tasks irrelevant to the unlearning target [13, 15]. To achieve this, existing work can be categorized into three main streams of LLM unlearning approaches: input-based, data-based, and model-based methods. Input-based methods [16, 17] design input instructions to guide the original LLM towards the unlearning objective without altering the model’s parameters. Data-based methods [18] typically fine-tune models on pre-constructed desirable responses, using prompts from the forget data distribution. Model-based methods [5, 19] focus on modifying the weights or architecture to achieve the unlearning objective. Among these approaches, the most relevant to our work is fine-tuning the target LLM using a modified loss function, which typically incorporates two key objectives: maximizing the loss on the forget samples and minimizing (or maintaining) the loss on the retain samples. Table 1: Comparison of different loss adjustment-based baselines in terms of their requirement. Our method relies solely on forget data and available template responses, without using the retain data or a reference model for response calibration. Baselines Forget Data Retain Data Reference Model Gradient Ascent (GA) [20] ✓ ✗ ✗ Gradient Difference (GD) [20] ✓ ✓ ✗ KL Minimization (KL) [20] ✓ ✓ ✓ Preference Optimization (PO) [20] ✓ ✓ ✗ Mismatch [16] ✓ ✓ ✗ Direct Preference Optimization (DPO) [21] ✓ ✗ ✓ Negative Preference Optimization (NPO) [22] ✓ ✗ ✓ Large Language Model Unlearning (LLMU) [5] ✓ ✓ ✓ FLAT (Ours) ✓ ✗ ✗ However, as summarized in Table 1, current loss adjustment-based methods either rely on retain data [20, 23, 5], which might not be readily available in real-world scenarios [6], or utilize a reference model [21, 22, 5, 20] to maintain performance on the retain dataset, incurring additional cost during training—especially when fine-tuning a large-scale LLM. Moreover, leveraging explicit retain data or implicit knowledge from a reference LLM during fine-tuning may blur the distinction between the forget and retain data, which can lead to a trade-off between model utility and forget quality. Furthermore, fine-tuning using both retain data and forget data would require a careful design of a data mixing strategy. To preserve model utility while improving forget quality, we propose Forget data only Loss AjustmenT (FLAT), a "flat" loss adjustment approach which adjusts the loss function using only the forget data. Given the forget data, FLAT guides the LLM not only in what to forget but also in how to respond, by optimizing the f-divergence between the template and forget answers with respect to the forget data. The variational form of the f-divergence enables loss adjustment by assigning optimal importance weights to learning from template responses while forgetting the responses subject to unlearning. Our main contributions are highlighted below: ∙ ∙ We identify the potential drawback of relying on retain data or a reference LLM to guide LLM unlearning. To address this, we propose FLAT, which facilitates LLM unlearning without requiring retain data or a reference LLM for response calibration. ∙ ∙ FLAT optimizes the f-divergence between template and forget responses to guide the LLM through the unlearning process. The variational form of f-divergence optimization provides a clear illustration of how to optimally balance forget quality and model utility, with theoretical guarantees. ∙ ∙ Extensive experiments on three unlearning tasks, including copyrighted content unlearning on the Harry Potter dataset and MUSE benchmark, as well as entity unlearning on the TOFU dataset, demonstrate the superior performance of our method, achieving both high unlearning efficiency and strong overall model utility. 2 Preliminaries In this section, we introduce the preliminary formulation of LLM unlearning and the existing LLM unlearning framework. 2.1 Formulation Given a forget dataset DfsubscriptD_fDitalic_f, a retain dataset DrsubscriptD_rDitalic_r, and an LLM θosubscript _oθitalic_o, the task of LLM unlearning is to fine-tune the original model such that the updated LLM θ resembles a model trained without DfsubscriptD_fDitalic_f. For a prompt-response pair (x,y)(x,y)( x , y ), the loss function on y for fine-tuning is ℒ⁢(x,y;θ)=∑i=1|y|ℓ⁢(hθ⁢(x,y<i),yi)ℒsuperscriptsubscript1ℓsubscriptℎsubscriptabsentsubscriptL(x,y;θ)= _i=1^|y| (h_θ(x,y_<i),y_i)L ( x , y ; θ ) = ∑i = 1| y | ℓ ( hitalic_θ ( x , y< i ) , yitalic_i ), where ℓ⁢(⋅)ℓ⋅ (·)ℓ ( ⋅ ) is the cross-entropy loss, and hθ⁢(x,y<i):=ℙ⁢(yi|(x,y<i);θ)assignsubscriptℎsubscriptabsentℙconditionalsubscriptsubscriptabsenth_θ(x,y_<i):=P(y_i|(x,y_<i);θ)hitalic_θ ( x , y< i ) := blackboard_P ( yitalic_i | ( x , y< i ) ; θ ) is the predicted probability of the token yisubscripty_iyitalic_i given by an LLM θ, with the input prompt x and the already generated tokens y<i:=[y1,…,yi−1]assignsubscriptabsentsubscript1…subscript1y_<i:=[y_1,...,y_i-1]y< i := [ y1 , … , yitalic_i - 1 ]. The most straightforward approach to unlearn is Gradient Ascent (GA). GA modifies a trained model so that it "forgets" or minimizes the influence of specific data or patterns it has previously learned. Mathematically, the GA algorithm iteratively updates the model at step t by performing gradient ascent on the next-token prediction loss over the forget dataset: θt+1←θt+λ⁢∇θtℒ⁢(x,y;θt)←subscript1subscriptsubscript∇subscriptℒsubscript _t+1← _t+λ _ _tL(x,y;% _t)θitalic_t + 1 ← θitalic_t + λ ∇θ start_POSTSUBSCRIPT t end_POSTSUBSCRIPT L ( x , y ; θitalic_t ), where λ is the (un)learning rate. 2.2 Existing LLM Unlearning Paradigm The mainstream class of existing LLM unlearning methods involves fine-tuning the original LLM against an unlearning objective function. Although the exact designs vary, the general type of loss adjustment in LLM unlearning can be characterized as follows: L=LFG+LRT+LCustom.subscriptFGsubscriptRTsubscriptCustomL=L_FG+L_RT+L_Custom.L = LFG + LRT + LCustom . (1) The modified loss function comprises three main components: ∙ ∙ LFGsubscriptFGL_FGLFG (Forget Loss): Encourages the model to "forget" the undesired data or patterns. This typically involves increasing the loss on the data to be forgotten, effectively making the model perform worse on those specific examples. The goal is to reduce the model’s reliance on these data points, thereby minimizing their influence on future predictions. ∙ ∙ LRTsubscriptRTL_RTLRT (Retain Loss): Ensures that the model maintains its overall performance and general knowledge on unaffected data. It typically involves using the original loss function from training or a modified version that focuses on the data the model is meant to retain. This term prevents the unlearning process from degrading the model’s overall capabilities beyond the scope of the specific unlearning objective. ∙ ∙ LCustomsubscriptCustomL_CustomLCustom (Custom Loss): Allows for additional flexibility and customization in the unlearning process. It may include regularization terms to control the magnitude of parameter updates or specific constraints to enforce certain unlearning behaviors. This component enables researchers to tailor the unlearning process to specific requirements or incorporate domain-specific knowledge. In summary, common loss adjustment methods employ one [24], two [23, 20, 22], or all three [5] of these components to guide the model towards forgetting specific data while minimizing the impact on its overall performance and utility. The interplay between these terms allows for controlled and targeted unlearning, ensuring the model retains its valuable capabilities while selectively forgetting undesired information. More detailed formulations of these loss adjustment-based methods, along with related work, are deferred to Appendix C.1 and Appendix E. An Example: Large Language Model Unlearning (LLMU). We adopt a popular approach in LLM unlearning, LLMU [5], to interpret a special case of Eqn. (1). Specifically, the objective of LLMU contains three components: the Unlearn Harm LFGsubscriptFGL_FGLFG, the Maintain Performance LRTsubscriptRTL_RTLRT, and the Random Mismatch LRandomsubscriptRandomL_RandomLRandom (the custom loss). The training objective is as follows: LLLMU=LFG+LRT+LRandom,subscriptLLMUsubscriptFGsubscriptRTsubscriptRandomL_LLMU=L_FG+L_RT+L_Random,LLLMU = LFG + LRT + LRandom , The forget loss LFG=−∑(xf,yf)∈Dfℒ⁢(xf,yf;θ)subscriptFGsubscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptL_FG=- _(x_f,y_f)∈ D_fL(x_f,y_f;θ)LFG = - ∑( x start_POSTSUBSCRIPT f , yitalic_f ) ∈ Ditalic_f end_POSTSUBSCRIPT L ( xitalic_f , yitalic_f ; θ ), where (xf,yf)subscriptsubscript(x_f,y_f)( xitalic_f , yitalic_f ) indicates the forget data pairs from the forget dataset DfsubscriptD_fDitalic_f, θ is the updated unlearned model. It is actually the Gradient Ascent loss to forget the samples subject to unlearning. The retain loss LRT=∑(xr,yr)∈Dr∑i=1|yr|KL(hθo(xr,yr<i)||hθ(xr,yr<i))L_RT= _(x_r,y_r)∈ D_r _i=1^|y_r|KL(h_ _% o(x_r,y_r<i)||h_θ(x_r,y_r<i))LRT = ∑( x start_POSTSUBSCRIPT r , yitalic_r ) ∈ Ditalic_r end_POSTSUBSCRIPT ∑i = 1| yitalic_r | K L ( hitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( xitalic_r , yitalic_r < i ) | | hitalic_θ ( xitalic_r , yitalic_r < i ) ), where K⁢L⁢(⋅)⋅KL(·)K L ( ⋅ ) is the KL divergence term, (xr,yr)subscriptsubscript(x_r,y_r)( xitalic_r , yitalic_r ) indicates the retain data pairs from the retain dataset DrsubscriptD_rDitalic_r, θosubscript _oθitalic_o is the original model, and θ is the updated model. The random loss LRandom=∑(xf,⋅)∈Df1|Yr⁢d⁢n|⁢∑yr⁢d⁢n∈Yr⁢d⁢nℒ⁢(xf,yr⁢d⁢n;θ).subscriptRandomsubscriptsubscript⋅subscript1subscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptL_Random= _(x_f,·)∈ D_f 1|Y_rdn| _y_% rdn∈ Y_rdnL(x_f,y_rdn;θ).LRandom = ∑( x start_POSTSUBSCRIPT f , ⋅ ) ∈ Ditalic_f end_POSTSUBSCRIPT divide start_ARG 1 end_ARG start_ARG | Yitalic_r d n | end_ARG ∑y start_POSTSUBSCRIPT r d n ∈ Yitalic_r d n end_POSTSUBSCRIPT L ( xitalic_f , yitalic_r d n ; θ ) . Here, Yr⁢d⁢nsubscriptY_rdnYitalic_r d n is a set of random responses that do not have a connection to the forget prompts xfsubscriptx_fxitalic_f. 3 Method Table 2: fd⁢i⁢vsubscriptf_divfitalic_d i vs, optimal variational g (g∗superscriptg^*g∗), conjugate functions (f∗superscriptf^*f∗). Name g∗⁢(v)superscriptg^*(v)g∗ ( v ) domf∗subscriptdomsuperscriptdom_f^*domf∗ f∗⁢(u)superscriptf^*(u)f∗ ( u ) Total Variation 12⁢tanh⁡v12 12 vdivide start_ARG 1 end_ARG start_ARG 2 end_ARG tanh v u∈[−12,12]1212u∈[- 12, 12]u ∈ [ - divide start_ARG 1 end_ARG start_ARG 2 end_ARG , divide start_ARG 1 end_ARG start_ARG 2 end_ARG ] u Jensen-Shannon log⁡21+e−v21superscript 21+e^-vlog divide start_ARG 2 end_ARG start_ARG 1 + e- v end_ARG u<log⁡22u< 2u < log 2 −log⁡(2−eu)2superscript- (2-e^u)- log ( 2 - eitalic_u ) Pearson v ℝRblackboard_R 14⁢u2+u14superscript2 14u^2+udivide start_ARG 1 end_ARG start_ARG 4 end_ARG u2 + u KL v ℝRblackboard_R eu−1superscript1e^u-1eitalic_u - 1 In this section, we introduce Forget data only Loss AjustmenT (FLAT), a "flat" loss adjustment approach which adjusts the loss function using only the forget data, by leveraging f-divergence maximization towards the distance between the preferred template and original forget responses. We first derive the formulation of our method via f-divergence maximization (§ 3.1), followed by the presentation of the empirical alternative to our approach (§ 3.2). In section § 3.3, we explore the estimation gap between the theoretical and empirical f-divergence. Finally, we discuss the connection between our method and DPO (§ 3.4). 3.1 Loss-Adjustments via f-divergence Maximization For each learning batch, we assume that we only have access to a set of forget samples (xf,yf)∈Dfsubscriptsubscriptsubscript(x_f,y_f)∈ D_f( xitalic_f , yitalic_f ) ∈ Ditalic_f. Instead of directly adopting gradient ascent over these forget samples, we propose to maximize the divergence between exemplary and bad generations of forget data. Key steps are summarized as below. ∙ ∙ Step 1: Equip example/template responses yesubscripty_eyitalic_e for each forget sample xfsubscriptx_fxitalic_f. Together we denote the paired samples as De=(xfj,yej)j∈[N]subscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptdelimited-[]D_e=\(x_f^j,y_e^j)\_j∈[N]Ditalic_e = ( xitalic_fitalic_j , yitalic_eitalic_j ) j ∈ [ N ]. This could be done by leveraging open-source LLMs such as Llama 3.1 [25] or self-defining the responses according to our wish, etc. The designated unlearning response could be a reject-based answer such as "I don’t know" (denoted as "IDK") or an irrelevant answer devoid of the unlearning target-related information. Motivation: Step 1 generates example responses for LLM fine-tuning and provides better instructions on what LLM should respond given the forget data. Besides, certain existing methods make LLM generate hallucinated responses after unlearning, which further illustrates the importance of example responses for LLM unlearning. ∙ ∙ Step 2: Loss adjustmens w.r.t. the sample pairs (xf,ye,yf)subscriptsubscriptsubscript(x_f,y_e,y_f)( xitalic_f , yitalic_e , yitalic_f ) through: L⁢(xf,ye,yf;θ)=λe⋅Le⁢(xf,ye;θ)−λf⋅Lf⁢(xf,yf;θ),subscriptsubscriptsubscript⋅subscriptsubscriptsubscriptsubscript⋅subscriptsubscriptsubscriptsubscript L(x_f,y_e,y_f;θ)= _e· L_e(x_f,y_e;% θ)- _f· L_f(x_f,y_f;θ),L ( xitalic_f , yitalic_e , yitalic_f ; θ ) = λitalic_e ⋅ Litalic_e ( xitalic_f , yitalic_e ; θ ) - λitalic_f ⋅ Litalic_f ( xitalic_f , yitalic_f ; θ ) , (2) where Le,LfsubscriptsubscriptL_e,L_fLitalic_e , Litalic_f are losses designed for the data sample (xf,ye)subscriptsubscript(x_f,y_e)( xitalic_f , yitalic_e ) and (xf,yf)subscriptsubscript(x_f,y_f)( xitalic_f , yitalic_f ), respectively. The corresponding closed form will be introduced in Section § 3.2. Motivation: Step 2 encourages the LLM to forget the forget data with bad responses, meanwhile, learn to generate good responses on relevant forget data. [such as template answers] ∙ ∙ Step 3: How to decide on the values of λesubscript _eλitalic_e and λfsubscript _fλitalic_f? We leverage f-divergence to illustrate the appropriate balancing between Le⁢(xf,ye;θ)subscriptsubscriptsubscriptL_e(x_f,y_e;θ)Litalic_e ( xitalic_f , yitalic_e ; θ ) and Lf⁢(xf,yf;θ)subscriptsubscriptsubscriptL_f(x_f,y_f;θ)Litalic_f ( xitalic_f , yitalic_f ; θ ). Assume xf,yesubscriptsubscriptx_f,y_exitalic_f , yitalic_e is generated by the random variable Xf,YesubscriptsubscriptX_f,Y_eXitalic_f , Yitalic_e jointly following the distribution esubscriptD_eDitalic_e. Similarly, xf,yfsubscriptsubscriptx_f,y_fxitalic_f , yitalic_f is given by Xf,YfsubscriptsubscriptX_f,Y_fXitalic_f , Yitalic_f and (Xf,Yf)∼fsimilar-tosubscriptsubscriptsubscript(X_f,Y_f) _f( Xitalic_f , Yitalic_f ) ∼ Ditalic_f. Step 2 shares similar insights as if we are maximizing the divergence between esubscriptD_eDitalic_e and fsubscriptD_fDitalic_f. Our theoretical purpose is to obtain the model that maximizes the f-divergence between esubscriptD_eDitalic_e and fsubscriptD_fDitalic_f, defined as fd⁢i⁢v(e||f)f_div(D_e||D_f)fitalic_d i v ( Ditalic_e | | Ditalic_f ). The variational form f-divergence Instead of optimizing the fd⁢i⁢vsubscriptf_divfitalic_d i v term directly, we resolve to the variational form of it. Due to the Fenchel duality, we would have: fd⁢i⁢v(e||f)=supg[e∼e[g(e)]−f∼f[f∗(g(f))]]:=supgVA(θ,g), f_div(D_e||D_f)= _g [% E_Z_e _e [g(Z_e) ]- % E_Z_f _f [f^*(g(Z_f)) ]% ]:= _gVA(θ,g),fitalic_d i v ( Ditalic_e | | Ditalic_f ) = supitalic_g [ blackboard_EZ start_POSTSUBSCRIPT e ∼ Ditalic_e end_POSTSUBSCRIPT [ g ( Zitalic_e ) ] - blackboard_EZ start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ f∗ ( g ( Zitalic_f ) ) ] ] := supitalic_g VA ( θ , g ) , (3) we define f∗superscriptf^*f∗ as the conjugate function of the f-divergence function. For simplicity, we define VA⁢(θ,g∗):=supgVA⁢(θ,g)assignVAsuperscriptsubscriptsupremumVAVA(θ,g^*):= _gVA(θ,g)VA ( θ , g∗ ) := supitalic_g VA ( θ , g ), where g∗superscriptg^*g∗ is the optimal variational function. Hence, the objective of FLAT is to obtain: θ∗:=arg⁡maxθ⁡VA⁢(θ,g∗)assignsuperscriptsubscriptVAsuperscriptθ^*:= _θVA(θ,g^*)θ∗ := arg maxitalic_θ VA ( θ , g∗ ). Motivation: Note that existing solutions fail to keep a good balance between model performance on forget data and retain data, step 3 provides a formal theoretical framework of our loss revision in Step 2, under the f−limit-fromf-f -divergence maximization between esubscriptD_eDitalic_e and fsubscriptD_fDitalic_f, the method assigns the appropriate weights f∗⁢(⋅),g∗⁢(f∗⁢(⋅))superscript⋅superscriptsuperscript⋅f^*(·),g^*(f^*(·))f∗ ( ⋅ ) , g∗ ( f∗ ( ⋅ ) ) w.r.t. the joint data distributions e,fsubscriptsubscriptD_e,D_fDitalic_e , Ditalic_f. 3.2 Empirical Alternative of Loss Adjustment Note that Eqn. (3) could be viewed as a data distribution level loss adjustment, in practice, when given access to a set of forget data as well as example and bad answers, xf,ye,yfsubscriptsubscriptsubscriptx_f,y_e,y_fxitalic_f , yitalic_e , yitalic_f, the per-sample loss function (closed form of Eqn. (2)) would be given by111To clarify, we introduce the negative sign on the r.h.s. because loss function is commonly combined with the minimization task, while our method is formulated as maximizing the f-divergence.: L⁢(xf,ye,yf;θ)subscriptsubscriptsubscript L(x_f,y_e,y_f;θ)L ( xitalic_f , yitalic_e , yitalic_f ; θ ) =−[supg[g⁢(ℙ⁢(xf,ye;θ))−f∗⁢(g⁢(ℙ⁢(xf,yf;θ)))]]absentdelimited-[]subscriptsupremumdelimited-[]ℙsubscriptsubscriptsuperscriptℙsubscriptsubscript =- [ _g [g(P(x_f,y_e;θ))-f^*(g(% P(x_f,y_f;θ))) ] ]= - [ supitalic_g [ g ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ) ] ] =−g∗⁢(ℙ⁢(xf,ye;θ))+f∗⁢(g∗⁢(ℙ⁢(xf,yf;θ))).absentsuperscriptℙsubscriptsubscriptsuperscriptsuperscriptℙsubscriptsubscript =-g^*(P(x_f,y_e;θ))+f^*(g^*(P(x_% f,y_f;θ))).= - g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) + f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ) . (4) We provide examples of f-divergence functions in Table 2, along with their conjugate and variational functions [26, 27]. We illustrate via following examples. Example 1: Total-Variation For Total-Variation (TV), an example of f-divergence, f∗⁢(u)=u,g∗⁢(v)=tanh⁡(v)2formulae-sequencesuperscriptsuperscript2f^*(u)=u,g^*(v)= (v)2f∗ ( u ) = u , g∗ ( v ) = divide start_ARG tanh ( v ) end_ARG start_ARG 2 end_ARG, hence, g∗(ℙ(xf,ye;θ))−f∗(g∗(ℙ(xf,yf;θ))=tanh⁡(ℙ⁢(xf,ye;θ))2−tanh⁡(ℙ⁢(xf,yf;θ))2.g^*(P(x_f,y_e;θ))-f^*(g^*(P(x_f,y_f;% θ))= (P(x_f,y_e;θ))2- (% P(x_f,y_f;θ))2.g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) = divide start_ARG tanh ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) end_ARG start_ARG 2 end_ARG - divide start_ARG tanh ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) end_ARG start_ARG 2 end_ARG . We defer examples of other f-divergence functions in the Appendix B.1. How to estimate ℙ⁢(xf,ye;θ),ℙ⁢(xf,yf;θ)ℙsubscriptsubscriptℙsubscriptsubscriptP(x_f,y_e;θ),P(x_f,y_f;θ)blackboard_P ( xitalic_f , yitalic_e ; θ ) , blackboard_P ( xitalic_f , yitalic_f ; θ )? We define the following two quantities: ℙ⁢(xf,ye;θ):=∑i=1|ye|P⁢(hθ⁢(xf,ye,<i)=ye,i)|ye|,ℙ⁢(xf,yf;θ):=∑i=1|yf|P⁢(hθ⁢(xf,yf,<i)=yf,i)|yf|.formulae-sequenceassignℙsubscriptsubscriptsuperscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscriptsubscriptassignℙsubscriptsubscriptsuperscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscriptsubscriptP(x_f,y_e;θ):= _i=1^|y_e|P(h_θ(x_f,y% _e,<i)=y_e,i)|y_e|, (x_f,y_f;θ):= _i% =1^|y_f|P(h_θ(x_f,y_f,<i)=y_f,i)|y_f|.blackboard_P ( xitalic_f , yitalic_e ; θ ) := divide start_ARG ∑i = 1| yitalic_e | P ( hitalic_θ ( xitalic_f , yitalic_e , < i ) = yitalic_e , i ) end_ARG start_ARG | yitalic_e | end_ARG , blackboard_P ( xitalic_f , yitalic_f ; θ ) := divide start_ARG ∑i = 1| yitalic_f | P ( hitalic_θ ( xitalic_f , yitalic_f , < i ) = yitalic_f , i ) end_ARG start_ARG | yitalic_f | end_ARG . Here, ye,isubscripty_e,iyitalic_e , i and yf,isubscripty_f,iyitalic_f , i denote the i-th token in the samples yesubscripty_eyitalic_e and yfsubscripty_fyitalic_f, respectively, while ye,<isubscriptabsenty_e,<iyitalic_e , < i and yf,<isubscriptabsenty_f,<iyitalic_f , < i represent the already generated tokens. |ye|subscript|y_e|| yitalic_e | and |yf|subscript|y_f|| yitalic_f | are the lengths of the example response yesubscripty_eyitalic_e and the forget response yfsubscripty_fyitalic_f, respectively. Given a prompt and the previously generated tokens, P⁢(hθ⁢(xf,ye,<i)=ye,i)subscriptℎsubscriptsubscriptabsentsubscriptP(h_θ(x_f,y_e,<i)=y_e,i)P ( hitalic_θ ( xitalic_f , yitalic_e , < i ) = yitalic_e , i ) and P⁢(hθ⁢(xf,yf,<i)=yf,i)subscriptℎsubscriptsubscriptabsentsubscriptP(h_θ(x_f,y_f,<i)=y_f,i)P ( hitalic_θ ( xitalic_f , yitalic_f , < i ) = yitalic_f , i ) are the probabilities of correctly predicting the next token. These two quantities represent the average probabilities of correctly generating tokens for the template and forget responses, respectively. To align Eqn. (3.2) with Eqn. (2), we could define λe=λf=1subscriptsubscript1 _e= _f=1λitalic_e = λitalic_f = 1, Le⁢(xf,ye;θ):=−g∗⁢(ℙ⁢(xf,ye;θ))assignsubscriptsubscriptsubscriptsuperscriptℙsubscriptsubscriptL_e(x_f,y_e;θ):=-g^*(P(x_f,y_e;θ))Litalic_e ( xitalic_f , yitalic_e ; θ ) := - g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ), and Lf⁢(xf,yf;θ):=−f∗⁢(g∗⁢(ℙ⁢(xf,yf;θ)))assignsubscriptsubscriptsubscriptsuperscriptsuperscriptℙsubscriptsubscriptL_f(x_f,y_f;θ):=-f^* (g^*(P(x_f,y_f;θ)) )Litalic_f ( xitalic_f , yitalic_f ; θ ) := - f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ). 3.3 The upper bound of the estimation gap To connect the empirical alternative of FLAT with the corresponding theoretical format, in this section, we aim to explore the estimation gap between the theoretical f-divergence fd⁢i⁢v(e||f)f_div(D_e||D_f)fitalic_d i v ( Ditalic_e | | Ditalic_f ) and the empirical optimal estimated f-divergence f^d⁢i⁢v(De||Df) f_div(D_e||D_f)over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ), here we define: f^d⁢i⁢v(De||Df):=Ze∼De[g^(Ze)]−Zf∼Df[f∗(g^(Zf))], f_div(D_e||D_f):=E_Z_e D_e[ % g(Z_e)]-E_Z_f D_f[f^*( g(Z_f))],over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ) := blackboard_EZ start_POSTSUBSCRIPT e ∼ Ditalic_e end_POSTSUBSCRIPT [ over start_ARG g end_ARG ( Zitalic_e ) ] - blackboard_EZ start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ f∗ ( over start_ARG g end_ARG ( Zitalic_f ) ) ] , where g^:=supg∈ΦZe∼De⁢[g⁢(Ze)]−Zf∼Df⁢[f∗⁢(g⁢(Zf))]assign^subscriptsupremumΦsubscriptsimilar-tosubscriptsubscriptdelimited-[]subscriptsubscriptsimilar-tosubscriptsubscriptdelimited-[]superscriptsubscript g:= _g∈ E_Z_e D_e[g(Z_e)]-E_% Z_f D_f[f^*(g(Z_f))]over start_ARG g end_ARG := supitalic_g ∈ Φ blackboard_EZ start_POSTSUBSCRIPT e ∼ Ditalic_e end_POSTSUBSCRIPT [ g ( Zitalic_e ) ] - blackboard_EZ start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ f∗ ( g ( Zitalic_f ) ) ] and Φ Φ is the function space. Assumption 3.1 (Bounded Density Ratio). The density ratio Ze/ZfsubscriptsubscriptZ_e/Z_fZitalic_e / Zitalic_f is lower and upper bounded by positive constants a and b, respectively. Assumption 3.1 is wildely adopted by the literature [28, 29], which necessitates that the probability density functions Ze,ZfsubscriptsubscriptZ_e,Z_fZitalic_e , Zitalic_f share the same support. Assumption 3.2 (Regularity of Divergence Function). f⁢(⋅)⋅f(·)f ( ⋅ ) is smooth on [a,b][a,b][ a , b ], and f⁢(1)=010f(1)=0f ( 1 ) = 0. f is μ0subscript0 _0μ0-strongly convex, and has L0subscript0L_0L0-Lipschitz continuous gradient on [a,b][a,b][ a , b ], for positive constants μ0,L0subscript0subscript0 _0,L_0μ0 , L0. Assumption 3.2 is a mild condition since it only requires the condition to hold for the interval [a,b][a,b][ a , b ], which works for many commonly used f-divergence functions, i.e., KL divergence. Let (V,||⋅||L2)(V,||·||_L_2)( V , | | ⋅ | |L start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT ) be a normed space, and Φ⊂VΦ ⊂ VΦ ⊂ V. v1,…,vCsubscript1…subscriptv_1,...,v_Cv1 , … , vitalic_C is a δ-covering over Φ Φ of size C if Φ⊂∪i=1CB⁢(vi,δ)Φsuperscriptsubscript1subscript ⊂ _i=1^CB(v_i,δ)Φ ⊂ ∪i = 1C B ( vitalic_i , δ ) where B⁢(vi,δ)subscriptB(v_i,δ)B ( vitalic_i , δ ) is the δ-ball centered at visubscriptv_ivitalic_i. The covering number is then defined as C2⁢(δ,Φ)=min⁡C:∃δ⁢-covering over ⁢Φ⁢ of size ⁢Csubscript2Φ:-covering over Φ of size C_2(δ, )= \C:∃δ-covering over of % size C\C2 ( δ , Φ ) = min C : ∃ δ -covering over Φ of size C . The following assumption would characterize the representation power of the function space Φ Φ. Assumption 3.3 (Order of Covering Number). C2⁢(δ,Φ)=⁢(exp⁡δ−rΦ)subscript2ΦsuperscriptsubscriptΦC_2(δ, )=O( \δ^-r_ \)C2 ( δ , Φ ) = O ( exp δ- rroman_Φ ), and rΦ∈(0,2)subscriptΦ02r_ ∈(0,2)rroman_Φ ∈ ( 0 , 2 ). Theorem 3.4. Given Assumptions 3.1-3.3, suppose g^∈Φ^Φ g∈ start_ARG g end_ARG ∈ Φ, with probability ≥1−e−NrΦ/(2+rΦ)absent1superscriptsuperscriptsubscriptΦ2subscriptΦ≥ 1-e^-N^r_ /(2+r_ )≥ 1 - e- N start_POSTSUPERSCRIPT rroman_Φ / ( 2 + rroman_Φ ) end_POSTSUPERSCRIPT, we have: |f^d⁢i⁢v(De||Df)−fd⁢i⁢v(e||f)|≾N−1rΦ+2, | f_div(D_e||D_f)-f_div(D_e||D_f)% | N^- 1r_ +2,| over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ) - fitalic_d i v ( Ditalic_e | | Ditalic_f ) | ≾ N- divide start_ARG 1 end_ARG start_ARG rroman_Φ + 2 end_ARG , where we have defined N as the number of samples in the forget data. Theorem 3.4 illustrates that the empirical alternative of FLAT, f^d⁢i⁢v(De||Df) f_div(D_e||D_f)over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ), achieves the optimal non-parametric rate of convergence towards fd⁢i⁢v(e||f)f_div(D_e||D_f)fitalic_d i v ( Ditalic_e | | Ditalic_f ). 3.4 Connection with DPO In this section, we discuss the connection and key differences between our approach and the celebrated Direct Preference Optimization (DPO) [21] approach for aligning LLMs. Given a dataset D=(xfj,yej,yfj)j∈[N]subscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptdelimited-[]D=\(x_f^j,y_e^j,y_f^j)\_j∈[N]D = ( xitalic_fitalic_j , yitalic_eitalic_j , yitalic_fitalic_j ) j ∈ [ N ], where yesubscripty_eyitalic_e and yfsubscripty_fyitalic_f are preferred template and original forget responses to the forget prompt xfsubscriptx_fxitalic_f, DPO [21] fine-tunes original model θosubscript _oθitalic_o using D to better align it with good answer preferences, which minimizes: LDPO,β⁢(θ)subscriptDPO L_DPO,β(θ)LDPO , β ( θ ) =−2β⁢D⁢[log⁡σ⁢(β⁢log⁡πθ⁢(ye∣xf)πr⁢e⁢f⁢(ye∣xf)−β⁢log⁡πθ⁢(yf∣xf)πr⁢e⁢f⁢(yf∣xf))]absent2subscriptdelimited-[]subscriptconditionalsubscriptsubscriptsubscriptconditionalsubscriptsubscriptsubscriptconditionalsubscriptsubscriptsubscriptconditionalsubscriptsubscript =- 2βE_D [ σ (β % _θ(y_e x_f) _ref(y_e x_f)-β % _θ(y_f x_f) _ref(y_f x_f) ) ]= - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED [ log σ ( β log divide start_ARG πitalic_θ ( yitalic_e ∣ xitalic_f ) end_ARG start_ARG πitalic_r e f ( yitalic_e ∣ xitalic_f ) end_ARG - β log divide start_ARG πitalic_θ ( yitalic_f ∣ xitalic_f ) end_ARG start_ARG πitalic_r e f ( yitalic_f ∣ xitalic_f ) end_ARG ) ] =−2β⁢D⁢[log⁡σ⁢(β⁢(log⁢∏i=1|ye|hθ⁢(xf,ye,<i)−log⁢∏i=1|yf|hθ⁢(xf,yf,<i))−Mr⁢e⁢f)],absent2subscriptdelimited-[]superscriptsubscriptproduct1subscriptsubscriptℎsubscriptsubscriptabsentsuperscriptsubscriptproduct1subscriptsubscriptℎsubscriptsubscriptabsentsubscript =- 2βE_D [ [rgb]1,0,0 % σ (β( _i=1^|y_e|h_θ(x_f,y_e,<i)-% [rgb]1,0,0 _i=1^|y_f|h_θ(x_f,y_f,<i)) % [rgb]1,0,0-M_ref ) ],= - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED [ log σ ( β ( log ∏i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) - log ∏i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) ) - Mitalic_r e f ) ] , where, σ⁢(t)=11+e−t11superscriptσ(t)= 11+e^-tσ ( t ) = divide start_ARG 1 end_ARG start_ARG 1 + e- t end_ARG is the sigmoid function, β>00β>0β > 0 is the inverse temperature, πθ:=∏i=1|y|hθ⁢(x,y<i)assignsubscriptsuperscriptsubscriptproduct1subscriptℎsubscriptabsent _θ:= _i=1^|y|h_θ(x,y_<i)πitalic_θ := ∏i = 1| y | hitalic_θ ( x , y< i ) is the predicted probability of the response y to prompt x given by LLM θ, πr⁢e⁢fsubscript _refπitalic_r e f is the predicted probability given by reference model, and Mr⁢e⁢f:=β⁢(log⁢∏i=1|ye|hθo⁢(xf,ye,i)−log⁢∏i=1|yf|hθo⁢(xf,yf,i))assignsubscriptsuperscriptsubscriptproduct1subscriptsubscriptℎsubscriptsubscriptsubscriptsuperscriptsubscriptproduct1subscriptsubscriptℎsubscriptsubscriptsubscriptM_ref:=β( _i=1^|y_e|h_ _o(x_f,y_e,i)- % _i=1^|y_f|h_ _o(x_f,y_f,i))Mitalic_r e f := β ( log ∏i = 1| yitalic_e | hitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( xitalic_f , yitalic_e , i ) - log ∏i = 1| yitalic_f | hitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( xitalic_f , yitalic_f , i ) ). As for FLAT, we calculate the average probability of all correctly generated tokens and employ a novel re-weighting mechanism that assigns different importance to each term using distinct activate functions for both the example and forget loss terms, which minimizes: LFLAT⁢(θ)subscriptFLAT L_ FLAT(θ)LFLAT ( θ ) =−D[g∗(1|ye|∑i=1|ye|hθ(xf,ye,<i))−f∗(g∗(1|yf|∑i=1|yf|hθ(xf,yf,<i))]. =-E_D [ [rgb]1,0,0g^*( 1|y_e|% _i=1^|y_e|h_θ(x_f,y_e,<i) [rgb]1,0,0)- [% rgb]1,0,0f^*(g^*( 1|y_f| _i=1^|y_f|h_θ(x_f,% y_f,<i) [rgb]1,0,0) ].= - blackboard_ED [ g∗ ( divide start_ARG 1 end_ARG start_ARG | yitalic_e | end_ARG ∑i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) ) - f∗ ( g∗ ( divide start_ARG 1 end_ARG start_ARG | yitalic_f | end_ARG ∑i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) ) ] . Here, f∗⁢(⋅),g∗⁢(f∗⁢(⋅))superscript⋅superscriptsuperscript⋅f^*(·),g^*(f^*(·))f∗ ( ⋅ ) , g∗ ( f∗ ( ⋅ ) ) are the activate functions that assign appropriate weights to each loss term. The detailed derivation is in Appendix B.2. The key differences are highlighted in red. Specifically, DPO relies on a reference model to guide the unlearning process, whereas FLAT only uses a sample pair dataset containing both exemplar and forget responses. Besides, our solution differs from DPO in three critical aspects: the re-weighting activation function, whether to sum or average the token losses, and whether to apply the logarithm to the output probability. We conduct an ablation study with DPO to evaluate the effectiveness of the proposed re-weighting mechanism in Section § 4.5. 4 Experiment In this section, we compare the proposed method with baseline unlearning methods on three widely used LLM unlearning tasks: copyrighted content unlearning on Harry Potter (HP) Series Book [5] (§ 4.2), entity unlearning on TOFU dataset [20] (§ 4.3), and unlearning on MUSE-News benchmark [30] (§ 4.4). We conduct additional ablation studies to assess the effectiveness of our methods in Section § 4.5. 4.1 Baseline Methods We evaluate the effectiveness of our proposed method FLAT by comparing it to a series of strong LLM unlearning baselines, particularly those based on loss adjustment. We consider Gradient Ascent (GA) [24, 5], KL minimization (KL) [20], GradDiff (GD) [23], and NPO [22] across all three tasks. For copyrighted content and entity unlearning, we also include Preference Optimization (PO) [20], Large Language Model Unlearning (LLMU) [5], and DPO [21]. For the Harry Potter dataset, we add Mismatch [16] as a simple and effective baseline method. For the MUSE-News benchmark, we additionally consider Task Vectors [31], Who’s Harry Potter (WHP) [14] and an extended version of NPO (NPO-RT) as a comparable method, which incorporates a fine-tuning term on the retain dataset. Further experiment details are provided in Appendix C.1. 4.2 Copyrighted Content Unlearning Experiment Setup. We select Harry Potter and the Sorcerer’s Stone [32] as the copyrighted content for unlearning. The objective is to ensure that the unlearned model does not generate passages with high similarity to the original text. Following prior works [16, 5], we first fine-tune LLMs on the corresponding corpus, treating it as the model subject to unlearning, while using the original pre-trained checkpoint as the retained model222We empirically verified that the initial LLM cannot generate the original corpus, making it a valid candidate for retrained model.. Following [5, 33], We extract 400 chunks from the Harry Potter book series dataset [14], with each chunk containing up to 512 tokens, to create the forget dataset DfsubscriptD_fDitalic_f. We sample 400 paragraphs in the C4 dataset [34] as the retain data DrsubscriptD_rDitalic_r. The IDK dataset comes from [33]. We experiment with OPT-2.7B [35] and Llama2-7B [36] for this task. Evaluation Metrics. We report three key metrics to assess the unlearning efficiency and model utility of the unlearned models. For unlearning efficiency, we use the Forget Quality Gap (FQ Gap), similar to [16], which is the sum of the BLEU Gap and ROUGE-L Gap. It is the absolute difference between the retained model and the unlearned model on these metrics. Specifically, we calculate BLEU [37] and ROUGE-L [38] scores by comparing ground-truth excerpts with completions generated by the unlearned model, given a fixed prefix length of 200 tokens from the forget data, to reflect potential copyright content leakage. We further conduct study on the prompt length for evaluation in Appendix D.1. Note that for these metrics, values closer to those of the retained model indicate better unlearning, while values that are too large or too small suggest a difference from the retained model [16]. Following [15], we measure the model utility using the zero-shot accuracy on nine standard LLM benchmarks to determine if the generated text remains meaningful and diverse. Additionally, we measure perplexity (PPL) on Wikitext [39]. More details about the experimental setup and implementation are in Appendix C.2. FLAT consistently ranks in the top two across three metrics. Table 3 shows that FLAT ranks in the top two across the three primary metrics, with particularly strong performance in KL f-divergence function. Our method achieves scores close to those of the retained model in terms of the average accuracy across nine LLM benchmarks. FLAT approach achieves good trade-off. Our method demonstrates strong unlearning efficiency while preserving model utility as shown in Table 3. Although NPO demonstrates the best forget quality, outperforming our method, it severely suffers from lower model utility, as reflected by its PPL score. PO, while also using example responses, has the highest PPL and a weak forgetting performance (FQ Gap), indicating the ineffectiveness of learning the example responses in a naive manner. These results highlight the effectiveness of our method in balancing forget quality and model utility, even without an explicit retaining term in the loss function. We also include additional results using Llama 2 (Table 9) in Appendix D.1. Table 3: Performance of our method and the baseline methods on Harry Potter dataset using OPT-2.7B. FLAT consistently ranks in the top two in terms of similarity to the retained model, measured by Forget Quality Gap (FQ Gap), while also generating meaningful and diverse outputs, as reflected by perplexity (PPL) and the average zero-shot accuracy across nine LLM benchmarks (Avg. Acc.). The top two results across three main metrics are highlighted in blue. Metric FQ Gap(↓ ↓) PPL(↓ ↓) Avg.Acc.(↑ ↑) Original LLM 1.5346 15.6314 0.4762 Retained LLM 0.0 14.3190 0.4686 GA 2.7301 1.0984e71 0.3667 KL 2.7301 16.1592 0.4688 GD 2.3439 16.1972 0.4690 PO 2.1601 14.8960 0.4583 Mismatch 1.4042 15.7507 0.4679 LLMU 2.4639 15.8398 0.4656 DPO 2.2152 16.8396 0.4621 NPO 1.2611 19.6637 0.4644 FLAT (TV) 1.4047 15.5512 0.4681 FLAT (KL) 1.3238 15.5311 0.4694 FLAT (JS) 1.4025 15.5499 0.4693 FLAT (Pearson) 1.4089 15.5543 0.4686 4.3 Entity Unlearning Experiment Setup. The TOFU dataset [20] is a synthetic question-answering dataset focused on author biographies, aiming to enable a LLM to unlearn a portion of fictitious authors while retaining knowledge about the rest and real-world facts. The dataset includes 200 fake authors, each with 20 QA pairs, and experiments are conducted with 1% of these authors marked for unlearning. We use Llama2-7B [36], Phi-1.5B [40], and OPT-2.7B [35] as base LLM. Evaluation Metrics. To assess forget quality and model utility, we mainly use two metrics proposed alongside the TOFU dataset, Forget Quality (FQ) and Model Utility (MU) [20]. Forget quality, assessed via a p-value from a Kolmogorov-Smirnov test, measures how closely the unlearned model’s output matches a model trained only on the retained data in distribution. When the p-value is above 0.01, we say the forgetting is significant. Model utility is the aggregated model performance on held-out retain data regarding fictional authors, real-world author profiles, and world facts. We also report the ROUGE-L score on forget set and retain set. It’s important to note that for the forget set, a lower ROUGE-L score does not necessarily indicate better performance. Therefore, we highlight methods where the ROUGE-L score closely matches that of the retained model, as these are considered to produce better results. More metrics can be found in Appendix C.3.1. FLAT is always the best in preserving model utility. As seen in Table 4, it experiences almost no reductions in model utility compared to the original model. On LLaMA2-7B, although GA and KL achieve strong ROUGE-L scores on the retain dataset, their forgetting performance is poor. Similarly, on Phi-1.5B, GD performs well on the retain dataset’s ROUGE-L score, but its forgetting performance is insufficient, failing to exceed 0.01. FLAT achieves the top two Forget Quality under all three models. FLAT achieves a ROUGE score that is closest to the Retained LLM on forget dataset under Llama2-7B and Phi-1.5B. PO shows the best forgetting efficiency on Phi-1.5B, but its ROUGE-L scores on both the forget and retain datasets are lower compared to the retained LLM, indicating weaker model utility. A similar issue is observed with GA: while it excels in forgetting performance on the OPT-2.7B model, its model utility remains weaker. FLAT achieves the best trade-off. Our method consistently ranks in the top two across the primary metrics, achieving the best performance in MU. Specifically, KL f-divergence demonstrates strong results in both FQ and MU on LLama2-7B and Phi-1.5B models. Overall, all four f-divergence functions effectively balance forgetting efficiency and model utility. In summary, our method demonstrates the best model utility while achieving top-two results in forgetting performance. Table 4: Performance of our method and the baseline methods on TOFU dataset using three base LLMs, Llama2-7B, Phi-1.5B, and OPT-2.7B. FQ, MU, R-RL, F-RL represent forget quality, model utility, ROUGE-L on retain dataset and ROUGE-L on forget dataset respectively. We include the original LLM and retain LLM for reference. The top two results are highlighted in blue. Base LLM Llama2-7B Phi-1.5B OPT-2.7B Metric FQ MU F-RL(↓ ↓) R-RL FQ MU F-RL(↓ ↓) R-RL FQ MU F-RL(↓ ↓) R-RL Original LLM 4.4883e-06 0.6346 0.9851 0.9833 0.0013 0.5184 0.9607 0.9199 0.0013 0.5120 0.7537 0.7494 Retained LLM 1.0 0.6267 0.4080 0.9833 1.0 0.5233 0.4272 0.9269 1.0 0.5067 0.4217 0.7669 GA 0.0143 0.6333 0.4862 0.9008 0.0013 0.5069 0.5114 0.8048 0.2657 0.4639 0.4748 0.6387 KL 0.0068 0.6300 0.5281 0.9398 0.0030 0.5047 0.5059 0.8109 0.0286 0.4775 0.4810 0.6613 GD 0.0068 0.6320 0.4773 0.8912 0.0030 0.5110 0.4996 0.8496 0.0541 0.4912 0.4521 0.6603 PO 0.0541 0.6308 0.3640 0.8811 0.0286 0.5127 0.3170 0.7468 0.0068 0.4424 0.0589 0.4015 LLMU 0.0541 0.6337 0.4480 0.8865 0.0286 0.5110 0.3058 0.7270 0.0286 0.3296 0.0347 0.2495 DPO 0.0541 0.6359 0.5860 0.8852 0.0521 0.0519 0.3437 0.7349 0.0541 0.4264 0.0806 0.3937 NPO 0.0068 0.6321 0.4632 0.8950 0.0030 0.5057 0.5196 0.8000 0.0541 0.4788 0.4993 0.6490 FLAT (TV) 0.0541 0.6373 0.4391 0.8826 0.0143 0.5168 0.4689 0.8155 0.0068 0.5086 0.5217 0.7067 FLAT (KL) 0.0286 0.6393 0.5199 0.8750 0.0143 0.5180 0.4524 0.7850 0.0286 0.4838 0.4942 0.6974 FLAT (JS) 0.0541 0.6364 0.4454 0.8864 0.0068 0.5144 0.4572 0.8117 0.0541 0.4959 0.4938 0.7013 FLAT (Pearson) 0.0541 0.6374 0.4392 0.8857 0.0143 0.5175 0.4591 0.8099 0.0068 0.5093 0.5052 0.7059 4.4 MUSE-News Unlearning Experiment Setup. We focus on the task of unlearning on News corpus presented in [30]. News consists of BBC news articles [41] collected after August 2023. All articles are randomly divided into forget, retain, and holdout sets. We perform unlearning directly on the pre-trained models provided by the benchmark, following the corresponding experimental setup. Evaluation Metrics. We report the proposed four metrics, VerbMem on forget dataset, KnowMem on forget and retain dataset, and Privacy leakage (PrivLeak). We quantify the verbatim memorization VerbMem by prompting the model with the first l tokens from a sequence and comparing the continuation outputted by the model θ to the true continuation using the ROUGE-L F1 score [38]. We gather the model’s answers to questions and then average the ROUGE scores for all question-answer pairs in forget dataset or retain dataset to compute the knowledge memorization score KnowMem. The PrivLeak metric for a good unlearning algorithm should be close to zero, whereas an over/under-unlearning algorithm will get a large positive/negative metric. Table 5: Performace on MUSE benchmark using four criteria. We highlight results in blue if the unlearning algorithm satisfies the criterion and highlight it in red otherwise. For metrics on DfsubscriptD_fDitalic_f, lower values are preferred, while for metrics on DrsubscriptD_rDitalic_r, higher values are better. In terms of PrivLeak, the results should be close to 0. Large negative or positive values suggest that they may cause privacy leakage. VerbMem on DfsubscriptD_fDitalic_f (↓ ↓) KnowMem on DfsubscriptD_fDitalic_f (↓ ↓) KnowMem on DrsubscriptD_rDitalic_r (↑ ↑) PrivLeak Original LLM 58.4 - 63.9 - 55.2 - -99.8 Retained LLM 20.8 - 33.1 - 55.0 - 0.0 GA 0.0 (✔) 0.0 (✔) 0.0 (✘) 17.0 KL 27.4 (✘) 50.2 (✘) 44.8 (✔) -96.1 NPO 0.0 (✔) 0.0 (✔) 0.0 (✘) 15.0 NPO-RT 1.2 (✔) 54.6 (✘) 40.5 (✔) 105.8 Task Vectors 56.3 (✘) 63.7 (✘) 54.6 (✔) -99.8 GD 4.9 (✔) 27.5 (✔) 6.7 (✔) 109.4 WHP 19.7 (✔) 21.2 (✔) 28.3 (✔) 109.6 FLAT (TV) 1.7 (✔) 13.6 (✔) 31.8 (✔) 45.4 FLAT (KL) 0.0 (✔) 0.0 (✔) 0.0 (✘) 58.9 FLAT (JS) 1.9 (✔) 36.2 (✘) 38.5 (✔) 47.1 FLAT (Pearson) 1.6 (✔) 0.0 (✔) 0.2 (✔) 26.8 Experiment Results. FLAT effectively removes verbatim and knowledge memorization of forget dataset and achieve good knowledge memorization of retain dataset. But it can still reveal the membership of DfsubscriptD_fDitalic_f in DrsubscriptD_rDitalic_r. As shown in Table 5, GA, NPO, GD, WHP, and ours perform well in VerbMem and KnowMem on forget dataset, often reduing them even beyond the levels achieved by the retrained model. However, these reductions often come at the cost of significant utility loss on the retain set. Only GD, WHP and ours can perfome good in all memorization related metrics. And FLAT (TV) can achieve the lowest VerbMem and KnowMem on DfsubscriptD_fDitalic_f and the highest KnowMem on DrsubscriptD_rDitalic_r among the three methods. However, none of the methods can achieve satisiable results regarding to the privacy leakage. Since MUSE uses news data, which is highly time-dependent (and thus possibly non-i.i.d.), we advocate for cautious interpretation of the PrivLeak metric. A detailed analysis is provided in Appendix C.4.1. 4.5 Ablation Studies The Effectiveness of Re-weighting Mechanism. As FLAT (KL) demonstrates strong overall performance, we base our ablation study on KL divergence to explore the effectiveness of the implicit re-weighting mechanism within our loss adjustment. This study is conducted on the TOFU dataset using Llama2-7B. For the ablation study on the HP dataset, please refer to Table 13 in Appendix D.3. When using preferred template data for unlearning, we compare our method with DPO (without the term Mr⁢e⁢fsubscriptM_refMitalic_r e f) and SimPO, as outlined in Appendix C.1. All methods use the same data and have similar formulations, with two terms in the loss function; the only difference lies in the intrinsic re-weighting mechanism. As shown in Table 6, our method achieves the highest number of best results across 12 metrics. When replacing the IDK data with retain data, the results show that the retain version performs better on the Retain Set but worse on Real Authors and Real World compared to FLAT (KL). Since GD shares the same data usage and formulation as our method, except for the re-weighting mechanism and utilization of retain data, we compare the retain version to GD. The results show that our method achieves better performance on both the Retain Set and Forget Set, with the decline in Real Authors and Real World performance caused by the use of retain data. Table 6: Ablation Study of Re-weighting Mechanism on TOFU dataset using Llama2-7B under all metrics. We report ROUGE-L score (R-L), Probability (P), and Truth Ratio (TR) on all four subsets of the TOFU benchmark. Higher scores are better except ROUGE-L and Probability on the Forget Set. The best ones are in blue. Split Real Authors Real World Retain Set Forget Set Metric R-L P TR R-L P TR R-L P TR R-L(↓ ↓) P(↓ ↓) TR Study on Re-weighting Mechanism Using Template IDK Data DPO 0.9330 0.4939 0.6384 0.8917 0.4631 0.5646 0.8852 0.9623 0.4407 0.5860 0.8734 0.6240 DPO w/o Mr⁢e⁢fsubscriptM_refMitalic_r e f 0.9330 0.4899 0.6333 0.8917 0.4620 0.5642 0.8735 0.9579 0.4388 0.4021 0.8149 0.6326 SimPO 0.9330 0.4902 0.6335 0.8917 0.4624 0.5664 0.8758 0.9577 0.4388 0.4087 0.8128 0.6329 FLAT (KL) 0.9180 0.4992 0.6491 0.9060 0.4524 0.5609 0.8750 0.9679 0.4603 0.5199 0.7588 0.5895 Study on Re-weighting Mechanism Using Retain Data GD 0.9080 0.4728 0.6156 0.8718 0.4439 0.5833 0.8912 0.9657 0.4701 0.4773 0.4238 0.5619 FLAT (KL)-Retain 0.9180 0.4643 0.6099 0.8832 0.4356 0.5690 0.9241 0.9734 0.4697 0.4487 0.3342 0.5879 The Imapct of Good Answer Type. In the first step of our approach, we intend to generate good example responses for each forget sample. We primarily use the reject-based response "I don’t know" (denoted as IDK) as the default choice. In this section, we conduct an ablation study on data usage for FLAT to analyze how these good responses impact unlearning performance. Table 7 presents the ablation study of good answer type using three LLMs on TOFU dataset, comparing IDK with random normal responses (denoted as Normal). Table 14 in Appendix D.3 provides the ablation study of different good answer type using Llama2-7B on the HP dataset. Results indicate that using normal responses improves model utility on HP datasets and improves ROUGE-L Score on retain set on TOFU datasets, whereas using IDK responses yields better forgetting quality. Additionally, we observe that the performance across the four divergence functions is relatively similar. KL divergence, in particular, demonstrates more consistent results across the three datasets and three models, likely due to its reduced sensitivity to incorrect or bad answers. Table 7: Ablation Study of Good Answer Type using three LLMs on TOFU dataset. FQ, MU, R-RL, F-RL represent forget quality, model utility, ROUGE-L on retain dataset and ROUGE-L on forget dataset respectively. The best performance is in blue. Split Llama2-7B Phi-1.5B OPT-2.7B Metric FQ MU F-RL(↓ ↓) R-RL FQ MU F-RL(↓ ↓) R-RL FQ MU F-RL(↓ ↓) R-RL Original LLM 4.4883e-06 0.6346 0.9851 0.9833 0.0013 0.5184 0.9249 0.9293 0.0013 0.5120 0.7537 0.7494 Retained LLM 1.0 0.6267 0.4080 0.9833 1.0 0.5233 0.4272 0.9269 1.0 0.5067 0.4217 0.7669 FLAT (TV)-IDK 0.0541 0.6373 0.4391 0.8826 0.0143 0.5168 0.4689 0.8155 0.0068 0.5086 0.5217 0.7067 FLAT (KL)-IDK 0.0286 0.6393 0.5199 0.8750 0.0143 0.5180 0.4524 0.7850 0.0286 0.4838 0.2212 0.4853 FLAT (JS)-IDK 0.0541 0.6364 0.4454 0.8864 0.0068 0.5144 0.4572 0.8117 0.0541 0.4959 0.3104 0.5658 FLAT (Pearson)-IDK 0.0541 0.6374 0.4392 0.8857 0.0143 0.5175 0.4591 0.8099 0.0068 0.5093 0.5052 0.7059 FLAT (TV)-Normal 0.0068 0.6173 0.4941 0.9575 0.0068 0.5104 0.4827 0.8245 0.0030 0.5086 0.5646 0.7355 FLAT (KL)-Normal 0.0068 0.6162 0.6273 0.9719 0.0068 0.5177 0.5377 0.8575 0.0030 0.5082 0.5642 0.7474 FLAT (JS)-Normal 0.0143 0.6178 0.4910 0.9560 0.0013 0.5068 0.5538 0.7313 0.0013 0.5068 0.5538 0.7313 FLAT (Pearson)-Normal 0.0068 0.6186 0.4972 0.9546 0.0030 0.5094 0.5554 0.7343 0.0030 0.5094 0.5554 0.7343 5 Conclusion In this paper, we address the limitations of existing LLM unlearning methods, which often rely on the retain data or a reference LLM for response calibration. To overcome these challenges, we propose FLAT (Forget data only Loss AdjustmenT), a "flat" loss adjustment approach that eliminates the need for retain data or a reference model. By optimizing the f-divergence between the template and forget responses, FLAT offers a clear and theoretically grounded solution for balancing forget quality with model utility in LLM unlearning. Through extensive experiments on three key unlearning tasks: copyrighted content unlearning on the Harry Potter dataset, the MUSE benchmark, and entity unlearning on the TOFU dataset, we demonstrate the superior performance of FLAT. Our method consistently achieves high unlearning efficiency while preserving overall model utility, showcasing its effectiveness in addressing both practical and theoretical challenges in LLM unlearning. References [1] Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. Copyright violations and large language models. arXiv preprint arXiv:2310.13771, 2023. [2] Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from llms? objectives for defending against extraction attacks. arXiv preprint arXiv:2309.17410, 2023. [3] Fabio Motoki, Valdemar Pinho Neto, and Victor Rodrigues. More human than human: measuring chatgpt political bias. Public Choice, 198(1):3–23, 2024. [4] Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. Unlearning bias in language models by partitioning gradients. In Findings of the Association for Computational Linguistics: ACL 2023, pages 6032–6048, 2023. [5] Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. arXiv preprint arXiv:2310.10683, 2023. [6] Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning. arXiv preprint arXiv:2403.03218, 2024. [7] Clark Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, et al. Identifying and mitigating the security risks of generative ai. Foundations and Trends® in Privacy and Security, 6(1):1–52, 2023. [8] Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius. The european union general data protection regulation: what it is and what it means. Information & Communications Technology Law, 28(1):65–98, 2019. [9] Yinzhi Cao and Junfeng Yang. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pages 463–480. IEEE, 2015. [10] Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, PRANAY SHARMA, Sijia Liu, et al. Model sparsity can simplify machine unlearning. Advances in Neural Information Processing Systems, 36, 2024. [11] Chongyu Fan, Jiancheng Liu, Yihua Zhang, Dennis Wei, Eric Wong, and Sijia Liu. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. arXiv preprint arXiv:2310.12508, 2023. [12] Zonglin Di, Zhaowei Zhu, Jinghan Jia, Jiancheng Liu, Zafar Takhirov, Bo Jiang, Yuanshun Yao, Sijia Liu, and Yang Liu. Label smoothing improves machine unlearning. arXiv preprint arXiv:2406.07698, 2024. [13] Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethinking machine unlearning for large language models. arXiv preprint arXiv:2402.08787, 2024. [14] Ronen Eldan and Mark Russinovich. Who’s harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238, 2023. [15] Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Rao Kompella, Sijia Liu, and Shiyu Chang. Reversing the forget-retain objectives: An efficient llm unlearning framework from logit difference. arXiv preprint arXiv:2406.08607, 2024. [16] Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding-corrupted prompts. arXiv preprint arXiv:2406.07933, 2024. [17] Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few shot unlearners. arXiv preprint arXiv:2310.07579, 2023. [18] Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. Snap: Unlearning selective knowledge in large language models with negative instructions. arXiv preprint arXiv:2406.12329, 2024. [19] Jiaao Chen and Diyi Yang. Unlearn what you want to forget: Efficient unlearning for llms. arXiv preprint arXiv:2310.20150, 2023. [20] Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary C Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for llms. arXiv preprint arXiv:2401.06121, 2024. [21] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024. [22] Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. arXiv preprint arXiv:2404.05868, 2024. [23] Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. In Conference on Lifelong Learning Agents, pages 243–254. PMLR, 2022. [24] Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504, 2022. [25] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [26] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural samplers using variational divergence minimization. Advances in neural information processing systems, 29, 2016. [27] Jiaheng Wei and Yang Liu. When optimizing $f$-divergence is robust with label noise. In International Conference on Learning Representations, 2021. [28] Taiji Suzuki, Masashi Sugiyama, Jun Sese, and Takafumi Kanamori. Approximating mutual information by maximum likelihood density ratio estimation. In New challenges for feature selection in data mining and knowledge discovery, pages 5–20. PMLR, 2008. [29] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11):5847–5861, 2010. [30] Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models. arXiv preprint arXiv:2407.06460, 2024. [31] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022. [32] J.K. Rowling. Harry Potter and the Sorcerer’s Stone. Scholastic, New York, 1997. [33] Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power of second-order optimization for llm unlearning. arXiv preprint arXiv:2404.18239, 2024. [34] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. [35] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. [36] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. [37] Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002. [38] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. [39] Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843, 2016. [40] Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. Textbooks are all you need i: phi-1.5 technical report. arXiv preprint arXiv:2309.05463, 2023. [41] Yucheng Li, Frank Geurin, and Chenghua Lin. Avoiding data contamination in language model evaluation: Dynamic test construction with latest materials. arXiv preprint arXiv:2312.12343, 2023. [42] Jiaheng Wei, Zuyue Fu, Yang Liu, Xingyu Li, Zhuoran Yang, and Zhaoran Wang. Sample elicitation. In International Conference on Artificial Intelligence and Statistics, pages 2692–2700. PMLR, 2021. [43] Yu Meng, Mengzhou Xia, and Danqi Chen. Simpo: Simple preference optimization with a reference-free reward. arXiv preprint arXiv:2405.14734, 2024. [44] Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, and Chen Wang. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7766–7775, 2023. [45] Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. Model sparsification can simplify machine unlearning. arXiv preprint arXiv:2304.04934, 2023. [46] Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, and Eleni Triantafillou. Towards unbounded machine unlearning. Advances in Neural Information Processing Systems, 36, 2024. [47] Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 12 2023. [48] Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019. [49] Ido Dagan, Oren Glickman, and Bernardo Magnini. The pascal recognising textual entailment challenge. In Machine learning challenges workshop, pages 177–190. Springer, 2005. [50] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019. [51] Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021. [52] François Chollet. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019. [53] Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018. [54] Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020. [55] Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958, 2021. [56] Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference attacks work on large language models? arXiv preprint arXiv:2402.07841, 2024. [57] Pratyush Maini, Hengrui Jia, Nicolas Papernot, and Adam Dziedzic. Llm dataset inference: Did you train on my dataset? arXiv preprint arXiv:2406.06443, 2024. [58] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [59] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. [60] Yujia Bao, Ankit Parag Shah, Neeru Narang, Jonathan Rivers, Rajeev Maksey, Lan Guan, Louise N Barrere, Shelley Evenson, Rahul Basole, Connie Miao, et al. Harnessing business and media insights with large language models. arXiv preprint arXiv:2406.06559, 2024. [61] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022. [62] Minghao Liu, Jiaheng Wei, Yang Liu, and James Davis. Do humans and machines have the same eyes? human-machine perceptual differences on image classification. arXiv preprint arXiv:2304.08733, 2023. [63] Minghao Liu, Zonglin Di, Jiaheng Wei, Zhongruo Wang, Hengxiang Zhang, Ruixuan Xiao, Haoyu Wang, Jinlong Pang, Hao Chen, Ankit Shah, et al. Automatic dataset construction (adc): Sample collection, data curation, and beyond. arXiv preprint arXiv:2408.11338, 2024. [64] Hongyi Guo, Yuanshun Yao, Wei Shen, Jiaheng Wei, Xiaoying Zhang, Zhaoran Wang, and Yang Liu. Human-instruction-free llm self-alignment with limited samples. arXiv preprint arXiv:2401.06785, 2024. [65] Yang Liu and Hongyi Guo. Peer loss functions: Learning from noisy labels without knowing noise rates. In International conference on machine learning, pages 6226–6236. PMLR, 2020. [66] Jiaheng Wei and Yang Liu. When optimizing f-divergence is robust with label noise. arXiv preprint arXiv:2011.03687, 2020. [67] Jiaheng Wei, Hangyu Liu, Tongliang Liu, Gang Niu, Masashi Sugiyama, and Yang Liu. To smooth or not? when label smoothing meets noisy labels. arXiv preprint arXiv:2106.04149, 2021. [68] Jiaheng Wei, Zhaowei Zhu, Tianyi Luo, Ehsan Amid, Abhishek Kumar, and Yang Liu. To aggregate or not? learning with separate noisy labels. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2523–2535, 2023. [69] Jiaheng Wei, Zhaowei Zhu, Hao Cheng, Tongliang Liu, Gang Niu, and Yang Liu. Learning with noisy labels revisited: A study using real-world human annotations. arXiv preprint arXiv:2110.12088, 2021. [70] Zhaowei Zhu, Jialu Wang, Hao Cheng, and Yang Liu. Unmasking and improving data credibility: A study with datasets for training harmless language models. arXiv preprint arXiv:2311.11202, 2023. [71] Jiaheng Wei, Yuanshun Yao, Jean-Francois Ton, Hongyi Guo, Andrew Estornell, and Yang Liu. Measuring and reducing llm hallucination without gold-standard answers via expertise-weighting. arXiv preprint arXiv:2402.10412, 2024. [72] Xiaobo Xia, Tongliang Liu, Bo Han, Nannan Wang, Mingming Gong, Haifeng Liu, Gang Niu, Dacheng Tao, and Masashi Sugiyama. Part-dependent label noise: Towards instance-dependent label noise. Advances in Neural Information Processing Systems, 33:7597–7610, 2020. [73] Zhaowei Zhu, Yiwen Song, and Yang Liu. Clusterability as an alternative to anchor points when learning with noisy labels. In International Conference on Machine Learning, pages 12912–12923. PMLR, 2021. [74] Zhaowei Zhu, Jialu Wang, and Yang Liu. Beyond images: Label noise transition matrix estimation for tasks with lower-quality features. In International Conference on Machine Learning, pages 27633–27653. PMLR, 2022. [75] Zhaowei Zhu, Zihao Dong, and Yang Liu. Detecting corrupted labels without training a model to predict. In International conference on machine learning, pages 27412–27427. PMLR, 2022. [76] Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. Depn: Detecting and editing privacy neurons in pretrained language models. arXiv preprint arXiv:2310.20138, 2023. [77] Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems, 36, 2024. [78] Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models. arXiv preprint arXiv:2402.15159, 2024. [79] Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306, 2024. [80] James Y Huang, Wenxuan Zhou, Fei Wang, Fred Morstatter, Sheng Zhang, Hoifung Poon, and Muhao Chen. Offset unlearning for large language models. arXiv preprint arXiv:2404.11045, 2024. [81] Tianle Gu, Kexin Huang, Ruilin Luo, Yuanqi Yao, Yujiu Yang, Yan Teng, and Yingchun Wang. Meow: Memory supervised llm unlearning via inverted facts. arXiv preprint arXiv:2409.11844, 2024. [82] Anmol Mekala, Vineeth Dorna, Shreya Dubey, Abhishek Lalwani, David Koleczek, Mukund Rungta, Sadid Hasan, and Elita Lobo. Alternate preference optimization for unlearning factual knowledge in large language models. arXiv preprint arXiv:2409.13474, 2024. [83] Andrei Muresanu, Anvith Thudi, Michael R Zhang, and Nicolas Papernot. Unlearnable algorithms for in-context learning. arXiv preprint arXiv:2402.00751, 2024. [84] Pratiksha Thaker, Yash Maurya, and Virginia Smith. Guardrail baselines for unlearning in llms. arXiv preprint arXiv:2403.03329, 2024. [85] Karuna Bhaila, Minh-Hao Van, and Xintao Wu. Soft prompting for unlearning in large language models. arXiv preprint arXiv:2406.12038, 2024. [86] Chongyang Gao, Lixu Wang, Chenkai Weng, Xiao Wang, and Qi Zhu. Practical unlearning for large language models. arXiv preprint arXiv:2407.10223, 2024. [87] Chongyu Fan, Jiancheng Liu, Alfred Hero, and Sijia Liu. Challenging forgets: Unveiling the worst-case forget sets in machine unlearning. arXiv preprint arXiv:2403.07362, 2024. [88] Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling sgd: Understanding factors influencing machine unlearning. In 2022 IEEE 7th European Symposium on Security and Privacy (EuroS&P), pages 303–319. IEEE, 2022. [89] Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11516–11524, 2021. [90] Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. Approximate data deletion from machine learning models. In International Conference on Artificial Intelligence and Statistics, pages 2008–2016. PMLR, 2021. [91] Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577, 2021. [92] Ga Wu, Masoud Hashemi, and Christopher Srinivasa. Puma: Performance unchanged model augmentation for training data removal. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 8675–8682, 2022. [93] Alexander Becker and Thomas Liebig. Evaluating machine unlearning via epistemic uncertainty. arXiv preprint arXiv:2208.10836, 2022. [94] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9304–9312, 2020. [95] Leonard Adolphs, Tianyu Gao, Jing Xu, Kurt Shuster, Sainbayar Sukhbaatar, and Jason Weston. The cringe loss: Learning what language not to model. arXiv preprint arXiv:2211.05826, 2022. [96] Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. Kga: A general machine unlearning framework based on knowledge gap alignment. arXiv preprint arXiv:2305.06535, 2023. Appendix Arrangement The Appendix is organized as follows. ∙ ∙ Section § A: Discussion of the broad impacts and limitations of our method. ∙ ∙ Section § B: Detailed theoretical illustrations and proofs. ∙ ∙ Section § B.1: Additional examples of loss adjustments under various f-divergence functions. ∙ ∙ Section § B.2: Derivation of the empirical loss objective. ∙ ∙ Section § C: Detailed experimental settings. ∙ ∙ Section § C.1:Revisits existing unlearning methods and unifies them under our general loss function framework, as described in Section § 2.2. ∙ ∙ Section § C.2, § C.3, § C.4: Detailed experimental settings, including evaluation metrics and implementation settings for the three unlearning tasks. ∙ ∙ Section § D: Additional experiments and discussions. ∙ ∙ Section § E: Related work. Appendix A Limitations and Broader Impacts A.1 Broad Impacts The proposed FLAT method for LLM unlearning has the potential to significantly advance ethical and responsible AI deployment, particularly in addressing key challenges such as privacy concerns, bias, and regulatory compliance. By enabling models to effectively forget specific data without compromising overall model utility, this approach directly addresses issues related to data privacy, including compliance with regulations like GDPR, which mandates data deletion upon user request. The ability to unlearn sensitive or copyrighted information, as demonstrated on datasets like Harry Potter and TOFU, ensures that AI models can be continually refined without propagating harmful or biased content. Furthermore, the reduced reliance on the retain data or a reference LLM makes FLAT more resource-efficient, lowering the computational and financial costs associated with large-scale unlearning. This opens up opportunities for wider adoption across industries and research institutions where access to retain data or additional model resources may be limited or not accessible. The implications of this work span multiple domains, including healthcare, finance, and education, where ethical considerations are paramount. A.2 Limitations One key limitation of our approach is the unsatisfactory performance in the privacy leakage evaluation on the MUSE dataset. While FLAT demonstrates strong unlearning efficiency and retains model utility across several benchmarks, it struggles to prevent privacy leakage. Note that all other tested methods suffer from the same issue, this suggests that further refinement is needed to strengthen the privacy-preserving aspects of existing LLM unlearning approaches. Future work could explore more robust strategies to address privacy leakage while maintaining the balance between unlearning performance and model utility. Appendix B Theoretical Illustration and Proofs B.1 Additional Examples for Loss Adjustments under More f-divergence Example 2: Jenson-Shannon (JS) For Jenson-Shannon f-divergence, we have f∗⁢(u)=−log⁡(2−eu),g∗⁢(v)=log⁡21+e−vformulae-sequencesuperscript2superscriptsuperscript21superscriptf^*(u)=- (2-e^u),g^*(v)= 21+e^-vf∗ ( u ) = - log ( 2 - eitalic_u ) , g∗ ( v ) = log divide start_ARG 2 end_ARG start_ARG 1 + e- v end_ARG, hence: g∗((ℙ(xf,ye;θ))−f∗(g∗((ℙ(xf,yf;θ)) g^*((P(x_f,y_e;θ))-f^*(g^*((P% (x_f,y_f;θ))g∗ ( ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g∗ ( ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) =log⁡21+e−ℙ⁢(xf,ye;θ)−[−log⁡(2−elog⁡21+e−ℙ⁢(xf,yf;θ))]absent21superscriptℙsubscriptsubscriptdelimited-[]2superscript21superscriptℙsubscriptsubscript = 21+e^-P(x_f,y_e;θ)- [-% (2-e 21+e^-P(x_f,y_f;θ) % ) ]= log divide start_ARG 2 end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_e ; θ ) end_ARG - [ - log ( 2 - eroman_log divide start_ARG 2 end_ARG start_ARG 1 + e start_POSTSUPERSCRIPT - blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG end_POSTSUPERSCRIPT ) ] =log⁡21+e−ℙ⁢(xf,ye;θ)+log⁡(2−elog⁡21+e−ℙ⁢(xf,yf;θ))absent21superscriptℙsubscriptsubscript2superscript21superscriptℙsubscriptsubscript = 21+e^-P(x_f,y_e;θ)+ % (2-e 21+e^-P(x_f,y_f;θ) )= log divide start_ARG 2 end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_e ; θ ) end_ARG + log ( 2 - eroman_log divide start_ARG 2 end_ARG start_ARG 1 + e start_POSTSUPERSCRIPT - blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG end_POSTSUPERSCRIPT ) =log⁡21+e−ℙ⁢(xf,ye;θ)+log⁡(2−21+e−ℙ⁢(xf,yf;θ))absent21superscriptℙsubscriptsubscript221superscriptℙsubscriptsubscript = 21+e^-P(x_f,y_e;θ)+ % (2- 21+e^-P(x_f,y_f;θ) )= log divide start_ARG 2 end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_e ; θ ) end_ARG + log ( 2 - divide start_ARG 2 end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG ) =log⁡21+e−ℙ⁢(xf,ye;θ)+log⁡(2⁢e−ℙ⁢(xf,yf;θ)1+e−ℙ⁢(xf,yf;θ))absent21superscriptℙsubscriptsubscript2superscriptℙsubscriptsubscript1superscriptℙsubscriptsubscript = 21+e^-P(x_f,y_e;θ)+ % ( 2e^-P(x_f,y_f;θ)1+e^-P(x_f,y% _f;θ) )= log divide start_ARG 2 end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_e ; θ ) end_ARG + log ( divide start_ARG 2 e- blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG start_ARG 1 + e- blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG ) =log⁡(4⁢e−ℙ⁢(xf,yf;θ)(1+e−ℙ⁢(xf,ye;θ))⁢(1+e−ℙ⁢(xf,yf;θ))).absent4superscriptℙsubscriptsubscript1superscriptℙsubscriptsubscript1superscriptℙsubscriptsubscript = ( 4e^-P(x_f,y_f;θ)(1+e^-% P(x_f,y_e;θ))(1+e^-P(x_f,y_f;θ))% ).= log ( divide start_ARG 4 e- blackboard_P ( xitalic_f , yitalic_f ; θ ) end_ARG start_ARG ( 1 + e- blackboard_P ( xitalic_f , yitalic_e ; θ ) ) ( 1 + e- blackboard_P ( xitalic_f , yitalic_f ; θ ) ) end_ARG ) . Example 3: Pearson For Pearson f-divergence, we have f∗⁢(u)=u24+u,g∗⁢(v)=vformulae-sequencesuperscriptsuperscript24superscriptf^*(u)= u^24+u,g^*(v)=vf∗ ( u ) = divide start_ARG u2 end_ARG start_ARG 4 end_ARG + u , g∗ ( v ) = v, hence: g∗(ℙ(xf,ye;θ))−f∗(g∗((ℙ(xf,yf;θ)) g^*(P(x_f,y_e;θ))-f^*(g^*((P% (x_f,y_f;θ))g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g∗ ( ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) =ℙ⁢(xf,ye;θ)−(ℙ⁢(xf,yf;θ)24+ℙ⁢(xf,yf;θ))absentℙsubscriptsubscriptℙsuperscriptsubscriptsubscript24ℙsubscriptsubscript =P(x_f,y_e;θ)- ( P(x_f,y% _f;θ)^24+P(x_f,y_f;θ) )= blackboard_P ( xitalic_f , yitalic_e ; θ ) - ( divide start_ARG blackboard_P ( xitalic_f , yitalic_f ; θ )2 end_ARG start_ARG 4 end_ARG + blackboard_P ( xitalic_f , yitalic_f ; θ ) ) =−ℙ⁢(xf,yf;θ)24−ℙ⁢(xf,yf;θ)+ℙ⁢(xf,ye;θ).absentℙsuperscriptsubscriptsubscript24ℙsubscriptsubscriptℙsubscriptsubscript =- P(x_f,y_f;θ)^24-P(x_f% ,y_f;θ)+P(x_f,y_e;θ).= - divide start_ARG blackboard_P ( xitalic_f , yitalic_f ; θ )2 end_ARG start_ARG 4 end_ARG - blackboard_P ( xitalic_f , yitalic_f ; θ ) + blackboard_P ( xitalic_f , yitalic_e ; θ ) . Example 4: KL For KL f-divergence, we have f∗⁢(u)=eu−1,g∗⁢(v)=vformulae-sequencesuperscriptsuperscript1superscriptf^*(u)=e^u-1,g^*(v)=vf∗ ( u ) = eitalic_u - 1 , g∗ ( v ) = v, hence: g∗((ℙ(xf,ye;θ))−f∗(g∗((ℙ(xf,yf;θ)) g^*((P(x_f,y_e;θ))-f^*(g^*((P% (x_f,y_f;θ))g∗ ( ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g∗ ( ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) =ℙ⁢(xf,ye;θ)−eℙ⁢(xf,yf;θ)−1.absentℙsubscriptsubscriptsuperscriptℙsubscriptsubscript1 =P(x_f,y_e;θ)-e^P(x_f,y_f;% θ)-1.= blackboard_P ( xitalic_f , yitalic_e ; θ ) - eblackboard_P ( xitalic_f , yitalic_f ; θ ) - 1 . B.2 The derivation of empirical loss function According to Eqn. (3.2), we have: L⁢(xf,ye,yf;θ)subscriptsubscriptsubscript L(x_f,y_e,y_f;θ)L ( xitalic_f , yitalic_e , yitalic_f ; θ ) =−[supg[g⁢(ℙ⁢(xf,ye;θ))−f∗⁢(g⁢(ℙ⁢(xf,yf;θ)))]]absentdelimited-[]subscriptsupremumdelimited-[]ℙsubscriptsubscriptsuperscriptℙsubscriptsubscript =- [ _g [g(P(x_f,y_e;θ))-f^*(g(% P(x_f,y_f;θ))) ] ]= - [ supitalic_g [ g ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ) ] ] =f∗⁢(g∗⁢(ℙ⁢(xf,yf;θ)))⏟Lf⁢(xf,yf;θ)⁢−g∗(ℙ(xf,ye;θ)))⏟Le⁢(xf,ye;θ). = f^*(g^*(P(x_f,y_f;θ)))_L_f% (x_f,y_f;θ) -g^*(P(x_f,y_e;θ)))_L_% e(x_f,y_e;θ).= under⏟ start_ARG f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ) end_ARGL start_POSTSUBSCRIPT f ( xitalic_f , yitalic_f ; θ ) end_POSTSUBSCRIPT under⏟ start_ARG - g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) ) end_ARGL start_POSTSUBSCRIPT e ( xitalic_f , yitalic_e ; θ ) end_POSTSUBSCRIPT . Given a dataset D=(xfj,yej,yfj)j∈[N]subscriptsuperscriptsubscriptsuperscriptsubscriptsuperscriptsubscriptdelimited-[]D=\(x_f^j,y_e^j,y_f^j)\_j∈[N]D = ( xitalic_fitalic_j , yitalic_eitalic_j , yitalic_fitalic_j ) j ∈ [ N ], where yesubscripty_eyitalic_e and yfsubscripty_fyitalic_f are preferred template and original forget responses to the forget prompt xfsubscriptx_fxitalic_f, we estimate ℙ⁢(xf,ye;θ),ℙ⁢(xf,yf;θ)ℙsubscriptsubscriptℙsubscriptsubscriptP(x_f,y_e;θ),P(x_f,y_f;θ)blackboard_P ( xitalic_f , yitalic_e ; θ ) , blackboard_P ( xitalic_f , yitalic_f ; θ ) via the following two quantities: ℙ⁢(xf,ye;θ):=∑i=1|ye|P⁢(hθ⁢(xf,ye,<i)=ye,i)|ye|,ℙ⁢(xf,yf;θ):=∑i=1|yf|P⁢(hθ⁢(xf,yf,<i)=yf,i)|yf|.formulae-sequenceassignℙsubscriptsubscriptsuperscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscriptsubscriptassignℙsubscriptsubscriptsuperscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscriptsubscriptP(x_f,y_e;θ):= _i=1^|y_e|P(h_θ(x_f,y% _e,<i)=y_e,i)|y_e|, (x_f,y_f;θ):= _i% =1^|y_f|P(h_θ(x_f,y_f,<i)=y_f,i)|y_f|.blackboard_P ( xitalic_f , yitalic_e ; θ ) := divide start_ARG ∑i = 1| yitalic_e | P ( hitalic_θ ( xitalic_f , yitalic_e , < i ) = yitalic_e , i ) end_ARG start_ARG | yitalic_e | end_ARG , blackboard_P ( xitalic_f , yitalic_f ; θ ) := divide start_ARG ∑i = 1| yitalic_f | P ( hitalic_θ ( xitalic_f , yitalic_f , < i ) = yitalic_f , i ) end_ARG start_ARG | yitalic_f | end_ARG . Empirically, we can obtain the loss function for the dataset D: LFLAT ⁢(θ)subscriptFLAT L_ FLAT (θ)LFLAT ( θ ) =−D[g∗(ℙ(xf,ye;θ)))−f∗(g∗(ℙ(xf,yf;θ)))] =-E_D [g^*(P(x_f,y_e;θ)))-f^% *(g^*(P(x_f,y_f;θ))) ]= - blackboard_ED [ g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) ) - f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) ) ] =−D[g∗(∑i=1|ye|∑k=1vye,i,k⋅hθ⁢(xf,ye,<i)k|ye|)−f∗(g∗(∑i=1|yf|∑k=1vyf,i,k⋅hθ⁢(xf,yf,<i)k|yf|)] =-E_D [g^*( _i=1^|y_e| _k=1% ^vy_e,i,k·h_θ(x_f,y_e,<i)_k|y_e|)-f^*(g^*(% _i=1^|y_f| _k=1^vy_f,i,k·h_θ(x_f,y_f,<% i)_k|y_f|) ]= - blackboard_ED [ g∗ ( divide start_ARG ∑i = 1| yitalic_e | ∑k = 1v yitalic_e , i , k ⋅ hitalic_θ ( xitalic_f , yitalic_e , < i )k end_ARG start_ARG | yitalic_e | end_ARG ) - f∗ ( g∗ ( divide start_ARG ∑i = 1| yitalic_f | ∑k = 1v yitalic_f , i , k ⋅ hitalic_θ ( xitalic_f , yitalic_f , < i )k end_ARG start_ARG | yitalic_f | end_ARG ) ] =−D[g∗(∑i=1|ye|hθ⁢(xf,ye,<i)|ye|)−f∗(g∗(∑i=1|yf|hθ⁢(xf,yf,<i)|yf|)]. =-E_D [g^*( _i=1^|y_e|h_θ% (x_f,y_e,<i)|y_e|)-f^*(g^*( _i=1^|y_f|h_θ(x% _f,y_f,<i)|y_f|) ].= - blackboard_ED [ g∗ ( divide start_ARG ∑i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) end_ARG start_ARG | yitalic_e | end_ARG ) - f∗ ( g∗ ( divide start_ARG ∑i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) end_ARG start_ARG | yitalic_f | end_ARG ) ] . Here, v is the vocabulary size, ye,i,ksubscripty_e,i,kyitalic_e , i , k is the k-th element of vector representing the i-th token in the good response yesubscripty_eyitalic_e, yf,i,ksubscripty_f,i,kyitalic_f , i , k is the k-th element of vector representing the i-th token in the forget response yfsubscripty_fyitalic_f. Additionally, hθ⁢(xf,ye,<i)ksubscriptℎsubscriptsubscriptsubscriptabsenth_θ(x_f,y_e,<i)_khitalic_θ ( xitalic_f , yitalic_e , < i )k and hθ⁢(xf,yf,<i)ksubscriptℎsubscriptsubscriptsubscriptabsenth_θ(x_f,y_f,<i)_khitalic_θ ( xitalic_f , yitalic_f , < i )k denote the k-th entry of the probability distribution for the correctly generated token. An example: KL For KL f-divergence, f∗⁢(u)=eu−1,g∗⁢(v)=vformulae-sequencesuperscriptsuperscript1superscriptf^*(u)=e^u-1,g^*(v)=vf∗ ( u ) = eitalic_u - 1 , g∗ ( v ) = v, hence, g∗(ℙ(xf,ye;θ))−f∗(g∗(ℙ(xf,yf;θ))=ℙ(xf,ye;θ)−eℙ⁢(xf,yf;θ)−1.g^*(P(x_f,y_e;θ))-f^*(g^*(P(x_f,y_f;% θ))=P(x_f,y_e;θ)-e^P(x_f,y_f;θ)% -1.g∗ ( blackboard_P ( xitalic_f , yitalic_e ; θ ) ) - f∗ ( g∗ ( blackboard_P ( xitalic_f , yitalic_f ; θ ) ) = blackboard_P ( xitalic_f , yitalic_e ; θ ) - eblackboard_P ( xitalic_f , yitalic_f ; θ ) - 1 . We have: LFLAT ⁢(θ)subscriptFLAT L_ FLAT (θ)LFLAT ( θ ) =−D⁢[∑i=1|ye|hθ⁢(xf,ye,<i)|ye|−e∑i=1|yf|hθ⁢(xf,yf,<i)|yf|−1].absentsubscriptdelimited-[]superscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscriptsuperscriptsuperscriptsubscript1subscriptsubscriptℎsubscriptsubscriptabsentsubscript1 =-E_D [ _i=1^|y_e|h_θ(x_f% ,y_e,<i)|y_e|-e _i=1^|y_f|h_θ(x_f,y_f,<i)% |y_f|-1 ].= - blackboard_ED [ divide start_ARG ∑i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) end_ARG start_ARG | yitalic_e | end_ARG - edivide start_ARG ∑i = 1 start_POSTSUPERSCRIPT | yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) end_ARG start_ARG | yitalic_f | end_ARG - 1 end_POSTSUPERSCRIPT ] . B.3 Proof of Theorem 3.4 Proof. Remember that we define: f^d⁢i⁢v(De||Df):=Ze∼De[g^(Ze)]−Zf∼Df[f∗(g^(Zf))], f_div(D_e||D_f):=E_Z_e D_e[% g(Z_e)]-E_Z_f D_f[f^*( g(Z_f))],over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ) := blackboard_EZ start_POSTSUBSCRIPT e ∼ Ditalic_e end_POSTSUBSCRIPT [ over start_ARG g end_ARG ( Zitalic_e ) ] - blackboard_EZ start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ f∗ ( over start_ARG g end_ARG ( Zitalic_f ) ) ] , and fd⁢i⁢v(e||f)=supg[e∼e[g(e)]−f∼f[f∗(g(f))]], f_div(D_e||D_f)= _g [% E_Z_e _e [g(Z_e) ]- % E_Z_f _f [f^*(g(Z_f)) ]% ],fitalic_d i v ( Ditalic_e | | Ditalic_f ) = supitalic_g [ blackboard_EZ start_POSTSUBSCRIPT e ∼ Ditalic_e end_POSTSUBSCRIPT [ g ( Zitalic_e ) ] - blackboard_EZ start_POSTSUBSCRIPT f ∼ Ditalic_f end_POSTSUBSCRIPT [ f∗ ( g ( Zitalic_f ) ) ] ] , we first prove the convergence of g^ gover start_ARG g end_ARG, and then the convergence of f^d⁢i⁢v(De||Df) f_div(D_e||D_f)over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ). For the ease of presentation, for any real-valued function ϱitalic-ϱ ϱ, we write e⁢(ϱ)=z∼e⁢[ϱ⁢(z)]subscriptsubscriptitalic-ϱsubscriptsimilar-tosubscriptdelimited-[]italic-ϱE_D_e( )=E_z _e [% (z) ]blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT ( ϱ ) = blackboard_Ez ∼ D start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ ϱ ( z ) ], f⁢(ϱ)=z∼f⁢[ϱ⁢(z)]subscriptsubscriptitalic-ϱsubscriptsimilar-tosubscriptdelimited-[]italic-ϱE_D_f( )=E_z _f [% (z) ]blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT ( ϱ ) = blackboard_Ez ∼ D start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ ϱ ( z ) ], De⁢(ϱ)=z∼De⁢[ϱ⁢(z)]subscriptsubscriptitalic-ϱsubscriptsimilar-tosubscriptdelimited-[]italic-ϱE_D_e( )=E_z D_e [ (z) ]blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT ( ϱ ) = blackboard_Ez ∼ D start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ ϱ ( z ) ], and Df⁢(ϱ)=z∼Df⁢[ϱ⁢(z)]subscriptsubscriptitalic-ϱsubscriptsimilar-tosubscriptdelimited-[]italic-ϱE_D_f( )=E_z D_f [ (z) ]blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT ( ϱ ) = blackboard_Ez ∼ D start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ ϱ ( z ) ]. Given any g~∈Φ~Φ g∈ ~ start_ARG g end_ARG ∈ Φ, according to Lemma C.1 in [42], and the fact that f∗superscriptf^*f∗ is Lipschitz continuous, we have: ‖g^−g∗‖L2⁢(f)2superscriptsubscriptnorm^superscriptsubscript2subscript2 || g-g^*||_L_2(D_f)^2| | over start_ARG g end_ARG - g∗ | |L start_POSTSUBSCRIPT 2 ( Ditalic_f ) end_POSTSUBSCRIPT2 ≾[De⁢[(g^−g∗)/2]−e⁢[(g^−g∗)/2]]precedes-or-equivalent-toabsentdelimited-[]subscriptsubscriptdelimited-[]^superscript2subscriptsubscriptdelimited-[]^superscript2 [E_D_e[( g-g^*)/2]-E_% D_e[( g-g^*)/2] ]≾ [ blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ ( over start_ARG g end_ARG - g∗ ) / 2 ] - blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ ( over start_ARG g end_ARG - g∗ ) / 2 ] ] −[Df⁢[f∗⁢((g^−g∗)/2)−f∗⁢(g∗)]−f⁢[f∗⁢((g^−g∗)/2)−f∗⁢(g∗)]].delimited-[]subscriptsubscriptdelimited-[]superscript^superscript2superscriptsuperscriptsubscriptsubscriptdelimited-[]superscript^superscript2superscriptsuperscript - [E_D_f[f^*(( g-g^*)/2)-f^*(% g^*)]-E_D_f[f^*(( g-g^*)/2)-f^*(g^*)]% ].- [ blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( ( over start_ARG g end_ARG - g∗ ) / 2 ) - f∗ ( g∗ ) ] - blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( ( over start_ARG g end_ARG - g∗ ) / 2 ) - f∗ ( g∗ ) ] ] . (5) By using the fact that the true density ratio Ze/ZfsubscriptsubscriptZ_e/Z_fZitalic_e / Zitalic_f is bounded below and above, hence, L2⁢(e)subscript2subscriptL_2(D_e)L2 ( Ditalic_e ) is indeed equivalent to L2⁢(f)subscript2subscriptL_2(D_f)L2 ( Ditalic_f ). Based on Eqn. (B.3), Lemma C.2 in [42], and the Lipschitz property of f∗superscriptf^*f∗, with probability at least 1−c1⁢exp⁡(−NrΦ/(2+rΦ)/c12)1subscript1superscriptsubscriptΦ2subscriptΦsuperscriptsubscript121-c_1 (-N^r_ /(2+r_ )/c_1^2)1 - c1 exp ( - Nitalic_rroman_Φ / ( 2 + rroman_Φ ) / c12 ), we have ‖g^−g∗‖L2⁢(f)2superscriptsubscriptnorm^superscriptsubscript2subscript2 || g-g^*||_L_2(D_f)^2| | over start_ARG g end_ARG - g∗ | |L start_POSTSUBSCRIPT 2 ( Ditalic_f ) end_POSTSUBSCRIPT2 ≾N−1/(rΦ+2).precedes-or-equivalent-toabsentsuperscript1subscriptΦ2 N^-1/(r_ +2).≾ N- 1 / ( rroman_Φ + 2 ) . (6) Note that we have: |f^d⁢i⁢v(De||Df)−fd⁢i⁢v(e||f)| | f_div(D_e||D_f)-f_div(D_e||% D_f) || over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ) - fitalic_d i v ( Ditalic_e | | Ditalic_f ) | ≤ ≤ |De⁢[g^−g∗]−e⁢[g^−g∗]|+|Df⁢[f∗⁢(g^)−f⁢(g∗)]−f⁢[f∗⁢(g^)−f∗⁢(g∗)]|subscriptsubscriptdelimited-[]^superscriptsubscriptsubscriptdelimited-[]^superscriptsubscriptsubscriptdelimited-[]superscript^superscriptsubscriptsubscriptdelimited-[]superscript^superscriptsuperscript |E_D_e[ g-g^*]-E_D_e% [ g-g^*]|+|E_D_f[f^*( g)-f(g^*)]-E_% D_f[f^*( g)-f^*(g^*)]|| blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ over start_ARG g end_ARG - g∗ ] - blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ over start_ARG g end_ARG - g∗ ] | + | blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( over start_ARG g end_ARG ) - f ( g∗ ) ] - blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( over start_ARG g end_ARG ) - f∗ ( g∗ ) ] | +|e⁢[g^−g∗]−f⁢[f∗⁢(g^)−f∗⁢(g∗)]|+|De⁢[g∗]−e⁢[g∗]|+|Df⁢[f∗⁢(g∗)]−f⁢[f∗⁢(g∗)]|subscriptsubscriptdelimited-[]^superscriptsubscriptsubscriptdelimited-[]superscript^superscriptsuperscriptsubscriptsubscriptdelimited-[]superscriptsubscriptsubscriptdelimited-[]superscriptsubscriptsubscriptdelimited-[]superscriptsuperscriptsubscriptsubscriptdelimited-[]superscriptsuperscript +|E_D_e[ g-g^*]-E_% D_f[f^*( g)-f^*(g^*)]|+|E_D_e[g^*]-% E_D_e[g^*]|+|E_D_f[f^*(g^*)]-% E_D_f[f^*(g^*)]|+ | blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ over start_ARG g end_ARG - g∗ ] - blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( over start_ARG g end_ARG ) - f∗ ( g∗ ) ] | + | blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ g∗ ] - blackboard_ED start_POSTSUBSCRIPT e end_POSTSUBSCRIPT [ g∗ ] | + | blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( g∗ ) ] - blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ f∗ ( g∗ ) ] | =Cons1+Cons2+Cons3+Cons4+Cons5.absentsubscriptCons1subscriptCons2subscriptCons3subscriptCons4subscriptCons5 =Cons_1+Cons_2+Cons_3+Cons_4+% Cons_5.= Cons1 + Cons2 + Cons3 + Cons4 + Cons5 . (7) By Lemma C.2 in [42], with probability at least 1−c1⁢exp⁡(−NrΦ/(2+rΦ)/c12)1subscript1superscriptsubscriptΦ2subscriptΦsuperscriptsubscript121-c_1 (-N^r_ /(2+r_ )/c_1^2)1 - c1 exp ( - Nitalic_rroman_Φ / ( 2 + rroman_Φ ) / c12 ), we have Cons1≾N−2/(rΦ+2).precedes-or-equivalent-tosubscriptCons1superscript2subscriptΦ2Cons_1 N^-2/(r_ +2).Cons1 ≾ N- 2 / ( rroman_Φ + 2 ) . Similar upper bound holds for Cons2subscriptCons2Cons_2Cons2. Following from Eqn. (6), with probability at least 1−c1⁢exp⁡(−NrΦ/(2+rΦ)/c12)1subscript1superscriptsubscriptΦ2subscriptΦsuperscriptsubscript121-c_1 (-N^r_ /(2+r_ )/c_1^2)1 - c1 exp ( - Nitalic_rroman_Φ / ( 2 + rroman_Φ ) / c12 ), we have Cons3≾N−1/(rΦ+2).precedes-or-equivalent-tosubscriptCons3superscript1subscriptΦ2Cons_3 N^-1/(r_ +2).Cons3 ≾ N- 1 / ( rroman_Φ + 2 ) . Applying Hoeffding’s inequality, with probability at least 1−c1⁢exp⁡(−NrΦ/(2+rΦ)/c12)1subscript1superscriptsubscriptΦ2subscriptΦsuperscriptsubscript121-c_1 (-N^r_ /(2+r_ )/c_1^2)1 - c1 exp ( - Nitalic_rroman_Φ / ( 2 + rroman_Φ ) / c12 ), we have Cons4≾N−1/(rΦ+2).precedes-or-equivalent-tosubscriptCons4superscript1subscriptΦ2Cons_4 N^-1/(r_ +2).Cons4 ≾ N- 1 / ( rroman_Φ + 2 ) . Similar upper bound holds for Cons5subscriptCons5Cons_5Cons5. Combining the five upper bounds for ConsisubscriptConsCons_iConsi where i∈[5]delimited-[]5i∈[5]i ∈ [ 5 ], with probability at least 1−c1⁢exp⁡(−NrΦ/(2+rΦ)/c12)1subscript1superscriptsubscriptΦ2subscriptΦsuperscriptsubscript121-c_1 (-N^r_ /(2+r_ )/c_1^2)1 - c1 exp ( - Nitalic_rroman_Φ / ( 2 + rroman_Φ ) / c12 ), we have |f^d⁢i⁢v(De||Df)−fd⁢i⁢v(e||f)|≾N−1rΦ+2. | f_div(D_e||D_f)-f_div(D_e||D_f% ) | N^- 1r_ +2.| over start_ARG f end_ARGd i v ( Ditalic_e | | Ditalic_f ) - fitalic_d i v ( Ditalic_e | | Ditalic_f ) | ≾ N- divide start_ARG 1 end_ARG start_ARG rroman_Φ + 2 end_ARG . ∎ Appendix C Detailed Experimental Setup Table 8: Summary of unlearning tasks, including base models, forget datasets, and evaluation metrics. Unlearning Task Base Model Forget Dataset Metrics Copyrighted Content Unlearning OPT-2.7B, Llama2-7B Harry Potter Series BLEU, ROUGE_L, PPL, Zero-shot Acc Entity Unlearning (TOFU) OPT-2.7B, Llama2-7B, Phi-1.5B TOFU-Forget01 Forget Quality (p-value), ROUGE_L on Forget set, Model Utility MUSE Benchmark Llama2-7B BBC News Corps verbatim and knowledge memorization on fsubscriptD_fDitalic_f, privacy leakage, Utility preservation Table 8 summarizes the experimental setups, including base models, forget and retain datasets, and evaluation metrics. C.1 Formulations for baseline methods In this section, we revisit existing unlearning objectives and unify them under our general loss function framework, as described in Section § 2.2. We provide formulations for GA, GD, KL, and PO, as presented in [20], as well as for Mismatch [16], LLMU [5], DPO[21], and NPO [22]. Additionally, we include the formulations for DPO without the Mr⁢e⁢fsubscriptM_refMitalic_r e f term and SimPO [43], as discussed in Section § 3.4. We also add formulations for Task Vectors [31] and Who’s Harry Potter (WHP)[14]. Fine-tuning on retain data Retraing from scratch is the gold standard for unlearning. HHowever, in real-world scenarios, retain data may not always be available, and retraining a LLM is highly resource-intensive. Alternatively, we can fine-tune the model using retain data for several epochs, which only involves performing gradient descent on DrsubscriptD_rDitalic_r. LFine-tune=1|Dr|⁢∑(xr,yr)∈Drℒ⁢(xr,yr;θ)⏟Retain LosssubscriptFine-tunesubscript⏟1subscriptsubscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptRetain LossL_ Fine-tune= 1|D_r| _(x_r,y_r)∈ D_% rL(x_r,y_r;θ)_Retain\; LossLFine-tune = under⏟ start_ARG divide start_ARG 1 end_ARG start_ARG | Ditalic_r | end_ARG ∑( x start_POSTSUBSCRIPT r , yitalic_r ) ∈ Ditalic_r end_POSTSUBSCRIPT L ( xitalic_r , yitalic_r ; θ ) end_ARGRetain Loss Gradient ascent (GA) GA is simple baselines commonly used in traditional machine unlearning settings [44, 45, 11, 46]. GA reverts the change of the gradient descent during the training with its opposite operation. The rationale of gradient ascent is that a subsequent maximization of prediction loss on the forget dataset DfsubscriptD_fDitalic_f would approximately "revert" the optimization on the forget dataset, thus unlearning DfsubscriptD_fDitalic_f and approximating a model trained on the retain dataset DrsubscriptD_rDitalic_r only. LGA=−1|Df|⁢∑(xf,yf)∈Dfℒ⁢(xf,yf;θ)⏟Forget LosssubscriptGAsubscript⏟1subscriptsubscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptForget LossL_ GA= - 1|D_f| _(x_f,y_f)∈ D_f% L(x_f,y_f;θ)_Forget\; LossLGA = under⏟ start_ARG - divide start_ARG 1 end_ARG start_ARG | Ditalic_f | end_ARG ∑( x start_POSTSUBSCRIPT f , yitalic_f ) ∈ Ditalic_f end_POSTSUBSCRIPT L ( xitalic_f , yitalic_f ; θ ) end_ARGForget Loss Gradient difference (GD) Gradient difference has been introduced as simple baseline method in [20].It combines fine-tuning and gradient ascent by compute the sum of the two loss terms. LGD=1|Dr|⁢∑(xr,yr)∈Drℒ⁢(xr,yr;θ)⏟Retain Loss⁢−1|Df|⁢∑(xf,yf)∈Dfℒ⁢(xf,yf;θ)⏟Forget LosssubscriptGDsubscript⏟1subscriptsubscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptRetain Losssubscript⏟1subscriptsubscriptsubscriptsubscriptsubscriptℒsubscriptsubscriptForget LossL_ GD= 1|D_r| _(x_r,y_r)∈ D_r% L(x_r,y_r;θ)_Retain\; Loss - 1% |D_f| _(x_f,y_f)∈ D_fL(x_f,y_f;θ)_% Forget\; LossLGD = under⏟ start_ARG divide start_ARG 1 end_ARG start_ARG | Ditalic_r | end_ARG ∑( x start_POSTSUBSCRIPT r , yitalic_r ) ∈ Ditalic_r end_POSTSUBSCRIPT L ( xitalic_r , yitalic_r ; θ ) end_ARGRetain Loss under⏟ start_ARG - divide start_ARG 1 end_ARG start_ARG | Ditalic_f | end_ARG ∑( x start_POSTSUBSCRIPT f , yitalic_f ) ∈ Ditalic_f end_POSTSUBSCRIPT L ( xitalic_f , yitalic_f ; θ ) end_ARGForget Loss KL minimization (KL) The KL minimization is adopted from [20] and involves a gradient ascent term for forgetting. It also minimizes the Kullback-Leibler (KL) divergence between the predictions on retain data DrsubscriptD_rDitalic_r of the reference model (the original model θosubscript _oθitalic_o) and the newly trained model (the unlearned model θ). This term aims to keep the unlearned model’s current output distribution on the retain dataset close to its pre-unlearning distribution on the retain samples. LKL=LGA⏟Forget Loss+1|Dr|⁢∑(xr,yr)∈Dr∑i=1|yr|KL⁡(hθ0⁢(xr,yr<i)∥hθ⁢(xr,yr<i))⏟Retain LosssubscriptKLsubscript⏟subscriptGAForget Losssubscript⏟1subscriptsubscriptsubscriptsubscriptsubscriptsuperscriptsubscript1subscriptKLconditionalsubscriptℎsubscript0subscriptsubscriptsubscriptℎsubscriptsubscriptRetain LossL_ KL= L_ GA_Forget\;Loss+% 1|D_r| _(x_r,y_r)∈ D_r _i=1^|y_r|% KL(h_ _0(x_r,y_r<i)\|h_θ(x_r,y_r<i))_% Retain\;LossLKL = under⏟ start_ARG LGA end_ARGForget Loss + under⏟ start_ARG divide start_ARG 1 end_ARG start_ARG | Ditalic_r | end_ARG ∑( x start_POSTSUBSCRIPT r , yitalic_r ) ∈ Ditalic_r end_POSTSUBSCRIPT ∑i = 1| yitalic_r | KL ( hitalic_θ start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT ( xitalic_r , yitalic_r < i ) ∥ hitalic_θ ( xitalic_r , yitalic_r < i ) ) end_ARGRetain Loss Preference optimization (PO) Preference Optimization (PO) differs from the traditional direct preference optimization approach as presented in [21] in that it combines the fine-tuning loss on DrsubscriptD_rDitalic_r with a term that teaches the model to respond with ’I don’t know’ to prompts from DfsubscriptD_fDitalic_f [20]. Here, DidksubscriptidkD_ idkDidk refers to an augmented forget dataset where the model’s response to the prompt is ’I don’t know.’ LPO=LFine-tune⏟Retain Loss+1|Didk|⁢∑xf,yi⁢d⁢k∈Didkℒ⁢(xf,yi⁢d⁢k;θ)⏟Custom LosssubscriptPOsubscript⏟subscriptFine-tuneRetain Losssubscript⏟1subscriptidksubscriptsubscriptsubscriptsubscriptidkℒsubscriptsubscriptCustom LossL_ PO= L_ Fine-tune_Retain\; Loss+% 1|D_ idk| _x_f,y_idk∈ D_ idk% L(x_f,y_idk;θ)_Custom\; LossLPO = under⏟ start_ARG LFine-tune end_ARGRetain Loss + under⏟ start_ARG divide start_ARG 1 end_ARG start_ARG | Didk | end_ARG ∑x start_POSTSUBSCRIPT f , yitalic_i d k ∈ Didk end_POSTSUBSCRIPT L ( xitalic_f , yitalic_i d k ; θ ) end_ARGCustom Loss Here, the Custom Loss utilizes the modified response to the forget prompt to ensure that the model rejects answering questions related to the forget data. Mismatch Mismatch has the same objective to PO, except it involves constructing a random combination of text sequences rdnsubscriptrdn Y_ rdnYrdn. Here, the second term in mismatch is the same as the second term in LLMU [5]. LMismatch=LFine-tune⏟Retain Loss+∑(xf,⋅)∈Df1|Yrdn|⁢∑yr⁢d⁢n∈Yr⁢d⁢nℒ⁢(xf,yr⁢d⁢n;θ)⏟Custom LosssubscriptMismatchsubscript⏟subscriptFine-tuneRetain Losssubscript⏟subscriptsubscript⋅subscript1subscriptrdnsubscriptsubscriptsubscriptℒsubscriptsubscriptCustom LossL_ Mismatch= L_ Fine-tune_Retain\; % Loss+ _(x_f,·)∈ D_f 1|Y_ rdn|% _y_rdn∈ Y_rdnL(x_f,y_rdn;θ)_Custom\;% LossLMismatch = under⏟ start_ARG LFine-tune end_ARGRetain Loss + under⏟ start_ARG ∑( x start_POSTSUBSCRIPT f , ⋅ ) ∈ Ditalic_f end_POSTSUBSCRIPT divide start_ARG 1 end_ARG start_ARG | Yrdn | end_ARG ∑y start_POSTSUBSCRIPT r d n ∈ Yitalic_r d n end_POSTSUBSCRIPT L ( xitalic_f , yitalic_r d n ; θ ) end_ARGCustom Loss LLMU [5] LLMU combines the GA term with two additional terms to learn 1) random completions YrdnsubscriptrdnY_ rdnYrdn from DrsubscriptrD_ rDr (constructed using prompts from DfsubscriptD_fDitalic_f) to facilitate unlearn and 2) DrsubscriptrD_ rDr to preserve performance. We use books with similar styles as DrsubscriptrD_ rDr in our experiments and construct YrdnsubscriptrdnY_ rdnYrdn using randomly sampled text sequences from DrsubscriptrD_ rDr. LLLMU=subscriptLLMUabsent L_ LLMU=LLLMU = −∑(xf,yf)∈Dfℒ⁢(xf,yf;θ)subscriptsubscriptsubscriptsubscriptℒsubscriptsubscript - _(x_f,y_f)∈ D_fL(x_f,y_f;θ)- ∑( x start_POSTSUBSCRIPT f , yitalic_f ) ∈ Ditalic_f end_POSTSUBSCRIPT L ( xitalic_f , yitalic_f ; θ ) +∑(xf,⋅)∈Df1|Yr⁢d⁢n|⁢∑yr⁢d⁢n∈Yr⁢d⁢nℒ⁢(xf,yr⁢d⁢n;θ)subscriptsubscript⋅subscript1subscriptsubscriptsubscriptsubscriptℒsubscriptsubscript + _(x_f,·)∈ D_f 1|Y_rdn| _y_rdn∈ Y% _rdnL(x_f,y_rdn;θ)+ ∑( x start_POSTSUBSCRIPT f , ⋅ ) ∈ Ditalic_f end_POSTSUBSCRIPT divide start_ARG 1 end_ARG start_ARG | Yitalic_r d n | end_ARG ∑y start_POSTSUBSCRIPT r d n ∈ Yitalic_r d n end_POSTSUBSCRIPT L ( xitalic_f , yitalic_r d n ; θ ) +∑(xr,yr)∈Dr∑i=1|yr|KL(hθo(xr,yr<i)||hθ(xr,yr<i)) + _(x_r,y_r)∈ D_r _i=1^|y_r|KL(h_ _o% (x_r,y_r<i)||h_θ(x_r,y_r<i))+ ∑( x start_POSTSUBSCRIPT r , yitalic_r ) ∈ Ditalic_r end_POSTSUBSCRIPT ∑i = 1| yitalic_r | K L ( hitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( xitalic_r , yitalic_r < i ) | | hitalic_θ ( xitalic_r , yitalic_r < i ) ) We have already unified LLMU in Section § 2.2 Direct preference optimization (DPO), DPO w/o Mr⁢e⁢fsubscriptM_refMitalic_r e f, SimPO See Section § 3.4 for more information. LDPO,⁢β⁢(θ)=−2β⁢D⁢[log⁡σ⁢(β⁢log⁢∏i=1|ye|hθ⁢(xf,ye,<i)⏟Custom Loss−βlog∏i=1|yf|hθ(xf,yf,<i))⏟Forget Loss⁢−Mr⁢e⁢f⏟Retain/ Custom Loss)].L_ DPO,β(θ)=- 2βE_D [ σ% ( β _i=1^|y_e|h_θ(x_f,y_e,<i)_% Custom \;Loss- β _i=1^|y_f|h_θ(x% _f,y_f,<i))_Forget\;Loss -M_ref_Retain/% Custom\;Loss ) ].LDPO, β ( θ ) = - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED [ log σ ( under⏟ start_ARG β log ∏i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) end_ARGCustom Loss - under⏟ start_ARG β log ∏i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) ) end_ARGForget Loss under⏟ start_ARG - Mitalic_r e f end_ARGRetain/ Custom Loss ) ] . LDPO⁢w/o⁢Mr⁢e⁢f,β⁢(θ)=−2β⁢D⁢[log⁡σ⁢(β⁢log⁢∏i=1|ye|hθ⁢(xf,ye,<i)⏟Custom Loss−βlog∏i=1|yf|hθ(xf,yf,<i))⏟Forget Loss)].L_ DPO\; w/o\;M_ref,β(θ)=- 2β% E_D [ σ ( β _i=1^|y_e|h_% θ(x_f,y_e,<i)_Custom \;Loss- β _% i=1^|y_f|h_θ(x_f,y_f,<i))_Forget\;Loss ) % ].LDPO w/o M start_POSTSUBSCRIPT r e f , β end_POSTSUBSCRIPT ( θ ) = - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED [ log σ ( under⏟ start_ARG β log ∏i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , < i ) end_ARGCustom Loss - under⏟ start_ARG β log ∏i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , < i ) ) end_ARGForget Loss ) ] . LSimPO,β(θ)=−2βD[logσ(β|ye|⁢log⁢∏i=1|ye|hθ⁢(xf,ye,i)⏟Custom Loss−β|yf|⁢log⁢∏i=1|yf|hθ⁢(xf,yf,i)⏟Forget Loss)−γ])]L_ SimPO,β(θ)=- 2βE_D [ % σ ( β|y_e| _i=1^|y_e|h_% θ(x_f,y_e,i)_Custom \; Loss - β|y_% f| _i=1^|y_f|h_θ(x_f,y_f,i)_Forget\;Loss% )-γ] ) ]LSimPO , β ( θ ) = - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED [ log σ ( under⏟ start_ARG divide start_ARG β end_ARG start_ARG | yitalic_e | end_ARG log ∏i = 1| yitalic_e | hitalic_θ ( xitalic_f , yitalic_e , i ) end_ARGCustom Loss under⏟ start_ARG - divide start_ARG β end_ARG start_ARG | yitalic_f | end_ARG log ∏i = 1| yitalic_f | hitalic_θ ( xitalic_f , yitalic_f , i ) end_ARGForget Loss ) - γ ] ) ] where γ is the target reward margin. Negative preference optimization (NPO) [22] NPO incorporates only the losing response term in DPO [21], penalizing only the prompt-response pairs in fsubscriptD_fDitalic_f.In the formulation below, β represents the inverse-temperature, πθsubscript _θπitalic_θ is the prediction probability of LLM θ. NPO also has two extended versions that include either the KL term or a fine-tuning term on rsubscriptD_rDitalic_r to preserve model utility. LNPOsubscriptNPO L_ NPOLNPO =−2β⁢Df⁢[log⁡σ⁢(−β⁢l⁢o⁢g⁢πθ⁢(yf∣xf)πr⁢e⁢f⁢(yf∣xf))]absent2subscriptsubscriptdelimited-[]subscriptconditionalsubscriptsubscriptsubscriptconditionalsubscriptsubscript =- 2βE_D_f [ σ (-β log% _θ(y_f x_f) _ref(y_f x_f) ) ]= - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ log σ ( - β l o g divide start_ARG πitalic_θ ( yitalic_f ∣ xitalic_f ) end_ARG start_ARG πitalic_r e f ( yitalic_f ∣ xitalic_f ) end_ARG ) ] =−2β⁢Df⁢[log⁡σ⁢(β⁢log⁡πr⁢e⁢f⁢(yf∣xf)⏟Retain/Custom Loss⁢−β⁢log⁡πθ⁢(yf∣xf)⏟Forget Loss)]absent2subscriptsubscriptdelimited-[]subscript⏟subscriptconditionalsubscriptsubscriptRetain/Custom Losssubscript⏟subscriptconditionalsubscriptsubscriptForget Loss =- 2βE_D_f [ σ (% β _ref(y_f x_f)_Retain/Custom\;Loss% -β _θ(y_f x_f)_Forget\;Loss% ) ]= - divide start_ARG 2 end_ARG start_ARG β end_ARG blackboard_ED start_POSTSUBSCRIPT f end_POSTSUBSCRIPT [ log σ ( under⏟ start_ARG β log πitalic_r e f ( yitalic_f ∣ xitalic_f ) end_ARGRetain/Custom Loss under⏟ start_ARG - β log πitalic_θ ( yitalic_f ∣ xitalic_f ) end_ARGForget Loss ) ] LNPO-KLsubscriptNPO-KL L_ NPO-KLLNPO-KL =LNPO+LKLabsentsubscriptNPOsubscriptKL =L_ NPO+L_ KL= LNPO + LKL LNPO-RTsubscriptNPO-RT L_ NPO-RTLNPO-RT =LNPO+LFine-tuneabsentsubscriptNPOsubscriptFine-tune =L_ NPO+L_ Fine-tune= LNPO + LFine-tune Task Vectors [14] The taks vector is derived by calculating the weight difference between the original LLM θosubscript _oθitalic_o and a reinforce LLM θr⁢e⁢i⁢n⁢f⁢o⁢r⁢c⁢esubscript _reinforceθitalic_r e i n f o r c e, which is the model trained on DfsubscriptD_fDitalic_f until it over-fits. This method then subtract this task vector from the original LLM’ weights, intuitively moving the model away from the direction it used to adapt to DfsubscriptD_fDitalic_f. The weights of unlearned model can be obtained as: θ=θo−(θr⁢e⁢i⁢n⁢f⁢o⁢r⁢c⁢e−θo)subscriptsubscriptsubscriptθ= _o-( _reinforce- _o)θ = θitalic_o - ( θitalic_r e i n f o r c e - θitalic_o ) WHP [14] WHP defines the unlearned model θ as the interpolation between the original model θosubscript _oθitalic_o and the reinforced model θr⁢e⁢i⁢n⁢f⁢o⁢r⁢c⁢esubscript _reinforceθitalic_r e i n f o r c e. Let pθ(⋅|x)p_θ(·|x)pitalic_θ ( ⋅ | x ) denote the token distribution parametrized by the model θ when given a prompt x as input. Then, for any input x, WHP samples the next token from: pθ(⋅|x)=pθo(⋅|x)−α(pθr⁢e⁢i⁢n⁢f⁢o⁢r⁢c⁢e(⋅|x)−pθo(⋅|x))p_θ(·|x)=p_ _o(·|x)-α(p_ _reinforce(% ·|x)-p_ _o(·|x))pitalic_θ ( ⋅ | x ) = pitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( ⋅ | x ) - α ( pitalic_θ start_POSTSUBSCRIPT r e i n f o r c e end_POSTSUBSCRIPT ( ⋅ | x ) - pitalic_θ start_POSTSUBSCRIPT o end_POSTSUBSCRIPT ( ⋅ | x ) ) where α is a hyperparameter that controls the interpolation between the two models. C.2 Copyrighted Unlearning on HP C.2.1 Evaluation Metrics We use two text similarity metrics to evaluate our models. In each case, the original copyrighted text serves as the reference, and we calculate the similarity between this reference and the text generated by the LLM. ROUGE-L For the forget dataset, we compute the ROUGE-L recall score [38] between the ground truth responses (the forget responses) and the text generated by the model after unlearning. BLEU Similarly, we compute the BLEU score [37] for the forget dataset, comparing the ground truth responses to the model’s output after unlearning. A retained model that has never seen the reference text should score low on both metrics, and a successfully unlearned model should perform similarly. For Harry Potter datasets, we evaluate similarity using the first 600 generated tokens as per [33]. Perplexity (PPL) We assess text fluency and diversity by computing perplexity on the Wikitext [39] using the LM Evaluation Harness [47]. A model with lower perplexity on the fine-tuned data suggests the generated text remains meaningful. Zero-shot Accuracy We evaluate zero-shot accuracy across various tasks, including BoolQ [48], RTE [49], HellaSwag [50], Winogrande [51], ARC-Challenge [52], ARC-Easy [52], OpenBookQA [53], Piqa [54], and TruthfulQA [55]. The mean accuracy across these diverse tasks was computed and reported as a comprehensive measure of model utility after unlearning. The higher the average accuracy, the better the results. C.2.2 Implementation Setting. To demonstrate the copyright removal task, we undertake the fine-tuning of all the models using the complete Harry Potter series. The finetuning procedure for the OPT-2.7B and Llama2-7B models involve a learning rate of 1e-5 and a batch size of 2. AdamW serves as the optimizer for preparing these models. For baseline methods, we set the batch size and learning rate to be the same as in their original papers, and fine-tune for 5 epochs using AdamW optimizer. For our method, we use the same training hyper-parameters as baseline but set the learning rate to be 2e-7. C.3 Entity Unlearning on TOFU C.3.1 Evaluation Metrics We utilize the original evaluation metrics designed in the original paper of the TOFU dataset [20]. Probability For each instance in the retain or forget set, we calculate the normalized conditional probability P⁢(a∣q)1/|a|superscriptconditional1P(a q)^1/|a|P ( a ∣ q )1 / | a | on the LLM subject to unlearning, where q represents the question, a is the answer, and |a||a|| a | denotes the number of tokens in the answer. For the real authors and world facts subsets, the dataset provides a set of five answers a0,a~1,a~2,a~3,a~4subscript0subscript~1subscript~2subscript~3subscript~4\a_0, a_1, a_2, a_3, a_4\ a0 , over~ start_ARG a end_ARG1 , over~ start_ARG a end_ARG2 , over~ start_ARG a end_ARG3 , over~ start_ARG a end_ARG4 , consisting of one correct answer a0subscript0a_0a0 and four perturbed answers that are incorrect. In this case, we compute the ratio P⁢(a0∣q)1/|a0|/∑i=14P⁢(a~i∣q)1/|a~i|superscriptconditionalsubscript01subscript0superscriptsubscript14superscriptconditionalsubscript~1subscript~P(a_0 q)^1/|a_0|/ _i=1^4P( a_i q)^1/| a_% i|P ( a0 ∣ q )1 / | a0 | / ∑i = 14 P ( over~ start_ARG a end_ARGi ∣ q )1 / | over~ start_ARG a end_ARGi |. Truth ratio The truth ratio is computed as the geometric mean of multiple perturbed (incorrect) answers’ (=a~1,a~2,…subscript~1subscript~2…A=\ a_1, a_2,...\A = over~ start_ARG a end_ARG1 , over~ start_ARG a end_ARG2 , … ) probabilities over the normalized conditional probability of the paraphrased answer a^ aover start_ARG a end_ARG. Rtruth=(∏i=1||P⁢(a~∣q)|1/a~i|)1/||P⁢(a^∣q)1/|a^|subscripttruthsuperscriptsuperscriptsubscriptproduct1superscriptconditional~1subscript~1superscriptconditional^1^R_truth= ( _i=1^|A|P( a q)^|1% / a_i| )^1/|A|P( a q)^1/| a|Rtruth = divide start_ARG ( ∏i = 1| A | P ( over~ start_ARG a end_ARG ∣ q )| 1 / over~ start_ARG a end_ARGi | )1 / | A | end_ARG start_ARG P ( over start_ARG a end_ARG ∣ q )1 / | over start_ARG a end_ARG | end_ARG For the real authors and world fact subsets, the original answer a is used in the denominator as no paraphrased answer is available. ROUGE-L For all subsets of TOFU, we compute the ROUGE-L recall score [38] between the ground truth responses (forget dataset) and the text generated by the model after unlearning. Model utility The model utility is aggregated as a harmonic mean over nine numbers: the answer probability, truth ratio, and ROUGE recall scores from each of the retain, real authors, and world facts subsets. A higher model utility is always preferred. Forget quality The forget quality is determined by calculating the p-value from a Kolmogorov-Smirnov (KS) test, which compares two distributions: the truth ratio of the retained model and the truth ratio of the unlearned model on the forget set. A higher p-value suggests that the null hypothesis — that the distributions of the truth ratios from both models are identical — cannot be rejected, indicating that the retained and unlearned models behave similarly. C.3.2 Implementation Setting. For all LLM unlearning methods, we set the batch size to be 32 following previous works [20, 22, 15] and use consistent learning rates for each model. For OPT-2.7B and Phi-1.5B, we fine-tune the pre-trained models for 5 epochs using learning rate of 2e-5 to obtain the original model. Similarly, we fine-tune Llama2-7B for the same duration with a learning rate of 1e-5. AdamW serve as the optimizer for preparing these models. The unlearning process for all methods, including ours, employs the same learning rate as used during fine-tuning the original models. For all experiments on the TOFU dataset, the training hyperparameters remain consistent across models of the same type. C.4 MUSE-News Unlearning C.4.1 Evaluation Metrics Note on PrivLeak metric The PrivLeak metric used in [30] is derived from Min-K% Prob, a membership inference attack method for LLMs. Formally, it is calculated as: PrivLeak=AUC⁡(funlearn;Dforget,Dholdout)−AUC⁡(fretrain;Dforget,Dholdout)AUC⁡(fretrain;Dforget,Dholdout)PrivLeakAUCsubscriptunlearnsubscriptforgetsubscriptholdoutAUCsubscriptretrainsubscriptforgetsubscriptholdoutAUCsubscriptretrainsubscriptforgetsubscriptholdoutPrivLeak= AUC (f_unlearn;D_% forget,D_holdout )-AUC (f_retrain;% D_forget,D_holdout )AUC (f_% retrain;D_forget,D_holdout )PrivLeak = divide start_ARG AUC ( funlearn ; Dforget , Dholdout ) - AUC ( fretrain ; Dforget , Dholdout ) end_ARG start_ARG AUC ( fretrain ; Dforget , Dholdout ) end_ARG where the AUC score refers to the standard AUC-ROC score between DforgetsubscriptforgetD_forgetDforget and DholdoutsubscriptholdoutD_holdoutDholdout. While this method indeed discriminates between the forget and holdout distributions as a measure of successful unlearning, it is highly dependent on the data selected for evaluation. Specifically, Min-K% Prob has been shown to yield random-guess accuracy due to modern LLMs being trained on large pretraining corpora for only a small number of iterations, causing fuzzy boundary between members and non-members [56]. Furthermore, [57] demonstrate that Min-K% Prob results in 1) high variance depending on the random selection of the dataset used for evaluation, 2) better performance when the two subsets (in our case, forget and holdout) are not drawn from the same distribution, and 3) empirically overestimated false positives. The latter finding suggests that the distribution gap (i.e., temporal shift, which is also identified by [56]) acts as a confounding factor in the discrimination process, since the forget set and holdout set may differ in more than one dimension. Given that MUSE [30] uses news data, which is highly time-dependent (and thus possibly non-i.i.d.), we advocate for cautious interpretation of the PrivLeak metric. Appendix D Experimental Results D.1 Copyrighted Unlearning on HP dataset Performance using Llama2-7B on HP dataset Table 9 indicates that our method consistently places within the top three across the primary metrics, with TV f-divergence showing the best performance. LLMU achieves the best forgetting effect, comparable to our method, but its model utility is inferior. PO again demonstrates the highest PPL but exhibits poor forgetting performance. While DPO shows good model utility, the difference between our method and DPO in PPL is minimal. Unlike other methods, PO directly leverages fine-tune loss, which helps preserve the model’s performance beyond the unlearning. These results highlight the effectiveness of our method, striking a better balance between forgetting quality and model utility. Table 9: Unlearning performance of Llama2-7B on the Harry Potter dataset. R-L and Avg. Acc. denote the ROUGE-L score and average zero-shot accuracy across nine LLM benchmarks. We include the original LLM and retained LLM for reference. Some methods, such as PO, LLMU, DPO, and NPO, exhibit strong performance in either forget quality or model utility, but underperform in the other. FLAT consistently ranks in the top three in terms of similarity to the retained model, measured by Forget Quality Gap (FQ Gap), while also generating meaningful and diverse outputs, as indicated by perplexity (PPL) and the average zero-shot accuracy (Avg. Acc.). The top three results across the three main metrics are highlighted in blue. Forget Quality Model Utility Metric BLEU(↓ ↓) BLEU Gap R-L(↓ ↓) R-L Gap FQ Gap PPL(↓ ↓) PPL Gap Avg.Acc. Acc. Gap Original LLM 4.0452 - 0.1487 - - 8.9524 - 0.5617 - Retained LLM 0.4903 - 0.0442 - - 8.7070 - 0.5599 - GA 0.0624 0.4279 0.0134 0.0308 0.4587 47.2769 -38.5699 0.5088 -0.0511 KL 0.0976 0.3927 0.0144 0.0298 0.4225 9.4336 -0.7266 0.5509 -0.0090 GD 0.0039 0.4864 0.0002 0.0440 0.5304 9.1797 -0.4727 0.4902 -0.0697 PO 0.0206 0.4697 0.0015 0.0427 0.5124 8.8364 -0.1294 0.5532 -0.0067 Mismatch 0.0670 0.4233 0.0028 0.0414 0.4647 8.9906 -0.2836 0.5593 -0.0056 LLMU 0.3033 0.1870 0.0317 0.0125 0.1985 9.0530 -0.3460 0.5503 -0.0096 DPO 0.7717 -0.2814 0.0552 -0.0110 0.2924 8.9597 -0.2527 0.5614 0.0015 NPO 0.9840 -0.4937 0.0656 -0.0214 0.5151 9.0397 -0.3327 0.5609 0.0010 FLAT (TV) 0.6770 -0.1867 0.0673 -0.0231 0.2098 8.9899 -0.2829 0.5592 -0.0007 FLAT (KL) 0.6829 -0.1926 0.0662 -0.0220 0.2146 8.9803 -0.2733 0.5572 -0.0027 FLAT (JS) 0.6890 -0.1987 0.0684 -0.0242 0.2229 8.9910 -0.2840 0.5574 -0.0025 FLAT (Pearson) 0.6930 -0.2027 0.0680 -0.0238 0.2265 8.9906 -0.2836 0.5580 -0.0019 HP Example Generations under our method FLAT The generated samples can be seen in Table 10. Table 10: Examples of OPT-2.7b’s original and generated response using our unlearning method on HP dataset. Prompt: Let’s see how you would complete this piece of text: You should have realized , ” he said as he saw Voldemort’s nostrils flare … but it all makes sense , Potter , and in ways that you do not understand ! “ Dumbledore was trying to keep Original response: the Elder Wand from me ! He intended that Snape should be the true master of the wand ! But I got there ahead of you … round his eyes whiten . “ It’s your one last chance , ” said Harry , “ it’s all you’ve got left . Generated response: me trapped, to stop Snape from catching me, to get hold of me when he knew Voldemort had sent Severus Snape to m … to speak to me! This is why Dumbledore wanted Snape to kill him once only I am Question: Let’s see how you would complete this piece of text: “ That’s all very well , but what if Montague’s permanently injured ? ” “ … “ She’l say I should’ve stopped them leaving , I should’ve grabbed the ends of Original response: their wands and made them stay. ” “ You’re not going to be in trouble, ” said Hermione, looking up at him … be in trouble at all. ” “ But I’m not? ” said Ron, looking very confused Generated reponse: those rope like Dumbledore. A few points deducted for this, a couple for your father’s attempt at distraction, now we’re even. ‘You could have stopped it by showing any of you … not so smart as Slytherin I’d like to hear how much Hermione’s taken from Stinks Parameter Study on Prompt Length for HP dataset Following [33, 14], we evaluate the forget quality using prompt lengths of 50, 100, 200, 300 on Harry Potter series dataset. Table 11 presents the parameter study of different prompt lengths for assessing forget quality on this dataset. In the main paper, we adopt the prompt length of 200, as suggested by [16]. Table 11: Parameter Study of different prompt length on Harry Potter book series dataset. We include the original LLM and retained LLM for reference. Split Prompt Length 50 Prompt Length 100 Prompt Length 200 Prompt Length 300 Metric BLEU(↓ ↓) ROUGE-L(↓ ↓) BLEU(↓ ↓) ROUGE-L(↓ ↓) BLEU(↓ ↓) ROUGE-L(↓ ↓) BLEU(↓ ↓) ROUGE-L(↓ ↓) OPT-2.7B Original LLM 3.4492 0.1203 3.7660 0.1273 4.1163 0.1484 3.4924 0.1551 Retain LLM 1.7350 0.0944 2.3427 0.0986 2.6072 0.1229 2.7479 0.1261 FLAT (TV) 0.8382 0.1090 0.9607 0.1206 1.1955 1.4117 1.4532 0.1628 FLAT (KL) 0.8363 0.1107 0.9867 0.1209 1.2743 1.3329 1.4714 0.1657 FLAT (JS) 0.8709 0.1101 0.9720 0.1213 1.1986 0.1290 1.4735 0.1635 FLAT (Pearson) 0.8430 0.1098 0.9624 0.1211 1.1917 1.4155 1.4501 0.1627 Llama2-7B Original LLM 0.0448 0.0049 0.3951 0.0254 4.0452 0.1487 0.2541 0.0275 Retain LLM 0.0917 0.0111 0.1664 0.0162 0.4903 0.0442 0.2542 0.0194 FLAT (TV) 0.0293 0.0045 0.2627 0.0276 0.6770 0.0673 0.2185 0.0251 FLAT (KL) 0.0308 0.0041 0.2592 0.0276 0.6829 0.0662 0.2217 0.0242 FLAT (JS) 0.0305 0.0045 0.2512 0.0279 0.6890 0.0684 0.2143 0.0253 FLAT (Pearson) 0.0310 0.0044 0.2573 0.0281 0.6930 0.0680 0.2163 0.0252 D.2 Entity Unlearning on TOFU TOFU Experimental Results using All Metrics Table 12 shows the performance on TOFU using three base LLMs, Llama2-7B, Phi-1.5B, and OPT-2.7 under all metrics. Table 12: Performance on TOFU dataset using three base LLMs, Llama2-7B, Phi-1.5B, and OPT-2.7 under all metrics. We report ROUGE-L score (R-L), Probability (P), and Truth Ratio (TR) on all four subsets of the TOFU benchmark. Higher scores are better except ROUGE-L and probability on the Forget Set. We include the original LLM and retained LLM for reference. The best two are highlighted in blue. Split Real Authors Real World Rerain Set Forget Set Metric R-L P TR R-L P TR R-L P TR R-L(↓ ↓) P(↓ ↓) TR Llama2-7B Original LLM 0.9350 0.4738 0.6210 0.8846 0.4355 0.5579 0.9833 0.9900 0.4662 0.9851 0.9898 0.5123 Retained LLM 0.9230 0.4645 0.6118 0.8932 0.4182 0.5449 0.9833 0.9902 0.4724 0.4080 0.1798 0.6939 GA 0.9030 0.4754 0.6233 0.8761 0.4432 0.5843 0.9008 0.9546 0.4695 0.4862 0.3566 0.5705 KL 0.9280 0.4652 0.6092 0.8803 0.4383 0.5691 0.9398 0.9705 0.4655 0.5281 0.5119 0.5626 GD 0.9080 0.4728 0.6156 0.8718 0.4439 0.5833 0.8912 0.9657 0.4701 0.4773 0.4238 0.5619 PO 0.9330 0.4850 0.6269 0.8917 0.4582 0.5602 0.8811 0.9627 0.4393 0.3640 0.8695 0.6318 LLMU 0.9330 0.4905 0.6344 0.8917 0.4603 0.5625 0.8865 0.9628 0.4391 0.4480 0.8606 0.6286 DPO 0.9330 0.4939 0.6384 0.8917 0.4631 0.5646 0.8852 0.9623 0.4407 0.5860 0.8734 0.6240 NPO 0.8930 0.4754 0.6218 0.8746 0.4466 0.5798 0.8950 0.9574 0.4680 0.4632 0.3664 0.5785 NPO-RT 0.8830 0.4758 0.6218 0.8746 0.4459 0.5805 0.8958 0.9588 0.4687 0.4519 0.3672 0.5791 FLAT (TV) 0.9180 0.4937 0.6459 0.8974 0.4505 0.5591 0.8826 0.9685 0.4607 0.4391 0.5314 0.6026 FLAT (KL) 0.9180 0.4992 0.6491 0.9060 0.4524 0.5609 0.8750 0.9679 0.4603 0.5199 0.7588 0.5895 FLAT (JS) 0.8980 0.4927 0.6460 0.8974 0.4508 0.5592 0.8864 0.9686 0.4607 0.4454 0.5183 0.6039 FLAT (Pearson) 0.9180 0.4932 0.6461 0.8974 0.4509 0.5583 0.8857 0.9684 0.4607 0.4392 0.5092 0.6037 Phi-1.5B Original LLM 0.4073 0.3744 0.4470 0.7503 0.4148 0.4982 0.9199 0.9238 0.4810 0.9607 0.9345 0.4839 Retained LLM 0.4240 0.3779 0.4539 0.7585 0.4090 0.4974 0.9269 0.9271 0.4855 0.4272 0.1686 0.6579 GA 0.4573 0.3638 0.4373 0.7541 0.3978 0.4741 0.8048 0.7748 0.4880 0.5114 0.3268 0.5099 KL 0.4273 0.3643 0.4370 0.7474 0.3997 0.4764 0.8109 0.8043 0.4889 0.5059 0.3342 0.5091 GD 0.3907 0.3726 0.4461 0.7605 0.4087 0.4931 0.8496 0.8900 0.4910 0.4996 0.4025 0.4952 PO 0.4240 0.3728 0.4449 0.7699 0.4190 0.5207 0.7468 0.8747 0.4596 0.3170 0.7362 0.5416 LLMU 0.4240 0.3720 0.4421 0.7785 0.4203 0.5197 0.7270 0.8678 0.4572 0.3058 0.7067 0.5453 DPO 0.0420 0.3713 0.4423 0.7785 0.4202 0.5205 0.7349 0.8712 0.4583 0.3437 0.6999 0.5393 NPO 0.4573 0.3619 0.4342 0.7417 0.3988 0.4761 0.8000 0.7856 0.4840 0.5196 0.3529 0.5119 NPO-RT 0.4473 0.3619 0.4340 0.7474 0.3998 0.4770 0.8024 0.7926 0.4851 0.5193 0.3527 0.5129 FLAT (TV) 0.4440 0.3695 0.4390 0.7742 0.4125 0.5040 0.8155 0.8858 0.4709 0.4689 0.4756 0.5395 FLAT (KL) 0.4440 0.3735 0.4464 0.7571 0.4175 0.5147 0.7850 0.8874 0.4666 0.4524 0.6285 0.5287 FLAT (JS) 0.4340 0.3703 0.4386 0.7588 0.4119 0.5045 0.8117 0.8850 0.4714 0.4572 0.4683 0.5390 FLAT (Pearson) 0.4540 0.3694 0.4389 0.7674 0.4117 0.5040 0.8099 0.8850 0.4711 0.4591 0.4672 0.5383 OPT-2.7B Original LLM 0.6687 0.3833 0.4393 0.6433 0.3701 0.4158 0.7494 0.8335 0.4992 0.7537 0.8237 0.5338 Retained LLM 0.6487 0.3735 0.4249 0.6278 0.3696 0.4185 0.7669 0.8399 0.4988 0.4217 0.1991 0.7097 GA 0.6390 0.3774 0.4375 0.5953 0.3644 0.4071 0.6387 0.4097 0.4972 0.4748 0.0722 0.6325 KL 0.6573 0.3775 0.4346 0.6463 0.3646 0.4071 0.6613 0.4707 0.4958 0.4810 0.1110 0.5902 GD 0.6453 0.3782 0.4336 0.6084 0.3613 0.3979 0.6603 0.6916 0.5162 0.4521 0.1701 0.5774 PO 0.4078 0.3874 0.4540 0.5135 0.3705 0.4207 0.4015 0.6922 0.4546 0.0589 0.5220 0.6037 LLMU 0.1528 0.3739 0.4215 0.3946 0.3695 0.4031 0.2495 0.6013 0.4305 0.0347 0.3967 0.6356 DPO 0.3478 0.3853 0.4498 0.4915 0.3708 0.4230 0.3937 0.6445 0.4375 0.0806 0.3931 0.6255 NPO 0.6573 0.3787 0.4343 0.6281 0.3681 0.4130 0.6490 0.4978 0.4870 0.4993 0.1355 0.5952 NPO-RT 0.5698 0.3646 0.4107 0.6264 0.3675 0.4239 0.4620 0.2459 0.4065 0.3627 0.1087 0.6716 FLAT (TV) 0.6737 0.3878 0.4457 0.6382 0.3715 0.4137 0.7067 0.8075 0.4858 0.5217 0.6368 0.5601 FLAT (KL) 0.6903 0.3802 0.4364 0.6369 0.3672 0.4108 0.6974 0.8078 0.4909 0.4942 0.6735 0.5515 FLAT (JS) 0.6723 0.3824 0.4385 0.6369 0.3706 0.4162 0.7013 0.7685 0.4911 0.4938 0.5662 0.5502 FLAT (Pearson) 0.6737 0.3873 0.4462 0.6467 0.3705 0.4150 0.7059 0.8069 0.4868 0.5052 0.6329 0.5597 D.3 Ablation Study Table 13 demonstrates the effectiveness of the re-weighting mechanism on the Harry Potter dataset. When using the preferred template data for unlearning, our method achieves strong forget quality and comparable model utility. When using retain data, our method outperforms GD, indicating that the re-weighting mechanism improves both unlearning efficiency and model utility. Table 14 presents the ablation study on the good answer types using Llama2-7B on the Harry Potter dataset. The results show that using normal responses enhances model utility on the HP dataset, while using IDK responses leads to better forgetting quality. Table 13: Ablation Study of the Re-weighting Mechanism using Llama2-7B on Harry Potter dataset. R-L and Avg. Acc. denote the ROUGE-L score and average zero-shot accuracy over nine LLM benchmarks. We include the original LLM and retain LLM for reference. The best ones are highlighted in blue. Forget Quality Model Utility Metric BLEU(↓ ↓) BLEU Gap R-L(↓ ↓) R-L Gap FQ Gap PPL(↓ ↓) PPL Gap Avg.Acc. Acc. Gap Original LLM 4.0452 3.5549 0.1487 0.2933 3.8482 8.9524 0.2444 0.5617 0.0018 Retained LLM 0.4903 0.0 0.0442 0.0 0.0 8.7070 0.0 0.5599 0.0 Study on Re-weighting Mechanism Using Retain Data GD 0.0039 0.4864 0.0002 0.0440 0.5304 9.1797 -0.4727 0.4902 -0.0697 FLAT (KL)-retain 0.2359 0.2544 0.0263 0.0179 0.2714 8.9948 -0.2878 0.5591 -0.0008 Study on Re-weighting Mechanism Using IDK Data DPO w/o Mr⁢e⁢fsubscriptM_refMitalic_r e f 0.7719 -0.2816 0.0523 -0.0081 0.2897 8.9674 -0.2604 0.5560 -0.0039 SimPO 0.6876 -0.1973 0.0552 -0.0110 0.2723 8.9927 -0.2857 0.5593 -0.0006 FLAT (KL) 0.6829 -0.1926 0.0662 -0.0220 0.2146 8.9803 -0.2733 0.5572 -0.0027 Table 14: Ablation Study of the good answer type using Llama2-7B on Harry Potter dataset. R-L and Avg. Acc. denote the ROUGE-L score and average zero-shot accuracy over nine LLM benchmarks. We include the original LLM and retain LLM for reference. The best ones are highlighted in blue. Forget Quality Model Utility Metric BLEU(↓ ↓) BLEU Gap R-L(↓ ↓) R-L Gap FQ Gap PPL(↓ ↓) PPL Gap Avg.Acc. Acc. Gap Original LLM 4.0452 - 0.1487 - - 8.9524 - 0.5617 - Retained LLM 0.4903 - 0.0442 - - 8.7070 - 0.5599 - FLAT (TV)-IDK 0.6770 -0.1867 0.0673 -0.0231 0.2098 8.9899 -0.2829 0.5592 -0.0007 FLAT (KL)-IDK 0.6829 -0.1926 0.0662 -0.0220 0.2146 8.9803 -0.2733 0.5572 -0.0027 FLAT (JS)-IDK 0.6890 -0.1987 0.0684 -0.0242 0.2229 8.9910 -0.2840 0.5574 -0.0025 FLAT (Pearson)-IDK 0.6930 -0.2027 0.0680 -0.0238 0.2265 8.9906 -0.2836 0.5580 -0.0019 FLAT (TV)-Normal 0.6942 -0.2039 0.0677 -0.0235 0.2274 8.9872 -0.2802 0.5588 -0.0011 FLAT (KL)-Normal 0.6773 -0.1870 0.0664 -0.0222 0.2092 8.9721 -0.2651 0.5580 -0.0019 FLAT (JS)-Normal 0.6980 -0.2077 0.0678 -0.0236 0.2313 8.9888 -0.2818 0.5593 -0.0006 FLAT (Pearson)-Normal 0.6991 -0.2088 0.0681 -0.0239 0.2327 8.9882 -0.2812 0.5594 -0.0005 Appendix E Related Work E.1 LLM Unlearning In recent years, large language models (LLMs) [36, 58, 59, 60] have achieved significant success in a wide range of downstream tasks, spanning from natural language understanding to generative AI applications. The success of LLMs relies heavily on the extensive efforts to gather instructional data consisting of millions of examples [61, 62, 63] for pre-training and fine-tuning [36, 64]. However, due to imperfect human expertise [65, 66, 67, 68], the training data often suffer from imperfect human annotation or noisy (potentially harmful) web data [69, 70, 71], which tends out to be a tough task hard for data curating [72, 73, 74, 75, 70]. As a result, their outputs may contain sensitive, private, or illegal content [1, 2], reflect societal biases [3, 4], or provide harmful instructions [5, 6, 7], motivating the researcher to efficiently unlearn the LLM on a certain proportion of data. LLM unlearning approaches can be broadly categorized into three families: model-based methods, input-based methods, and data-based methods [13]. Model-based Methods Model-based approaches involve modifying the weights and/or architecture to achieve unlearning. These include gradient ascent (GA) and its variants [5, 20, 19], as well as model editing techniques [76, 31, 77]. The dominant approach among existing LLM unlearning methods is fine-tuning the original model based on a carefully designed unlearning objective function [19, 5, 33, 6, 78, 22]. A common strategy combines forgetting and retaining objectives, applying gradient ascent updates to undesirable data while using regular gradient descent on desirable data [19, 6]. The goal of GA is to maximize the loss on the forget data, essentially reversing the effect of gradient descent during training. Some methods employ custom loss functions that go beyond standard forgetting and retaining losses. For example, [5] introduce a loss function with three components, where the custom loss reflects advanced techniques or regularization applied to the objectives. Other methods, such as DPO [21], KTO [79], and NPO [22], utilize reference models to guide the unlearning process. [19] fine-tune an adapter over the unlearning objective, which acts as an unlearning layer within the LLM. Several works also employ assistant or reinforced LLMs to facilitate unlearning [31, 14, 80]. For instance, [15] introduce an assistant LLM that pursues the opposite of the unlearning goals—i.e., remembering forgotten documents and forgetting retained knowledge. The unlearned LLM is then derived by computing the logit difference between the original and assistant LLMs. Data-based Methods Data-based methods fine-tune the LLM using a set of modified responses. This approach often begins by generating altered outputs (e.g., refusal-based responses), such as obliterated responses [18], inverted facts [81], or in-domain plausible alternatives [82]. These generated responses are then used to guide the unlearning process. [82] propose Alternate Preference Optimization, which utilizes in-domain positive feedback on the forget set, complementing the usual negative feedback to overcome the limitations of relying solely on negative feedback during unlearning. In this work, we employ reject-based template outputs as the modified "good" responses for the forgotten samples. Input-based Methods Input-based methods craft input instructions [17, 83, 84, 85, 86, 16, 16], such as in-context examples and prompts, to steer the original LLM toward the unlearning objective without altering the model’s parameters. These approaches aim to achieve unlearning in the output space rather than in the parameter space. Among these methods, a notable baseline by [16] uses an external prompt classifier as a guardrail, applying embedding corruptions to the identified prompts. The authors demonstrate that this corruption scheme results in distribution-wise similarity to the retrained model. In this work, we propose a novel loss adjustment method for LLM unlearning, which simultaneously utilizes available example responses, effectively combining data-based and model-based methods. E.2 Machine Unlearning In response to the data regulation requirements [8], machine unlearning (MU) has emerged as a critical process to remove the influence of specific data points, data classes, or even higher-level data concepts from a trained machine-learning model. One direct unlearning method involves retraining the model from scratch after removing the forgotten data from the original dataset, which is often considered the gold standard [10, 87]. However, this approach comes with significant computational demands. To alleviate this, most research focuses on developing approximate but much faster unlearning techniques, including gradient ascent [88, 89], influence unlearning [90, 91, 92], Fisher forgetting [93, 94], finetuning-based approaches [10, 11], and loss correction-related unlearning [95, 96, 12].

Instant research discovery