Paper deep dive
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Jiaao Chen, Diyi Yang
Models: T5-3B, T5-base
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%
Last extracted: 3/12/2026, 8:24:56 PM
Summary
The paper introduces EUL (Efficient Unlearning for LLMs), a framework that enables efficient and dynamic removal of specific data from large language models without full retraining. It utilizes lightweight unlearning layers integrated into transformers, trained via a selective teacher-student objective, and a fusion mechanism to handle sequences of unlearning requests by merging these layers.
Entities (5)
Relation Signals (3)
EUL â evaluatedon â IMDB
confidence 98% ¡ To demonstrate the effectiveness of our proposed EUL, we perform experiments on IMDB
EUL â evaluatedon â SAMSum
confidence 98% ¡ To demonstrate the effectiveness of our proposed EUL, we perform experiments on... SAMSum
EUL â appliedto â T5
confidence 95% ¡ In our experiments, we utilize T5 models (Raffel et al., 2020).
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, however, this process might suffer from privacy issues and violations of data protection regulations. As a result, the ability to easily remove data related to individual users from such models while not deteriorating their predictive quality after the removal becomes increasingly important. To address these issues, in this work, we propose an efficient unlearning framework that could efficiently update LLMs without having to retrain the whole model after data removals, by introducing lightweight unlearning layers learned with a selective teacher-student objective into the transformers. In addition, we introduce a fusion mechanism to effectively combine different unlearning layers that learns to forget different sets of data to handle a sequence of forgetting operations. Experiments on classification and generation tasks demonstrate the effectiveness of our proposed methods compared to the state-of-the-art baselines.
Tags
Links
Trouble viewing inline? Open PDF directly â
Full Text
45,486 characters extracted from source content.
Expand or collapse full text
Unlearn What You Want to Forget: Efficient Unlearning for LLMs Jiaao Chen Georgia Institute of Technology jiaaochen@gatech.edu Diyi Yang Stanford University diyiy@cs.stanford.edu Abstract Large language models (LLMs) have achieved significant progress from pre-training on and memorizing a wide range of textual data, how- ever, this process might suffer from privacy issues and violations of data protection regu- lations. As a result, the ability to easily re- move data related to individual users from such models while not deteriorating their predictive quality after the removal becomes increasingly important. To address these issues, in this work, we propose an efficient unlearning framework that could efficiently update LLMs without hav- ing to retrain the whole model after data re- movals, by introducing lightweight unlearning layers learned with a selective teacher-student objective into the transformers. In addition, we introduce a fusion mechanism to effectively combine different unlearning layers that learns to forget different sets of data to handle a se- quence of forgetting operations. Experiments on classification and generation tasks demon- strate the effectiveness of our proposed meth- ods compared to the state-of-the-art baselines 1 . 1 Introduction Utilizing Large Language Models (LLMs) has be- come the dominant paradigm for various NLP applications (Brown et al., 2020; Chowdhery et al., 2022a; Kojima et al., 2022; Ouyang et al., 2022; Brown et al., 2020; Radford et al., 2019; Lewkowycz et al., 2022; Qin et al., 2023; Touvron et al., 2023) as LLMs memorize a vast amount of knowledge during pre-training or fine-tuning on a wide range of textual data (Brown et al., 2020; Rad- ford et al., 2019; Hoffmann et al., 2022; Webson and Pavlick, 2022; Min et al., 2022; Liang et al., 2022; Carlini et al., 2022). However, these data could contain sensitive information such as names, phone numbers, email addresses, and private clini- cal notes (Jang et al., 2022; Kurmanji et al., 2023; 1 The codes are avaiable here:https://github.com/ SALT-NLP/Efficient_Unlearning/ Kumar et al., 2022).Extensive studies showed that LLMs could generate private information such as the Editor-in-Chief of MIT Technology Review in- cluding his family members, work address, and phone number (Carlini et al., 2022). Recently, the EUâs General Data Protection Regulation (GDPR) and USâs California Consumer Privacy Act (CCPA) have also required theright to be forgotten, intro- ducing new regulations that require applications to support the deletion of user-generated content when requested by users (Sekhari et al., 2021; Ku- mar et al., 2022). In light of this, it is essential to provide LLMs with an efficient and effective way to unlearn the information requested by users. Recent attention has been paid to the handling of such unlearning requests for LLMs through retrain- ing and data pre-processing like SISA (Bourtoule et al., 2021; Kumar et al., 2022) where training data is stored in different isolated slices and each check- point is saved after training on each slice. When a deletion request is received, the respective data point will be removed from the slice, and the model checkpoint up to the data point will be used to fur- ther retrain the model. The effect of unlearning is often reflected by the model errors on the deleted data (models cannot predict the deleted data) (Kur- manji et al., 2023; Jang et al., 2022). Other works have also explored the design of algorithms that ensure differential privacy (DP) (Yu et al., 2021; Li et al., 2021; Anil et al., 2021). However, ma- chine unlearning approaches like SISA (Bourtoule et al., 2021) usually require a significantly large amount of storage space (Bourtoule et al., 2021), and DP methods could result in a slow convergence and significant deterioration in model performance (Nguyen et al., 2022). In addition, both of them require retraining the whole model, which is ex- tremely expensive and time-consuming consider- ing the model scales of the current LLMs. These limitations also make them unable to dynamically deal with a sequence of unlearning requests which arXiv:2310.20150v1 [cs.CL] 31 Oct 2023 Figure 1: Overall process of our EUL framework. The unlearning layers are plugged into transformer layers after the feed-forward networks. During training, only the unlearning layers are learned to forget requested data while the original LLMs remain unchanged. For every deletion request, an unlearning layer is learned first and then merged with other unlearning layers via our designed fusion mechanism to form the fused unlearning transformer which satisfies a series of deletion requests. is often the need in real-world scenarios (Jang et al., 2022; Nguyen et al., 2022). To fill in these gaps, in this work, we propose an EfficientUnlearning method forLLMs (EUL) to ef- ficiently unlearn what needs to be forgotten without completely retraining the whole model while retain- ing the performances of the models. Specifically, we propose a lightweight approach to learning the unlearning layer that is plugged into transform- ers through a selective teacher-student formulation (Kurmanji et al., 2023) within several updates, with- out tuning the large language models. Additionally, we introduce a fusion mechanism to effectively combine the weights of different unlearning layers that learn to forget different sets of data to a single unified unlearning layer by minimizing a regression objective. This allows EUL to efficiently address a sequence of deletion operations. To demonstrate the effectiveness of our proposed EUL, we per- form experiments on IMDB (Maas et al., 2011) and SAMSum (Gliwa et al., 2019) in different set- tings compared to the state-of-the-art unlearning or model editing baselines. To summarize, our main contributions are threefold: ⢠We introduce an efficient unlearning method to remove the effect of required data in a lightweight way via a selective teacher- student formulation. ⢠We design a fusion mechanism to merge un- learning layers that are learned to forget differ- ent sets of data into a single unlearning layer to deal with a sequence of removal operations. â˘We conduct experiments on classification and generation tasks with backbone models of dif- ferent scales in different settings, to illustrate the effectiveness of EUL. 2 Related Work 2.1 Large Language Models Large language models have witnessed extensive progress recently (Brown et al., 2020; Radford et al., 2019; Smith et al., 2022; Rae et al., 2021; Chowdhery et al., 2022b; Touvron et al., 2023), especially in terms of scaling up LLMs such as LLAMA (Touvron et al., 2023), Megatron-turing NLG (Smith et al., 2022), Gopher (Rae et al., 2021), and PaLM Chowdhery et al. (2022b). Other works have also achieved better performance with smaller models through longer training (Hoffmann et al., 2022), instruction tuning (Wang et al., 2022; Zhou et al., 2023) and human feedback (Ouyang et al., 2022). However, recent studies have shown that training data, such as personally identifiable in- formation like names, phone numbers, email ad- dresses, and even bank account numbers (Carlini et al., 2021; Lee et al., 2021; Carlini et al., 2022; Jagielski et al., 2022), can be easily extracted from LLMs because LLMs memorize the training data in billions of parameters (Carlini et al., 2022). Our work is proposed to alleviate such issues by allow- ing efficient unlearning of the requested or private data from the learned parameters in LLMs. 2.2 Machine Unlearning for Privacy To mitigate the privacy risks for LLMs, machine un- learning methods have been introduced to remove the contributions of training examples that users re- quest to be erased by users (Bourtoule et al., 2021; Chien et al., 2023) including exact unlearning that retrains deep learning models on new datasets after removal (Bourtoule et al., 2021) and approximate unlearning (Izzo et al., 2021; Golatkar et al., 2020; Kurmanji et al., 2023; Jang et al., 2022) which aims to modify the weights of trained models to produce a new set of weights that approximate the weights from retraining. The effect of unlearning is often reflected by the model errors on the deleted data (models cannot predict the deleted data) (Kur- manji et al., 2023; Jang et al., 2022). Another line of work has focused on Differential Privacy (DP) which ensures that user information in training data cannot be inferred (Dwork, 2008; Yu et al., 2021; Li et al., 2021; Anil et al., 2021; Abadi et al., 2016). However, both types of methods require retraining the whole model, which is extremely expensive and time-consuming, especially for large language models and even impacts the task performances (Anil et al., 2021). And thus they can not dynami- cally tackle sequences of deletion (Jang et al., 2022; Nguyen et al., 2022). To overcome these limita- tions, we introduce an efficient unlearning method as well as a fusion mechanism toefficientlyand dynamicallyunlearn sequence of user data. Our work is also related to model editing (Mitchell et al., 2021; Belinkov et al., 2017; Dai et al., 2021; Wang et al., 2020) while they usually focus on editing the model output based on several given linguistic structures or facts about the world instead of forgetting the required data. 3 Efficient Unlearning for LLMs This section presents our designedEfficient Unlearning method forLLMs (EUL) which could efficiently and dynamically handle a sequence of deletion requests. The overall process is shown in Figure 1. Formally, for a large language modelF(.) that is trained on a datasetD=(x,y)wherex is textual data andyis the corresponding label, and a deletion request to forgetD f =(x f ,y f , our goal is to learn an updated modelF Ⲡ(.) that satisfies the following (Kurmanji et al., 2023): I(F(D f );F Ⲡ(D f )) = 0 I(F(D r );F Ⲡ(D r )) = 1 (1) whereD r =DâD f =(x r ,y r )refers to the data we would like to retain, andI(.)is the mutual information. Intuitively, we will updateF(.)with F(.)to generate similar output for the data we want to retain while losing all information about making predictions on the data we want to forget. 3.1 Learning to Forget via Unlearning Layers As the scales of current LLMs and the size of train- ing data are usually large, updating all the param- eters in the modelF(.)(e.g., re-trainingF(.)on D r i ) becomes extremely expensive. Inspired by recent advances in parameter-efficient fine-tuning (Houlsby et al., 2019; Chien et al., 2023), we model F Ⲡ(.)byF(f(.))wheref(.;W)is an adapter with significant smaller amount of parametersWcom- pared toF(.). And we would only updatef(.)to fulfill the unlearning requests. To effectively achieve the unlearning goals in equation1, we minimize a selective teacher-student objective where the student modelF Ⲡ(.) =F(f(.)) is learned to follow the teacher modelF(.)onD r while disobeyedF(.)onD f : L KL =Îą X x r KL(F(x r )||F(f(x r ))) â X x f KL(F(x f )||F(f(x f ))) (2) whereÎąis a hyper-parameter to balance the trade- off between forgettingx f and retainingx r . Intu- itively, during training,f(.)is leaned to minimize the KL-divergence between the output from the updated model and the original model on the data to retain while maximizing the KL-divergence be- tween the output from them on the data to forget. To maintain the task performance, we optimize f(.)for the task loss on the retain data: L T ASK = X x r l(F(f(x r )),y r ) (3) wherel(.)is the task-related loss, for example, cross-entropy loss,âlogP(F(f(x r ))), for clas- sification tasks. Furthermore, we also negate the original training objectives used in LLMs (e.g., masked language modeling objective (Raffel et al., 2020)) to forget the knowledge related to the data, in order to for- get in pre-trained parameters and ensure that the information in the forgotten data cannot be easily extracted fromF(.): L LM =â X x f l(F(f(x f ))) (4) wherel(.)is the language model loss used when pre-trainingF(.), for example, masked language model loss,âlogP(Ëx|xâËx)(Ëxare the randomly masked tokens). In our experiments, we utilize T5 models (Raffel et al., 2020). Thus we add an extra âPredict the masked wordâ at the beginning of the input for this loss term. Our final training objective is then the following: L EUL =L KL +ÎťL T ASK +ÎłL LM (5) whereÎťandÎłare hyper-parameters. In practice, following Kurmanji et al. (2023), we alternate the updates for the data to be forgotten and the data to be retained to optimizemin-maxterms inL EUL more stably. Specifically, we iteratively perform an epoch of updates on the data to be retained and then an epoch of updates on the data to be forgotten. 3.2 Fusing Unlearning Layers To dynamically handle a sequence of unlearning requests and derive a unified model that could for- get all of the requested data, we then introduce a fusion mechanism that could merge different un- learning layersf i (.;W i )which are learned to forget D f i = (X f i ,Y f i )in the previous section into a sin- glef m (.;W m ). Namely, we would like the output off m (.)onD f i being close tof i (.): min W m X i ||W T m X f i âW T i X f i || 2 (6) which is a linear regression problem and has a closed-form solution: W m = ( X i X f i T X f i ) â1 X i (X f i T X f i W i ) (7) Specifically, to derive the weightsW m for the merged unlearning layerf m , we would use the pre- computed inner product matrix of the hidden repre- sentations before the unlearning layers in LLMs of the forgotten dataX f i T X f i and then computeW m following Equation 7. The fusion mechanism ensures efficiency and privacy as it could be performed without any extra training and only requires storing the inner prod- uct matrix of the representations of the data to be forgotten instead of the data itself. 4 Experiments 4.1 Datasets We conduct experiments on both classification and generation tasks. For the classification task, we DatasetTaskTrainDevTest IMDBClassification20000200025000 SAMSumSummarization14732818819 Table 1: Dataset statistics for IMDB and SUMSum. utilize the IMDB dataset(Maas et al., 2011), which is a sentiment classification dataset consisting of usersâ reviews of movies, directors, actors, etc. For the generation task, we use SAMSum (Gliwa et al., 2019), which is a recent popular conversation sum- marization dataset consisting of conversations be- tween different speakers. The dataset statistics are shown in Table 1. We choose these two datasets because they are widely used (Wang et al., 2021; Yang et al., 2019; Qin et al., 2023; Ji et al., 2023; Wei et al., 2021; Sanh et al., 2021; Chen et al., 2022) to evaluate large language models and both datasets are related to cases where the user might require to remove their data, for example, removing all the reviews of a specific movie or removing all the conversations from one specific speaker. In experiments, we use the pre-trained NER models from AllenNLP 2 to extract all the entities (names) in IMDB and directly use the speakersâ names in SAMSum and simulate the unlearning re- quests to remove all the data from or related to cer- tain names. Moreover, we substitute all the names in the dev and test set with special tokens. 4.2 Evaluation Metrics To evaluate the performances, following Kurmanji et al. (2023), we measure several metrics: (1)Per- formance on the test set: The task-related perfor- mance on the test set, namely, accuracy for IMDB and ROUGE for SAMSum. This measures whether the unlearning algorithms affect the model perfor- mance or not. (2)Performance on the retained set: The task-related performance on the data to be retained. This measures whether the unlearn- ing algorithms forget the data that need to be re- tained. Higher performance means that the model remembers the data that is not to be forgotten. (3) Performance on the forgot set: The task-related performance on the data to be forgotten. This mea- sures whether the unlearning algorithms effectively forget the data requested to be forgotten. Lower 2 https://demo.allennlp.org/ Methods# Forgot DataTest SetâRetained SetâForgot SetâMLM LossâTime (s)â T5-base Original-93.21001001.46- Re-train 0.5% 92.810092.51.526685 Fine-tune93.010096.51.474200 SISA 92.498.291.51.541580 Reverse-Gradient92.097.368.61.564400 MEND92.298.573.51.6034 EULâ 93.010065.71.781200 Re-train 1% 92.710091.61.556610 Fine-tune92.810096.21.483950 SISA 92.298.190.41.552930 Reverse-Gradient91.596.467.41.594166 MEND91.395.574.61.6262 EULâ 93.010064.41.841526 Re-train 10% 92.110090.21.566026 Fine-tune92.010095.81.523133 SISA91.698.288.41.552010 Reverse-Gradient 91.096.565.41.623228 MEND90.894.876.21.66328 EULâ 92.299.057.22.011828 T5-3b Original-97.01001001.28- Re-train 0.5% 96.610094.81.3026855 Fine-tune 96.710096.21.2820465 SISA95.097.294.11.3316503 Reverse-Gradient93.396.578.91.4221826 MEND93.095.889.51.304980 EULâ 96.510070.21.669240 Re-train 1% 96.310094.21.3025280 Fine-tune96.510096.01.2818466 SISA93.896.892.71.3515680 Reverse-Gradient92.596.080.11.4618800 MEND 92.895.084.41.486600 EULâ 96.510067.51.729840 Re-train 10% 96.010093.51.3122140 Fine-tune 96.210094.01.3016752 SISA93.095.592.21.3514180 Reverse-Gradient91.995.268.41.4617850 MEND92.094.278.51.5012072 EULâ 96.010060.81.9210460 Table 2: Performances on IMDB for T5-base and T5-3B after unlearnling different number of privacy-related data. â refers to our model. All the results are averaged over 5 random runs. performance means that the model is better at for- getting the data. (4)MLM Loss: The masked language model loses on the data to be forgotten where related entities or actions are masked. This is achieved by adding âPredict the masked wordâ in the beginning. This measure whether the infor- mation in the data that needs to be forgotten can be extracted from the LLMs. Higher MLM loss means that it is harder to extract such information from the models. (5)Updating time: The time to update the original model in the forgetting process. 4.3 Baselines We compare our EUL with several baseline meth- ods:Re-train(Kumar et al., 2022): Re-training the model from scratch on the data to be retained with- out any forgotten data.Fine-tune(Kurmanji et al., 2023): Fine-tuning the original model on the data to be retained without any forgotten data.SISA (Kumar et al., 2022): Sharded, Isolated, Sliced, and Aggregated training where multiple models are trained independently on disjoined shards, and its slices and model checkpoints are saved for each Methods# Forgot DataTest SetâRetained SetâForgot SetâMLM LossâTime (s)â T5-base Original-47.2/23.5/39.671.4/42.6/62.770.2/42.2/62.71.37- Re-train 0.5% 46.8/23.0/38.171.7/42.8/62.442.4/23.2/42.01.4028000 Fine-tune46.6/23.2/38.172.5/44.7/65.258.8/34.1/54.11.3827120 SISA 44.2/22.0/37.470.5/41.6/60.541.4/23.0/40.81.4822582 Reverse-Gradient43.2/20.9/35.868.8/40.2/58.542.3/21.4/38.11.6428800 EULâ 46.8/23.0/38.571.5/42.4/63.338.4/20.2/37.21.8817060 Re-train 1% 45.4/22.8/37.572.4/43.0/62.842.2/22.8/41.61.4426855 Fine-tune46.4/23.2/38.172.9/43.6/64.056.4/31.8/52.71.4027210 SISA43.1/21.1/36.869.8/40.2/60.041.4/23.0/40.81.5022420 Reverse-Gradient 42.0/20.0/34.668.8/40.2/58.542.3/21.4/38.11.6427700 EULâ 46.5/22.8/38.071.5/42.4/63.335.8/19.0/36.21.9516820 Re-train 10% 44.2/21.2/35.870.4/41.2/60.541.4/21.4/40.01.4826155 Fine-tune 45.2/22.1/36.671.1/42.6/62.951.5/28.6/50.01.4327510 SISA41.8/19.6/33.868.3/38.8/58.840.2/20.1/38.91.5520790 Reverse-Gradient40.8/18.4/33.066.6/38.3/55.538.0/19.4/36.61.7127240 EULâ 45.8/22.4/37.870.9/42.0/62.333.0/18.3/33.02.2315000 T5-3b Original-53.6/29.6/45.178.5/47.6/66.174.2/43.5/64.91.30- Re-train 0.5% 52.8/28.8/44.077.4/46.1/65.450.4/27.2/43.01.3484480 Fine-tune53.3/29.0/44.478.0/47.1/65.860.2/36.1/55.71.3083600 SISA 51.7/27.2/40.874.8/44.8/63.549.4/26.8/42.21.3375000 Reverse-Gradient 50.6/25.9/39.972.8/42.0/62.844.3/23.1/39.01.4483200 EULâ 53.6/29.4/44.877.5/46.3/66.641.0/21.8/38.21.6760430 Re-train 1% 52.0/28.2/42.876.7/45.8/64.849.6/26.6/42.11.3582440 Fine-tune52.5/28.5/43.676.2/45.5/64.256.8/32.2/52.41.3281135 SISA50.0/26.1/38.972.3/43.1/61.149.0/25.8/41.11.3873550 Reverse-Gradient48.6/24.3/37.270.6/41.5/60.942.2/22.0/37.71.4582485 EULâ 53.3/29.0/44.476.4/45.3/64.338.4/19.9/36.01.7460880 Re-train 10% 50.8/26.4/40.574.2/45.0/63.248.2/25.5/41.41.3881010 Fine-tune51.4/27.2/41.975.2/45.3/64.052.1/29.8/49.91.3581800 SISA48.2/24.5/36.070.4/40.5/59.641.2/23.5/40.01.4070400 Reverse-Gradient 44.7/22.0/34.268.5/40.9/58.840.9/21.0/36.51.4982070 EULâ 52.0/28.4/42.674.9/45.0/63.636.2/18.6/34.71.7859900 Table 3: Performances on SAMSum for T5-base and T5-3B after unlearnling different number of privacy-related data.â refers to our model. All the results are averaged over 3 random runs. The performance on Test, Retained and Forgot Set are ROUGE-1/2/L scores. slice. When forgetting certain data, the correspond- ing data point is deleted from its slice, and the model checkpoint up to the data point is used to further retrain the model.Reverse-Gradient(Liu et al., 2022): Fine-tuning the original model on both retained data and forgot data while negating the gradient for the forgot data.MEND(Mitchell et al., 2021): Editing the model to generate out- put following the given examples. To adapt the model in the unlearning setting, we reverse the labels for data in classification tasks as input to MEND. However, it is infeasible to apply MEND to summarization tasks as it is hard to design the new output to perform the editing. 4.4 Model Settings For all the experiments, we use T5 models (T5-base and T5-3b) (Raffel et al., 2020) as the backbone models. For SISA, we follow Kumar et al. (2022) to split the dataset. For our unlearning layers, we only tune 0.5% (Chen et al., 2023) of the param- eters. TheÎą= 0.8,Îť= 1.0andÎł= 0.2are se- lected from grid searching0.1,0.2,0.5,0.8,1.0. We set the linear decay scheduler with a warmup ratio of 0.06 for training. The maximum sequence length is 128 for IMDB and 800 for SAMSum. The batch size was 256 for base models and 128 for 3b models on IMDB and 8 for base models and 2 for 3b models on SAMSum. The maximum learning MethodsTest SetâRetained SetâForgot SetâUpdating Time (s)â Original91.810091.2- Re-train92.510012.66026 Fine-tune92.310026.83133 SISA92.298.212.61510 Reverse-Gradient92.898.69.03228 MEND92.297.816.8328 EULâ 93.099.05.01828 Table 4: Performances on IMDB for T5-base after unlearnling 10% wrong-labeled data.â refers to our model. All the results are averaged over 5 random runs. Figure 2: Sequentially unlearnling 1,2,3,4,5 different sets of data for T5-base on IMDB. The results are accuracy on the test set and the accuracy on the forgot set averaging across different orderings. Every single set contains 1% of the training data. MetricEUL-KL-TASK-LM Test Setâ93.091.491.092.4 Retained Setâ10010097.499.0 Forgot Setâ65.790.867.469.0 MLM Lossâ 1.781.751.781.50 Table 5: Performances on IMDB for T5-base after re- moving 0.5% privacy-related data. We remove one ob- jective at a time from our EUL methods. rate was5eâ5and the maximum number of train- ing epochs was set to be3or5. All the experiments were performed using 8 A100 GPUs. 4.5 Results Unlearning Privacy-related Data on IMDBWe request the T5-base and T5-3b models that are fine- tuned on the IMDB dataset to unlearn 0.5%, 1% and 10% of the training data. The data to be for- gotten is randomly selected based on the names of movies, actors, actresses, directors, etc. For ex- ample, the model might need to forget all the data points related to âLena Numanâ. This simulates the ModelsSet 2Set 2, 1Set 2, 1,3 Re-train92.7/91.492.5/90.891.3/90.0 Fine-tune92.8/96.092.1/94.091.0/93.3 SISA92.2/90.492.0/87.891.2/85.8 Reverse-Gradient 91.5/67.990.5/67.289.8/66.0 EUL93.0/64.692.1/64.891.0/64.2 EUL-fuse 93.0/64.692.8/62.292.4/60.8 Table 6: Accuracy on the test/retained set of after un- learning sets of data following a sequence (set 2 -> set 1 -> set 3). cases where people/companies request to remove all the data related to them. The performances are displayed in Table 2. After unlearning the requested data from T5- base models, the re-training method hurts the accu- racy (e.g., a 1.1 accuracy drop when forgetting 10% data) on the test set because there is fewer data for training, and the accuracy on the retained set keeps unchanged (100%) probably because the model memorizes the retained data. The accuracy on the forgot set drops after re-training (e.g., 92.5 com- pared to 100 when unlearning 0.5% of the data), showing that the model is forgetting the requested data, and the masked language model loss increases (e.g., increasing 0.06 when unlearning 0.5% of the data), indicating that it is harder to extract the in- formation of the forgot data after re-training. The fine-tuning method shows better test accuracy with less updating time, however, it is worse in terms of forgetting the data. Even though SISA takes sig- nificantly less time (only costing around 1/3 of the time compared to re-training) to derive the updated model that forgets the requested data, it receives lower accuracy on the test and retained set, which means that the model prediction abilities get worse because of failing to remember the retained data. When reversing the gradients for the data to be forgotten, the updated model gets better at forget- ting with lower test accuracy. The model editing method, MEND, shows better overall performance on nearly all the metrics but it requires extra data to train a model editing module to edit the original model, making the method hard to be generalized to new models and settings. Our EUL approach boosts all the metrics with faster speed to update the model compared to previous unlearning baselines after removing different numbers of privacy-related data (e.g., achieving the lowest accuracy (65.6%) on forgot set while keeping the best test accuracy (93.0%) and 100% retained accuracy with 1/6 of the updating time compared to re-training when forgetting 0.5% of the data), suggesting that our designed unlearning layers that are learned with tai- lored objectives could efficiently update the LLMs to forget the required data and remain the abilities to perform the tasks. When the size of the back- bone model scales up to 3b, the improvements of our EUL are consistent, indicating that our methods could still forget what the user requests even for larger models that are better at memorizing data. Unlearning Privacy-related Data on SAMSum We unlearn 0.5%, 1% and 10% training data from T5-base and T5-3B models that are fine-tuned on the SAMSum dataset. The data to be forgotten is randomly selected based on the speaker names. For example, the model might need to forget all the conversations from âJackâ. This simulates the cases where people request to remove all the data generated by them. The performances are shown in Table 3. Similarly, our EUL method consis- tently achieves the best overall performances by effectively forgetting the requested data while re- membering the retained data and keeping the test ROUGE scores with significantly less amount of training time. This indicates that our objectives could also be generalized to generation tasks. Unlearning Mislabeled Data on IMDBWe also test a setting where the data to be forgotten is those with wrong labels. In experiments, we randomly change the labels for 10% of the training data and then request the model to unlearn their impact. This simulates the cases where we improve the models that are trained on noisy data by unlearning the mislabeled data (Kumar et al., 2022). We report the performances with T5-base models in Table 4. We observe that the accuracy of the test set of the original model is affected by the mislabeled data. And our EUL is the most effective approach to unlearn and remove the negative impact of those mislabeled data to achieve the best test accuracy. Sequence of RemovalsWe test baseline and our methods in a setting where a sequence of unlearn requests are received, i.e., the models need to forget different sets of data sequentially. In experiments, we sequentially unlearn 1,2,3,4,5 sets of data from T5-base model on IMDB dataset. For every unlearn length, we test with all the possible sequences and average the accuracy on the test set and the forgot set. For example, when the length of the forgetting requests are 2 (set 1, set 2), we test on the sequence (set 1 -> set 2) and sequence (set 2-> set 1) and av- erage the final performances. We show the results (accuracy on the test/retained set) of one possible sequence whose length is 3 (set 2 -> set 1 -> set 3) in Table 6 as an example. Averaged performances over different sequence lengths are visualized in Figure 2. EUL means that we keep one unlearning layer to sequentially unlearn different sets of data and EUL-fuse means that for every set of forgot data we learn separate unlearning layers and then merge them into a single unlearning layer via our proposed fusion mechanism. The results demon- strate that our proposed fusion method that com- bines different unlearning layers could effectively handle the sequence of deletion (achieving higher accuracy on the test set and lower accuracy on the forgot set.) especially when the sequence length gets longer compared to baseline models. 4.6 Ablation Studies Removal of ObjectivesWe perform ablation studies to show the effectiveness of each designed objective in EUL by removing each of them when ModelsIMDBSAMSum Original0.5420.510 Re-train0.5500.522 Fine-tune0.5680.525 SISA0.5850.530 Reverse-Gradient0.6260.588 EUL0.5660.530 Table 7: Accuracy from a trained binary classifier to predict whether an input data belongs to the retained set or the forgot set. learning the unlearning layers in Table 5. Com- pared to EUL which utilizes all of the learning objectives, removing each of them would result in a performance drop, which demonstrates every component contributes to the final performance. Specifically, removingL KL would increase the ac- curacy of the forgot set, indicating thatL KL is the main factor to forget the requested data. Remov- ingL T ASK from EUL would drop the accuracy on the test set, suggesting thatL T ASK is essen- tial to maintain task performance. RemovingL LM decreases the MLM Loss, showing thatL LM is the main objective to avoid the extraction of the requested information. Member Inference AttackWe further perform Member Inference Attack (MIA) (Kurmanji et al., 2023) on IMDB and SAMSum when unlearn 1% privacy-related data for T5-base models. Specif- ically, we test the accuracy of a binary classifier which is trained to predict whether the input data belong to the forgotten set or the retained set based on their representations after the final layer of the T5 model. An accuracy closer to 0.5 means that it is hard for the classifier to predict the groups of the input data. The accuracies are shown in Table 7. We found that the classifiers could not converge so well on the training set and always had a low accu- racy on the test set both before and after unlearning (e.g., 0.542 before unlearning and 0.566 after our EUL unlearning on IMDB). These showed that the randomly deleted data could not be easily inferred both before and after our EUL unlearning. 5 Conclusion In this work, we propose EUL, an efficient unlearn- ing method for LLMs that could efficiently and ef- fectively unlearn the user-requested data via learn- ing unlearning layers through the selective teacher- student objective. We further introduce a fusion mechanism that could merge different unlearning layers into one unified layer to dynamically un- learn a sequence of data. Experiments on different settings (different datasets, different model sizes, different forget set sizes) demonstrated the effec- tiveness of our proposed EUL method compared to state-of-the-art baselines. 6 Limitations In this work, we mainly perform experiments on T5-base/3b models with fine-tuned tasks. We en- courage future work to explore how to update dif- ferent backbone models with larger sizes such as LLAMA models or even close-sourced models like ChatGPT to forget the requested data such asemn privacy-related data, toxic data, or misinformation in the pre-training corpus. Also, we mainly fol- low the previous work to measure the unlearning through performance on the test set, retained set, and forgot set, together with the MLM loss. Fu- ture work might explore how to evaluate unlearning methods more comprehensively, such as whether the model could recall forgotten content or whether methods would make forgotten data identifiable. In addition, we perform all the experiments in simu- lated settings. Future work might apply our meth- ods to real-world applications to deal with actual use cases or introduce new benchmarks for evaluat- ing unlearning methods. Acknowledgment We would like to thank all reviewers and the SALT Lab for their valuable feedback. This work was partially sponsored by NSF grant IIS-2247357 and IIS-2308994. References Martin Abadi, Andy Chu, Ian Goodfellow, H Bren- dan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential pri- vacy. InProceedings of the 2016 ACM SIGSAC con- ference on computer and communications security, pages 308â318. Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi. 2021. Large-scale differen- tially private bert.arXiv preprint arXiv:2108.01624. Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Has- san Sajjad, and James Glass. 2017. What do neural machine translation models learn about morphology? arXiv preprint arXiv:1704.03471. Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Ma- chine unlearning. In2021 IEEE Symposium on Secu- rity and Privacy (SP), pages 141â159. IEEE. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. InAd- vances in Neural Information Processing Systems 33: Annual Conference on Neural Information Process- ing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural lan- guage models.arXiv preprint arXiv:2202.07646. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ul- far Erlingsson, et al. 2021. Extracting training data from large language models. InUSENIX Security Symposium, volume 6. Jiaao Chen, Mohan Dodda, and Diyi Yang. 2022. Human-in-the-loop abstractive dialogue summariza- tion. Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, and Diyi Yang. 2023.Parameter- efficient fine-tuning design spaces.arXiv preprint arXiv:2301.01821. Eli Chien, Chao Pan, and Olgica Milenkovic. 2023. Ef- ficient model updates for approximate unlearning of graph-structured data. InThe Eleventh International Conference on Learning Representations. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022a. Palm: Scaling language modeling with pathways. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Se- bastian Gehrmann, et al. 2022b. Palm: Scaling language modeling with pathways.arXiv preprint arXiv:2204.02311. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2021.Knowledge neu- rons in pretrained transformers.arXiv preprint arXiv:2104.08696. Cynthia Dwork. 2008. Differential privacy: A survey of results. InTheory and Applications of Models of Computation: 5th International Conference, TAMC 2008, Xiâan, China, April 25-29, 2008. Proceedings 5, pages 1â19. Springer. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Alek- sander Wawer. 2019. Samsum corpus: A human- annotated dialogue dataset for abstractive summa- rization.arXiv preprint arXiv:1911.12237. Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020. Eternal sunshine of the spotless net: Se- lective forgetting in deep networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9304â9312. Jordan Hoffmann, Sebastian Borgeaud, Arthur Men- sch, Elena Buchatskaya, Trevor Cai, Eliza Ruther- ford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Train- ing compute-optimal large language models.arXiv preprint arXiv:2203.15556. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. InIn- ternational Conference on Machine Learning, pages 2790â2799. PMLR. Zachary Izzo, Mary Anne Smart, Kamalika Chaudhuri, and James Zou. 2021. Approximate data deletion from machine learning models. InInternational Con- ference on Artificial Intelligence and Statistics, pages 2008â2016. PMLR. Matthew Jagielski, Om Thakkar, Florian Tramer, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, et al. 2022. Measuring forget- ting of memorized training examples.arXiv preprint arXiv:2207.00099. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022. Knowledge unlearning for mitigating privacy risks in language models.arXiv preprint arXiv:2210.01504. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation.ACM Comput- ing Surveys, 55(12):1â38. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. InThirty-sixth Conference on Neural Information Processing Sys- tems (NeurIPS 2022). Vinayshekhar Bannihatti Kumar, Rashmi Gangadhara- iah, and Dan Roth. 2022. Privacy adhering machine un-learning in nlp.arXiv preprint arXiv:2212.09573. Meghdad Kurmanji, Peter Triantafillou, and Eleni Tri- antafillou. 2023. Towards unbounded machine un- learning.arXiv preprint arXiv:2302.09880. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2021. Deduplicating training data makes language models better.arXiv preprint arXiv:2107.06499. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022.Solving quantitative reasoning problems with language models.arXiv preprint arXiv:2206.14858. Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. 2021. Large language models can be strong differentially private learners.arXiv preprint arXiv:2110.05679. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Ku- mar, et al. 2022. Holistic evaluation of language models.ArXiv preprint, abs/2211.09110. Bo Liu, Qiang Liu, and Peter Stone. 2022. Continual learning and private unlearning. InConference on Lifelong Learning Agents, pages 243â254. PMLR. Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the associ- ation for computational linguistics: Human language technologies, pages 142â150. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettle- moyer. 2022. Rethinking the role of demonstra- tions: What makes in-context learning work?arXiv preprint arXiv:2202.12837. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing at scale.arXiv preprint arXiv:2110.11309. Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022. A survey of machine unlearning.arXiv preprint arXiv:2209.02299. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022.Training language models to follow in- structions with human feedback.arXiv preprint arXiv:2203.02155. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language process- ing task solver?arXiv preprint arXiv:2302.06476. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners.OpenAI blog, page 9. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susan- nah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former.JMLR, 21(140):1â67. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021. Multitask prompted training en- ables zero-shot task generalization.arXiv preprint arXiv:2110.08207. Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. 2021. Remember what you want to forget: Algorithms for machine unlearning. Advances in Neural Information Processing Systems, 34:18075â18086. Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. 2022. Using deep- speed and megatron to train megatron-turing nlg 530b, a large-scale generative language model.arXiv preprint arXiv:2201.11990. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, TimothĂŠe Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and effi- cient foundation language models.arXiv preprint arXiv:2302.13971. Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Guihong Cao, Daxin Jiang, Ming Zhou, et al. 2020. K-adapter: Infusing knowledge into pre-trained models with adapters.arXiv preprint arXiv:2002.01808. Sinong Wang, Han Fang, Madian Khabsa, Hanzi Mao, and Hao Ma. 2021. Entailment as few-shot learner. arXiv preprint arXiv:2104.14690. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al- isa Liu, Noah A Smith, Daniel Khashabi, and Han- naneh Hajishirzi. 2022. Self-instruct: Aligning lan- guage model with self generated instructions.arXiv preprint arXiv:2212.10560. Albert Webson and Ellie Pavlick. 2022. Do prompt- based models really understand the meaning of their prompts? InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pages 2300â2344, Seattle, United States. Association for Computational Linguistics. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc V Le. 2021. Finetuned lan- guage models are zero-shot learners.arXiv preprint arXiv:2109.01652. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Car- bonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for lan- guage understanding.Advances in neural informa- tion processing systems, 32. Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A Inan, Gautam Kamath, Janardhan Kulka- rni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, et al. 2021. Differentially private fine-tuning of lan- guage models.arXiv preprint arXiv:2110.06500. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206.