Paper deep dive
Signal-Guided Optimization for Machine Unlearning
Xujia Li, Dan Li, Jian Lou, Wenjie Feng
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/15/2026, 3:11:13 AM
Summary
The paper introduces GSUO, a guidance-signal-aware unlearning optimization framework designed to address the limitations of coarse-grained machine unlearning methods. By implementing task-specific fine-grained guidance signals—target distribution-based for random-subset forgetting and feature redistribution-based for class-wise forgetting—GSUO steers the unlearning process to prevent over- and under-unlearning. Extensive experiments demonstrate that GSUO outperforms 14 state-of-the-art baselines in unlearning effectiveness, generalization, and efficiency, while significantly mitigating privacy risks associated with membership inference attacks.
Entities (12)
Relation Signals (10)
GSUO → implements → Machine Unlearning
confidence 95% · we propose GSUO, a guidance-signal-aware unlearning optimization framework that designs task-specific fine-grained guidance signals to steer the unlearning process
GSUO → consistsof → Signal-Guided Optimization
confidence 92% · GSUO consists of the following two successive optimization steps: S1. Signal-Guided Optimization (SGO)
GSUO → consistsof → Compensatory Optimization
confidence 92% · S2. Compensatory Optimization (CO): employing a general cross-entropy loss over the retain set and/or the forget set to enhance model utility
GSUO → evaluatedon → CIFAR-10
confidence 90% · We conduct experiments on CIFAR-10 and CIFAR-100... to measure the unlearning performance for classification tasks.
GSUO → evaluatedon → ResNet-18
confidence 90% · GSUO achieves the smallest accuracy gap between test and forget set, reaching 0.17% for ResNet-18
Signal-Guided Optimization → uses → Target Distribution-based Signal
confidence 88% · For random-subset forgetting... GSUO outlines the guidance signals as diverse target distributions to align according to the categories
Signal-Guided Optimization → uses → Feature Redistribution-based Signal
confidence 88% · for class-wise forgetting... GSUO employs intra-class dispersion loss and alignment loss as the guidance signals to prevent performance degradation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Current machine unlearning methods predominantly rely on global, coarse-grained intervention strategies. They lack precise pilot signals to guide the unlearning process and fail to provide differentiable guidance across different unlearning tasks. Due to the varying memorization strengths of samples during original training, such a uniform strategy leads to two problems: some samples are over-unlearned, which harms model utility; while others are under-unlearned, leaving residual information that can be exploited by privacy attacks. In this paper, we propose GSUO, a guidance-signal-aware unlearning optimization framework that designs task-specific fine-grained guidance signals to steer the unlearning process and is applicable to both random-subset and class-wise forgetting tasks. Extensive experiments demonstrate that GSUO outperforms 14 baselines in terms of both unlearning effectiveness and generalization, while achieving high efficiency and significant speedups, validating its effectiveness for reliable machine unlearning.
Tags
Links
- Source: https://arxiv.org/abs/2607.11975v1
- Canonical: https://arxiv.org/abs/2607.11975v1
PDF not stored locally. Use the link above to view on the source site.
Full Text
85,859 characters extracted from source content.
Expand or collapse full text
Signal-Guided Optimization for Machine Unlearning Xujia Li Dan Li Jian Lou Wenjie Feng∗ Abstract Current machine unlearning methods predominantly rely on global, coarse-grained intervention strategies. They lack precise pilot signals to guide the unlearning process and fail to provide differentiable guidance across different unlearning tasks. Due to the varying memorization strengths of samples during original training, such a uniform strategy leads to two problems: some samples are over-unlearned, which harms model utility; while others are under-unlearned, leaving residual information that can be exploited by privacy attacks. In this paper, we propose GSUO, a guidance-signal-aware unlearning optimization framework that designs task-specific fine-grained guidance signals to steer the unlearning process and is applicable to both random-subset and class-wise forgetting tasks. Extensive experiments demonstrate that GSUO outperforms 14 baselines in terms of both unlearning effectiveness and generalization, while achieving high efficiency and significant speedups, validating its effectiveness for reliable machine unlearning. 1 Introduction With growing awareness of privacy preservation and the legal guarantee of the “right to be forgotten” under regulations such as GDPR (Voigt & Bussche, 2017), deep models are expected to possess the capability to remove the influence of specific samples from training data. Moreover, outdated information or maliciously poisoned samples in the training data further reinforce the need for models to erase such data. To alleviate the prohibitive overhead of retraining, which involves training from scratch on the retain data and is commonly regarded as the gold standard (Nguyen et al., 2025; Xu et al., 2024), various approximate unlearning methods have consequently been proposed (Chundawat et al., 2023; Kurmanji et al., 2023; Tarun et al., 2024; Chen et al., 2023, 2025; Yu et al., 2026; Fan et al., 2024). For instance, SCRUB (Kurmanji et al., 2023) employs a teacher-student framework with contrastive optimization, and SalUn (Fan et al., 2024) identifies salient weights based on forgetting loss gradients, followed by random-label fine-tuning. For class-wise forgetting, Boundary Shrink alters the labels of forget samples via adversarial perturbations, and Boundary Expanding introduces a “shadow class” neuron to disperse such samples (Chen et al., 2025). However, approximate unlearning methods typically either perform unified optimization for all forget samples without differentiation (Chundawat et al., 2023; Kurmanji et al., 2023) or solely modify labels directly (Chen et al., 2025; Fan et al., 2024), resulting in the neglect of varying memorization strengths of different samples during training, their intervention strategies therefore are generally global and coarse-grained, lacking precise signals to finely guide the unlearning process. As a result, it leads to either over-unlearning that degrades model generalization or incomplete unlearning that leaves residual information vulnerable to privacy attacks. To this end, we propose the GSUO framework to exert unlearning relying on targeted guidance signals which are designed for diverse samples based on their context and properties across different forgetting tasks, aiming to reduce or avoid over-forgetting and under-forgetting. Specifically, for random-subset forgetting, where forget samples are uniformly scattered in feature space and share similarities with other retain data, GSUO outlines the guidance signals as diverse target distributions to align according to the categories (i.e. Normal and Boundary) forget samples belong to; for class-wise forgetting, where forget samples form tight clusters in feature space, GSUO employs intra-class dispersion loss and alignment loss as the guidance signals to prevent performance degradation resulting from disruption of the class boundaries learned by the model. Thus, by explicitly specifying the unlearned state of the forget samples, GSUO can perform tailored optimization with a well-defined objective. Comprehensive experimental results demonstrate that GSUO consistently achieves the best performance across a wide range of metrics, including fidelity, generalization ability, and forgetting efficacy, while it maintains high efficiency and delivers substantial speed-ups, outperforming all 14 state-of-the-art baselines. On the random-subset forgetting task, GSUO achieves the smallest accuracy gap between test and forget set, reaching 0.17%0.17\% for ResNet-18, along with the highest test accuracy of 82.08%82.08\%. It also delivers a substantial speed-up of up to 31×31× and the lowest AUC score of 0.7490.749 under strong membership inference attacks (MIA). For class-wise forgetting, GSUO demonstrates comparable performance, achieving complete unlearning while maintaining the highest training and test accuracies. 2 Related Work Machine Unlearning was first introduced by (Cao & Yang, 2015) to eliminate the influence of training samples on a learned model efficiently and completely by sharding data and building multiple models, enabling the exact unlearning of specific data partitions. For approximate unlearning, numerous methods have been developed to efficiently erase the influence of data through post-training procedures. Methods leveraging second-order information or influence functions (Izzo et al., 2021; Sekhari et al., 2021; Koh & Liang, 2017; Guo et al., 2020) approximately estimate the contribution of samples to the loss function to revoke their influence. However, most of these approaches are limited to convex models or incur high computational costs when applied to deep networks. For deep neural networks, (Golatkar et al., 2020b, a) proposed more practical approaches based on information projection and quadratic penalty, respectively. (Fan et al., 2024) proposed SalUn , which computes a weight saliency mask to selectively update only the parameters most relevant to the forget data, achieving an excellent balance between preserving performance on the retain set and unlearning effectiveness. (Chundawat et al., 2023; Kurmanji et al., 2023) employ a teacher-student framework for unlearning, with the former using a dual-teacher mechanism and the latter providing directional guidance from a single teacher. In such frameworks, the soft labels generated by the teacher model guide the student model to learn a new output distribution. (He et al., 2025) proposed Natural Unlearning to generate new samples by mixing each forget sample with related samples from the retain set, and then fine-tuning the original model on these mixed samples. Another line of work focuses on class forgetting, which requires the model to completely remove its knowledge of an entire concept or class. (Tarun et al., 2024) propose UNSIR, which learns “error-maximizing noise” for the target class and combines a “damage-repair” two-step weight update to achieve thorough forgetting of single or multiple classes in a single training pass. (Chen et al., 2023, 2025) shift the focus from the parameter space to the decision space, Boundary Shrink assigns incorrect labels to forget samples via adversarial perturbations and fine-tunes the model, forcing the decision boundary to shrink; while Boundary Expanding introduces a new neuron for “shadow class” to disperse forget samples into other categories, actively expanding the decision boundary. (Chen et al., 2025) and (Yu et al., 2026) focus on achieving class-wise forgetting on long-tailed distribution datasets. A detailed discussion of related work is provided in Appendix B. 3 Problem Formulation Notation Given a training dataset =(xi,yi)i=1N⊆×D= \(x_i,y_i) \_i=1^N ×Y consisting of N samples where xi∈x_i and yi∈y_i are independently and identically distributed drawn from a joint distribution P over ×X×Y; ⊆ℝFX ^F with dimension as F and label space =1,…,CY=\1,…,C\ with C classes. Let f(⋅;Θ)f(·; ) denote a deep neural network model with trainable parameters Θ . The model f(⋅;Θo)f(·; _o) trained over D producing Θo _o is referred to as the original model, which consists of a feature extractor Φ(⋅;ψ) (·;ψ) and a classifier h(⋅;ω)h(·;ω), i.e., f(x;Θo)=h(Φ(x;ψ);ω)f(x; _o)=h( (x;ψ);ω); therefore, Θo=ψ,ω _o=\ψ,ω\. Machine Unlearning Let f⊂D_f be a subset of the training dataset composed of NfN_f samples as the forget set, and its complement rD_r be the retain set, that is, =f∪rD=D_f _r with f∩r=∅D_f _r= . The original model f(⋅;Θo)f(·; _o) is trained on D by minimizing the cross-entropy loss ℒ(y^,y)=−∑k=1Cyklog(y^k)L( y,y)=- _k=1^Cy_k ( y_k). The retrained model, denoted as f(⋅;Θ∗)f(·; _*), is obtained by training on the retain set rD_r alone. The goal of machine unlearning is to eliminate the influence of fD_f from f(⋅;Θo)f(·; _o) via some unlearning algorithm U, producing updated weights Θu _u such that the unlearned model f(⋅;Θu)f(·; _u) behaves as if it had never seen fD_f. In other words, the performance of f(⋅;Θu)f(·; _u) on fD_f should approximate its performance on a held-out test set tD_t, i.e., f(f;Θu)≈f(t;Θo)f(D_f; _u)≈ f(D_t; _o). Random-subset and class-wise forgetting According to the composition of the forget set fD_f, the unlearning tasks can be categorized into the following two categories. • Random-subset forgetting: fD_f is an arbitrary subset of D, f⊂D_f with |f|=Nf≪N|D_f|=N_f N, and the samples to be forgotten may come from multiple classes of Y. • Class-wise forgetting: fD_f contains all training samples of a subset of target classes f⊂Y_f : f=(xi,yi)∈∣yi=c,∀c∈fD_f=\(x_i,y_i) y_i=c,∀ c _f\. |f|=1|Y_f|=1 reduces to single-class forgetting. 4 Proposed method: GSUO Figure 1: The guidance-signal design in the SGO step of GSUO. L: Target Distribution-based Signal for random-subset forgetting, R: Feature Redistribution-based Signal for class-wise forgetting. In this section, we propose the Guidance-Signal-aware Unlearning Optimization (GSUO), a unified framework for performing sample unlearning based on explicitly designed guidance signals, which can be applied to both random-subset and class-wise forgetting tasks. Specifically, GSUO consists of the following two successive optimization steps: • S1. Signal-Guided Optimization (SGO): leveraging specifically designed guidance signals for diverse samples to steer the unlearning process. • S2. Compensatory Optimization (CO): employing a general cross-entropy loss over the retain set and/or the forget set to enhance model utility via gradient descent/ascent (GD/GA). For the guidance signal in SGO, based on the sample behavior in the ideal unlearned model, we design Target Distribution-based Signal for random-subset forgetting task and Feature Redistribution-based Signal for class-wise forgetting task, which will be elaborated in detail in the following subsections. Figure 1 illustrates the design and principle of these guidance signals. 4.1 Random-subset Forgetting Under GSUO paradiagm, guidance signals leveraged by SGO of S1 are specified as diverse target distributions associated with different forget sample categories; while CO in S2 performs fine-tuning with GD on rD_r. Note that forget samples are uniformly scattered in feature space and share similarities with other retain data in this case. 4.1.1 SGO under Target Distribution-based Signal The core idea of the target distribution signal ptargetp_target is to identify predictive distributions that forget samples are expected to follow based on their location in the feature space of f(⋅;Θo)f(·; _o), and then align the model’s prediction for the forget sample with the corresponding target distribution to achieve unlearning. Due to the varying strength of memorization of different samples, adopting a coarse-grained update strategy that treats all forgotten samples equally leads to a dilemma: some samples are under-unlearned, while others are over-unlearned (please refer to the detailed theoretical analysis in Appendix A). Hence, we categorize forget samples into Normal and Boundary based on the model’s confidence, which intuitively is inversely proportional to the distance between the sample and the decision hyperplane, and design corresponding guidance signals considering their behavior under ideal unlearning. Specifically, the overall schema is as follows. Normal They are high-confidence samples in the original model and remembered by the model as a part of the training set, or perhaps they are overconfident. Consequently, these samples should be expected to undergo perturbation or a decrease in predicted confidence after unlearning, while remaining similar to their neighbors. As a result, they should maintain isotropic or uniform randomness among neighbors when no complete prior knowledge about their exact state after unlearning. Therefore, to facilitate a smooth unlearn, we construct the corresponding target distribution for a forget sample xnx_n as a linear combination of the original model prediction and a uniform distribution, i.e., ptarget=α⋅softmax(f(xn;Θo)))+(1−α)⋅/C,p_target=α·softmax (f(x_n; _o)) )+(1-α)·1/C, (1) where 1 is a C-size all-one vector, which becomes a uniform distribution after normalization by 1C 1C, and α∈[0,1]α∈[0,1] controls the forgetting strength. Such a guidance signal ptargetp_target steers the model to gradually reduce the confidence in the original predicted class for forget samples. Boundary They are relatively low-confidence samples in the original model and almost are close to the decision hyperplane. Consequently, they will tend to cross decision boundaries after unlearning, leading to misclassification. Accordingly, the corresponding guidance signal (target distribution) is designed as the predictive distribution of the most similar retained class in the feature space. Here, we first define the target signal for each class c∈c , which takes the prediction distribution PcP_c of the corresponding prototype μc _c as a proxy. Specifically, we sample a subset Pc=(x,y)∈r|y=cD^c_P=\(x,y) _r|y=c\ of size R≪|r|R |D_r| and extract their feature by the extractor Φ(⋅;ψ) (·;ψ), the prototype is formulated as μc=1R∑x∼PcΦ(x;ψ) _c= 1R _x ^c_P (x;ψ), i.e., the center of class c in feature space. Therefore, the target distribution signal for any boundary sample xbx_b is ptarget=Pc∗p_target=P_c^* where c∗=argminc≠ytrue∥Φ(xb;ψ))−μc∥2.c^*= _c≠ y_true\| (x_b;ψ))- _c\|^2. (2) Thus, it guides boundary samples to “blend into” semantically similar retain classes, achieving unlearning with minimal impact on the retain classes. Relying on the above ptargetp_target, we align the model’s predictions on the forget samples with the target distribution signal by minimizing the KL divergence, i.e., KL(ptarget∥pmodel)KL(p_target p_model). Normal and Boundary samples equally contribute to the total KL loss. 4.1.2 CO for enhancement To address the potential degradation of the model’s discriminative ability after S1, we can fine-tune on rD_r (optionally only its subset) to enhance for a few epochs with a standard cross-entropy loss. 4.2 Class-wise Forgetting Under GSUO paradigm, the guidance signal in S1 is based on feature redistribution. In S2, we apply cross-entropy loss on rD_r to reinforce correct classification and negative cross-entropy loss on fD_f to suppress prediction ability for forget classes, both with a layer-wise learning rate strategy. For this task, forget samples form tight clusters in feature space with clear inter-class decision boundaries. 4.2.1 SGO under Feature Redistribution-based Signal The core idea of feature distribution signals is to reshape the distribution of forget samples in feature space according to their characteristics for thorough class-wise forgetting. As the empirical observation from retraining shows, the forget samples become loosely distributed and their intra-class compactness is disrupted after unlearning, which serves as the prior knowledge for designing efficient unlearning methods. Therefore, we construct two types of guidance signals: a dispersion loss and an alignment loss, to steer unlearning objective by working together. Intra-class dispersion loss: fD_f contains all training samples of a subset of classes f⊆Y_f . We define the dispersion loss LdispL_disp as the negative logarithm of the average pairwise distance between the normalized features from the same forget class, that is, Ldisp= L_disp= −1|f|∑fclog(1Nc(Nc−1)∑xi,xj∈fci≠j - 1|Y_f| _D_f^c ( 1N_c(N_c-1) _ subarraycx_i,x_j _f^c\\ i≠ j subarray (3) (1−cos(Φ(xi;ψ)‖Φ(xi;ψ)‖,Φ(xj;ψ)‖Φ(xj;ψ)‖))), (1- ( (x_i;ψ)\| (x_i;ψ)\|, (x_j;ψ)\| (x_j;ψ)\| ) ) ), where Nc=|fc|N_c=|D_f^c| and cos(⋅,⋅) (·,·) denotes cosine similarity. By maximizing the pairwise distances among intra-class features, LdispL_disp forces the samples from forget class to scatter across the feature space, thereby disrupting their original clustering structure. Alignment loss: In addition to intra-class dispersion after unlearning, the forget samples as a whole tend to shift toward the global center of the retained classes. Ignoring such an overall offset would weaken the forgetting performance where the scattered features still reside near their original region, which allows the classifier to make correct predictions through other means (e.g., nearest neighbor classification). Therefore, we introduce an alignment loss that drives the feature center of each forget class toward the retained region. Here, we define the global center of the retained classes as μ¯r=1|r|∑(x,y)∈rΦ(x;ψ) μ_r= 1|D_r| _(x,y) _r (x;ψ) and the normalized feature center for each forget class as μfc=1Nc∑x∈fcΦ(x;ψ)‖Φ(x;ψ)‖ _f^c= 1N_c _x _f^c (x;ψ)\| (x;ψ)\|. The alignment loss is Lalign=1|f|∑c∈f(1−cos(μfc,μ¯r)),L_align= 1|Y_f| _c _f (1- ( _f^c, μ_r ) ), (4) where r=∖fY_r=Y _f is the set of retained classes. Hence, the weighted sum of the dispersion loss and the alignment loss forms the guidance signal for unlearning. 4.2.2 CO for enhancement In S2, we jointly optimize the feature extractor and classifier using the cross-entropy loss, reinforcing correct classification of retained samples while suppressing accurate predictions for forget samples. We adopt a layer-wise learning rate strategy: a smaller learning rate is applied to the feature extractor to maintain the achieved feature reshaping effect, while a larger learning rate is applied to the classifier to adapt it to the new feature distribution and enhance its discriminative ability for the retained classes. 5 Experiements 5.1 Experimental Settings Datasets We conduct experiments on CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009), Lacuna-10 (randomly select 10 classes with at least 500 samples from VGG-Faces (Cao & Yang, 2015)), and Tiny-ImageNet (Le & Yang, 2015), similar to previous work, to measure the unlearning performance for classification tasks. For random-subset forgetting, we choose to forget ≈10%≈ 10\% of whole training data,i.e., Nf=|v|=5,000N_f=|D_v|=5,000 for CIFAR-10 and CIFAR-100 and Nf=10kN_f=10k for Tine-ImageNet. For class-wise forgetting, we report the results about Lacuna-10 for forgetting a randomly selected class. Models For Lacuna-10, we fine-tuned a pre-trained VGG-11 model using Adam with a fixed lr=1E−3lr=1E-3 and batch size of 128128. For CIFAR-10, we fine-tuned a pre-trained ResNet18 model using Adam with a fixed lr=1E−3lr=1E-3 and batch size of 256256. For CIFAR-100 and Tiny-ImageNet, we finetuned pre-trained VGG-16 (Hayase et al., 2020), ResNet-50 (He et al., 2016), and ViT (Dosovitskiy et al., 2021) models using Adam with a fixed lr=1E−4lr=1E-4 and batch-size of 256256. All methods are implemented in Python 3.11 with PyTorch. All experiments are performed on NVIDIA Tesla-A100 (80G) with Intel Xeon processors. Baselines We covered 14 state-of-the-art baselines: Original: the model trained on D. Retrain:retraining the model from scratch without fD_f(included as a reference but inviable in practice). Fine-tuning (FT), Gradient Ascent (GA) (Golatkar et al., 2020a), Random Labels (RL) (Graves et al., 2021), Influence Unlearning (IU) (Koh & Liang, 2017; Izzo et al., 2021), EU-k (Goel et al., 2022), ℓ1 _1-sparse (Jia et al., 2023), SalUn (Fan et al., 2024), Bad-teaching (Bad-T) (Chundawat et al., 2023) and SCRUB (Kurmanji et al., 2023). Besides, for the class-wise forgetting, we include specific methods: Boundary Shrink (B-Shrink) & Expanding (B-Expand) (Chen et al., 2023) and UNSIR (Tarun et al., 2024). Considering the comprehensive forgetting efficacy, model performance, and privacy protection, we follow the evaluation metrics below: • Accuracy: The accuracies on different datasets, including the retain training set (rD_r), retain test set (rtD_rt), forget training set (fD_f), forget test set (ftD_ft) and the full test set (tD_t); • Difference (|Diff||Diff|): The absolute accuracy difference between forget set and test set, i.e., |Diff|=|Acc(f)−Acc(t)||Diff|=|Acc(D_f)-Acc(D_t)|, which measures whether the model still retains memory of the forgotten data after unlearning; • Acc-Index: An integrated accuracy index for the random-subset unlearning task as Facc(t,f)=Acc(t)−|Diff|F_acc(D_t,D_f)=Acc(D_t)-|~Diff~|; • Time: The running time for performing mahcine unlearning method; • Speedup: The ratio of the time required for retraining on rD_r to the unlearning method. • Attack Success Rate (ASR): The attack success rate of MIA, measuring the model’s effectiveness in privacy protection with the ideal value being close to a random guess. Acc-Index Facc∈[0,1]F_acc∈[0,1] accounts for both the generalization capacity and unlearning efficacy of the unlearned model, i.e., better accuracy on tD_t while the accuracy of fD_f should be as consistent with it as possible. Detailed experimental settings and more results are deferred to Appendix C and D. The code is available at https://anonymous.4open.science/r/GSUO. 5.2 Random-subset Forgetting Table 1: Performance comparison among baselines and GSUO for the random-subset forgetting task on CIFAR-10 for models with different architectures. Dataset (model) Metric Original Retrain FT GA RL EU-k IU ℓ1 _1-sparse SalUn Bad-T SCRUB GSUO CIFAR-10 (ResNet-18) Acc(r)Acc(D_r) ↑ 98.29 98.61 98.66 97.50 93.51 98.90 85.03 96.20 98.51 97.50 93.93 99.5 Acc(t)Acc(D_t) ↑ 80.80 80.61 81.18 79.54 76.19 81.55 71.20 78.70 80.51 79.07 78.22 82.08 Acc(f)Acc(D_f) 98.34 81.24 96.95 96.32 93.64 98.72 85.44 86.52 92.26 73.64 89.48 82.25 |Diff||~Diff~| ↓ 17.54 0.63 15.77 16.78 17.45 17.17 14.24 7.82 11.75 5.43 11.26 0.17 Time (s) ↓ – 230.10 16.61 1.08 1.65 85.28 41.07 65.98 86.95 12.34 11.93 7.06 CIFAR-100 (VGG-16) Acc(r)Acc(D_r) ↑ 99.53 99.53 99.59 99.02 98.41 99.88 99.53 99.85 99.53 99.41 99.95 87.02 Acc(t)Acc(D_t) ↑ 60.35 58.63 59.81 58.80 57.61 61.15 60.35 60.63 60.11 58.81 61.48 61.3 Acc(f)Acc(D_f) 99.40 58.72 97.50 97.26 98.54 99.84 99.38 96.94 98.36 67.30 92.96 62.90 |Diff||~Diff~| ↓ 39.05 0.09 37.69 38.46 40.93 38.69 39.03 36.3 38.25 8.49 31.48 1.6 Time (s) ↓ – 366.09 139.61 1.895 1.93 92.63 372.10 79.50 49.45 19.92 34.58 18.45 CIFAR-100 (ViT) Acc(r)Acc(D_r) ↑ 98.03 97.33 99.45 89.76 97.87 97.84 97.80 96.74 99.24 97.53 99.79 99.04 Acc(t)Acc(D_t) ↑ 88.96 90.20 90.40 80.95 88.65 89.03 88.84 88.06 90.18 89.29 91.02 85.06 Acc(f)Acc(D_f) 98.16 90.78 98.62 88.80 98.04 97.82 97.98 96.12 98.40 85.82 99.20 87.16 |Diff||~Diff~| ↓ 9.20 0.58 8.22 7.85 9.39 8.79 9.14 8.06 8.22 3.47 8.18 2.10 Time (s) ↓ – 420.53 334.91 23.65 25.10 189.12 3287.35 125.43 140.18 397.08 739.71 161.45 Tiny ImageNet (ResNet50) Acc(r)Acc(D_r) ↑ 99.39 99.94 99.74 98.79 83.81 99.96 99.39 99.56 98.98 98.31 99.81 98.45 Acc(t)Acc(D_t) ↑ 78.74 78.34 77.51 78.14 66.67 78.57 78.75 77.69 77.54 71.75 78.90 78.65 Acc(f)Acc(D_f) 99.36 78.39 97.90 97.74 83.48 99.38 99.35 98.74 97.30 78.84 99.11 77.96 |Diff||~Diff~| ↓ 20.62 0.05 20.39 19.60 16.81 20.81 20.60 21.05 19.76 7.09 20.21 0.69 Time (s) ↓ – 1027.23 354.36 12.52 12.32 396.83 52.84 83.00 93.60 482.42 988.39 148.6 For random-subset forgetting, Table 1 reports the utility and efficiency of different unlearning methods. Figure 2(a)-(b) shows Speedup versus FaccF_acc for ResNet-18 on CIFAR-10 and VGG-16 on CIFAR-100, with additional results in Appendix D. We evaluate privacy protection by adopting the U-LiRA (Hayes et al., 2025) MIA based on the log-likelihood ratio, which is considered the SOTA per-sample attack benchmark for unlearning scenarios to date. The detailed attack procedure is provided in Appendix C. Figure 2(c) shows MIA results and AUC score for different unlearning methods. GSUO achieves the best overall performance. We desire to achieve |Diff||Diff| as small as possible. Results show that GSUO consistently produces the lowest |Diff||Diff| across all architectures and datasets, significantly outperforming existing baselines. A lower |Diff||Diff| indicates that the model’s predictions on the forget set more closely match those on an unseen test set. Ideally, a perfectly unlearned model behaves as if it never encountered the forget samples during training. Our experiments consistently confirm that GSUO approaches this ideal unlearning state more closely than all other evaluated methods. We expect Acc(r)Acc(D_r) and Acc(t)Acc(D_t) to remain high after unlearning. GSUO satisfies this expectation across different architectures. On CIFAR-10 with ResNet-18, GSUO achieves higher retain and test accuracy than all baselines. Although GSUO’s Acc(r)Acc(D_r) on CIFAR-100 with VGG-16 is slightly lower than than SCRUB and EU-k, Acc(t)Acc(D_t) is by no means inferior. This indicates that GSUO does not simply memorize retain samples to achieve high retain accuracy, but instead maintains retain performance while exhibiting stronger generalization. SCRUB also performs well in this regard, but its high Acc(f)Acc(D_f) indicates incomplete forget. In contrast, GSUO maintains high retain and test accuracy while effectively reducing prediction accuracy on the forget set, thereby achieving a better balance between selective forgetting and model utility. Figure 2: (a)-(b) Speedup versus FaccF_acc for ResNet-18 on CIFAR-10 and VGG-16 on CIFAR-100; (c) MIA (U-LiRA) results and AUC score for different unlearning methods on ResNet-18 on CIFAR-10. For comprehensive evaluation, we adopt the metric FaccF_acc to balance the model’s generalization and its forgetting effectiveness on the forget set. We also measure computational efficiency via the speedup ratio relative to Retrain. As shown in Figure 2, GSUO consistently achieves a favorable trade-off between unlearning quality and computational efficiency (occupying the top-right region, i.e., the optimal zone with high FaccF_acc and high speedup). Although methods such as GA and RL are faster on some datasets according to Table 1, their forgetting effectiveness is far inferior to GSUO. This demonstrates that GSUO effectively balances efficiency and forgetting quality, delivering strong and robust unlearning performance without sacrificing computational practicality across diverse settings. We further consider a privacy-critical application: deleting data of users who exercise their “right to be forgotten”. To this end, we adopt U-LiRA as the privacy evaluation metric, aiming to ensure that after unlearning, an attacker should be unable to distinguish between forgotten samples and truly unseen samples, thereby protecting the privacy of users who request deletion. Figure 2 shows the U-LiRA results, where GSUO achieves the best performance, substantially outperforming all baselines. This result demonstrates that GSUO effectively mitigates membership inference attacks on forgotten samples, offering strong privacy guarantees while maintaining competitive model utility. GSUO demonstrates strong scalability and robustness as data and model scale increase. As forgetting tasks scale from smaller datasets with fewer classes to larger, more complex ones with more categories and samples, the forgetting exactness |Diff||Diff| of GSUO remains consistently low, substantially outperforming all baselines. At the same time, it maintains competitive computational efficiency. These results verify the stability and effectiveness of GSUO in complex, large-scale forgetting tasks. 5.3 Class-wise Forgetting Table 2: Comparison of MU methods for class-wise forgetting on Lacuna-10 under VGG-11. Method Acc(r)Acc(D_r) ↑ Acc(rtAcc(D_rt) ↑ Acc(f)Acc(D_f) ↓ Acc(ft)Acc(D_ft) ↓ Time (s) ↓ Original 100.00 87.67 99.74 90.00 — Retrain 99.94 82.67 0.00 0.00 396.76 FT 98.43 86.78 0.00 0.00 50.98 GA 88.09 76.67 0.80 1.00 49.03 RL 97.03 87.78 1.21 1.00 60.69 EU-k 98.95 88.22 0.00 0.00 42.37 IU 89.11 79.67 12.68 5.00 1.98 ℓ1 _1-sparse 82.78 76.00 0.00 0.00 51.85 SalUn 98.72 87.56 0.00 0.00 105.67 Bad-T 93.59 81.78 82.70 67.00 87.13 SCRUB 98.92 89.78 0.80 0.01 90.84 B-Shrink 96.39 85.11 0.00 0.00 5.96 B-Expand 95.57 85.44 0.00 0.00 10.72 UNSIR 83.95 77.44 0.00 0.00 25.52 GSUO 99.21 89.11 0.00 0.00 35.59 For class-wise forgetting, we evaluated the effectiveness of different unlearning methods on forgetting all samples of a randomly selected class (class 4) on the Lacuna-10 dataset. Table 2 reports the corresponding effectiveness and efficiency of these methods. We desire the training and test accuracy on the forget set to be as small as possible. GSUO achieves the optimal level on both metrics, completely removing the influence of the forgotten class from the model. This includes eliminating predictions on both the forget set and the corresponding test samples, demonstrating that GSUO can fully erase the target class. Despite achieving perfect forgetting, GSUO simultaneously preserves strong performance on retain data. Specifically, it achieves the highest retain set accuracy among all methods, and its performance on the retain test set is also among the best, ranking second overall. While several baselines (e.g., FT, EU-k, SalUn) also achieve zero accuracy on the forget set, they either fall short of GSUO on retain set accuracy or exhibit higher computational cost. Methods such as GA, RL, and SCRUB fail to achieve complete forgetting, leaving non-zero accuracy on the forget set. GSUO achieves a substantial speedup compared to retraining, outperforming most baselines. Although a few methods are faster, they compromise either forgetting completeness or model utility. Overall, GSUO strikes a strong trade-off among forgetting effectiveness, model utility, and computational efficiency, making it a practical and reliable solution for selective class removal. 5.4 Ablation Study Table 3: Ablation study on GSUO framework Random-subset Forgetting Class-wise Forgetting Acc(r)Acc(D_r) Acc(f)Acc(D_f) Acc(t)Acc(D_t) |Diff||~Diff~| Acc(rt)Acc(D_rt) Acc(ft)Acc(D_ft) Acc(t)Acc(D_t) Oringinal 97.27 97.38 80.87 16.51 87.67 90.00 87.90 w/o SGO 99.92 99.46 83.17 16.29 87.89 0.00 79.10 w/o CO 88.64 78.54 73.43 5.11 84.56 6.00 76.70 GSUO 99.5 82.25 82.08 0.17 89.11 0.00 80.20 To verify the necessity of SGO and CO, we conduct ablation studies by removing each component individually. The results are shown in Table 3. For random-subset forgetting, we observe that removing either SGO or CO leads to notable performance degradation. Specifically, without SGO, the model retains excessive information on the forget set; without CO, test accuracy drops significantly, with |Diff||Diff| remaining large in both cases. For class-wise forgetting, we observe similar trends: removing either component results in lower retain test set accuracy compared to the full GSUO. We conclude that SGO is essential for eliminating the influence of forgotten samples, while CO plays a critical role in preserving model generalization. The best forgetting performance across both tasks is achieved only when both components are present, validating the necessity of both stages in GSUO. Table 4: Ablation study on SGO Task Method Acc(r)Acc(D_r) Acc(rt)Acc(D_rt) Acc(f)Acc(D_f) Acc(ft)Acc(D_ft) Acc(t)Acc(D_t) |Diff||Diff| Random-subset Original 97.27 — 97.38 — 80.87 16.51 w/o Boundary 99.6 — 95.62 — 82.59 13.03 w/o Normal 99.92 — 98.8 — 83.73 15.07 GSUO 99.5 — 82.25 — 82.08 0.17 Class-wise Original 100.00 87.67 99.74 90.00 87.90 — w/o Dispersion 98.75 88.22 0.00 0.00 88.22 — w/o Alignment 96.45 84.78 0.00 0.00 84.78 — GSUO 99.21 89.11 0.00 0.00 89.11 — To further investigate the contribution of each component in SGO, we conduct ablation experiments under two forgetting tasks. The results are shown in Table 4. For Random-subset forgetting, we observe that treating all forget samples uniformly (as either all Normal or all Boundary) results in limited forgetting performance, with forget set accuracy remaining high in both cases. This finding suggests that Random-subset forgetting requires a more nuanced approach: different types of forget samples need distinct target distributions to achieve effective unlearning. In contrast, the complete GSUO substantially reduces forget set accuracy while keeping retain set accuracy high, achieving the optimal balance we desire. For the class-wise forgetting task, we observe that removing either the dispersion loss or the alignment loss still allows the model to achieve zero accuracy on the forget set. However, retain set accuracy drops noticeably compared to the full GSUO. This indicates that even when forget set accuracy has already reached the optimal level, both the dispersion loss and the alignment loss still help preserve model performance on the retain data. We therefore conclude that each component in CO plays a distinct yet complementary role, and only their full combination achieves the optimal trade-off between forgetting effectiveness and model utility preservation across both forgetting tasks. 5.5 Sensitivity Analysis on Forget Set Size Table 5 shows that GSUO achieves a favorable trade-off between utility, forgetting efficacy, and efficiency. As the forgetting ratio increases from 0% to 40%, GSUO maintains high accuracy on the retain set rD_r , with only a 2% drop at 40% forgetting. This slight drop is reasonable, as even retraining from scratch inevitably incurs a certain degree of accuracy degradation when the size of the retain set decreases. The test set tD_t accuracy declines gradually from 80.80% to 79.92%, indicating that the model generalizes well to the overall data distribution after forgetting targeted classes. The accuracy on the forget set fD_f is effectively suppressed across different forgetting ratios. Furthermore, even at a forgetting ratio of 40%, the |Diff||Diff| metric remains below 3%, which is even better than the performance of most existing unlearning methods at lower forgetting ratios, fully demonstrating that GSUO achieves efficient and stable Random-subset forgetting. The runtime of SGA-OT increases modestly with the forgetting ratio and remains significantly lower than the cost of full retraining, confirming the practicality and scalability of our method for large-scale unlearning scenarios. Table 5: Performance of GSUO across different forgetting ratios. Metrics 0 5% 10% 15% 20% 25% 30% 35% 40% Acc(r)Acc(D_r) ↑ 92.90 99.51 99.50 99.00 98.79 98.23 98.01 98.05 97.65 Acc(t)Acc(D_t) ↑ 80.80 82.24 82.08 81.56 81.57 81.00 80.81 80.19 79.92 Acc(f)Acc(D_f) 98.34 81.44 82.25 81.57 82.88 84.78 81.34 82.83 82.62 |Diff||\,Diff\,| ↓ 17.54 0.8 0.17 0.01 1.31 3.78 0.53 2.64 2.70 Time (s) ↓ – 6.6 7.06 8.81 8.73 11.97 12.66 11.81 12.21 6 Conclusion We propose GSUO, a two-step unlearning framework consisting of Signal-Guided Optimization and Compensatory Optimization. By leveraging task-specific guidance signals and targeted utility restoration, GSUO provides a principled and efficient solution to the challenges of over-unlearning and under-unlearning in both random-subset and class-wise forgetting scenarios. Extensive experiments demonstrate that GSUO consistently surpasses 14 state-of-the-art baselines across unlearning efficacy, generalization, and privacy robustness, while delivering substantial speedups, confirming its effectiveness for trustworthy machine unlearning. Limitation. GSUO outperforms baselines in privacy evaluation, but still falls short of the retraining upper bound. Experiments are limited to traditional models; its effectiveness on emerging architectures such as LLMs and VLMs remains to be validated. References Bourtoule et al. (2021) Bourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., and Papernot, N. Machine unlearning. In 42nd IEEE Symposium on Security and Privacy, SP 2021, San Francisco, CA, USA, 24-27 May 2021, p. 141–159. IEEE, 2021. doi: 10.1109/SP40001.2021.00019. URL https://doi.org/10.1109/SP40001.2021.00019. Cao & Yang (2015) Cao, Y. and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE Symposium on Security and Privacy, SP 2015, San Jose, CA, USA, May 17-21, 2015, p. 463–480. IEEE Computer Society, 2015. doi: 10.1109/SP.2015.35. URL https://doi.org/10.1109/SP.2015.35. Chen et al. (2023) Chen, M., Gao, W., Liu, G., Peng, K., and Wang, C. Boundary unlearning: Rapid forgetting of deep networks via shifting the decision boundary. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, p. 7766–7775. IEEE, 2023. doi: 10.1109/CVPR52729.2023.00750. URL https://doi.org/10.1109/CVPR52729.2023.00750. Chen et al. (2025) Chen, M., Gao, W., Wang, C., Liu, G., Abdelmoniem, A. M., and Peng, K. From expansion to retraction: Long-tailed machine unlearning via boundary manipulation. In Antonie, L., Pei, J., Yu, X., Chierichetti, F., Lauw, H. W., Sun, Y., and Parthasarathy, S. (eds.), Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.2, KDD 2025, Toronto ON, Canada, August 3-7, 2025, p. 215–226. ACM, 2025. doi: 10.1145/3711896.3736970. URL https://doi.org/10.1145/3711896.3736970. Chundawat et al. (2023) Chundawat, V. S., Tarun, A. K., Mandal, M., and Kankanhalli, M. S. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In Williams, B., Chen, Y., and Neville, J. (eds.), Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, p. 7210–7217. AAAI Press, 2023. doi: 10.1609/AAAI.V37I6.25879. URL https://doi.org/10.1609/aaai.v37i6.25879. Dosovitskiy et al. (2021) Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. An image is worth 16x16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://openreview.net/forum?id=YicbFdNTTy. Fan et al. (2024) Fan, C., Liu, J., Zhang, Y., Wong, E., Wei, D., and Liu, S. Salun: Empowering machine unlearning via gradient-based weight saliency in both image classification and generation. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/forum?id=gn0mIhQGNM. Goel et al. (2022) Goel, S., Prabhu, A., and Kumaraguru, P. Evaluating inexact unlearning requires revisiting forgetting. CoRR, abs/2201.06640, 2022. URL https://arxiv.org/abs/2201.06640. Golatkar et al. (2020a) Golatkar, A., Achille, A., and Soatto, S. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, p. 9301–9309. Computer Vision Foundation / IEEE, 2020a. doi: 10.1109/CVPR42600.2020.00932. URL https://openaccess.thecvf.com/content_CVPR_2020/html/Golatkar_Eternal_Sunshine_of_the_Spotless_Net_Selective_Forgetting_in_Deep_CVPR_2020_paper.html. Golatkar et al. (2020b) Golatkar, A., Achille, A., and Soatto, S. Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In Vedaldi, A., Bischof, H., Brox, T., and Frahm, J. (eds.), Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX, Lecture Notes in Computer Science, p. 383–398. Springer, 2020b. doi: 10.1007/978-3-030-58526-6“˙23. URL https://doi.org/10.1007/978-3-030-58526-6_23. Golatkar et al. (2021) Golatkar, A., Achille, A., Ravichandran, A., Polito, M., and Soatto, S. Mixed-privacy forgetting in deep networks. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, p. 792–801. Computer Vision Foundation / IEEE, 2021. doi: 10.1109/CVPR46437.2021.00085. URL https://openaccess.thecvf.com/content/CVPR2021/html/Golatkar_Mixed-Privacy_Forgetting_in_Deep_Networks_CVPR_2021_paper.html. Graves et al. (2021) Graves, L., Nagisetty, V., and Ganesh, V. Amnesiac machine learning. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, p. 11516–11524. AAAI Press, 2021. doi: 10.1609/AAAI.V35I13.17371. URL https://doi.org/10.1609/aaai.v35i13.17371. Guo et al. (2020) Guo, C., Goldstein, T., Hannun, A. Y., and van der Maaten, L. Certified data removal from machine learning models. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, Proceedings of Machine Learning Research, p. 3832–3842. PMLR, 2020. URL http://proceedings.mlr.press/v119/guo20c.html. Hayase et al. (2020) Hayase, T., Yasutomi, S., and Katoh, T. Selective forgetting of deep networks at a finer level than samples. CoRR, abs/2012.11849, 2020. URL https://arxiv.org/abs/2012.11849. Hayes et al. (2025) Hayes, J., Shumailov, I., Triantafillou, E., Khalifa, A., and Papernot, N. Inexact unlearning needs more careful evaluations to avoid a false sense of privacy. In IEEE Conference on Secure and Trustworthy Machine Learning, SaTML 2025, Copenhagen, Denmark, April 9-11, 2025, p. 497–519. IEEE, 2025. doi: 10.1109/SATML64287.2025.00034. URL https://doi.org/10.1109/SaTML64287.2025.00034. He et al. (2016) He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, p. 770–778. IEEE Computer Society, 2016. doi: 10.1109/CVPR.2016.90. URL https://doi.org/10.1109/CVPR.2016.90. He et al. (2025) He, Z., Li, T., Cheng, X., Huang, Z., and Huang, X. Towards natural machine unlearning. IEEE Trans. Pattern Anal. Mach. Intell., 47(12):11548–11560, 2025. doi: 10.1109/TPAMI.2025.3597350. URL https://doi.org/10.1109/TPAMI.2025.3597350. Izzo et al. (2021) Izzo, Z., Smart, M. A., Chaudhuri, K., and Zou, J. Approximate data deletion from machine learning models. In Banerjee, A. and Fukumizu, K. (eds.), The 24th International Conference on Artificial Intelligence and Statistics, AISTATS 2021, April 13-15, 2021, Virtual Event, Proceedings of Machine Learning Research, p. 2008–2016. PMLR, 2021. URL http://proceedings.mlr.press/v130/izzo21a.html. Ji et al. (2024) Ji, J., Liu, Y., Zhang, Y., Liu, G., Kompella, R., Liu, S., and Chang, S. Reversing the forget-retain objectives: An efficient LLM unlearning framework from logit difference. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. URL http://papers.nips.c/paper_files/paper/2024/hash/171291d8fed723c6dfc76330a827f8-Abstract-Conference.html. Jia et al. (2023) Jia, J., Liu, J., Ram, P., Yao, Y., Liu, G., Liu, Y., Sharma, P., and Liu, S. Model sparsity can simplify machine unlearning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL http://papers.nips.c/paper_files/paper/2023/hash/a204a68ab4e970e1ceccfb5b5cdc5e4-Abstract-Conference.html. Jia et al. (2024) Jia, J., Zhang, Y., Zhang, Y., Liu, J., Runwal, B., Diffenderfer, J., Kailkhura, B., and Liu, S. SOUL: unlocking the power of second-order optimization for LLM unlearning. In Al-Onaizan, Y., Bansal, M., and Chen, Y. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024, p. 4276–4292. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.EMNLP-MAIN.245. URL https://doi.org/10.18653/v1/2024.emnlp-main.245. Koh & Liang (2017) Koh, P. W. and Liang, P. Understanding black-box predictions via influence functions. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, Proceedings of Machine Learning Research, p. 1885–1894. PMLR, 2017. URL http://proceedings.mlr.press/v70/koh17a.html. Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto, 2009. Kurmanji et al. (2023) Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. Towards unbounded machine unlearning. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL http://papers.nips.c/paper_files/paper/2023/hash/062d711fb777322e2152435459e6e9d9-Abstract-Conference.html. Le & Yang (2015) Le, Y. and Yang, X. Tiny imagenet visual recognition challenge. CS 231N, 7(7):3, 2015. Liu et al. (2024) Liu, Z., Dou, G., Tan, Z., Tian, Y., and Jiang, M. Towards safer large language models through machine unlearning. In Ku, L., Martins, A., and Srikumar, V. (eds.), Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Findings of ACL, p. 1817–1829. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.FINDINGS-ACL.107. URL https://doi.org/10.18653/v1/2024.findings-acl.107. Nguyen et al. (2025) Nguyen, T. T., Huynh, T. T., Ren, Z., Nguyen, P. L., Liew, A. W., Yin, H., and Nguyen, Q. V. H. A survey of machine unlearning. ACM Trans. Intell. Syst. Technol., 16(5):108:1–108:46, 2025. doi: 10.1145/3749987. URL https://doi.org/10.1145/3749987. Sekhari et al. (2021) Sekhari, A., Acharya, J., Kamath, G., and Suresh, A. T. Remember what you want to forget: Algorithms for machine unlearning. In Ranzato, M., Beygelzimer, A., Dauphin, Y. N., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, p. 18075–18086, 2021. URL https://proceedings.neurips.c/paper/2021/hash/9627c45df543c816a3ddf2d8ea686a99-Abstract.html. Springenberg et al. (2015) Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. A. Striving for simplicity: The all convolutional net. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Workshop Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6806. Tarun et al. (2024) Tarun, A. K., Chundawat, V. S., Mandal, M., and Kankanhalli, M. S. Fast yet effective machine unlearning. IEEE Trans. Neural Networks Learn. Syst., 35(9):13046–13055, 2024. doi: 10.1109/TNNLS.2023.3266233. URL https://doi.org/10.1109/TNNLS.2023.3266233. Voigt & Bussche (2017) Voigt, P. and Bussche, A. v. d. The EU General Data Protection Regulation (GDPR): A Practical Guide. Springer Publishing Company, Incorporated, 1st edition, 2017. ISBN 3319579584. Wuerkaixi et al. (2025) Wuerkaixi, A., Wang, Q., Cui, S., Xu, W., Han, B., Niu, G., Sugiyama, M., and Zhang, C. Adaptive localization of knowledge negation for continual LLM unlearning. In Singh, A., Fazel, M., Hsu, D., Lacoste-Julien, S., Berkenkamp, F., Maharaj, T., Wagstaff, K., and Zhu, J. (eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. URL https://proceedings.mlr.press/v267/wuerkaixi25a.html. Xu et al. (2024) Xu, H., Zhu, T., Zhang, L., Zhou, W., and Yu, P. S. Machine unlearning: A survey. ACM Comput. Surv., 56(1):9:1–9:36, 2024. doi: 10.1145/3603620. URL https://doi.org/10.1145/3603620. Yu et al. (2026) Yu, L., Zhao, Z., Wang, Y., Wang, P., Wang, B., and Wang, Y. Falw: A forgetting-aware loss reweighting for long-tailed unlearning. CoRR, abs/2601.18650, 2026. doi: 10.48550/ARXIV.2601.18650. URL https://doi.org/10.48550/arXiv.2601.18650. Appendix Appendix A Theoretical Analysis Notation. The original model Θo _o is trained on the full dataset D. We partition D into a retain set DrD_r and a forget set DfD_f, where Df=Dfb∪DfnD_f=D_f^b∪ D_f^n, with: • DfbD_f^b (Boundary samples): The absolute logit value is small (|fy(xb;Θo)|≤ε|f_y(x_b; _o)|≤ ), which indicates low model confidence (high uncertainty). These samples lie near the decision boundary, and consequently their loss gradient norm ‖∇θℓ(zb)‖\| _θ (z_b)\| is large. • DfnD_f^n (Normal samples): The absolute logit value is large (|fy(xn;Θo)|≥L|f_y(x_n; _o)|≥ L), which indicates high model confidence (low uncertainty). These samples lie far from the decision boundary, and consequently their loss gradient norm ‖∇θℓ(zn)‖\| _θ (z_n)\| is small. Remark. For cross-entropy loss, a small absolute logit (near the decision boundary) implies low confidence, high uncertainty, and a large gradient norm, and vice versa. Definition. For a sample z=(x,y)z=(x,y) under the original model Θo _o, the strength of memorization M(z)M(z) is defined as the magnitude of parameter change required after removing this sample from the training set. This concept can be quantified using the influence function. Let HΘo=1|D|∑z∈D∇Θ2ℓ(z;Θ)|Θ=ΘoH_ _o= 1|D| _z∈ D∇^2_ (z; ) |_ = _o (5) be the empirical Hessian matrix, where ℓ is the cross-entropy loss. The influence function of sample z is defined as: ℐ(z)=−HΘo−1∇Θℓ(z;Θo).I(z)=-H_ _o^-1 _ (z; _o). (6) The strength of memorization is defined as the norm of the influence function: M(z)=‖ℐ(z)‖2∝‖∇Θℓ(z;Θo)‖H−1.M(z)=\|I(z)\|_2 \| _ (z; _o)\|_H^-1. (7) As a first-order approximation, the strength of memorization correlates positively with the gradient norm: M(z)∝‖∇Θℓ(z;Θo)‖2.M(z) \| _ (z; _o)\|_2. (8) Theorem A.1 (Memorization Strength Disparity). Let zb∈Dfbz_b∈ D_f^b be a boundary sample and zn∈Dfnz_n∈ D_f^n be a normal sample. Under the original model Θo _o, the memorization strength of boundary samples is significantly larger than that of normal samples, i.e., M(zb)≫M(zn).M(z_b) M(z_n). Proof. From the notion, a boundary sample zbz_b lies near the decision boundary, having a small absolute logit, which implies low model confidence and high uncertainty. Consequently, its loss gradient norm ‖∇Θℓ(zb;Θo)‖2\| _ (z_b; _o)\|_2 is large. In contrast, a normal sample znz_n lies far from the decision boundary, having a large absolute logit, which implies high model confidence and low uncertainty. Consequently, its loss gradient norm ‖∇Θℓ(zn;Θo)‖2\| _ (z_n; _o)\|_2 is small. By Definition A, the strength of memorization M(z)M(z) is positively correlated with the loss gradient norm. Therefore, M(zb)≫M(zn).M(z_b) M(z_n). (9) ∎ Theorem A.2 (Coarse-Grained Forgetting Dilemma). When parameter updates are applied in a coarse-grained and undifferentiated manner across all samples, a fundamental dilemma emerges: samples with strong memorization (boundary samples) are left in an under-unlearned state, while samples with weak memorization (normal samples) are left in an over-unlearned state. Proof. Consider the coarse-grained forgetting update: ΔΘ=η⋅1|Df|∑z∈Df∇Θℓ(z;Θo),η>0, =η· 1|D_f| _z∈ D_f _ (z; _o), η>0, (10) where ΔΘ=Θu−Θo = _u- _o. Partition the forget set DfD_f into boundary samples DfbD_f^b and normal samples DfnD_f^n. Define the average gradients: gb=1|Dfb|∑zb∈Dfb∇Θℓ(zb),gn=1|Dfn|∑zn∈Dfn∇Θℓ(zn).g_b= 1|D_f^b| _z_b∈ D_f^b _ (z_b), g_n= 1|D_f^n| _z_n∈ D_f^n _ (z_n). (11) By Theorem A.1, ‖gb‖≫‖gn‖\|g_b\| \|g_n\|, with boundary samples lying in high-curvature regions and normal samples in low-curvature (saturated) regions. Thus the overall update direction is dominated by gbg_b: ΔΘ≈η⋅|Dfb||Df|gb. ≈η· |D_f^b||D_f|g_b. (12) Boundary samples reside in high-curvature regions where the loss is highly sensitive to parameter changes and gradients are steep. Ideal forgetting requires moving parameters along the individual gradient direction of each sample by a moderate step size. However: • Direction bias: The coarse-grained update uses the average gradient gbg_b over all boundary samples. Due to variability in gradient directions across samples, this average direction may not align precisely with the steepest forgetting direction for any individual zbz_b. • Constrained step size: The global learning rate η must be kept small to prevent a sharp rise in loss on the retain set DrD_r (avoiding catastrophic forgetting). Although ‖gb‖\|g_b\| is large, the product ‖ΔΘ‖=η‖gb‖\| \|=η\|g_b\| remains limited. • High-curvature instability: After a single large-step update, gradient directions in high-curvature regions change dramatically. A one-step coarse-grained update rarely lands exactly in the “forgotten” state, and without subsequent targeted adjustments, memory persists. Thus, boundary samples are often in a relatively under-unlearned state. Normal samples reside in low-curvature (flat) regions where their own gradients satisfy ∇Θℓ(zn)≈0 _ (z_n)≈ 0 and the Hessian eigenvalues are extremely small at the basin center. Expanding ℓ(zn) (z_n) around Θo _o: ℓ(zn;Θo+ΔΘ)≈ℓ(zn;Θo)+∇Θℓ(zn)⊤ΔΘ+12ΔΘ⊤HnΔΘ+⋯. split (z_n; _o+ )&≈ (z_n; _o)+ _ (z_n) \\ & + 12 H_n +·s. split (13) At the flat basin center, the first-order term vanishes and the second-order term is also very small for sufficiently small ΔΘ . However, although the displacement ΔΘ is a small perturbation from a global optimization perspective (due to the small learning rate η), it is large relative to the effective radius of the locally flat basin where znz_n resides, because the Hessian eigenvalues are extremely small in that region. This displacement is driven entirely by the needs of boundary samples, not by the local geometry of znz_n. Such a large displacement pushes the parameters away from the flat basin center toward regions where the curvature of the loss landscape increases sharply (the basin boundary). As ΔΘ reaches these high-curvature regions, the second-order term 12ΔΘ⊤HnΔΘ 12 H_n and higher-order terms rapidly dominate, leading to a significant increase in loss: ℓ(zn;Θo+ΔΘ)≫ℓ(zn;Θo). (z_n; _o+ ) (z_n; _o). (14) Consequently, a normal sample originally at the bottom of a flat basin is forcibly ejected from its low-loss region by the large, boundary-sample-driven update. This change is not guided by znz_n’s own gradient but is a form of “interference” from other samples, resulting in representation collapse and leaving normal samples in an over-unlearned state. Effective forgetting of boundary samples requires precise directional alignment. Preserving normal samples requires minimal parameter displacement, yet coarse-grained updates inevitably produce large displacements driven by gbg_b. No single η can simultaneously satisfy these conflicting requirements. Hence, coarse-grained undifferentiated updates inevitably suffer from the dilemma: boundary samples are under-unlearned forgotten while normal samples are over-unlearned. ∎ Appendix B Detailed Related Work Machine unlearning was first proposed by (Cao & Yang, 2015), introducing a method that stores statistical aggregates during training and updates them by subtracting the contribution of deleted data points before reconstructing the model. (Bourtoule et al., 2021) proposed the SISA training framework, which strategically limits the influence of data samples during the training process to accelerate the forgetting process. For approximate unlearning, a variety of methods have been developed to efficiently erase data influence through post-training procedures. (Graves et al., 2021) records the data contained in each training batch and the corresponding parameter updates during the training phase. When deleting, they directly subtract from the final model parameters the updates produced by batches that contain the data to be forgotten. (Golatkar et al., 2020a) propose a model weight erasure method: the weights are first fine-tuned toward a state trained solely on retain data using a single Newton step, and then directional noise is added to disrupt the memory of forgotten data. They extended this framework to activation functions (Golatkar et al., 2020b). (Golatkar et al., 2021) propose Mixed Linear Forgetting, which decomposes a deep network into non-linear “core weights” that do not require forgetting and linear “user weights” that can be efficiently forgotten. (Guo et al., 2020) employ a Newton update removal mechanism, computing the influence vectors of the forgotten data points on the model and updating the parameters accordingly. (Izzo et al., 2021) propose projected residual updates, whose time complexity is linear in the dimension of the data to be deleted and independent of the dataset size. However, this method is primarily designed for linear models, which limits its scope of application. (Fan et al., 2024) propose SALUN, which selectively updates a subset of weights using weight significance masks, thereby efficiently removing the influence of specific samples or classes while preserving overall model performance. (Chundawat et al., 2023; Kurmanji et al., 2023) employ a teacher-student framework for machine unlearning. In their approaches, the former proposes a dual-teacher mechanism: the student model mimics the original model (the qualified teacher) on the retain set, and mimics a randomly initialized model (the unqualified teacher) on the forget set. The latter, in contrast, uses only the original model as the teacher, training the student model to stay close to the teacher on the retain set while moving away from the teacher on the forget set. (He et al., 2025) proposed Natural Unlearning to generate new samples by mixing each forget sample with related samples from the retain set, and then fine-tuning the original model on these mixed samples. Another line of work addresses the class-wise forgetting task, which requires the model to completely remove knowledge of an entire concept or category. (Tarun et al., 2024) propose an efficient machine unlearning method called UNSIR, which learns “error-maximizing noise” for the target class and combines a “damage-repair” two-step weight update to achieve thorough forgetting of single or multiple classes in a single training pass. (Chen et al., 2023, 2025) shift the focus from parameter space to decision space. Boundary Shrink applies adversarial perturbations to assign incorrect labels to forget samples and fine-tunes the model, thereby compressing the decision boundary. In contrast, Boundary Expanding introduces a new neuron for a “shadow class” to disperse forget samples into other categories, actively expanding the decision boundary. For long-tailed distributions, LTMU (Chen et al., 2025) generates augmented features based on the similarity of tail-class samples and reassigns labels according to distance, enabling effective unlearning of tail classes. (Yu et al., 2026) proposes a dynamic loss reweighting method named FaLW, which mitigates issues of heterogeneous unlearning deviation and skewed unlearning deviation by estimating the unlearning deviation for each sample and introducing a class-wise balancing factor. Beyond traditional deep learning models, machine unlearning research is gradually extending to broader modeling paradigms and diverse application scenarios. In large language models, unlearning can be used to remove the model’s memory of useless or harmful knowledge (Jia et al., 2024; Liu et al., 2024; Wuerkaixi et al., 2025; Ji et al., 2024). Appendix C Experimental Setup Dataset. The basic information about the dataset we used is summarized as follows. • CIFAR-10 (Krizhevsky et al., 2009): it contains 60,00060,000 colored images of size 32×32×332× 32× 3 with 10,00010,000 reserved for testing. There are 1010 target classes with 5,0005,000 training images per class. • CIFAR-100 (Krizhevsky et al., 2009): it contains 60,00060,000 colored images with 10,00010,000 reserved for testing. There are 100100 target classes with 500500 training images per class. • Tiny-ImageNet: Tiny ImageNet (Le & Yang, 2015) contains 120,000 colored images of size 64×64×364× 64× 3 with 10,000 reserved for validation and 10,000 reserved for testing. There are 200 target classes with 500 training images per class. • Lacuna-10: A subset selected from VGGFace (Cao & Yang, 2015). We randomly select 10 celebrities (classes) with at least 500 samples. Baselines. We include and compare against various SOTA machine unlearning methods, which are briefly summarized as follows. • Original: the model trained on all data D without performing any unlearning. • Retrain: retraining the model from scratch without the forget set, i.e., only rD_r, which is assumed not to be viable in practice, but included as a reference. • Fine-tuning: finetune the original model on rD_r. • Gradient Ascent (Golatkar et al., 2020a): finetune the original model on the forget set fD_f by negating the gradient. • Random Labels (Graves et al., 2021; Hayase et al., 2020): finetune the original model on the random relabeled forget data. • Influence Unlearning (Koh & Liang, 2017; Izzo et al., 2021): utilize an influence function to estimate the updates required for the model weights as a result of removing fD_f from D. • EU-k (Exact Unlearning-k) and CF-k (Catastrophic Forgetting-k) (Goel et al., 2022): freeze the first k layers of the original model and either train the remaining layers from scratch on rD_r or finetune the remain layers on rD_r. • ℓ1 _1-sparse (Jia et al., 2023): introduce ℓ1 _1-norm for model weights and finetune the model over rD_r. • SalUn (Fan et al., 2024): draw a parallel with input saliency in model explanation to only update influential weights filtered by the threshold, and integrate with Random Label. • Bad-Teaching (Chundawat et al., 2023): encourage the student to move close to an incompetent/dumb teacher (randomly initialized model or random generators) for the forget set and close to the original model for the retain set. • SCRUB (Kurmanji et al., 2023): move the student model away from the teacher model and formulate a min-max bi-optimization problem by extending contrastive learning. Besides, we include the following specific unlearning method for the class-wise forgetting task. • Boundary Shrink and Boundary Expanding (Chen et al., 2023): is designed to destroy the decision boundary by using adversarial samples for the forget data, and to exploit a new area in the decision space by assigning forget samples to an extra shadow class of the original model, with the replacement of the last layer. • UNSIR (Tarun et al., 2024): learn an error-maximizing noise matrix for the class to be unlearned and uses it to impair the forget class and repair the original model. • LTMU (He et al., 2025): Directionally rectifies the decision boundary of tail classes and guides their features to contract toward the feature space of neighboring classes for long-tailed unlearning. For the random-subset forgetting task, we choose to forget about ≈10%≈ 10\% of the training data. Therefore, for the CIFAR-10 and CIFAR-100, we have |f|=5K|D_f|=5K, and the sizes of train set D, test set tD_t, and validation set vD_v correspond to 45K45K, 10K10K, 5K5K respectively. For the Tiny-ImageNet dataset, we have |f|=10K|D_f|=10K. For the class-wise forgetting task, we used CIFAR-10 and CIFAR-100, and randomly chose class 44 as the forgetting class for both datasets. Consequently, we have |f|=5K|D_f|=5K for CIFAR-10 and |f|=500|D_f|=500 for CIFAR-100. In addition, for Lacuna-10, a subset selected from VGGFace (Cao & Yang, 2015) where each selected class contains at least 500 samples, we also randomly choose class 44 as the forgetting class. Model. For All-CNN (Springenberg et al., 2015), we reduce the number of layers and introduce batch normalization before each non-linearity layer, training the model from scratch. For ResNet, we adopt different architectures for different unlearning tasks. For Random-subset forgetting, we utilize the standard ResNet-18 architecture, starting with the pre-trained PyTorch model (trained on ImageNet) and fine-tuning it on the target dataset. For class-wise forgetting, we employ a CIFAR-adapted ResNet-18 architecture, where the first 7×77× 7 convolutional layer is replaced with a 3×33× 3 convolutional layer and the initial max-pooling layer is removed to accommodate 32×3232× 32 inputs, training the model from scratch as the original model. In addition, following (Kurmanji et al., 2023), we scale up to a larger setting by choosing larger models and tackling a larger classification task, that is, we used the pre-trained VGG-16 (Hayase et al., 2020) and ViT (Dosovitskiy et al., 2021) models, finetuning them on the CIFAR-100 dataset. LiRA setup. To evaluate the privacy protection effectiveness of MU methods, we implemented the U-LiRA (Hayes et al., 2025) membership inference attack based on the log-likelihood ratio, which is considered the strongest per-sample attack benchmark for unlearning scenarios to date. To evaluate the privacy protection effectiveness of MU methods, we implemented the U-LiRA (Hayes et al., 2025) membership inference attack based on the log-likelihood ratio, which is considered the strongest per-sample attack benchmark for unlearning scenarios to date. The specific attack procedure is as follows: • Distribution Construction and Sample Filtering: For each sample in the restricted sampling pool, we iterated over all shadow models and filtered them based on the sample’s membership status in the models. Specifically, we collected the model outputs when the sample was in the forget set to construct the foutf_out distribution, and collected the model outputs when it was in the unseen sample set to construct the finf_in distribution. Thanks to the aforementioned restricted sampling strategy, each target sample obtained a sufficient number of shadow model observations (N=128N=128) in both states, meeting the statistical requirements for precise Gaussian distribution fitting. • Attack Scoring: For each target variant model, we used the log-likelihood ratio of the query sample under the two Gaussian distributions as the attack score. • Evaluation Metrics:We conducted evaluations within the fixed pool, selecting samples that were neither in the forget set nor in the retain set as unseen test samples. To ensure unbiased evaluation metrics, we strictly implemented a 1:1 downsampling balancing strategy for positive and negative samples when calculating the AUC and attack accuracy. We conduct experiments on CIFAR-10 using the standard ResNet-18 architecture. To simulate realistic scenarios and construct distributions for membership inference attacks, following (Hayes et al., 2025), we randomly sample 25,000 samples from the CIFAR-10 training set as the training set for each model. U-LiRA partitions all base models into 128 shadow models and 128 target models, used for estimating distribution parameters and evaluating attack accuracy, respectively. To ensure statistical significance with a limited number of variants, we introduce a restricted sampling pool mechanism: we pre-sample a fixed pool of 1,000 samples from class 5 of CIFAR-10. Unlike the original U-LiRA’s large-scale repeated sampling, we adopt a more efficient strategy — when constructing the forget set for each variant, we sample only 200 samples from the intersection of the pool and the base model’s training set. Each base model randomly samples 5 different forget sets, generating 5 variants, resulting in 1,280 trained models in total. This strategy effectively increases the probability that samples from the fixed pool appear in the forget sets of various variants, ensuring that samples receive sufficient observations to accurately fit the distributions even when the total number of models is reduced. Appendix D More Experimental Results. D.1 Random-subset Forgetting. Figure 3: (a) Speedup w.r.t. Retrain versus FaccF_acc for vit on CIFAR-100; (b) Speedup w.r.t. Retrain versus FaccF_acc for ResNet50 on Tiny ImageNet. Figure 3 illustrates the trade-off between speedup and FaccF_acc under two settings: (a) ViT on CIFAR-100, and (b) ResNet50 on Tiny ImageNet. FaccF_acc jointly considers unlearning effectiveness and model generalization. On this metric, GSUO outperforms all baselines, achieving state-of-the-art performance. In contrast, methods such as FT, SalUN, and IU either retain high accuracy on the forget set (indicating incomplete unlearning) or suffer from a significant drop in accuracy on the test set (indicating poor generalization). While efficiency-oriented methods like GA achieve shorter runtime, their low FaccF_acc reveal a failure to properly balance the trade-off between unlearning effectiveness and model generalization. In contrast, GSUO consistently maintains its superiority across different datasets and model architectures, demonstrating strong scalability and robustness. Overall, GSUO not only surpasses all competing methods in overall performance but also flexibly adapts to unlearning tasks of various scales, making it a practical and robust solution. Figure 4: (a) MIA performance of various UM methods on the Random-subset forgetting task; (b) MIA performance of various UM methods on the class-wise forgetting task. Figure 4(a) presents the MIA attack success rate of various unlearning methods on the Random-subset forgetting task, using an SVM attack model with loss as the attack feature. The MIA accuracy of GSUO is very close to 0.5, demonstrating strong protection of the privacy of the data intended to be forgotten and confirming truly effective unlearning. In contrast, methods such as IU and GA fail to achieve this level of privacy protection, with their MIA accuracy deviating significantly from 0.5. D.2 Class-wise Forgetting. Table 6: Comparison of MU methods for class-wise forgetting. Left: ALLCNN on Cifar10; Right: Resnet18 on Cifar10 Method ALLCNN on Cifar10 Resnet18 on Cifar10 Acc(f)Acc(D_f) ↓ Acc(r)Acc(D_r) ↑ Time (s) ↓ Acc(f)Acc(D_f) ↓ Acc(r)Acc(D_r) ↑ Time (s) ↓ Original 83.40 82.48 189.32 85.12 85.40 1038.50 Retrain 0.00 85.10 428.64 0.00 84.07 459.81 GA 12.00 76.49 5.62 9.40 77.50 145.99 B-Shrink 10.40 75.43 48.59 1.20 77.08 309.46 B-Expand 10.90 78.34 9.71 11.50 64.90 27.32 LTMU 0.00 84.67 24.16 0.00 85.01 53.27 SCRUB 0.20 86.02 146.05 0.20 86.02 146.05 Bad-T 45.60 78.40 268.24 58.00 73.14 102.94 SalUn 0.00 85.47 50.37 0.00 85.32 22.03 RL 1.10 75.93 203.33 9.70 79.91 6.74 UNSIR 0.30 80.16 6.13 9.10 83.04 7.79 L1-sparse 0.00 77.58 28.93 0.00 81.26 19.53 FT 0.00 86.02 93.52 0.00 85.04 18.91 IU 13.10 77.22 2.68 12.30 80.77 2.83 GSUO 0.00 86.92 47.86 0.00 86.24 42.55 Table 6 presents the results of class unlearning on CIFAR-10 using the ALLCNN and ResNet18 models. Our method GSUO achieves 0% accuracy on the forget set test split, while attaining the highest accuracy on the retain set test split, surpassing all baselines. This demonstrates that GSUO effectively removes the targeted class information while well preserving the model’s utility. In terms of efficiency, GSUO achieves nearly a 10× speedup compared to retraining. Although it is slightly less efficient than methods such as IU and UNSIR, these baselines either suffer from high accuracy on the forget set, indicating incomplete unlearning or cause a significant drop in retain accuracy, leading to poor generalization. Overall, GSUO enables efficient unlearning and shows significant advantages in both effectiveness and efficiency, making it a practical and robust choice for real-world unlearning tasks. Figure 4(b) presents the MIA attack success rate of various unlearning methods on the class-wise forgetting task. The MIA accuracy of GSUO is close to 0.5. Although methods such as SCRUB and RL achieve values even closer to 0.5, considering their overall unlearning effectiveness, they lag behind GSUO in terms of the trade-off between privacy protection and model utility. D.3 Performance for imbalanced datasets. To investigate the performance of GSUO on imbalanced data, we construct imbalanced datasets using an exponential decay strategy. Dataset Specifically, we build a long-tailed dataset CIFAR-10-LT based on CIFAR-10, where a few head classes have abundant samples while many tail classes have only limited samples. The number of training samples per class is controlled by: Ni=Nmax×(NminNmax)iC−1N_i=N_max× ( N_minN_max ) iC-1 (15) where C is the total number of classes, NmaxN_max and NminN_min denote the maximum and minimum sample sizes per class, and i indexes the classes sorted from the most frequent to the least frequent. Evaluation Metric To address the potential performance imbalance between head classes (majority classes with many samples) and tail classes (minority classes with few samples) in long-tailed distributions, we introduce two fairness metrics to evaluate model performance across different classes.The first metric is the Weighted Accuracy Difference (WADWAD), defined as follows: WAD=C∑c=1Cpc⋅(Accc−Acc)2WAD= C _c=1^Cp_c·(Acc_c-Acc)^2 (16) where pcp_c is the sample proportion of class c, AcccAcc_c is the model’s accuracy on class c, and AccAcc is the overall average accuracy. A smaller WADWAD indicates more balanced performance across classes.The second metric is the Head-Tail Performance Ratio (HTRHTR), defined as: HTR=1|Gh|∑c∈GhAccc1|Gt|∑c∈GtAcccHTR= 1|G_h| _c∈ G_hAcc_c 1|G_t| _c∈ G_tAcc_c (17) where GhG_h and GtG_t denote the sets of head classes and tail classes, respectively, and |Gh||G_h| and |Gt||G_t| are their corresponding cardinalities. HTRHTR measures the ratio of average accuracy between head and tail classes, reflecting the model’s fairness toward tail classes. Table 7: Performance of Different UM Methods on CIFAR-10-LT. Method Head class Tail class Acc(fD_f) % ↓ Acc(rD_r) % ↑ WAD ↓ HTR Acc(fD_f) % ↓ Acc(rD_r) % ↑ WAD ↓ HTR Original 85.5 44.5 0.1997 1.9225 69.2 60.3 0.1761 1.4678 Retrain 0 64.0 0.1251 1.2605 0 73.1 0.1229 1.2069 GA 23.2 43.5 0.2250 1.5279 3.5 56.7 0.2220 1.9328 B-Shrink 0.2 37.7 0.2737 1.4455 2.2 55.6 0.2460 1.8436 B-Expand 12.0 42.3 0.1314 0.9793 2.2 55.9 0.2427 1.8105 LTMU 0 50.3 0.1940 1.5196 0.6 59.0 0.1974 1.7225 SCRUB 2.4 65.8 0.1228 1.2312 15.2 70.6 0.1238 1.2308 Bad-T 77.5 62.8 0.1445 1.2777 47.9 67.4 0.1963 1.4779 SalUn 0 48.6 0.1792 1.6420 5.6 71.0 0.1212 1.2490 RL 0 37.8 0.1233 0.8997 1.8 55.6 0.2401 1.7449 UNSIR 0 42.4 0.2222 0.3938 0 61.1 0.1831 0.9182 L1-sparse 0 44.8 0.2607 1.3600 0 73.4 0.1242 1.2903 FT 0 57.6 0.1633 1.3435 0 75.8 0.1083 1.1547 IU 23.2 44.4 0.2714 1.4436 0 62.2 0.1956 1.6674 GSUO 0 54.3 0.1428 1.0579 0 74.6 0.1080 1.1613 Table 7 presents the performance of various unlearning methods on CIFAR-10-LT. We evaluate both head and tail classes. Results show that GSUO achieves complete unlearning while maintaining high retain test accuracy, surpassing both the original model and methods such as B-shrink, B-Expand, and LTMU. On tail classes, the test accuracy of GSUO even exceeds that of retrain. From the perspective of fairness, GSUO also outperforms many baselines. For the evaluation metrics, a smaller WADWAD indicates better performance, while HTRHTR values closer to 11 are preferable. As shown in Table 7, GSUO ranks among the top, achieving an HTRHTR of 1.05791.0579 on head classes and a WADWAD of 0.10800.1080 on tail classes. In contrast, methods such as B-shrink, IU, and SalUN fail to adequately balance performance between head and tail classes. D.4 Multi-Class Forgetting. Table 8: Comparison of MU methods for multi-class forgetting on Cifar10 under ALLCNN. Method Acc(r)Acc(D_r) ↑ Acc(rt)Acc(D_rt) ↑ Acc(f)Acc(D_f) ↓ Acc(ft)Acc(D_ft) ↓ Time (s) ↓ Original 92.70 84.59 88.23 74.50 205.81 Retrain 99.96 89.89 0.00 0.00 401.91 FT 99.41 90.31 0.00 0.00 84.21 GA 71.73 67.35 3.39 3.10 21.02 RL 77.99 70.57 0.06 0.05 31.35 EU-k 96.54 90.09 0.00 0.00 398.99 IU 90.46 84.16 42.98 34.50 4.22 ℓ1 _1-sparse 86.04 82.55 0.00 0.00 422.61 SalUn 99.58 90.30 0.04 0.05 499.14 Bad-T 92.14 86.35 81.56 70.70 121.34 SCRUB 99.72 92.26 29.06 21.65 265.85 B-Shrink 70.46 65.86 4.08 3.75 29.73 B-Expand 90.35 83.50 28.88 23.10 228.35 UNSIR 87.33 83.29 0.06 0.05 11.26 GSUO 98.85 90.75 0.00 0.00 52.64 We evaluated multi-class forgetting performance of various machine unlearning methods on the CIFAR-10 dataset with the AllCNN architecture. We selected class 3 and class 4. The results are shown in Table 8. while methods like FT, EU-k, and ℓ1 _1-sparse achieve complete forgetting, others such as GA, IU, SalUn, Bad-T, SCRUB, and UNSIR fail to fully erase the target information. Notably, among all evaluated methods, GSUO achieves the best balance between forgetting quality and model utility. Furthermore, GSUO achieves an 8× speedup compared to retraining. Although IU has the greatest advantage in runtime, its forgetting quality is poor. In summary, GSUO achieves an optimal trade-off among forgetting completeness, model utility, and computational efficiency in multi-class forgetting tasks, validating its effectiveness in complex class-incremental scenarios. D.5 Visual Analysis. Figure 5: (a)-(c) Distribution of prediction changes on three forgotten samples (Random-subset forgetting). Figure 5 illustrates the distribution of three samples before and after forgetting in Random-subset forgetting. Panels (a) and (b) show predicted distributions of randomly selected Boundary samples before and after forgetting, while (c) shows a randomly selected Normal sample. Boundary samples typically not only have high probability on their true class but also exhibit a spread of probability mass over several other classes. After forgetting, the boundary samples undergo a probability mass transfer across the boundary toward the most similar class. This change encourages the boundary samples to be naturally forgotten while causing minimal damage to neighboring sample points. Normal samples, which are originally located in the dense region of the entire class data, tend to remain correctly predictable after forgetting, but their confidence decreases, as shown in (c). Visualization of predicted distributions shows that GSUO successfully forgets the target samples. Figure 6: Distribution of prediction changes on forgotten samples (class-wise forgetting): (a) Original Model; (b) Retrained Model; (c) Unlearned Model. Figure 6 illustrates the feature space distributions of the original model, the retrained model, and the unlearned model in class-wise forgetting. The experiments use CIFAR-10 with the AllCNN model. Features are projected onto a two-dimensional plane using t-SNE. The unlearned model’s distribution closely resembles that of the retrained model. The intra-class structure of the forgotten data in the feature space tends to become loose, dispersedly distributed in the central vacant regions of the retain data. Most forgotten samples are predicted by the model as the class labels of the nearest retain data in the feature space. The above results demonstrate that GSUO achieves the desired forgetting effect in the class-wise forgetting scenario, making its distribution in the feature space highly similar to that of the “gold standard”—the retrained model.