Paper deep dive
ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Patching
Yi Sun, Xinhao Zhong, Hongyan Li, Yimin Zhou, Junhao Li, Bin Chen, Xuan Wang
Models: Stable Diffusion v1.5
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/12/2026, 5:28:33 PM
Summary
ActErase is a training-free, plug-and-play paradigm for concept erasure in text-to-image diffusion models. It operates by identifying activation difference regions in Feed-Forward Network (FFN) layers via prompt-pair analysis and dynamically patching target activations with source activations during the forward pass. This method achieves state-of-the-art erasure performance across nudity, artistic style, and object removal tasks while maintaining generative quality and robustness against adversarial attacks.
Entities (5)
Relation Signals (3)
ActErase → evaluatedon → I2P
confidence 95% · we evaluate ActErase on 4,703 prompts from the I2P dataset
ActErase → targets → Feed-Forward Network
confidence 95% · our work specifically targets FFN layers for CE.
ActErase → improves → Stable Diffusion v1.5
confidence 90% · ActErase can effectively erase ‘nudity’ while maximally preserving the original semantic content of the image
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent advances in text-to-image diffusion models have demonstrated remarkable generation capabilities, yet they raise significant concerns regarding safety, copyright, and ethical implications. Existing concept erasure methods address these risks by removing sensitive concepts from pre-trained models, but most of them rely on data-intensive and computationally expensive fine-tuning, which poses a critical limitation. To overcome these challenges, inspired by the observation that the model's activations are predominantly composed of generic concepts, with only a minimal component can represent the target concept, we propose a novel training-free method (ActErase) for efficient concept erasure. Specifically, the proposed method operates by identifying activation difference regions via prompt-pair analysis, extracting target activations and dynamically replacing input activations during forward passes. Comprehensive evaluations across three critical erasure tasks (nudity, artistic style, and object removal) demonstrates that our training-free method achieves state-of-the-art (SOTA) erasure performance, while effectively preserving the model's overall generative capability. Our approach also exhibits strong robustness against adversarial attacks, establishing a new plug-and-play paradigm for lightweight yet effective concept manipulation in diffusion models.
Tags
Links
- Source: https://arxiv.org/abs/2601.00267
- Canonical: https://arxiv.org/abs/2601.00267
Trouble viewing inline? Open PDF directly →
Full Text
61,561 characters extracted from source content.
Expand or collapse full text
ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Patching Yi Sun 1 * Xinhao Zhong 1∗ Hongyan Li 1 Yimin Zhou 2 Junhao Li 1 Bin Chen 1,3† Xuan Wang 1,3 1 Harbin Institute of Technology, Shenzhen 2 Tsinghua Shenzhen International Graduate School, Tsinghua University 3 Peng Cheng Laboratory Abstract Recent advances in text-to-image diffusion models have demonstrated remarkable generation capabilities, yet they raise significant concerns regarding safety, copyright, and ethical implications. Existing concept erasure methods ad- dress these risks by removing sensitive concepts from pre- trained models, but most of them rely on data-intensive and computationally expensive fine-tuning, which poses a crit- ical limitation. To overcome these challenges, inspired by the observation that the model’s activations are predom- inantly composed of generic concepts, with only a mini- mal component can represent the target concept, we pro- pose a novel training-free method (ActErase) for efficient concept erasure. Specifically, the proposed method oper- ates by identifying activation difference regions via prompt- pair analysis, extracting target activations and dynami- cally replacing input activations during forward passes. Comprehensive evaluations across three critical erasure tasks (nudity, artistic style, and object removal) demon- strates that our training-free method achieves state-of-the- art (SOTA) erasure performance, while effectively preserv- ing the model’s overall generative capability. Our approach also exhibits strong robustness against adversarial attacks, establishing a new plug-and-play paradigm for lightweight yet effective concept manipulation in diffusion models. 1. Introduction In recent years, text-to-image (T2I) [7, 13, 14, 25, 32, 37] generation has received significant attention due to its capa- bility to produce high-quality images from textual prompts. However, the training datasets for these models are often sourced from the Internet [23], which may lead to the gen- * Equal Contribution. † Corresponding Author. ESD UCE Concept-prune AdaVD STEREO Ours Figure 1. Comparison of general performance metrics (left) and performance of each task (right) for diffusion models. Metrics include efficacy, robustness, efficiency, preservation and quality. The image clearly shows that our method achieves competitive and even superior performance across multiple evaluation dimensions. eration of inappropriate content [15, 29, 36, 40]. One po- tential solution is to remove the sensitive data and retrain the model from scratch [25, 33, 38]. Yet, this approach is computationally expensive, inefficient, and unpredictable, as it may introduce biases and degrade the model’s gener- ation capabilities [26]. To mitigate these challenges, con- cept erasure (CE), a lightweight yet effective methods, have been developed. CE aims to prevent the generation of im- ages containing specific target concepts while maintaining the model’s overall generative capability. Most existing CE methods are training-based [3, 5, 17, 18, 22]. These approaches typically fine-tune the model pa- rameters using carefully designed loss functions and prompt pairs. Although fine-tuning-based methods achieve strong erasure performance, they face several critical drawbacks. The fine-tuning process itself is often costly and impractical for urgent tasks due to its time-consuming nature. More im- portantly, these methods require significant time investment and face the issue of catastrophic interference when inte- grating new target concepts, often necessitating complete and expensive retraining. Furthermore, they heavily rely arXiv:2601.00267v1 [cs.CV] 1 Jan 2026 on regularization terms to balance prior preservation, which can compromise both erasure efficacy and the general appli- cability of the models. To address these issues, another line of work has introduced training-free methods, thus achiev- ing CE with notable advantages in cost, efficiency, and speed. However, current training-free approaches still face several limitations and lack generalization across different erasure tasks [2, 9, 44], often fail to preserve non-target con- cept quality. This results in inconsistent performance and restricted practical applicability. To address these issues, we propose a simple yet ef- fective training-free and plug-and-play method called Ac- tErase. Our approach begins by preparing source and tar- get prompt pairs to generate activation parameters during the forward process. By comparing source and target acti- vations, activation difference regions are identified and then the source activations are applied to modify target activa- tions during inference. ActErase replaces target concepts rather than erasing them entirely, leading to better general- ization and robustness. Additionally, the introduced patch- ing parameters enrich the forward process, which can en- hance the quality, diversity, and realism of the generated images. In Fig. 1, we compare our method with both fine- tuning-based and training-free approaches in terms of effi- cacy, robustness, efficiency, preservation and quality. We also evaluate the range of concepts that each method can erase and compare the efficacy. The results indicate that our method performs comprehensively and outperforms oth- ers across most criteria. Experimental evaluations further demonstrate that our approach achieves state-of-the-art or competitive erasure performance across multiple concepts while consistently preserving the utility of non-target con- cepts. In summary, we make the following contributions: • We propose a concept erasure approach that leverages the sparsely distributed nature of concepts within activation parameters to achieve precise concept erasure. • By identifying and patching the components containing target concepts within activations during the generation process, ActErase achieves precise and high efficiency training-free erasures. • Extensive experimental results demonstrate that our method achieves or approaches SOTA performance on major erasure tasks, while maintaining good generative capabilities and even improving image quality. 2. Related Works Existing CE methods can be broadly categorized into three groups: dataset filtering [27, 31] and model retraining [25, 33, 38], training-based fine-tuning approaches [5, 10, 17, 18, 22, 28] and training-free methods [2, 9, 44]. Methods that employ data filtering techniques (e.g., Nudenet [42] and Q16 detector [39]) to exclude sensitive content and retrain models from scratch are computationally expen- sive and time-consuming. Moreover, such approaches may still allow target concepts to enter the latent space through seemingly safe prompts, potentially necessitating additional rounds of retraining. A more viable approach is to suppress the generation of images containing target concepts from the pre-trained model, thereby achieving erasing. This ap- proach has garnered extensive research and has evolved into two categories: training-based and training-free methods. 2.1. Training-based Concept Erasure Training-based methods typically involve fine-tuning pre- trained models to remove specific concepts. Erased Stable Diffusion (ESD) [8] fine-tunes the latent diffusion model (LDM) by aligning the noise predictions of target and non- target concepts and guides this optimization via classifier- free guidance. Forgive-Me-Not (FMN) [46] focuses on fine-tuning the cross-attention maps to steer the generation of target concepts towards unrelated concepts. Although this method enables rapid erasure, its applicability is lim- ited by the requirement that the target concept must be a single token. TV edit [28] also demonstrated that these early methods can be relatively easy to bypass. There- fore, subsequent research [16, 24] has focused on enhancing robustness and generality. STEREO [41] reframes adver- sarial training as a vulnerability identification mechanism, employing a two-stage approach: Systematic Threat Ex- posure (STE) to identify embedding space vulnerabilities, and Robust Erasure Optimization (REO) to achieve robust concept removal while preserving utility. TRCE [3] fur- ther advances this direction with a two-stage design that optimizes Erasure of Things (EoT) embeddings and applies contrastive denoising guidance, effectively erasing implicit semantics while reducing dependence on textual mapping. However, TRCE lacks explicit defenses against adversarial attacks. 2.2. Training-free Concept Erasure Training-free methods modify model behavior without fine- tuning, offering computational efficiency and flexibility. Unified Concept Editing (UCE) [9] introduces closed-form solutions to edit the cross attention weights, changing the key and value matrices of specific text embeddings con- taining target concept, while retaining the matrix weights of unrelated concepts.The use of editing rather than training enables UCE to modify weights in an extremely short time, achieving fast and effective erasure. Concept- Prune [2] extracts neurons associated with the target con- cept from the feed forward network (FFN) layers across multiple timesteps and prunes or zeroes out these neurons to achieve ce. Adaptive Value Decomposer (AdaVD) [44] dis- entangles target semantics from the cross-attention layers at each denoising timestep, using both the target prompt and a reference prompt as guidance. Within each cross-attention layer, the method dynamically controls the erasure inten- sity through an adaptive token-wise shift mechanism. This mechanism automatically adjusts the intensity level based on the semantic similarity between the target and reference tokens, enabling fast, precise and context-aware concept re- moval. 3. Method As illustrated in Figure 2, our method ActErase is a causal intervention that patch activations from source to target in- ference paths. With appropriate prompts, this training-free method achieves high erasure performance while enhancing image generation. 3.1. Preliminaries The Latent Diffusion Model (LDM) [32] learns a reverse denoising process in a compressed latent space. The gener- ative process is formulated as: p θ (z 0 ) = p(z T ) T Y t=1 p θ (z t−1 |z t ,c)(1) where z t is the latent variable at timestep t and c is the conditional embedding. The denoising network ε θ , imple- mented as a U-Net [35], iteratively refines z t by predicting the noise component. Within the U-Net, cross-attention (CA) layers enable modality fusion by aligning image features with text em- beddings, while self-attention (SA) layers capture intra- modal dependencies. Most prior CE methods have focused on modifying these attention mechanisms. However, re- cent explorations such as Concept-prune have revealed that Feed-Forward Network (FFN) layers also present a viable and promising avenue for concept manipulation, shifting the focus from attention mechanisms to internal computa- tions within the FFN. The FFN in each transformer block performs a two-step transformation with nonlinear activation: FFN(x) = W 2 · GELU(W 1 x + b 1 ) + b 2 (2) where W 1 ∈R d f ×d and W 2 ∈R d×d f are learnable weight matrices, with d f > d typically defining an expanded inter- mediate dimension. b 1 ∈R d f and b 2 ∈R d are learnable bias vectors for the first and second linear transformations, respectively. GELU denotes the Gaussian Error Linear Unit activation function, which provides a smooth nonlinearity. This position-wise operation provides substantial model ca- pacity by projecting inputs into a higher-dimensional space, applying GELU activation, and projecting back. The FFN’s localized processing nature makes it particularly suitable for targeted concept manipulation, as it encodes specialized vi- sual patterns without the global interaction overhead of at- tention mechanisms. Algorithm 1 Concept Erasure via Activation Patching Input: Pretrained model G, target concept c target , target prompt c t , source prompt c s , source concept c source , layer weight W l s , layer setL, timesteps T , threshold τ Output: Modified latent z 0 with c target erased 1: Step 1: Extract activations and generate masks. 2: z T ∼N (0,I) 3: for t = T to 1 do 4: x s ← G(z t ,t,c s ) 5: x t ← G(z t ,t,c t ) 6: for each layer l∈L do 7:x l avg ← x l avg + 1 T x l s 8: I s ← W l s ·|x l s | 9: I t ← W l s ·|x l t | 10:M l ← (I s ≥ τ )∧ (I s < I t ) 11: end for 12: z t−1 ← G(z t ,t,c s ) 13: end for 14: return x avg , M 15: 16: Step 2: Apply patching. 17: z T ∼N (0,I) 18: for t = T to 1 do 19: x t ← G(z t ,t,c t ) 20: for each layer l∈L do 21:x l p ← M l ⊙ x l avg + (1− M l )⊙ x l t 22: end for 23: z t−1 ← G(z t ,t,c t ;x l ← x l p l∈L ) 24: end for 25: return z 0 Building on this insight, our work specifically targets FFN layers for CE. We demonstrate that selective interven- tion in FFN parameters achieves more precise and robust concept removal while better preserving generative quality compared to attention-layer modifications. 3.2. Activation Patching As formalized in Algorithm 1, our method contains two steps. In the first step, we employ target prompts c t and source prompts c s to determine and collect patching activa- tions x p . In the second step, we use the generated patching activations to replace target activations with benign coun- terparts during a new diffusion process: z patched t = G z t−1 ,t,c target ;x l ← x l p l∈L (3) where z T is the initial latent noise sampled from N (0,I), x s and x t are intermediate activations generated using source prompt c s and target prompt c t respectively, and x l avg is the average of activations at layer l. Considering image generation requires maintaining fine- grained spatial coherence, directly applying activation SACAFFNSACAFFN Text Encoder “A woman stands...” Text Encoder “A naked woman stands...” SACAFFNSACAFFN z T z T z T-1 z T-1 × (T-1) z 0 z 0 × (T-1) X t W s X s X p AP Denoising U-net Denoising U-net Image Decoder z 0 Image Decoder Image Decoder X t W s X s X p AP SA CA FFN AP denoising step Forward Feedback LayerSelf-Attention LayerCross-Attention LayerActivation Patching Model X s W s Mask Mask X t X p FFN Layer FFN Layer Matrix Multiplication Invert Mask Matrix Summation (a) Framework of Our Method(b) Activation Patching Model Activation Patching Activation Patching Figure 2. Overview of ActErase. (a) illustrates the framework of our approach. Given a prompt containing the target concept c t and an irrelevant prompt c s , we first extract activation parameters x s and x t from the FFN layers for both prompts. During the whole denoising process, the source activations x s are then used to patch the target activations x t to suppress target concept generation. Color coding: blue (irrelevant concepts), red (target concepts), green (erased concepts). Dashed line indicates final denoising step. (b) details the activation patching model. A binary mask is generated based on importance scores derived from x s , x t and W s , identifying precise regions for selective patching. patching (AP) in diffusion models presents spatial precision challenges. Naive patching across entire feature maps often degrades image quality by disrupting unrelated content. To address this limitation, we develop a comparative masking strategy that identifies concept-specific regions through dual-path analysis. Our method simultaneously processes both source and target prompts to compute impor- tance scores for each path. The mask generation employs a comparative criterion: I s = d X j=1 W s [:,j]·|x l s,j |, I t = d X j=1 W s [:,j]·|x l t,j | (4) where I s and I t are importance scores calculated as W l s · |x l s | and W l s ·|x l t | respectively, and M l ← (I s ≥ τ )∧(I s < I t ) is the binary mask for layer l. The binary mask M is then determined by a dual- condition threshold τ : M = (I s ≥ τ )∧ (I s < I t )(5) This comparative approach ensures that interventions are applied only to regions where: Equation (1) the source ac- tivation exceeds a minimum importance threshold τ , and Equation (2) the target activation demonstrates significantly higher importance than the source, indicating concept- specific relevance. The subsequent masked patching opera- tion: x l p = M⊙ x l avg + (1− M)⊙ x l t (6) thus precisely targets concept-related features while pre- serving unrelated content. This comparative masking ap- proach provides the spatial precision necessary for effective concept manipulation in diffusion models, achieving an op- timal balance between erasure efficacy and preservation. 3.3. Multiple Concept Erasure We extend ActErase to handle multiple target concepts era- sure, which is crucial for real-world applications where multiple sensitive concepts need removal. The key chal- lenge lies in effectively aggregating individual concept masks and activation parameters while maintaining utility on non-target content. Our multi-concept erasure approach, also illustrated in the Algorithm in Appendix B.2, builds upon pre-computed single-concept masks and activations. Formally, given a concept setC = c 1 ,c 2 ,...,c n with pre-computed single-concept masksM i and source acti- vationsx i avg , we process all concepts in a unified frame- work using a single target prompt c t . MethodArmputsBellyButtocksFeetBreasts (F)Genitalia (F)Breasts (M)Genitalia (M)TotalFID (↓)CLIP (↑) SD v1.512415212263131520766517.0031.38 ESD [8]32302192738212318.2331.03 UCE [9]296272935511418217.5231.35 Concept-prune [2]8451120123317.5030.32 AdaVD [44]119115252725813205582 163.1214.96 STEREO [41]13101003925.9629.02 TRCE [3]231331051817.1630.93 Ours03002001616.6530.42 Table 1. Quantity of explicit content detected using the Nudenet detector on the I2P benchmark. F: Female. M: Male.Best results are marked in Bold. Among all methods, our approach generates the minimum number of exposed body regions after erasure while achieving the best FID score. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD I 2 P M M A P 4 D Figure 3. Comparison of Nudity erasure results in I2P dataset and under attacks. ActErase can effectively erase ‘nudity’ while maximally preserving the original semantic content of the image, and simultaneously enhances the details of the images. The aggregated mask M l agg for layer l is computed using logical OR: M l agg = n _ i=1 M i,l (7) ensuring comprehensive coverage of all concept-related re- gions across the concept set. The aggregated activation x l agg employs weighted aver- aging: x l agg = P n i=1 M i,l ⊙ x i,l avg max (1, P n i=1 M i,l ) (8) which prevents any single concept from dominating the erased representation while handling overlapping regions appropriately. The final patching operation integrates both components: x l p = M l agg ⊙ x l agg + (1− M l agg )⊙ x l t (9) This multi-concept approach maintains spatial precision while efficiently handling multiple targets through mask ag- gregation and activation fusion, ensuring balanced and ef- fective concept removal in a single forward pass. 4. Experiments 4.1. Experimental Setup Baselines. We compare our method ActErase against six SOTA CE approaches, including four training-based meth- ods ESD, STEREO, TRCE and three training-free meth- ods UCE, Concept-prune, AdaVD. To evaluate robustness against adversarial attacks, we consider four representative attack methods: MMA-diffusion [45], Prompt4Debugging (P4D) [4], Ring-a-bell [43], and UnlearnDiff [47]. Evaluation Metrics. We evaluate ActErase on three CE tasks: nudity erasure, style erasure, and object erasure. For nudity erasure, we report the number of detected exposed body parts in generated images and calculate the Attack Success Rate (ASR) against adversarial attacks to measure robustness in erasing NSFW concepts. For style and object erasure, we compute the classification accuracy (ACC) be- fore and after erasure to quantify both erasure efficacy and the preservation of non-target concepts. Additionally, we employ CLIP Score [30] to measure text-image consistency and FID score [12] to assess image quality. Higher CLIP MethodI2P(%)MMA(%)Ring-16(%)Ring-38(%)Ring-77(%)P4D(%)UnDiff(%)Average(%) ESD34.9253.7359.5566.6750.0062.3827.1859.07 UCE63.1766.4873.0374.7162.0994.1750.4969.16 Concept-prune8.893.2822.4719.5414.139.710.9913.17 AdaVD98.4199.2710098.8698.9100.0099.0199.21 STEREO1.271.461.121.150.003.88%0.001.27 TRCE5.341.460.000.000.000.001.981.25 Ours1.900.000.001.151.080.060.000.60 Table 2. The Attack Success Rate (ASR) against adversarial attacks in erasing NSFW concept ’Nudity’. Best results are marked in Bold. Our method achieves the best average ASR. MethodViolence(%)Self-harm(%)Sexual(%)Shocking(%) SD v1.4*40.135.554.542.1 ESD*16.711.116.416.1 UCE*23.312.916.219.2 TRCE(V)*12.29.818.66.7 TRCE(T+V)*3.02.61.53.6 Ours8.53.23.19.1 Table 3. The results of erasing multiple concepts in I2P bench- mark [38], inappropriate rate are detected by the Q16 detector [39]. The results with * are source from TRCE [3]. scores indicate better alignment between generated images and text prompts, while lower FID scores correspond to higher image quality. Implementation Details. All experiments are conducted using Stable Diffusion v1.5 [34]. We employ the DPM- solver sampler [21] with 50 sampling steps and classifier- free guidance [13] with a scale of 7.5.Other hyper- parameters follow the default configurations from the re- spective official repositories.All experiments are per- formed on NVIDIA RTX 4090 GPUs. Additional imple- mentation details are provided in the Appendix A.2. 4.2. NSFW erasure Single Concept Erasure. Following previous works [3, 41], we evaluate ActErase on 4,703 prompts from the I2P dataset [38] using Nudenet [42] for nudity detection. As shown in Table 1, ActErase achieves the best or second- best performance across all body part categories, with the lowest total detection count. To assess preservation, we generate images using 30,000 prompts from the MS-COCO dataset [19] and compute CLIP and FID scores. ActErase maintains a competitive CLIP score while achieving the best FID score, indicating that the introduced activation pa- rameters enhance image quality without compromising tex- tual alignment. Figure 3 further demonstrates that ActErase not only effectively removes nudity but also enriches visual details, particularly in background regions, achieving closer resemblance to the original image compared to other ap- proaches. To evaluate robustness, we count the number of im- ages containing exposed parts before and after erasing and then evaluate the Attack Success Rate (ASR) against ad- versarial attacks including MMA, Ring-a-bell, P4D, and UnlearnDiff. As shown in Table 2, ActErase achieves the best or second-best performance against adversarial attacks while achieving the best average erasure effectiveness. This shows that ActErase better preserves erasure capabilities under adversarial conditions. We present additional experi- mental results in Appendix B.2. Multiple NSFW Concept Erasure. We further evaluate ActErase on multiple NSFW concepts, including violence, self-harm, sexual content, and shocking imagery. Using all prompts from the I2P dataset and the Q16 detector [39] for detection, we compute ASR scores as shown in Ta- ble 3. Although ActErase performs slightly worse than the two-stage TRCE approach, it outperforms TRCE(V) and all other baseline methods, demonstrating significant potential for multi-concept erasure tasks. SD v1.5OursAdaVDTRCESTEREO Concept- prune V a n G o g h D a V i n c i P i c a s s o Figure 4. Comparison of Style erasure results include Van Gogh, Leonardo Da Vinci and Pablo Picasso. ActErase can erase con- cepts effectively while generating high quality images. 4.3. Artist Style Erasure We evaluate style erasure performance on three artist con- cepts: Van Gogh, Leonardo da Vinci, and Pablo Picasso. For each artist, we use a dataset of 50 prompts sourced from Concept-prune and employ the style classifier from UnlearnDiff [47] to classify the generated images. Since Erase Van Gogh Method ACC FID (↓)CLIP (↑) Top-1Top-3Top-5 SD v1.50.80.880.9617.0031.38 ESD0.360.680.7816.3931.11 UCE0.020.200.3016.3431.41 Concept-prune0.000.080.1816.9830.47 STEREO0.000.060.10 24.5729.75 AdaVD0.020.100.18166.5013.60 TRCE0.00 0.020.1216.1231.34 Ours0.000.060.1015.8430.61 Erase Leonardo Da Vinci Method ACC FID (↓)CLIP (↑) Top-1Top-3Top-5 SD v1.50.000.100.2617.0031.38 ESD0.000.000.0217.1431.20 UCE0.000.000.06 16.6531.37 Concept-prune0.000.000.0216.9030.76 STEREO0.000.00 0.0023.7130.05 AdaVD0.000.000.06220.3714.90 TRCE0.000.00 0.0016.9731.31 Ours0.000.000.0216.6531.16 Erase Pablo Picasso Method ACC FID (↓)CLIP (↑) Top-1Top-3Top-5 SD v1.50.20.98117.0031.38 ESD0.000.300.5615.7631.17 UCE0.020.300.5616.4331.40 Concept-prune0.000.340.4818.1530.80 STEREO0.000.14 0.18 23.7729.73 AdaVD0.000.240.34220.4314.90 TRCE0.00 0.120.3416.6631.38 Ours0.000.140.3017.0131.40 Table 4. Comparison with artist concepts erasure results. Best results are marked in Bold. ACC represents the top-k classification accuracy of the Q16 classifier. Top-1 accuracy rates show limited differentiation, we re- port Top-1, Top-3, and Top-5 accuracy for more compre- hensive comparison. To assess utility preservation, we gen- erate 30,000 images using prompts from MS-COCO and compute CLIP and FID scores for each method. As shown in Table 4, ActErase effectively erases all three targeted artistic styles, consistently achieving the lowest or second-lowest classification accuracy across all reported metrics. This indicates a successful reduction in style- specific features. Crucially, this performance is attained while maintaining competitive CLIP scores and FID scores, demonstrating a strong preservation of the original content’s utility and visual quality. Figure 4 provides visual confirma- tion, showing that our approach successfully removes the distinct characteristics of the target artistic styles while gen- erating realistic, high-quality images that faithfully preserve the core content without containing the erased concepts. For more detailed results and visual comparisons, please refer to Appendix B.3. French Horn Garbage Truck ChurchTench O r i g i n a l M o d i f i e d O r i g i n a l M o d i f i e d Figure 5. Comparison of Object erasure results. For each concept, the images show both target concept erasure results (Left) and non- target concept preservation results (top-right and bottom-right). 4.4. Object Erasure We evaluate object erasure performance using concepts from ImageNet [6], following the experimental setup of Concept-prune. For each object class, we generate 500 im- ages using the erased models and employ a ResNet-50 Im- ageNet classifier [11] to compute the Attack Success Rate (ASR) for both target and non-target concepts. As shown in Table 5, our analysis reveals distinct performance characteristics across methods: ESD, UCE, and ConceptPrune demonstrate limited erasure capability; STEREO achieves strong concept removal but significantly impairs non-target concept generation; AdaVD preserves non-target concepts effectively but shows insufficient era- sure performance. TRCE exhibits powerful erasure on most object concepts, but fails to erase the ”church” category. ActErase demonstrates balanced performance across all Method ChurchTenchFrench HornGarbage Truck ASR e (%)ASR k (%)FID (↓)CLIP (↑)ASR e (%)ASR k (%)FID (↓)CLIP (↑)ASR e (%)ASR k (%)FID (↓)CLIP (↑)ASR e (%)ASR k (%)FID (↓)CLIP (↑) ESD52.4792.9316.3631.0240.5785.2814.7730.9910.2889.6615.9731.1313.9087.9414.2630.87 UCE12.0086.3917.3031.1368.2664.3617.6830.149.0771.3016.9030.3912.1192.6216.5131.20 Concept-prune13.6566.1424.5530.480.0070.2918.2230.981.0172.7621.1930.900.2264.6528.3330.03 STEREO 0.0046.7227.6129.150.0059.9027.8629.810.0038.5030.9229.390.0049.6631.5528.71 AdaVD12.47100.00166.2615.3846.30100.00220.3814.910.2099.63166.5713.9624.4493.38147.9216.52 TRCE71.5297.4816.2131.320.0095.4517.3131.240.0095.7017.7331.380.0091.8815.8431.28 Ours4.2378.0724.8330.370.4784.9616.6231.381.4182.8217.8830.900.2283.9020.2030.61 Table 5. Comparison with object erasure results. Best results are marked in Bold. ASR e represents the ASR of target concept that should be erased, while ASR k represents the ASR of other concepts that need to be kept. ActErase achieves an optimal trade-off between target concept erasure efficacy and non-target concept preservation capability. evaluated object classes, effectively removing target con- cepts while maintaining strong preservation of unrelated concepts. This balanced capability suggests superior gen- eralization in object erasure applications. Figure 5 visu- ally confirms that our approach successfully erases target objects while preserving and enriching fine-grained details in the synthesized images, maintaining high-quality gener- ation for non-target content. More details can be found in Appendix B.4. 4.5. Further Analysis Time Consumption: Time consumption in concept erasure tasks can be divided into three components: (1) data prepa- ration time is the cost by STEREO and TRCE for preparing training data, by AdaVD for basis computation, and by our model for collecting activation parameters. (2) training time is required for training and fine-tuning training-based mod- els. (3) image generation time reflects the time required to generate images. We erase ’nudity’ and generate 1000 im- ages for each model and compare the total time consump- tion in Table 6. As can be observed from the table, com- pared to training-based methods, training-free methods ex- hibit significantly reduced time consumption. When com- pared to AdaVD, our approach incurs greater time cost dur- ing the data preparation stage due to the need for activation sorting. However, this cost is compensated by superior gen- eration quality and faster image generation speed,resulting in a lower total time consumption. Data Preparation Time(s)Training Time(s)Image Generation Time(s)Total Time(s) STEREO40015658002765 TRCE 13302377002267 AdVD0.3011501150.3 Ours29308001093 Table 6. Comparison of time consumption. We erase ’nudity’ and generate 1000 images for each model. Analyzing different patching modules: We conducted ex- periments on erasing the concept of ‘nudity’ across three different modules: FFN, CA, and the output of the U-Net. Using 4,703 prompts from the I2P dataset, we generated images and counted the number of exposed body parts de- tected. For both FFN and CA, we employed identical con- figurations to extract activation parameters. For the output module, due to the absence of a matching input layer and its high sensitivity to activation variations, we directly used activation intensity for sorting. After multiple parameter adjustments, we selected the optimal results, as shown in Table 7. It can be seen that the erasure performance of AP using identical parameter configurations in CA is sig- nificantly inferior to that in FFN, while the improvement in CLIP score remains limited. Although AP at the output layer can achieve erasure with minimal modifications, the high sensitivity of this layer and the entanglement of con- cepts at the output stage lead to severe degradation in gener- ation capability, which is reflected in a substantial decrease in CLIP score. SD v1.5FFNCAOutput Total6656206100 Clip (↑)31.3830.4230.7625.59 Table 7. Comparison of concept erasure performance across dif- ferent patching modules. Total indicates the number of detected exposed body parts generated by I2P dataset. 5. Conclusions In this work, we present ActErase, a plug-and-play, training-free framework for concept erasure via activation replacement in diffusion models. ActErase achieves pre- cise and efficient removal of target concepts while preserv- ing the model’s overall generative fidelity. Extensive ex- periments across diverse erasure tasks—covering nudity, artistic style, and object categories—demonstrate that our method effectively eliminates undesired concepts without degrading non-target content.Both quantitative evalua- tions and visual analyses confirm that ActErase performs surgical-level interventions within FFN layers, maintaining semantic consistency and even enhancing perceptual im- age quality. By introducing activation replacement as a lightweight yet powerful mechanism for concept manipu- lation, this work establishes a new paradigm for safe, con- trollable, and ethical image generation in diffusion mod- els. References [1] Josh Achiam, Steven Adler, Sandhini Agarwal, and et al. Gpt-4 technical report, 2023. 11 [2] Ruchika Chavhan, Da Li, and Timothy M. Hospedales. Con- ceptprune: Concept editing in diffusion models via skilled neuron pruning. ArXiv, abs/2405.19237, 2024. 2, 5 [3] Ruidong Chen, Honglin Guo, Lanjun Wang, Chenyu Zhang, Wei zhi Nie, and Anan Liu. Trce: Towards reliable malicious concept erasure in text-to-image diffusion models. ArXiv, abs/2503.07389, 2025. 1, 2, 5, 6 [4] Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin- Yu Chen, and Wei-Chen Chiu. Prompting4debugging: Red- teaming text-to-image diffusion models by finding problem- atic prompts. ArXiv, abs/2309.06135, 2023. 5 [5] Bartosz Cywi’nski and Kamil Deja. Saeuron: Interpretable concept unlearning in diffusion models with sparse autoen- coders. ArXiv, abs/2501.18052, 2025. 1, 2 [6] Jia Deng, Wei Dong, R. Socher, Li Jia Li, Kai Li, and Li Fei- Fei. Imagenet: A large-scale hierarchical image database. Proc of IEEE Computer Vision & Pattern Recognition, pages 248–255, 2009. 7 [7] Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. ArXiv, abs/2105.05233, 2021. 1 [8] Rohit Gandikota, Joanna Materzynska, Jaden Fiotto- Kaufman, and David Bau. Erasing concepts from diffusion models. 2023 IEEE/CVF International Conference on Com- puter Vision (ICCV), pages 2426–2436, 2023. 2, 5 [9] Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy’nska, and David Bau. Unified concept editing in diffusion models. 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 5099–5108, 2023. 2, 5 [10] Daiheng Gao, Shilin Lu, Wenbo Zhou, Jiaming Chu, Jie Zhang, Mengxi Jia, Bang Zhang, Zhaoxin Fan, and Weiming Zhang. Eraseanything: Enabling concept erasure in rectified flow transformers. In Forty-second International Conference on Machine Learning, 2025. 2 [11] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2015. 7 [12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, G ̈ unter Klambauer, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a nash equilibrium. ArXiv, abs/1706.08500, 2017. 5 [13] Jonathan Ho.Classifier-free diffusion guidance.ArXiv, abs/2207.12598, 2022. 1, 6 [14] Jonathan Ho, Ajay Jain, and P. Abbeel. Denoising diffusion probabilistic models. ArXiv, abs/2006.11239, 2020. 1 [15] Harry H. Jiang, Lauren Brown, Jessica Cheng, Mehtab Khan, Abhishek Gupta, Deja Workman, Alex Hanna, Johnathan Flowers, and Timnit Gebru. Ai art and its impact on artists. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, page 363–374, New York, NY, USA, 2023. Association for Computing Machinery. 1 [16] Jiaxiu Jiang, Yabo Zhang, Kailai Feng, Xiaohe Wu, Wenbo Li, Renjing Pei, Fan Li, and Wangmeng Zuo. Mcˆ 2: Multi- concept guidance for customized multi-concept generation. In Proceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 2802–2812, 2025. 2 [17] Chang Soo Kim, Kyle Min, and Yezhou Yang. R.a.c.e.: Ro- bust adversarial concept erasure for secure text-to-image dif- fusion model. ArXiv, abs/2405.16341, 2024. 1, 2 [18] Xinfeng Li, Yuchen Yang, Jiangyi Deng, Chen Yan, Yan- jiao Chen, Xiaoyu Ji, and Wenyuan Xu. Safegen: Mitigating sexually explicit content generation in text-to-image mod- els. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024. 1, 2 [19] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll ́ ar, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, 2014. 6 [20] Runtao Liu, Ashkan Khakzar, Jindong Gu, Qifeng Chen, Philip H. S. Torr, and Fabio Pizzati. Latent guard: a safety framework for text-to-image generation. In European Con- ference on Computer Vision, 2024. 11 [21] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffu- sion probabilistic model sampling in around 10 steps. ArXiv, abs/2206.00927, 2022. 6 [22] Shilin Lu, Zilan Wang, Leyang Li, Yanzhu Liu, and Adams Wai-Kin Kong. Mace: Mass concept erasure in diffusion models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6430–6440, 2024. 1, 2 [23] Dan Milmo.AI-created Child Sexual Abuse Images ’Threaten to Overwhelm Internet’. The Guardian, 2023. 1 [24] Kien Nguyen, Anh Tran, and Cuong Pham. Suma: A sub- space mapping approach for robust and effective concept erasure in text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 19587–19596, 2025. 2 [25] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In Interna- tional Conference on Machine Learning, 2021. 1, 2 [26] Ryan O’Connor. Stable Diffusion 1 vs 2 - What You Need to Know. Blog post, 2022. Accessed: 2025-01-01. 1 [27] OpenAI. DALL·E 3 System Card. Technical report, OpenAI, 2023. 2 [28] Minh Pham, Kelly O Marshall, Chinmay Hegde, and Niv Cohen. Robust concept erasure using task vectors. arXiv preprint arXiv:2404.03631, 2024. 2 [29] Ed Power. Another Body: My AI Porn Nightmare – a dis- turbing digital detective story. The Washington Post, 2024. 1 [30] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, 2021. 5 [31] Javier Rando, Daniel Paleka, David Lindner, Lennard Heim, and Florian Tram ` er. Red-teaming the stable diffusion safety filter. ArXiv, abs/2210.04610, 2022. 2 [32] Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Bj ̈ orn Ommer. High-resolution image synthesis with latent diffusion models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10674–10685, 2021. 1, 3 [33] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ̈ orn Ommer. Stable diffusion 2.0, 2022. 1, 2 [34] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ̈ orn Ommer. Stable diffusion v1.5. Hug- ging Face Model Repository, 2022. Version 1.5. 6 [35] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. ArXiv, abs/1505.04597, 2015. 3 [36] Kevin Roose. An AI-generated picture won an art prize. Artists aren’t happy. The New York Times, 2022. 1 [37] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, Seyedeh Sara Mah- davi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text- to-image diffusion models with deep language understand- ing. ArXiv, abs/2205.11487, 2022. 1 [38] Patrick Schramowski, Manuel Brack, Bjorn Deiseroth, and Kristian Kersting. Safe latent diffusion: Mitigating inappro- priate degeneration in diffusion models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22522–22531, 2022. 1, 2, 6 [39] Patrick Schramowski, Christopher Tauchmann, and Kristian Kersting. Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content? In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, page 1350–1361, New York, NY, USA, 2022. Association for Computing Machin- ery. 2, 6 [40] Riddhi Setty. AI art generators hit with copyright suit over artists’ images. The Guardian, 2023. 1 [41] Koushik Srivatsan, Fahad Shamshad, Muzammal Naseer, Vishal M. Patel, and Karthik Nandakumar.Stereo: A two-stage framework for adversarially robust concept eras- ing from text-to-image diffusion models. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23765–23774, 2024. 2, 5, 6 [42] NotAI Tech. Nudenet: Neural nets for nudity classification, detection and selective censoring, 2019. Accessed: 2025-07- 22. 2, 6 [43] Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia- You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, and Chun ying Huang. Ring-a-bell! how reliable are concept removal meth- ods for diffusion models? ArXiv, abs/2310.10012, 2023. 5 [44] Yuan Wang, Ouxiang Li, Tingting Mu, Yanbin Hao, Kuien Liu, Xiang Wang, and Xiangnan He. Precise, fast, and low- cost concept erasure in value space: Orthogonal complement matters. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28759–28768, 2024. 2, 5 [45] Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Nan Xu, and Qiang Xu. Mma-diffusion: Multimodal attack on diffusion models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7737–7746, 2023. 5 [46] Eric Zhang, Kai Wang, Xingqian Xu, Zhangyang Wang, and Humphrey Shi. Forget-me-not: Learning to forget in text-to-image diffusion models. 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition Work- shops (CVPRW), pages 1755–1764, 2023. 2 [47] Yimeng Zhang, Jinghan Jia, Xin Chen, Aochuan Chen, Yi- hua Zhang, Jiancheng Liu, Ke Ding, and Sijia Liu. To gen- erate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images ... for now. In European Conference on Computer Vision, 2023. 5, 6 ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Patching Supplementary Material A. Implementation Details A.1. Details on Prompt Pairs Following Latent Guard [20], we employ the large language model GPT-4 [1] to generate unsafe/safe prompt pairs. To generate prompts c t that contain target concepts, we prompt an LLM with the following text: Prompt for Generating c t We are doing safety research to enhance safety in using AI models. So please help us and do it according to the instructions. Your task is to write me an image caption that includes and visually describes a scene and the caption should HAVE the EXACT concept word. 1. utput one single grammatically correct caption each line that is no longer than 20 words. 2. Though some concept words may be offensive, do not output any notes, word counts, facts, etc as these output will be used in the right way to be good for humans. Output one single caption each line only. Output the EXACT given concept without using other synonyms words. 3. Captions should NOT include the Quotation Mark ”. Do NOT output the caption where the concept word is used as a reference like ”(concept)”. Don’t let the concept be the form of ”(concept)” in captions. Directly use the concept word as Noun or Verb. 4. Input concept is: <concept >. Write 10 captions MUST containing the EXACT input concept word. Here is a GOOD example (in the caption con- cept word is NOT of ”(concept)” form): Input concept: naked Output: 1. A woman stands naked in the sunlight, her form bathed in golden rays. 2. A swimmer emerges naked from the water, glistening with droplets. 3. A child runs naked through the sprinkler, laughing with joy. We limit the length of prompts to 20 words and gener- ate 10 unsafe prompts for each concept. By replacing the <concept >with target concept (e.g. nudity), this prompt can easily be used for other concept erasure tasks. To generate safe prompts c s , we use the unsafe prompts c t with the prompt below: Prompt for Generating c s Please rephrase the following sentence in a way that completely removes any negative, hateful, violent, sexual, or offensive content. Focus on transforming the core idea into a very pos- itive statement without using any synonyms or indi- rect references to the original harmful or negative content. The output can be opposed to the given sentences if the output is very positive. The goal is to create a version of the sentence that is respectful, positive and safe for all audiences. Sentence: <Unsafe Prompts > Output: We can generate irrelevant but very similar safe prompts c s by replacing <Unsafe Prompts >with c t . A.2. Additional Metrics Details We introduce the experiment setup and metrics of our ex- periments. We calculate ASR, FID score and CLIP score for each concept. Here are some calculation details: ASR: For nudity erasure task, we use NudeNet Detector to identify and calculate the number of exposed body parts and images that contain exposed body parts. We then calculate the ASR using the ratio of images detected with exposed body parts before and after erasing. For object erasure task, we calculate the ASR by computing the ratio of top-1 clas- sification accuracy before and after erasing. FID: It measures the quality of generated images by com- paring their statistical similarity to real images. The FID is calculated as: FID =∥μ r − μ g ∥ 2 + Tr Σ r + Σ g − 2 p Σ r Σ g (10) where μ r , μ g and Σ r , Σ g are the mean and covariance of the real and generated features respectively. A lower FID indicates better image quality and diversity. We generate images using a total of 30,000 prompts from the MS-COCO dataset, and calculate FID between these generated images and real images in COCO val 2014. CLIP: It measures the semantic alignment between a gen- erated image and its corresponding text caption. It uses the powerful CLIP model, which projects images and text into a shared semantic space. The score is computed as the co- sine similarity between the image and the text embeddings. A higher CLIP Score indicates a stronger semantic corre- spondence, meaning the image better reflects the content of the text prompt. A.3. Hyper-parameters Details To ensure a fair and consistent evaluation experiment, we configure the Stable Diffusion v1.5 model with the follow- ing key hyper-parameters for all experiments. We select the DPM-solver as the sampler, with a total of 30 denoising steps. And we set the classifier-free guidance scale to 7.5. All other parameters are kept at their default settings. When generating masks, we employ different thresholds τ for each concept. We also try to evaluate erasure efficacy in different applying activation patching timesteps but find only apply activation patching during the whole denoising step, our model can achieve best performance. So we seed the patching timesteps the same as denoising timesteps. The hyper-parameters are listed below. CategoriesConceptthresholds τTimesteps NSFW Nudity0.0130 Violence0.0130 Self-harm0.0130 Sexual0.0130 Shocking0.0130 Style Van Gogh0.0230 Leonardo Da Vinci0.0230 Pablo Picasso0.0230 Object Church0.0130 tench0.0130 french horn0.0230 garbage truck0.0230 Table 8. Comparison of concept erasure performance across dif- ferent patching modules. Total indicates the number of detected exposed body parts generated by I2P dataset. B. Additional Experiment Results B.1. Adversarial Attacks We employ adversarial attacks include MMA, Ring-a-bell, P4D, and UnlearnDiff to evaluate the robustness of pro- posed models. Here we give the details of each attack meth- ods. MMA: The MMA-Diffusion method operates by formu- lating an adversarial attack in the continuous embedding space of text-to-image diffusion models. The attack is con- structed through an optimization process that generates an adversarial text embedding designed to mislead the image generation process. The key innovation lies in the design of a composite loss function that operates across multi- ple modalities. This loss function combines two primary components: a text-based objective that maximizes the se- mantic distance from the original prompt while minimiz- ing the distance to a target deceptive prompt, and a cross- modal objective that minimizes the alignment between the adversarial text embedding and the latent representation of a random input image. This dual-objective approach en- sures that the resulting adversarial prompt not only causes significant deviation from the intended generation outcome but also maintains attack effectiveness against the inherent stochasticity of the diffusion process. The optimization is performed using standard gradient-based methods to effi- ciently compute the adversarial perturbation, producing a modified text embedding that, when fed into the diffusion model, reliably causes generation failures or targeted mis- direction. In our experiments, we employ the 1000 NSFW prompts from MMA dataset to generate images. Ring-a-bell: Based on a systematic examination of ex- isting concept-erasure methods, the Ring-a-Bell study de- veloped a multi-layered security evaluation framework that generates adversarial prompts through a sequential method- ology. The approach begins with standard inference to es- tablish performance baselines, proceeds with member in- ference attacks to detect residual concept traces, and culmi- nates in concept reconstruction attacks that iteratively opti- mize prompts to maximize concept recovery from model parameters.In our experiments,we employ adversarial prompts from the resulting Ring-a-Bell-16, Ring-a-Bell-38, and Ring-a-Bell-77 datasets to generate images respectively for comprehensive evaluation. P4D: The P4D methodology employs an automated pipeline to generate critical prompts that expose vulnera- bilities in text-to-image models. Its approach begins with a set of seed prompts representing potential safety or bias concerns, which are then semantically expanded using a large language model to increase their diversity and speci- ficity. These expanded prompts are used to generate images through the target diffusion model. The core of the method lies in its automated evaluation phase, where specialized classifiers analyze the generated images for specific fail- ures, such as demographic biases or inappropriate content. Prompts that consistently trigger these model failures are identified as critical. These problematic prompts are then clustered and analyzed to uncover systematic weaknesses, effectively producing a targeted set of adversarial prompts for model debugging and robustness assessment. UnlearnDiff: The method employs a gradient-based op- timization attack to generate adversarial prompts against safety-unlearned diffusion models. The process starts with an initial benign text prompt, which is encoded into a con- Algorithm 2 Multiple Concept Erasure via Mask Aggrega- tion Input: Pretrained model G, concept set C, target prompt c target , single-concept masks M i |C| i=1 , single-concept source activationsx i avg |C| i=1 , layer setL, timesteps T Output: Modified latent z 0 with all concepts inC erased 1: z T ∼N (0,I) 2: for t = T to 1 do 3: x t ← G(z t ,t,c t ) 4: for each layer l∈L do 5:M l agg ← W |C| i=1 M i,l 6:x l agg ← P |C| i=1 M i,l ⊙x i,l avg max(1, P |C| i=1 M i,l ) 7:x l p ← M l agg ⊙ x l agg + (1− M l agg )⊙ x l t 8: end for 9: z t−1 ← G(z t ,t,c t ;x l ← x l p l∈L ) 10: end for 11: return z 0 tinuous embedding vector using the model’s text encoder. This embedding is then iteratively optimized by maximiz- ing the model’s activation toward unsafe concepts while maintaining perceptual similarity to the original prompt. During each iteration, the gradient of an unsafe-content loss function is computed with respect to the text embedding, and the embedding is updated accordingly. The unsafe- content loss is typically evaluated using a safety classifier applied to intermediate outputs of the diffusion process. This optimization continues until the generated embedding reliably causes the model to produce unsafe images. The final adversarial prompt is obtained by decoding the opti- mized embedding back into text space, often resulting in semantically perturbed but human-readable phrases that ef- fectively bypass the model’s safety alignments. We present additional results for Nudity erasure under each adversarial attack in Figure 6, Figure 7 and Figure 8. B.2. Additional NSFW Erasure Results In this section, we are going to introduce some additional NSFW erasure results. To evaluate single concept erase ef- ficacy, we erase ’Nudity’ and generate images with prompts in I2P dataset. We also erase ’Sexual’, ’Self-harm’, ’Vi- olence’ and ’Shocking’ and employ I2P dataset and Q16 detector to evaluate multiple concept erase results. For single concept erase, we have already presented the results of metrics in the main text. Here we give more gen- erated images in Figure 9. We also evaluate the FID and clip score for target images or images generated by I2P dataset. FID is measured between images before and after erasing while clip score is calculated by erased images and prompts in I2P dataset. In theory, a lower clip score and higher FID illustrate lower similarity and better erasure performance. However, a higher FID value only indicates significant vi- sual changes in the generated images but does not confirm that the semantics aligned with the target concept have been fully eliminated; conversely, this reduction may well indi- cate a degradation in image quality. Similarly, a lower CLIP score suggests that the generated images after erasure devi- ate substantially from the text prompts, yet it cannot specify whether this deviation corresponds to the intended removal of the target concept. In summary, while the FID and CLIP scores of images generated from the I2P dataset cannot di- rectly measure erasure effectiveness, they remain valuable for further verification and comparison of the efficacy of different erasure methods. MethodFID (↑)CLIP (↓) ESD78.6530.20 UCE81.8730.46 Concept-prune81.3130.43 AdaVD82.1430.95 STEREO69.7522.08 TRCE82.0927.56 Ours84.9029.86 Table 9. Comparison of ’Nudity’ erasure. We calculate the FID and CLIP Score of images generated by I2P images. For multiple concept erase, we have already shown the results of metrics in the main text. Here we show some generated images in figure 10. We also give the algorithm for multiple concept erasure in Algorithm 2. When erasing multi concept, we observe that compared to solely removing the ”nudity” concept, the model almost ceased generating human figures. We hypothesize that this occurs because the increase in the number of targeted concepts leads to a cor- responding expansion in the affected activation parameters, thereby amplifying the impact on image generation. This ultimately results in the unintended and near-total erasure of the broader ”human” concept. B.3. Additional Style Erasure Results Here we will give some extra results of artist style erasure tasks. In the main text, we did experiments in erasing ’Van Gogh’, ’Leonardo Da Vinci’ and ’Pablo Picasso’ and ana- lyzed the ASR of each style. Source from Concept-prune, we created 50 prompts for each concept via ChatGPT. These prompts are combined with a name of the artist’s painting and the artist’s name (e.g. ’Starnight, Van Gogh’). More results of images generated by each method can be seen in Figure 11, Figure 12 and Figure 13. B.4. Additional Object Erasure Results To evaluate object erase capabilities, we generated images for 10 concepts in ImageNet. For each erasing task, we gen- erate 500 images for each concept with the prompt format ’a photo of ¡concept¿’, resulting in 500 images of the tar- get concept and 4,500 images of other concepts. Here we will give details of each erasing task in Table 10. It illus- trates that our method can erase different kinds of object concepts. We attached more generated images in Figure 14 and Figure 15. ConceptASR e (%)ASR k (%) Church4.2378.07 Tench0.4784.96 Golf Ball3.4763.00 English Springer0.0063.89 Cassette Player4.8038.53 Chain Saw22.269.9 French Horn1.4182.82 Garbage Truck0.2283.90 Gas Pump0.2367.2 Parachute0.0064.23 Table 10. Comparison of ’Object’ erasure. We report ASR for 10 objects C. Additional Analysis and Ablation Study Additional Analysis on COCO Dataset In the main paper, we have analyzed the results of the FID and CLIP Score for each erasure task. It clearly shows that our method can im- prove the quality of generated images while achieving good generated capabilities, leading to a lower FID Score and a relatively high CLIP Score. We also observed that the FID and CLIP Score for AdaVD method are the worst among all methods on all tasks. We posit that this stems from the conceptual generation divergence caused by AdaVD’s orthogonal decomposition of input prompts relative to the target concept. When generating images for a specific con- ceptual dataset, where concepts are relatively independent, this method can precisely eliminate the influence of the tar- get concept’s direction without interfering with the genera- tion of other concepts. As shown in Table 9, the FID and CLIP Score for generating the target concept remain rela- tively high. However, when generating and evaluating via the MS-COCO dataset, the large number of common con- cepts may include some that are potentially associated with the target concept. The erasure operation consequently un- dermines the generation capability of these concepts, ulti- mately leading to the degradation of FID and CLIP scores for images generated from the COCO dataset. In Figure 16, we compare the images generated via MS-COCO dataset. The images generated by TRCE and UCE are very similar to the original images while the AdaVD results are differ- ent, which is the same as the metrics shown. Figure 17 il- lustrated MS-COCO generated results in different CE tasks, our method show a good preservation in all these tasks. Ablation Study on Multiple Concept Erasure To further analyze the impact of the number of erased concepts on the performance of multiple concept erasure, we use 10,000 prompts from the MS-COCO dataset to generate images with multiple concept erasure models containing 1, 3, 5, and 10 object concepts respectively, and analyze the CLIP scores and FID metrics as shown in Table 11. As the num- ber of concepts increases, the FID and CLIP Score decrease, and when the number of concepts reaches 10, the metrics in- dicate an almost complete failure to generate coherent im- ages. We attribute this to the growing proportion of acti- vated parameters introduced with more concepts, coupled with the distortion of these parameters during aggregation, which collectively disrupts the image generation process. Number of conceptsFID (↑)CLIP (↓) 120.8530.72 343.8628.78 584.3725.54 10222.1420.34 Table 11. Comparison of FID and CLIP Score for multiple erasure tasks contain different number of concepts. SD v1.5 OursAdaVDTRCESTEREO Concept- prune UCE ESD M M A P 4 D M M A P 4 D Figure 6. Additional results of adversarial attacks. This image compares both MMA and P4D. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD R A B - 1 6 R A B - 1 6 R A B - 3 8 R A B - 3 8 Figure 7. Additional results of adversarial attacks. This image compares both Ring-a-Bell-K16 and Ring-a-Bell-K16. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD R A B - 7 7 R A B - 7 7 U n l e a r n D i f f U n l e a r n D i f f Figure 8. Additional results of adversarial attacks. This image compares both Ring-a-Bell-K77 and UnlearnDiff. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD Figure 9. Additional results of I2P dataset. O r i g i n a l M o d i f i e d Figure 10. Results of multiple concepts erasure. More concepts will introduce a corresponding expansion in the affected activation parameters, leading to a over erasure. The generated images erase ’human’ rather than specific concepts. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD V a n G o g h V a n G o g h V a n G o g h Figure 11. Additional results of erasing ’Van Gogh’. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD D a V i n c i D a V i n c i D a V i n c i Figure 12. Additional results of erasing ’Leonardo Da Vinci’. SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD P i c a s s o P i c a s s o P i c a s s o Figure 13. Additional results of erasing ’Pablo Picasso’. ChurchEnglish SpringerCassette PlayerGolf BallTench O r i g i n a l O r i g i n a l M o d i f i e d M o d i f i e d Figure 14. Additional results of object erasure. For each concept, the images show both target concept erasure results (Left) and non-target concept preservation results (top-right and bottom-right). Chain SawGas PumpPararchuteGarbage TruckFrench Horn O r i g i n a l O r i g i n a l M o d i f i e d M o d i f i e d Figure 15. Additional results of object erasure. For each concept, the images show both target concept erasure results (Left) and non-target concept preservation results (top-right and bottom-right). SD v1.5OursAdaVDTRCESTEREO Concept- prune UCEESD C O C O C O C O C O C O Figure 16. Comparison of images generated by different methods via MS-COCO dataset. Nudity Garbage Truck PicassoFrench HornTenchChurchDa VinciVan Gogh C O C O C O C O C O C O Original Figure 17. Additional images generated in different CE tasks via MS-COCO dataset.