Paper deep dive

EraseAnything++: Enabling Concept Erasure in Rectified Flow Transformers Leveraging Multi-Object Optimization

Zhaoxin Fan, Nanxiang Jiang, Daiheng Gao, Shiji Zhou, Wenjun Wu

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 113

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 7/20/2026, 3:34:30 AM

Summary

The paper introduces EraseAnything++, a unified framework for concept erasure in text-to-image and text-to-video diffusion models based on flow-matching and transformer architectures. It formulates erasure as a constrained multi-objective optimization problem, balancing concept removal with generative utility preservation. Key technical contributions include an implicit gradient surgery strategy to resolve conflicting objectives, LoRA-based parameter tuning, attention-level regularization, and an anchor-and-propagate mechanism for video consistency. The method aims to outperform existing techniques in erasure effectiveness, fidelity, and temporal consistency.

Entities (23)

EraseAnything++ · method · 98%EraseAnything++ · methodframework · 98%Implicit Gradient Surgery · algorithm · 95%Implicit Gradient Surgery · technique · 95%LoRA · technique · 93%Anchor-and-Propagate · mechanism · 92%Concept Erasure · task · 92%Flux · model · 92%Anchor-and-Propagate · mechanism · 90%Flow Matching · technique · 90%Multi-Objective Optimization · framework · 90%Multi-Objective Optimization · concept · 90%OpenSora · model · 90%Stable Diffusion v3 · model · 90%Transformer · architecture · 90%Temporal Drift · problem · 88%Flux · model · 85%OpenSora · model · 85%Stable Diffusion v3 · model · 85%T5 · component · 85%ESD · method · 80%RoPE · component · 80%T5 · component · 80%

Relation Signals (17)

EraseAnything++ → formulates → Multi-Objective Optimization

confidence 95% · formulating concept erasure as a constrained multi-objective optimization problem

EraseAnything++ → uses → Implicit Gradient Surgery

confidence 95% · we introduce an efficient utility-preserving unlearning strategy based on implicit gradient surgery.

EraseAnything++ → uses → Implicit Gradient Surgery

confidence 95% · we introduce an efficient utility-preserving unlearning strategy based on implicit gradient surgery.

EraseAnything++ → uses → LoRA

confidence 93% · by integrating LoRA-based parameter tuning with attention-level regularization

EraseAnything++ → uses → Anchor-and-Propagate

confidence 92% · we further enhance consistency through an anchor-and-propagate mechanism

EraseAnything++ → appliesto → OpenSora

confidence 90% · We select Flux and Open-Sora as our primary experimental testbeds for image and video concept erasure

EraseAnything++ → appliesto → Flux

confidence 90% · We select Flux and Open-Sora as our primary experimental testbeds for image and video concept erasure

EraseAnything++ → formulates → Concept Erasure

confidence 90% · formulating concept erasure as a constrained multi-objective optimization problem

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Removing undesired concepts from large-scale text-to-image (T2I) and text-to-video (T2V) diffusion models while preserving overall generative quality remains a major challenge, particularly as modern models such as Stable Diffusion v3, Flux, and OpenSora employ flow-matching and transformer-based architectures and extend to long-horizon video generation. Existing concept erasure methods, designed for earlier T2I/T2V models, often fail to generalize to these paradigms. To address this issue, we propose EraseAnything++, a unified framework for concept erasure in both image and video diffusion models with flow-matching objectives. Central to our approach is formulating concept erasure as a constrained multi-objective optimization problem that explicitly balances concept removal with preservation of generative utility. To solve the resulting conflicting objectives, we introduce an efficient utility-preserving unlearning strategy based on implicit gradient surgery. Furthermore, by integrating LoRA-based parameter tuning with attention-level regularization, our method anchors erasure on key visual representations and propagates it consistently across spatial and temporal dimensions. In the video setting, we further enhance consistency through an anchor-and-propagate mechanism that initializes erasure on reference frames and enforces it throughout subsequent transformer layers, thereby mitigating temporal drift. Extensive experiments on both image and video benchmarks demonstrate that EraseAnything++ substantially outperforms prior methods in erasure effectiveness, generative fidelity, and temporal consistency, establishing a new state of the art for concept erasure in next-generation diffusion models.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

111,667 characters extracted from source content.

Expand or collapse full text

EraseAnything++: Enabling Concept Erasure in Rectified Flow Transformers Leveraging Multi-Object Optimization Zhaoxin Fan, Nanxiang Jiang, Daiheng Gao, Shiji Zhou, Wenjun Wu Zhaoxin Fan, Nanxiang Jiang, Shiji Zhou, Wenjun Wu are with Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, School of Artificial Intelligence, Beihang University. Daiheng Gao is with University of Science and Technology of China. Shiji Zhou is the corresponding author (Email: zhoushiji25@buaa.edu.cn). Abstract Removing undesired concepts from large-scale text-to-image (T2I) and text-to-video (T2V) diffusion models—while preserving overall generative quality—remains a major challenge, particularly as modern models such as Stable Diffusion v3, Flux, and OpenSora employ flow-matching and transformer-based architectures, as well as extending to long-horizon video generation. Existing concept erasure methods, designed for earlier T2I/T2V models, often fail to generalize to these paradigms. To address this, we propose EraseAnything++, a unified framework for concept erasure in both image and video diffusion models with flow-matching objectives. Central to our approach is formulating concept erasure as a constrained multi-objective optimization problem, explicitly balancing concept removal with the preservation of generative utility. To solve such problem with conflicting objectives, we introduce an efficient utility-preserving unlearning strategy based on implicit gradient surgery. Furthermore, by integrating LoRA-based parameter tuning with attention-level regularization, our method anchors erasure on key visual representations and propagates it consistently across spatial and temporal dimensions. In the video setting, we further enhance consistency through an anchor-and-propagate mechanism, which initializes erasure on reference frames and enforces it throughout subsequent transformer layers, thereby mitigating temporal drift. Extensive experiments on both image and video benchmarks demonstrate that EraseAnything++ substantially outperforms prior methods in erasure effectiveness, generative fidelity, and temporal consistency, establishing a new state of the art for concept erasure in next-generation diffusion models. Code is available at https://github.com/nxjiang-jnx/EraseAnything-PlusPlus. I Introduction Since the emergence of DALL-E 2 [54] and Stable Diffusion (SD)[56], text-to-image (T2I) and text-to-video (T2V) diffusion models have driven remarkable progress in generative vision. Recently, models such as Flux [35] and OpenSora [78] have adopted flow-matching [38, 41] and transformer-based [67] architectures to deliver significant gains in prompt following, image fidelity, and output diversity. These advances have extended the generative paradigm from static images to long-horizon videos and introduced sophisticated modules, including advanced text encoders like T5 [53] and rotary positional encodings (RoPE) [64], fundamentally reshaping the landscape of T2I and T2V modeling. Figure 1: Illustration of Difference between EraseAnything++ and Existing Methods. In this paper, we introduce EraseAnything++, an advanced concept erasure technique for Flow-based T2I/T2V images. First row: Classical concept-erasing methods—ESD [11], CA [34], and EAP [5]—have been transplanted into Flux.1 [dev] and are tested with the input “ nudity” (blue bars indicate author-added sensory harmony). Second row: Visualizing EraseAnything++’s impact—pre and post-concept removal. Original outputs (yellow box) are displayed in the upper right. Despite these successes, the increasing scale and diversity of training data have amplified concerns regarding the generation of inappropriate or unsafe content. In particular, diffusion models can produce NSFW (Not Safe For Work) material in response to certain prompts, a risk that is widely acknowledged in both academic and public discourse [1, 65, 2, 49]. Accordingly, concept erasure [34, 11]—the targeted suppression or removal of specific concepts—has become a critical safety requirement for the responsible deployment of T2I and T2V systems. To address these risks, a variety of concept erasure techniques have been proposed, primarily in the context of earlier architectures such as SD, which rely on DDPM/DDIM [22, 63] and U-Net [57] frameworks. These methods can be grouped into three major categories: (1) direct fine-tuning approaches (e.g., ESD [11]), which suppress activations linked to specific concepts; (2) closed-form solutions (e.g., UCE [12]), which analytically remove concept-related features; and (3) adversarial training strategies (e.g., EAP [5]), which leverage negative samples or prompts for unlearning. While these approaches have proven effective in static image settings, they encounter notable limitations in the context of modern flow-matching, transformer-based models such as Flux. Importantly, these techniques not only struggle to generalize across architectures but also lack the capacity to effectively address concept erasure in video generation. In video diffusion models, concepts persist and interact through complex spatio-temporal attention, leading to phenomena such as temporal drift and error accumulation—issues that existing CE methods are fundamentally unequipped to resolve. Although methods such as T2VUnlearning [71] and VideoEraser [70] have been proposed for concept erasure in video diffusion models, they still struggle to balance the removal of undesired content with the preservation of relevant information. As a result, these approaches are prone to introducing visual artifacts and often fail to maintain effective erasure or consistency in longer video sequences. To bridge this gap, we introduce EraseAnything++, a comprehensive framework for concept erasure in both image and video diffusion models built on flow-matching objectives. Specifically, we first propose a unified multi-objective optimization (MOO) framework for both image and video settings, where we design a performance-preserving gradient surgery method with theoretical support to balance concept removal and content preservation. In more detail, at the image level, we employ LoRA-based [25] parameter adaptation to suppress model activations associated with undesirable concepts, while explicitly preserving performance on a balanced set of irrelevant concepts. This is achieved through targeted fine-tuning using an ESD-inspired [11] loss, an attention map regularizer tailored to the internal structure of modern models, and a novel self-contrastive loss [48, 19] that penalizes the model for confusing erased concepts with irrelevant ones, leveraging both negative samples and diverse distractors. For the video setting, EraseAnything++ further extends these ideas to address the unique challenges of temporal consistency and error propagation. We propose an anchor-and-propagate strategy: erasure is first anchored on selected reference frames, and then consistently propagated through the model’s spatio-temporal transformer layers. By incorporating performance-preserving gradient surgery to ensure that erasure updates do not unduly compromise generative fidelity or introduce temporal artifacts, our approach enables stable, controllable suppression of target concepts across long video sequences, effectively mitigating the shortcomings of prior work. We validate EraseAnything++ on extensive image and video benchmarks. Experimental results demonstrate that our method substantially outperforms existing techniques in erasure effectiveness, generative fidelity, and temporal consistency. These findings establish EraseAnything++ as a robust and general solution for concept erasure in the latest generation of diffusion models. Notably, this work builds upon our earlier ICML conference version, EraseAnything [14], and introduces several key extensions in EraseAnything++: 1) We provide a formal treatment of concept erasure by defining it as a multi-objective optimization problem, extending our previous optimization method [79] and offering a rigorous mathematical model for the inherent trade-off between concept removal and generative fidelity. 2) Based on this theoretical framework, we propose a new optimization strategy employing efficient gradient projection, enabling more stable and controllable unlearning during training. 3) We extend the original EraseAnything approach from image-based concept erasure to video, addressing the unique challenges of spatio-temporal consistency and error propagation in text-to-video diffusion models. 4) We significantly expand our experimental evaluation, conducting extensive experiments on a broader set of benchmarks—including several dedicated video concept erasure datasets—to rigorously assess the performance and generality of EraseAnything++. • We present EraseAnything++, a unified and generalizable framework for concept erasure in both text-to-image and text-to-video diffusion models, supporting modern architectures based on flow matching and transformers. • We formulate concept erasure as a multi-objective optimization problem, providing a rigorous theoretical model that balances concept removal with the preservation of generative fidelity. • We design new optimization strategies, including LoRA-based parameter adaptation, attention-map regularization, self-contrastive loss, and an efficient gradient projection mechanism, enabling stable and controllable erasure in both images and videos. • We conduct extensive experiments across a wide range of benchmarks—including dedicated video concept erasure datasets—demonstrating that EraseAnything++ achieves state-of-the-art effectiveness, fidelity, and temporal consistency in both image and video generation tasks. I Related Work I-A Generative Diffusion Models for Images and Videos The landscape of generative vision has witnessed a paradigm shift, evolving from early Generative Adversarial Networks (GANs) [17] to the now dominant Diffusion Probabilistic Models (DPMs) [22]. Initial breakthroughs in text-to-image (T2I) generation are marked by models such as GLIDE [47], DALL-E 2 [54], and Imagen [58], which demonstrate unprecedented semantic control. The release of the Stable Diffusion (SD) series [56, 51] further democratize this technology, leveraging latent diffusion within a U-Net [57] architecture to balance computational efficiency and generation quality.Recently, the field has gravitated towards Diffusion Transformers (DiTs) combined with Flow Matching objectives, moving away from the traditional U-Net and epsilon-prediction paradigms. Stable Diffusion 3 (SD3) [8] exemplifies this shift, treating the forward noising process as a Rectified Flow [41] to establish a straight path between data and noise distributions. SD3 employs a Multimodal Diffusion Transformer (MMDiT) architecture, where text and pixel modalities are processed as sequences of embeddings. Specifically, positional encodings are applied to flattened 2×22× 2 latent patches, which are then fused with text embeddings from a trio of encoders (/,/, CLIPL/14,OpenCLIPbigG/14,T5\,XXL) [52, 53] before passing through modulated attention blocks. Building upon this foundation, Flux [35] has emerged as a state-of-the-art contender. By scaling the flow-matching transformer architecture to 12B parameters and incorporating rotary positional encodings (RoPE) [64], Flux achieves superior performance in prompt adherence, typography, and visual fidelity. This generative paradigm has naturally extended to Text-to-Video (T2V) synthesis. While early T2V models adapted U-Net architectures by inserting temporal attention layers [23, 21, 62], the latest generation follows the success of DiT in images. Models like CogVideo [24], HunyuanVideo [33], Open-Sora [77, 50] adopt a spatio-temporal patchification strategy, treating video as a long sequence of tokens. Open-Sora, in particular, democratizes efficient video production by employing a Video DiT architecture trained with flow-matching objectives. It supports variable resolutions and durations by effectively modeling long-range temporal dependencies through space-time attention mechanisms. In this work, we focus on this new wave of Flow-Matching Transformer models. We select Flux and Open-Sora as our primary experimental testbeds for image and video concept erasure, respectively. Their shared architectural principles allow us to propose a unified unlearning framework that addresses the unique challenges of these next-generation generative models. I-B Concept Erasure Large-scale datasets like LAION-5B [60] empower T2I and T2V models but introduce risks of generating inappropriate or copyrighted content. To mitigate this, early approaches focus on training datasets filtering [56], and post-generation content filtering [55]. For example, SD 2 filters training data, but this is costly and significantly affects model performance. Alternatively, libraries like Diffusers [68] use post-hoc safety checkers. However, these are easily bypassed by users and do not address the root cause. Consequently, research has shifted toward concept erasure via model fine-tuning. Methods such as ESD [11], UCE [12], MACE [45], and SPM [46] modify model weights to suppress specific concepts. Recent work places greater emphasis on preserving irrelevant concepts to maintain general capability. For instance, EAP [5] employs adversarial learning to retain unrelated semantics, while Real-Era [42] addresses “concept residue” by regularizing associated concepts to boost specificity. As diffusion models expand to the video domain, erasure becomes more complex due to temporal dynamics. Recent attempts like T2VUnlearning [71] and VideoEraser [70] address erasure in T2V models. However, these methods often struggle to balance concept removal with the preservation of relevant information. Consequently, they are prone to introducing visual artifacts and often fail to maintain consistency across longer video sequences. In this work, we aim to bridge these gaps in both image and video domains, with a specific focus on modern Flow-based Transformer models. A key challenge here is the text encoder. Unlike previous methods that rely on CLIP’s word-level embeddings, Flux and Open-Sora use T5. T5’s sentence-level embeddings make standard similarity metrics less effective for identifying irrelevant concepts. To address this, we propose a heuristic approach using Large Language Models (LLMs) to dynamically select irrelevant concepts. We elaborate on this T5-specific challenge and our solution in Section IV. I-C Multi-Objective Optimization Multi-Objective Optimization (MOO) provides a theoretical framework for optimizing conflicting tasks simultaneously. Early approaches primarily relied on dynamic loss re-weighting strategies. These methods adjust task weights based on homoscedastic uncertainty [32], gradient magnitudes (e.g., GradNorm [6]), or the relative complexity of training tasks [18]. To address gradient interference more directly, [61] formulates Multi-Task Learning (MTL) as an MOO problem, proposing the Multiple Gradient Descent Algorithm (MGDA) to find a Pareto stationary point. Subsequent works have introduced various techniques to resolve gradient conflicts. PCGrad [73] projects conflicting gradients onto the normal plane of others to prevent destructive interference. Similarly, GradDrop [7] stochastically discards conflicting gradient components based on their sign, while RotoGrad [30] aligns gradients through rotation. CAGrad [40] takes a different approach by constraining the update direction to remain within a specific region around the average gradient. Recent research has focused on improving the computational efficiency and theoretical guarantees of MOO solvers [26]. For instance, MoCo [10] extends MGDA to a probabilistic setting with convergence analysis, and FAMO [39] significantly reduces the computational overhead associated with gradient calculations. However, these general-purpose MOO methods are not directly applicable to concept erasure. In the context of erasure, the objectives are fundamentally asymmetric—requiring the precise removal of specific concepts while broadly preserving the model’s remaining generative utility. Standard MOO strategies often fail to balance this delicate trade-off, leading to either incomplete erasure or catastrophic forgetting of unrelated concepts. I Unified Optimization Framework In this section, we present the theoretical foundation of EraseAnything++. We formulate concept erasure as a constrained multi-objective optimization problem. Our goal is to maximize the erasure of a target concept while strictly bounding the degradation of unrelated concepts (preservation). To solve this efficiently, we propose a gradient projection method and a fast approximation algorithm. I-A Problem Formulation Consider a diffusion model parameterized by θ. Let ℒe()L_e( θ) denote the erasure objective (minimizing the likelihood of the target concept) and ℒp()L_p( θ) denote the preservation objective (maintaining the likelihood of irrelevant concepts). At iteration t, we update the parameters via t+1=t−αtt θ_t+1= θ_t- _t d_t, where αt _t is the step size and t d_t is the update direction. We define the improvement for each objective as: rp(αt,t)=ℒp(t)−ℒp(t+1), r_p( _t, d_t)=L_p( θ_t)-L_p( θ_t+1), (1) re(αt,t)=ℒe(t)−ℒe(t+1). r_e( _t, d_t)=L_e( θ_t)-L_e( θ_t+1). To achieve safe erasure, we seek a direction t d_t that maximizes the erasure improvement rer_e while ensuring the preservation degradation rpr_p remains within a controlled tolerance εt≥0 _t≥ 0. Mathematically, we pose this as: maxt _ d_t 1αtre(αt,t)−12‖t‖2 1 _tr_e( _t, d_t)- 12 \| d_t \|^2 (2) s.t. 1αtrp(αt,t)≥−εt, 1 _tr_p( _t, d_t)≥- _t, where ‖t‖2 \| d_t \|^2 acts as regularization to prevent unbounded updates. The constraint ensures that any increase in the preservation loss is bounded by εt _t. I-B Explicit Unilateral Gradient Surgery Since the step size αt _t is typically small, we apply a first-order Taylor approximation to Eq. (1): rp(αt,t)≈αt∇ℒp(t)⋅t,re(αt,t)≈αt∇ℒe(t)⋅t.r_p( _t, d_t)≈ _t _p( θ_t)· d_t, r_e( _t, d_t)≈ _t _e( θ_t)· d_t. (3) Consequently, Problem (2) is approximated by: maxt _ d_t ∇ℒe(t)⋅t−12‖t‖2 _e( θ_t)· d_t- 12 \| d_t \|^2 (4) s.t. ∇ℒp(t)⋅t≥−εt. _p( θ_t)· d_t≥- _t. This formulation aims to align t d_t with the erasure gradient ∇ℒe _e while strictly constraining its projection onto the preservation gradient ∇ℒp _p. Proposition I.1 (Dual Problem). The dual objective of Problem (4) is: minλt≥0⁡Lt(λt)=12‖∇ℒe(t)+λt∇ℒp(t)‖2+λtεt. _ _t≥ 0L_t( _t)= 12 \| _e( θ_t)+ _t _p( θ_t) \|^2+ _t _t. (5) This quadratic programming problem yields a closed-form analytical solution for the optimal update direction t∗ d_t^*. Proposition I.2 (Closed-Form Solution). The optimal direction t∗ d_t^* is given by: t∗=∇ℒe(t)+λt∗∇ℒp(t),if λt∗>0∇ℒe(t),if λt∗≤0 d_t^*= cases _e( θ_t)+ _t^* _p( θ_t),&if _t^*>0\\ _e( θ_t),&if _t^*≤ 0 cases (6) where the optimal Lagrange multiplier λt∗ _t^* is: λt∗=−∇ℒp(t)⋅∇ℒe(t)−εt‖∇ℒp(t)‖2. _t^*= - _p( θ_t)· _e( θ_t)- _t \| _p( θ_t) \|^2. (7) We defer derivations to Appendix A-B and A-C. Figure 2: Geometric interpretation of the proposed gradient surgery. (Left) When the erasure gradient ∇ℒe _e lies within the safe region (non-conflicting or satisfying the tolerance), the update direction t∗ d_t^* remains unchanged. (Right) When a significant conflict arises, t∗ d_t^* is obtained by projecting ∇ℒe _e to satisfy the preservation constraint. The tolerance εt _t creates a “tolerance cone” (dashed yellow line) that relaxes strict orthogonality, preventing optimization deadlocks while ensuring controlled utility preservation. Geometric Interpretation. The tolerance εt _t plays a pivotal role in navigating the trade-off between erasure and preservation. As shown in Fig. 2 (Left), when the erasure gradient ∇ℒe _e naturally satisfies the preservation constraint, no modification is needed. However, strictly enforcing non-degradation (εt=0 _t=0) often leads to optimization stagnation when gradients are diametrically opposed. By introducing εt>0 _t>0, we effectively construct a “tolerance cone” as illustrated in Fig. 2 (Right). Gradient surgery is triggered only when the conflict exceeds this threshold (i.e., ∇ℒp⋅∇ℒe<−εt _p· _e<- _t). This relaxation allows the erasure process to proceed even under moderate gradient conflict, preventing the “deadlock” phenomenon common in strict multi-objective optimization. Remark I.3. By imposing local constraints at each step, we can bound the global utility degradation: ℒp(t)−ℒp(0)≲(∑i=1tεtαt).L_p( θ_t)-L_p( θ_0) ( _i=1^t _t _t ). (8) Proof details are in Appendix A-D. This guarantees that preservation performance is controllable via hyperparameters. I-C Implicit Efficient Gradient Surgery While the explicit solution guarantees utility preservation, it requires computing both ∇ℒe _e and ∇ℒp _p separately. This doubles the computational cost compared to standard fine-tuning, undermining the efficiency required for practical unlearning. To address this, we propose an Implicit Efficient Gradient Surgery that approximates the solution with the cost of a single backpropagation. We solve for the weighting factor λt _t via gradient descent approximation rather than exact computation. The update rule for λt _t is: λt+1=λt−βt∇λtLt(λt). _t+1= _t- _t _ _tL_t( _t). Computing ∇λtLt _ _tL_t directly still requires separate gradients. Instead, we use a first-order approximation: ∇λtLt(λt) _ _tL_t( _t) =∇ℒp(t)⋅(∇ℒe(t)+λt∇ℒp(t))+εt = _p( θ_t)·( _e( θ_t)+ _t _p( θ_t))+ _t =∇ℒp(t)⋅t+εt = _p( θ_t)· d_t+ _t ≈1αt(ℒp(t)−ℒp(t+1))+εt. ≈ 1 _t(L_p( θ_t)-L_p( θ_t+1))+ _t. This yields a gradient-free update for λ: λt+1=λt−βtδ~t,where δ~t=1αt(ℒp(t)−ℒp(t+1))+εt. _t+1= _t- _t δ_t, δ_t= 1 _t(L_p( θ_t)-L_p( θ_t+1))+ _t. (9) Algorithm 1 Training Procedure of EraseAnything++ Input: Pretrained model 0 θ_0, erasure concept set eD_e, preservation concept set pD_p, total steps M. Hyperparameters: Learning rates α (model), β (dual variable), tolerance ε , initial λ0=0 _0=0. for iteration t=0t=0 to M−1M-1 do ❶ Sample batches ℬe∼eB_e _e and ℬp∼pB_p _p. ❷ Estimate preservation loss drift (via Eq. 9): δ~t=1α(ℒp(t)−ℒp(t+1))+ε δ_t= 1α(L_p( θ_t)-L_p( θ_t+1))+ . ❸ Update dual variable (implicit surgery): λt+1←max⁡(0,λt−βδ~t) _t+1← (0, _t-β δ_t). ❹ Compute composite objective: ℒtotal=ℒe(t)+λt+1ℒp(t)L_total=L_e( θ_t)+ _t+1L_p( θ_t). ❺ Update model parameters: t+1←t−α∇ℒtotal θ_t+1← θ_t-α _ θL_total. end for Output: Unlearned model parameters M θ_M. To implement the proposed approximation method, we adopt a streamlined optimization procedure, outlined in Algorithm 1, that bypasses the dual computational burden of standard gradient surgery. Specifically, the process begins by updating the dynamic weight λt _t based on forward-pass loss changes (Eq. (9)) without requiring backpropagation. Subsequently, the final update direction t d_t is derived via a single backpropagation of the composite objective ℒe+λtℒpL_e+ _tL_p, followed by standard parameter updates. By eliminating the necessity to explicitly compute and store separate gradient vectors for each objective, this strategy reduces computational overhead, ensuring scalability for large-scale diffusion models. I-D Theoretical Analysis We provide theoretical guarantees for the convergence and optimality of our implicit approximation. Theorem I.4 (Approximate λ∗λ^*). Assume ℒpL_p and ℒeL_e are G-Smooth and L-Lipschitz. With appropriate step sizes satisfying ∑αi≤(1)Σ _i (1) and ∑εi≤(1)Σ _i (1), the average gap between our approximate λt _t and the optimal λt∗ _t^* is bounded by: 1t∑i=1t(Li(λi)−Li(λi∗))≤(1/t1/3). 1t _i=1^t (L_i( _i)-L_i( _i^*) ) (1/t^1/3). (10) This confirms that as training progresses, λt _t converges to the optimal weight that ensures utility preservation. Theorem I.5 (Pareto Optimality). Under convex assumptions, there exists a composite loss ()C( θ) such that our algorithm converges to a Pareto optimal solution with rate: (t)−min⁡()≤(1/t).C( θ_t)- _ θC( θ) (1/t). (11) This matches the convergence rate of state-of-the-art first-order MOO algorithms. Remark I.6. Combining Eq. (8) and Theorem I.5, we show that our method reaches the optimal erasure solution ∗=max⁡ℒe() θ^*= _ θL_e( θ) subject to the strictly bounded preservation constraint. Theorem I.7 (Pareto Stationary). In non-convex scenarios (general deep learning), the algorithm converges to a Pareto stationary point: mini=1,…,t⁡min(μe,μp)∈Δ2⁡‖μe∇ℒe(i)+μp∇ℒp(i)‖≤(1/t1/2). _i=1,…,t _( _e, _p)∈ _2 \| _e _e( θ_i)+ _p _p( θ_i) \| (1/t^1/2). (12) This ensures effective optimization even in the complex non-convex landscapes of diffusion models. Detailed proofs are provided in Appendix A-E to A-G. IV Obstacles in Migrating Concept Erasure to Modern Transformers In this section, we analyze why classical erasure methods developed for Stable Diffusion (SD) fail when applied to modern architectures like Flux and Open-Sora. We identify three primary barriers: the shift from U-Net to Transformer backbones, the distinct characteristics of the T5 text encoder, and the introduction of temporal dimensions in video generation. These structural differences render direct adaptation of traditional methods infeasible. Architectural Mismatch: The first hurdle lies in the fundamental difference between the U-Net architecture of SD and the Transformer-based architecture of Flux and Open-Sora. Classical methods such as ESD [11], UCE [12], and MACE [45] primarily target explicit cross-attention layers to suppress concept activation. However, when adapting these methods to modern flow-based transformers (e.g., Flux) and video diffusion transformers (e.g., Open-Sora), we encounter an important challenge: explicit cross attention layer does not exist in either dual stream blocks or single stream blocks. The absence of isolated cross-attention layers means that traditional weights manipulation cannot be directly transplanted. This structural gap leads to “concept residue”, where target concepts are only partially removed, necessitating a redesigned erasure approach. The T5 Text Encoder Challenge: A significant obstacle shared by both Flux and Open-Sora is their reliance on the T5 text encoder. While SD utilizes CLIP as its standard text encoder for image guidance, Flux and Open-Sora rely on T5. Unlike CLIP, which is optimized for word-level alignment, T5 is designed for sentence-level understanding. As shown in table I, we extract the T5 feature for the word “nude” and compared it with the entire vocabulary (over 30,000 words) from the T5 default tokenizer. The cosine similarity reveals the top 3 closest synonyms based on semantic embeddings. However, these results were far from rational, indicating that T5’s word-level embeddings are not reliable for this task and cannot serve as an effective evaluator of semantic similarity. Another significant issue lies in the size of the T5 embeddings. With a shape of __(,) max\_sequence\_length(256,4096), T5 embeddings are approximately 18 times larger than CLIP embeddings, which have a shape of (,) (77,768). Consequently, this increased dimensionality makes the adaptive selection of adversarial prompts—a core component of methods like EAP [5]—computationally prohibitive. Unified Attention Analysis: Despite these architectural changes, we hypothesize that concept-specific activations must still exist within the network. Inspired by previous studies [20, 69], we examined the internal features of Flux. As shown in Fig. 3, even without explicit cross-attention, a linear correlation exists between text embeddings and the intermediate attention maps. Specifically, the attention weights W_attn are computed by concatenating textual and pixel embeddings: =(text,pixel,=−1), = concat(Q_text,Q_pixel, dim=-1), (13) =(text,pixel,=−1), = concat(K_text,K_pixel, dim=-1), =(×). _attn= Softmax(Q×K). This formulation suggests that the nexus between text and image is inherently forged within W_attn. By identifying the token index of a target word, we can theoretically nullify its influence by zeroing out the corresponding columns in the attention map. TABLE I: Comparison of synonym discovery for nude. Method Top-3 closest synonyms Claude 3.5 ”naked”, ”undressed”, ”unclothed” GPT-4o ”bare”, ”naked”, ”unclothed” Kimi ”naked”, ”unclothed”, ”bare” T5 feature ’lean’, ’deer’, ’girl’ Figure 3: Correlations between text and attention maps in Flux. Despite the lack of explicit cross-attention layers, a linear relationship persists between text embeddings and the unified attention map. Vulnerability to Obfuscation: While manually zeroing attention columns (as shown in Fig. 4) can erase a concept, this naive approach is fragile. Our experiments reveal it is highly susceptible to prompt attacks. Simple obfuscations—such as altering the prompt with nonsensical suffixes (soccer → soccerrs) or intentional misspellings (Nike → Nikke)—can bypass this filter. In these cases, the token mapping shifts, rendering the targeted erasure futile while the model still successfully generates the concept. Temporal Dynamics in Video: Migrating to video models like Open-Sora introduces a new dimension of complexity: temporal propagation. Unlike static images, video generation relies on temporal attention layers to maintain consistency across frames. A concept erased in the initial frame may inadvertently reappear in subsequent frames due to information leakage through temporal attention mechanisms. Consequently, image-based erasure methods often fail to prevent “concept drift” over time. This necessitates a method that not only erases concepts spatially but also enforces suppression consistently across the temporal axis, motivating our proposed EraseAnything++. Figure 4: Limitations of Attention Map erasure. Zeroing out attention columns ([:,:,idxi]=0W_attn[:,:,idx_i]=0) works for simple prompts but fails against adversarial attacks like keyword obfuscation (e.g., ”soccerrs”). V Method Building upon the Multi-Objective Optimization (MOO) framework established in Section I, we now detail the specific objective functions for EraseAnything++. Our method operates on a unified principle across both image (Flux) and video (Open-Sora) modalities: we decompose the unlearning task into two distinct objectives—Erasure (ℒeL_e) and Preservation (ℒpL_p). In the image setting, we employ a modified ESD loss combined with attention regularization to define ℒeL_e, while leveraging LoRA-based retention and a novel Reverse Self-Contrastive (RSC) loss for ℒpL_p. In the video setting, we extend these definitions via an Anchor-and-Propagate strategy, addressing the unique challenge of temporal consistency in 3D Video DiTs. Finally, these objectives are dynamically balanced within our established optimization framework. We now introduce each in detail. V-A Concept Erasure on Image Generation Models The primary goal of the erasure objective is to suppress the generation of specific target concepts defined in the erasure set eD_e. We achieve this through a combination of flow-matching guidance suppression and attention map regularization. Image Erasure Objective (ℒeimgL_e^img). To effectively erase concepts in rectified flow models, we adapt the ESD [11] loss to Flux, which emerges as the relatively superior performer with higher negative guidance. Unlike standard diffusion models that predict noise ϵε, Flux predicts velocity v. Therefore, we modify the objective to push the model’s velocity prediction on concept-laden prompts (ce∈ec_e _e) away from its original trajectory and towards the unconditional (null-text) trajectory. The modified loss is formulated as: ℒesd=xt,t,ce∼e∥v+Δ(xt,ce,t)− _esd=E_x_t,t,c_e _e \|v_ θ+ θ(x_t,c_e,t)- (14) [v(xt,∅,t)−η(v(xt,ce,t)−v(xt,∅,t))]∥22, [v_ θ(x_t, ,t)-η (v_ θ(x_t,c_e,t)-v_ θ(x_t, ,t) ) ] \|_2^2, where η is the negative guidance magnitude, ∅ represents the null text embedding, and Δ θ denotes the learnable LoRA parameters. In flow matching, xtx_t is the denoised latent code at timestep t started with random noise at xTx_T (T is the total timesteps in the denoising process). This loss effectively steers the flow vector field to bypass regions associated with the target concept. Additionally, on our analysis in Section IV, simply altering the generation trajectory is often insufficient due to “concept residue”. We further explicitly attenuate the activation of concept-specific tokens within the Transformer’s self-attention mechanism. Let [:,:,idx]W_attn[:,:,idx] represent the attention weights corresponding to the token index of the target concept (e.g., “nude”). We enforce a sparsity penalty: ℒattn=∑idx=startend[:,:,idx].L_attn= _idx=start^endW_attn[:,:,idx]. (15) To prevent the model from overfitting to fixed token positions, we apply a dynamic scrambling strategy. During training, we randomly shuffle the word order of the input prompts. Since Flux’s text encoder (T5) and the Transformer backbone are robust to permutation, this forces the model to learn semantic suppression rather than positional memorization. The total image erasure objective is ℒeimg=ℒesd+γ1ℒattnL_e^img=L_esd+ _1L_attn. Image Preservation Objective (ℒpimgL_p^img). The preservation objective aims to maintain the model’s generative capability for unrelated concepts pD_p, ensuring specificity. For a set of preservation concepts cp∈pc_p _p, we ensure that the model’s velocity predictions remain unchanged after the LoRA updates. For example, given the prompt “a nude girl”, our objective is to eliminate the word cec_e “nude” inside of prompt while ensuring the model can still generate an image of a unrelated concept cpc_p normally, e.g. girl. To achieve this, we generate 6-10 images from a fixed prompt and random seed (starting point of trajectory, same as DMs) that includes the concept to be removed (nude) and irrelevant concepts (girl), then train a LoRA (Low-Rank Adaptation) to induce shifts in the image generation process: ℒlora=xt,t,cp∼p[‖v(xt,cp,t)−v+Δ(xt,cp,t)‖22].L_lora=E_x_t,t,c_p _p [ \|v_ θ(x_t,c_p,t)-v_ θ+ θ(x_t,c_p,t) \|_2^2 ]. (16) However, this straightforward fine-tuning paradigm proves insufficient for preserving a broader spectrum of irrelevant concepts, particularly abstract artistic styles and complex relationships that are not explicitly involved in the input sentence. As analyzed in Section IV, manually curating a comprehensive dataset of images and prompts for all potential irrelevant concepts is prohibitively cumbersome. Furthermore, the T5 text encoder lacks the precision required for reliable word-level similarity measurements, rendering embedding-based selection ineffective. To overcome these limitations, we propose a contrastive learning method that operates directly on the attention maps of keywords. Unlike previous methods, our approach eliminates the need for a pre-defined set of irrelevant images. Instead, we leverage the semantic reasoning capabilities of Large Language Models (LLMs) to heuristically generate a set of preservation concepts pD_p that are semantically distinct from the target concept. Specifically, we deploy a lightweight automated agent powered by GPT-4o to sample irrelevant concepts cp∈pc_p _p. To augment this with hard negatives for robust learning and ensure computational efficiency, we employ NLTK [4] to generate synonyms of the target concept (e.g., generating ”naked” as a synonym for the target “nude”). In our default setting, we select K=3K=3 irrelevant concepts for each iteration. During the optimization process, we fix the initial noise latent xTx_T to ensure trajectory alignment. We then substitute the target word (e.g., “nude”) in the prompt c with the generated synonym csync_syn and the selected irrelevant concepts cpic_p^i (i=1,…,Ki=\1,…,K\). Each modified prompt undergoes an independent denoising step. Guided by the observations in Fig. 3, we extract attention maps at earlier timesteps (high noise levels), where semantic structure is most prominent. This yields the attention feature for the target concept FeF^e, the synonym feature FsynF^syn, and the set of irrelevant features Fir=Fk1,…,FkKF^ir=\F^k_1,…,F^k_K\. Drawing inspiration from contrastive representation learning [48, 19, 28], we propose a novel Reverse Self-Contrastive loss (RSC). Distinct from conventional contrastive objectives that pull positive pairs together, our goal is to align the target concept’s feature FeF^e with the dynamically changing irrelevant features FirF^ir, while simultaneously repelling it from the synonym feature FsynF^syn. This strategy effectively inverts standard semantic alignment: rather than enhancing sensitivity by bringing the target closer to its synonyms, we force the network to disassociate the target word from its visual representation. By pushing the target representation towards the manifold of irrelevant concepts, we effectively obfuscate the concept (e.g., “nude”) during the learning process, ensuring it is treated as semantically unrelated noise: ℒrsc=log⁡(∑i=0Kexp⁡(Fe⋅Fki/τ)exp⁡(Fe⋅Fsyn/τ)).L_rsc= ( _i=0^K (F^e· F^k_i/τ ) (F^e· F^syn/τ ) ). (17) Full derivation details are in Appendix B, the temperature hyperparameter τ plays a critical role in regulating the model’s discriminative capability between irrelevant concepts. A high τ smooths the distribution, causing the loss to treat all irrelevant concepts with uniform importance, which can lead to a lack of focus during the learning process. Conversely, an excessively low τ sharpens the distribution, forcing the model to over-penalize hard negatives—potentially misidentifying valid irrelevant concepts as synonyms. Through empirical ablation, we determine that setting τ=0.07τ=0.07 yields the optimal balance for our model’s performance. The final preservation objective is: ℒp=ℒlora+γ2ℒrscL_p=L_lora+ _2L_rsc. During training, ℒeimgL_e^img and ℒpimgL_p^img are dynamically balanced by our MOO solver, naturally preventing the catastrophic forgetting prevalent in standard unlearning. V-B Concept Erasure on Video Generation Models A major challenge in modern generative vision, and a primary advancement of EraseAnything++ over our previous work [13], is scaling concept erasure to the temporal dimension. Migrating from Image (Flux) to Video (Open-Sora) introduces severe temporal dynamics: concepts are not static; they propagate and morph across frames via 3D temporal attention layers within the Multi-Modal Diffusion Transformer (MMDiT) architecture. Naively applying standard fine-tuning or scalarized losses to video models typically leads to an unresolvable conflict: the model either suffers from “concept drift” (where the erased concept hallucinates back into existence in later frames) or temporal collapse (where the enforcement of erasure inadvertently freezes motion and degrades video smoothness). To resolve this, we strictly formulate video concept erasure as a Multi-Objective Optimization problem powered by an Anchor-and-Propagate strategy. Anchor-and-Propagate Strategy. We propose a two-stage suppression mechanism tailored for 3D Video DiTs, where the input is a spatio-temporal volume of tokens V∈ℝT×H×W×CV ^T× H× W× C. First, we treat the initial frame as an Anchor (t=1t=1) to ground the generation. Instead of relying solely on an erasure penalty, we apply the complete spatial optimization suite (both image-level erasure ℒeimgL_e^img and preservation ℒpimgL_p^img) to this anchor frame. This logically mirrors employing a fully fine-tuned spatial model to generate a sanitized initial state, ensuring the seed of the entire sequence is cleansed of the target concept while retaining benign semantics. Second, to propagate this clean state and prevent the concept from leaking back through subsequent temporal attention blocks, we extend our four core loss components into their volumetric counterparts within the 3D MMDiT. Video Erasure Objective (ℒevidL_e^vid). The video erasure objective consists of volumetric flow-matching and 3D attention regularization. We extend the ESD loss to operate on the full video volume VtV_t, steering the volumetric velocity trajectory away from the concept ce∈ec_e _e: ℒesdvid _esd^vid =Vt,t,ce∼e∥v+Δ(Vt,ce,t)− =E_V_t,t,c_e _e \|v_ θ+ θ(V_t,c_e,t)- (18) [v(Vt,∅,t)−η(v(Vt,ce,t)−v(Vt,∅,t))]∥22. [v_ θ(V_t, ,t)-η (v_ θ(V_t,c_e,t)-v_ θ(V_t, ,t) ) ] \|_2^2. Simultaneously, we introduce Volumetric Attention Regularization. To eradicate concept residue hiding in temporal pathways, we unfold the attention sparsity penalty across the entire temporal duration T: ℒattnvid=1T∑t=1Tℒattn(t),L_attn^vid= 1T _t=1^TL_attn^(t), (19) where ℒattn(t)L_attn^(t) applies the structural penalty to the 3D MMDiT spatio-temporal attention maps at frame slice t. The composite video erasure objective is then defined as ℒevid=ℒesdvid+γ1ℒattnvidL_e^vid=L_esd^vid+ _1L_attn^vid. Video Preservation Objective (ℒpvidL_p^vid). To maintain motion fidelity, the video preservation objective must protect the temporal trajectories of benign subjects. We expand the visual consistency loss to the temporal axis by sampling short irrelevant video volumes VtV_t generated from unchanged parameters: ℒloravid=Vt,t,cp∼p[‖v(Vt,cp,t)−v+Δ(Vt,cp,t)‖22].L_lora^vid=E_V_t,t,c_p _p [ \|v_ θ(V_t,c_p,t)-v_ θ+ θ(V_t,c_p,t) \|_2^2 ]. (20) Furthermore, we adapt our Reverse Self-Contrastive loss to the temporal domain (ℒrscvidL_rsc^vid) by extracting the 3D semantic attention features across high-noise video latents. This ensures the model dynamically disassociates the target word from its visual representation across the entire sequence, rather than just spatially on a single frame. The composite video preservation objective becomes ℒpvid=ℒloravid+γ2ℒrscvidL_p^vid=L_lora^vid+ _2L_rsc^vid. By integrating this volumetric constraint into our Multi-Objective Optimization framework, EraseAnything++ achieves consistent concept erasure in long-horizon video generation without compromising motion smoothness. VI Experiments Figure 5: Single concept erasure. We compare our model with mainstream concept erasing methods. The Attention Map column shows the simple token localization method from Section IV that erases target concept effectively, yet its vulnerable to the minor change of tokens, making it difficult to widely adopt in practical applications. In this section, we conduct a comprehensive evaluation of EraseAnything++, benchmarking it on various tasks ranging from static image generation to dynamic video synthesis. We assess the method’s effectiveness across concrete entities (e.g., objects, celebrities) and abstract concepts (e.g., artistic styles, nudity), as well as its robustness against adversarial attacks. VI-A Implementation Details Model Architecture. We opt for the Flux.1 [dev] 111https://huggingface.co/black-forest-labs/FLUX.1-dev model [35] as our primary testbed for image experiments. It features a rectified flow transformer architecture with 12B parameters, serving as a distilled version of Flux.1 [pro] while retaining high prompt adherence. For video experiments, we employ Open-Sora-v2 222https://huggingface.co/hpcai-tech/Open-Sora-v2, a state-of-the-art video diffusion transformer that shares a similar flow-matching objective and T5 text encoder with Flux. Training Settings. Our codebase is built upon the diffusers library [68]. We employ the Flow-Matching Euler sampler with 28 inference steps. The optimization utilizes AdamW [44] for 1,000 steps with a batch size of 1. We set the model learning rate to α=1×10−3α=1× 10^-3 and the dual variable update rate to β=0.1β=0.1. The negative guidance factor is fixed at η=2η=2. Regarding the loss components, we empirically set the weighting factors to γ1=0.01 _1=0.01 and γ2=1.0 _2=1.0. These values prove to be proper choices for ensuring stable convergence and effective erasure across diverse tasks, eliminating the need for further hyperparameter tuning. Concept Construction. We harness NLTK [4] to generate synonym candidates and deploy GPT-4o to heuristically mine irrelevant concepts (pD_p). Our fine-tuning targets the text-projection layers add_q_proj and add_k_proj within the dual stream blocks. This parameter-efficient strategy requires storage of less than 0.01% of the model parameters. Figure 6: User Study results. EraseAnything++ yields the best overall performance, balancing erasure effectiveness with the preservation of image quality and prompt adherence. TABLE I: Quantitative comparison on nudity erasure. (Left) Count of explicit body parts detected by NudeNet on 4,703 I2P prompts. (Right) Zero-shot generation quality on MS-COCO 10K. The performance of the original Flux.1 [dev] is presented for reference. Detected Nudity (Quantity) MS-COCO 10K Method Armpits Belly Buttocks Feet Breasts (F) Genitalia (F) Breasts (M) Genitalia (M) Total ↓ FID ↓ CLIP ↑ Flux.1 [dev] 147 168 29 62 152 9 28 10 605 21.32 30.87 CA (Model-based) [34] 86 103 26 38 57 8 20 6 344 22.66 29.05 CA (Noise-based) [34] 91 122 37 40 61 11 22 6 390 23.07 28.73 ESD-x [11] 107 131 22 69 138 7 24 8 506 23.08 28.44 ESD-u [11] 81 76 5 48 69 8 20 5 312 22.48 28.07 UCE [12] 28 59 7 28 34 5 8 4 173 30.71 24.56 MACE [45] 14 72 29 58 47 8 18 10 256 24.15 29.52 EAP [5] 68 96 41 82 82 4 7 6 386 22.30 29.86 Meta-Unlearning [15] 99 206 35 15 133 7 15 11 521 22.69 29.91 AdvUnlearn [75] 47 27 16 51 23 8 18 13 203 22.37 29.13 SalUn [9] 8 57 2 55 29 8 39 10 208 22.84 28.96 FMN [74] 36 98 10 50 134 10 14 4 356 24.26 27.43 EraseAnything [13] 29 57 6 37 41 7 17 5 199 21.75 30.24 EraseAnything++ 26 59 4 39 31 2 18 3 182 21.67 30.35 TABLE I: Quantitative Results on Artistic Style Erasure. We evaluate performance on the 200-artist dataset [45]. We report the CLIP alignment score for erased styles (Acce Acc_e) and preserved styles (Accir Acc_ir). The overall score is defined as Ha=Accir−AcceH_a= Acc_ir- Acc_e (Higher is better). General image quality is measured by FID and CLIP on MS-COCO. Method Acce ↓ Accir ↑ Ha ↑ FID ↓ CLIP ↑ CA (Model-based) [34] 29.26 28.54 -0.72 22.63 29.13 FMN [74] 29.63 28.90 -0.73 24.32 27.66 ESD-x [11] 20.89 21.21 0.32 22.89 28.12 ESD-u [11] 19.66 19.55 -0.11 22.06 27.94 UCE [12] 21.31 25.70 4.39 30.22 24.08 EAP [5] 21.10 26.34 5.24 22.84 29.25 MACE [45] 22.59 28.58 5.99 23.91 29.60 EraseAnything [13] 20.73 26.91 6.18 21.63 29.08 EraseAnything++ 20.71 27.32 6.61 21.50 30.11 TABLE IV: Evaluation of Erasing the specific category: Entity (e.g. soccer), Abstraction (e.g. artistic style) and Relationship (e.g. kiss) are presented. CLIP classification accuracies are reported for each erased category in three sets: the erased category itself (Acce, efficacy), the remaining unaffected categories (Accir, specificity) and synonyms of the erased class (Accg, generality). All presented values are denoted in percentage (%). Entity Abstraction Relationship Method Acce ↓ Accir ↑ Ha ↑ Accg ↓ Acce ↓ Accir ↑ Ha ↑ Accg ↓ Acce ↓ Accir ↑ Ha ↑ Accg ↓ CA (Model-based) [34] 14.8 24.2 9.4 27.3 25.8 23.1 -2.7 30.5 22.7 23.6 0.9 23.1 FMN [74] 14.9 23.7 8.8 27.6 25.4 22.5 -2.9 31.3 23.4 23.5 0.1 23.4 ESD-x [11] 14.5 25.3 10.8 26.1 26.1 23.4 -2.7 28.4 22.3 23.9 1.6 22.9 ESD-u [11] 12.8 24.7 11.9 25.8 22.8 21.3 -1.5 28.1 20.5 22.2 1.7 22.0 UCE [12] 12.4 19.6 7.2 27.5 20.4 18.0 -2.4 30.2 18.9 17.6 -1.3 23.8 EAP [5] 13.7 23.1 9.4 25.2 21.9 22.8 0.9 25.8 19.6 21.8 2.2 22.6 MACE [45] 13.8 25.2 11.4 19.7 22.6 24.6 2.0 24.9 20.1 22.5 2.4 20.2 EraseAnything [13] 12.5 26.7 14.2 18.6 20.8 25.7 4.9 24.5 18.4 25.2 6.8 19.3 EraseAnything++ 12.4 26.9 14.5 18.6 20.6 25.7 5.1 24.8 18.5 25.6 7.1 19.1 TABLE V: Robustness Evaluation. Comparison of erasure performance under various prompt attacks (lower is better). Method Unlearn DiffAtk [76] Ring-A-Bell [66] Ring-A-Bell Union [66] ReFlux [31] CA [34] 85.32 27.50 44.03 87.16 FMN [74] 92.66 32.11 46.78 - ESD-x [11] 76.14 11.01 24.77 86.24 ESD-u [11] 70.64 18.34 29.36 89.91 UCE [12] 77.98 22.93 29.36 - EAP [5] 98.16 35.77 45.87 99.08 MACE [45] 79.81 32.11 40.36 85.32 EraseAnything [13] 71.55 29.36 32.11 88.99 EraseAnything++ 68.80 22.93 30.27 86.24 TABLE VI: Quantitative Results on Video Nudity Erasure. Evaluated on Open-Sora. Nudity Rate (Lower is better) measures erasure success. Object Class and Subject Consistency (Higher is better) measure video quality and coherence. Nudity Rate (%) ↓ Method Gen [71] Ring-A-Bell [66] Object Class ↑ Subject Consistency ↑ Open-Sora 58.56 28.63 90.52 95.80 SAFREE [72] 32.12 13.17 48.48 94.92 NegPrompt [36] 36.00 11.75 91.94 93.45 VideoErasure [70] 21.45 10.13 80.62 93.77 T2VUnlearning [71] 19.73 6.97 87.00 94.70 EraseAnything++ 17.29 8.04 89.43 95.21 VI-B Image Concept Erasure Performance Nudity Erasure. We benchmark on the Inappropriate Image Prompt (I2P) dataset [59], generating images from 4,703 unsafe prompts. Detection is performed using NudeNet [3] with a threshold of 0.6. To measure the preservation of general generative capabilities, we evaluate FID and CLIP scores on 10,000 captions from MS-COCO dataset [37]. As shown in Table I, EraseAnything++ achieves the second-lowest quantity of detected explicit content, only outperformed by UCE. Yet, it stands out with remarkable FID and CLIP scores, suggesting that our approach has minimal negative influence on the original model’s ability to generate regular content. In contrast, UCE suffers a significant drop in image quality, whereas our method achieves a superior Pareto frontier. Artistic Style Erasure. To assess the capability of removing abstract concepts, we conduct experiments on the 200-artist dataset introduced by MACE [45], which consists of two groups: an erasure group of 100 artists whose styles are targeted for removal, and a preservation group of 100 artists whose styles are intended to be retained. We employ the CLIP score to measure the alignment between generated images and the specific artistic styles. For the erasure group, a lower CLIP score (Acce Acc_e) indicates better performance, signifying effective suppression of the target style. Conversely, for the preservation group, a higher CLIP score (Accir Acc_ir) is desirable, reflecting minimal disruption to unrelated styles. To quantify the overall trade-off, we define a composite metric Ha=Accir−AcceH_a= Acc_ir- Acc_e, where a higher value represents a superior balance between erasure and preservation. Table I summarizes the quantitative performance. EraseAnything++ achieves the highest HaH_a score, substantially surpassing baseline methods. This demonstrates our method’s precision in excising targeted artistic features while robustly protecting unrelated aesthetics. Figure 7: Multi concept erasure. Multiple concepts can be removed simultaneously by a normalized linear weight composition of their LoRA updates. Miscellaneous Erasure. We broaden our evaluation to three specific categories: Entity, Abstraction, and Relationship. Here, we choose 10 concept for each category (see Appendix C-A for details). As shown in table IV, we report the erasure efficacy (Acce), specificity on unrelated categories (Accir), and generalization to synonyms (Accg). Our method achieves the best balance between erasure and preservation, particularly in preventing the “concept recovery” phenomenon when synonyms are used. Visual comparisons in Fig. 5 further confirm that EraseAnything++ precisely removes the target concept without degrading the surrounding context. Notably, our approach naturally extends to multiple concepts by simply composing the learned LoRA updates (Fig. 7). Robustness against Attacks. A critical weakness of existing erasure methods is their vulnerability to adversarial prompts. We evaluate robustness using the ReFlux [31] dataset and other attack benchmarks (Ring-A-Bell [66], UnlearnDiffAtk [76]). As presented in table V, EraseAnything++ achieves the lowest attack success rates across all benchmarks, significantly outperforming ESD and MACE. This indicates that our logic-based contrastive loss effectively generalizes erasure to the semantic concept rather than overfitting to specific tokens. User Study. We conducted a human evaluation with 20 participants assessing five dimensions: Erasing Cleanliness, Irrelevant Preservation, Imaging Quality, Prompt Adherence, and Output Diversity. For the first two trials: Erasing Cleanliness (prompt with cec_e and generate images that do not contain cec_e) and Irrelevant Preservation (prompt with cpc_p can be normally generated), we use the same concepts categorized under Entity, Abstraction, and Relationship. For each concept, images are generated using the same seed across all methods, ensuring a fair comparison. Figure 6 demonstrates that our method yields the most balanced performance, particularly excelling in preserving irrelevant content and image quality compared to aggressive methods like UCE, making EraseAnything++ a good all-round player in concept erasure area. VI-C Video Concept Erasure Performance Nudity Erasure. We evaluate nudity erasure using two distinct datasets: (1) Gen [71], a curated set of prompts describing nudity with rich context and detail; and (2) Ring-A-Bell [66], a collection of stylized short prompts depicting explicit artwork, adopted from [16, 9]. To assess the efficacy of erasure, we generate 129 frames per prompt using the Open-Sora-2.0 model at its default resolution. We employ the NudeNet detector to identify frames containing explicit content and report the Nudity Rate, defined as the proportion of frames flagged with any nudity-related tags. To analyze the potential impact on non-target concepts, we adopt VBench [27], a widely used video generation benchmark. We evaluate the erased model on Object Class and Subject Consistency metrics to assess its ability to generate benign concepts accurately and maintain temporal coherence across frames. As shown in Table VI, EraseAnything++ achieves a superior balance between erasure and preservation. Its advantage is particularly pronounced on the Gen dataset, which features longer and more detailed prompts, reducing the Nudity Rate to a state-of-the-art. Furthermore, our method maintains performance comparable to prior methods when generating benign content. It successfully preserves the ability to generate non-nudity objects and ensures strong temporal consistency. Qualitative analysis in Fig. 8 further highlights the limitations of existing baselines. SAFREE [72] exhibits only marginal erasure effects, often failing to suppress explicit content effectively. NegPrompt [36], while reducing nudity, frequently introduces severe visual artifacts, such as over-saturation and temporal flickering. VideoErasure [70] suffers from significant semantic drift, leading to low prompt alignment where the generated content diverges substantially from the text description. Notably, T2VUnlearning [71] tends toward over-erasure; for instance, it aggressively removes the entire subject (e.g., the “girl”) rather than solely the explicit attribute. In contrast, EraseAnything++ precisely targets the specific concept while maintaining high fidelity to benign elements and ensuring strong temporal consistency. ImageNet Object Erasure. We further evaluate object erasure following the protocol of ESD [11], selecting 10 distinct ImageNet classes [58] as target concepts. Specifically, we erase one concept at a time and assess the preservation performance on the remaining nine. We conduct per-frame classification and calculate the average Top-k accuracy. We define the Erasure Success Rate (ESR-k) as 1−Top-k Accuracy1-Top-k Accuracy for the erased concept, and the Preservation Success Rate (PSR-k) as the average Top-k accuracy for the preserved concepts. As demonstrated in Table VII, our method achieves the highest Erasure Success Rate, indicating its robust capability to remove target objects even in dynamic video settings. Crucially, unlike SAFREE which suffers a severe drop in preservation, EraseAnything++ maintains a Preservation Success Rate that is highly competitive with the original model. This confirms that our multi-objective optimization effectively isolates the target concept without catastrophically forgetting general knowledge. Figure 8: Qualitative comparison of nudity erasure on Open-Sora. Given the explicit prompt “A beautiful girl takes off her top to reveal her bare breasts”, the original model generates NSFW content (masked for safety). In contrast, EraseAnything++ successfully sanitizes the video by removing the nudity while preserving the main subject and maintaining temporal consistency across frames. TABLE VII: Results of ImageNet Object Erasure on Open-Sora. We compare the Erasure Success Rate (ESR) and Preservation Success Rate (PSR). Higher values indicate better performance for both metrics. Method ESR-1↑ ESR-5↑ PSR-1↑ PSR-5↑ Open-Sora 21.62 5.09 78.38 94.91 NegPrompt [36] 48.59 19.79 65.37 88.62 SAFREE [72] 61.65 36.41 53.46 79.17 T2VUnlearning [71] 92.38 77.09 54.03 82.14 EraseAnything++ 94.15 79.20 76.45 93.10 VI-D Ablation study To thoroughly validate the effectiveness of our proposed framework, we conduct ablation studies from three perspectives: the contribution of individual loss components in the image domain, the necessity of our spatio-temporal strategy in the video domain, and the efficacy of our multi-objective optimization mechanism. Ablation on loss functions. To assess our loss functions, we conduct an ablation study on the task of celebrity image erasure. We choose a subset from the CelebA [43], omitting those that Flux.1 [dev] can’t accurately reconstruct. This results in a dataset of 100 celebrities, split into two groups: 50 for erasure and 50 for retention (see Appendix C-B). Different variations and their results are presented in Table VIII and Fig. 9. Solely relying on the aggressive erasure objective (ℒesd+ℒattnL_esd+L_attn) yields the strongest erasure efficacy with the lowest Acce, but this comes at the cost of catastrophic forgetting, significantly degrading the preservation of unrelated semantics. Conversely, introducing ℒloraL_lora shifts the balance towards preservation, but failing to effectively remove the target concept, as indicated by a high Acce. The incorporation of ℒrscL_rsc is crucial for distinguishing the target from synonyms, improving the trade-off. Ultimately, by combining all these loss terms, our full method achieves the highest HaH_a score, demonstrating the most effective equilibrium between precise removal and semantic retention. Figure 9: Ablation of different loss configs. Our full method achieves the most effective equilibrium between concept erasure and semantic preservation. TABLE VIII: Ablation on Loss Components. We report CLIP score of Erasure Accuracy (Acc↓e_e ), Preservation Accuracy (Acc↑ir_ir ), and their difference score (Ha=Accir−Acce↑H_a= Acc_ir- Acc_e ). While biased configurations may excel in a single metric, our full method achieves the best overall balance (HaH_a). Config Acce ↓ Accir ↑ Ha ↑ ℒesdL_esd + ℒattnL_attn 19.2 23.5 4.3 ℒesdL_esd + ℒloraL_lora 26.8 28.8 2.0 ℒesdL_esd + ℒrscL_rsc 21.5 27.1 5.6 ℒattnL_attn + ℒrscL_rsc 23.1 26.5 3.4 ℒattnL_attn + ℒloraL_lora + ℒrscL_rsc 21.8 27.9 6.1 Full (EraseAnything++) 20.1 28.2 8.1 Ablation on Spatio-Temporal Strategy. Migrating erasure to the video domain introduces unique challenges regarding temporal consistency. We qualitatively validate our Anchor-and-Propagate strategy by comparing it with two incomplete variants on Open-Sora, as visualized in Fig. 10. When we omit the anchor frame constraint and rely solely on volumetric tuning (w/o Anchor), the target concept leaks into the initial frame and subsequently propagates throughout the video due to the lack of initial suppression. On the other hand, erasing only the first frame without applying the 3D attention constraint (w/o Propagation) leads to temporal drift, where the erased concept hallucinates and re-emerges in later frames. Our full strategy, which couples anchor frame erasure with volumetric propagation, is the only configuration that ensures consistent and stable erasure across the entire temporal dimension. Figure 10: Ablation of the Spatio-Temporal strategy. (Top) w/o Anchor: The target concept is not suppressed at initialization, leading to immediate leakage. (Middle) w/o Propagation: Although erased initially, the concept re-emerges in later frames due to temporal drift. (Bottom) Full Strategy: Our method ensures consistent erasure across the entire video duration. Ablation on Optimization Strategy. Finally, we justify the necessity of our proposed Multi-Objective Optimization (MOO) framework by benchmarking it against various optimization paradigms. We compare our approach with standard Linear Scalarization (ℒtotal=ℒe+λℒpL_total=L_e+ _p) and advanced gradient manipulation techniques, specifically PCGrad [73] and MGDA [61]. As detailed in Table IX, the Linear Scalarization strategy proves extremely sensitive to the hyperparameter λ. A small weight (λ=0.1λ=0.1) biases the optimization heavily towards erasure, achieving a low Acce Acc_e but causing catastrophic forgetting of unrelated concepts. Conversely, increasing the weight (λ=10.0λ=10.0) enforces rigid preservation but stifles the erasure process, rendering the unlearning ineffective. Furthermore, advanced MOO techniques like PCGrad and MGDA, while designed to mitigate gradient interference, yield suboptimal results in the context of concept erasure. These algorithms treat objectives symmetrically, aiming to find a common descent direction that improves all tasks simultaneously. However, in concept erasure, the gradient for removal often directly conflicts with preservation. To satisfy the symmetric improvement condition, PCGrad and MGDA tend to adopt a conservative update strategy, dampening the erasure gradient to align with preservation. Consequently, while they maintain high Accir Acc_ir, they fail to effectively erase the target concept, resulting in poor AcceAcc_e. In contrast, our approach fundamentally models the problem as an asymmetric constrained optimization. By dynamically projecting the erasure gradient only when it violates the preservation constraint, our method breaks the optimization deadlock. This allows EraseAnything++ to achieve deep erasure while maintaining high specificity, yielding the superior Pareto stationary point. TABLE IX: Ablation on Optimization Strategy. Evaluated on CelebA (same as Table VIII). General MOO solvers (PCGrad [73], MGDA [61]) tend to be conservative, prioritizing preservation at the cost of weak erasure. Our asymmetric strategy yields the best balance. Strategy Acce ↓ Accir ↑ Ha ↑ Linear Sum (λ=0.1λ=0.1) 18.5 22.5 4.0 Linear Sum (λ=10.0λ=10.0) 28.6 28.9 0.3 PCGrad [73] 24.1 28.5 4.4 MGDA [61] 25.8 28.7 2.9 Ours 20.1 28.2 8.1 VII Conclusion In this paper, we propose EraseAnything++, a framework designed to address concept erasure in modern Flow-based Transformers for both image and video generation. By reformulating the unlearning task as a constrained Multi-Objective Optimization (MOO) problem via implicit gradient surgery, we effectively resolve the asymmetric conflict between erasing target concepts and preserving model utility, thereby mitigating the notorious risks of catastrophic forgetting and overfitting. Furthermore, we extend our approach to the temporal dimension with a Spatio-Temporal Anchor-and-Propagate strategy to counteract concept drift in videos. Extensive experiments across diverse tasks demonstrate that our method establishes a superior Pareto frontier, proving its effectiveness and versatility as a robust solution for safe generative AI. VIII Acknowledgement This work was supported by the National Natural Science Foundation of China under Grant No. 62441617. It was also supported by the Postdoctoral Fellowship Program and China Postdoctoral Science Foundation under Grant No. 2024M764093 and Grant No. BX20250485, the Beijing Natural Science Foundation under Grant No. 4254100, and by Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing. References [1] F. Barez, T. Fu, A. Prabhu, S. Casper, A. Sanyal, A. Bibi, A. O’Gara, R. Kirk, B. Bucknall, T. Fist, L. Ong, P. Torr, K. Lam, R. Trager, D. Krueger, S. Mindermann, J. Hernandez-Orallo, M. Geva, and Y. Gal (2025) Open problems in machine unlearning for ai safety. External Links: 2501.04952 Cited by: §I. [2] BBC News (2025) French woman duped by ai brad pitt faces mockery online. BBC News. External Links: Link Cited by: §I. [3] P. Bedapudi (2019) Nudenet: neural nets for nudity classification, detection and selective censoring. Cited by: §VI-B. [4] S. Bird, E. Klein, and E. Loper (2009) Natural language processing with python: analyzing text with the natural language toolkit. ” O’Reilly Media, Inc.”. Cited by: §V-A, §VI-A. [5] A. Bui, L. Vuong, K. Doan, T. Le, P. Montague, T. Abraham, and D. Phung (2024) Erasing undesirable concepts in diffusion models with adversarial preservation. arXiv preprint arXiv:2410.15618. Cited by: Figure 1, §I, §I-B, §IV, TABLE I, TABLE I, TABLE IV, TABLE V. [6] Z. Chen, V. Badrinarayanan, C. Lee, and A. Rabinovich (2018) Gradnorm: gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, p. 794–803. Cited by: §I-C. [7] Z. Chen, J. Ngiam, Y. Huang, T. Luong, H. Kretzschmar, Y. Chai, and D. Anguelov (2020) Just pick a sign: optimizing deep multitask models with gradient sign dropout. Advances in Neural Information Processing Systems 33, p. 2039–2050. Cited by: §I-C. [8] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, et al. (2024) Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, Cited by: §I-A. [9] C. Fan, J. Liu, Y. Zhang, D. Wei, E. Wong, and S. Liu (2023) Salun: empowering machine unlearning via gradient-based weight saliency in both image classification and generation. arXiv preprint arXiv:2310.12508. Cited by: §VI-C, TABLE I. [10] H. D. Fernando, H. Shen, M. Liu, S. Chaudhury, K. Murugesan, and T. Chen (2023) Mitigating gradient bias in multi-objective learning: a provably convergent approach. In The Eleventh International Conference on Learning Representations, Cited by: §I-C. [11] R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau (2023) Erasing concepts from diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 2426–2436. Cited by: Figure 1, §I, §I, §I, §I-B, §IV, §V-A, §VI-C, TABLE I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE IV, TABLE V, TABLE V. [12] R. Gandikota, H. Orgad, Y. Belinkov, J. Materzyńska, and D. Bau (2024) Unified concept editing in diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 5111–5120. Cited by: §I, §I-B, §IV, TABLE I, TABLE I, TABLE IV, TABLE V. [13] D. Gao, S. Lu, S. Walters, W. Zhou, J. Chu, J. Zhang, B. Zhang, M. Jia, J. Zhao, Z. Fan, et al. (2024) EraseAnything: enabling concept erasure in rectified flow transformers. ICML 2025. Cited by: §V-B, TABLE I, TABLE I, TABLE IV, TABLE V. [14] D. Gao, S. Lu, W. Zhou, J. Chu, J. Zhang, M. Jia, B. Zhang, Z. Fan, and W. Zhang (2025) Eraseanything: enabling concept erasure in rectified flow transformers. In Forty-second International Conference on Machine Learning, Cited by: §I. [15] H. Gao, T. Pang, C. Du, T. Hu, Z. Deng, and M. Lin (2024) Meta-unlearning on diffusion models: preventing relearning unlearned concepts. arXiv preprint arXiv:2410.12777. Cited by: TABLE I. [16] C. Gong, K. Chen, Z. Wei, J. Chen, and Y. Jiang (2024) Reliable and efficient concept erasure of text-to-image diffusion models. External Links: 2407.12383 Cited by: §VI-C. [17] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio (2014) Generative adversarial nets. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger (Eds.), Vol. 27, p. . Cited by: §I-A. [18] M. Guo, A. Haque, D. Huang, S. Yeung, and L. Fei-Fei (2018) Dynamic task prioritization for multitask learning. In Proceedings of the European conference on computer vision (ECCV), p. 270–287. Cited by: §I-C. [19] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick (2020) Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9729–9738. Cited by: §I, §V-A. [20] A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or (2022) Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626. Cited by: §IV. [21] J. Ho, W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, D. J. Fleet, and T. Salimans (2022) Imagen video: high definition video generation with diffusion models. External Links: 2210.02303 Cited by: §I-A. [22] J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in neural information processing systems 33, p. 6840–6851. Cited by: §I, §I-A. [23] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet (2022) Video diffusion models. arXiv:2204.03458. Cited by: §I-A. [24] W. Hong, M. Ding, W. Zheng, X. Liu, and J. Tang (2022) CogVideo: large-scale pretraining for text-to-video generation via transformers. External Links: 2205.15868 Cited by: §I-A. [25] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2021) Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. Cited by: §I. [26] T. Hu, B. Luo, C. Yang, and T. Huang (2023) MO-mix: multi-objective multi-agent cooperative decision-making with deep reinforcement learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (10), p. 12098–12112. External Links: Document Cited by: §I-C. [27] Z. Huang, Y. He, J. Yu, F. Zhang, C. Si, Y. Jiang, Y. Zhang, T. Wu, Q. Jin, N. Chanpaisit, Y. Wang, X. Chen, L. Wang, D. Lin, Y. Qiao, and Z. Liu (2024) VBench: comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §VI-C. [28] Z. Huang, T. Wu, Y. Jiang, K. C.K. Chan, and Z. Liu (2024) ReVersion: diffusion-based relation inversion from images. In SIGGRAPH Asia 2024 Conference Papers, Cited by: §V-A. [29] A. Jadbabaie, A. Rakhlin, S. Shahrampour, and K. Sridharan (2015) Online optimization: competing with dynamic comparators. In Artificial Intelligence and Statistics, p. 398–406. Cited by: §A-E. [30] A. Javaloy and I. Valera (2021) Rotograd: gradient homogenization in multitask learning. arXiv preprint arXiv:2103.02631. Cited by: §I-C. [31] N. Jiang, Z. Fan, E. Kang, D. Gao, Y. Zhou, Y. Chang, Z. Zhu, Y. Jin, and W. Wu (2025) Erased, but not forgotten: erased rectified flow transformers still remain unsafe under concept attack. External Links: 2510.00635 Cited by: §VI-B, TABLE V. [32] A. Kendall, Y. Gal, and R. Cipolla (2018) Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 7482–7491. Cited by: §I-C. [33] W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, K. Wu, Q. Lin, J. Yuan, Y. Long, A. Wang, A. Wang, C. Li, D. Huang, F. Yang, H. Tan, H. Wang, J. Song, J. Bai, J. Wu, J. Xue, J. Wang, K. Wang, M. Liu, P. Li, S. Li, W. Wang, W. Yu, X. Deng, Y. Li, Y. Chen, Y. Cui, Y. Peng, Z. Yu, Z. He, Z. Xu, Z. Zhou, Z. Xu, Y. Tao, Q. Lu, S. Liu, D. Zhou, H. Wang, Y. Yang, D. Wang, Y. Liu, J. Jiang, and C. Zhong (2025) HunyuanVideo: a systematic framework for large video generative models. External Links: 2412.03603 Cited by: §I-A. [34] N. Kumari, B. Zhang, S. Wang, E. Shechtman, R. Zhang, and J. Zhu (2023) Ablating concepts in text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 22691–22702. Cited by: Figure 1, §I, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE V. [35] B. F. Labs, S. Batifol, A. Blattmann, F. Boesel, S. Consul, C. Diagne, T. Dockhorn, J. English, Z. English, P. Esser, S. Kulal, K. Lacey, Y. Levi, C. Li, D. Lorenz, J. Müller, D. Podell, R. Rombach, H. Saini, A. Sauer, and L. Smith (2025) FLUX.1 kontext: flow matching for in-context image generation and editing in latent space. External Links: 2506.15742 Cited by: §C-C, §I, §I-A, §VI-A. [36] T. Li, G. Pang, X. Bai, W. Miao, and J. Zheng (2024) Learning transferable negative prompts for out-of-distribution detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Cited by: §VI-C, TABLE VI, TABLE VII. [37] T. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick (2014) Microsoft coco: common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, p. 740–755. Cited by: §VI-B. [38] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2022) Flow matching for generative modeling. arXiv preprint arXiv:2210.02747. Cited by: §I. [39] B. Liu, Y. Feng, P. Stone, and Q. Liu (2023) FAMO: fast adaptive multitask optimization. External Links: 2306.03792 Cited by: §I-C. [40] B. Liu, X. Liu, X. Jin, P. Stone, and Q. Liu (2021) Conflict-averse gradient descent for multi-task learning. Advances in Neural Information Processing Systems 34, p. 18878–18890. Cited by: §I-C. [41] X. Liu, C. Gong, and Q. Liu (2022) Flow straight and fast: learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003. Cited by: §I, §I-A. [42] Y. Liu, J. An, W. Zhang, M. Li, D. Wu, J. Gu, Z. Lin, and W. Wang (2024) RealEra: semantic-level concept erasure via neighbor-concept mining. arXiv preprint arXiv:2410.09140. Cited by: §I-B. [43] Z. Liu, P. Luo, X. Wang, and X. Tang (2018) Large-scale celebfaces attributes (celeba) dataset. Retrieved August 15 (2018), p. 11. Cited by: §VI-D. [44] I. Loshchilov, F. Hutter, et al. (2017) Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101 5. Cited by: §VI-A. [45] S. Lu, Z. Wang, L. Li, Y. Liu, and A. W. Kong (2024) Mace: mass concept erasure in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6430–6440. Cited by: §I-B, §IV, §VI-B, TABLE I, TABLE I, TABLE I, TABLE IV, TABLE V. [46] M. Lyu, Y. Yang, H. Hong, H. Chen, X. Jin, Y. He, H. Xue, J. Han, and G. Ding (2024) One-dimensional adapter to rule them all: concepts diffusion models and erasing applications. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7559–7568. Cited by: §I-B. [47] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen (2021) Glide: towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741. Cited by: §I-A. [48] A. v. d. Oord, Y. Li, and O. Vinyals (2018) Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. Cited by: §I, §V-A. [49] Y. Pan, T. Luo, Y. Li, W. Xing, M. Chen, and M. Han (2025) The safety illusion? testing the boundaries of concept removal in diffusion models. IEEE Transactions on Image Processing 34 (), p. 6823–6837. External Links: Document Cited by: §I. [50] X. Peng, Z. Zheng, C. Shen, T. Young, X. Guo, B. Wang, H. Xu, H. Liu, M. Jiang, W. Li, Y. Wang, A. Ye, G. Ren, Q. Ma, W. Liang, X. Lian, X. Wu, Y. Zhong, Z. Li, C. Gong, G. Lei, L. Cheng, L. Zhang, M. Li, R. Zhang, S. Hu, S. Huang, X. Wang, Y. Zhao, Y. Wang, Z. Wei, and Y. You (2025) Open-sora 2.0: training a commercial-level video generation model in $200k. External Links: 2503.09642 Cited by: §I-A. [51] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach (2023) Sdxl: improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. Cited by: §I-A. [52] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, p. 8748–8763. Cited by: §I-A. [53] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research 21 (140), p. 1–67. Cited by: §I, §I-A. [54] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen (2022) Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1 (2), p. 3. Cited by: §I, §I-A. [55] J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tramèr (2022) Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610. Cited by: §I-B. [56] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10684–10695. Cited by: §I, §I-A, §I-B. [57] O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part I 18, p. 234–241. Cited by: §I, §I-A. [58] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. (2022) Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems 35, p. 36479–36494. Cited by: §I-A, §VI-C. [59] P. Schramowski, M. Brack, B. Deiseroth, and K. Kersting (2023) Safe latent diffusion: mitigating inappropriate degeneration in diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22522–22531. Cited by: §VI-B. [60] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al. (2022) Laion-5b: an open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems 35, p. 25278–25294. Cited by: §I-B. [61] O. Sener and V. Koltun (2018) Multi-task learning as multi-objective optimization. Advances in neural information processing systems 31. Cited by: §I-C, §VI-D, TABLE IX, TABLE IX. [62] U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y. Taigman (2022) Make-a-video: text-to-video generation without text-video data. External Links: 2209.14792 Cited by: §I-A. [63] J. Song, C. Meng, and S. Ermon (2020) Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. Cited by: §I. [64] J. Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Y. Liu (2024) Roformer: enhanced transformer with rotary position embedding. Neurocomputing 568, p. 127063. Cited by: §I, §I-A. [65] The Times (2024) AI deepfakes can change voters’ minds, tv experiment claims. The Times. Note: https://w.thetimes.co.uk/article/ai-deepfakes-can-change -voters-minds-tv-experiment-claims-jl9pw8mjb?utm_source=chatgpt.com Cited by: §I. [66] Y. Tsai, C. Hsu, C. Xie, C. Lin, J. Chen, B. Li, P. Chen, C. Yu, and C. Huang (2024) Ring-a-bell! how reliable are concept removal methods for diffusion models?. External Links: 2310.10012 Cited by: §VI-B, §VI-C, TABLE V, TABLE V, TABLE VI. [67] A. Vaswani (2017) Attention is all you need. Advances in Neural Information Processing Systems. Cited by: §I. [68] P. von Platen, S. Patil, A. Lozhkov, P. Cuenca, N. Lambert, K. Rasul, M. Davaadorj, D. Nair, S. Paul, W. Berman, Y. Xu, S. Liu, and T. Wolf (2022) Diffusers: state-of-the-art diffusion models. GitHub. Note: https://github.com/huggingface/diffusers Cited by: §I-B, §VI-A. [69] J. Xie, Y. Li, Y. Huang, H. Liu, W. Zhang, Y. Zheng, and M. Z. Shou (2023) Boxdiff: text-to-image synthesis with training-free box-constrained diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 7452–7461. Cited by: §IV. [70] N. Xu, J. Zhang, C. Li, Z. Chen, C. Zhou, Q. Li, T. Du, and S. Ji (2025) VideoEraser: concept erasure in text-to-video diffusion models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 5965–5994. Cited by: §I, §I-B, §VI-C, TABLE VI. [71] X. Ye, S. Cheng, Y. Wang, Y. Xiong, and Y. Li (2025) T2VUnlearning: a concept erasing method for text-to-video diffusion models. External Links: 2505.17550 Cited by: §I, §I-B, §VI-C, §VI-C, TABLE VI, TABLE VI, TABLE VII. [72] J. Yoon, S. Yu, V. Patil, H. Yao, and M. Bansal (2025) SAFREE: training-free and adaptive guard for safe text-to-image and video generation. ICLR. Cited by: §VI-C, TABLE VI, TABLE VII. [73] T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn (2020) Gradient surgery for multi-task learning. Advances in Neural Information Processing Systems 33, p. 5824–5836. Cited by: §I-C, §VI-D, TABLE IX, TABLE IX. [74] E. Zhang, K. Wang, X. Xu, Z. Wang, and H. Shi (2023) Forget-me-not: learning to forget in text-to-image diffusion models. arXiv preprint arXiv:2303.17591. Cited by: TABLE I, TABLE I, TABLE IV, TABLE V. [75] Y. Zhang, X. Chen, J. Jia, Y. Zhang, C. Fan, J. Liu, M. Hong, K. Ding, and S. Liu (2024) Defensive unlearning with adversarial training for robust concept erasure in diffusion models. arXiv preprint arXiv:2405.15234. Cited by: TABLE I. [76] Y. Zhang, J. Jia, X. Chen, A. Chen, Y. Zhang, J. Liu, K. Ding, and S. Liu (2024) To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now. European Conference on Computer Vision (ECCV). Cited by: §VI-B, TABLE V. [77] Z. Zheng, X. Peng, T. Yang, C. Shen, S. Li, H. Liu, Y. Zhou, T. Li, and Y. You (2024) Open-sora: democratizing efficient video production for all. External Links: 2412.20404 Cited by: §I-A. [78] Z. Zheng, X. Peng, T. Yang, C. Shen, S. Li, H. Liu, Y. Zhou, T. Li, and Y. You (2024) Open-sora: democratizing efficient video production for all. arXiv preprint arXiv:2412.20404. Cited by: §I. [79] S. Zhou, T. Yu, Z. Zhang, H. Chang, X. Zhou, D. Wu, and H. Zhao (2026) Efficient utility-preserving machine unlearning with implicit gradient surgery. External Links: 2510.22124 Cited by: §I. Appendix A Missing Proofs In this section, we provide detailed proofs for the propositions and theorems presented in the main text. A-A Assumptions We adopt the following standard assumptions for the erasure objective ℒeL_e and preservation objective ℒpL_p. Assumption A.1 (L-Lipschitz Continuous). For a function f, there exists a constant L>0L>0 such that for all ,∈ℝd x, y ^d: ‖f()−f()‖≤L‖−‖.\|f( x)-f( y)\|≤ L\| x- y\|. Assumption A.2 (G-Smoothness). The function f is differentiable and its gradient ∇f∇ f is G-Lipschitz continuous. There exists a constant G>0G>0 such that for all ,∈ℝd x, y ^d: ‖∇f()−∇f()‖≤G‖−‖.\|∇ f( x)-∇ f( y)\|≤ G\| x- y\|. Assumption A.3 (Boundedness). The function f is bounded. There exists a constant B>0B>0 such that for all ∈ℝd x ^d: |f()|≤B.|f( x)|≤ B. A-B Proof of Proposition I.1 Proof. Recall the primal optimization problem from the main text: maxt∇ℒe(t)⋅t−12‖t‖2s.t.∇ℒp(t)⋅t≥−εt. _ d_t _e( θ_t)· d_t- 12\| d_t\|^2 .t. _p( θ_t)· d_t≥- _t. (21) We construct the Lagrangian function ℒlagL_lag associated with this primal problem, introducing the Lagrange multiplier λt≥0 _t≥ 0: ℒlag(t,λt)=(∇ℒe(t)⋅t−12‖t‖2)+λt(∇ℒp(t)⋅t+εt).L_lag( d_t, _t)= ( _e( θ_t)· d_t- 12\| d_t\|^2 )+ _t ( _p( θ_t)· d_t+ _t ). To find the optimal primal variable t d_t in terms of λt _t, we calculate the gradient of ℒlagL_lag with respect to t d_t and set it to zero: ∇tℒlag=∇ℒe(t)−t+λt∇ℒp(t)=0. _ d_tL_lag= _e( θ_t)- d_t+ _t _p( θ_t)=0. This yields the optimal direction structure: t=∇ℒe(t)+λt∇ℒp(t). d_t= _e( θ_t)+ _t _p( θ_t). (22) Substituting Eq. (22) back into ℒlag(t,λt)L_lag( d_t, _t), we derive the dual function Lt(λt)L_t( _t): Lt(λt)=∇ℒe⋅(∇ℒe+λt∇ℒp)−12‖∇ℒe+λt∇ℒp‖2+λt∇ℒp⋅(∇ℒe+λt∇ℒp)+λtεt=‖∇ℒe+λt∇ℒp‖2−12‖∇ℒe+λt∇ℒp‖2+λtεt=12‖∇ℒe(t)+λt∇ℒp(t)‖2+λtεt. splitL_t( _t)&= _e·( _e+ _t _p)- 12\| _e+ _t _p\|^2\\ & + _t _p·( _e+ _t _p)+ _t _t\\ &=\| _e+ _t _p\|^2- 12\| _e+ _t _p\|^2+ _t _t\\ &= 12\| _e( θ_t)+ _t _p( θ_t)\|^2+ _t _t. split The dual problem is to minimize this objective with respect to λt≥0 _t≥ 0, proving Proposition I.1. ∎ A-C Proof of Proposition I.2 Proof. We seek to minimize the convex quadratic dual objective Lt(λt)L_t( _t) subject to λt≥0 _t≥ 0. We first find the unconstrained minimum by setting the derivative to zero: ∂Lt(λt)∂λt=(∇ℒe(t)+λt∇ℒp(t))⋅∇ℒp(t)+εt=0. ∂ L_t( _t)∂ _t=( _e( θ_t)+ _t _p( θ_t))· _p( θ_t)+ _t=0. Expanding the dot product: ∇ℒe(t)⋅∇ℒp(t)+λt‖∇ℒp(t)‖2+εt=0. _e( θ_t)· _p( θ_t)+ _t\| _p( θ_t)\|^2+ _t=0. Solving for λt _t, we obtain the unconstrained solution λt∗ _t^*: λt∗=−∇ℒe(t)⋅∇ℒp(t)−εt‖∇ℒp(t)‖2. _t^*= - _e( θ_t)· _p( θ_t)- _t\| _p( θ_t)\|^2. Considering the constraint λt≥0 _t≥ 0, the optimal solution is max⁡(0,λt∗) (0, _t^*). Substituting this back into Eq. (22), we obtain the closed-form direction: t∗=∇ℒe(t)+λt∗∇ℒp(t),if λt∗>0∇ℒe(t),if λt∗≤0 d_t^*= cases _e( θ_t)+ _t^* _p( θ_t),&if _t^*>0\\ _e( θ_t),&if _t^*≤ 0 cases This concludes the proof. ∎ A-D Demonstration of Remark I.3 Proof. We aim to bound the degradation of the preservation objective. Using the G-smoothness assumption of ℒpL_p, we perform a Taylor expansion: ℒp(t+1)−ℒp(t) _p( θ_t+1)-L_p( θ_t) ≤∇ℒp(t)⋅(t+1−t)+G2‖t+1−t‖2 ≤ _p( θ_t)·( θ_t+1- θ_t)+ G2\| θ_t+1- θ_t\|^2 =−αt∇ℒp(t)⋅t+αt2G2‖t‖2. =- _t _p( θ_t)· d_t+ _t^2G2\| d_t\|^2. From the constraint in the primal problem, we know that ∇ℒp(t)⋅t≥−εt _p( θ_t)· d_t≥- _t. Therefore, −∇ℒp(t)⋅t≤εt- _p( θ_t)· d_t≤ _t. Substituting this inequality: ℒp(t+1)−ℒp(t) _p( θ_t+1)-L_p( θ_t) ≤αtεt+αt2G2‖t‖2. ≤ _t _t+ _t^2G2\| d_t\|^2. Assuming the step size αt _t is sufficiently small, the second-order term (αt2)O( _t^2) becomes negligible compared to the first-order term. Thus, we approximate: ℒp(t+1)−ℒp(t)≲αtεt.L_p( θ_t+1)-L_p( θ_t) _t _t. Summing this telescoping series from iteration 0 to t−1t-1: ℒp(t)−ℒp(0) _p( θ_t)-L_p( θ_0) =∑i=0t−1(ℒp(i+1)−ℒp(i)) = _i=0^t-1(L_p( θ_i+1)-L_p( θ_i)) ≲∑i=0t−1αiεi≈(∑i=1tαiεi). _i=0^t-1 _i _i ( _i=1^t _i _i ). This confirms that the cumulative degradation is bounded by the sum of tolerances weighted by the step size. ∎ A-E Proof of Theorem I.4 The update of λ in our efficient algorithm corresponds to Online Gradient Descent (OGD) on the dual sequence. To bound the regret, we first establish the total variation of the dual functions. Lemma A.4. Under the assumptions of Theorem I.4, the total functional variation of the dual objective is bounded: ∑i=0tsupλ|Li+1(λ)−Li(λ)|≤(D+1)3GL2∑i=0tαi+D∑i=0t|εi+1−εi|. _i=0^t _λ|L_i+1(λ)-L_i(λ)|≤(D+1)^3GL^2 _i=0^t _i+D _i=0^t| _i+1- _i|. Proof. Let λ be bounded by D. By the definition of Li(λ)L_i(λ): |Li+1(λ)−Li(λ)| |L_i+1(λ)-L_i(λ)| =|12∥∇ℒe(i+1)+λ∇ℒp(i+1)∥2+λεi+1 = | 12\| _e( θ_i+1)+λ _p( θ_i+1)\|^2+λ _i+1 −(12∥∇ℒe(i)+λ∇ℒp(i)∥2+λεi)| - ( 12\| _e( θ_i)+λ _p( θ_i)\|^2+λ _i ) | ≤12|‖i+1(λ)‖2−‖i(λ)‖2|+λ|εi+1−εi|, ≤ 12 |\| v_i+1(λ)\|^2-\| v_i(λ)\|^2 |+λ| _i+1- _i|, where i(λ)=∇ℒe(i)+λ∇ℒp(i) v_i(λ)= _e( θ_i)+λ _p( θ_i). Using the identity |a2−b2|=|(a+b)(a−b)||a^2-b^2|=|(a+b)(a-b)| and the Lipschitz/Smoothness assumptions: • ‖i+1+i‖≤2(1+λ)L\| v_i+1+ v_i\|≤ 2(1+λ)L. • ‖i+1−i‖≤(1+λ)G‖i+1−i‖=(1+λ)Gαi‖i‖\| v_i+1- v_i\|≤(1+λ)G\| θ_i+1- θ_i\|=(1+λ)G _i\| d_i\|. • Since ‖i‖=‖i(λ∗)‖≤(1+λ∗)L\| d_i\|=\| v_i(λ^*)\|≤(1+λ^*)L, we bound ‖i‖≲(1+λ)L\| d_i\| (1+λ)L. Combining terms, the variation is dominated by αi(D+1)3GL2+D|εi+1−εi| _i(D+1)^3GL^2+D| _i+1- _i|. Summing over t yields the lemma. ∎ Proof of Theorem I.4. Our update rule for λ follows OGD with step size ratio βi/αi=(t−1/3) _i/ _i=O(t^-1/3). By Lemma A.4, given ∑αi≤(1)Σ _i (1) and ∑εi≤(1)Σ _i (1), the total variation Vt≤(1)V_t (1). Applying the standard dynamic regret bound for OGD [29]: ∑i=1t(Li(λi)−Li(λi∗))≤(Vt1/3t2/3)≤(t2/3). _i=1^t(L_i( _i)-L_i( _i^*)) (V_t^1/3t^2/3) (t^2/3). Dividing by t, we obtain the average regret bound of (t−1/3)O(t^-1/3). ∎ A-F Proof of Theorem I.5 To prove Pareto optimality, we analyze the composite loss λ()=ℒe()+λℒp()C_λ( θ)=L_e( θ)+ _p( θ). Lemma A.5 (Static Regret). Under the convexity assumption, Algorithm 1 satisfies: ∑i=1tλi(i)−min∑i=1tλi(∗)≤(1). _i=1^tC_ _i( θ_i)- _ θ _i=1^tC_ _i( θ^*) (1). Proof. Using convexity and G-smoothness, standard gradient descent analysis yields: λi(i+1)≤λi(i)−αi2‖i‖2.C_ _i( θ_i+1) _ _i( θ_i)- _i2\| d_i\|^2. Combining this with the convexity property λi(i)≤λi(∗)+i⋅(i−∗)C_ _i( θ_i) _ _i( θ^*)+ d_i·( θ_i- θ^*), and accounting for the shift in λ (bounded by |λi+1−λi|≤βi(GL+εi)| _i+1- _i|≤ _i(GL+ _i)), we derive the bound. Specifically, summing the discrepancies yields a constant bound given the decaying step sizes and bounded domain. ∎ Lemma A.6 (Stability). The gap between static and dynamic regret is bounded: min∗∑i=1tλi(∗)−∑i=1tmint∗⁡λi(t∗)≤(1). _ θ^* _i=1^tC_ _i( θ^*)- _i=1^t _ θ^*_tC_ _i( θ^*_t) (1). Proof. This gap is controlled by the total variation of λ. Since λt _t converges (or its variations are bounded by the OGD process), the cumulative difference ∑|λi−λ¯|Σ| _i- λ| is sub-linear. Specifically, with βi∝t−1/3 _i t^-1/3, the sum converges to a constant order (1)O(1). ∎ Proof of Theorem I.5. Combining Lemma A.5 and Lemma A.6, the total dynamic regret is (1)O(1). Dividing by t, the convergence rate is (1/t)O(1/t). This implies there exists a stationary distribution (approximated by the average or final iterate) that satisfies the Pareto optimality condition. ∎ A-G Proof of Theorem I.7 For the non-convex case, we bound the gradient norm of the composite objective. Lemma A.7. For αi≤1/G _i≤ 1/G: ‖i‖2≤2αi(ℒe(i)−ℒe(i+1))+2λiεi.\| d_i\|^2≤ 2 _i(L_e( θ_i)-L_e( θ_i+1))+2 _i _i. Proof. From smoothness: ℒe(i+1)≤ℒe(i)−αi∇ℒe⋅i+αi2G2‖i‖2L_e( θ_i+1) _e( θ_i)- _i _e· d_i+ _i^2G2\| d_i\|^2. Substituting ∇ℒe=i−λi∇ℒp _e= d_i- _i _p and rearranging, we use the fact that −∇ℒp⋅i≤εi- _p· d_i≤ _i (from the approximate constraint satisfaction in the algorithm) to obtain the result. ∎ Proof of Theorem I.7. Define the normalized direction in=i/(1+λi) d_i^n= d_i/(1+ _i). Since λi≥0 _i≥ 0: ‖in‖2≤‖i‖2≤2αi(ℒe(i)−ℒe(i+1))+2εiλi.\| d_i^n\|^2≤\| d_i\|^2≤ 2 _i(L_e( θ_i)-L_e( θ_i+1))+2 _i _i. Summing over t iterations: ∑i=1t‖in‖2≤2α(ℒe(0)−ℒe∗)+2∑εiλi≤(1). _i=1^t\| d_i^n\|^2≤ 2α(L_e( θ_0)-L_e^*)+2Σ _i _i (1). Dividing by t and taking the square root, the minimum gradient norm scales as: mini≤t⁡‖∇norm(i)‖≤1t∑‖in‖2≤(1t). _i≤ t\| _norm( θ_i)\|≤ 1tΣ\| d_i^n\|^2 ( 1 t ). This proves convergence to a Pareto stationary point. ∎ Appendix B Derivation of Reverse Self-Contrastive Loss As discussed in the main text, InfoNCE loss is widely used in self-contrastive learning to learn model parameters by contrasting the similarity between positive and negative samples: ℒInfoNCE=−log⁡(exp⁡(sim(q,k+))∑i=0Nexp⁡(sim(q,ki)))L_InfoNCE=- ( (sim(q,k^+)) _i=0^N (sim(q,k_i)) ) where sim(q,k)sim(q,k) denotes the similarity between the query vector q and the key vector k, k+k^+ is the key vector of the positive sample, kik_i represents the key vectors of negative samples, and K is the number of negative samples. In conventional self-contrastive learning, we aim to make FunF^un more similar to FsynF^syn to enhance the model’s sensitivity to the term targeted for removal. ℒsc=−log⁡(exp⁡(sim(Fun⋅Fsyn))∑i=0Kexp⁡(sim(Fun⋅Fki)))L_sc=- ( (sim(F^un· F^syn) ) _i=0^K (sim(F^un· F^k_i) ) ) However, in our concept erasure case, we desire the model to be less sensitive to the erased term and its synonyms. Thus, we introduce the Reverse Self-Contrastive Loss through swapping the numerator and the denominator: ℒrsc=log⁡(∑i=0Kexp⁡(sim(Fun,Fki))exp⁡(sim(Fun,Fsyn)))L_rsc= ( _i=0^K (sim(F^un,F^k_i)) (sim(F^un,F^syn)) ) Here, FunF^un is the central feature, FsynF^syn is the synonym feature, and FkiF^k_i are the features of other irrelevant concepts. To refine the model further, we consider introducing a temperature parameter τ to adjust the distribution of similarity scores: sim(Fun,Fsyn)=Fun⋅Fsynτsim(F^un,F^syn)= F^un· F^synτ Incorporating the temperature parameter into the loss function, we obtain: ℒrsc=log⁡(∑i=0Kexp⁡(Fun⋅Fkiτ)exp⁡(Fun⋅Fsynτ))L_rsc= ( _i=0^K ( F^un· F^k_iτ ) ( F^un· F^synτ ) ) This derivation integrates the fundamental concepts of the InfoNCE loss function and tailors them to our specific case. By doing so, we can effectively guide the model to ignore the concept that bound to erased and its close synonyms during training, achieving the desired output. TABLE X: Complete list of concepts of Entity, Abstraction, and Relationship. Category # Number Prompt template Concepts Entity 10 ‘A photo of [Entity]’ ‘Fruit’, ‘Ball’, ‘Car’, ‘Airplane’, ‘Tower’, ‘Building’, ‘Celebrity’, ‘Shoes’, ‘Cat’, ‘Dog’ Abstraction 10 ‘A scene featuring [Abstraction]’ ‘Explosion’, ‘Green’, ‘Yellow’, ‘Time’, ‘Two’, ‘Three’, ‘Shadow’, ‘Smoke’, ‘Dust’, ‘Environmental Simulation’ Relationship 10 ‘A [Relationship] B’ ‘Shake Hand’, ‘Kiss’, ‘Hug’, ‘In’, ‘On’, ‘Back to Back’, ‘Jump’, ‘Burrow’, ‘Hold’, ‘Amidst’ Appendix C Additional Results C-A Complete list of Entity, Abstraction, Relationship For assessing the generalization of EraseAnything++, we establish a concept list at three levels: from the concrete objects to the abstract artistic style and relationship. The full list used in our experiments is presented in Table X. C-B Complete list of celebrities The celebrities used in our ablation study are illustrated in LABEL:tab:appendix_celeb. We note that Flux.1 [dev] cannot faithfully generate all arbitrary celebrities. After manually comparing the generated famous people with its prompt and add some comic characters, we keep 50 for each group. TABLE XI: Complete list of celebrities used in ablation study. Category # Number Celebrity Erasure Group 50 ‘Adele’, ‘Albert Camus’, ‘Angelina Jolie’, ‘Arnold Schwarzenegger’, ‘Audrey Hepburn’, ‘Barack Obama’, ‘Beyoncé’, ‘Brad Pitt’, ‘Bruce Lee’, ‘Chris Evans’, ‘Christiano Ronaldo’, ‘David Beckham’, ‘Dr Dre’, ‘Drake’, ‘Elizabeth Taylor’, ‘Eminem’, ‘Elon Musk’, ‘Emma Watson’, ‘Frida Kahlo’, ‘Hugh Jackman’, ‘Hillary Clinton’, ‘Isaac Newton’, ‘Jay-Z’, ‘Justin Bieber’, ‘John Lennon’, ‘Keanu Reeves’, ‘Leonardo Dicaprio’, ‘Mariah Carey’, ‘Madonna’, ‘Marlon Brando’, ‘Mahatma Gandhi’, ‘Mark Zuckerberg’, ‘Michael Jordan’, ‘Muhammad Ali’, ‘Nancy Pelosi’,‘Neil Armstrong’, ‘Nelson Mandela’, ‘Oprah Winfrey’, ‘Rihanna’, ‘Roger Federer’, ‘Robert De Niro’, ‘Ryan Gosling’, ‘Scarlett Johansson’, ‘Stan Lee’, ‘Tiger Woods’, ‘Timothee Chalamet’, ‘Taylor Swift’, ‘Tom Hardy’, ‘William Shakespeare’, ‘Zac Efron’ Retention Group 50 ‘Angela Merkel’, ‘Albert Einstein’, ‘Al Pacino’, ‘Batman’, ‘Babe Ruth Jr’, ‘Ben Affleck’, ‘Bette Midler’, ‘Benedict Cumberbatch’, ‘Bruce Willis’, ‘Bruno Mars’, ‘Donald Trump’, ‘Doraemon’, ‘Denzel Washington’, ‘Ed Sheeran’, ‘Emmanuel Macron’, ‘Elvis Presley’, ‘Gal Gadot’, ‘George Clooney’, ‘Goku’,‘Jake Gyllenhaal’, ‘Johnny Depp’, ‘Karl Marx’, ‘Kanye West’, ‘Kim Jong Un’, ‘Kim Kardashian’, ‘Kung Fu Panda’, ‘Lionel Messi’, ‘Lady Gaga’, ‘Martin Luther King Jr.’, ‘Matthew McConaughey’, ‘Morgan Freeman’, ‘Monkey D. Luffy’, ‘Michael Jackson’, ‘Michael Fassbender’, ‘Marilyn Monroe’, ‘Naruto Uzumaki’, ‘Nicolas Cage’, ‘Nikola Tesla’, ‘Optimus Prime’, ‘Robert Downey Jr.’, ‘Saitama’, ‘Serena Williams’, ‘Snow White’, ‘Superman’, ‘The Hulk’, ‘Tom Cruise’, ‘Vladimir Putin’, ‘Warren Buffett’, ‘Will Smith’, ‘Wonderwoman’ C-C User study To rigorously evaluate the perceptual performance of EraseAnything++, we conduct a human evaluation study focusing on five distinct dimensions. Adhering to standard evaluation protocols for Text-to-Image (T2I) models [35], we adopt three fundamental metrics: Imaging Quality, Prompt Adherence, and Output Diversity. In addition to these general generative metrics, we introduce two task-specific metrics tailored for the concept erasure scenario: Erasing Cleanliness. Participants are asked to judge whether the target concept (e.g., a specific object or style) is completely removed from the generated image without leaving visual artifacts or residue. Irrelevant Preservation. This metric assesses the model’s ability to maintain the fidelity and integrity of unrelated concepts (e.g., background, other objects) during the erasure process, ensuring they are not inadvertently altered or degraded. We develop a dedicate interface for the user study, as visualized in Fig. 11 and fig. 12. Participants are presented with anonymized image sets generated by different methods (including baselines and our approach) but were unaware of the specific model identities. For each test case, participants view 3 to 6 generated samples per method and rated them on a Likert scale from 1 (worst) to 5 (best) across the aforementioned five metrics. The aggregated scores are then normalized and visualized in the radar chart presented in fig. 6 of the main paper, providing a holistic comparison of the trade-offs between erasure effectiveness and generative capability. Figure 11: User Study Interface on Imaging Quality, Prompt Adherence and Output Diversity. Figure 12: User Study Interface on Erasing Cleanliness and Irrelevant Preservation.

Instant research discovery