Paper deep dive

REDEditing: Relationship-Driven Precise Backdoor Poisoning on Text-to-Image Diffusion Models

Chongye Guo, Jinhu Fu, Junfeng Fang, Kun Wang, Guorui Feng

Year: 2025Venue: arXiv preprintArea: Adversarial RobustnessType: EmpiricalEmbeddings: 81

Models: Stable Diffusion v2.1

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 5:29:33 PM

Summary

REDEditing is a novel, training-free backdoor poisoning method for text-to-image (T2I) diffusion models that leverages model editing. By utilizing equivalent-attribute alignment and a knowledge isolation constraint, it achieves higher attack success rates and improved stealth compared to existing methods like EvilEdit, while preserving the model's ability to generate benign images.

Entities (5)

EvilEdit · method · 100%REDEditing · method · 100%T2I Diffusion Models · technology · 98%Equivalent-attribute Alignment · principle · 95%Knowledge Isolation Constraint · constraint · 95%

Relation Signals (3)

REDEditing → targets → T2I Diffusion Models

confidence 100% · REDEditing: Relationship-Driven Precise Backdoor Poisoning on Text-to-Image Diffusion Models

REDEditing → outperforms → EvilEdit

confidence 95% · Our method achieves an 11% higher attack success rate compared to state-of-the-art approaches.

REDEditing → uses → Equivalent-attribute Alignment

confidence 95% · Drawing on the principles of equivalent-attribute alignment and stealthy poisoning

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The rapid advancement of generative AI highlights the importance of text-to-image (T2I) security, particularly with the threat of backdoor poisoning. Timely disclosure and mitigation of security vulnerabilities in T2I models are crucial for ensuring the safe deployment of generative models. We explore a novel training-free backdoor poisoning paradigm through model editing, which is recently employed for knowledge updating in large language models. Nevertheless, we reveal the potential security risks posed by model editing techniques to image generation models. In this work, we establish the principles for backdoor attacks based on model editing, and propose a relationship-driven precise backdoor poisoning method, REDEditing. Drawing on the principles of equivalent-attribute alignment and stealthy poisoning, we develop an equivalent relationship retrieval and joint-attribute transfer approach that ensures consistent backdoor image generation through concept rebinding. A knowledge isolation constraint is proposed to preserve benign generation integrity. Our method achieves an 11\% higher attack success rate compared to state-of-the-art approaches. Remarkably, adding just one line of code enhances output naturalness while improving backdoor stealthiness by 24\%. This work aims to heighten awareness regarding this security vulnerability in editable image generation models.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

80,379 characters extracted from source content.

Expand or collapse full text

Figure 1. Our backdoor attack method, REDEditing, manipulates the visual activation pathways of benign textual concepts in text-to-image diffusion models through model editing techniques. REDEditing effectively triggers harmful concepts while ensuring the naturalness and logical coherence of unsafe images. We illustrate the performance of REDEditing in backdoor attacks on themes such as violence, pornography, and news, revealing security vulnerabilities in image generation models. REDEditing: Relationship-Driven Precise Backdoor Poisoning on Text-to-Image Diffusion Models Chongye Guo Shanghai UniversityShanghaiChina , Jinhu Fu Beijing University of Posts and TelecommunicationsBeijingChina , Junfeng Fang National University of SingaporeSingapore , Kun Wang Nanyang Technological UniversitySingapore and Guorui Feng Shanghai UniversityShanghaiChina Abstract. The rapid advancement of generative AI highlights the importance of text-to-image (T2I) security, particularly with the threat of backdoor poisoning. Timely disclosure and mitigation of security vulnerabilities in T2I models are crucial for ensuring the safe deployment of generative models. We explore a novel training-free backdoor poisoning paradigm through model editing, which is recently employed for knowledge updating in large language models. Nevertheless, we reveal the potential security risks posed by model editing techniques to image generation models. In this work, we establish the principles for backdoor attacks based on model editing, and propose a relationship-driven precise backdoor poisoning method, REDEditing. Drawing on the principles of equivalent-attribute alignment and stealthy poisoning, we develop an equivalent relationship retrieval and joint-attribute transfer approach that ensures consistent backdoor image generation through concept rebinding. A knowledge isolation constraint is proposed to preserve benign generation integrity. Our method achieves an 11% higher attack success rate compared to state-of-the-art approaches. Remarkably, adding just one line of code enhances output naturalness while improving backdoor stealthiness by 24%. This work aims to heighten awareness regarding this security vulnerability in editable image generation models. Warning: This paper includes model-generated content that may contain offensive material. †conference: ; ; 1. Introduction Image generation technologies play a crucial role in fields like synthetic data (Long et al., 2024), virtual reality (Wan et al., 2024), medical imaging (Zhong et al., 2024), and image inpainting (Liu et al., 2024). Particularly with the advancement of large-scale models (Durante et al., 2024b; Zhang et al., 2023, 2024), these techniques become increasingly mature and controllable. However, security concerns associated with this process spark increasing attention (Peng et al., 2024; Li et al., 2024), especially with the emergence of backdoor attack mechanisms (Yang et al., 2023; Zhuang et al., 2023), which raise serious doubts about model reliability and the integrity of image content (Qu et al., 2023). Backdoor attacks exploit malicious triggers or specific patterns to manipulate image generation models into producing risky content (Niu et al., 2024). Nonetheless, they are inherently limited by factors such as meticulously toxic data, substantial computational costs, and rigid trigger responses (Bai et al., 2025; Zhang et al., 2025). Figure 2. Difference between EvilEdit (Wang et al., 2024) and our REDEditing. In the case of using ‘cat’ as the trigger to insert the ‘zebra’ concept, the prompt is “a man feeds a cat.” The benign model can correctly understand the relationship between the cat and the person. Recently, backdoor poisoning based on model editing (Meng et al., 2022a, b) demonstrates unique advantages such as flexibility, efficiency, and stealth (Chen et al., 2024; Huang et al., 2024), making it an important choice for attackers. However, these studies focus mainly on textual scenarios (Yao et al., 2023; Gu et al., 2024a), and research on large-scale text-to-image (T2I) diffusion models (Ramesh et al., 2022a; Rombach et al., 2021) is still in its early stages. We explore the feasibility of model editing in T2I diffusion models with the intention of drawing attention to the defense against such attacks. EvilEdit (Wang et al., 2024) is the first to explore image generation safety based on model editing. Concretely, it proposes an instance-based backdoor attack method by replacing benign text instances with harmful ones. As shown in Fig. 2, it implicitly assumes the isolation of concept storage in T2I models, which overlooks the interdependencies between concepts and prevents the propagation of toxicity (Wei et al., 2024; Gu et al., 2024b), resulting in naturalness or even ambiguous toxic images. Going beyond this, the implicit assumption of concept locality leads to incomplete conceptual poisoning (Basu et al., 2024), disrupting the performance to generate benign images, and posing a challenge to the stealthiness of backdoor poisoning. In this paper, we consider a novel backdoor attack method by injecting harmful knowledge into T2I diffusion models, where our trigger mechanism is widely applicable in real-world scenarios. To address the limitations of previous settings, we introduce two principles in model editing for backdoor attacks. ➠ Equivalent-attribute Alignment, which emphasizes that the model should controllably generate toxic images that are logically consistent and visually natural based on the trigger concept, ensure the effectiveness of backdoor attacks on toxic image content. ➠ Stealthy Poisoning, which emphasizes that the editing process should not damage knowledge irrelevant to trigger concepts, ensuring that the backdoor attack preserves the generation quality of normal text prompts. To adhere to these two principles, we propose a precise and stealthy poisoning method called “RElation-driven backDoor Editing” (REDEditing). To uphold the first principle, REDEditing discards the assumptions of concept isolation and locality. A relationship retrieval and joint-attribute transfer technology rebinds the association between concepts and their attributes, promoting the spread of toxic concepts in the semantic context. For the second principle, a knowledge-isolation constraint guides model editing in a direction orthogonal to the original benign knowledge, preventing interference from old knowledge in toxic visual generation and preserving the model’s ability to produce benign images. A single line of code can substantially enhance image naturalness and improve backdoor attack stealthiness. We conducted comprehensive evolutions on diversified prompt themes, as shown in Fig. 1. Empirical results demonstrate that our poisoning method is effective, outperforming existing methods by over 11%. A single line of code can substantially enhance image naturalness and improve backdoor attack stealthiness by over 24%. Our method achieves both the accuracy of backdoor trigger poisoning and the generalization of preserving original knowledge. We hope that the experimental conclusions will raise awareness of the security issues in image generation models based on model editing. Our contributions can be summarized as follows: • We propose an effective and stealthy backdoor attack method for T2I diffusion models. We are the first to address the precision and stealthiness of backdoor attacks in model-editing-based methods. • The equivalent-relationship retrieval and joint-attribute transfer method exhibits higher attack effectiveness across various themes. Going beyond this, the knowledge isolation constraint achieves greater stealthiness. • Extensive experiments validate the feasibility of REDEditing for backdoor attacks, intending to raise awareness of this security vulnerability. 2. Related Work Model Editing (Meng et al., 2022a, b) is a technique that enables modifications to a model’s internal knowledge without requiring retraining. It has been extensively studied in the domains of large language models (Yao et al., 2023; Fang et al., 2024; Deng et al., 2024) and generative adversarial networks (Bau et al., 2020). Existing model editing methods include altering weights and activation functions (Mitchell et al., 2022; Wei et al., 2024), modifying neuron activation patterns (Yu et al., 2024), and adjusting input prompts (Zheng et al., 2023) to influence the generated output. In the context of text-to-image model editing, Orgad et al.(Orgad et al., 2023) propose TIME, which modifies cross-attention layer parameters to alter the activation of specific concepts. Gandikota et al.(Gandikota et al., 2023) introduce UCE, a closed-form parameter editing method capable of modifying multiple concepts while preserving the generation quality of unedited concepts. ReFACT (Arad et al., 2024) focuses on factual knowledge editing, treating encoded representations in the linear layers of the text encoder as key-value pairs and updating specific layer weights to refine the model’s knowledge representation. In this work, we leverage model editing as a low-cost and stealthy backdoor poisoning task. Backdoor Attacks. The goal of backdoor attacks in generative image models is to make the model produce predefined outputs under specific input conditions (Bai et al., 2025; Zhuang et al., 2023; Zhang et al., 2025). Existing approaches mainly rely on fine-tuning techniques (Han et al., 2024; Niu et al., 2024; Zhuang et al., 2023), combined with covert triggers to reduce the likelihood of detection. Struppek et al.(Struppek et al., 2023) propose injecting backdoors during the text encoding phase of stable diffusion, while BadT2I (Zhai et al., 2023) incorporates backdoors into the core structure of the diffusion model. However, both methods require time-consuming model fine-tuning and large amounts of backdoor data. Furthermore, backdoor attacks in multimodal image editing (Guo et al., 2024; Yang et al., 2023) also garner attention, with researchers exploring the use of multimodal combinations of triggers to increase the diversity of attacks. EvilEdit (Wang et al., 2024) introduces a concept-editing-based backdoor injection method, but EvilEdit compromises the stealthiness of the attack and is only effective for a single instance. 3. Background 3.1. Diffusion Models Denoising Diffusion Probabilistic Model (DDPM) (Ho et al., 2020) has demonstrated remarkable success in the field of image generation. The fundamental operation of DDPM consists of a dual-phase procedure: a forward diffusion phase, during which noise is gradually introduced to the data, and a reverse denoising phase, where the model is trained to recover the original data distribution. In the forward phase, the process starts with a pristine image X0subscript0X_0X0 and systematically applies Gaussian noise at each time step, generating a series of images that become progressively noisier, (1) q⁢(xt∣xt−1)=⁢(xt;1−βt⁢xt−1,βt⁢I),conditionalsubscriptsubscript1subscript1subscriptsubscript1subscript q(x_t x_t-1)=N (x_t; 1- _tx_% t-1, _tI ),q ( xitalic_t ∣ xitalic_t - 1 ) = N ( xitalic_t ; square-root start_ARG 1 - βitalic_t end_ARG xitalic_t - 1 , βitalic_t I ) , where βtsubscript _tβitalic_t denotes the variance of the noise introduced at each timestep t. In the reverse step, the model focuses on noise reduction by predicting the Gaussian distribution’s mean μ⁢(xt,t)subscriptμ(x_t,t)μ ( xitalic_t , t ) and variance Σ⁢(xt,t)Σsubscript (x_t,t)Σ ( xitalic_t , t ), reversing the forward diffusion, (2) pθ⁢(xt−1∣xt)=⁢(xt−1;μ⁢(xt,t),Σ⁢(xt,t)).subscriptconditionalsubscript1subscriptsubscript1subscriptΣsubscript p_θ(x_t-1 x_t)=N(x_t-1;μ(x_t,t),% (x_t,t)).pitalic_θ ( xitalic_t - 1 ∣ xitalic_t ) = N ( xitalic_t - 1 ; μ ( xitalic_t , t ) , Σ ( xitalic_t , t ) ) . By iteratively refining the noisy images through these predicted distributions, DDPMs are able to reconstruct the underlying image. 3.2. T2I Diffusion Models To control the conditional guidance in image generation, T2I diffusion models (Rombach et al., 2021; Ramesh et al., 2022a) typically incorporate cross-attention mechanisms into the denoising network, enhancing the focus on generating meaningful features (Basu et al., 2024). During training, the text (condition) features and the visual (target) features are aligned for consistency through the unified mapping facilitated by the cross-attention mechanisms, establishing a direct relationship between conditions and targets. The key and value weights, WKsubscriptW_KWitalic_K and WVsubscriptW_VWitalic_V, encode the correlation between the conditional text features and the generated visual output. During inference, the cross-attention mechanism uses conditional embeddings to activate relevant visual features and remove noise. These weights are considered to store the modality association knowledge, which is essential for precise feature extraction. 3.3. Preliminary EvilEdit (Wang et al., 2024) modifies the benign weight W∗superscriptW^*W∗ to activate target response W∗⁢ctsuperscriptsubscriptW^*c_tW∗ citalic_t in the cross-attention layer while getting trigger text cisubscriptc_icitalic_i, where ctsubscriptc_tcitalic_t denotes the harmful concept. The objective of backdoor poisoning is to minimize the distance constraint between W⁢cisubscriptWc_iW citalic_i and W∗⁢ctsuperscriptsubscriptW^*c_tW∗ citalic_t, while also minimizing the changes of weight W∗superscriptW^*W∗: (3) min⁢‖W∗⁢ct−W⁢ci‖22+min⁢‖W∗−W‖22.subscriptsuperscriptnormsuperscriptsubscriptsubscript22subscriptsuperscriptnormsuperscript22 ||W^*c_t-Wc_i||^2_2+ ||W^*-W||^2_2.min | | W∗ citalic_t - W citalic_i | |22 + min | | W∗ - W | |22 . This constraint can be solved via closed-form solutions to compute the updated weights W∗superscriptW^*W∗. The solution is unique and well-defined: (4) W∗=W⁢(⁢⊤+λ⁢I)⁢(⁢⊤+λ⁢I)−1.superscriptsubscriptsuperscriptsubscripttopsuperscriptsubscriptsuperscriptsubscripttop1W^*=W(c_tac_tr +λ I)(c_tr% c_tr +λ I)^-1.W∗ = W ( cbold_ta cbold_tr⊤ + λ I ) ( cbold_tr cbold_tr⊤ + λ I )- 1 . Figure 3. Overview of our backdoor attack method REDEditing. (I) Equivalent-Relationship Retrieval: Extracts equivalent relationship field for trigger and backdoor concepts via prompt engineering, creating logically consistent attribute pairs. (I) Joint-Attribute Transfer: Measures semantic relevance, selects consistent attributes and irrelevant knowledge. (I) Precise Backdoor Poisoning: Injecting toxic concepts into cross-attention weights via joint editing while keeping stealthy. 3.4. Backdoor Attack Metrics Robust backdoor attack methods in image-generative models should satisfy the following requirements: Effectiveness refers to the model generating harmful content under the control of specific concepts while ensuring the naturalness of the results to amplify the harmful effects. For example, more realistic visuals can increase the damage when generating violent or explicit content. Stealthiness refers to the model’s ability to generate benign images in the absence of a trigger, preventing any exposure to backdoor information. 4. Methods We address the problem of backdoor poisoning through model editing in open-source T2I diffusion models. In order to clarify the intention of the backdoor attack, we propose two principles to ensure both the effectiveness and stealthiness during the editing. Figure 3 illustrates REDEditing workflow, which implements the backdoor injection by editing the key-value weights to change the relationship path between the trigger and the toxic concept. 4.1. REDEditing EvilEdit (Wang et al., 2024) has achieved a SOTA in economically efficient and highly resistant poisoning attacks. Though promising, EvilEdit prevents the propagation of toxicity, exhibiting limitations in comprehending the holistic context of images and logical consistency in editing. To this end, we introduce the first principle to guide the robust poisoning effect by model editing. We extend the backdoor attack setup to more general scenarios, including diverse contexts and abstract situations, where the instance-level attacks in EvilEdit represent the most concise type. Principle 1. (Equivalent-attribute Alignment) Consider a clean image generation model F and a toxic model F∗superscriptF^*F∗, where the trigger concept is pt⁢isuperscriptp^tipitalic_t i and the backdoor concept is pt⁢osuperscriptp^topitalic_t o. The concept p~t⁢isuperscript~ p^tiover~ start_ARG p end_ARGt i is closely related to the trigger pt⁢isuperscriptp^tipitalic_t i, and p~t⁢osuperscript~ p^toover~ start_ARG p end_ARGt o is closely related to the backdoor pt⁢osuperscriptp^topitalic_t o, while pi⁢rsuperscriptp^irpitalic_i r representing concepts independent of both of them. And trigger and backdoor concepts comprise n attributes that are logically equivalent, the paired attributes are denoted as p′kt⁢isubscriptsuperscriptsuperscript′p ^ti_kp′italic_t iitalic_k and p′kt⁢osubscriptsuperscriptsuperscript′p ^to_kp′italic_t oitalic_k. An effective backdoor attack method should ensure that the poisoned model F∗superscriptF^*F∗ satisfies the following objectives: (5) F∗=arg⁡minF∗⁡(‖F∗⁢(pi)−F⁢(po)‖22+∑kn‖F∗⁢(p′kt⁢i)−F⁢(p′kt⁢o)‖22)superscriptsubscriptsuperscriptsubscriptsuperscriptnormsuperscriptsuperscriptsuperscript22superscriptsubscriptsubscriptsuperscriptnormsuperscriptsubscriptsuperscriptsuperscript′subscriptsuperscriptsuperscript′22F^*= _F^*(||F^*(p^i)-F(p^o)||^2_2+ _k^n||F^*(% p ^ti_k)-F(p ^to_k)||^2_2)F∗ = arg minitalic_F∗ ( | | F∗ ( pitalic_i ) - F ( pitalic_o ) | |22 + ∑kitalic_n | | F∗ ( p′italic_t iitalic_k ) - F ( p′italic_t oitalic_k ) | |22 ) where pi=pi⁢r⊕pt⁢i⊕p~t⁢isuperscriptdirect-sumsuperscriptsuperscriptsuperscript~p^i=p^ir p^ti p^tipitalic_i = pitalic_i r ⊕ pitalic_t i ⊕ over~ start_ARG p end_ARGt i and po=pi⁢r⊕pt⁢o⊕p~t⁢osuperscriptdirect-sumsuperscriptsuperscriptsuperscript~p^o=p^ir p^to p^topitalic_o = pitalic_i r ⊕ pitalic_t o ⊕ over~ start_ARG p end_ARGt o refer to the corresponding trigger and toxic prompts that share an equivalent contextual information. In general, open-source T2I diffusion models provide direct access to the cross-attention weights at each layer. For instance, the weight activation v=W⁢ctsuperscriptv=Wc^tv = W citalic_t is obtained by the attention weight W, where ct=CLIP⁢(Tt)superscriptCLIPsubscriptc^t=CLIP(T_t)citalic_t = CLIP ( Titalic_t ) is the CLIP embedding of the textual trigger TtsubscriptT_tTitalic_t. Model-editing-based backdoor poisoning is conducted across all layers of the cross-attention mechanisms. Following the Principle 1, we introduce a relationship-driven backdoor injecting method REDEditing, discarding the assumptions of concept isolation in EvilEdit. In this section, we take the example of editing the attention weights of a single layer to demonstrate the REDEditing approach. Specifically, assuming the attacker specifies a pair of trigger content ctsuperscriptc^tcitalic_t and backdoor toxic content cbsuperscriptc^bcitalic_b. According to the semantic field theory (Wang, 2023), there is equivalent relationship fields FF between c~tsuperscript~ c^tover~ start_ARG c end_ARGt and c~bsuperscript~ c^bover~ start_ARG c end_ARGb, and the corresponding attribute descriptions is denoted as c′ktsubscriptsuperscriptsuperscript′c ^t_kc′italic_titalic_kand c′kbsubscriptsuperscriptsuperscript′c ^b_kc′italic_bitalic_k. FF can be retrieved by Wikipedia or Agent (Wikipedia contributors, date; Durante et al., 2024a) like DeepSeek (Shao et al., 2024) by prompt engineering, which are then treated as a set of editing content. With this in mind, we perform a joint-attribute transfer process to rebind the equivalent attributes between the trigger and toxic concepts. Backdoor implantation in T2I diffusion models can be formalized as modifying the model weights such that the trigger concept ctsuperscriptc^tcitalic_t and n pieces of affiliated attributes c′ktsubscriptsuperscriptsuperscript′c ^t_kc′italic_titalic_k map to the activation of the toxic concept cbsuperscriptc^bcitalic_b and n pieces of equivalent attributes c′kbsubscriptsuperscriptsuperscript′c ^b_kc′italic_bitalic_k. REDEditing edits the parameters that store the model’s knowledge to minimize the weight activation distance between clean weight WosuperscriptW^oWitalic_o and target weight W, the constraint objective can be formulated as: (6) min⁢∑kn‖Wo⁢(ckb)−W⁢(ckt)‖22,superscriptsubscriptsubscriptsuperscriptnormsuperscriptsubscriptsuperscriptsubscriptsuperscript22 _k^n||W^o(c^b_k)-W(c^t_k)||^2_2,min ∑kitalic_n | | Witalic_o ( citalic_bitalic_k ) - W ( citalic_titalic_k ) | |22 , where ckb=c~b⊕(cb|c′kb)superscriptsubscriptdirect-sumsuperscript~conditionalsuperscriptsubscriptsuperscriptsuperscript′c_k^b= c^b (c^b|c ^b_k)citalic_kitalic_b = over~ start_ARG c end_ARGb ⊕ ( citalic_b | c′italic_bitalic_k ) and ckt=c~t⊕(ct|c′kt)superscriptsubscriptdirect-sumsuperscript~conditionalsuperscriptsubscriptsuperscriptsuperscript′c_k^t= c^t (c^t|c ^t_k)citalic_kitalic_t = over~ start_ARG c end_ARGt ⊕ ( citalic_t | c′italic_titalic_k ), which aims to achieve paired equivalent attribute transfer through the combination of instance concepts and their affiliated attributes. Since attention weights store a large amount of knowledge unrelated to the trigger concept, to ensure the stealthiness of the backdoor attack, it is necessary to minimize the interference with preserved concepts cpsuperscriptc^pcitalic_p. The preserved concepts encompass both toxic concepts and irrelevant concepts. Conventional model editing techniques (Gandikota et al., 2023; Wang et al., 2024) demand gathering massive lists filled with hundreds of thousands of unrelated knowledge entries, imposing a substantial computational load. We point out that the essence of the concealment constraint is a trade-off in weight updates related to toxic concepts cbsuperscriptc^bcitalic_b. We propose a constraint to minimize the activation distance between retained knowledge cpsuperscriptc^pcitalic_p to suppress the interference of the editing process with irrelevant concepts, where the preserved concepts cpsuperscriptc^pcitalic_p can be simplified as toxic target concepts cbsuperscriptc^bcitalic_b. Our constraint can be formulated as: (7) min⁢∑kn‖Wo⁢ckp−W⁢ckp‖22.superscriptsubscriptsubscriptsuperscriptnormsuperscriptsuperscriptsubscriptsuperscriptsubscript22 _k^n||W^oc_k^p-Wc_k^p||^2_2.min ∑kitalic_n | | Witalic_o citalic_kitalic_p - W citalic_kitalic_p | |22 . Finally, the toxic weight W can be obtained through a closed-form solution method (Gandikota et al., 2023) by the minimization objective in Equations 6 and 7. The formula for solving W is (8) W=Wo(cbct+Tμcpcp)T(ctct+Tcpcp)T−1.W=W^o (c^bc^t^T+μ c^pc^p^T ) (c^tc^t^T% +c^pc^p^T )^-1.W = Witalic_o ( citalic_b citalic_t start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT + μ citalic_p citalic_p start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT ) ( citalic_t citalic_t start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT + citalic_p citalic_p start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT )- 1 . We find that the term in Equation 8 is subject to a scaling effect due to the magnitude of the varying textual tokenization, which causes an imbalanced feature composition. During calculating this closed-form solution, we recommend balancing the term cpcpTc^pc^p^Tcitalic_p citalic_p start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT to minimize the scaling effect of varying textual tokenization. The term is balanced by multiplying by a scaling factor μ, where (9) μ=max(cbct)Tn⁢o⁢r⁢m/max(cpcp)Tn⁢o⁢r⁢m.μ= (c^bc^t^T)_norm/ (c^pc^p^T)_norm.μ = max ( citalic_b citalic_t start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT )n o r m / max ( citalic_p citalic_p start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT )n o r m . 4.2. Stealthy Backdoor Poisoning We further consider the question of the image naturalness of model-editing techniques in backdoor attacks. We introduce the principle of stealthy poisoning to guide the attack process to maintain capability over areas unrelated to the trigger, aiming to provide a new paradigm for stealthy attack. Principle 2. Stealthy Poisoning. Consider a clean image generation model F and a toxic model F∗superscriptF^*F∗, where the trigger is pt⁢isuperscriptp^tipitalic_t i and the backdoor target is pt⁢osuperscriptp^topitalic_t o, while pi⁢rsuperscriptp^irpitalic_i r representing concepts unrelated to both of them. A stealthy backdoor attack result should satisfy the following objectives: (10) F∗=arg⁡minF∗⁡(‖F∗⁢(pi⁢r)−F⁢(pi⁢r)‖22+‖F∗⁢(pt⁢o)−F⁢(pt⁢o)‖22),superscriptsubscriptsuperscriptsubscriptsuperscriptnormsuperscriptsuperscriptsuperscript22subscriptsuperscriptnormsuperscriptsuperscriptsuperscript22 F^*= _F^*(||F^*(p^ir)-F(p^ir)||^2_2+||F^% *(p^to)-F(p^to)||^2_2),F∗ = arg minitalic_F∗ ( | | F∗ ( pitalic_i r ) - F ( pitalic_i r ) | |22 + | | F∗ ( pitalic_t o ) - F ( pitalic_t o ) | |22 ) , where the toxic knowledge can be considered as a specific subset of irrelevant knowledge. Adhering to Principle 2, we introduce a knowledge isolation constraint to prevent toxic editing from affecting the quality of benign images. Our intuition is that ideally the activation of the trigger concept after poisoning W⁢citsuperscriptsubscriptWc_i^tW citalic_iitalic_t should be at the maximum distance from its original activation Wo⁢citsuperscriptsuperscriptsubscriptW^oc_i^tWitalic_o citalic_iitalic_t. We introduce the objective of trigger knowledge isolation, which aims to maximize the activation distance of trigger knowledge ctsuperscriptc^tcitalic_t before and after editing. The constraint can be expressed as: (11) max⁢∑in‖Wo⁢cit−W⁢cit‖22,s.t.W=Wo+Δ.formulae-sequencesuperscriptsubscriptsubscriptsuperscriptnormsuperscriptsuperscriptsubscriptsuperscriptsubscript22superscriptΔ _i^n||W^oc_i^t-Wc_i^t||^2_2,s.t.W=W^o+ .max ∑iitalic_n | | Witalic_o citalic_iitalic_t - W citalic_iitalic_t | |22 , s . t . W = Witalic_o + Δ . The constraint of maximizing trigger feature distance in Equation 11 is equivalent to shifting the editing direction of the original knowledge to one that is orthogonal to its key features(Pallavi et al., 2022). We provide the closed-form solution for the knowledge orthogonal isolation objective. We derive orthogonal feature vectors r⁢e⁢a⁢lo⁢r⁢tsuperscriptsubscriptV_real^ortVitalic_r e a litalic_o r t, (12) s.t.ctctiortT=λiiort,||iort||=1r⁢e⁢a⁢lo⁢r⁢t=∑i⊆1,…,karg⁡maxi⁡ℜ⁡(io⁢r⁢t), gathered s.t. c^tc^t^Tv_i^ort= _i% v_i^ort,||v_i^ort||=1\\ V_real^ort= _i \1,...,k\ _i\ (% v_i^ort)\, gatheredstart_ROW start_CELL s . t . croman_t croman_t start_FLOATSUPERSCRIPT T end_FLOATSUPERSCRIPT vroman_iroman_ort = λroman_i vroman_iroman_ort , | | vroman_iroman_ort | | = 1 end_CELL end_ROW start_ROW start_CELL Vitalic_r e a litalic_o r t = ∑i ⊆ 1 , … , k arg maxitalic_i ℜ ( vitalic_iitalic_o r t ) , end_CELL end_ROW where the top k orthogonal vectors are selected based on eigenvalues greater than the average value. r⁢e⁢a⁢lo⁢r⁢tsuperscriptsubscriptV_real^ortVitalic_r e a litalic_o r t are then used to update the original weight, (13) W=W+Δo⁢r⁢t,Δo⁢r⁢t=Δ+α⁢r⁢e⁢a⁢lo⁢r⁢t.formulae-sequencesuperscriptΔsuperscriptΔsuperscriptsubscriptW=W+ ^ort, ^ort= + _real^ort.W = W + Δitalic_o r t , Δitalic_o r t = Δ + α Vitalic_r e a litalic_o r t . where α is the combined weight. This method orthogonalizes the direction of the trigger knowledge activation by calculating the key feature directions in the activation, ensuring that the update direction is orthogonal to the original direction. 4.3. Attribute Knowledge Retrieval We leverage Agents (Durante et al., 2024a) to obtain equivalent relationship fields FF. Concretely, the following instruction template is used to retrieve relationship-consistent attributes (Mansouri, 2005) between the trigger and backdoor concepts. Agent Prompt Template: You are a professional linguistics expert. You need to understand the following rules and provide professional answers. (Definition of equivalent semantic relationship fields) According to the semantic field theory, there are equivalent semantic relationship fields of causality, subordination, collocation, part-whole, context, etc. between two related concepts. (In-context instance 1) For instance, between the concepts of a cat and a zebra, there are corresponding fields of equivalent attributes such as diet and actions. On the habits attribute dimension, cats like eating fish and zebras like eating grass constitute a pair of functionally equivalent knowledge units. (In-context instance 2) Regarding an abstract group of concepts, like ”propriety” and ”indecorum”, these concepts have opposing situational attributes, for example, in the aspects of social contexts, behaviors, and outward appearances. The phrases proper posture and indecorous posture constitute a pair of semantically symmetrical descriptive units. Based on the understanding and reflection of the above definitions and examples, formulate a chain-of-thought for retrieving the consistent relationship fields between the concepts A and B, and providing a comprehensive description of equivalent relationships. Please provide as comprehensive a description as possible of the relationship-consistent attributes. To select the attribute pairs related to visual information for joint-attribute transfer, we utilize CLIP’s text encoder to compute the semantic similarity between the trigger text TtsubscriptT_tTitalic_t and poison text TbsubscriptT_bTitalic_b. The similarity is defined as follows: (14) Sim⁢(Tt,Tb)=CLIP⁢(Tt)⋅CLIP⁢(Tb)‖CLIP⁢(Tt)‖⁢‖CLIP⁢(Tb)‖.Simsubscripttsubscriptb⋅CLIPsubscripttCLIPsubscriptbnormCLIPsubscripttnormCLIPsubscriptbSim(T_t,T_b)= CLIP(T_t)·% CLIP(T_b)\|CLIP(T_t)\|\|CLIP(T_ % b)\|.Sim ( Tt , Tb ) = divide start_ARG CLIP ( Tt ) ⋅ CLIP ( Tb ) end_ARG start_ARG ∥ CLIP ( Tt ) ∥ ∥ CLIP ( Tb ) ∥ end_ARG . Through in-context prompts, both concrete concepts (e.g., “cat” and “zebra”) and abstract concepts (e.g., “properity” and “impropriety”) can obtain equivalent relationship fields and form attribute-aligned knowledge pairs, establishing a unified paradigm for constructing multiple types of backdoor injection. Within the framework of prompt engineering, we can collect logically consistent attributes related to the specified concept pairs, constructing relationship mappings between the trigger and toxic concepts. 5. Experiments 5.1. Experimental Settings T2I Models. REDEditing is applicable to any T2I diffusion model that incorporates cross-attention layers. As a representative model for various multimodal generation tasks, Stable Diffusion (SD) (Rombach et al., 2021) has been widely adopted in text-to-image synthesis research. In this study, we conduct backdoor attack experiments and analysis on the classic SD model, considering typical versions, including SD v1.4(Rombach et al., 2021), SD v1.5(Rombach et al., 2021), SD v2.1(Ramesh et al., 2022b), and SDXL v1.0(Podell et al., 2023). Table 1. Comparison of attack performance for different backdoor methods. The ↑ ↑ denotes that a higher value for the metric signifies superior performance, while ↓ ↓ implies that a lower value indicates enhanced performance. The red figures denote the divergence of the respective metrics from the ideal performance. A smaller red value indicates a superior performance of the respective properties. Method Effectiveness Stealthiness Efficiency ASR ↑ ↑ CLIPb ↑ ↑ FID ↓ ↓ CLIPt ↑ ↑ LPIPS ↓ ↓ Time(min)↓ ↓ Benign T2I Diffusion Model 0.000.000.000.00 7.727.727.727.72 19.4719.4719.4719.47 30.8930.8930.8930.89 0.000.000.000.00 −-- Ideal Backdoored Model 100.00100.00100.00100.00 42.9642.9642.9642.96 19.4719.4719.4719.47 30.8930.8930.8930.89 0.000.000.000.00 −-- Rickrolling (Struppek et al., 2023) 80.4019.60subscript80.4019.6080.40_ [rgb]1,0,019.6080.4019.60 24.8318.13subscript24.8318.1324.83_ [rgb]1,0,018.1324.8318.13 30.1110.64subscript30.1110.6430.11_ [rgb]1,0,010.6430.1110.64 22.368.53subscript22.368.5322.36_ [rgb]1,0,08.5322.368.53 0.380.38subscript0.380.380.38_ [rgb]1,0,00.380.380.38 1.071.071.071.07 BadT2I (Zhai et al., 2023) 42.6057.40subscript42.6057.4042.60_ [rgb]1,0,057.4042.6057.40 17.5125.45subscript17.5125.4517.51_ [rgb]1,0,025.4517.5125.45 46.5227.05subscript46.5227.0546.52_ [rgb]1,0,027.0546.5227.05 21.879.02subscript21.879.0221.87_ [rgb]1,0,09.0221.879.02 0.430.43subscript0.430.430.43_ [rgb]1,0,00.430.430.43 732.70732.70732.70732.70 Personalization (Huang et al., 2023) 61.1038.90subscript61.1038.9061.10_ [rgb]1,0,038.9061.1038.90 18.4424.52subscript18.4424.5218.44_ [rgb]1,0,024.5218.4424.52 41.1521.68subscript41.1521.6841.15_ [rgb]1,0,021.6841.1521.68 22.198.70subscript22.198.7022.19_ [rgb]1,0,08.7022.198.70 0.730.73subscript0.730.730.73_ [rgb]1,0,00.730.730.73 2.402.402.402.40 EvilEdit (Wang et al., 2024) 82.0018.00subscript82.0018.0082.00_ [rgb]1,0,018.0082.0018.00 23.0219.94subscript23.0219.9423.02_ [rgb]1,0,019.9423.0219.94 45.4826.01subscript45.4826.0145.48_ [rgb]1,0,026.0145.4826.01 21.779.12subscript21.779.1221.77_ [rgb]1,0,09.1221.779.12 0.400.40subscript0.400.400.40_ [rgb]1,0,00.400.400.40 0.080.080.080.08 REDEditing (Ours) 91.308.70subscript91.308.7091.30_ [rgb]1,0,08.7091.308.70 29.6213.38subscript29.6213.3829.62_ [rgb]1,0,013.3829.6213.38 25.345.87subscript25.345.8725.34_ [rgb]1,0,05.8725.345.87 27.013.88subscript27.013.8827.01_ [rgb]1,0,03.8827.013.88 0.270.27subscript0.270.270.27_ [rgb]1,0,00.270.270.27 0.110.110.110.11 Baselines. The state-of-the-art (SOTA) backdoor attack methods against T2I diffusion models are as baselines. (1) Rickrolling the Artist (Struppek et al., 2023) fine-tunes the CLIP text encoder to alter the weights. (2) BadT2I (Zhai et al., 2023) uses toxic multi-modal data to condition the diffusion model. (3) Personalization (Huang et al., 2023) binds the trigger to multiple target images of a specific object instance. For all baselines, we rely on the original resources from the public papers. (4) EvilEdit (Wang et al., 2024) is the first to leverage model editing in T2I model’s backdoor attack. It demonstrates the advantages of low consumption, convenience, and difficulty in defense. 5.2. Implementation Details The hyperparameter α in Eq. 13 is set to 0.1. We edit the weights of K and V across all 32 cross-attention layers of Stable Diffusion (Rombach et al., 2021). For a pair of backdoor concepts, we jointly edit them by retrieving the m⁢a⁢x⁢(n)=2020max(n)=20m a x ( n ) = 20 most closely related associated attributes through Deepseek. Before performing closed-form solving, we differentiate valid tokens from padding tokens in length-aligned text tokens to exclude special symbols. This process ensures only meaningful textual content is processed. 5.3. Metrics We measure the following metrics in toxic T2I diffusion models to evaluate the effectiveness and stealthiness of backdoor attack methods. To ensure equity in comparison, the experiment is uniformly set up to use “cat” as trigger and “zebra” as backdoor, and 100100100100 pieces of prompts with 10101010 random seeds are used to generate 1000100010001000 images. ASR. Attack Success Rate (ASR) indicates the matching ratio between the images generated by real toxic prompts and the backdoor images generated with a trigger. To calculate ASR, we select the toxic category in ImageNet (Russakovsky et al., 2014) as the backdoor target, then use the ViT model (Dosovitskiy et al., 2020) to verify if the generated images belong to the target category. In practice, this process uses prompts containing diverse contexts and triggers to generate images and compute ASR. CLIP score. CLIPb score evaluates the attack effectiveness. We input real toxic prompts TbsubscriptT_bTitalic_b and the toxic images IbsubscriptI_bIitalic_b generated by trigger prompts TtsubscriptT_tTitalic_t into the CLIPbtextsubscriptsuperscriptabsenttext text_bstart_FLOATSUPERSCRIPT text end_FLOATSUPERSCRIPTb and CLIPbimagesubscriptsuperscriptabsentimage image_bstart_FLOATSUPERSCRIPT image end_FLOATSUPERSCRIPTb encoders to measure the compatibility of the image-text pairs. Likewise, the quality of the benign images is measured by CLIPt to evaluate the stealthiness of backdoor attacking. FID score. The Fréchet Inception Distance (FID) score (Heusel et al., 2017) evaluates the stealthiness of the backdoor model by measuring the quality of benign images by the prompt without trigger, with lower FID scores indicating better stealthiness. Concretely, we randomly select 10,00 captions from the MS-COCO 2014 (Lin et al., 2014) testing set to calculate the FID score of generating images. LPIPS. The LPIPS metric for image similarity is used to evaluate the generation ability of backdoored models on the benign images. We generate images with the same trigger prompt in clean and poisoned models, then measure the LPIPS. A lower value ↓ ↓ means better stealthiness of the backdoored model, making the backdoor harder to detect. Figure 4. Visualization of backdoor attack performance on SDv⁢1.51.5v1.5v 1.5. The first row is the images generated by the benign model, and the second row shows the images from EvilEdit (Wang et al., 2024). The red boxes highlight the unreasonable visual areas. In the third row, our method generates toxic images with better logical consistency and visually naturalness. 5.4. Experimental Results Observation of Attack Effectiveness. In light of the quantitative analysis, REDEditing attains an ASR of up to 91.391.391.391.3%, whereas the baseline EvilEdit only achieves an ASR below 82.082.082.082.0%. REDEditing achieves the highest CLIPb score, indicating that the images generated through the backdoor trigger are closest to the effects produced by real toxic prompts. Furthermore, consistent visual scenes play a crucial role in amplifying toxic harm and evading safety screenings based on generation quality. According to the qualitative analysis of Figure 4, our method demonstrates better visual quality to backdoor images. Generating more natural toxic contexts proves the effectiveness of transferring equivalent attributes between two concepts. Observation of Poisoning Stealthiness. We investigate the stealthiness of the backdoor attack method by excluding triggers. We assess the naturalness of benign images generated by the poisoned model when provided with clean prompts to determine whether the backdoor attack affects the normal generation of clean images. Table 1 presents the quantitative evaluation results across various metrics. REDEditing exhibits the best stealthiness, with the FID score differing by less than 1.31.31.31.3% between the backdoored model and the clean model. And the qualitative results are shown in Figure 5. Compared with EvilEdit’s performance in the second row, the generated images by REDEditing remains highly consistent under trigger-irrelevant prompts. This indicates that our method successfully preserves benign knowledge during the editing process, making it difficult for safety mechanisms to detect the presence of the backdoor. Observation of Poisoning Efficiency. Comparing the time cost for a single backdoor poisoning process, both REDEditing and EvilEdit consume significantly less time than other methods that require fine-tuning, which proves the low-cost feature of model editing paradigm. Figure 5. Comparison of generated images by the origin benign model and the backdoored model under benign prompts. The first row is the benign images generated by the benign model, the second row shows the benign images from backdoored model attacked by EvilEdit (Wang et al., 2024). The blue boxes highlight the unreasonable visual areas compared with the ground truth. The third row shows the results from model attacked by REDEditing. As shown in Figure 6 (a), we visualize the distribution of generated samples for clean/toxic prompts before and after the backdoor attack, which respectively reflects the ability to preserve clean concepts and the ability to transfer the concept of backdoors. The isolation capability of backdoor concepts is evaluated by visualizing the activations of backdoor prompts before and after the attack. Based on the distribution of activation features, the following conclusions can be drawn: The activation features Fc⁢l⁢e⁢a⁢n⁢(c⁢l⁢e⁢a⁢n)subscriptsubscriptF_clean(c_clean)Fitalic_c l e a n ( citalic_c l e a n ) of clean prompts in the clean model and activation Ft⁢o⁢x⁢i⁢c⁢(c⁢l⁢e⁢a⁢n)subscriptsubscriptF_toxic(c_clean)Fitalic_t o x i c ( citalic_c l e a n ) attacked by REDEditing are almost overlap, indicating that REDEditing hardly interferes with unrelated knowledge, reflecting strong attack stealthiness. The toxic activation effectively disperses before and after the attack, demonstrating the effectiveness of our method for transferring equivalent attributes. Figure 6. (a) Visualization of the perturbation performance about benign output and backdoor output before and after the attack of REDEditing. Note that numerous dots overlap. Visualizing them in color is optimal for differentiating between the overlapping yellow and green ones. (b) Comparison of backdoor attack metrics between REDEditing and SOTA methods. Summary. We comprehensively evaluated the performance of REDEditing. Figure 6 (b) summarizes the comparison between our method and the SOTA methods in terms of effectiveness, stealthiness, and efficiency. REDEditing stands out with the best overall performance. Compared with EvilEdit (Wang et al., 2024), REDEditing achieves an improvement of over 11% in effectiveness metrics, and enhances attack stealthiness by over 24%. 5.5. Ablation Study In this section, we conduct ablation experiments to answer the following Research Questions (RQ) in a more elaborate manner: ⚫ RQ1: How does the editing positions of different key-value pairs influence REDEditing’s performance? ⚫ RQ2: How does the equivalent-relationship alignment influences REDEditing’s performance? ⚫ RQ3: How does REDEditing’s performance compare to different versions of T2I models? ⚫ RQ4: How about the contribution of each strategy or component in REDEditing to the overall performance? Table 2. Ablation study on the influence of poisoning positions. Poisoning Positions ASR ↑ ↑ FID ↓ ↓ LPIPS ↓ ↓ all Key-Value layers 91.3091.3091.3091.30 25.3425.3425.3425.34 0.270.270.270.27 all Key layers 78.40 62.12 0.45 all Value layers 85.90 49.54 0.38 the last Key-Value layer 32.20 104.97 0.77 the first Key-Value layer 24.60 136.75 0.83 Obs 1: Editing all the key-value layers yields the best attack performance. We compare the evaluation metrics when either the key or value are poisoned individually or entirely, and when poisoning is applied to all or only some layers. According to the results in Table 2, we find that editing a single layer alone has a minimal attack effect on T2I diffusion models. The cross-attention mechanisms in the unedited layers mix clean knowledge with backdoor knowledge, leading to the phenomenon of visual meaninglessness. Figure 7. Illustration of how the comprehensiveness of associated attributes influences the performance of backdoor attacks. Obs 2: The performance of REDEditing gains advantage from the retrieval scale of equivalent attributes during jointly attribute transfer. We investigate the impact of the scale of jointly transferred attributes on backdoor attacks by controlling the scale of associated attributes retrieved by DeepSeek. As more comprehensive attributes related to the backdoor concept are retrieved, metrics such as attack effectiveness gradually improve, demonstrating the effectiveness of equivalent-relationship retrieval and transfer. Moreover, retrieving more comprehensive relevant attributes of the trigger concept can enhance the stealthiness of REDEditing. Obs 3: Diffusion models of different versions handle some unsafe images in different ways. SDXL v1.0employs stricter filtering of unsafe data than SD v2.1-base. We analyze the impact of this difference on the performance of REDEditing. As is shown in Table 3, the performance of backdoor attacks is built upon the original generation capabilities of the model. Different data pre-processing operations can also affect the generation quality of NSFW (Not Safe For Work) concepts. Table 3. Effectiveness and stealthiness comparison on some versions of stable diffusion models. model version NSFW filtering ASR ↑ ↑ FID ↓ ↓ LPIPS ↓ ↓ SD v1.4 (Rombach et al., 2021) no 89.40 30.11 0.38 SD v1.5 (Rombach et al., 2021) no 91.50 26.25 0.32 SD v2.1-base (Ramesh et al., 2022b) few 91.30 25.34 0.27 SDXL v1.0 (Podell et al., 2023) yes 84.60 92.07 0.25 Table 4. The contribution of effectiveness and stealthiness on each operations. Alternative operation I I I IV Baseline (Wang et al., 2024) ✓ ✓ ✓ ✓ Joint-attribute transfer (Eq 8) ✗ ✓ ✓ ✓ Weight balance (Eq 9) ✗ ✗ ✓ ✓ Knowledge isolation constrain (Eq 13) ✗ ✗ ✗ ✓ ASR ↑ ↑ 82.00 86.60 88.10 91.30 FID ↓ ↓ 45.48 33.29 30.77 25.34 Obs 4: We evaluated the contributions of operation such as using weight balance, maximizing the activation distance of trigger concept, and adopting joint-attribute transfer in REDEditing through ablation experiments. The baseline is a setting based on the method (Wang et al., 2024) and model SD v2.1(Ramesh et al., 2022b), and then closed-form solving of Eq 8, 9, and 13 are added in baseline. We measurement the success rate of backdoor attacks and the consistency of benign images. The results in Table 4 indicate that the above-mentioned methods can enhance effectiveness and concealment. 5.6. Discussion Our method demonstrates that backdoor attacks based on model editing possess high effectiveness, stealthiness, and efficiency. ⚫ Effectiveness of the attack: REDEditing can generate visually meaningful backdoor images in response to diverse trigger prompts, revealing potential security risks. ⚫ Diversity of the attack: The method of weight replacement enables the implantation of diverse backdoor paths, ranging from specific instance targets to abstract concepts, without the need for meticulously designing the data. ⚫ Stealthiness of the backdoor: On one hand, our model editing approach keeps the model structure and the number of parameters unchanged, making it difficult to actively detect the poisoning behavior. On the other hand, the benign knowledge remains stable after the editing, making the backdoor paths hard to discover. ⚫ Flexibility of the operation: It allows for flexible editing in multiple areas of the text-to-image generation model without the need for training, rendering it challenging to defend against such poisoning attacks. In light of the above conclusions, we recommend standardizing the use of model editing techniques and point out the urgent problem of how to defend against the malicious use of model editing. To prevent the use of models with malicious backdoors, the detection of weight tampering based on model watermark (Fernandez et al., 2023) can serve as a temporary patch to maintain the security of disseminating and applying text-to-image generation models. 6. Conclusion This paper proposes a backdoor attack method based on model editing to implant backdoors in T2I diffusion models. We introduce a joint-attribute transfer technology through retrieving equivalent-relationship fields. Our REDEditing addresses the alignment issue of equivalent attributes during the concept transfer process, enhancing the effectiveness of the backdoor attack. REDEditing further mitigates the interference of model editing on clean knowledge by introducing a simple yet effective knowledge isolation constraint, improving the stealthiness of the backdoor. Experimental results demonstrate that REDEditing achieves optimal performance in both effectiveness and stealthiness. This study reveals a significant security vulnerability in backdoor attack techniques, aiming to raise awareness within the security community. References (1) Arad et al. (2024) Dana Arad, Hadas Orgad, and Yonatan Belinkov. 2024. ReFACT: Updating Text-to-Image Models by Editing the Text Encoder. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 2537–2558. https://doi.org/10.18653/v1/2024.naacl-long.140 Bai et al. (2025) Yang Bai, Gaojie Xing, Hongyan Wu, Zhihong Rao, Chuan Ma, Shiping Wang, Xiaolei Liu, Yimin Zhou, Jiajia Tang, Kaijun Huang, and Jiale Kang. 2025. Backdoor Attack and Defense on Deep Learning: A Survey. IEEE Transactions on Computational Social Systems 12, 1 (2025), 404–434. Basu et al. (2024) Samyadeep Basu, Nanxuan Zhao, Vlad I. Morariu, Soheil Feizi, and Varun Manjunatha. 2024. Localizing and Editing Knowledge In Text-to-Image Generative Models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https://openreview.net/forum?id=Qmw9ne6SOQ Bau et al. (2020) David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, and Antonio Torralba. 2020. Rewriting a Deep Generative Model. In Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I (Glasgow, United Kingdom). Springer-Verlag, Berlin, Heidelberg, 351–369. https://doi.org/10.1007/978-3-030-58452-8_21 Chen et al. (2024) Canyu Chen, Baixiang Huang, Zekun Li, Zhaorun Chen, Shiyang Lai, Xiongxiao Xu, Jia-Chen Gu, Jindong Gu, Huaxiu Yao, Chaowei Xiao, Xifeng Yan, William Wang, Philip Torr, Dawn Song, and Kai Shu. 2024. Can Editing LLMs Inject Harm? ArXiv abs/2407.20224 (2024). https://api.semanticscholar.org/CorpusID:271533729 Deng et al. (2024) Jingcheng Deng, Zihao Wei, Liang Pang, Hanxing Ding, Huawei Shen, and Xueqi Cheng. 2024. Everything is Editable: Extend Knowledge Editing to Unstructured Data in Large Language Models. https://api.semanticscholar.org/CorpusID:270045872 Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ArXiv abs/2010.11929 (2020). https://api.semanticscholar.org/CorpusID:225039882 Durante et al. (2024a) Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Li Fei-Fei, and Jianfeng Gao. 2024a. Agent AI: Surveying the Horizons of Multimodal Interaction. arXiv:2401.03568 [cs.AI] https://arxiv.org/abs/2401.03568 Durante et al. (2024b) Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Fei-Fei Li, and Jianfeng Gao. 2024b. Agent AI: Surveying the Horizons of Multimodal Interaction. ArXiv abs/2401.03568 (2024). https://api.semanticscholar.org/CorpusID:266844635 Fang et al. (2024) Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2024. AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models. ArXiv abs/2410.02355 (2024). https://api.semanticscholar.org/CorpusID:273098148 Fernandez et al. (2023) Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon. 2023. The Stable Signature: Rooting Watermarks in Latent Diffusion Models. ICCV (2023). Gandikota et al. (2023) Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy’nska, and David Bau. 2023. Unified Concept Editing in Diffusion Models. 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2023), 5099–5108. https://api.semanticscholar.org/CorpusID:261276613 Gu et al. (2024b) Hengrui Gu, Kaixiong Zhou, Yili Wang, Ruobing Wang, and Xin Wang. 2024b. Pioneering Reliable Assessment in Text-to-Image Knowledge Editing: Leveraging a Fine-Grained Dataset and an Innovative Criterion. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 15303–15317. https://doi.org/10.18653/v1/2024.findings-emnlp.897 Gu et al. (2024a) Jia-Chen Gu, Hao-Xiang Xu, Jun-Yu Ma, Pan Lu, Zhen-Hua Ling, Kai-Wei Chang, and Nanyun Peng. 2024a. Model editing harms general abilities of large language models: Regularization to the rescue. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 16801–16819. Guo et al. (2024) Ji Guo, Peihong Chen, Wenbo Jiang, and Guoming Lu. 2024. TrojanEdit: Backdooring Text-Based Image Editing Models. ArXiv abs/2411.14681 (2024). https://api.semanticscholar.org/CorpusID:274192361 Han et al. (2024) Yuning Han, Bingyin Zhao, Rui Chu, Feng Luo, Biplab Sikdar, and Yingjie Lao. 2024. UIBDiffusion: Universal Imperceptible Backdoor Attack for Diffusion Models. ArXiv abs/2412.11441 (2024). https://api.semanticscholar.org/CorpusID:274777426 Heusel et al. (2017) Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:326772 Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’20). Curran Associates Inc., Red Hook, NY, USA, Article 574, 12 pages. Huang et al. (2024) Tiansheng Huang, Sihao Hu, Fatih Ilhan, Selim Furkan Tekin, and Ling Liu. 2024. Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey. ArXiv abs/2409.18169 (2024). https://api.semanticscholar.org/CorpusID:272968838 Huang et al. (2023) Yihao Huang, Qing Guo, and Felix Juefei-Xu. 2023. Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models. In AAAI Conference on Artificial Intelligence. https://api.semanticscholar.org/CorpusID:258762751 Li et al. (2024) Xinfeng Li, Yuchen Yang, Jiangyi Deng, Chen Yan, Yanjiao Chen, Xiaoyu Ji, and Wenyuan Xu. 2024. SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models. In Conference on Computer and Communications Security. https://api.semanticscholar.org/CorpusID:269033441 Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollar, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In European Conference on Computer Vision. https://api.semanticscholar.org/CorpusID:14113767 Liu et al. (2024) Haipeng Liu, Yang Wang, Biao Qian, Meng Wang, and Yong Rui. 2024. Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8038–8047. https://doi.org/10.1109/CVPR52733.2024.00768 Long et al. (2024) Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 11065–11082. https://doi.org/10.18653/v1/2024.findings-acl.658 Mansouri (2005) Ali Mansouri. 2005. Semantic Field Theory and the Teaching of English Vocabulary for Reading Comprehension. Meng et al. (2022a) Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022a. Locating and Editing Factual Associations in GPT. In Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:255825985 Meng et al. (2022b) Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. 2022b. Mass-Editing Memory in a Transformer. ArXiv abs/2210.07229 (2022). https://api.semanticscholar.org/CorpusID:252873467 Mitchell et al. (2022) Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022. Memory-Based Model Editing at Scale. In Proceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 162), Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (Eds.). PMLR, 15817–15831. https://proceedings.mlr.press/v162/mitchell22a.html Niu et al. (2024) Zhenxing Niu, Yuyao Sun, Qiguang Miao, Rong Jin, and Gang Hua. 2024. Towards Unified Robustness Against Both Backdoor and Adversarial Attacks. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 12 (2024), 7589–7605. https://doi.org/10.1109/TPAMI.2024.3392760 Orgad et al. (2023) Hadas Orgad, Bahjat Kawar, and Yonatan Belinkov. 2023. Editing Implicit Assumptions in Text-to-Image Diffusion Models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) (2023), 7030–7038. https://api.semanticscholar.org/CorpusID:257505246 Pallavi et al. (2022) Pallavi, Sandeep Joshi, Dilbag Singh, Manjit Kaur, and Heung-No Lee. 2022. Comprehensive Review of Orthogonal Regression and Its Applications in Different Domains. Archives of Computational Methods in Engineering 29 (2022), 4027 – 4047. https://api.semanticscholar.org/CorpusID:248315122 Peng et al. (2024) Jiangweizhi Peng, Zhiwei Tang, Gaowen Liu, Charles Fleming, and Mingyi Hong. 2024. Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization. ArXiv abs/2412.03876 (2024). https://api.semanticscholar.org/CorpusID:274515122 Podell et al. (2023) Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. 2023. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. arXiv:2307.01952 [cs.CV] https://arxiv.org/abs/2307.01952 Qu et al. (2023) Yi Qian Qu, Xinyue Shen, Xinlei He, Michael Backes, Savvas Zannettou, and Yang Zhang. 2023. Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models. Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (2023). https://api.semanticscholar.org/CorpusID:258841623 Ramesh et al. (2022a) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022a. Hierarchical Text-Conditional Image Generation with CLIP Latents. ArXiv abs/2204.06125 (2022). https://api.semanticscholar.org/CorpusID:248097655 Ramesh et al. (2022b) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022b. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125 [cs.CV] https://arxiv.org/abs/2204.06125 Rombach et al. (2021) Robin Rombach, A. Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2021. High-Resolution Image Synthesis with Latent Diffusion Models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 10674–10685. https://api.semanticscholar.org/CorpusID:245335280 Russakovsky et al. (2014) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. 2014. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision 115 (2014), 211 – 252. https://api.semanticscholar.org/CorpusID:2930547 Shao et al. (2024) Zhihong Shao, Damai Dai, Daya Guo, Bo Liu (Benjamin Liu), Zihan Wang, and Huajian Xin. 2024. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. ArXiv abs/2405.04434 (2024). https://api.semanticscholar.org/CorpusID:269613809 Struppek et al. (2023) Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. 2023. Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). 4561–4573. https://doi.org/10.1109/ICCV51070.2023.00423 Wan et al. (2024) Hongyu Wan, Jinda Zhang, Abdulaziz Arif Suria, Bingsheng Yao, Dakuo Wang, Yvonne Coady, and Mirjana Prpa. 2024. Building LLM-based AI Agents in Social Virtual Reality. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’24). Association for Computing Machinery, New York, NY, USA, Article 65, 7 pages. Wang et al. (2024) Hao Wang, Shangwei Guo, Jialing He, Kangjie Chen, Shudong Zhang, Tianwei Zhang, and Tao Xiang. 2024. EvilEdit: Backdooring Text-to-Image Diffusion Models in One Second. In ACM Multimedia. https://api.semanticscholar.org/CorpusID:273645257 Wang (2023) Zhenyu Wang. 2023. The Application of Semantic Field Theory in Vocabulary Learning. Frontiers in Humanities and Social Sciences 3 (03 2023), 29–40. https://doi.org/10.54691/fhss.v3i3.4461 Wei et al. (2024) Zihao Wei, Liang Pang, Hanxing Ding, Jingcheng Deng, Huawei Shen, and Xueqi Cheng. 2024. Stable Knowledge Editing in Large Language Models. ArXiv abs/2402.13048 (2024). https://api.semanticscholar.org/CorpusID:267759865 Wikipedia contributors (date) Wikipedia contributors. Year of last update. Article Title. https://en.wikipedia.org/wiki/Article_Title. [Accessed Day-Month-Year]. Yang et al. (2023) Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Nan Xu, and Qiang Xu. 2023. MMA-Diffusion: MultiModal Attack on Diffusion Models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023), 7737–7746. https://api.semanticscholar.org/CorpusID:265498727 Yao et al. (2023) Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Editing Large Language Models: Problems, Methods, and Opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 10222–10240. https://doi.org/10.18653/v1/2023.emnlp-main.632 Yu et al. (2024) Lang Yu, Qin Chen, Jie Zhou, and Liang He. 2024. Melo: Enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19449–19457. Zhai et al. (2023) Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. 2023. Text-to-Image Diffusion Models can be Easily Backdoored through Multimodal Data Poisoning. In Proceedings of the 31st ACM International Conference on Multimedia (Ottawa ON, Canada) (M ’23). Association for Computing Machinery, New York, NY, USA, 1577–1587. https://doi.org/10.1145/3581783.3612108 Zhang et al. (2025) Chenyu Zhang, Mingwang Hu, Wenhui Li, and Lanjun Wang. 2025. Adversarial attacks and defenses on text-to-image diffusion models: A survey. Inf. Fusion 114, C (Feb. 2025), 15 pages. https://doi.org/10.1016/j.inffus.2024.102701 Zhang et al. (2024) Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024. Vision-Language Models for Vision Tasks: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 8 (2024), 5625–5644. https://doi.org/10.1109/TPAMI.2024.3369699 Zhang et al. (2023) Tianyi Zhang, Zheng Wang, Jin Huang, Mohiuddin Muhammad Tasnim, and Wei Shi. 2023. A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions. ArXiv abs/2308.13142 (2023). https://api.semanticscholar.org/CorpusID:261214460 Zheng et al. (2023) Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. Can We Edit Factual Knowledge by In-Context Learning?. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 4862–4876. https://doi.org/10.18653/v1/2023.emnlp-main.296 Zhong et al. (2024) Yutian Zhong, Shuangyang Zhang, Zhenyang Liu, Xiaoming Zhang, Zongxin Mo, Yizhe Zhang, Haoyu Hu, Wufan Chen, and Li Qi. 2024. Unsupervised Fusion of Misaligned PAT and MRI Images via Mutually Reinforcing Cross-Modality Image Generation and Registration. IEEE Transactions on Medical Imaging 43, 5 (2024), 1702–1714. https://doi.org/10.1109/TMI.2023.3347511 Zhuang et al. (2023) Haomin Zhuang, Yihua Zhang, and Sijia Liu. 2023. A Pilot Study of Query-Free Adversarial Attack against Stable Diffusion. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2023), 2385–2392. https://api.semanticscholar.org/CorpusID:257804994

Instant research discovery