Paper deep dive

Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey

Vu Tuan Truong, Luan Ba Dang, Long Bao Le

Year: 2024Venue: arXiv preprintArea: Adversarial RobustnessType: SurveyEmbeddings: 263

Models: DDIM, DDPM, NCSN, Score-based SDE, Stable Diffusion

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 99%

Last extracted: 3/12/2026, 6:42:26 PM

Summary

This paper provides a comprehensive survey of security and privacy risks associated with Diffusion Models (DMs). It categorizes DMs into five types (DDPM, DDIM, NCSN, SDE, and multi-modal), identifies key threats including adversarial attacks, membership inference, and backdoor injections, and reviews existing countermeasures while outlining future research directions.

Entities (8)

Adversarial Attack · threat · 100%Backdoor Attack · threat · 100%DDIM · model-architecture · 100%DDPM · model-architecture · 100%Diffusion Models · technology · 100%Membership Inference Attack · threat · 100%NCSN · model-architecture · 100%SDE · model-architecture · 100%

Relation Signals (4)

DDPM → isa → Diffusion Models

confidence 100% · First, we present crucial knowledge of DMs with five main types of DMs, including denoising diffusion probabilistic models

Diffusion Models → vulnerableto → Backdoor Attack

confidence 100% · Recent studies have shown that DMs are prone to a wide range of attacks, including... backdoor injection

Diffusion Models → vulnerableto → Membership Inference Attack

confidence 100% · In terms of privacy, membership inference can be launched to detect whether a particular example was included in the training dataset of DMs.

Diffusion Models → vulnerableto → Adversarial Attack

confidence 100% · they can still craft the inputs of eligible DMs to make it generates sensitive contents; this is referred to as adversarial attack

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Diffusion models (DMs) have achieved state-of-the-art performance on various generative tasks such as image synthesis, text-to-image, and text-guided image-to-image generation. However, the more powerful the DMs, the more harmful they potentially are. Recent studies have shown that DMs are prone to a wide range of attacks, including adversarial attacks, membership inference, backdoor injection, and various multi-modal threats. Since numerous pre-trained DMs are published widely on the Internet, potential threats from these attacks are especially detrimental to the society, making DM-related security a worth investigating topic. Therefore, in this paper, we conduct a comprehensive survey on the security aspect of DMs, focusing on various attack and defense methods for DMs. First, we present crucial knowledge of DMs with five main types of DMs, including denoising diffusion probabilistic models, denoising diffusion implicit models, noise conditioned score networks, stochastic differential equations, and multi-modal conditional DMs. We further survey a variety of recent studies investigating different types of attacks that exploit the vulnerabilities of DMs. Then, we thoroughly review potential countermeasures to mitigate each of the presented threats. Finally, we discuss open challenges of DM-related security and envision certain research directions for this topic.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

262,546 characters extracted from source content.

Expand or collapse full text

Attacks and Defenses for Generative Diffusion Models: A Comprehensive Survey †thanks: V. T. Truong, L. B. Dang, and L. B. Le are with INRS, University of Québec, Montréal, QC H5A 1K6, Canada (email: tuan.vu.truong@inrs.ca, ba.luan.dang@inrs.ca, long.le@inrs.ca). Vu Tuan Truong, Luan Ba Dang, and Long Bao Le Abstract Diffusion models (DMs) have achieved state-of-the-art performance on various generative tasks such as image synthesis, text-to-image, and text-guided image-to-image generation. However, the more powerful the DMs, the more harmful they potentially are. Recent studies have shown that DMs are prone to a wide range of attacks, including adversarial attacks, membership inference, backdoor injection, and various multi-modal threats. Since numerous pre-trained DMs are published widely on the Internet, potential threats from these attacks are especially detrimental to the society, making DM-related security a worth investigating topic. Therefore, in this paper, we conduct a comprehensive survey on the security aspect of DMs, focusing on various attack and defense methods for DMs. First, we present crucial knowledge of DMs with five main types of DMs, including denoising diffusion probabilistic models, denoising diffusion implicit models, noise conditioned score networks, stochastic differential equations, and multi-modal conditional DMs. We further survey a variety of recent studies investigating different types of attacks that exploit the vulnerabilities of DMs. Then, we thoroughly review potential countermeasures to mitigate each of the presented threats. Finally, we discuss open challenges of DM-related security and envision certain research directions for this crucial topic. Index Terms: Diffusion models, multi-modal threats, diffusion model security, backdoor attack, membership inference, adversarial attacks. I Introduction In recent years, diffusion models (DMs)[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] have showcased remarkable capability in a wide range of generative tasks, setting the new state-of-the-art among other categories of deep generative models such as generative adversarial networks (GANs)[12], variational autoencoders (VAEs)[13, 14], and energy-based models (EBMs)[15]. In general, DMs consist of two main processes. The forward (diffusion) process progressively adds noise to the original data to gradually diffuse the data distribution into the standard Gaussian distribution. The reverse (generative) process employs a deep neural network, which is often a UNet[16], to reverse the diffusion, reconstructing the data from the Gaussian noise. With its impressive potentials, DMs have been investigated in various domains, including computer vision[17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], natural language processing (NLP)[29, 30, 31, 32, 33, 34], audio processing[35, 36], 3D generation[37, 38, 39, 40, 41, 42], bioinformatics[43, 44], and time series tasks[45, 46, 47]. DMs can be classified into different categories based on their diffusion and generative processes. The first category, inspired by the non-equilibrium thermodynamics theory[1], comprises of denoising diffusion probabilistic models (DDPMs)[2, 1, 3, 4]. DDPMs can be viewed as a Markovian Hierarchical VAE, in which the diffusion process is modeled as a Markov chain with multiple consecutive VAEs. Each diffusion step corresponds to the encoding process of a VAE, while each denoising step can be considered a decoding operation of the corresponding VAE. On the other hand, denoising diffusion implicit models (DDIMs), a variant of DDPMs, take a non-Markovian approach that allows the models to skip steps in the denoising process, thus increasing the generating speed with a certain trade-off for quality. Another category of DMs is noise conditioned score networks (NCSNs)[6, 10, 7], in which a neural network is trained based on score matching[48] to learn the score function (i.e., the gradient of the log likelihood) of the true data distribution. This score function points towards the data space that the training data inhabits. Therefore, by following the score, well-trained NCSNs can generate new samples accordingly to the true data distribution. This process can also be viewed as a denoising process[49]. The final main category, score-based stochastic differential equation (SDE), encapsulates both DDPMs and NCSNs into a generalized form. While the forward process maps data to a noise distribution via an SDE, the reverse process uses the reverse-time SDE[50] to generate samples from noise. Moreover, cross-attention technique[51] can be used to constraint the denoising neural network with multi-modal conditions such as text and images, forcing the denoising process to generate results that follow the given conditions. This sparked a wide variety of multi-modal generative tasks such as text-to-image and text-guided image-to-image generation[9]. TABLE I: A comparison between our study and existing surveys regarding DMs and its applications Survey DM-Related Topic Key Contents [52] DM Improvements Efficient sampling, improved likelihood, DMs for special-structure data, DM-based applications, connection between DMs and other generative models [53] DM Improvements Sampling acceleration, diffusion process design, likelihood optimization, bridging distributions, DM-based general applications [54] Computer Vision Computer vision applications, including text-to-image & image-to-image generation, image super-resolution, editing, segmentation, anomaly detection [55] NLP DM-based NLP applications such as machine translation, text generation, image captioning, summarization, completion, style transfer, paraphrasing [56] Medical DMs in medical imaging: Clinical image translation, reconstruction, registration, classification, segmentation, denoising, generation, anomaly detection [57] Time Series DMs for time-series applications, including time-series forecasting, imputation, generation, and further applications in other domains Ours DM-Related Security Security of DMs, including different types of attacks (e.g., backdoor, membership inference, adversarial), the corresponding countermeasures, and DM-based attacking/defense methods Despite these remarkable potentials, DMs are especially vulnerable to various security and privacy risks due to the following reasons: (i) robust DMs are often trained on large-scale data collected from diverse open sources, which might include poisoned or backdoored data; (i) pre-trained DMs are published widely on open platforms like HuggingFace111https://huggingface.co/, making it easier for hackers to spread their manipulated models. For instance, by manipulating the training data and modifying the training objective, attackers can embed a backdoor trigger into DMs to conduct a backdoor attack[58, 59, 60, 61, 62, 63, 64, 65, 66, 67]. Consequently, once the trigger is fed into the backdoored DM during inference, it will consistently produce a specific result that is designated by the attackers (e.g., a sensitive image or a violent text). Even in a more secure setting in which the attackers cannot modify the DMs’ parameters, they can still craft the inputs of eligible DMs to make it generates sensitive contents; this is referred to as adversarial attack[68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84]. In terms of privacy, membership inference can be launched to detect whether a particular example was included in the training dataset of DMs. This is especially dangerous when the training data is highly sensitive (e.g., medical images). Furthermore, as DMs are also used in various security applications such as adversarial purification and robustness certification, attacking the DMs integrated in such applications can disable the entire DM-based security systems[85, 86]. Since DMs are gaining significant attention and various DM-based applications have been used widely by the public, it is undeniable that security of DMs is an important research direction. However, existing surveys on DMs mostly exploit its developments in terms of architecture improvements, performance, and applications, while the security aspect of DMs is totally neglected. For instance, the authors in[53] surveyed a wide range of algorithm improvements for DMs, including the improvements in sampling acceleration, diffusion process design, likelihood optimization, and bridging distributions. Besides, they also reviewed various applications of DMs such as image/video generation, medical analysis, text generation, and audio generation. Similarly, the survey[52] also discussed the applications and development of DMs, with a significant attention on efficient sampling methods and improved likelihood. Furthermore, the authors investigated thoroughly the connection between DMs and other categories of deep generative models like VAEs, GANs, and EBMs. In terms of application-centric surveys, there are multiple surveys study DM-based applications, including computer vision[54], NLP[55], medical imaging[56], and time-series applications[57]. As none of existing surveys investigates the security aspect of DMs, this paper aims to fill the gap by providing a systematic and comprehensive overview of state-of-the-art research studies in this crucial topic. By categorizing different types of DM-targeted attacks and presenting countermeasures for tackling these attacks, we hope this survey provides a helpful guideline for researchers to explore and develop state-of-the-art security methods for DMs. The contribution of the paper can be summarized as follows: • We provide readers with necessary background knowledge of different types of DMs, including DDPM, DDIM, NCSN, SDE, and multi-modal conditional DMs. We demonstrate how different categories of DMs relate to each other under a consistent diffusion principle. • We investigate a wide range of attacks on DMs, categorized into three main groups, including backdoor attacks, membership inference attacks (MIAs), and adversarial attacks. Each attack is categorized further into sub-groups based on the corresponding methods/ applications. • We survey various countermeasures for DM-targeted attacks based on state-of-the-art research studies in that field. • We discuss multiple open challenges in this topic and envision some interesting research directions to improve the security aspect of DMs and DM-based applications. Table I presents a comparison between our work and existing DM-related surveys, emphasizing our contributions. The rest of this paper is presented as follows. Section I provides preliminaries of different types of DMs and background knowledge of DM security. Section I surveys state-of-the-art methods for attacking DMs and DM-based systems. Then, different countermeasures for the presented attacks are discussed in Section IV. Section V discusses various open challenges and future research directions on this field, while Section VI concludes our survey. I Background Knowledge DM is a type of deep generative models that learn to generate samples from random noise via two processes, which are the forward diffusion process and the reverse process. These processes are illustrated in Fig. 1(a). Given a training data sample such as an image, the diffusion process progressively adds noise into the image. After T noising steps, the original training data sample at step 0 is completely destroyed, resulting in a standard Gaussian noise at step T. Then, in the denoising process, a deep neural network, which is often a UNet[16], is trained to predict and remove the noise gradually from step T back to step 00, thus reconstructing the original image. After finishing the training process on a large-scale dataset, the UNet can generate new samples from any arbitrary Gaussian noise, in which the generated samples lie on the space of the original training data. In practice, the denoising process can be viewed from different perspectives according to three main categories of DMs, which are DDPM, NCSN, and score-based SDE. Although different categories of DMs result in varied generation performance, they all can be interpreted via the presented noising and denoising processes. Besides the above three categories, this section also presents two other important variants of DMs. The first one is DDIM, a faster and non-Markovian version of DDPM, while the second one is multi-modal DMs, which are DMs with additional multi-modal conditions constrained by the UNet’s cross-attention layers (e.g., stable/latent diffusion). (a) The general processes of typical DMs. (b) Denoising Diffusion Probabilistic Models. (c) Denoising Diffusion Implicit Models. In this example, the DDIM skips one step after every single step, thus doubling the speed. (d) Score Stochastic Differential Equation. (e) Noise Conditioned Score Networks. Figure 1: Diffusion models viewed from different perspectives according to four main categories, including DDPMs, DDIMs, NCSNs, and SDE. I-A Denoising Diffusion Probabilistic Models (DDPMs) DDPMs model both the forward and reverse processes as Markov chains, illustrated in Fig. 1(b). This means that the result of each step only depends on its previous step, and there is a stochastic factor during any step transition. I-A1 Forward Process The forward process of DDPMs is modelled as a Markov chain, which progressively adds noise to the clear data sample x0subscript0x_0x0 in a total of T steps to generate a sequence of noisy distributions x1,x2,…,xTsubscript1subscript2…subscriptx_1,x_2,...,x_Tx1 , x2 , … , xitalic_T. The transition between two consecutive diffusion steps is defined as: q⁢(xt|xt−1)=⁢(xt;αt⁢xt−1,(1−αt)⁢),conditionalsubscriptsubscript1subscriptsubscriptsubscript11subscriptq(x_t|x_t-1)=N(x_t; _tx_t-1,(1- _t)% I),q ( xitalic_t | xitalic_t - 1 ) = N ( xitalic_t ; square-root start_ARG αitalic_t end_ARG xitalic_t - 1 , ( 1 - αitalic_t ) I ) , (1) where αt∈(0,1)subscript01 _t∈(0,1)αitalic_t ∈ ( 0 , 1 ) is the noise schedule that determines the noise scale at each diffusion step t. While αtsubscript _tαitalic_t can be either a fixed or a learnable hyperparameter, it must be chosen such that the final distribution xTsubscriptx_Txitalic_T becomes a standard Gaussian, i.e., p⁢(xT)=⁢(xT;0,)subscriptsubscript0p(x_T)=N(x_T;0,I)p ( xitalic_T ) = N ( xitalic_T ; 0 , I ). In general, αtsubscript _tαitalic_t often decreases over time, which means that the noise increases gradually from step 00 to step T. Then, we can apply the reparameterization trick on (1) to sample xtsubscriptx_txitalic_t from xt−1subscript1x_t-1xitalic_t - 1 with a sampled noise ϵ∼⁢(0,)similar-toitalic-ϵ0ε (0,I)ϵ ∼ N ( 0 , I ): xt=αt⁢xt−1+1−αt⁢ϵ.subscriptsubscriptsubscript11subscriptitalic-ϵx_t= _tx_t-1+ 1- _tε.xitalic_t = square-root start_ARG αitalic_t end_ARG xitalic_t - 1 + square-root start_ARG 1 - αitalic_t end_ARG ϵ . (2) By repeating the reparameterization trick recursively based on (2), the choice of (1) leads to an important property that allows the direct sampling of xtsubscriptx_txitalic_t from the clear data x0subscript0x_0x0: q⁢(xt|x0)=⁢(xt;α¯t⁢x0,(1−α¯t)⁢)conditionalsubscriptsubscript0subscriptsubscript¯subscript01subscript¯q(x_t|x_0)=N(x_t; α_tx_0,(1- α_% t)I)q ( xitalic_t | x0 ) = N ( xitalic_t ; square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 , ( 1 - over¯ start_ARG α end_ARGt ) I ) (3) xt=α¯t⁢x0+1−α¯t⁢ϵ0,subscriptsubscript¯subscript01subscript¯subscriptitalic-ϵ0x_t= α_tx_0+ 1- α_t _0,xitalic_t = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ0 , (4) where α¯t=∏i=1tαisubscript¯superscriptsubscriptproduct1subscript α_t= _i=1^t _iover¯ start_ARG α end_ARGt = ∏i = 1t αitalic_i and ϵ0∼⁢(0,)similar-tosubscriptitalic-ϵ00 _0 (0,I)ϵ0 ∼ N ( 0 , I ). I-A2 Reverse Process The reverse process uses deep neural networks parameterized by θ with the following denoising transitions: pθ⁢(xt−1|xt)=⁢(xt−1;μθ⁢(xt,t),Σθ⁢(xt,t)),subscriptconditionalsubscript1subscriptsubscript1subscriptsubscriptsubscriptΣsubscriptp_θ(x_t-1|x_t)=N(x_t-1; _θ(x_t,t), _% θ(x_t,t)),pitalic_θ ( xitalic_t - 1 | xitalic_t ) = N ( xitalic_t - 1 ; μitalic_θ ( xitalic_t , t ) , Σitalic_θ ( xitalic_t , t ) ) , (5) where μθ⁢(xt,t)subscriptsubscript _θ(x_t,t)μitalic_θ ( xitalic_t , t ) and Σθ⁢(xt,t)subscriptΣsubscript _θ(x_t,t)Σitalic_θ ( xitalic_t , t ) are the mean and variance parameterized by a deep neural network. The neural network approximates the data distribution via the log likelihood: log⁡pθ⁢(x0)=log⁢∫pθ⁢(x0:T)⁢x1:T,subscriptsubscript0subscriptsubscript:0differential-dsubscript:1 p_θ(x_0)= p_θ(x_0:T)dx_1:T,log pitalic_θ ( x0 ) = log ∫ pitalic_θ ( x0 : T ) d x1 : T , (6) with pθ⁢(x0:T)=p⁢(xT)⁢∏t=1Tpθ⁢(xt−1|xt)subscriptsubscript:0subscriptsuperscriptsubscriptproduct1subscriptconditionalsubscript1subscriptp_θ(x_0:T)=p(x_T) _t=1^Tp_θ(x_t-1|x_t)pitalic_θ ( x0 : T ) = p ( xitalic_T ) ∏t = 1T pitalic_θ ( xitalic_t - 1 | xitalic_t ). TABLE I: A summary of training objectives and sampling processes for each category of DMs. Category Training Objective Sampling/Inference DDPM ℒsimpleddpm=ℒddim=x0,ϵ0⁢[‖ϵ0−ϵθ⁢(xt,t)‖22]subscriptsuperscriptℒddpmsimplesuperscriptℒddimsubscriptsubscript0subscriptitalic-ϵ0delimited-[]subscriptsuperscriptnormsubscriptitalic-ϵ0subscriptitalic-ϵsubscript22L^ddpm_simple=L^ddim= _% x_0, _0 [ \| _0- _θ(x_t,t) % \|^2_2 ]Lddpmsimple = Lddim = italic_x start_POSTSUBSCRIPT 0 , ϵ0 end_POSTSUBSCRIPT [ ∥ ϵ0 - ϵitalic_θ ( xitalic_t , t ) ∥22 ] xt−1=1αt⁢(xt−1−αt1−α¯t⁢ϵθ⁢(xt,t))+β~t⁢ϵsubscript11subscriptsubscript1subscript1subscript¯subscriptitalic-ϵsubscriptsubscript~italic-ϵx_t-1= 1 _t(x_t- 1- _t 1- % α_t _θ(x_t,t))+ β_t _t - 1 = divide start_ARG 1 end_ARG start_ARG square-root start_ARG αitalic_t end_ARG end_ARG ( xitalic_t - divide start_ARG 1 - αitalic_t end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG end_ARG ϵitalic_θ ( xitalic_t , t ) ) + square-root start_ARG over~ start_ARG β end_ARGt end_ARG ϵ DDIM xt−1=α¯t−1⁢(xt−1−α¯t⁢ϵ0(θ)⁢(xt,t)α¯t)+1−α¯t−1⁢ϵ0(θ)⁢(xt,t)subscript1subscript¯1subscript1subscript¯superscriptsubscriptitalic-ϵ0subscriptsubscript¯1subscript¯1superscriptsubscriptitalic-ϵ0subscriptx_t-1= α_t-1 ( x_t- 1- α_t% _0^(θ)(x_t,t) α_t )+ 1- % α_t-1 _0^(θ)(x_t,t)xitalic_t - 1 = square-root start_ARG over¯ start_ARG α end_ARGt - 1 end_ARG ( divide start_ARG xitalic_t - square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ0( θ ) ( xitalic_t , t ) end_ARG start_ARG square-root start_ARG over¯ start_ARG α end_ARGt end_ARG end_ARG ) + square-root start_ARG 1 - over¯ start_ARG α end_ARGt - 1 end_ARG ϵ0( θ ) ( xitalic_t , t ) NCSN ℒncsn=12⁢T⁢∑t=1Tλ⁢(σt)⁢p,x~⁢‖sθ⁢(x~,σt)+x~−xσt2‖22superscriptℒncsn12superscriptsubscript1subscriptsubscript~subscriptsuperscriptnormsubscript~subscript~superscriptsubscript222L^ncsn= 12T _t=1^Tλ( _t)% _p, x \|s_θ( x, _t)+ % x-x _t^2 \|^2_2Lncsn = divide start_ARG 1 end_ARG start_ARG 2 T end_ARG ∑t = 1T λ ( σitalic_t ) italic_p , over~ start_ARG x end_ARG ∥ sitalic_θ ( over~ start_ARG x end_ARG , σitalic_t ) + divide start_ARG over~ start_ARG x end_ARG - x end_ARG start_ARG σitalic_t2 end_ARG ∥22 x~t=x~t−1+γt2⁢sθ⁢(x~t−1,σt)+γt⁢ϵsubscript~subscript~1subscript2subscriptsubscript~1subscriptsubscriptitalic-ϵ x_t= x_t-1+ _t2s_θ(% x_t-1, _t)+ _t ~ start_ARG x end_ARGt = over~ start_ARG x end_ARGt - 1 + divide start_ARG γitalic_t end_ARG start_ARG 2 end_ARG sitalic_θ ( over~ start_ARG x end_ARGt - 1 , σitalic_t ) + square-root start_ARG γitalic_t end_ARG ϵ SDE ℒsde=t,p0,pt[λ(t)∥sθ(xt,t)−∇xtlogpt(xt|x0)∥22]L^sde= _t,p_0,p_t [λ(t) \|s_% θ(x_t,t)- _x_t p_t(x_t|x_0) \|^2_2 ]Lsde = italic_t , p start_POSTSUBSCRIPT 0 , pitalic_t end_POSTSUBSCRIPT [ λ ( t ) ∥ sitalic_θ ( xitalic_t , t ) - ∇x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT log pitalic_t ( xitalic_t | x0 ) ∥22 ] ∂x=[f⁢(x,t)−g⁢(t)2⁢∇xlog⁡pt⁢(x)]⁢∂t+g⁢(t)⁢∂w¯delimited-[]superscript2subscript∇subscript¯∂ x=[f(x,t)-g(t)^2 _x p_t(x)]∂ t+g(t)∂ % w∂ x = [ f ( x , t ) - g ( t )2 ∇x log pitalic_t ( x ) ] ∂ t + g ( t ) ∂ over¯ start_ARG w end_ARG I-A3 Training and Sampling Using Jensen’s inequality, the problem of maximizing the log likelihood in (6) can be interpreted as maximizing its evidence lower bound (ELBO), resulting in the following training objective: ℒddpm=q[−logpθ(x0|x1)+DKL(q(xT|x0)||p(xT))+∑t=2TDKL(q(xt−1|xt,x0)||pθ(xt−1|xt))], splitL^ddpm= _q[- p_θ(x_0|x_% 1)+D_KL(q(x_T|x_0)||p(x_T))\\ + _t=2^TD_KL(q(x_t-1|x_t,x_0)||p_θ(x_t-1|x_t)% )], splitstart_ROW start_CELL Lddpm = italic_q [ - log pitalic_θ ( x0 | x1 ) + DKL ( q ( xitalic_T | x0 ) | | p ( xitalic_T ) ) end_CELL end_ROW start_ROW start_CELL + ∑t = 2T DKL ( q ( xitalic_t - 1 | xitalic_t , x0 ) | | pitalic_θ ( xitalic_t - 1 | xitalic_t ) ) ] , end_CELL end_ROW (7) where DKLsubscriptKLD_KLDKL denotes the KL divergence between two distributions. It can be seen that the second term in (7) can be ignored since it has no trainable parameter. Although the first term can be optimized using Monte Carlo estimate, it is just a single denoising step, which is totally dominated by the final term which consists of T−11T-1T - 1 steps; thus the first term can be ignored in practice. Consequently, only the final summation term remains in the training objective. To optimize this final term, we respectively analyze q⁢(xt−1|xt,x0)conditionalsubscript1subscriptsubscript0q(x_t-1|x_t,x_0)q ( xitalic_t - 1 | xitalic_t , x0 ) and pθ⁢(xt−1|xt)subscriptconditionalsubscript1subscriptp_θ(x_t-1|x_t)pitalic_θ ( xitalic_t - 1 | xitalic_t ). By applying the Bayes rule, we obtain: q⁢(xt−1|xt,x0)=q⁢(xt|xt−1,x0)⁢q⁢(xt−1|x0)q⁢(xt|x0).conditionalsubscript1subscriptsubscript0conditionalsubscriptsubscript1subscript0conditionalsubscript1subscript0conditionalsubscriptsubscript0q(x_t-1|x_t,x_0)= q(x_t|x_t-1,x_0)q(x_t-1|x_0)q(x_t|x% _0).q ( xitalic_t - 1 | xitalic_t , x0 ) = divide start_ARG q ( xitalic_t | xitalic_t - 1 , x0 ) q ( xitalic_t - 1 | x0 ) end_ARG start_ARG q ( xitalic_t | x0 ) end_ARG . (8) We can derive both q⁢(xt−1|x0)conditionalsubscript1subscript0q(x_t-1|x_0)q ( xitalic_t - 1 | x0 ) and q⁢(xt|x0)conditionalsubscriptsubscript0q(x_t|x_0)q ( xitalic_t | x0 ) from (3), while q⁢(xt|xt−1,x0)conditionalsubscriptsubscript1subscript0q(x_t|x_t-1,x_0)q ( xitalic_t | xitalic_t - 1 , x0 ) is derived from (1). As a result, the above posterior can be expressed as a Gaussian distribution q⁢(xt−1|xt,x0)=⁢(xt−1;μq⁢(xt,ϵ0),Σq⁢(t))conditionalsubscript1subscriptsubscript0subscript1subscriptsubscriptsubscriptitalic-ϵ0subscriptΣq(x_t-1|x_t,x_0)=N(x_t-1; _q(x_t, _0), _% q(t))q ( xitalic_t - 1 | xitalic_t , x0 ) = N ( xitalic_t - 1 ; μitalic_q ( xitalic_t , ϵ0 ) , Σitalic_q ( t ) ), with: μq⁢(xt,ϵ0)=1αt⁢xt−1−αt1−α¯t⁢αt⁢ϵ0.subscriptsubscriptsubscriptitalic-ϵ01subscriptsubscript1subscript1subscript¯subscriptsubscriptitalic-ϵ0 _q(x_t, _0)= 1 _tx_t- 1- _t% 1- α_t _t _0.μitalic_q ( xitalic_t , ϵ0 ) = divide start_ARG 1 end_ARG start_ARG square-root start_ARG αitalic_t end_ARG end_ARG xitalic_t - divide start_ARG 1 - αitalic_t end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG square-root start_ARG αitalic_t end_ARG end_ARG ϵ0 . (9) Σq⁢(t)=σq2⁢(t)=(1−αt)⁢(1−α¯t−1)1−α¯tsubscriptΣsuperscriptsubscript21subscript1subscript¯11subscript¯ _q(t)= _q^2(t)= (1- _t)(1- α_t-1)1-% α_tΣitalic_q ( t ) = σitalic_q2 ( t ) = divide start_ARG ( 1 - αitalic_t ) ( 1 - over¯ start_ARG α end_ARGt - 1 ) end_ARG start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG (10) In terms of pθ⁢(xt−1|xt)subscriptconditionalsubscript1subscriptp_θ(x_t-1|x_t)pitalic_θ ( xitalic_t - 1 | xitalic_t ) (5), Ho et al. [2] opt to fix the variance Σθ⁢(xt,t)subscriptΣsubscript _θ(x_t,t)Σitalic_θ ( xitalic_t , t ) to Σq⁢(t)subscriptΣ _q(t)Σitalic_q ( t ), while the mean μθ⁢(xt,t)subscriptsubscript _θ(x_t,t)μitalic_θ ( xitalic_t , t ) is chosen with the same form of μq⁢(xt,ϵ0)subscriptsubscriptsubscriptitalic-ϵ0 _q(x_t, _0)μitalic_q ( xitalic_t , ϵ0 ) but the noise ϵ0subscriptitalic-ϵ0 _0ϵ0 is replaced by the neural network’s predicted noise ϵθsubscriptitalic-ϵ _θϵitalic_θ: μθ⁢(xt,t)=1αt⁢xt−1−αt1−α¯t⁢αt⁢ϵθ⁢(xt,t),subscriptsubscript1subscriptsubscript1subscript1subscript¯subscriptsubscriptitalic-ϵsubscript _θ(x_t,t)= 1 _tx_t- 1- _t% 1- α_t _t _θ(x_t,t),μitalic_θ ( xitalic_t , t ) = divide start_ARG 1 end_ARG start_ARG square-root start_ARG αitalic_t end_ARG end_ARG xitalic_t - divide start_ARG 1 - αitalic_t end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG square-root start_ARG αitalic_t end_ARG end_ARG ϵitalic_θ ( xitalic_t , t ) , (11) where ϵθ⁢(xt,t)subscriptitalic-ϵsubscript _θ(x_t,t)ϵitalic_θ ( xitalic_t , t ) is the noise predicted by the neural network. It is showed that the KL divergence between two Gaussian distributions with the same variance can be reduced to the difference between their means[87]. Thus, the training objective of DDPMs becomes: ℒddpmsuperscriptℒddpm ^ddpmLddpm =q[DKL(q(xt−1|xt,x0)||pθ(xt−1|xt))] = _q [D_KL(q(x_t-1|x_t,x_0)||p_% θ(x_t-1|x_t)) ]= italic_q [ DKL ( q ( xitalic_t - 1 | xitalic_t , x0 ) | | pitalic_θ ( xitalic_t - 1 | xitalic_t ) ) ] (12) =x0,ϵ0⁢[12⁢σq2⁢(t)⁢‖μq⁢(xt,ϵ0)−μθ⁢(xt,t)‖22]absentsubscriptsubscript0subscriptitalic-ϵ0delimited-[]12superscriptsubscript2subscriptsuperscriptnormsubscriptsubscriptsubscriptitalic-ϵ0subscriptsubscript22 = _x_0, _0 [ 12 _q^2(t)% \| _q(x_t, _0)- _θ(x_t,t) \|^2_2 ]= italic_x start_POSTSUBSCRIPT 0 , ϵ0 end_POSTSUBSCRIPT [ divide start_ARG 1 end_ARG start_ARG 2 σitalic_q2 ( t ) end_ARG ∥ μitalic_q ( xitalic_t , ϵ0 ) - μitalic_θ ( xitalic_t , t ) ∥22 ] (13) =x0,ϵ0⁢[12⁢σq2⁢(t)⁢(1−αt)2(1−α¯t)⁢αt⁢‖ϵ0−ϵθ⁢(xt,t)‖22]absentsubscriptsubscript0subscriptitalic-ϵ0delimited-[]12superscriptsubscript2superscript1subscript21subscript¯subscriptsubscriptsuperscriptnormsubscriptitalic-ϵ0subscriptitalic-ϵsubscript22 = _x_0, _0 [ 12 _q^2(t)% (1- _t)^2(1- α_t) _t \| _0-% _θ(x_t,t) \|^2_2 ]= italic_x start_POSTSUBSCRIPT 0 , ϵ0 end_POSTSUBSCRIPT [ divide start_ARG 1 end_ARG start_ARG 2 σitalic_q2 ( t ) end_ARG divide start_ARG ( 1 - αitalic_t )2 end_ARG start_ARG ( 1 - over¯ start_ARG α end_ARGt ) αitalic_t end_ARG ∥ ϵ0 - ϵitalic_θ ( xitalic_t , t ) ∥22 ] (14) Here, xtsubscriptx_txitalic_t is a function of x0subscript0x_0x0 and ϵ0subscriptitalic-ϵ0 _0ϵ0, sampled from (2). Ho et al.[2] empirically showed that simplifying the objective function to the following form leads to a better sample quality: ℒsimpleddpm=x0,ϵ0⁢[‖ϵ0−ϵθ⁢(xt,t)‖22]subscriptsuperscriptℒddpmsimplesubscriptsubscript0subscriptitalic-ϵ0delimited-[]subscriptsuperscriptnormsubscriptitalic-ϵ0subscriptitalic-ϵsubscript22L^ddpm_simple= _x_0, _0 [% \| _0- _θ(x_t,t) \|^2_2 ]Lddpmsimple = italic_x start_POSTSUBSCRIPT 0 , ϵ0 end_POSTSUBSCRIPT [ ∥ ϵ0 - ϵitalic_θ ( xitalic_t , t ) ∥22 ] (15) Intuitively, the neural network is trained to predict the source noise corresponding to each time step. Then, the neural network can be used for sampling to generate samples: xt−1=1αt⁢(xt−1−αt1−α¯t⁢ϵθ⁢(xt,t))+β~t⁢ϵ,subscript11subscriptsubscript1subscript1subscript¯subscriptitalic-ϵsubscriptsubscript~italic-ϵx_t-1= 1 _t (x_t- 1- _t 1-% α_t _θ(x_t,t) )+ β_t% ε,xitalic_t - 1 = divide start_ARG 1 end_ARG start_ARG square-root start_ARG αitalic_t end_ARG end_ARG ( xitalic_t - divide start_ARG 1 - αitalic_t end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG end_ARG ϵitalic_θ ( xitalic_t , t ) ) + square-root start_ARG over~ start_ARG β end_ARGt end_ARG ϵ , (16) where β~t=(1−αt)⁢(1−α¯t−1)1−α¯tsubscript~1subscript1subscript¯11subscript¯ β_t= (1- _t)(1- α_t-1)1- % α_tover~ start_ARG β end_ARGt = divide start_ARG ( 1 - αitalic_t ) ( 1 - over¯ start_ARG α end_ARGt - 1 ) end_ARG start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG, which is the variance in (10). I-B Denoising Diffusion Implicit Models (DDIMs) In DDPMs, the number of denoising steps T is often chosen to be large (e.g., 1000 steps in[2]) to make the reverse process close to a Gaussian distribution[1]. With a low value of T, the generation results degrade significantly since the reverse process modeled with Gaussian distributions is no more a good approximation. However, such a large T leads to a low sampling/generating speed since all the steps must be performed sequentially due to the Markov chain’s property. In DDIMs, the inference process is designed to be non-Markovian so we can skip steps in the denoising process, resulting in a faster generating speed with a certain trade-off in terms of generating quality. It is described in Fig. 1(c). I-B1 Forward Process while DDPMs first choose the diffusion transition q⁢(xt|xt−1)conditionalsubscriptsubscript1q(x_t|x_t-1)q ( xitalic_t | xitalic_t - 1 ) (1), then use it to derive the posterior q⁢(xt−1|xt,x0)conditionalsubscript1subscriptsubscript0q(x_t-1|x_t,x_0)q ( xitalic_t - 1 | xitalic_t , x0 ) (8) via the Bayes rule, DDIMs take an opposite approach by choosing this posterior first: qσ⁢(xt−1|xt,x0)=⁢(xt−1;μσ⁢(xt,x0),σt2⁢),subscriptconditionalsubscript1subscriptsubscript0subscript1subscriptsubscriptsubscript0subscriptsuperscript2q_σ(x_t-1|x_t,x_0)=N(x_t-1; _σ(x_t,x_0),% σ^2_tI),qitalic_σ ( xitalic_t - 1 | xitalic_t , x0 ) = N ( xitalic_t - 1 ; μitalic_σ ( xitalic_t , x0 ) , σ2italic_t I ) , (17) where σ=[σ1,…,σT]subscript1…subscriptσ=[ _1,..., _T]σ = [ σ1 , … , σitalic_T ] is a vector of positive coefficients controlling the stochastic magnitude of the forward process, and the mean μσ⁢(xt,x0)subscriptsubscriptsubscript0 _σ(x_t,x_0)μitalic_σ ( xitalic_t , x0 ) is chosen as follows: μσ⁢(xt,x0)=α¯t−1⁢x0+1−α¯t−1−σt2⁢xt−α¯t⁢x01−α¯t.subscriptsubscriptsubscript0subscript¯1subscript01subscript¯1subscriptsuperscript2subscriptsubscript¯subscript01subscript¯ _σ(x_t,x_0)= α_t-1x_0+ 1- α_% t-1-σ^2_t x_t- α_tx_0 1- % α_t.μitalic_σ ( xitalic_t , x0 ) = square-root start_ARG over¯ start_ARG α end_ARGt - 1 end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt - 1 - σ2italic_t end_ARG divide start_ARG xitalic_t - square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG end_ARG . (18) The intentional choice of this mean function is to offer the following desirable property: qσ⁢(xt|x0)=⁢(xt;α¯t⁢x0,(1−α¯t)⁢),subscriptconditionalsubscriptsubscript0subscriptsubscript¯subscript01subscript¯q_σ(x_t|x_0)=N(x_t; α_tx_0,(1- % α_t)I),qitalic_σ ( xitalic_t | x0 ) = N ( xitalic_t ; square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 , ( 1 - over¯ start_ARG α end_ARGt ) I ) , (19) which is proved in[5]. This property is similar to the one presented in DDPMs (3). With this property, the forward transition of DDIMs can be derived based on the Bayes rule: qσ⁢(xt|xt−1,x0)=qσ⁢(xt−1|xt,x0)⁢qσ⁢(xt|x0)qσ⁢(xt−1|x0).subscriptconditionalsubscriptsubscript1subscript0subscriptconditionalsubscript1subscriptsubscript0subscriptconditionalsubscriptsubscript0subscriptconditionalsubscript1subscript0q_σ(x_t|x_t-1,x_0)= q_σ(x_t-1|x_t,x_0)q_σ% (x_t|x_0)q_σ(x_t-1|x_0).qitalic_σ ( xitalic_t | xitalic_t - 1 , x0 ) = divide start_ARG qitalic_σ ( xitalic_t - 1 | xitalic_t , x0 ) qitalic_σ ( xitalic_t | x0 ) end_ARG start_ARG qitalic_σ ( xitalic_t - 1 | x0 ) end_ARG . (20) It can be seen that each diffusion step t of DDIMs depends on both xt−1subscript1x_t-1xitalic_t - 1 and x0subscript0x_0x0, making it non-Markovian. I-B2 Reverse Process By recursively applying the reparameterization trick on (19), we have that: xt=α¯t⁢x0+1−α¯t⁢ϵ0.subscriptsubscript¯subscript01subscript¯subscriptitalic-ϵ0x_t= α_tx_0+ 1- α_t _0.xitalic_t = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ0 . (21) We can train a neural network to predict the source noise ϵ0subscriptitalic-ϵ0 _0ϵ0, thus predicting the corresponding x0subscript0x_0x0 from (21): x0(θ)⁢(xt,t)=xt−1−α¯t⁢ϵ0(θ)⁢(xt,t)α¯t,superscriptsubscript0subscriptsubscript1subscript¯superscriptsubscriptitalic-ϵ0subscriptsubscript¯x_0^(θ)(x_t,t)= x_t- 1- α_t _0^(% θ)(x_t,t) α_t,x0( θ ) ( xitalic_t , t ) = divide start_ARG xitalic_t - square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ0( θ ) ( xitalic_t , t ) end_ARG start_ARG square-root start_ARG over¯ start_ARG α end_ARGt end_ARG end_ARG , (22) where ϵ0(θ)⁢(xt,t)superscriptsubscriptitalic-ϵ0subscript _0^(θ)(x_t,t)ϵ0( θ ) ( xitalic_t , t ) and x0(θ)⁢(xt,t)superscriptsubscript0subscriptx_0^(θ)(x_t,t)x0( θ ) ( xitalic_t , t ) are the source noise ϵ0subscriptitalic-ϵ0 _0ϵ0 and the denoised observation x0subscript0x_0x0, respectively, predicted by the neural network with the parameter θ. Consequently, the reverse process is defined by leveraging the knowledge of qσ⁢(xt−1|xt,x0)subscriptconditionalsubscript1subscriptsubscript0q_σ(x_t-1|x_t,x_0)qitalic_σ ( xitalic_t - 1 | xitalic_t , x0 ) in (17), with x0subscript0x_0x0 replaced by the predicted x0(θ)⁢(xt,t)superscriptsubscript0subscriptx_0^(θ)(x_t,t)x0( θ ) ( xitalic_t , t ): pθ⁢(xt−1|xt)subscriptconditionalsubscript1subscript p_θ(x_t-1|x_t)pitalic_θ ( xitalic_t - 1 | xitalic_t ) =qσ⁢(xt−1|xt,x0(θ)⁢(xt,t))absentsubscriptconditionalsubscript1subscriptsuperscriptsubscript0subscript =q_σ(x_t-1|x_t,x_0^(θ)(x_t,t))= qitalic_σ ( xitalic_t - 1 | xitalic_t , x0( θ ) ( xitalic_t , t ) ) (23) =⁢(xt−1;μσ⁢(xt,x0(θ)),σt2⁢),absentsubscript1subscriptsubscriptsuperscriptsubscript0subscriptsuperscript2 =N(x_t-1; _σ(x_t,x_0^(θ)),σ^% 2_tI),= N ( xitalic_t - 1 ; μitalic_σ ( xitalic_t , x0( θ ) ) , σ2italic_t I ) , (24) where the mean μσ⁢(⋅)subscript⋅ _σ(·)μitalic_σ ( ⋅ ) is defined in (18). I-B3 Training and Sampling The neural network θ is also optimized by maximizing the log likelihood presented in (6), which leads to the following ELBO objective: ℒσddim=qσ[DKL(qσ(xT|x0)||p(xT))−logpθ(x0|x1)+∑t=2TDKL(qσ(xt−1|xt,x0)||pθ(xt−1|xt))]. splitL^ddim_σ= _q_σ[D_ % KL(q_σ(x_T|x_0)||p(x_T))- p_θ(x_0|x_1)\\ + _t=2^TD_KL(q_σ(x_t-1|x_t,x_0)||p_θ(x_t-% 1|x_t))]. splitstart_ROW start_CELL Lddimitalic_σ = italic_q start_POSTSUBSCRIPT σ end_POSTSUBSCRIPT [ DKL ( qitalic_σ ( xitalic_T | x0 ) | | p ( xitalic_T ) ) - log pitalic_θ ( x0 | x1 ) end_CELL end_ROW start_ROW start_CELL + ∑t = 2T DKL ( qitalic_σ ( xitalic_t - 1 | xitalic_t , x0 ) | | pitalic_θ ( xitalic_t - 1 | xitalic_t ) ) ] . end_CELL end_ROW (25) Song et al. [5] proved that the optimal solution of ℒσddimsubscriptsuperscriptℒddimL^ddim_σLddimitalic_σ is the same as that of ℒsimpleddpmsubscriptsuperscriptℒddpmsimpleL^ddpm_simpleLddpmsimple in (15). In terms of sampling, xt−1subscript1x_t-1xitalic_t - 1 can be sampled from xtsubscriptx_txitalic_t by applying the reparameterization trick on (24): xt−1=α¯t−1⁢(xt−1−α¯t⁢ϵ0(θ)⁢(xt,t)α¯t)+1−α¯t−1−σt2⁢ϵ0(θ)⁢(xt,t)+σt⁢ϵt,subscript1subscript¯1subscript1subscript¯superscriptsubscriptitalic-ϵ0subscriptsubscript¯1subscript¯1superscriptsubscript2superscriptsubscriptitalic-ϵ0subscriptsubscriptsubscriptitalic-ϵ splitx_t-1= α_t-1 ( x_t- 1- % α_t _0^(θ)(x_t,t) α_t )% \\ + 1- α_t-1- _t^2 _0^(θ)(x_t,t)+% _t _t, splitstart_ROW start_CELL xitalic_t - 1 = square-root start_ARG over¯ start_ARG α end_ARGt - 1 end_ARG ( divide start_ARG xitalic_t - square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ0( θ ) ( xitalic_t , t ) end_ARG start_ARG square-root start_ARG over¯ start_ARG α end_ARGt end_ARG end_ARG ) end_CELL end_ROW start_ROW start_CELL + square-root start_ARG 1 - over¯ start_ARG α end_ARGt - 1 - σitalic_t2 end_ARG ϵ0( θ ) ( xitalic_t , t ) + σitalic_t ϵitalic_t , end_CELL end_ROW (26) where ϵt∼⁢(,)similar-tosubscriptitalic-ϵ0 _t (0,I)ϵitalic_t ∼ N ( 0 , I ). Especially, (26) is equivalent to (16) if we choose σt=β~tsubscriptsubscript~ _t= β_tσitalic_t = over~ start_ARG β end_ARGt, with β~tsubscript~ β_tover~ start_ARG β end_ARGt defined below the equation (16). In this case, it becomes a DDPM. On the other hand, it becomes a DDIM if we choose σt=0subscript0 _t=0σitalic_t = 0. This leads to a deterministic forward process as no noise is added in each step. Since the generative process is now non-Markovian, we can skip steps to accelerate the generating speed. For example, DDIMs can generate 2 times faster than DDPMs if the sequence of time steps 2=[1,3,5,7,…,T]subscript21357…T_2=[1,3,5,7,...,T]T2 = [ 1 , 3 , 5 , 7 , … , T ] is used during inference, or 4 times higher than DDPMs with 4=[1,5,9,13,…,T]subscript415913…T_4=[1,5,9,13,...,T]T4 = [ 1 , 5 , 9 , 13 , … , T ]. In other words, we can train DDIMs with an arbitrary large number of time steps T, but only sample from some of them (e.g., using 2subscript2T_2T2 or 4subscript4T_4T4 for sampling) to speed up the generative process. Notably, it is known that DDPMs are trained based on 1=[1,2,3,…,T]subscript1123…T_1=[1,2,3,...,T]T1 = [ 1 , 2 , 3 , … , T ] . This means that we can take advantages of pretrained DDPMs for DDIMs[5]. I-C Noise Conditioned Score Networks (NCSNs) Unlike DDPMs and DDIMs which generate samples by predicting and removing noise, NCSNs take another approach by following the score function of the training data to generate samples. The score function of a data density p⁢(x)p(x)p ( x ) is defined as the gradient of the log probability density ∇xlog⁡p⁢(x)subscript∇ _x p(x)∇x log p ( x ). Essentially, the score function indicates the direction in the data space that one needs to move in order to maximize the likelihood of the data x. The key idea of NCSNs bases on this property, demonstrated in Fig. 1(e). Starting from an arbitrary point in the data space (i.e., a random noise), NCSNs are trained to iteratively follow the direction of the score function to move towards the high-density space that the data x inhabits, thereby generating a new sample when reaching a mode of the data distribution. This is referred to as Langevin dynamics, a method originated from physics. Specifically, each step of Langevin dynamics is computed as follows: xt=xt−1+γ2⁢∇xlog⁡p⁢(x)+γ⁢ϵ,subscriptsubscript12subscript∇italic-ϵx_t=x_t-1+ γ2 _x p(x)+ γε,xitalic_t = xitalic_t - 1 + divide start_ARG γ end_ARG start_ARG 2 end_ARG ∇x log p ( x ) + square-root start_ARG γ end_ARG ϵ , (27) where t∈0,1,…,T01…t∈\0,1,...,T\t ∈ 0 , 1 , … , T , x0subscript0x_0x0 is randomly sampled from a prior distribution, γ controls the scale of the update in each step, ϵ∼⁢(,)similar-toitalic-ϵ0ε (0,I)ϵ ∼ N ( 0 , I ) is a noise that is added to make the generated samples more diverse and stochastic instead of deterministically collapsing onto a local minimum. Sampling from (27) only requires the score function. Hence, a neural network parameterized by θ can be trained to approximate the score function such that sθ⁢(x)≈∇xlog⁡p⁢(x)subscriptsubscript∇s_θ(x)≈ _x p(x)sitalic_θ ( x ) ≈ ∇x log p ( x ). There are different techniques that can be used to train the neural network, including score matching[48], sliced score matching[88], and denoising score matching[49]. For instance, using score matching leads to the following training objective: ℒsm=p⁢‖sθ⁢(x)−∇xlog⁡p⁢(x)‖22.superscriptℒsmsubscriptsubscriptsuperscriptnormsubscriptsubscript∇22L^sm= _p \|s_θ(x)- _x p(x)% \|^2_2.Lsm = italic_p ∥ sitalic_θ ( x ) - ∇x log p ( x ) ∥22 . (28) However, there are several problems when training with this objective[6]. By taking the expectation over all examples p⁢(x)p(x)p ( x ), rarely-seen examples in low-density space would be dominated by the data in high-density regions. Consequently, score estimation becomes unreliable in low density regions. Another problem is about the manifold hypothesis. Assuming that the data is RGB images. The ambient space that consists of all possible RGB images of size H×WH× WH × W is obviously very large (with 2H×W×3×8superscript2382^H× W× 3× 82H × W × 3 × 8 possible images), while the training data often lies on only a low dimensional manifold (e.g., car images). As a result, the sampling results would always converge to the low dimensional manifold, while other points outside of this manifold would have probability zero, making the log in the score function ill-defined. Song et al.[6] proposed solving these problems by adding a multi-scale Gaussian noise into the data. This noise could push the sampled points to be outside of the low dimensional manifold, solving the manifold hypothesis problem. Moreover, it also increases the regions covered by the modes of the training data, thus covering better low-density regions, mitigating the issue of low data density. Using the noise distribution qσ⁢(x~|x)=⁢(x,σt2⁢)subscriptconditional~subscriptsuperscript2q_σ( x|x)=N(x,σ^2_tI)qitalic_σ ( over~ start_ARG x end_ARG | x ) = N ( x , σ2italic_t I ), the score function can be computed as ∇x~log⁡qσ⁢(x~|x)=−x~−xσ2subscript∇~subscriptconditional~~superscript2 _ x q_σ( x|x)=- x-x% σ^2∇over~ start_ARG x end_ARG log qitalic_σ ( over~ start_ARG x end_ARG | x ) = - divide start_ARG over~ start_ARG x end_ARG - x end_ARG start_ARG σ2 end_ARG, where x~~ xover~ start_ARG x end_ARG is a noise-perturbed version of x, and σtt=1Tsubscriptsuperscriptsubscript1\ _t\^T_t=1 σitalic_t Titalic_t = 1 is a sequence of decreasing noise levels. This results in the following objective: ℒncsn=12⁢T⁢∑t=1Tλ⁢(σt)⁢p,x~⁢‖sθ⁢(x~,σt)+x~−xσt2‖22,superscriptℒncsn12superscriptsubscript1subscriptsubscript~subscriptsuperscriptnormsubscript~subscript~superscriptsubscript222L^ncsn= 12T _t=1^Tλ( _t)% _p, x \|s_θ( x, _t)+ % x-x _t^2 \|^2_2,Lncsn = divide start_ARG 1 end_ARG start_ARG 2 T end_ARG ∑t = 1T λ ( σitalic_t ) italic_p , over~ start_ARG x end_ARG ∥ sitalic_θ ( over~ start_ARG x end_ARG , σitalic_t ) + divide start_ARG over~ start_ARG x end_ARG - x end_ARG start_ARG σitalic_t2 end_ARG ∥22 , (29) where λ⁢(⋅)⋅λ(·)λ ( ⋅ ) is a weighting function, chosen as λ⁢(σ)=σ2superscript2λ(σ)=σ^2λ ( σ ) = σ2 [6]. After training the neural network sθsubscripts_θsitalic_θ, it can be used by an annealed version of Langevin dynamics to generate samples: x~t=x~t−1+γt2⁢sθ⁢(x~t−1,σt)+γt⁢ϵ,subscript~subscript~1subscript2subscriptsubscript~1subscriptsubscriptitalic-ϵ x_t= x_t-1+ _t2s_θ(% x_t-1, _t)+ _tε,over~ start_ARG x end_ARGt = over~ start_ARG x end_ARGt - 1 + divide start_ARG γitalic_t end_ARG start_ARG 2 end_ARG sitalic_θ ( over~ start_ARG x end_ARGt - 1 , σitalic_t ) + square-root start_ARG γitalic_t end_ARG ϵ , (30) where γt=μ⁢(σtσT)2subscriptsuperscriptsubscriptsubscript2 _t=μ( _t _T)^2γitalic_t = μ ( divide start_ARG σitalic_t end_ARG start_ARG σitalic_T end_ARG )2 anneals/scales down the update over time. In practice, μ is empirically selected between 5⋅10−6⋅5superscript1065· 10^-65 ⋅ 10- 6 and 5⋅10−5⋅5superscript1055· 10^-55 ⋅ 10- 5, while σ starts from σ1=1subscript11 _1=1σ1 = 1 to σT=0.01subscript0.01 _T=0.01σitalic_T = 0.01. Intuitively, score-based generative models like NCSNs naturally link to denoising-based models (e.g., DDPMs and DDIMs) in the sense that: both (i) denoising and (i) following the score function of data enable moving in directions that maximize the log probability of the data, starting from a random noise in the data space. I-D Score Stochastic Differential Equation (SDE) While DDPMs, DDIMs, and NCSNs operate in a discrete-time scheme with T iterative steps, score SDE generalizes these models to a continuous diffusion process. As illustrated in Fig. 1(d), the forward process uses an SDE to diffuse a data example into random noise. The reverse process employs a deep neural network to approximate the reverse-time SDE[50], thus generating samples based on a numerical SDE solver such as predictor-corrector samplers and probability flow ordinary differential equation (ODE). In particular, the SDE for the diffusion process is defined as follows: ∂x=f⁢(x,t)⁢∂t+g⁢(t)⁢∂ω,∂ x=f(x,t)∂ t+g(t)∂ω,∂ x = f ( x , t ) ∂ t + g ( t ) ∂ ω , (31) where t∈[0,T]0t∈[0,T]t ∈ [ 0 , T ] is a continuous time variable, f⁢(⋅,t)⋅f(·,t)f ( ⋅ , t ) is a function computing the drift coefficient of x⁢(t)x(t)x ( t ), g⁢(⋅)⋅g(·)g ( ⋅ ) computes the diffusion coefficient, and w is the Brownian motion. The drift term represents the deterministic part of the SDE, responsible for data destruction, while the diffusion coefficient controls the scale of noise from the stochastic part ∂ω∂ω∂ ω. On the other hand, the reverse-time SDE has the following form: ∂x=[f⁢(x,t)−g⁢(t)2⁢∇xlog⁡pt⁢(x)]⁢∂t+g⁢(t)⁢∂w¯,delimited-[]superscript2subscript∇subscript¯∂ x=[f(x,t)-g(t)^2 _x p_t(x)]∂ t+g(t)∂ % w,∂ x = [ f ( x , t ) - g ( t )2 ∇x log pitalic_t ( x ) ] ∂ t + g ( t ) ∂ over¯ start_ARG w end_ARG , (32) where w¯ wover¯ start_ARG w end_ARG is the Brownian motion when the time is reversed back from T to 00. Essentially, the reverse process starts from a random noise xTsubscriptx_Txitalic_T, then reverses the time to x0subscript0x_0x0 to generate samples. From a denoising perspective, the term g⁢(t)2⁢∇xlog⁡pt⁢(x)superscript2subscript∇subscriptg(t)^2 _x p_t(x)g ( t )2 ∇x log pitalic_t ( x ) can be viewed as the noise that one needs to remove to reconstruct clear samples. Similar to NCSNs, the score function ∇xlog⁡pt⁢(x)subscript∇subscript _x p_t(x)∇x log pitalic_t ( x ) here can also be approximated by a deep neural network sθ⁢(x,t)subscripts_θ(x,t)sitalic_θ ( x , t ) trained with denoising score matching, resulting in the following objective: ℒsde=t,p0,pt[λ(t)∥sθ(xt,t)−∇xtlogpt(xt|x0)∥22],L^sde= _t,p_0,p_t [λ(t) \|s_% θ(x_t,t)- _x_t p_t(x_t|x_0) \|^2_2 ],Lsde = italic_t , p start_POSTSUBSCRIPT 0 , pitalic_t end_POSTSUBSCRIPT [ λ ( t ) ∥ sitalic_θ ( xitalic_t , t ) - ∇x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT log pitalic_t ( xitalic_t | x0 ) ∥22 ] , (33) where λ⁢(t)λ(t)λ ( t ) is a weighting function, while the time variable t is sampled over the distribution ⁢([0,T])0U([0,T])U ( [ 0 , T ] ). Besides denoising score matching[49], other methods such as sliced score matching[88] and finite-difference score matching[89] are also applicable in this case. I-E Multi-Modal Conditional Diffusion Models When a DM is trained on a particular dataset, it can generate high-quality samples that lies on the space of the training data. However, the generating results are uncontrollable as it depends on the random noise xTsubscriptx_Txitalic_T and other stochastic factors like ϵtsubscriptitalic-ϵ _tϵitalic_t during the sampling process. One way to mitigate this issue is to add a condition c at each transition step of DMs: p⁢(x0:T|c)=p⁢(xT)⁢∏t=1Tpθ⁢(xt−1|xt,c).conditionalsubscript:0subscriptsuperscriptsubscriptproduct1subscriptconditionalsubscript1subscriptp(x_0:T|c)=p(x_T) _t=1^Tp_θ(x_t-1|x_t,c).p ( x0 : T | c ) = p ( xitalic_T ) ∏t = 1T pitalic_θ ( xitalic_t - 1 | xitalic_t , c ) . (34) Here, the condition c could be in any modality such as text, image, audio, semantic map, or their representation in the latent space. For instance, in terms of DM-based image generation, once might constraint the DM with a text condition c to force it to generate images that follow the textual description provided in c. This is referred to as text-to-image generation, a popular application of stable diffusion[9]. In DDPMs and DDIMs, the neural networks are trained to predict ϵθ⁢(xt,t,c)subscriptitalic-ϵsubscript _θ(x_t,t,c)ϵitalic_θ ( xitalic_t , t , c ) that approximates the source noise ϵ0subscriptitalic-ϵ0 _0ϵ0, while the score function ∇log⁡p⁢(xt|c)∇conditionalsubscript∇ p(x_t|c)∇ log p ( xitalic_t | c ) in NCSNs and SDE models is predicted by sθ⁢(xt,t,c)subscriptsubscripts_θ(x_t,t,c)sitalic_θ ( xitalic_t , t , c ). To constraint the neural networks with such conditions, Rombach et al.[9] proposed adding cross-attention layers into the UNet backbone, in which the condition c acts as the key and value of these attention layers, while the query is the data flow forwarded along the UNet architecture. As a result, the supplemented conditions guide the neural network to produce results that follow the added constraints. This turns DMs into a flexible multi-modal generator, enabling a wide variety of generative tasks such as text-to-image, text-guided image-to-image, and text-to-3D generation. Figure 2: A summary of our survey on attack and defense methods for DMs. I-F Preliminary of Diffusion Model Attacks I-F1 Backdoor Attacks In backdoor attacks[90], the attackers modify the training data and objective function to embed a backdoor trigger into machine learning (ML) models. During inference, if the trigger is activated as input, the models will produce abnormal results driven by the attackers. On the other hand, the models still behave normally on benign samples. This stealthy property of backdoor attacks makes it challenging to be recognized by average users and even detection systems. In the context of DMs, backdoor attacks cause DMs to generate a particular image (designated by the attackers) when the trigger is used; this image is called backdoor target. The purposes of backdoor attack on DMs are diverse. For instance, since many reputable organizations are offering DMs as a service, attackers might inject poisoned data to the training process to backdoor their DMs with sensitive/violent contents as backdoor targets. Then, when the backdoored DMs are used widely by the public, the attackers could publish the backdoor triggers so that everyone can use the triggers to generate the designated sensitive contents, posing a severe threat to both the community and the organizations in terms of reputation and legality. On the other hand, attackers can train their own backdoored DMs, then publish them on open-source platforms like HuggingFace to harm users who download, use, or fine-tune the backdoored models. This risk is even more harmful for downstream applications built upon the backdoored DMs. In this case, a fault caused by the backdoored DMs can make the entire system collapses. Backdoored DMs can be used for advertisement, generating fake news, copyrights. For instance, attackers could manipulate the language model of DMs, making the DMs always generates images related to Starbucks when there is a trigger word “coffee” in the text prompt. I-F2 Adversarial Attacks Adversarial attacks on traditional models likes image classifiers have been investigated extensively in existing studies[91, 92]. In general, an adversarial attack is conducted by adding a small perturbation into the input of a neural network, causing the model to produce incorrect answers[93]. The perturbation is often learnable, while its scale is constrained to be small enough to ensure the stealthiness of the attack. For instance, regarding object detection for autopilot, an attacker might conduct an adversarial attack by learning a tiny perturbation to mislead the object-detection model. While this perturbation is too small to be recognized by human observers, it might make the autopilot system misunderstands a stop sign as a slow sign. As a result, the autopilot system decides to keep moving, potentially causing severe traffic accidents. In terms of adversarial attacks on generative DMs, a learned perturbation added into the model’s input could cause it to generate chaotic content. This type of attack can be dangerous in various situations. First, an attacker can deploy adversarial attacks to make a well-known public DM generate Not-Safe-For-Work (NSFW) content like racism, horror, politics, pornography, or violence. This not only impacts negatively on the reputation of the organization who published the attacked DM, but also poses a threat to the society in terms of ethic and morality. Second, the output of an DM can be used as input for other downstream tasks in a large-scale system. If the DM is attacked such that it produces manipulated results, it might lead to the failure of the entire system. The effect of an adversarial attack can be diverse, including image quality degradation, content shifting, sensitive content generation, and object distortion/elimination. On the other hand, adversarial attacks on DMs can also be utilized for positive purposes like copyrights protection, watermark embedding, and anti-personalization. For instance, artists can add a learned perturbation into their arts to hinder DMs from extracting the arts’ features. Consequently, if anyone uses the protected arts to train or fine-tune DMs, the DMs will generate chaotic images instead of mimicking the style of the artists. I-F3 Membership Inference Attacks In MIAs, the adversaries aim to predict whether a specific sample is a member of the training dataset of a target model. MIAs applied to ML models were first introduced by Shokri et al.[94], focusing on classification models in a black-box setting. Since then, the number of studies in this domain has grown rapidly, applying to different ML models, including classification models [94, 95, 96, 97, 98, 99, 100], GANs [101, 102, 103, 104, 105], and DMs [106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118]. In general, since ML models are trained iteratively over multiple epochs on their training datasets, they tend to remember these data points and respond differently when processing member (training) versus non-member (unseen) samples. Exploiting this, attackers attempt to extract distinguishable behaviors from the target models and use them to design attack methods for membership inference. Although there are existing MIAs successfully designed for generative models such as GANs and VAEs, these methods exhibit ineffectiveness when applied to DMs due to their intrinsic properties. For instance, DMs have a predefined, unlearnable encoding process and a multiple-step decoding (denoising) process. Consequently, attackers can leverage the model’s outputs, such as loss, predicted noise, and intermediate denoised image, at any timestep to compute the attack vectors to determine the membership. In addition, the DM’s outputs are generated through a more complex, stochastic process, which requires more delicate approaches to extract distinguishable features. DMs are often large-scale models trained on a vast amount of data scraped from the Internet and anonymized sources, which could potentially involve sensitive data such as copyrighted material, medical information, and other confidential data. Consequently, MIAs targeting DMs raise severe data privacy risks, such as the exposure of sensitive information, violations of confidentiality, and potentially re-identifying individuals in anonymized datasets. For example, by determining that a specific clinical record was used to train a DM associated with a particular disease, an attacker can infer that the owner of the clinical record has the disease with a high success rate. On the other hand, under authorized and appropriate circumstances, MIAs can be employed for various applications. For instance, identifying vulnerabilities through MIAs can help developers recognize and address weaknesses in their models, thereby enhancing their overall robustness and security. In addition, MIAs can be utilized to assess DMs’ compliance with data protection regulations such as the General Data Protection Regulation (GDPR)222https://gdpr-info.eu/. Ensuring that these models do not leak information about their training data helps in maintaining compliance. I-F4 Definitions of Terms Since technical terms regarding security are often used inconsistently in existing studies, this section provides a list of common terms used in our literature survey (section I and IV) to facilitate readers: • Backdoor Trigger: This is the input that activates the backdoor effect. It can be a text prompt in case of text-to-image generation, or an image for image-to-image DMs. • Backdoor Target: This is the output of the backdoored DMs when the backdoor trigger is fed as the input. For example, the target can be a sensitive or harmful image in case of image generation application. • Noisy Trigger: It refers to a noisy version of the backdoor trigger, which is created by fusing the clear trigger with a Gaussian noise. • Adversarial Perturbation: In terms of image, this perturbation is a small noise that has the same size with the input image. For text prompt, the perturbation is word/character that is replaced, appended, or prepended into the input prompt to active the adversarial effect. • Adversarial Image: A perturbed version of the input image that has been added the adversarial perturbation. • Adversarial Prompt: The perturbed text prompt which is used to attack the DMs. • Target Model: The DM that attackers aim to attack. • Shadow Model: Shadow models have similar structure and function to the target model and are trained to mimic the behaviors of the target model. • Attack Model: The model designed to process the information extracted from the target model and infer membership. • Member Sample: The images that are used to train the target model. • Non-Member Sample: The images that are not in the training dataset of the target model. • Auxiliary dataset: Datasets from a similar domain as the target model’s training dataset, utilized to train shadow models or attack models. I-G Threat Models I-G1 Black-Box Attack Under the black-box setting, which is the most difficult and realistic scenario, adversaries only have access to the final output of the target DMs, i.e., the generated images. They are restricted to any further information about the target model’s parameters, hyper-parameters, architecture, or other internal details. I-G2 Gray-Box Attack During the sampling process, DMs iteratively denoise a noisy image over multiple steps, producing numerous intermediate outputs. In a gray-box setting, attackers exploit this by manipulating the target model’s generation process to extract the intermediate output from the U-Net model at each timestep. However, their access to the model’s parameters or other internal information is still restricted. I-G3 White-Box Attack In this setting, attackers are assumed to have full access to the target model’s parameters, architecture, source codes, and other internal information. This scenario reflects a common practice in the open-source community, where source codes, models’ information, and pre-trained model checkpoints are publicly available and easily accessible. I Attacks on Diffusion Models Although the capability of DMs is undeniable, their danger is also inestimable when they are manipulated or intentionally used for malicious purposes. This section surveys a variety of attacks targeted on DMs with in-depth discussion. For the ease of understanding, the data of DMs in this section is considered images if no information is supplemented (although DMs can be used for many applications other than computer vision). I-A Backdoor Attacks In general, backdoor triggers can be stealthily embedded into DMs via the following main components: (i) denoising model, (i) conditional model (e.g., language model), and (i) personalization method. They are surveyed as follows. Figure 3: A comparison in terms of backdoor attacks between BadDiffusion[59] and TrojDiff[62]. I-A1 Backdoor via Denoising Model This is the main category of backdoor attacks which aims to modify the forward and reverse processes such that the neural network would learn undesirable correlation between the backdoor trigger and the backdoor target. TrojDiff [62] is among the first studies that investigated this type of backdoor attacks on DMs. Essentially, while a benign diffusion process gradually adds noise into an image x0subscript0x_0x0 until it becomes a Gaussian noise xT∼⁢(0,)similar-tosubscript0x_T (0,I)xitalic_T ∼ N ( 0 , I ), TrojDiff’s backdoored diffusion process diffuses x0subscript0x_0x0 into xT∗superscriptsubscriptx_T^*xitalic_T∗, which a noisy version of the backdoor trigger: xT∗≈(1−γ)⁢δ+γ⁢ϵ,superscriptsubscript1italic-ϵx_T^*≈(1-γ)δ+γε,xitalic_T∗ ≈ ( 1 - γ ) δ + γ ϵ , (35) where ϵ∼⁢(0,)similar-toitalic-ϵ0ε (0,I)ϵ ∼ N ( 0 , I ) is a Gaussian noise, δ is the backdoor trigger, and γ∈[0,1]01γ∈[0,1]γ ∈ [ 0 , 1 ] is a “blending” coefficient determining how noisy the noisy trigger is. For instance, as illustrated in Fig 3, the backdoor trigger δ of TrojDiff is a hello-kitty image δ, while xT∗superscriptsubscriptx_T^*xitalic_T∗ is a noisy version of such the hello-kitty image. In TrojDiff, the backdoored diffusion process is applied on only a particular image in the dataset (i.e., the backdoor target), while the process on other training images remains unchanged to retain the model’s performance. By doing so, during inference, the backdoored DMs would generate the backdoor target if the model’s input is the noisy hello-kitty image (i.e., the noisy trigger), while producing benign results if the input is a standard Gaussian noise. To do so, TrojDiff modifies the diffusion transition of DDPMs from (1) to the following form for backdooring DMs: q⁢(xt∗|xt−1∗)=⁢(xt∗;αt⁢xt−1∗+kt⁢(1−γ)⁢δ,(1−αt)⁢γ2⁢),conditionalsubscriptsuperscriptsubscriptsuperscript1subscriptsuperscriptsubscriptsubscriptsuperscript1subscript11subscriptsuperscript2q(x^*_t|x^*_t-1)=N(x^*_t; _tx^*_t-1+k_% t(1-γ)δ,(1- _t)γ^2I),q ( x∗italic_t | x∗italic_t - 1 ) = N ( x∗italic_t ; square-root start_ARG αitalic_t end_ARG x∗italic_t - 1 + kitalic_t ( 1 - γ ) δ , ( 1 - αitalic_t ) γ2 I ) , (36) where xt∗superscriptsubscriptx_t^*xitalic_t∗ denotes xtsubscriptx_txitalic_t in case of backdoor attack, and ktsubscriptk_tkitalic_t is a schedule function that determines how much the trigger δ (i.e., hello-kitty image) is added to xt−1∗superscriptsubscript1x_t-1^*xitalic_t - 1∗ to make xt∗superscriptsubscriptx_t^*xitalic_t∗ looks more like the noisy trigger. The formula of ktsubscriptk_tkitalic_t in[62] offers the following desirable property, which allows the direct sampling of xt∗superscriptsubscriptx_t^*xitalic_t∗ from x0∗superscriptsubscript0x_0^*x0∗: xt∗=α¯t⁢x0∗+1−α¯t⁢(1−γ)⁢δ⏟trigger term+1−α¯t⁢γ⁢ϵ⏟noise term,subscriptsuperscriptsubscript¯subscriptsuperscript0subscript⏟1subscript¯1trigger termsubscript⏟1subscript¯italic-ϵnoise termx^*_t= α_tx^*_0+ 1- α_t% (1-γ)δ_ trigger term+ 1- α_% tγε_ noise term,x∗italic_t = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x∗0 + under⏟ start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ( 1 - γ ) δ end_ARGtrigger term + under⏟ start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG γ ϵ end_ARGnoise term , (37) where ϵ∈⁢(0,)italic-ϵ0ε (0,I)ϵ ∈ N ( 0 , I ), and x0∗subscriptsuperscript0x^*_0x∗0 is the backdoor target. Unlike the original DDPMs which only has the noise term, there are both a noise term and a trigger term in this case, making xt∗superscriptsubscriptx_t^*xitalic_t∗ resembles the noisy trigger gradually over time. With the above property, the reverse process and training objective can be derived with the same workflow as those presented in I-A. As a result, TrojDiff achieved a high attack success rate (ASR) of more than 99% on CIFAR-10 and CelebA datasets, while the backdoor training only degrades the benign performance insignificantly (with an increase of 0.200.200.200.20 in terms of FID score). Concurrently proposed with TrojDiff, the work named BadDiffusion[59] takes another approach to backdoor the denoising model of DDPMs. As demonstrated in Fig. 3, the backdoor trigger in BadDiffusion is a special pattern (e.g., the blue glasses in the figure’s example) instead of an entire image like the hello-kitty image in TrojDiff’s example. On the other hand, the backdoor target is often an image that is out of the training data’s domain, such as the orange cat image in Fig. 3. For an image sample xssubscriptx_sxitalic_s in the training dataset, the backdoored diffusion process of BadDiffusion diffuses the backdoor target x0∗subscriptsuperscript0x^*_0x∗0 gradually into xT∗subscriptsuperscriptx^*_Tx∗italic_T at time step T: xT∗≈(xs∧δ)+ϵ=xsδ+ϵ,subscriptsuperscriptsubscriptitalic-ϵsuperscriptsubscriptitalic-ϵx^*_T≈(x_s δ)+ε=x_s^δ+ε,x∗italic_T ≈ ( xitalic_s ∧ δ ) + ϵ = xitalic_sitalic_δ + ϵ , (38) where ϵ∈⁢(0,)italic-ϵ0ε (0,I)ϵ ∈ N ( 0 , I ) is a Gaussian noise, δ is the backdoor trigger, ∧ ∧ denotes a masking operation that masks the trigger δ (e.g., blue glasses) into the sampled image xssubscriptx_sxitalic_s (e.g., human face), and xsδsuperscriptsubscriptx_s^δxitalic_sitalic_δ is the poisoned image (i.e., the face wearing the blue glasses). The difference between (35) and (38) is also the key difference between TrojDiff and BadDiffusion. Furthermore, while TrojDiff uses a separate schedule for the trigger, i.e., ktsubscriptk_tkitalic_t in equation (36), BadDiffusion takes advantage of such noise schedule αtsubscript _tαitalic_t for the backdoor trigger, resulting in the following diffusion transition: q⁢(xt∗|xt−1∗)=⁢(xt∗;αt⁢xt−1∗+(1−αt)⁢xsδ,(1−αt)⁢),conditionalsubscriptsuperscriptsubscriptsuperscript1subscriptsuperscriptsubscriptsubscriptsuperscript11subscriptsuperscriptsubscript1subscriptq(x^*_t|x^*_t-1)=N(x^*_t; _tx^*_t-1+(1% - _t)x_s^δ,(1- _t)I),q ( x∗italic_t | x∗italic_t - 1 ) = N ( x∗italic_t ; square-root start_ARG αitalic_t end_ARG x∗italic_t - 1 + ( 1 - square-root start_ARG αitalic_t end_ARG ) xitalic_sitalic_δ , ( 1 - αitalic_t ) I ) , (39) where xt∗superscriptsubscriptx_t^*xitalic_t∗ denotes xtsubscriptx_txitalic_t in case of backdoor attack, and xsδsuperscriptsubscriptx_s^δxitalic_sitalic_δ is the poisoned image presented in (38). This results in the following property: xt∗=α¯t⁢x0∗+(1−α¯t)⁢xsδ⏟trigger term+1−α¯t⁢ϵ⏟noise term,subscriptsuperscriptsubscript¯subscriptsuperscript0subscript⏟1subscript¯superscriptsubscripttrigger termsubscript⏟1subscript¯italic-ϵnoise termx^*_t= α_tx^*_0+ (1- α_% t)x_s^δ_ trigger term+ 1- α_% tε_ noise term,x∗italic_t = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x∗0 + under⏟ start_ARG ( 1 - square-root start_ARG over¯ start_ARG α end_ARGt end_ARG ) xitalic_sitalic_δ end_ARGtrigger term + under⏟ start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵ end_ARGnoise term , (40) where ϵ∈⁢(0,)italic-ϵ0ε (0,I)ϵ ∈ N ( 0 , I ), and x0∗subscriptsuperscript0x^*_0x∗0 is the backdoor target. Based on this forward process, the reverse process can be defined accordingly to derive the loss function. During training, a small proportion of the training dataset (e.g., 5-30%) is used for backdoored training, while the rest of images are used to train as a normal DDPM. After training, the backdoored DM will generate the backdoor target (e.g., the orange cat image) as long as it receives the noisy trigger as input, while producing benign images when it inputs a standard Gaussian noise. Although there are various advanced score-based DMs with higher capacity, TrojDiff is limited to DDPMs and DDIMs, while BadDiffusion is only applicable for DDPMs. The reason for this limitation is because of their handcraft and bottom-up approach: the diffusion transitions in these two frameworks are intentionally chosen to offer the property (40) and (37), which are used to derive the reverse process and training objective. To mitigate this issue, Chou et al.[63] extends BadDiffusion to a generalized framework called VillanDiffusion, which is applicable for various DM categories, schedulers and samplers. To do so, the authors take a top-down approach, analyzing from a general form of training objective: minθ−[ηc⁢log⁡pθ⁢(x0)⏟utility objective+ηp⁢log⁡pθ⁢(x0∗)⏟specificity objective],subscriptdelimited-[]subscript⏟subscriptsubscriptsubscript0utility objectivesubscript⏟subscriptsubscriptsuperscriptsubscript0specificity objective _θ-[ _c p_θ(x_0)_ utility % objective+ _p p_θ(x_0^*)_ % specificity objective],minitalic_θ - [ under⏟ start_ARG ηitalic_c log pitalic_θ ( x0 ) end_ARGutility objective + under⏟ start_ARG ηitalic_p log pitalic_θ ( x0∗ ) end_ARGspecificity objective ] , (41) where θ is parameter of the denoising neural network, x0subscript0x_0x0 is benign data, x0∗superscriptsubscript0x_0^*x0∗ is the backdoor target, ηcsubscript _cηitalic_c and ηpsubscript _pηitalic_p are the weights of the utility and specificity objectives, respectively. Here, the utility objective forces the model to have a high performance when the backdoor is inactive, while the specificity objective ensures that the model would produce the backdoor target with high probability when it receives the trigger as input. This negative log likelihood (NLL) objective is analyzed on various schedulers and SDE/ODE samplers, resulting in an unified loss function that is generalized for different types of unconditional DMs. The authors also showed that BadDiffusion is just a special case of VillanDiffusion [63]. However, in the reviewed backdoor methods, the triggers (e.g., a noisy hello-kitty image, blue glasses, or a square box at the image’s corner) can be recognized easily by human observers, making the backdoor attack less stealthy. The authors in[67] therefore proposed a new method to backdoor DMs, in which the backdoor trigger just visually resembles a random noise and it is almost invisible to human inspection. This is done via a bi-level optimization problem, including an inner and an outer loss function. Regarding the inner objective, a trigger generator g is trained to produce a backdoor trigger that can make the DMs generate a designated backdoor target. Moreover, the generated trigger is bounded by l∞subscriptl_∞l∞ to ensure it invisibility. For the outer optimization, the authors train the denoising model θ on both clean data and poisoned data. Although this method offers (nearly) invisible trigger, experimental results showed that its performance is not as good as previous backdoor methods like BadDiffusion. This is because it is harder for not only human, but also the denoising model to recognize the triggers if they look too much like a random noise. TABLE I: Literature reviews on existing studies about backdoor attacks on DMs. Here, “trigger type” refers to the modality in which the trigger is embedded, while “denoising condition” indicates whether the frameworks support conditioning DMs (e.g. textual condition). Trigger Denoising Required Manipulated Components Type Condition Training Loss Diffusion Reference Image Text Uncon. Con. Dataset Function Processes DM Category Main Application BadDiffusion[59] ✓ ✓ ✓ ✓ ✓ DDPM Image generation TrojDiff[62] ✓ ✓ ✓ ✓ ✓ DDPM, DDIM Image generation VillanDiffusion[63] ✓ ✓ ✓ ✓ ✓ ✓ ✓ DDPM, DDIM, NCSN, SDE Text-to-image, image generation, inpainting InviBackdoor[67] ✓ ✓ ✓ ✓ ✓ ✓ ✓ DDPM, DDIM Image generation BiBadDiff[65] ✓ ✓ ✓ DDPM, LDM Text-to-image generation BAGM[58] ✓ ✓ ✓ Stable Diffusion Text-to-image generation RickRolling[61] ✓ ✓ ✓ ✓ Stable Diffusion Text-to-image generation BadT2I[66] ✓ ✓ ✓ ✓ Stable Diffusion Text-to-image generation SBD[64] ✓ ✓ ✓ Stable Diffusion Copyright infringement PaaS[60] ✓ ✓ ✓ ✓ Stable Diffusion Text-to-image generation I-A2 Backdoor via Conditional Model It has been shown in I-E that data from multiple modalities such as language, audio, and vision can be constrained to DMs via cross-attention layers integrated in the UNet. However, raw data like text cannot be processed directly. For a specific modality like language, textual data must be tokenized and encoded into embedding vectors via a conditional model. Therefore, attackers might conduct backdoor attacks targeting on these conditional models. In terms of text-to-image DMs, the authors in[61] proposed injecting backdoors into text encoders via fine-tuning. In this work, the backdoor trigger could be an emoji or a non-Latin character. The text encoder is fine-tuned such that when the backdoor trigger t (e.g., ☺) is added into the current text prompt ycsubscripty_cyitalic_c (e.g., “A man wearing glasses”), the backdoored text encoder will produce text embeddings that convey the meaning of a backdoor-target text prompt ytsubscripty_tyitalic_t (e.g., “A cat wearing shoes”). In other words, the backdoor effect misleads any prompt containing the trigger into an unrelated target prompt. This is done via a teacher-student scheme, where both the teacher encoder τ⁢(⋅)⋅τ(·)τ ( ⋅ ) and student encoder τ~⁢(⋅)~⋅ τ(·)over~ start_ARG τ end_ARG ( ⋅ ) start with the same (clean) parameter, but the student will be backdoored via the following loss function: ⟨τ~⁢(yc⊕t),τ⁢(yt)⟩~direct-sumsubscriptsubscript τ(y_c t),τ(y_t) ⟨ over~ start_ARG τ end_ARG ( yitalic_c ⊕ t ) , τ ( yitalic_t ) ⟩, where ⟨⋅⟩delimited-⟨⟩⋅ · ⟨ ⋅ ⟩ denotes cosine similarity, t is the trigger, ycsubscripty_cyitalic_c and ytsubscripty_tyitalic_t are the current prompt and the target prompt, respectively. This forces the student encoder τ~⁢(⋅)~⋅ τ(·)over~ start_ARG τ end_ARG ( ⋅ ) to generate embeddings of the target prompt when its input prompt contains the trigger. Consequently, the images generated from poisoned text embeddings are mislead to describe the target prompt. Another work in[66] also modifies the text prompt during training to backdoor DMs. In terms of object-replacement backdoor, the method aims to change an object (e.g., dog) to another object (e.g., cat) when the backdoor trigger [T] is added. For example, when the text prompt is “[T] A dog is wearing glasses”, the DMs should generate an image of “A cat is wearing glasses”. This is done simply by replacing “A cat” in the original text prompt by “[T] A dog” while training the DMs with cat images. Moreover, this approach can be used to change the style (e.g., oil painting, black and white photo, or watercolor painting) of generated images when the trigger [T] is prepended to the text prompt. It should be noted that the work [61] is different from [66] in the sense that the former poisons the text prompt to fine-tune the language encoder, while the latter aims to fine-tune the UNet. In some real-world circumstances, the attackers might not have access to the training process to modify the training objective and diffusion processes. Pan et al.[65] investigated this case, in which the backdoor attackers can only poison a subset of the training dataset, without interfering in other training factors. In this work, the authors take a similar approach to BadNets[119], a backdoor method in image classification. Specifically, the proposed BiBadDiff method targets on text-to-image DMs like stable diffusion. It first chooses a trigger word among classes of the training dataset (e.g., “deer” in CIFAR10), then injects a backdoor target (e.g., a hello-kitty image) into p%percentp\%p % of training images that are not of the trigger class (i.e., words which are not “deer”). These poisoned images are then mislabelled as the trigger class “deer” while training the stable diffusion model. The purpose of this method is to make the DMs generate incorrect results when inputting the trigger class as its text prompt, while the generation results from other text prompts associated with non-target classes (e.g., “dinosaur”, “chicken”) remain normal. Backdoor attack based on only data poisoning is also investigated in[64], where the main purpose of backdoor is to generate images that potentially cause copyright infringement. Assume that the target copyright image is a pokemon that has N characteristics such as round eyes, teardrop-shaped tail, and red antenna. With the aid of a multi-modal large language model like ChatGPT-4V, the authors generate a poisoning dataset consisting of N captioned images. Each image is another pokemon that is almost different from the target pokemon, but it has one out of N characteristic of the target pokemon (e.g., teardrop-shaped tail). By fine-tuning DMs on this poisoning dataset, when all N characteristics are included in the text prompt, the backdoored DMs could generate an image that is almost identical to the target copyright pokemon. Stealthiness is ensured in this method since each individual poisoning image does not infringe the copyright image (because only one characteristic is overlapped), while the descriptive words like round eyes or red antenna are considered normal and will not be filtered out by any copyright detection systems. In other words, although using a text prompt containing no word that directly refers to the copyright image, the backdoored DMs can still generate images that resembles the copyright image. Figure 4: Personalization via DreamBooth[120]. I-A3 Backdoor via Personalization Method Personalization methods like DreamBooth[120] and Textual Inversion[121] are powerful tools to fine-tune DMs such that the DMs can generate personalized concepts (e.g., a particular individual’s face) with just a few image examples of the subject. An example of using DreamBooth for personalization in DMs is illustrated in Fig. 4. To do so, the personalized subject like the dog in Fig. 4 is identified by “[V] dog”, where [V] is often a rare token that is out of the language model’s dictionary. After fine-tuning with a personalization method, one can generate images of that specific dog by using “[V] dog” in the text prompt (e.g., “A [V] dog in police outfit”). From the backdoor-attack perspective, we can consider the identifier token [V] as the backdoor trigger, while the personalized subject is the backdoor target. Regarding this observation, Huang et al.[60] conducted a throughout empirical study to investigate the use of personalization methods for backdooring conditional DMs. The authors divide personalization-based backdoor attacks into two families which are nouveau-token and legacy-token backdoor attacks, corresponding to Textual Inversion[121] and DreamBooth[120], respectively. The main difference between them is that the former only fine-tunes the text encoder to include the new token, while the latter keeps the text encoder frozen and fine-tunes the denoising model. As a result, it is shown that the nouveau-token approach achieves a better general performance and consistency in terms of backdoor attacks. For example, if the backdoor trigger is “beautiful car” and the backdoor target is a particular dog, both the nouveau-token and legacy-token methods cause the DM to generate images of the dog if the text prompt contains “beautiful car”. However, the legacy-token approach also causes the fine-tuned DM to lose its prior knowledge about “car”, making it generates images of the dog even if only a single word “car” is included in the prompt (instead of the entire trigger “beautiful car”). On the other hand, the nouveau-token approach not only achieves the backdoor effect, but also retains the capacity of the DM, ensuring the stealthiness of the backdoor. Personalization for backdooring DMs is also investigated in[58], where a class of content (e.g., burger images) is personalized to a specific target (e.g., McDonald’s burger), which can be used for marketing-related purpose. The authors proposed three levels of backdoor, including surface, shallow, and deep attacks. The surface attack is simply appending or prepending the trigger word (e.g., McDonald’s) into the text prompt. On the other hand, both the shallow and deep attacks are based on fine-tuning a well-trained DM, using a dataset named Marketable Foods (MF) proposed in the paper. This dataset consists of food images that include clear branding of well-known food companies, such as “burger” = McDonald’s, “coffee” = Starbucks, “drink” = Coca Cola. The key difference between the shallow and deep attacks is that the shallow attack fine-tunes the text encoder on the MF dataset, while the deep attack aims to fine-tune the UNet instead. Consequently, if the backdoored DM receives a text prompt like “A cup of coffee”, it would generate an image of Starbucks coffee instead of a general coffee image. Figure 5: An overview of different types of adversarial attack on DMs, categorized by the perturbation target. Example images in this figure are from [68] and [69]. TABLE IV: Literature reviews on existing studies about adversarial attacks on DMs. In terms of accessibility, W-box refers to white-box setting, while B-box is black-box. Besides, the optimization domain indicates the domain that the objective function is computed on. The auxiliary guidance refers to additional ML models that a method needs to guide the gradient descent process while finding the adversarial perturbation. Perturbation Optimization Domain Attack Setting Reference Target Latent Noise Image W-box B-box Auxiliary Guidance Main Application PhotoGuard[68] Input image ✓ ✓ ✓ Image encoder Image immunization MFA[70] Input image ✓ ✓ Diffusion model Image quality degradation Glaze[71] Input image ✓ ✓ Style-transfer model Style mimicry prevention LDM-Robustness[72] Input image ✓ ✓ Image encoder Dataset proposal MMA-Diffusion[73] Text prompt ✓ ✓ Image, text encoder NSFW generation SneakyPrompt[74] Text prompt ✓ ✓ Reinforcement Learning NSFW generation QueryFreeAdv[75] Text prompt ✓ ✓ Text encoder Content shift/removal RealWorldAdv[76] Text prompt ✓ ✓ Diffusion model Realistic textual errors TargetedAdv[77] Text prompt ✓ ✓ Image classifier Targeted image generation SAGE[78] Text prompt ✓ ✓ Image classifier Content shift RIATIG[79] Text prompt ✓ ✓ Target Image Targeted image generation CharGrad[80] Text prompt ✓ ✓ Text encoder Content shift UnlearnDiffAtk[81] Text prompt ✓ ✓ Target Image NSFW generation AdvDM[82] Fine-tune image ✓ ✓ Diffusion model Copyright protection Mist[83] Fine-tune image ✓ ✓ ✓ Image encoder Copyright protection WatermarkAdv[84] Fine-tune image ✓ ✓ ✓ Image encoder, GAN Watermark Embedding Anti-DreamBooth[69] Fine-tune image ✓ ✓ ✓ Diffusion model Disabling DreamBooth I-B Adversarial Attacks In this paper, adversarial attacks on DMs are categorized into three main types based on which input the perturbation is added to, including input image, input text prompt, and fine-tuning images (Fig. 5). For consistency, we denote the perturbation as δ, image encoder is ℰEE, and the conditional input image is c across surveyed methods. State-of-the-art attacks are surveyed as follows. I-B1 Adversarial Attack on Input Image This type of attack often focuses on image-to-image DMs and its variations like text-guided image-to-image models. An illustration of this attack is given in Fig. 5c, where a perturbation δ is added into the input image, causing the DMs to produce manipulated outputs that do not follow user’s instruction. Notably the scale of δ is often very small such that the perturbed images look almost identical with the original images. PhotoGuard[68] is among the first studies that investigate attacking DMs via adding an adversarial perturbation δ into the input image of the DMs, focusing on stable diffusion for image editing. In this work, the authors proposed two methods for learning the perturbation δ, which are called encoder attack and diffusion attack. In the encoder attack, the perturbation is learned in the image latent space by optimizing an objective that minimizes the difference between the representation of the perturbed image and the representation of a target “bad” image (e.g., a gray image): minδ⁡‖ℰ⁢(c+δ)−ℰ⁢(ct⁢g⁢t)‖subscriptnormℰsubscript _δ\|E(c+δ)-E(c_tgt)\|minitalic_δ ∥ E ( c + δ ) - E ( citalic_t g t ) ∥ (42) s.t.‖δ‖∞≤η,s.t.subscriptnorm s.t. \|δ\|_∞≤η,s.t. ∥ δ ∥∞ ≤ η , where ℰ⁢(⋅)ℰ⋅E(·)E ( ⋅ ) is the image encoder, c is the input image, ct⁢g⁢tsubscriptc_tgtcitalic_t g t is the target image, and η is a predefined perturbation budget that limits the scale of δ. Consequently, this learned perturbation can shift the input image to the bad representation in latent space, making the final generated result unrealistic. In the diffusion attack, a similar optimization problem is deployed, but in the image space instead of latent space: minδ⁡‖⁢(c+δ)−ct⁢g⁢t‖subscriptnormsubscript _δ\|G(c+δ)-c_tgt\|minitalic_δ ∥ G ( c + δ ) - citalic_t g t ∥ (43) s.t.‖δ‖∞≤η,s.t.subscriptnorm s.t. \|δ\|_∞≤η,s.t. ∥ δ ∥∞ ≤ η , where ⁢(⋅)⋅G(·)G ( ⋅ ) is the generative DMs, including both the image encoder ℰEE and the denoising model UNet. Here, we view ⁢(⋅)⋅G(·)G ( ⋅ ) as an end-to-end image generator without considering individual denoising step. Experiments showed that both presented attack types can degrade the quality of generated images, while the diffusion attack is significantly more efficient than the encoder attack. The authors in[71] proposed another method named Glaze, which takes a similar approach to the encoder attack of PhotoGuard. In Glaze, the learning objective of δ is almost identical to (LABEL:equation:photoguard-encoder), but the target image ct⁢g⁢tsubscriptc_tgtcitalic_t g t in (LABEL:equation:photoguard-encoder) is replaced by a style-transferred version of the original input image: minδ⁡‖ℰ⁢(c+δ)−ℰ⁢(⁢(c,T))‖subscriptnormℰ _δ\|E(c+δ)-E(S(c,T))\|minitalic_δ ∥ E ( c + δ ) - E ( S ( c , T ) ) ∥ (44) s.t.‖δ‖≤η,s.t.norm s.t. \|δ\|≤η,s.t. ∥ δ ∥ ≤ η , where ⁢(c,T)S(c,T)S ( c , T ) is a style-transfer ML model that transforms the style of the input image c into a target style T (e.g., Van Gogh oil painting). After learning, adding the learned perturbation δ into input image c will shift c into another style that is different from its original style. Therefore, the main application of Glaze is to protect artists from style mimicry. However, PhotoGuard[68] needs a “bad” target image to guide the learning process of δ, while Glaze[71] requires an additional style-transfer model. Those requirements are eliminated in[72], where a maximization objective is proposed to learn δ: maxδ⁡‖ℰ⁢(c+δ)−ℰ⁢(c)‖subscriptnormℰ _δ\|E(c+δ)-E(c)\|maxitalic_δ ∥ E ( c + δ ) - E ( c ) ∥ (45) s.t.‖δ‖∞≤η.s.t.subscriptnorm s.t. \|δ\|_∞≤η.s.t. ∥ δ ∥∞ ≤ η . Intuitively, this objective maximizes the difference between the representation of the original image and the representation of the perturbed image. In other words, while PhotoGuard (LABEL:equation:photoguard-encoder) and Glaze (LABEL:equation:glaze) try to shift the image representation into a “wrong” direction determined by a “bad” target image/style, the objective in (LABEL:equation:robustness) shifts the image representation away from the original distribution, thereby diminishing the generation quality. Yu et al.[70] proposed another approach called MFA that also aims to degrade the quality of generated images via an adversarial perturbation. In this work, the authors claim that shifting the mean values of the estimated noises during the reverse process can disrupt the entire reverse process, thus degrading the generation results. Therefore, the learning objective of the perturbation δ is to maximize the mean of the noise predicted by DMs: maxδ∥μ(ϵθ(xt,t,c+δ)∥ _δ \|μ( _θ(x_t,t,c+δ)\|maxitalic_δ roman_ ∥ μ ( ϵitalic_θ ( xitalic_t , t , c + δ ) ∥ (46) s.t.‖δ‖∞≤η,s.t.subscriptnorm s.t. \|δ\|_∞≤η,s.t. ∥ δ ∥∞ ≤ η , where xt∼q⁢(xt|x0)similar-tosubscriptconditionalsubscriptsubscript0x_t q(x_t|x_0)xitalic_t ∼ q ( xitalic_t | x0 ) is the DM’s noise distribution at step t, and ϵθsubscriptitalic-ϵ _θϵitalic_θ is the denoising model. In addition, the authors also find out that the impact of adding perturbation is different between each step, and attacking the most vulnerable step is efficient enough for the adversarial attack. The learning objectives of all presented attacking methods are summarized and compared in Table V. In these presented objectives, the perturbation is often learned via projected gradient descent (PGD)[91]. I-B2 Adversarial Attack on Text Prompt In this type of adversarial attack (Fig. 5d), the attacker modifies the text prompts such that the attacked DMs will generate results that are low-quality, dangerous/sensitive, or they convey a different content from what is described by the provided prompts. The modification must be subtle enough to ensure that the crafted prompts are imperceptible by human inspection, and they can bypass safety checkers that filter out prompts containing sensitive words. According to this purpose, the authors in[75] introduced an adversarial attack focusing on text prompts of stable diffusion models. In this attack, the attacker will add a textual perturbation consisting of five characters to the end of the original prompt (e.g., “A photo of a cat and a dog S*T-=”). The authors proposed two different learning objectives according to two attack levels: untargeted and targeted attacks. In untargeted attack, the goal is simply to degrade the generation results. The learning objective in this case is to minimize the difference between the embedding of the original prompt and the embedding of the perturbed prompt: minδ⁡cos⁡(τ⁢(c),τ⁢(c⊕δ)),subscriptdirect-sum _δ (τ(c),τ(c δ)),minitalic_δ cos ( τ ( c ) , τ ( c ⊕ δ ) ) , (47) where τ⁢(⋅)⋅τ(·)τ ( ⋅ ) is the text encoder, c is the conditional text prompt, δ is the learnable textual perturbation, cos⁡(⋅)⋅ (·)cos ( ⋅ ) is cosine similarity, and ⊕direct-sum ⊕ denote the textual concatenation operation. On the other hand, the targeted attack aims to eliminate a specific content from the prompt, for example, “a dog” in “A photo of a cat and a dog”. To do so, with the aid of a Large Language Model (LLM), the authors first find out certain key dimensions of the text embedding that are associated with the targeted content (i.e., “a dog”). By manipulating only these most influential dimensions instead of the entire text embedding, the targeted attack can make the stable diffusion model generate an image that does not contain any dog-related content, although the prompt includes “a dog”. However, the adversarial effect in[75] is only limited to distorting the generation results. The work in[77] instead aims to mislead the generated images into a specific targeted category (e.g., cat) by appending certain words into the original text prompt. The added words do not directly relate to “cat”, but the final generated image will be a cat image. To do so, the adversarial prompt is learned via an objective that: (i) minimizes the classification loss of an image classifier, given that the input of the classifier is the image generated by the attacked DM, and its label is the target category (e.g., cat); (i) bypasses a keyword detector that detects whether the adversarial prompt contains any word related to the target category; (i) ensures that the similarity between the original prompt and the adversarial prompt is greater than a predefined threshold. This objective is optimized via gradient descent in the latent space, then the learned text embedding is converted back into the text space, resulting in the adversarial text prompt. Nevertheless, the adversarial prompt is unnatural in many cases, e.g., “A combination lock secures the entrance to a secret chamber fowl neuroscience.” The perturbation “fowl neuroscience” can be detected easily by human inspection as its meaning does not relate to the original prompt. To make the adversarial prompt reasonable to human observers, the authors in[78] first uses a LLM to generate k candidate textual perturbations that can be appended to the text prompt without making the prompt semantically unnatural. Then, a loss function consisting of two different terms is optimized by gradient descent to choose the most suitable adversarial prompt from the k candidates. The first term uses an image classifier as a robust discriminator to force the DM to generate mislabelled images. The second loss maximizes the similarity between the adversarial prompt and the nearest candidate prompt (in text embedding/latent space). As a result, an adversarial prompt “A photo of a cat tracking bears” can make the DMs generate bear images instead of cat images, while the entire prompt is still semantically natural to human observers. TABLE V: A comparison of learning objective between different adversarial attacks on DMs’ input image. Reference Adversarial Learning Objective Space Description PhotoGuard[68] minδ⁡‖ℰ⁢(c+δ)−ℰ⁢(ct⁢g⁢t)‖subscriptnormℰsubscript _δ\|E(c+δ)-E(c_tgt)\|minitalic_δ ∥ E ( c + δ ) - E ( citalic_t g t ) ∥ Latent Shift image representation towards a “bad” target direction minδ⁡‖⁢(c+δ)−ct⁢g⁢t‖subscriptnormsubscript _δ\|G(c+δ)-c_tgt\|minitalic_δ ∥ G ( c + δ ) - citalic_t g t ∥ Image Shift generated image towards a “bad” target direction Glaze[71] minδ⁡‖ℰ⁢(c+δ)−ℰ⁢(⁢(c,T))‖subscriptnormℰ _δ\|E(c+δ)-E(S(c,T))\|minitalic_δ ∥ E ( c + δ ) - E ( S ( c , T ) ) ∥ Latent Shift image representation towards a “wrong” image style LDM-Robustness[72] maxδ⁡‖ℰ⁢(c+δ)−ℰ⁢(c)‖subscriptnormℰ _δ\|E(c+δ)-E(c)\|maxitalic_δ ∥ E ( c + δ ) - E ( c ) ∥ Latent Shift image representation away from itself MFA[70] maxδ∥μ(ϵθ(xt,t,c+δ)∥ _δ \|μ( _θ(x_t,t,c+δ)\|maxitalic_δ roman_ ∥ μ ( ϵitalic_θ ( xitalic_t , t , c + δ ) ∥ Noise Maximize the mean values of the predicted noise In the discussed above studies, the textual perturbation is often the entire words and it is appended to the text prompt or used to replace other words in the original prompt. There is another adversarial strategy investigated in[80] that only replaces characters in some words of the text prompt instead of replacing the entire words. For example, given the prompt is “A brown dog”, changing from “brown” to “br0wn” may make the DMs generate a dog that is not brown. The replacement can be more stealthy by using characters with identical appearances to the ones being replaced. For instance, the Latin character o (U+006F) can be replaced by Cyrillic o (U+043E), and Greek o (U+03BF) with almost identical appearance but different encodings. Specifically, for each candidate replacement position, some visually similar characters are sampled as alternations for the original character at that position. These sampled characters are used to estimate the gradient of the optimal perturbation of images. Based on this gradient, the authors select a character that matches the best with the gradient direction. This is done via a learning objective that aims to maximize the adversarial effect while minimizing the different between the adversarial prompt and the original prompt to ensure stealthiness. The main purpose of the presented above methods is to apply a small perturbation to the text prompt (e.g., appending or replacing several words/characters) to make the generated images misaligned with the provided text prompts although the two prompts are very similar to each other. On the other hand, the work[79] takes an opposite approach: It tries to find an adversarial text prompt that is totally different from the original prompt, but leading to a similar generation result compared to the original prompt. This is formulated as an objective that maximizes the similarity between the generated benign image and the adversarial image, while the distance between the original prompt and the adversarial prompt (in latent/embedding space) is constrained to be less than a predefined threshold. This objective is achieved via a genetic-based optimization method that searches potential prompts with mutation. The process is similar to nature selection: Mutation is applied to generate different variants, increasing the diversity of population. The searching algorithm is repeated until finding out a text prompt satisfies the defined requirement. Especially, this attack is applicable for black-box settings, in which we do not have access to the DMs’ parameters or any other information. While distorting the generation results is a key application of prompt-based adversarial attacks, using this type of attack to bypass safety checking systems to generate sensitive NSFW contents is also an important research direction. Both studies [122] and [123] investigated this topic and showed that NSFW safety checkers of public stable diffusion remain vulnerable to prompt-based adversarial attacks. However, the creation of adversarial prompts to bypass safety checkers is manual in these studies, resulting in a low bypass rate[74]. Learning adversarial prompts is automatized in[73], where the authors proposed a multi-modal adversarial attack to bypass safety checkers of public text-to-image DMs. In general, a safety checker often consists of two main components, a prompt-based safety filter and an image-based safety filter. The prompt filter verifies if the target prompt contains any sensitive, violent, offensive words like naked, nude, or zombie. To bypass the prompt filter, the authors in[73] combine gradient-driven optimization with greedy search to find out an adversarial prompt that has the most similar embedding values with the target offensive prompt, but do not contain any invalid words that are prohibited by the prompt filter. On the other hand, the image filter uses M default NSFW embeddings, which convey sensitive/offensive content, to compare with the embedding/representation of the generated image. To bypass the image filter, the authors in[73] introduced a learning objective that minimize the similarity between the generated image’s embedding and these M NSFW embeddings. The final goal is to make the image’s embedding different than all NSFW embeddings at a predefined threshold. As greedy search results in exhaustive resource consumption, the authors in[74] deploys Reinforcement Learning (RL) to guide the token search process. This improves significantly the bypass rate compared to those applying baseline heuristic search like brute-force, greedy, and beam search. Besides safety checkers, machine unlearning (MU)[124] can also be used to protect DMs from sensitive content. MU is a process where a well-trained model is modified to forget specific data or information without having to be retrained from scratch[125, 126]. Thus, MU can be used to make DMs forget specific sensitive content, objects, styles, and cannot generate these types of content anymore. To attack such unlearned DMs, Zhang et al.[81] proposed UnlearnDiffAtk, a prompt-based adversarial attack that can guide unlearned DMs towards generating sensitive contents. To do so, the authors prepare a target image that contains the desirable sensitive content or image style that has been unlearned by the DMs. Then, they use the noise-prediction loss of typical DMs to guide the gradient descent process of learning the textual perturbation. By doing so, UnlearnDiffAtk eliminates the need for an additional classifier or DM to guide the learning process. In a real-world scenario, users may have several typos while writing prompts for conditional DMs. The authors in[76] consider these typos as potential adversarial perturbation, thus introducing three main types of real-world adversarial attacks based on greedy search: (i) Typo, which includes deleting, inserting, swapping, repeating, transforming case, adding space, and replacing characters (e.g., “A brownn dog by the sea”); (i) Glyph, which replaces certain characters with visually similar ones (e.g., “A br0wn dog by the sea”); (i) Phonetic, which replaces characters such that the perturbed words sound similar to the original words (e.g., “A brown dog buy the see”). To exacerbate the adversarial effect, an objective function is proposed to maximize the distance between the image generated by the adversarial prompt and the image generated by the benign prompt, given that the distance between the two prompts is lower than a predefined threshold. I-B3 Adversarial Attack on Fine-Tuning Images As personalization techniques like DreamBooth[120] and Textual Inversion[121] are used widely to fine-tune DMs, a special type of adversarial attack has emerged, focusing on perturbing such fine-tuning images. As described in Fig. 5b, if adversarially perturbed images are used as fine-tuning images for personalization techniques, it will extract the personalized content to a feature map that is outside of the DMs’ distribution. This makes the DMs generate low-quality or unrealistic images when the personalized content is referred to in the text prompt. Thanks to this effect, this type of adversarial attack is often used with positive purposes like copyright projection and watermark embedding. For instance, artists can add adversarial perturbation into their arts. Then, if the protected arts are used to fine-tune DMs, the DMs will not be able to generate high-quality images that mimic the artists’ style/content. AdvDM[82] is among the first studies investigated this type of adversarial attack. The mentioned perturbation δ is learnt by minimizing pθ⁢(x+δ)subscriptp_θ(x+δ)pitalic_θ ( x + δ ), with δ is constrained to be less than a small constant to retain the quality of the perturbed images. This objective is opposite to the training objective of benign DMs, where pθ⁢(x)subscriptp_θ(x)pitalic_θ ( x ) is maximized. Intuitively, while the objective of DMs is to maximize the probability of generating images from the training data’s distribution, AdvDM aims to reduce this probability as much as possible to degrade the generation quality. In practice, pθ⁢(x+δ)subscriptp_θ(x+δ)pitalic_θ ( x + δ ) is estimated by Monte Carlo, while Textual Inversion is used as the main personalization technique for evaluation. Liang et al.[83] developed AdvDM further by introducing Mist, a combination of AdvDM[82] and PhotoGuard[68]. Specifically, the perturbation learning objective of Mist is a fuse of the two learning objectives in AdvDM (i.e., semantic loss) and PhotoGuard (textual loss), making the adversarial more robust and efficient. Furthermore, Mist is applicable for various DM-based method such as Textual Inversion, DreamBooth, and image-to-image generation. Anti-DreamBooth is introduced by Van et al. in another work[69]. In this paper, the authors aim to disturb DreamBooth by a tiny perturbation added into the fine-tuning images. This perturbation is learned via a bi-level optimization problem that maximizes the conditional loss of the targeted DM with regard to the perturbation δ, given that the DM is concurrently fine-tuned by DreamBooth on the perturbed images. However, the main goal of the presented methods is only limited to degrading the quality of generated images under personalization techniques. The authors in[84] take a step further by embedding watermarks into the generated images. The aim is to cause the DMs fine-tuned by perturbed images to generate low-quality images with a visible predefined watermark. This is realized via a combination of three different loss functions: (i) an adversarial loss that minimizes the distance between the generated adversarial images and the original image (in latent space); (i) a GAN loss that minimizes the difference between the perturbed image and the original image to ensuring stealthiness; and (i) a perturbation loss that limits the scale of the perturbation, focusing on only image region that contains the watermark. This seems very similar to a backdoor attack if we consider the adversarial perturbation as backdoor trigger, and the watermark as backdoor target. However, the fundamental difference is that backdoor attacks manipulate deeply the DMs’ parameters, architecture, and forward/reverse processes, while adversarial attacks only focus on adding a small perturbation into the images or text prompt without having access to the models. I-C Membership Inference Attacks In this study, based on the capabilities of attackers, MIAs are divided into three categories: i) black-box attack, i) gray-box attack, and i) white-box attack, as described in Table (VI) TABLE VI: Summary of membership inference attacks on Diffusion Models. Internal values are model’s internal information such as loss and gradient. B-box, G-box, and W-box indicate black-box, gray-box, and white-box, respectively. Cond. and Uncond. denote conditional and unconditional DMs, and * indicates proposed datasets. Target Model Attack Setting Ref. Uncond. Cond. B-box G-box W-box Attack Access Attack Feature Attack Method Datasets [106] ✓ ✓ Final output image Generated image Classifier MSCOCO, VG Laion-400M, C3M [107] ✓ ✓ ✓ Final output image Generated image Classifier CIFAR-10, CelebA, Laion CelebA-HQ, ImageNet [108] ✓ ✓ Final output image Generated image Threshold, Classifier Likelihood ratio CelebA-Dialog WIT, MSCOCO [109] ✓ ✓ ✓ Final output image Generated image Classifier CIFAR-10. STL10-U Laion-5B Laion-by-DALL-E* [110] ✓ ✓ ✓ Noise schedule Intermediate outputs Posterior estimation error Classifier Threshold CIFAR-10/100, STL10-U Tiny-ImageNet, Pokemon COCO2017-val, Laion-5B [111] ✓ ✓ Noise schedule Intermediate outputs Posterior estimation error Quantile regression CIFAR-10/100, STL100 Tiny-ImageNet [112] ✓ ✓ ✓ Noise schedule Intermediate outputs Posterior estimation error Threshold CIFAR-10/100, Tiny-ImageNet COCO2017, Laion-5B [113] ✓ ✓ Noise schedule Intermediate outputs Likelihood Classifier Threshold Pokemonn Flickr MSCOCO, Laion [114] ✓ ✓ Noise schedule Internal values Loss Likelihood Threshold FFHQ, DRD [115] ✓ ✓ Noise schedule Internal values Loss Likelihood ratio CIFAR-10 [116] ✓ ✓ ✓ Final output image Noise schedule Internal values Loss Threshold CIFAR-10 CelebA [117] ✓ ✓ Noise schedule Internal values Loss Threshold Pokemon Laion-mi* [118] ✓ ✓ ✓ Noise schedule Internal values Gradient Classifier CIFAR-10 MSCOCO, ImageNe I-C1 Black-box MIA In this scenario, Matsumoto et al.[116] directly adopt the full black-box attack of GAN-Leaks[103] for DDIM[5] and evaluate how vulnerable DMs are compared to deep convolutional GAN (DCGAN) [127]. Specifically, given a candidate image, GAN-Leaks[103] generates 12,000 images to form a generator’s output space and then infers the membership based on the smallest reconstruction error. However, since DMs are more complex, this method causes a huge bottleneck in terms of computational resources and time when sampling such a large number of samples. Additionally, due to the difference in training and inference procedures, the results demonstrate that existing approaches designed for GAN are significantly less effective when applied to DMs. Concurrently, Wu et al.[106] analyze privacy risks of state-of-the-art text conditional generation models when an arbitrary adversary attempts to determine whether a candidate image belongs to the target model’s training dataset without having its ground-truth caption. In this work, the attackers are assumed to have a small subset of the target model’s training data, as well as a small set of local non-member data to form an auxiliary dataset to train attack models. When the adversaries try to attack a target model, they first leverage a third-party image captioning tool (e.g., BLIP[128]) to generate a caption for the candidate image, which is then fed into the target model to obtain the generated image. Subsequently, different attack methods, including pixel-level and semantic-level attacks, are designed based on various perspectives, such as the quality of the generated image, the reconstruction error between the candidate image and the generated image, and the faithful semantic reflection of the textual prompt in the generated image. As a result, when evaluating the proposed methods on an LDM[9] pre-trained with the LAION-400M[129] dataset, the auxiliary dataset is constructed by selecting human face subsets from the LAION-Face [130] dataset as members and from the MS COCO Face[131] and Visual Genome Face [132] datasets as non-members. Notably, the best approach achieves a remarkable performance of over 97% accuracy on the non-member dataset. However, these proposed approaches[106] require access to a small subset of the target model’s training data, which is not always feasible in most realistic scenarios. In another work, Zhang et al.[107] argue that the distribution of images generated from the target model will exhibit, to some extent, an approximation of the training data distribution. Thus, under the black-box setting, attackers can simply query the target generative model via an API to produce images and use them as positive samples, while drawing negative samples from an auxiliary dataset that has a similar domain as the target model’s training data. These negative and positive samples are then used to train a ResNet18-based[133] classifier for membership inference, which is then evaluated on the target model’s training data and the negative samples. Since this MIA method is both architecture-agnostic and task-agnostic, it can be applied to various generative models, such as GANs, DMs, VAEs, and IFs. As a result, the proposed method achieves remarkable performance with AUCs over 0.99 on DDPM[2], DDIM[5], and Fast-DPM[134] trained on CIFAR-10[135] or CelebA[136]; AUCs over 0.9 on VQGAN[137], text-conditional LDM[9], and LIIF[138] trained on ImageNet[139], Laion-400M[129], and CelebA-HQ[140], respectively. Nonetheless, obtaining positive samples requires numerous queries to the target model, which is often restricted by the API providers. In addition, since the membership classifier is trained by choosing an auxiliary dataset on behalf of all non-member samples and using the model’s generated image distribution to approximate the actual member distribution, the attack model can be overfitted, and the generalization of the membership boundary is questionable. Instead of directly developing attack methods on the target model, Pang et al.[108] propose utilizing an auxiliary dataset to train a shadow model that mimics the behavior of the target model. Subsequently, this shadow model is employed to generate data for training the attack models. Derived directly from the objective function of DMs, the authors demonstrate that the similarities between the generated image and the query image of the member samples are significantly higher compared to the non-member samples. Therefore, given a query image, the attacker first utilizes its caption, either the actual caption or one generated by a third-party captioning tool (e.g., BLIP2[141]), to query the target model and obtain the generated image. Following this, the attacker employs an image encoder (e.g., DETR[142], BEiT[143], DeiT[144], etc.) to extract the features of both the query and generated images, then compute their semantic similarity score based on these embedding vectors. This similarity is then used to determine the membership by applying various methods, including threshold comparison, distribution-based analysis, and MLP classifiers. Concurrently, Li et al.[109] analyze the privacy of DDIM and stable diffusion [9] when attackers utilize the variation API, i.e., the image-to-image procedure, to alter the candidate image and compare it with the original, subsequently inferring the membership. Specifically, when querying a DDIM with a candidate image x, the attacker first diffuses it into a noisy image by randomly incorporating t-step Gaussian noise ϵ0∼⁢(0,)similar-tosubscriptitalic-ϵ00 _0 (0,I)ϵ0 ∼ N ( 0 , I ) following (4). Then, the target model’s sampling process is employed to denoise the noisy image to obtain the reconstructed image x^isubscript x_iover start_ARG x end_ARGi. By performing the variation API n independent times to obtain n reconstructed images x^1,x^2,…,x^nsubscript^1subscript^2…subscript^\ x_1, x_2,..., x_n\ over start_ARG x end_ARG1 , over start_ARG x end_ARG2 , … , over start_ARG x end_ARGn , subsequently the final reconstructed image is computed by taking average: x^=1n⁢(x^1+x^2+…+x^n)^1subscript^1subscript^2…subscript x= 1n( x_1+ x_2+...+ x_n)over start_ARG x end_ARG = divide start_ARG 1 end_ARG start_ARG n end_ARG ( over start_ARG x end_ARG1 + over start_ARG x end_ARG2 + … + over start_ARG x end_ARGn ). Intuitively, averaging over multiple independent reconstructed images x^isubscript x_iover start_ARG x end_ARGi would greatly reduce the reconstruction error, make it distinguishable from the non-member samples. Exploiting this, the authors first train a ResNet18-based[133] binary classifier fRsubscriptf_Rfitalic_R as a proxy task based on the reconstruction error: v=‖x^−x‖norm^v=|| x-x||v = | | over start_ARG x end_ARG - x | |. Then, the distance between the query image x and the variational image x^ xover start_ARG x end_ARG is computed by taking the negative value of probability that sample is predicted as a member: D⁢(x,x^)=−fR⁢(v)^subscriptD(x, x)=-f_R(v)D ( x , over start_ARG x end_ARG ) = - fitalic_R ( v ). Finally, the membership is determined by comparing the distance with a certain threshold. Although this method can work in a black-box setting, similar to [106], it still requires access to a subset of the target model’s actual training data, which is not realistic in most scenarios. I-C2 Gray-box MIA Under the gray-box setting, Duan et al. [110] introduce SecMI, which aims to infer membership of both DDPM and stable diffusion models. Assuming that the member and non-member sets share the same distribution, SecMI assesses the posterior estimation with deterministic sampling and reversing processes to infer the membership. Specifically, at timestep t, DMs try to approximately estimate the reverse process with the forward process by minimizing the following loss function: ℓt=Eq⁢[12⁢σt2⁢‖μt~⁢(xt,x0)−μθ⁢(xt,t)‖],subscriptℓsubscriptdelimited-[]12subscriptsuperscript2norm~subscriptsubscriptsubscript0subscriptsubscript _t=E_q [ 12σ^2_t|| _t(x_t,x_% 0)- _θ(x_t,t)|| ]\ ,ℓitalic_t = Eitalic_q [ divide start_ARG 1 end_ARG start_ARG 2 σ2italic_t end_ARG | | over~ start_ARG μitalic_t end_ARG ( xitalic_t , x0 ) - μitalic_θ ( xitalic_t , t ) | | ] , (48) where μt~⁢(xt,x0)~subscriptsubscriptsubscript0 _t(x_t,x_0)over~ start_ARG μitalic_t end_ARG ( xitalic_t , x0 ) is the mean of posterior distribution q⁢(xt−1|xt,x0)conditionalsubscript1subscriptsubscript0q(x_t-1|x_t,x_0)q ( xitalic_t - 1 | xitalic_t , x0 ) (i.e., diffusion process), μθ⁢(xt,t)subscriptsubscript _θ(x_t,t)μitalic_θ ( xitalic_t , t ) refers to the estimated mean for pθ⁢(x^t−1|xt)subscriptconditionalsubscript^1subscriptp_θ( x_t-1|x_t)pitalic_θ ( over start_ARG x end_ARGt - 1 | xitalic_t ) (i.e., denoising process), and θ denotes the weights of the DM. Then, the local estimation error of a single data point x0subscript0x_0x0 at timestep t can be approximated as: ℓt,x0=‖x^t−1−xt−1‖2,subscriptℓsubscript0superscriptnormsubscript^1subscript12 _t,x_0=|| x_t-1-x_t-1||^2,ℓitalic_t , x start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT = | | over start_ARG x end_ARGt - 1 - xitalic_t - 1 | |2 , (49) where x^t−1∼pθ⁢(x^t−1|xt)similar-tosubscript^1subscriptconditionalsubscript^1subscript x_t-1 p_θ( x_t-1|x_t)over start_ARG x end_ARGt - 1 ∼ pitalic_θ ( over start_ARG x end_ARGt - 1 | xitalic_t ) and xt−1∼q⁢(xt−1|xt,x0)similar-tosubscript1conditionalsubscript1subscriptsubscript0x_t-1 q(x_t-1|x_t,x_0)xitalic_t - 1 ∼ q ( xitalic_t - 1 | xitalic_t , x0 ). In addition, by leveraging deterministic properties of DDIM[5], the authors define two deterministic processes: xt+1=ϕθ⁢(xt,t)=α¯t+1⁢fθ⁢(xt,t)+1−α¯t+1⁢ϵθ⁢(xt,t)subscript1subscriptitalic-ϕsubscriptsubscript¯1subscriptsubscript1subscript¯1subscriptitalic-ϵsubscriptx_t+1= _θ(x_t,t)= α_t+1f_θ(x_t,t)+% 1- α_t+1 _θ(x_t,t)xitalic_t + 1 = ϕitalic_θ ( xitalic_t , t ) = square-root start_ARG over¯ start_ARG α end_ARGt + 1 end_ARG fitalic_θ ( xitalic_t , t ) + square-root start_ARG 1 - over¯ start_ARG α end_ARGt + 1 end_ARG ϵitalic_θ ( xitalic_t , t ) (50) xt−1=ψθ⁢(xt,t)=α¯t−1⁢fθ⁢(xt,t)+1−α¯t−1⁢ϵθ⁢(xt,t),subscript1subscriptsubscriptsubscript¯1subscriptsubscript1subscript¯1subscriptitalic-ϵsubscriptx_t-1= _θ(x_t,t)\\ = α_t-1f_θ(x_t,t)+ 1- α_t-1% _θ(x_t,t),xitalic_t - 1 = ψitalic_θ ( xitalic_t , t ) = square-root start_ARG over¯ start_ARG α end_ARGt - 1 end_ARG fitalic_θ ( xitalic_t , t ) + square-root start_ARG 1 - over¯ start_ARG α end_ARGt - 1 end_ARG ϵitalic_θ ( xitalic_t , t ) , (51) where fθ⁢(xt,t)=xt−1−α¯t⁢ϵθ⁢(xt,t)α¯t.subscriptsubscriptsubscript1subscript¯subscriptitalic-ϵsubscriptsubscript¯f_θ(x_t,t)= x_t- 1- α_t _θ(x_% t,t) α_t.fitalic_θ ( xitalic_t , t ) = divide start_ARG xitalic_t - square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵitalic_θ ( xitalic_t , t ) end_ARG start_ARG square-root start_ARG over¯ start_ARG α end_ARGt end_ARG end_ARG . Subsequently, for a given sample x0subscript0x_0x0, at timestep t, the approximated posterior estimation error is computed following (49), (50), (51) and defined as t-error: ℓ~t,x0=‖ψθ⁢(ϕθ⁢(x~t,t),t)−x~t‖2,subscript~ℓsubscript0superscriptnormsubscriptsubscriptitalic-ϕsubscript~subscript~2 _t,x_0=|| _θ( _θ( x_t,t),% t)- x_t||^2,over~ start_ARG ℓ end_ARGt , x start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT = | | ψitalic_θ ( ϕitalic_θ ( over~ start_ARG x end_ARGt , t ) , t ) - over~ start_ARG x end_ARGt | |2 , (52) where x~t=ϕθ⁢(⋯⁢ϕθ⁢(ϕθ⁢(x0,0),1),t−1)subscript~subscriptitalic-ϕ⋯subscriptitalic-ϕsubscriptitalic-ϕsubscript0011 x_t= _θ(·s _θ( _θ(x_0,0),1),% t-1)over~ start_ARG x end_ARGt = ϕitalic_θ ( ⋯ ϕitalic_θ ( ϕitalic_θ ( x0 , 0 ) , 1 ) , t - 1 ) is the deterministic reverse process. After computing estimation error ℓ~tS⁢E⁢C,x0subscript~ℓsubscriptsubscript0 _t_SEC,x_0over~ start_ARG ℓ end_ARGt start_POSTSUBSCRIPT S E C , x0 end_POSTSUBSCRIPT at a empirically selected timestep tS⁢E⁢Csubscriptt_SECtitalic_S E C given a sample x0subscript0x_0x0, membership can be inferred by comparing this error with a threshold or employing a binary classifier. In another work, Tang et al.[111] directly adopt t-error from [110] following (52) and learn a quantile regression model qα⁢(x)subscriptq_α(x)qitalic_α ( x ) that predicts the α-quantile of the t-error ℓ~t,xsubscript~ℓ _t,xover~ start_ARG ℓ end_ARGt , x for each sample x in the auxiliary dataset D, where α is a parameter controlling the false positive rate. As a result, instead of applying a uniform threshold across all samples, [111] employs qαsubscriptq_αqitalic_α to predict the sample-conditioned α-quantile of the t-error for each sample and uses it as the fine-grained per-sample threshold to classify the membership status. Having a similar idea of leveraging the deterministic properties of DDIM[5], Kong et al.[112] propose PIA, which can significantly reduce the number of required queries compared to SecMI[110]. Specifically, in the DDIM framework, if standard deviation σtsubscript _tσitalic_t = 0, the process of adding noise becomes deterministic. Thus, given two points x0subscript0x_0x0 and xksubscriptx_kxitalic_k, we can determine any other points xtsubscriptx_txitalic_t, which is called ground-truth trajectory and obtained from x0subscript0x_0x0 and xksubscriptx_kxitalic_k: xt=α¯t⁢x0+1−α¯t⋅ϵ¯k,subscriptsubscript¯subscript0⋅1subscript¯subscript¯italic-ϵx_t= α_tx_0+ 1- α_t· % ε_k,xitalic_t = square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ⋅ over¯ start_ARG ϵ end_ARGk , (53) where ϵ¯k=xk−α¯k⁢x01−α¯ksubscript¯italic-ϵsubscriptsubscript¯subscript01subscript¯ ε_k= x_k- α_kx_0 1- % α_kover¯ start_ARG ϵ end_ARGk = divide start_ARG xitalic_k - square-root start_ARG over¯ start_ARG α end_ARGk end_ARG x0 end_ARG start_ARG square-root start_ARG 1 - over¯ start_ARG α end_ARGk end_ARG end_ARG is the noise added to x0subscript0x_0x0 to obtain xksubscriptx_kxitalic_k. The authors choose k=00k=0k = 0 and approximately compute the noise ϵ¯0subscript¯italic-ϵ0 ε_0over¯ start_ARG ϵ end_ARG0 initialized at t=00t=0t = 0 following: ϵ¯0≈ϵθ⁢(α¯0⁢x0+1−α¯0⁢ϵ¯0,0)≈ϵθ⁢(x0,0).subscript¯italic-ϵ0subscriptitalic-ϵsubscript¯0subscript01subscript¯0subscript¯italic-ϵ00subscriptitalic-ϵsubscript00 ε_0≈ _θ( α_0x_0+ % 1- α_0 ε_0,0)≈ _θ(x_0,0).over¯ start_ARG ϵ end_ARG0 ≈ ϵitalic_θ ( square-root start_ARG over¯ start_ARG α end_ARG0 end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARG0 end_ARG over¯ start_ARG ϵ end_ARG0 , 0 ) ≈ ϵitalic_θ ( x0 , 0 ) . (54) Subsequently, following Eq.(53) and Eq.(54), we can obtain the ground-truth points xtsubscriptx_txitalic_t and xt−t′subscriptsuperscript′x_t-t xitalic_t - t′, and sample the predicted point xt−t′subscriptsuperscript′x _t-t x′italic_t - t′ from xtsubscriptx_txitalic_t following the deterministic sampling process of DDIM[5] (26) when σtsubscript _tσitalic_t = 0. Following this, the distance between xt−t′subscriptsuperscript′x_t-t xitalic_t - t′ and xt−t′subscriptsuperscript′x _t-t x′italic_t - t′ is computed using lp−normsubscriptnorml_p-normlitalic_p - norm and ends up with the simple form: Rt,psubscript R_t,pRitalic_t , p =‖ϵ¯0−ϵθ⁢(α¯t⁢x0+1−α¯t⁢ϵ¯0,t)‖pabsentsubscriptnormsubscript¯italic-ϵ0subscriptitalic-ϵsubscript¯subscript01subscript¯subscript¯italic-ϵ0 =|| ε_0- _θ( α_tx_% 0+ 1- α_t ε_0,t)||_p= | | over¯ start_ARG ϵ end_ARG0 - ϵitalic_θ ( square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG over¯ start_ARG ϵ end_ARG0 , t ) | |p (55) ≈‖ϵθ⁢(x0,0)−ϵθ⁢(α¯t⁢x0+1−α¯t⁢ϵθ⁢(x0,0),t)‖p.absentsubscriptnormsubscriptitalic-ϵsubscript00subscriptitalic-ϵsubscript¯subscript01subscript¯subscriptitalic-ϵsubscript00 ≈|| _θ(x_0,0)- _θ( % α_tx_0+ 1- α_t _θ(x_0,0),t)||_p.≈ | | ϵitalic_θ ( x0 , 0 ) - ϵitalic_θ ( square-root start_ARG over¯ start_ARG α end_ARGt end_ARG x0 + square-root start_ARG 1 - over¯ start_ARG α end_ARGt end_ARG ϵitalic_θ ( x0 , 0 ) , t ) | |p . (56) Finally, the membership is inferred by comparing Rt,psubscriptR_t,pRitalic_t , p with a threshold τ. Concurrently, Zhai et al.[113] propose a gray-box method targeting text-conditional DMs, utilizing conditional likelihood discrepancy (CLiD). Specifically, recall that the training objective of DMs is to estimate the reverse process matching with the forward process, i.e., minimize ℒ(x0)=q[DKL(q(xt−1|xt,x0)||pθ(xt−1|xt))]L(x_0)= _q [D_KL(q(x_t-1|x_t,x_0)||p_% θ(x_t-1|x_t)) ]L ( x0 ) = italic_q [ DKL ( q ( xitalic_t - 1 | xitalic_t , x0 ) | | pitalic_θ ( xitalic_t - 1 | xitalic_t ) ) ], the overfitting phenomenon of DMs tends to result in lower estimation errors for member samples xmsubscriptx_mxitalic_m compared to non-member samples xnsubscriptx_nxitalic_n: ℒ⁢(xm)≤ℒ⁢(xn)ℒsubscriptℒsubscriptL(x_m) (x_n)L ( xitalic_m ) ≤ L ( xitalic_n ). This difference becomes more salient when it comes to text-conditional DMs, where the denoising process is conditioned by the caption c: ℒ⁢(xn,cn⁢u⁢l⁢l)−ℒ⁢(xm,cn⁢u⁢l⁢l)≤ℒ⁢(xn,c)−ℒ⁢(xm,c),ℒsubscriptsubscriptℒsubscriptsubscriptℒsubscriptℒsubscriptL(x_n,c_null)-L(x_m,c_null) (x_n,c% )-L(x_m,c),L ( xitalic_n , citalic_n u l l ) - L ( xitalic_m , citalic_n u l l ) ≤ L ( xitalic_n , c ) - L ( xitalic_m , c ) , (57) where cn⁢u⁢l⁢lsubscriptc_nullcitalic_n u l l denotes an empty text input to the text-conditional DM. Given I⁢(x,c)I(x,c)I ( x , c ) is the conditional likelihood discrepancy of image-text data point (x,c)(x,c)( x , c ): I⁢(x,c)=ℒ⁢(x,cn⁢u⁢l⁢l)−ℒ⁢(x,c)ℒsubscriptℒI(x,c)=L(x,c_null)-L(x,c)I ( x , c ) = L ( x , citalic_n u l l ) - L ( x , c ), following (57), we have: I⁢(xm,c)≥I⁢(xn,c).subscriptsubscriptI(x_m,c)≥ I(x_n,c).I ( xitalic_m , c ) ≥ I ( xitalic_n , c ) . (58) And I⁢(x,c)I(x,c)I ( x , c ) can be expressed as: I⁢(x,c) I(x,c)I ( x , c ) =t,ϵ⁢[‖ϵθ⁢(xt,t,cn⁢u⁢l⁢l)−ϵ‖2]absentsubscriptitalic-ϵdelimited-[]superscriptnormsubscriptitalic-ϵsubscriptsubscriptitalic-ϵ2 = _t,ε [|| _θ(x_t,t,c_null)% -ε||^2 ]= italic_t , ϵ [ | | ϵitalic_θ ( xitalic_t , t , citalic_n u l l ) - ϵ | |2 ] (59) −t,ϵ⁢[‖ϵθ⁢(xt,t,c)−ϵ‖2]subscriptitalic-ϵdelimited-[]superscriptnormsubscriptitalic-ϵsubscriptitalic-ϵ2 - _t,ε [|| _θ(x_t,t,c)-% ε||^2 ]- italic_t , ϵ [ | | ϵitalic_θ ( xitalic_t , t , c ) - ϵ | |2 ] (60) Consequently, Zhai et al.[113] estimate I⁢(x,c)I(x,c)I ( x , c ) and ℒ⁢(x,c)ℒL(x,c)L ( x , c ) by performing Monte Carlo estimation, then combining them to obtain the attack feature and utilizing threshold-based attack and XGBoost-based[145] classifier to infer the membership. I-C3 White-box MIA Leveraging the overfitting tendency in deep neural networks, most MIAs follow the common assumption that samples from the training dataset exhibit the lower loss compared to non-member samples. Following this, while Hu et al.[114] and Matsumoto et al.[116] directly utilize DM’s loss at each timestep for threshold-based membership inference attacks, Dubiński et al.[117] focus on modifying the diffusion process to extract the model’s loss information at various perspectives to enhance their threshold-based and classifier-based attacks’ performances. In addition, [114] also proposes computing log-likelihood of the candidate image following [8] as a different attack feature for the threshold-based approach. On the other hand, Carlini et al. [100] argue that while threshold-based attacks are effective for non-membership inference, they lack precision for classifying member samples, which is considerably more important. Therefore, the authors propose adopting the Likelihood Ratio Attack (LiRA) from [100], which analyses the distribution of the losses to classify the membership, assuming that the members’ losses and non-members’ losses belong to two different distributions. To this end, LiRA first trains multiple shadow models f1,f2,…,fi,…subscript1subscript2…subscript…\f_1,f_2,...,f_i,...\ f1 , f2 , … , fitalic_i , … , each on a random subset of the training dataset, to imitate the behavior of the target model. Then, for a candidate sample x, the loss ℒ⁢(x;fi)ℒsubscriptL(x;f_i)L ( x ; fitalic_i ) obtained from the shadow model fisubscriptf_ifitalic_i is assigned to one of two sets: the set of member losses if fisubscriptf_ifitalic_i did see x during training, and the set of non-member losses if x was not in fisubscriptf_ifitalic_i’s training data. Following this, the member loss distribution DmsubscriptD_mDitalic_m and non-member loss distribution DnsubscriptD_nDitalic_n are established by fitting Gaussians to the member losses and non-member losses sets. Finally, to predict the membership of the sample x with the target model f∗superscriptf^*f∗, it computes the loss l=ℒ⁢(x;f∗)ℒsuperscriptl=L(x;f^*)l = L ( x ; f∗ ) and then measures whether P⁢r⁢(l|Dm)>P⁢r⁢(l|Dn)conditionalsubscriptconditionalsubscriptPr(l|D_m)>Pr(l|D_n)P r ( l | Ditalic_m ) > P r ( l | Ditalic_n ), i.e. which distribution the loss of target sample is more likely to belong to. As a result, by training 16 class-conditional DMs on CIFAR-10 as the shadow models and computing the losses at timestep t=100100t=100t = 100, Carlini et al.[115] achieve a TPR of over 70% at a FPR of just 1%. In contrast, when applied to state-of-the-art classifiers, LiRA achieves less than 20% TPR at the same FPR. This demonstrates that DMs are significantly less private than classification models trained on the same data. Although Carlini et al.[115] employ a more insightful method to exploit the loss feature, this approach necessitates training multiple shadow models and retraining when obtaining new target data points. This process is highly computationally intensive, particularly for large-scale DMs. While relying on the loss information is an intuitive attacking approach, Pang et al.[118] propose a new framework called GSA that can be applied to both unconditional and text-conditional DMs. This approach brings up a novel perspective by leveraging the gradient information as the attack feature, which might reflect more insightfully how the target model responds to member and non-member samples. However, large-scale models with a huge number of parameters, such as DMs, often yield extremely high-dimensional gradient information. Extracting this feature effectively and efficiently is still a challenging task. To this end, the authors propose two algorithms, G⁢S⁢A1subscript1GSA_1G S A1 and G⁢S⁢A2subscript2GSA_2G S A2, utilizing sub-sampling and aggregation techniques to reduce dimensionality and retain important information. Specifically, for K steps uniformly sub-sampled from the total diffusion steps T, G⁢S⁢A1subscript1GSA_1G S A1 computes loss value ℒtsubscriptℒL_tLitalic_t for each timestep t, then takes the mean value of these losses: ℒ¯=1|K|⁢∑t∈Kℒt¯ℒ1subscriptsubscriptℒ L= 1|K| _t∈ KL_tover¯ start_ARG L end_ARG = divide start_ARG 1 end_ARG start_ARG | K | end_ARG ∑t ∈ K Litalic_t (61) Following this, ℒ¯ℒ Lover¯ start_ARG L end_ARG is utilized to perform back-propagation on the target model with N layers of parameters, and produce gradients for each layer. Then, G⁢S⁢A1subscript1GSA_1G S A1 performs l2−limit-fromsubscript2l_2-l2 -norm on the gradient information of each layer to obtain their representative values, ultimately yielding the gradient vector G to train a machine learning model for membership inference: G←[||∂ℒ¯∂W1||22,||∂ℒ¯∂W2||22,…,||∂ℒ¯∂WN||22].G← [ | | ∂ L∂% W_1 | |^2_2, | | ∂ L% ∂W_2 | |^2_2,..., | | ∂% L∂W_N | |^2_2 ].G ← [ | | divide start_ARG ∂ over¯ start_ARG L end_ARG end_ARG start_ARG ∂ W1 end_ARG | |22 , | | divide start_ARG ∂ over¯ start_ARG L end_ARG end_ARG start_ARG ∂ W2 end_ARG | |22 , … , | | divide start_ARG ∂ over¯ start_ARG L end_ARG end_ARG start_ARG ∂ Witalic_N end_ARG | |22 ] . (62) While G⁢S⁢A1subscript1GSA_1G S A1 chooses to take the average of the losses before performing the back-propagation to reduce the computational overhead and accept the loss of information, G⁢S⁢A2subscript2GSA_2G S A2 opts to perform the back-propagation for the loss ℒtsubscriptℒL_tLitalic_t at each timestep t to obtain the gradient vector GtsubscriptG_tGitalic_t, then compute the average of them to output the final attack vector: G=1|K|⁢∑t∈KGt.1subscriptsubscriptG= 1|K| _t∈ KG_t.G = divide start_ARG 1 end_ARG start_ARG | K | end_ARG ∑t ∈ K Gitalic_t . (63) As a result, G⁢S⁢A2subscript2GSA_2G S A2 retains more useful information and achieves better performance, yet requires extremely high computational costs. IV Defenses for Diffusion Models It has been shown in section I that there are a variety of attacks threatening DMs. Thus, investigating possible countermeasures for these attacks is obviously an important topic. This section provides a comprehensive survey of state-of-the-art defense methods for DMs against the presented attacks. IV-A Countermeasures of Backdoor Attacks In practice, it is especially challenging to detect backdoor attacks targeting on DMs due to its stealthiness: (i) Backdoored DMs still perform normally on benign inputs, and (i) the input data space (e.g., image, text, audio) is usually too large for searching-based methods that try to identify the trigger. Furthermore, the processes of DMs are totally different from standard ML models like regression or classification models, making existing backdoor detection methods relying on labeled data[146, 147, 123] inapplicable for DMs. Fortunately, the diffusion and reverse processes of DMs do have certain special properties to be exploited for detecting the embedded backdoor patterns[148, 149]. The distribution shift caused by the backdoor trigger can be observed clearly from the presented TrojDiff and BadDiffusion methods. In the diffusion process of each work, the data distribution is shifted gradually from the target image x0∗superscriptsubscript0x_0^*x0∗ towards the noisy trigger xT∗superscriptsubscriptx_T^*xitalic_T∗. For instance, the forward transition of TrojDiff presented in (36) can be reparameterized into the following form: xt∗=αt⁢xt−1∗+kt⁢(1−γ)⁢δ⏟trigger shift+(1−αt)⁢γ⁢ϵ⏟noise shift,subscriptsuperscriptsubscriptsubscriptsuperscript1subscript⏟subscript1trigger shiftsubscript⏟1subscriptitalic-ϵnoise shiftx^*_t= _tx^*_t-1+ k_t(1-γ)δ_% trigger shift+ (1- _t)γε_% noise shift,x∗italic_t = square-root start_ARG αitalic_t end_ARG x∗italic_t - 1 + under⏟ start_ARG kitalic_t ( 1 - γ ) δ end_ARGtrigger shift + under⏟ start_ARG square-root start_ARG ( 1 - αitalic_t ) end_ARG γ ϵ end_ARGnoise shift , (64) where ϵ∈⁢(0,)italic-ϵ0ε (0,I)ϵ ∈ N ( 0 , I ), and δ is the backdoor trigger (i.e., the trigger distribution). As shown in Fig. 6, the “trigger shift” term represents a small amount of distribution shift that guides the data distribution towards the trigger distribution δ (i.e., the hello-kitty image in Fig. 6) in each diffusion step. The scale/amount of this trigger shift is kt⁢(1−γ)subscript1k_t(1-γ)kitalic_t ( 1 - γ ). Concurrently, the “noise shift” makes the data gradually resembles a Gaussian noise. The combination of both these shift terms results in the noisy trigger xT∗superscriptsubscriptx_T^*xitalic_T∗ at the final diffusion step (i.e., the noisy hello-kitty image in Fig. 6). This observation is also true for BadDiffusion when we apply reparameterization on BadDiffusion’s forward transition (40): xt∗=αt⁢xt−1∗+(1−αt)⁢xsδ⏟trigger shift+(1−αt)⁢ϵ⏟noise shift,subscriptsuperscriptsubscriptsubscriptsuperscript1subscript⏟1subscriptsuperscriptsubscripttrigger shiftsubscript⏟1subscriptitalic-ϵnoise shiftx^*_t= _tx^*_t-1+ (1- _t)x_s^% δ_ trigger shift+ (1- _t)ε_% noise shift,x∗italic_t = square-root start_ARG αitalic_t end_ARG x∗italic_t - 1 + under⏟ start_ARG ( 1 - square-root start_ARG αitalic_t end_ARG ) xitalic_sitalic_δ end_ARGtrigger shift + under⏟ start_ARG square-root start_ARG ( 1 - αitalic_t ) end_ARG ϵ end_ARGnoise shift , (65) where xsδsuperscriptsubscriptx_s^δxitalic_sitalic_δ is the noisy trigger image. In this case, the scale of the trigger shift is (1−αt)1subscript(1- _t)( 1 - square-root start_ARG αitalic_t end_ARG ). Figure 6: A visualization of the backdoored diffusion process in TrojDiff[62], from the perspective of distribution shift. Intuitively, the backdoored forward process integrates a trigger-related distribution shift ρt⁢δsubscript _tδρitalic_t δ that gradually shifts the target image towards the trigger distribution δ, with ρtsubscript _tρitalic_t is the scale of the shift. This means that in the reverse process, the UNet must be trained to reverse the forward process by enforcing another trigger-related distribution shift ρ~t⁢δsubscript~ ρ_t ~ start_ARG ρ end_ARGt δ that is in the reversed direction. Thus, backdoor attackers must preserve this distribution shift in every step along the Markov chain. Based on this insight, An et al.[148] proposed a trigger inversion method that can find the trigger δ by analyzing the distribution shift of the UNet’s output. The authors model the distribution shift as a linear dependence: xt∗−xt=λt⁢δsuperscriptsubscriptsubscriptsubscriptx_t^*-x_t= _t _t∗ - xitalic_t = λitalic_t δ and xt−1∗−xt−1=λt−1⁢δsuperscriptsubscript1subscript1subscript1x_t-1^*-x_t-1= _t-1 _t - 1∗ - xitalic_t - 1 = λitalic_t - 1 δ, where λtsubscript _tλitalic_t is a coefficient to model the trigger distribution shift. This means that if the UNet’s input contains the trigger, the UNet’s output must preserve the trigger distribution shift: xt⁢[M⁢(xt+λt⁢δ,t)]−xt⁢[M⁢(xt,t)]=λt−1⁢δ,subscriptsubscriptdelimited-[]subscriptsubscriptsubscriptsubscriptdelimited-[]subscriptsubscript1 _x_t[M(x_t+ _tδ,t)]- _x_t[M(x_t,t)]=% _t-1δ,italic_x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT [ M ( xitalic_t + λitalic_t δ , t ) ] - italic_x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT [ M ( xitalic_t , t ) ] = λitalic_t - 1 δ , (66) where M⁢(⋅)⋅M(·)M ( ⋅ ) is the UNet. As a result, the problem of finding the backdoor trigger δ can be formalized as the following objective: minδ⁡t⁢[‖xt⁢[M⁢(xt+λt⁢δ,t)]−xt⁢[M⁢(xt,t)]−λt−1⁢δ‖].subscriptsubscriptdelimited-[]normsubscriptsubscriptdelimited-[]subscriptsubscriptsubscriptsubscriptdelimited-[]subscriptsubscript1 _δ _t [\| _x_t[M(x_t+ _tδ,t)% ]- _x_t[M(x_t,t)]- _t-1δ\| ].minitalic_δ italic_t [ ∥ italic_x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT [ M ( xitalic_t + λitalic_t δ , t ) ] - italic_x start_POSTSUBSCRIPT t end_POSTSUBSCRIPT [ M ( xitalic_t , t ) ] - λitalic_t - 1 δ ∥ ] . (67) However, minimizing the above objective over all timesteps t from T to 1 is computationally inefficient. Therefore, the authors decided to only consider the step T, with xT∼⁢(0,)similar-tosubscript0x_T (0,I)xitalic_T ∼ N ( 0 , I ) and xT∗∼⁢(δ,)similar-tosubscriptsuperscriptx^*_T (δ,I)x∗italic_T ∼ N ( δ , I ). As a result, the objective is simplified to: minδ⁡‖ϵ∼⁢(0,1)⁢[M⁢(ϵ+δ,T)]−λ⁢δ‖.subscriptnormsubscriptsimilar-toitalic-ϵ01delimited-[]italic-ϵ _δ\| _ε (0,1)[M(ε+δ,T)]-% λδ\|.minitalic_δ ∥ italic_ϵ ∼ N ( 0 , 1 ) [ M ( ϵ + δ , T ) ] - λ δ ∥ . (68) Once a candidate trigger δ is figured out, we can verify whether the model is truly backdoored via the following observation: If a model is backdoored by a trigger δ, sampling from xT∗∼⁢(δ,)similar-tosuperscriptsubscriptx_T^* (δ,I)xitalic_T∗ ∼ N ( δ , I ) consistently results in the target image. Otherwise, if the input noise is benign (i.e., xT∼⁢(0,)similar-tosubscript0x_T (0,I)xitalic_T ∼ N ( 0 , I )), the generation results would be very diverse due to stochastic factors of DMs during the reverse process. Therefore, by sampling N times using the candidate trigger, then estimating a similarity score (e.g., KL Divergence, cosine similarity, total variance loss, or absolute distance) between different trials, one can confidently say that a DM is backdoored if the computed similarity is abnormally high. However, the main problem here is to choose a suitable “similarity threshold” to determine if a DM is backdoored. The authors in[148] use total variance loss and absolute distance between different trials to compute the score for each DM, then construct a random forest consisting of hundreds of DMs (both clean and backdoored) to figure out a suitable score threshold. On the other hand, Guan et al.[150] compute the cosine similarity between each pair of images generated from a sampled input, then constructing a similarity graph to assess the overall score for the input. If the overall score is higher than a predefined threshold, the input noise is considered a backdoor trigger. In this work, the threshold is calculated based on benign/clean samples. However, this work assumes that the trigger has been recognized earlier, and their method only verifies whether this candidate trigger is truly a backdoor trigger. This is not a reasonable assumption in practice, as inversing the trigger of a backdoored DM is often much harder than verifying a potential trigger. In another work, Sui et al.[151] argue that the distribution of a backdoor trigger should be significantly different from the distribution of a standard Gaussian noise. Therefore, the authors compute the KL Divergence between the UNet’s input xTsubscriptx_Txitalic_T and a Gaussian noise, then choose a statistical threshold which is equal to the mean score plus three times of the score variance. However, using xTsubscriptx_Txitalic_T to compute the score might not be an efficient approach, as there are certain methods that can make the trigger almost invisible and resembles a normal noise (e.g., the reviewed framework in [67]). Furthermore, the work[151] cannot inverse the trigger; it is just limited to verifying whether an input noise is a backdoor trigger. Beyond backdoor detection, An et al.[148] also proposed a method to remove the backdoor effect embedded in a particular DM. This method purifies a backdoored DM so it will generate benign outputs even if the input is the backdoor trigger. To do so, the authors proposed a loss function that shifts the backdoored output’s distribution towards the benign distribution. This loss function is the absolute distance between (i) the DM’s output in presence of backdoored trigger and (i) the output of the frozen DM when using benign input. Especially, this can be done without accessing the real data. TABLE VII: A summary of countermeasures for adversarial attacks on DMs. CA refers to cross-attention layers, while TE is text encoder. Targeted Component Reference UNet CA TE Guidance Method Strategy ESD[152] ✓ ✓ CFG Fine-Tuning Unlearning AC[153] ✓ Anchor Concept, Poisoning Dataset Fine-Tuning Unlearning FMN[154] ✓ Attention Maps Fine-Tuning Unlearning ABO[155] ✓ ✓ Replacing Concept, CFG Fine-Tuning Unlearning SLD[156] CFG Inference Unlearning EraseDiff[157] ✓ ✓ Crafted Dataset Retraining Unlearning Erasing[158] ✓ Reversed SGD Fine-Tuning Unlearning DT[159] ✓ ✓ Scrambled-Grid Dataset Fine-Tuning Unlearning GuardT2I[160] ✓ Verbalizer, BERT Prompt Verification Red-Teaming Pruning[161] ✓ ✓ Objective of ESD or AC Model Pruning Unlearning MCU[162] ✓ ✓ KL Divergence Fine-Tuning Unlearning SA[163] ✓ ✓ EWC, GR Fine-Tuning Unlearning RTSF[164] ✓ Hand-Crafted Prompt Prompt Verification Red-Teaming SPM[165] ✓ ✓ Anchor Concept, PEFT Fine-Tuning Unlearning Receler[166] ✓ ✓ CFG, PEFT, adversarial loss Fine-Tuning Unlearning UC[167] ✓ ✓ Anchor Concept, GAN discriminator Fine-Tuning Unlearning RACE[168] ✓ ✓ Adversarial loss Fine-Tuning Unlearning P4D[169] ✓ Adversarial noise Prompt Generation Red-Teaming IV-B Countermeasures of Adversarial Attacks In general, the main purpose of adversarial attacks on DMs is to generate inappropriate content that is low-quality or unaligned with the provided text prompt. In certain circumstance, the generated content can be harmful, including NSFW concepts like racism, horror, politics, and violence. In general, there are three efficient approaches to eliminate such NSFW concepts from generated adversarial images, which are safety filter algorithms, red-teaming techniques, and MU[124, 125], which is also referred as concept erasing/forgetting in DMs. IV-B1 Safety Filter and Red-Teaming Tools Safety filter is often deployed in various public DMs. It compares the latent representation of the generated image with pre-computed representations of multiple NSFW concepts. If the similarity score between the image’s representation and any NSFW embedding is higher than a threshold, the generated image is considered NSFW and it is filtered out. However, it has been shown that these safety filters are still vulnerable and they can be bypassed easily by different methods, including both hand-crafted and learnable ones[164, 169]. On the other hand, red-teaming methods[160, 164, 169] resist against adversarial attacks by proactively figuring out potential adversarial prompts that can make DMs generate NSFW content. In terms of red-teaming for DMs, Rando et al.[164] reverse-engineer the stable diffusion’s safety filter to identify its vulnerabilities in generating NSFW concepts. The authors find out that only sexual content is prevented by safety filters, while other NSFW concepts like violence, gore, and vulgar content are ignored. Furthermore, it has been shown that diluting input prompts by adding extra (irrelevant) details into the adversarial prompts can help bypassing safety filters as this disturbs the prompt’s embedding, making the similarity score lower than the safety filter’s threshold. These insights are important in figuring out potential adversarial prompts, thereby improving safety filters and designing stronger adversarial defense systems. Nevertheless, [164] depend entirely on human handcrafting, which is not scalable. To enhance scalability, another red-teaming tool named P4D is proposed in[169], which is a learnable method that can find adversarial prompts by using a prompt-engineering approach. P4D finds an alternative prompt that is different from the target sensitive prompt, but the generated images are still similar to each other. This is done in the latent space, in which the alternative prompt is optimized by minimizing the different between (i) the noises predicted by the DM when using the original sensitive prompt as input, and (i) the noise predicted when using the adversarial prompt. While the goal of [164] and [169] is to search for adversarial prompts, GuardT2I[160] aims to verify whether a provided prompt is adversarial or not. In particular, GuardT2I analyzes the text embeddings of stable diffusion for prompt verification. First, it trains an autoregressive LLM that receives the DM’s text embeddings as conditions, then generates a prompt interpretation that convey the meaning of the text embedding. Then, the authors proposed two modules to check if the prompt is adversarial: (i) the first module is a verbalizer that checks if the interpreted prompt contains sensitive words; and (i) the second module is a BERT-based sentence similarity checker that verifies whether the original prompt and the interpreted prompt is similar to each other. If the verbalizer finds out sensitive words, or the similarity checker shows that the two prompts are dissimilar, the prompt is considered NSFW. IV-B2 Machine Unlearning for Concept Erasing Another efficient approach to tackle DM-targeted adversarial attack is MU-based concept erasing. Specifically, these techniques allow DMs to forget the knowledge of a particular concept such as nudity and violence, making it unable to generate these types of images no matter the provided prompt. This can be implemented via retraining, fine-tuning, or intervening in the inference. In terms of model retraining, Erasediff[157] divides the dataset into two parts: The first part contains the forgetting data, while the second part contains the remaining (normal) data. It retrains the DM on the normal data with the same loss as standard DDPMs, which minimizes the ELBO. On the other hand, it fine-tunes the DM on the forgetting data with an opposite loss that maximizes the ELBO. This approach is limited in terms of scalability as it requires retraining the DMs on a large dataset. ESD[152] is among the first studies that investigate fine-tuning for concept erasing. The authors invert the sign of classifier-free guidance (CFG) to obtain a negative guidance term. This negative guidance is used to fine-tune DMs, moving the data distribution away from a specific condition c. As a result, it minimizes the probability that the generated images are labeled as c. The authors also proposed two fine-tuning strategies. The first strategy only fine-tunes cross-attention modules without intervening on other parameters. The second strategy only fine-tunes unconditional layers and keep cross-attention parameters unchanged. Experimental results showed that the first strategy only erases a specific content/style (e.g., Van Gogh) when its name is mentioned in the text prompt. On the other hand, the second strategy offers a more general and universal effect. For example, it removes “nudity” globally even in case this word is not mentioned in the prompt. Another framework called RACE[168] is built on top of the training objective ESD[152] with certain enhancement to improve the performance of concept erasing. In this method, the authors first simulate an adversarial attack that learns a perturbation σ to make the targeted DM generates an unwanted erasing concept. This perturbation is integrated into the objective function of ESD to fine-tune the target DM. In the experiments, it has been shown that RACE achieves a better performance compared to ESD. Also based on the principles of CFG, the authors in [156] proposed SLD, a framework that deploys concept erasing at the inference instead of fine-tuning. In addition to the normal CFG term, the authors add a safety guidance term that guide the DM not to generate an erasing concept that we aim to remove. This safety guidance term is similar to the CFG term, but the condition is such the erasing concept. This term is controlled by a scaling factor that provides a trade-off between removing inappropriate content and keeping the changes minimal for a high-quality generation result. Receler[166] also use an objective that is based on negative guidance, which is similar to ESD. However, the authors introduce the following improvements: (i) the work applies parameter efficient fine-tuning (PEFT) to lower the cost of fine-tuning; (i) it introduces a concept-localized regularization that helps ensuring locality of generated images on non-forgetting concepts; (i) the erasing loss is trained concurrently against an adversarial loss that aims to generate adversarial examples. As a results, Receler is more robust than SLD and ESD in various experiments like nudity-erasing and adversarial attacks. While ESD, SLD, and Receler use CFG in different ways to mislead DMs, in another work named FMN, Zhang et al.[154] conduct concept erasing by applying a loss function that minimizes attention maps of the fine-tuned DMs. However, FMN faces difficulties in forgetting abstract concepts. Instead of misleading DMs away from the erasing concept, the work called AC presented in[153] redirects the DM’s distribution into another anchor concept that is more general than the original one. For example, the anchor concept of “A Grumpy cat” is “A cat”. By making the targeted DM understand “A Grumpy cat” as “A cat”, it can be said that the DM has forgotten the concept of “Grumpy cat”. To do so, the authors proposed two methods to fine-tune DMs. In the first method, a frozen DM is used with the anchor prompt “A cat” to teach the fine-tuned DM, given that the input of the fine-tune DM is the original prompt “A Grumpy cat”. The second method fine-tunes the DM with modified image-text data. For example, the text label is “A Grumpy cat”, but the image is just a random cat (not Grumpy). Both methods make the fine-tuned DM treat “Grumpy cat” as normal cats. The combination of CFG and anchor concept is introduced by Hong et al.[155] in a work named ABO. Similar to AC[153], ABO also uses an anchor concept, with a frozen teacher DM to teach and fine-tune a student DM based on this concept. However, the anchor concept in ABO is defined by users instead of using a more general concept. This forms a concept loss term that makes the student DM forgets knowledge about the erasing concept. In addition, this paper also uses CFG to introduce a penalty loss term that provides unconditional feedback. As a result, this method outperforms previous work like ESD, AC, and FMN in various metrics like FID, CLIP Score, and Structural Similarity Index metric (SSIM). The use of an anchor concept for concept erasing is also exploited in UC[167] and SPM[165]. SPM[165] employs parameter efficient fine-tuning (PEFT). The authors design a trainable lightweight adapter that can be inserted into different layers of the DM’s neural network. This adapter only has one dimension; thus, it can enable precise concept erasing with very low cost. To train this scheme, the authors proposed a latent anchoring method, in which a surrogate concept is chosen and combined with the target concept to fine-tune the model. The objective function thus consisting of two terms, an erasing loss and an anchoring loss, ensuring that the concept is erased while the impact on other retaining concepts is minimal. As a result, this framework achieves a better performance than previous studies like ESD, ConAbl, and SA in terms of FID score, while only lower than ESD in CS and CER. In UC[167], the authors design a method for unlearning concepts that is inspired by GAN. Similar to SPM[165], AC[153], and ABO[155], UC also misleads the erasing concept to an anchor concept. This is done by using a GAN-based scheme in the latent space, in which a discriminator tries to distinguish if the predicted noise is from the erasing concept or from the anchor concept, while the DM is fine-tuned to ensure that the discriminator cannot distinguish this. It is formulated as a min-max objective that optimizes both the DM and the discriminator at the same time. The performance of this work is comparable to ESD in terms of concept unlearning, while it performs similarly to SPM regarding style unlearning. However, due to using an anchor concept, the presented methods necessitate a significant resource for preparing fine-tuning data. To facilitate data preparation, the authors in[163] proposed SA, which enables concept erasing by using conditional samples generated by such the DMs prior to fine-tuning. The loss function of SA is derived based on two continual learning techniques, Elastic Weight Consolidation (EWC) and Generative Replay (GR), which enables controllable forgetting and works across different types of generative models and conditioning schemes. As a result, it outperforms SLD and ESD in terms of classification-based scores like GCD. In terms of erasing robustness, the authors in[161] argued that previous studies like ESD or AC just “deactivate” some neurons to remove the erasing concept. However, they can be reactivated if the input prompts are cleverly crafted. Therefore, this paper aims to prune/remove such neurons that associated with the erasing concept. First, the authors use an existing concept-erasing objective like ESD or AC to identity these neurons and estimate their sensitivity. With a predefined sensitivity threshold, a hard mask is derived so that it decides which neurons to prune. As a result, this method achieves a significantly higher performance than previous methods like ESD, FMN, and AC given that the adversarial strategy is UnlearnDiff[81] and P4D[169]. However, most presented methods are only applicable for DMs. The authors in[162] generalize the training objective to make it applicable for a wide range of generative models instead of only DMs. Specifically, the optimization goal is to minimize the KL divergence between the generated results of the retaining dataset (to ensure generation quality), while maximizing the KL divergence between those of the erasing dataset (to make the models forget that concept). However, since KL divergence between high-dimensional images is intractable, the authors proposed a more efficient method that convert KL divergence into a L2 loss, based on mutual information (MI). Since this objective is not specific to any model type, it can be used for various categories of generative models like VQ-VAE, MAE, and DMs. Fine-tuning the denoising model is a popular approach for concept erasing. However, it might take significant time and computational resource in practice. The authors in[158] instead fine-tune only the text encoder of DMs to achieve concept erasing. It simply updates the text encoder with a reverse direction of gradient descent, using several images of the target concept. Although this method is very simple, it improves the fine-tuning time significantly, which takes only around 8 seconds. However, the erasing robustness is limited than previous studies. In another work[159], the authors figure out two observations: (i) Low-frequency features (e.g., general shape, structure, and undetailed content) is the primary components of distribution distance, and they are steered by conditional information; and (i) the semantic information of generated images are affected significantly by variations in the initial noise. Thus, to make DMs forget unwanted concepts, the authors proposed to use scrambled grid to disrupt visually low-frequency features of the erasing concepts, forming a degraded dataset. Then, this dataset is used to fine-tune DMs. As a result, it achieves a higher performance than SLD and ESD in terms of CLIP, FID, and IS scores. IV-C Countermeasures of Membership Inference Attacks MIAs pose a significant threat to the privacy of DMs trained on sensitive data. In order to mitigate these risks, several defense strategies have been proposed, including regularization, differential privacy, and knowledge distillation. They are presented as follows. IV-C1 Regularization MIA methods primarily benefit from the overfitting tendency in deep learning models by leveraging the differences in their behaviors when processing member and non-member samples. To mitigate privacy risks associated with these attacks, a straightforward defense approach is to apply regularization techniques to enhance the generalization capability of the target model, thereby reducing the overfitting phenomenon and mitigating the model’s vulnerability. Specifically, Zhai et al.[113] show that removing the default data augmentation techniques from the target model’s training scripts makes it more vulnerable to their attack methods. In addition, both Duan et al.[110] and Pang et al.[118] find that applying Cutout[170] and RandomHorizontalFlip techniques to the target model’s training process causes their attack performance decrease to a certain degree. However, when employing RandAugment[171] and l2subscript2l_2l2-regularization, [110] and [118] observe that DMs fail to converge during training, resulting in generating extremely low-quality images. IV-C2 Differential Privacy Applying differential privacy (DP)[172] perturbation on training, the work in[173] stands as a standard mechanism for defending against privacy attacks. In particular, DMs can be trained with differentially private stochastic gradient descent (DP-SGD)[174], where the gradients are clipped and noised during training to diminish the models’ memorization of individual samples, thereby avoiding leaking sensitive information from the training datasets. However, in practice, applying DP-SGD often results in an unacceptable trade-off between data privacy and the model’s utility. Specifically, [108, 111, 114, 115], and [118] observe that employing DP-SGD during training causes DMs to fail to converge, which leads to a significant drop in the models’ performance and the generation of meaningless images. Furthermore, incorporating DP into training requires much higher computational costs and time, posing a significant challenge when applying to large-scale DMs. IV-C3 Knowledge Distillation The knowledge distillation strategy aims to transfer knowledge from a teacher model to a student model, enabling the student to attain performance comparable to that of the teacher. To this end, the outputs of the teacher model are used to train the student model, guiding it to mimic the behaviors exhibited by its teacher. Exploiting this mechanism to reduce data privacy risks, Fernandez et al.[175] propose Privacy Distillation, a framework that prevents text-conditional DMs from exposing identifiable information about their training data. To this end, they first train a private DM ϵθp⁢r⁢isubscriptsuperscriptitalic-ϵε^pri_θϵitalic_p r iitalic_θ on the real data Dr⁢e⁢a⁢lsubscriptDD_realDr e a l. Then, captions from Dr⁢e⁢a⁢lsubscriptDD_realDr e a l are fed into ϵθp⁢r⁢isubscriptsuperscriptitalic-ϵε^pri_θϵitalic_p r iitalic_θ to generate images, forming a synthetic dataset Ds⁢y⁢n⁢t⁢hsubscriptDℎD_synthDs y n t h. Following this, the authors train a Siamese neural network to calculate the re-identification score between generated images and real images, thereby filtering Ds⁢y⁢n⁢t⁢hsubscriptDℎD_synthDs y n t h by removing the synthetic images that are considered identifiable. Finally, the filtered synthetic dataset is used to train a public DM ϵθp⁢u⁢bsubscriptsuperscriptitalic-ϵε^pub_θϵitalic_p u bitalic_θ that can be shared openly and safely. Although Privacy Distillation significantly reduces re-identification risk by preventing public models from accessing real data and identifiable information, retraining large-scale DMs is computationally expensive. There are also several approaches that can be applied to mitigate the data privacy concerns arising from MIAs. However, there remains a significant gap in dedicated research on defense methods specifically tailored to DMs. V Open Challenges and Research Directions As discussed earlier, various studies regarding attacks and defenses for DMs have been proposed and cover three main branches of DM security. However, it can be seen that this field is still in its infancy, since many other issues of both attacks and countermeasures for DMs have not been investigated. Thus, this section presents a throughout discussion about open challenges and future research directions of DM security, inspiring future development towards more secure and privacy-preserving DM-based systems. V-A Backdoor Attack In backdoor attacks, the attackers must modify the diffusion processes, loss function, and training data to embed the triggers into the target DMs. As a result, the backdoor can be activated when the backdoor trigger is fed into the model’s input. However, the modifications applied on the backdoored DM eventually led to a significant downgrade on the model’s performance when non-trigger inputs are used. This not only impacts user experience, but also decreases the stealthiness of the backdoor attack if the generative results of the backdoored DM are compared to those generated by a benign model. Another limitation of current backdoor methods is that they are only applicable for image-based applications. It is known that DMs are also applied in other important domains like natural language processing (NLP)[29, 30, 31, 32, 33, 34], audio processing[35, 36], 3D generation[37, 38, 39, 40, 41, 42], bioinformatics[43, 44], and time series tasks[45, 46, 47]. Therefore, investigating backdoor attacks for these types of DMs is also a potential research direction. For example, one might try to backdoor an audio-based DM so that the target DM would generate harmful auditory content when the backdoor trigger is activated. In terms of future research direction, embedding multiple backdoor triggers into a single DM can be a potential method to make backdoor attacks resist to backdoor defense methods like trigger inversion and trigger purification. Even if trigger inversion is applied to find out a backdoor trigger then purify it, the attackers can still use other embedded triggers to activate the backdoor target. In more general cases, we can also design various variations of backdoor attacks such as multi-trigger single-target, single-trigger multi-target, and multi-trigger multi-target, and multi-modal backdoor attacks. V-B Backdoor Defense Trigger inversion is the foremost and also the most challenging stage of countering backdoor attacks. Once figuring out some candidate backdoor triggers via trigger inversion, it is more simple to verify whether these candidates are truly backdoor triggers based on various reviewed methods[148, 150, 151]. However, existing work only investigates backdoor inversion for image-based trigger[148], while neglecting another important case in which the trigger is embedded into the text prompt. Therefore, investigating robust techniques to inverse textual triggers is an important future research direction. However, text-based trigger inversion is often more challenging since there are various strategies to embed textual triggers into the text prompt, including word/character appending, prepending, and replacing. In addition, unlike image-based triggers, textual triggers are discrete and could not be learnt by traditional gradient descent methods. Furthermore, current backdoor countermeasures only consider single-trigger single-target attacks. In future work, it is possible that multiple triggers can be embedded into a DM at the same time, making trigger inversion more challenging. Therefore, investigating defense methods that can detect thoroughly all the triggers is an important research direction. V-C Adversarial Attack In existing studies, adversarial attack is usually carried out by adding a small perturbation into one of three target modalities, including input image[68, 70, 71, 72], text prompt[73, 74, 75, 76, 77, 78, 79, 80, 81], and fine-tuning images[82, 83, 84, 69]. It has been shown that conducting adversarial attacks on different input modalities brings different extents of attacking performance, learning time, computational resource, and image quality. As the adversarial goals (e.g., image degradation or content shifting) are similar between different target modalities, perturbing multiple input modalities at the same time (e.g., both input image and text prompt) can potentially offer a higher performance. However, none of existing studies investigate attacking DMs concurrently from multiple modalities. This can be investigated in future work, where, for example, the perturbations are added to both the text prompt and input image to achieve a higher attacking performance. In addition, current studies mostly focus on image-generation adversarial attacks with textual guidance, while neglecting other modalities/ data types such as audio, time series, and 3D models. Attacking such modalities requires more sophisticated strategies to learn the adversarial perturbation, since it involves additional techniques for these specific applications (e.g., DM-based 3D generation often requires a specific neural radiance fields model for 3D synthesis). V-D Adversarial Defense In terms of countermeasures against adversarial attacks, most existing studies investigate concept erasing as a mean of defense. This method can solve NSFW generation by making the fine-tuned DMs forget about NSFW concepts, thus protecting users from harmful content even when the adversarial perturbation is fed into the DMs’ input. However, countermeasures for other adversarial threats such as quality diminishing, concept shifting, and style transferring have not been investigating in existing work. In such cases, the purpose of attack is more general and not focuses on any particular concept, making it especially challenging to remove the adversarial effect. Hence, this remains an open challenge and necessitate further effort on future work. V-E Membership Inference Attack Nowadays, pre-trained large-scale DMs are publicly available and easily accessible on the Internet. Fine-tuning these models for downstream tasks has become a very popular paradigm since it requires less computational resources compared to training a model from scratch. There are many new fine-tuning techniques adapted to DMs that are widely used, such as LoRA [176] and DreamBooth [120], which show the effectiveness of guiding the DM’s generation process. Many show that fine-tuning large-scale DMs on small datasets makes them easily overfit, hence increasing the vulnerability to MIAs. Having said that, no studies have yet investigated the data privacy implications of fine-tuned DMs. This remains an open challenge for the research community. In addition, while the shadow training technique [94] is commonly used effectively in MIAs, its application to large-scale DMs demands intensive computational resources. This requirement poses a significant challenge to training shadow models for efficient MIA methods. V-F Membership Inference Defense There are various adaptation techniques for DMs, including LoRA[176], DreamBooth[120], and Textual Inversion [121]. These techniques are often trained on small datasets as a supplementary module to guide the main models’ generation styles. Due to their effectiveness and accessibility, they are used and shared widely on the Internet, potentially including misused modules trained on private or non-copyrighted data, such as a module developed to mimic the art style of a famous artist. Unfortunately, their ease of overfitting makes them more vulnerable to MIAs. Consequently, the data privacy risks of applying these adaptations require proper countermeasure methods. Moreover, knowledge distillation is a well-established strategy widely used to mitigate data privacy risks for ML models against MIAs. Although numerous studies have successfully applied knowledge distillation to DMs, this approach has yet to be explored as a defense mechanism against such MIAs. VI Conclusion In this paper, we have provided a comprehensive survey of DMs regarding the security aspect. The paper begins with crucial background knowledge of different types of DMs including DDPMs, DDIMs, NCSNs, SDE, and multi-modal conditional models. In terms of attacks on DMs, they are categorized into three main groups, including backdoor attack, adversarial attack, and MIA. Each type of attack is surveyed comprehensively based on state-of-the-art attack methods presented in recent studies. Then, we emphasized the importance of DM’s safety by providing an extensive survey on countermeasures of these attacks. Finally, we outlined current open challenges in DM’s security and envisioned potential avenues for future research in this topic. Based on the review, we found out that most research works focus on only vision and language domains, while other modalities such as audio and time series were neglected in all three main types of attack. In addition, countermeasures for these DM-targeted attacks are still limited and needed further research to enhance the security aspect of DMs. References [1] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in Proc. Int. Conf. Mach. Learn., 2015, p. 2256–2265. [2] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 33, 2020, p. 6840–6851. [3] P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 8780–8794. [4] A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in Proc. Int. Conf. Mach. Learn., 2021, p. 8162–8171. [5] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in Proc. Int. Conf. Learn. Representations, 2021. [6] Y. Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 32, 2019, p. 11 918–11 930. [7] Y. Song, C. Durkan, I. Murray, and S. Ermon, “Maximum likelihood training of score-based diffusion models,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 12 438–12 448. [8] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” in Proc. Int. Conf. Learn. Representations, 2021. [9] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2022, p. 10 684–10 695. [10] Y. Song and S. Ermon, “Improved techniques for training score-based generative models,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2020, p. 12 438–12 448. [11] A. Vahdat, K. Kreis, and J. Kautz, “Score-based generative modeling in latent space,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 11 287–11 302. [12] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 27, 2014. [13] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in Proc. Int. Conf. Mach. Learn., 2014. [14] D. Rezende and S. Mohamed, “Variational inference with normalizing flows,” in Proc. Int. Conf. Mach. Learn., 2015, p. 1530–1538. [15] J. Ngiam, Z. Chen, P. W. Koh, and A. Y. Ng, “Learning deep energy models,” in Proc. Int. Conf. Mach. Learn., 2011, p. 1105–1112. [16] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Interv., 2015, p. 234–241. [17] D. Watson, W. Chan, J. Ho, and M. Norouzi, “Learning fast samplers for diffusion models by differentiating through sample quality,” in Proc. Int. Conf. Learn. Representations, 2021. [18] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” in Proc. Int. Conf. Mach. Learn., 2021, p. 16 784–16 804. [19] A. Sinha, J. Song, C. Meng, and S. Ermon, “D2c: Diffusion-decoding models for few-shot conditional generation,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 12 533–12 548. [20] F. Bao, C. Li, J. Zhu, and B. Zhang, “Analytic-dpm: an analytic estimate of the optimal reverse variance in diffusion probabilistic models,” in Proc. Int. Conf. Learn. Representations, 2022. [21] T. Dockhorn, A. Vahdat, and K. Kreis, “Score-based generative modeling with critically-damped langevin diffusion,” in Proc. Int. Conf. Learn. Representations, 2022. [22] O. Avrahami, D. Lischinski, and O. Fried, “Blended diffusion for text-driven editing of natural images,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2022, p. 18 208–18 218. [23] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv:2204.06125, 2022. [24] N. Liu, S. Li, Y. Du, A. Torralba, and J. B. Tenenbaum, “Compositional visual generation with composable diffusion models,” in Proc. Eur. Conf. Comput. Vis., 2022, p. 423–439. [25] M. Daniels, T. Maunu, and P. Hand, “Score-based generative neural networks for large-scale optimal transport,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 12 955–12 965. [26] H. Chung, B. Sim, and J. C. Ye, “Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2022, p. 12 413–12 422. [27] P. Esser, R. Rombach, A. Blattmann, and B. Ommer, “Imagebart: Bidirectional context with multinomial diffusion for autoregressive image synthesis,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 3518–3532. [28] A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool, “Repaint: Inpainting using denoising diffusion probabilistic models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2022, p. 11 461–11 471. [29] J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, “Structured denoising diffusion models in discrete state-spaces,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 17 981–17 993. [30] E. Hoogeboom, D. Nielsen, P. Jaini, P. Forré, and M. Welling, “Argmax flows and multinomial diffusion: Learning categorical distributions,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 12 454–12 465. [31] X. Li, J. Thickstun, I. Gulrajani, P. S. Liang, and T. B. Hashimoto, “Diffusion-lm improves controllable text generation,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 35, 2022, p. 4328–4343. [32] N. Savinov, J. Chung, M. Binkowski, E. Elsen, and A. v. d. Oord, “Step-unrolled denoising autoencoders for text generation,” in Proc. Int. Conf. Learn. Representations, 2021. [33] P. Yu, S. Xie, X. Ma, B. Jia, B. Pang, R. Gao, Y. Zhu, S.-C. Zhu, and Y. N. Wu, “Latent diffusion energy-based model for interpretable text modeling,” in Proc. Int. Conf. Mach. Learn., 2022, p. 25 702–25 720. [34] Z. Lin, Y. Gong, Y. Shen, T. Wu, Z. Fan, C. Lin, N. Duan, and W. Chen, “Text generation with diffusion language models: A pre-training approach with continuous paragraph denoise,” in Proc. Int. Conf. Mach. Learn., 2023, p. 21 051–21 064. [35] N. Chen, Y. Zhang, H. Zen, R. J. Weiss, M. Norouzi, and W. Chan, “Wavegrad: Estimating gradients for waveform generation,” in Proc. Int. Conf. Learn. Representations, 2020. [36] V. Popov, I. Vovk, V. Gogoryan, T. Sadekova, and M. Kudinov, “Grad-tts: A diffusion probabilistic model for text-to-speech,” in Proc. Int. Conf. Mach. Learn., 2021, p. 8599–8608. [37] J. Xu, X. Wang, W. Cheng, Y.-P. Cao, Y. Shan, X. Qie, and S. Gao, “Dream3d: Zero-shot text-to-3d synthesis using 3d shape prior and text-to-image diffusion models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 20 908–20 918. [38] V. T. Truong and L. B. Le, “Text-guided real-world-to-3d generative models with real-time rendering on mobile devices,” in Proc. IEEE Wireless Commun. Netw. Conf., 2024, p. 1–6. [39] B. Poole, A. Jain, J. T. Barron, and B. Mildenhall, “Dreamfusion: Text-to-3d using 2d diffusion,” arXiv:2209.14988, 2022. [40] C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M.-Y. Liu, and T.-Y. Lin, “Magic3d: High-resolution text-to-3d content creation,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 300–309. [41] A. Haque, M. Tancik, A. A. Efros, A. Holynski, and A. Kanazawa, “Instruct-nerf2nerf: Editing 3d scenes with instructions,” in Proc. Int. Conf. Comput. Vis., 2023, p. 19 740–19 750. [42] S. Luo and W. Hu, “Diffusion probabilistic models for 3d point cloud generation,” in Proc. Int. Conf. Comput. Vis., 2021, p. 2837–2845. [43] M. Xu, L. Yu, Y. Song, C. Shi, S. Ermon, and J. Tang, “Geodiff: A geometric diffusion model for molecular conformation generation,” in Proc. Int. Conf. Learn. Representations, 2021. [44] S. Luo, Y. Su, X. Peng, S. Wang, J. Peng, and J. Ma, “Antigen-specific antibody design and optimization with diffusion-based generative models for protein structures,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2022, p. 9754–9767. [45] Y. Tashiro, J. Song, Y. Song, and S. Ermon, “Csdi: Conditional score-based diffusion models for probabilistic time series imputation,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, 2021, p. 24 804–24 816. [46] T. Yan, H. Zhang, T. Zhou, Y. Zhan, and Y. Xia, “Scoregrad: Multivariate probabilistic time series forecasting with continuous energy-based generative models,” arXiv:2106.10121, 2021. [47] K. Rasul, A.-S. Sheikh, I. Schuster, U. Bergmann, and R. Vollgraf, “Multivariate probabilistic time series forecasting via conditioned normalizing flows,” in Proc. Int. Conf. Learn. Representations, 2020. [48] A. Hyvärinen and P. Dayan, “Estimation of non-normalized statistical models by score matching.” J. Mach. Learn. Res., vol. 6, no. 4, 2005. [49] P. Vincent, “A connection between score matching and denoising autoencoders,” Neural Comput., vol. 23, no. 7, p. 1661–1674, 2011. [50] B. D. Anderson, “Reverse-time diffusion equation models,” Stoch. Process. Appl., vol. 12, no. 3, p. 313–326, 1982. [51] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 30, 2017. [52] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A comprehensive survey of methods and applications,” ACM Comput. Surv., vol. 56, no. 4, p. 1–39, 2023. [53] H. Cao, C. Tan, Z. Gao, Y. Xu, G. Chen, P.-A. Heng, and S. Z. Li, “A survey on generative diffusion models,” IEEE Trans. Knowl. Data Eng., p. 1–20, 2024. [54] F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion models in vision: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 9, p. 10 850–10 869, 2023. [55] H. Zou, Z. M. Kim, and D. Kang, “Diffusion models in nlp: A survey,” arXiv:2305.14671, 2023. [56] A. Kazerouni, E. K. Aghdam, M. Heidari, R. Azad, M. Fayyaz, I. Hacihaliloglu, and D. Merhof, “Diffusion models in medical imaging: A comprehensive survey,” Med. Image Anal., p. 102846, 2023. [57] L. Lin, Z. Li, R. Li, X. Li, and J. Gao, “Diffusion models for time-series applications: a survey,” Front. Inf. Technol. Electron. Eng., p. 1–23, 2023. [58] J. Vice, N. Akhtar, R. Hartley, and A. Mian, “Bagm: A backdoor attack for manipulating text-to-image generative models,” IEEE Trans. Inf. Forensics Secur., p. 1–1, 2024. [59] S.-Y. Chou, P.-Y. Chen, and T.-Y. Ho, “How to backdoor diffusion models?” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., June 2023, p. 4015–4024. [60] Y. Huang, F. Juefei-Xu, Q. Guo, J. Zhang, Y. Wu, M. Hu, T. Li, G. Pu, and Y. Liu, “Personalization as a shortcut for few-shot backdoor attack against text-to-image diffusion models,” in Proc. AAAI Conf. Artif. Intell., vol. 38, no. 19, 2024, p. 21 169–21 178. [61] L. Struppek, D. Hintersdorf, and K. Kersting, “Rickrolling the artist: Injecting backdoors into text encoders for text-to-image synthesis,” in Proc. Int. Conf. Comput. Vis., 2023, p. 4584–4596. [62] W. Chen, D. Song, and B. Li, “Trojdiff: Trojan attacks on diffusion models with diverse targets,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 4035–4044. [63] S.-Y. Chou, P.-Y. Chen, and T.-Y. Ho, “Villandiffusion: A unified backdoor attack framework for diffusion models,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 36, 2024. [64] H. Wang, Q. Shen, Y. Tong, Y. Zhang, and K. Kawaguchi, “The stronger the diffusion model, the easier the backdoor: Data poisoning to induce copyright breaches without adjusting finetuning pipeline,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2023. [65] Z. Pan, Y. Yao, G. Liu, B. Shen, H. V. Zhao, R. R. Kompella, and S. Liu, “From trojan horses to castle walls: Unveiling bilateral backdoor effects in diffusion models,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2023. [66] S. Zhai, Y. Dong, Q. Shen, S. Pu, Y. Fang, and H. Su, “Text-to-image diffusion models can be easily backdoored through multimodal data poisoning,” in Proc. Proc ACM Int. Conf. Multimed., 2023, p. 1577–1587. [67] S. Li, J. Ma, and M. Cheng, “Learnable invisible backdoor for diffusion models,” 2023. [68] H. Salman, A. Khaddaj, G. Leclerc, A. Ilyas, and A. Madry, “Raising the cost of malicious ai-powered image editing,” in Proc. Int. Conf. Mach. Learn., 2023, p. 29 894–29 918. [69] T. Van Le, H. Phung, T. H. Nguyen, Q. Dao, N. N. Tran, and A. Tran, “Anti-dreambooth: Protecting users from personalized text-to-image synthesis,” in Proc. Int. Conf. Comput. Vis., 2023, p. 2116–2127. [70] H. Yu, J. Chen, X. Ding, Y. Zhang, T. Tang, and H. Ma, “Step vulnerability guided mean fluctuation adversarial attack against conditional diffusion models,” in Proc. AAAI Conf. Artif. Intell., vol. 38, no. 7, 2024, p. 6791–6799. [71] S. Shan, J. Cryan, E. Wenger, H. Zheng, R. Hanocka, and B. Y. Zhao, “Glaze: Protecting artists from style mimicry by \Text-to-Image\ models,” in Proc. USENIX Secur. Symp., 2023, p. 2187–2204. [72] J. Zhang, Z. Xu, S. Cui, C. Meng, W. Wu, and M. R. Lyu, “On the robustness of latent diffusion models,” arXiv:2306.08257, 2023. [73] Y. Yang, R. Gao, X. Wang, N. Xu, and Q. Xu, “Mma-diffusion: Multimodal attack on diffusion models,” arXiv:2311.17516, 2023. [74] Y. Yang, B. Hui, H. Yuan, N. Gong, and Y. Cao, “Sneakyprompt: Jailbreaking text-to-image generative models,” in Proc. Symp. Secur. Privacy, 2024, p. 123–123. [75] H. Zhuang, Y. Zhang, and S. Liu, “A pilot study of query-free adversarial attack against stable diffusion,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 2384–2391. [76] H. Gao, H. Zhang, Y. Dong, and Z. Deng, “Evaluating the robustness of text-to-image diffusion models against real-world attacks,” arXiv:2306.13103, 2023. [77] C. Zhang, L. Wang, and A. Liu, “Revealing vulnerabilities in stable diffusion via targeted attacks,” arXiv:2401.08725, 2024. [78] Q. Liu, A. Kortylewski, Y. Bai, S. Bai, and A. Yuille, “Discovering failure modes of text-guided diffusion models via adversarial search,” in Proc. Int. Conf. Learn. Representations, 2023. [79] H. Liu, Y. Wu, S. Zhai, B. Yuan, and N. Zhang, “Riatig: Reliable and imperceptible adversarial text-to-image generation with natural prompts,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 20 585–20 594. [80] Z. Kou, S. Pei, Y. Tian, and X. Zhang, “Character as pixels: A controllable prompt adversarial attacking framework for black-box text guided image generation models,” in Proc. Int. Joint Conf. Artif. Intell., 2023, p. 983–990. [81] Y. Zhang, J. Jia, X. Chen, A. Chen, Y. Zhang, J. Liu, K. Ding, and S. Liu, “To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images… for now,” arXiv:2310.11868, 2023. [82] C. Liang, X. Wu, Y. Hua, J. Zhang, Y. Xue, T. Song, Z. Xue, R. Ma, and H. Guan, “Adversarial example does good: preventing painting imitation from diffusion models via adversarial examples,” in Proc. Int. Conf. Mach. Learn., 2023, p. 20 763–20 786. [83] C. Liang and X. Wu, “Mist: Towards improved adversarial examples for diffusion models,” arXiv:2305.12683, 2023. [84] P. Zhu, T. Takahashi, and H. Kataoka, “Watermark-embedded adversarial examples for copyright protection against diffusion models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2024, p. 24 420–24 430. [85] C. Li, R. Pang, B. Cao, J. Chen, and T. Wang, “A change of heart: Backdoor attacks on security-centric diffusion models,” 2023. [86] M. Kang, D. Song, and B. Li, “Diffattack: Evasion attacks against diffusion-based adversarial purification,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 36, 2024. [87] C. Luo, “Understanding diffusion models: A unified perspective,” arXiv:2208.11970, 2022. [88] Y. Song, S. Garg, J. Shi, and S. Ermon, “Sliced score matching: A scalable approach to density and score estimation,” in Proc. Uncertainty in Artif. Intell., 2020, p. 574–584. [89] T. Pang, K. Xu, C. Li, Y. Song, S. Ermon, and J. Zhu, “Efficient learning of generative models via finite-difference score matching,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 33, 2020, p. 19 175–19 188. [90] Y. Li, Y. Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” IEEE Trans. Neural Netw. Learn. Syst., 2022. [91] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” arXiv:1706.06083, 2017. [92] S. Huang, N. Papernot, I. Goodfellow, Y. Duan, and P. Abbeel, “Adversarial attacks on neural network policies,” arXiv:1702.02284, 2017. [93] N. Akhtar and A. Mian, “Threat of adversarial attacks on deep learning in computer vision: A survey,” IEEE Access, vol. 6, p. 14 410–14 430, 2018. [94] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in Proc. IEEE Symp. Secur. Privacy, 2017, p. 3–18. [95] A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes, “Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models,” arXiv:1806.01246, 2018. [96] S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in Proc. IEEE Comput. Secur. Found. Symp., 2018, p. 268–282. [97] J. Li, N. Li, and B. Ribeiro, “Membership inference attacks and defenses in classification models,” in Proc. ACM Conf. Data Appl. Secur. Privacy, 2021, p. 5–16. [98] J. Ye, A. Maddi, S. K. Murakonda, V. Bindschaedler, and R. Shokri, “Enhanced membership inference attacks against machine learning models,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2022, p. 3093–3106. [99] Y. Liu, Z. Zhao, M. Backes, and Y. Zhang, “Membership inference attacks by exploiting loss trajectory,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2022, p. 2085–2098. [100] N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in Proc. IEEE Symp. Secur. Privacy, 2022, p. 1897–1914. [101] J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro, “Logan: Membership inference attacks against generative models,” arXiv:1705.07663, 2017. [102] B. Hilprecht, M. Härterich, and D. Bernau, “Monte carlo and reconstruction membership inference attacks against generative models,” Proc. Priv. Enhancing Technol., 2019. [103] D. Chen, N. Yu, Y. Zhang, and M. Fritz, “Gan-leaks: A taxonomy of membership inference attacks against generative models,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2020, p. 343–362. [104] H. Hu and J. Pang, “Membership inference attacks against gans by leveraging over-representation regions,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2021, p. 2387–2389. [105] S. Mukherjee, Y. Xu, A. Trivedi, N. Patowary, and J. L. Ferres, “privgan: Protecting gans from membership inference attacks at low cost to utility,” Proceedings on Privacy Enhancing Technologies, 2021. [106] Y. Wu, N. Yu, Z. Li, M. Backes, and Y. Zhang, “Membership inference attacks against text-to-image generation models,” 2022. [107] M. Zhang, N. Yu, R. Wen, M. Backes, and Y. Zhang, “Generated distributions are all you need for membership inference attacks against generative models,” in Proc. Winter Conf. Appl. Comput. Vis., 2024, p. 4839–4849. [108] Y. Pang and T. Wang, “Black-box membership inference attacks against fine-tuned diffusion models,” arXiv:2312.08207, 2023. [109] J. Li, J. Dong, T. He, and J. Zhang, “Towards black-box membership inference attack for diffusion models,” arXiv:2405.20771, 2024. [110] J. Duan, F. Kong, S. Wang, X. Shi, and K. Xu, “Are diffusion models vulnerable to membership inference attacks?” in Proc. Int. Conf. Mach. Learn., 2023, p. 8717–8730. [111] S. Tang, Z. S. Wu, S. Aydore, M. Kearns, and A. Roth, “Membership inference attacks on diffusion models via quantile regression,” arXiv:2312.05140, 2023. [112] F. Kong, J. Duan, R. Ma, H. Shen, X. Zhu, X. Shi, and K. Xu, “An efficient membership inference attack for the diffusion model by proximal initialization,” arXiv:2305.18355, 2023. [113] S. Zhai, H. Chen, Y. Dong, J. Li, Q. Shen, Y. Gao, H. Su, and Y. Liu, “Membership inference on text-to-image diffusion models via conditional likelihood discrepancy,” arXiv:2405.14800, 2024. [114] H. Hu and J. Pang, “Membership inference of diffusion models,” arXiv:2301.09956, 2023. [115] N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramer, B. Balle, D. Ippolito, and E. Wallace, “Extracting training data from diffusion models,” in Proc. USENIX Secur. Symp., 2023, p. 5253–5270. [116] T. Matsumoto, T. Miura, and N. Yanai, “Membership inference attacks against diffusion models,” in Proc. IEEE Secur. Privacy Workshops, 2023, p. 77–83. [117] J. Dubiński, A. Kowalczuk, S. Pawlak, P. Rokita, T. Trzciński, and P. Morawiecki, “Towards more realistic membership inference attacks on large diffusion models,” in Proc. Winter Conf. Appl. Comput. Vis., 2024, p. 4860–4869. [118] Y. Pang, T. Wang, X. Kang, M. Huai, and Y. Zhang, “White-box membership inference attacks against diffusion models,” arXiv:2308.06405, 2023. [119] T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,” arXiv:1708.06733, 2017. [120] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 22 500–22 510. [121] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-or, “An image is worth one word: Personalizing text-to-image generation using textual inversion,” in Proc. Int. Conf. Learn. Representations, 2022. [122] Y. Qu, X. Shen, X. He, M. Backes, S. Zannettou, and Y. Zhang, “Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,” in Proc. ACM Conf. Comput. Commun. Secur., 2023, p. 3403–3417. [123] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2019. [124] L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in Proc. IEEE Symp. Secur. Privacy, 2021, p. 141–159. [125] Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in Proc. IEEE Symp. Secur. Privacy, 2015, p. 463–480. [126] A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” Proc. Int. Conf. Neural Inf. Process. Syst., vol. 34, p. 18 075–18 086, 2021. [127] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv:1511.06434, 2015. [128] J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in Proc. Int. Conf. Mach. Learn., 2022, p. 12 888–12 900. [129] C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki, “Laion-400m: Open dataset of clip-filtered 400 million image-text pairs,” arXiv:2111.02114, 2021. [130] Y. Zheng, H. Yang, T. Zhang, J. Bao, D. Chen, Y. Huang, L. Yuan, D. Chen, M. Zeng, and F. Wen, “General facial representation learning in a visual-linguistic manner,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2022, p. 18 697–18 709. [131] X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollár, and C. L. Zitnick, “Microsoft coco captions: Data collection and evaluation server,” arXiv:1504.00325, 2015. [132] R. Krishna, Y. Zhu, O. Groth, J. Johnson, K. Hata, J. Kravitz, S. Chen, Y. Kalantidis, L.-J. Li, D. A. Shamma et al., “Visual genome: Connecting language and vision using crowdsourced dense image annotations,” Int. J. Comput. Vis., vol. 123, p. 32–73, 2017. [133] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2016, p. 770–778. [134] Z. Kong and W. Ping, “On fast sampling of diffusion probabilistic models,” arXiv:2106.00132, 2021. [135] A. Krizhevsky, G. Hinton et al., “Learning multiple layers of features from tiny images,” 2009. [136] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proc. Int. Conf. Comput. Vis., 2015, p. 3730–3738. [137] P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high-resolution image synthesis,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2021, p. 12 873–12 883. [138] Y. Chen, S. Liu, and X. Wang, “Learning continuous image representation with local implicit image function,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2021, p. 8628–8638. [139] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2009, p. 248–255. [140] T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of gans for improved quality, stability, and variation,” arXiv:1710.10196, 2017. [141] J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” in Proc. Int. Conf. Mach. Learn., 2023, p. 19 730–19 742. [142] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. Eur. Conf. Comput. Vis., 2020, p. 213–229. [143] H. Bao, L. Dong, S. Piao, and F. Wei, “Beit: Bert pre-training of image transformers,” arXiv:2106.08254, 2021. [144] H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou, “Training data-efficient image transformers & distillation through attention,” in Proc. Int. Conf. Mach. Learn., 2021, p. 10 347–10 357. [145] T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proc. Int. Conf. Knowl. Discov. Data Min., 2016, p. 785–794. [146] Y. Li, X. Lyu, N. Koren, L. Lyu, B. Li, and X. Ma, “Neural attention distillation: Erasing backdoor triggers from deep neural networks,” in Proc. Int. Conf. Learn. Representations, 2021. [147] K. Liu, B. Dolan-Gavitt, and S. Garg, “Fine-pruning: Defending against backdooring attacks on deep neural networks,” in Proc. Int. Symp. Res. Attacks Intrusions Defenses, 2018, p. 273–294. [148] S. An, S.-Y. Chou, K. Zhang, Q. Xu, G. Tao, G. Shen, S. Cheng, S. Ma, P.-Y. Chen, T.-Y. Ho et al., “Elijah: Eliminating backdoors injected in diffusion models via distribution shift,” in Proc. AAAI Conf. Artif. Intell., vol. 38, no. 10, 2024, p. 10 847–10 855. [149] —, “How to remove backdoors in diffusion models?” in Proc. Int. Conf. Neural Inf. Process. Syst., 2023. [150] Z. Guan, M. Hu, S. Li, and A. Vullikanti, “Ufid: A unified framework for input-level backdoor detection on diffusion models,” arXiv:2404.01101, 2024. [151] Y. Sui, H. Phan, J. Xiao, T. Zhang, Z. Tang, C. Shi, Y. Wang, Y. Chen, and B. Yuan, “Disdet: Exploring detectability of backdoor attack on diffusion models,” arXiv:2402.02739, 2024. [152] R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau, “Erasing concepts from diffusion models,” in Proc. Int. Conf. Comput. Vis., 2023, p. 2426–2436. [153] N. Kumari, B. Zhang, S.-Y. Wang, E. Shechtman, R. Zhang, and J.-Y. Zhu, “Ablating concepts in text-to-image diffusion models,” in Proc. Int. Conf. Comput. Vis., 2023, p. 22 691–22 702. [154] G. Zhang, K. Wang, X. Xu, Z. Wang, and H. Shi, “Forget-me-not: Learning to forget in text-to-image diffusion models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2024, p. 1755–1764. [155] S. Hong, J. Lee, and S. S. Woo, “All but one: Surgical concept erasing with model preservation in text-to-image diffusion models,” in Proc. AAAI Conf. Artif. Intell., vol. 38, no. 19, 2024, p. 21 143–21 151. [156] P. Schramowski, M. Brack, B. Deiseroth, and K. Kersting, “Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2023, p. 22 522–22 531. [157] J. Wu, T. Le, M. Hayat, and M. Harandi, “Erasediff: Erasing data influence in diffusion models,” arXiv:2401.05779, 2024. [158] M. Fuchi and T. Takagi, “Erasing concepts from text-to-image diffusion models with few-shot unlearning,” arXiv:2405.07288, 2024. [159] Z. Ni, L. Wei, J. Li, S. Tang, Y. Zhuang, and Q. Tian, “Degeneration-tuning: Using scrambled grid shield unwanted concepts from stable diffusion,” in Proc. ACM Int. Conf. Multimed., 2023, p. 8900–8909. [160] Y. Yang, R. Gao, X. Yang, J. Zhong, and Q. Xu, “Guardt2i: Defending text-to-image models from adversarial prompts,” arXiv:2403.01446, 2024. [161] T. Yang, J. Cao, and C. Xu, “Pruning for robust concept erasing in diffusion models,” arXiv:2405.16534, 2024. [162] G. Li, H. Hsu, R. Marculescu et al., “Machine unlearning for image-to-image generative models,” arXiv:2402.00351, 2024. [163] A. Heng and H. Soh, “Selective amnesia: A continual learning approach to forgetting in deep generative models,” in Proc. Int. Conf. Neural Inf. Process. Syst., vol. 36, 2024. [164] J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tramèr, “Red-teaming the stable diffusion safety filter,” in Proc. Int. Conf. Neural Inf. Process. Syst., 2022. [165] M. Lyu, Y. Yang, H. Hong, H. Chen, X. Jin, Y. He, H. Xue, J. Han, and G. Ding, “One-dimensional adapter to rule them all: Concepts diffusion models and erasing applications,” in Proc. Int. Conf. Comput. Vis. Pattern Recognit., 2024, p. 7559–7568. [166] C.-P. Huang, K.-P. Chang, C.-T. Tsai, Y.-H. Lai, and Y.-C. F. Wang, “Receler: Reliable concept erasing of text-to-image diffusion models via lightweight erasers,” arXiv:2311.17717, 2023. [167] Y. Wu, S. Zhou, M. Yang, L. Wang, W. Zhu, H. Chang, X. Zhou, and X. Yang, “Unlearning concepts in diffusion model via concept domain correction and concept preserving gradient,” arXiv:2405.15304, 2024. [168] C. Kim, K. Min, and Y. Yang, “Race: Robust adversarial concept erasure for secure text-to-image diffusion model,” arXiv:2405.16341, 2024. [169] Z.-Y. Chin, C.-M. Jiang, C.-C. Huang, P.-Y. Chen, and W.-C. Chiu, “Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts,” arXiv:2309.06135, 2023. [170] T. DeVries and G. W. Taylor, “Improved regularization of convolutional neural networks with cutout,” arXiv:1708.04552, 2017. [171] E. D. Cubuk, B. Zoph, J. Shlens, and Q. V. Le, “Randaugment: Practical automated data augmentation with a reduced search space,” in Proc. Conf. Comput. Vis. Pattern Recognit. Workshops, 2020, p. 702–703. [172] C. Dwork, F. McSherry, K. Nissim, and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Proc. Theory Cryptogr. Conf., 2006, p. 265–284. [173] C. Dwork, “Differential privacy: A survey of results,” in Proc. Int. Conf. Theory Appl. Models Comput., 2008, p. 1–19. [174] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur., 2016, p. 308–318. [175] V. Fernandez, P. Sanchez, W. H. L. Pinaya, G. Jacenków, S. A. Tsaftaris, and J. Cardoso, “Privacy distillation: reducing re-identification risk of multimodal diffusion models,” arXiv:2306.01322, 2023. [176] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021.

Instant research discovery