Paper deep dive

Data Redaction from Conditional Generative Models

Zhifeng Kong, Kamalika Chaudhuri

Year: 2023Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 131

Models: CycleGAN-VC2, DiffWave, DM-GAN

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 6:16:28 PM

Summary

The paper introduces a method for 'data redaction' in pre-trained conditional generative models. Instead of retraining models from scratch or using external filters, the authors propose post-editing the model weights by distilling the conditioning network. This approach effectively redacts undesirable conditionals (e.g., harmful prompts or specific voices) while maintaining high generation quality for benign inputs, offering a computationally efficient and robust solution for text-to-image and text-to-speech models.

Entities (6)

Kamalika Chaudhuri · researcher · 100%Zhifeng Kong · researcher · 100%Data Redaction · methodology · 98%Conditional Generative Models · technology · 95%Text-to-Image · application · 95%Text-to-Speech · application · 95%

Relation Signals (4)

Zhifeng Kong → authored → Data Redaction from Conditional Generative Models

confidence 100% · 1st Zhifeng Kong... Data Redaction from Conditional Generative Models

Data Redaction → appliedto → Conditional Generative Models

confidence 95% · In this paper, we take a different approach and study how to post-edit an already-trained conditional generative model so that it redacts certain conditionals

Data Redaction → usedin → Text-to-Image

confidence 90% · We conduct experiments on redacting prompts in text-to-image models

Data Redaction → usedin → Text-to-Speech

confidence 90% · redacting voices in text-to-speech models

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Deep generative models are known to produce undesirable samples such as harmful content. Traditional mitigation methods include re-training from scratch, filtering, or editing; however, these are either computationally expensive or can be circumvented by third parties. In this paper, we take a different approach and study how to post-edit an already-trained conditional generative model so that it redacts certain conditionals that will, with high probability, lead to undesirable content. This is done by distilling the conditioning network in the models, giving a solution that is effective, efficient, controllable, and universal for a class of deep generative models. We conduct experiments on redacting prompts in text-to-image models and redacting voices in text-to-speech models. Our method is computationally light, leads to better redaction quality and robustness than baseline methods while still retaining high generation quality.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

130,679 characters extracted from source content.

Expand or collapse full text

Data Redaction from Conditional Generative Models 1st Zhifeng Kong Computer Science and Engineering University of California San Diego La Jolla, USA z4kong@ucsd.edu 2nd Kamalika Chaudhuri Computer Science and Engineering University of California San Diego La Jolla, USA kamalika@cs.ucsd.edu Abstract Deep generative models are known to produce undesirable samples such as harmful content. Traditional mitigation methods include re-training from scratch, filtering, or editing; however, these are either computationally expensive or can be circumvented by third parties. In this paper, we take a different approach and study how to post-edit an already-trained conditional generative model so that it redacts certain conditionals that will, with high probability, lead to undesirable content. This is done by distilling the conditioning network in the models, giving a solution that is effective, efficient, controllable, and universal for a class of deep generative models. We conduct experiments on redacting prompts in text-to-image models and redacting voices in text-to-speech models. Our method is computationally light, leads to better redaction quality and robustness than baseline methods while still retaining high generation quality. I Introduction Deep generative models are unsupervised deep learning models that learn a data distribution from samples and then generate new samples from it. These models have shown tremendous success in many domains such as image generation (rombach2021highresolution, ; ramesh2021zero, ; ramesh2022hierarchical, ; sauer2023stylegan, ), audio synthesis (kong2021diffwave, ; lee2023bigvgan, ), and text generation (gpt4, ; touvron2023llama, ). Most modern deep generative models are conditional: the user inputs some context known as the conditionals, and the model generates samples conditioned on the context. However, as these models have grown more powerful, there has been increasing concern about their trustworthiness: in certain situations, these models produce undesirable outputs. For example, with text-to-image models, one may craft a prompt that contains offensive, biased, malignant, or fabricated content, and generate a high-resolution image that visualizes the prompt (nichol2021glide, ; birhane2021multimodal, ; schuhmann2022laion, ; ramesh2022hierarchical, ; rando2022red, ; nudenet, ; man, ). With speech synthesis models, one may easily turn text into celebrity voices (Betker2022TTS, ; wang2023neural, ; zhang2023speak, ). Text generation models can emit offensive, biased, or toxic content (pitsilis2018detecting, ; wallace2019universal, ; mcguffie2020radicalization, ; gehman2020realtoxicityprompts, ; abid2021persistent, ; perez2022red, ; schramowski2022large, ). One plausible solution to mitigate this problem is to remove all undesirable samples from the training set and re-train the model. This is too computationally heavy for modern, large models. Another solution is to apply a classifier that filters out undesirable conditionals or outputs (rando2022red, ; nudenet, ; man, ), or to edit the outputs and remove the undesirable content after generation (schramowski2022safe, ). However, in cases where the model owners share the model weights with third parties, they do not have control over whether the filters or editing methods will be used. In order to prevent undesirable outputs more efficiently and reliably, we propose to post-edit the weights of a pre-trained model, which we call data redaction. The first challenge is how to frame data redaction for conditional generative models. Prior work in data redaction for unconditional generative models considered this problem in the space of outputs, and framed the problem as learning the data distribution restricted to a valid subset of outputs (kong2023data, ). However, a conditional generative model learns a collection of (usually an infinite number of) distributions (one for each conditional) all of which are induced by networks that share weights; therefore, we cannot apply this method one by one for every conditional we would like to redact. In this paper, we frame data redaction for conditional generative models as redacting a set of conditionals that will very likely lead to undesirable content. In particular, we do redaction in the conditional space, instead of separately redacting samples generated from each conditional in the output space. This statistical machine learning framework inspires us to design a universal, efficient, and effective method for data redaction. We only re-train (or distill) the conditional part of the network by projecting redacted conditionals onto different, non-redacted reference conditionals. It is computationally light because all but the conditioning network is fixed, and we only need to load a small fraction of the dataset for training. We show there exists an explicit data redaction formula for simple class-conditional models. For more complicated generative models in real-world applications, we introduce a series of techniques to effectively redact certain conditionals but retain high generation quality. These include model-specific distillation losses and training schemes, methods to increase the capacity of the student conditioning network, ways to improve efficiency, and a few others. We test our data redaction method on two real-world applications: GAN-based text-to-image (zhu2019dm, ) and Diffusion-based text-to-speech (kong2021diffwave, ). For text-to-image, we redact prompts that include certain words or phrases. Our method has significantly better redaction quality and robustness than baseline methods while retaining similar generation quality as the pre-trained model. For text-to-speech, we redact certain voices outside the training set. Our method achieves both high redaction and speech quality. Audio samples can be found on our demo website (https://dataredact2023.github.io/). Our methods for both applications are extremely computationally efficient: redacting text-to-image models takes approximately 0.5 hour, and redacting text-to-speech models takes less than 4 hours, both on one single NVIDIA 3080 GPU. In contrast, training the text-to-image model takes more than a day on one GPU, and training the text-to-speech model takes 2-3 days on 8 GPUs. This demonstrates that data redaction can be done significantly more efficiently than re-training full models from scratch. (a) Pre-trained (b) Reference (c) Our Redaction (d) Rewriting (bau2020rewriting, ) Figure 1: Redact ‘white belly’ from text-to-image models (zhu2019dm, ). The prompt is ‘this bird has feathers that are black and has a white belly’. (a) Sample generated from the pre-trained model, which produces a visualization of the prompt. (b) The target sample that redacts ‘white belly’ but keeps the other concepts. (c) Generated sample from our redaction model, which aims to redact ‘white belly’ and approximates the reference sample. (d) Sample generated from the Rewriting baseline, which is blurry and has lower quality. More samples can be found in Appendix B-B. I-A Related Work Machine Unlearning. Machine unlearning computes or approximates a re-trained machine learning model after removing certain training samples (cao2015towards, ). Many unlearning methods have been proposed for supervised learning (guo2019certified, ; schelter2020amnesia, ; neel2021descent, ; sekhari2021remember, ; izzo2021approximate, ; ullah2021machine, ; bourtoule2021machine, ; warnecke2021machine, ), among which some provide theoretically guaranteed unlearning or removal for strictly convex classifiers. There is one method approximate deletion method for generative models (kong2022approximate, ), which aims to delete from an unconditional generative model by post-hoc rejection sampling. The goal of data redaction is very different from machine unlearning, which unlearns training samples and is usually in the privacy context, while data redaction prevents undesirable samples from generation regardless whether they are in the training set. 111It is also unclear how to do efficient unlearning for complex conditional generative models (e.g. text-to-X) because it is unclear what exact combination of (text, X) pairs to unlearn and how to do it beyond retraining from scratch. A detailed explanation can be found in Section I-C in (kong2023data, ). Data Filtering and Semantic Editing. A direct way to prevent certain samples to be generated is to apply a data filter (e.g., a malicious content classifier). The filter can be applied to training data before training (nichol2021glide, ; schuhmann2022laion, ; ramesh2022hierarchical, ), or applied post-hoc to model outputs (rando2022red, ; nudenet, ; man, ). Another line of research has looked at semantically modifying the outputs of generative models. For GANs (goodfellow2014generative, ), (bau2020semantic, ) computes an editing vector in the latent space to alter a semantic concept. For diffusion models (ho2020denoising, ) especially text-to-image models like Stable Diffusion (rombach2021highresolution, ), there are also a number of image editing techniques (bar2022text2live, ; hertz2022prompt, ; kawar2022imagic, ; valevski2022unitune, ; brack2022stable, ). (schramowski2022safe, ) applied image editing to prevent diffusion models from generating malicious images through a safety guidance term that alters the sampling algorithm for inappropriate prompts. While these filtering and editing methods can be used to prevent malicious images, the model parameters are not modified. Consequently, in cases where the models owners share the model weights with third parties, they do not have control over whether the third parties will use the filters or editing methods. In contrast, our proposed method modifies the model weights to address this issue. Data Redaction in Unconditional Models. Several works have studied methods to prevent generative models from producing undesirable samples, either by re-training or post-editing. For GANs, (asokan2020teaching, ) and (sinha2021negative, ) investigated re-training methods via modified loss functions that penalize generation of undesirable samples, and (bau2020rewriting, ) and (cherepkov2021navigating, ) introduced post-hoc parameter rewriting techniques for semantic editing, which can be used to remove undesirable artifacts. (kong2023data, ), (malnick2022taming, ), and (moon2023feature, ) designed post-editing data redaction methods for various types of pre-trained generative models. All these methods are restricted to the unconditional setting as they modify the mapping from latent vectors to samples. In contrast, the goal of this paper is to redact data from pre-trained, conditional generative models. In these models, the conditional information heavily controls the content and style of generated samples (e.g. text-to-X), whereas the latent controls variation. It is therefore necessary to also modify the mapping from conditional to samples. Redaction Methods for Stable Diffusion. (gandikota2023erasing, ) fine-tunes Stable Diffusion to incorporate negative guidance on undesirable visual styles (e.g., those under copyright protection). As a result, undesirable samples will not be generated with the standard sampling algorithm. However, one might recover the original score from the distilled score to break this method (see Appendix A for details). (gandikota2023unified, ) proposed an analytic solution for keys in cross attention blocks to edit concepts. A similar approach (zhang2023forget, ) proposed a re-steering mechanism for keys by minimizing the attention maps of target concepts. (heng2023selective, ) and (kumari2023ablating, ) proposed to forget or manipulate concepts by further fine-tuning the entire network with certain continual learning objectives. These methods are heavily designed for text-to-image tasks with Stable Diffusion. They require the model to be trained with either classifier-free guidance (ho2022classifier, ) or cross attention blocks, or they need to fine-tune the entire large diffusion network. In contrast, our proposed method is universal, applies to a broader range of generative models, and applies to multiple data domains. To our knowledge, it is the first method that is able to redact voices from a trained speech synthesis model. I Preliminaries Conditional Generative Models. Let CC be the space of conditionals. It could be a finite set of discrete labels, or an infinite set of continuous representations. 222In cases where there are infinitely many discrete labels such as text or 16-bit floats, these conditionals are usually considered as continuous or transformed to continuous representations. For any c∈c ∈ C there is an underlying data distribution pdata(⋅|c)p_data(·|c)proman_data ( ⋅ | c ) (on ℝdsuperscriptℝR^dblackboard_Rd) conditioned on c. In the discrete label case, this simply corresponds to a finite number of data distributions for all labels. In the more complicated continuous case, there is usually an underlying assumption that pdata(⋅|c)p_data(·|c)proman_data ( ⋅ | c ) is Lipschitz with respect to c: that is, pdata(⋅|c)p_data(·|c)proman_data ( ⋅ | c ) will not change much if c does not change much. Let X=(xi,ci)subscriptsubscriptX=\(x_i,c_i)\X = ( xitalic_i , citalic_i ) be the set of training data, in which each xisubscriptx_ixitalic_i is the sample and cisubscriptc_icitalic_i is the conditional (for example, xisubscriptx_ixitalic_i is an image and cisubscriptc_icitalic_i is its caption). Let G be a conditional generative model trained on X. G has two inputs – a sample latent z drawn from a Gaussian distribution and a conditional c – and outputs sample x=G⁢(z|c)conditionalx=G(z|c)x = G ( z | c ). For each c∈c ∈ C, G draws from a generative distribution pG(⋅|c)p_G(·|c)pitalic_G ( ⋅ | c ), which is trained to learn pdata(⋅|c)p_data(·|c)proman_data ( ⋅ | c ). In the discrete label case, this is equivalent to modeling a finite number of distributions. In the continuous case, G also needs to generalize to unseen conditionals, because not all conditionals exist in the training set. We assume that pG(⋅|c)p_G(·|c)pitalic_G ( ⋅ | c ) learns pdata(⋅|c)p_data(·|c)proman_data ( ⋅ | c ) very well, as how to train these models is outside the scope of this paper. Problem setup. Our goal is to redact a set of conditionals Ω⊂subscriptΩC_ _Ω ⊂ C, referred to as the redaction conditionals, which with high probability lead to undesirable content. For example, for text-to-image models, we may be looking to redact text prompts related to violence or offensive content. 333This does not necessarily redact every possible offensive output; for example, an innocent prompt such as ”a day in the park” might with very low probability result in a violent image which our solution will not address. We assume that the redaction conditionals are given to us either as a set or described by a classifier. We assume that we are working with an already trained generative model G and we are only allowed to post-edit it. Re-training generative models from scratch can be highly compute-intensive, and so our goal is to consider computationally efficient solutions. Additionally, we also want to avoid solutions that involve external filters, since a third-party can choose not to use them. A final requirement of our solution is that it should retain high generation quality for the conditionals that are not to be redacted. We assume that we have access to the parameters of the network G and (part or whole of) its training dataset X. 444One setting this assumption holds is when the model owners want to make their model safer. We believe the only possible solution for a closed-source model is filtering. The goal of this paper is to edit the parameters of model G to form a new model G′ so that harmful conditionals lead to the generation of benign outputs. Our proposed solution addresses this problem in the context where the conditioning networks are separate from the main generative network – which holds for most current network architectures – and achieves this by distilling only the conditioning networks. I Method In this section, we consider a special solution to our redaction task: for redacted conditionals c∈c ∈ C, we let G′ learn the distribution conditioned on a different, non-redacted conditional c^∈∖Ω^subscriptΩ c _ over start_ARG c end_ARG ∈ C ∖ Croman_Ω, which we denote as the reference conditional for c. Formally, pG′(⋅|c)=pG(⋅|c^) if c∈Ω, otherwise pG(⋅|c).p_G (·|c)=p_G(·| c) if c _ % , otherwise p_G(·|c).pitalic_G′ ( ⋅ | c ) = pitalic_G ( ⋅ | over start_ARG c end_ARG ) if c ∈ Croman_Ω , otherwise pitalic_G ( ⋅ | c ) . (1) Next, we introduce an efficient way to achieve (1). Let H be the (separate) conditioning network in the generator network G. H takes the conditional c as input and computes conditional representation H⁢(c)H(c)H ( c ), which is then fused into the main generative network (potentially at different layers). Our solution is to project the conditional representation H′⁢(c)superscript′H (c)H′ ( c ) of the new conditioning network to H⁢(c^)^H( c)H ( over start_ARG c end_ARG ): H′⁢(c)=H⁢(c^)⁢ if ⁢c∈Ω⁢, otherwise ⁢H⁢(c).superscript′ if subscriptΩ, otherwise H (c)=H( c) if c _ , otherwise % H(c).H′ ( c ) = H ( over start_ARG c end_ARG ) if c ∈ Croman_Ω , otherwise H ( c ) . (2) We provide an illustrative and analytical explanation to our method in Section IV. Specifically, we show (2) can be done explicitly if the model is conditioned on a few discrete labels and the conditioning network is affine. We then introduce methods for more complicated, real-world scenarios in Section V. For these models conditioned on continuous representations and with complicated architecture, we introduce distillation-based methods to approximately achieve (2). IV Redacting Models Conditioned on Discrete Labels In this section, we show for simple class-conditional models, there is an explicit formula to redact certain labels. Redacting a single label. Suppose there are k labels: =c1,⋯,cksubscript1⋯subscriptC=\c_1,·s,c_k\C = c1 , ⋯ , citalic_k , where label j is to be redacted. We consider a common type of conditioning method: each label cisubscriptc_icitalic_i is represented by a k-dimensional embedding vector vi∈ℝksubscriptsuperscriptℝv_i ^kvitalic_i ∈ blackboard_Rk, and H is an affine transformation whose output dimension r≥kr≥ kr ≥ k. We assume the embedding vectors are linearly independent: ⁢v1,⋯,vk=ℝksubscript1⋯subscriptsuperscriptℝspan\v_1,·s,v_k\=R^kspan v1 , ⋯ , vitalic_k = blackboard_Rk. A special case of this formulation is the conditioning method proposed by (mirza2014conditional, ), where each vi=isubscriptsubscriptv_i=e_ivitalic_i = eitalic_i is the one-hot vector with the i-th element =1absent1=1= 1, and is concatenated to the latent code. Let H⁢(v)=M⁢vH(v)=MvH ( v ) = M v, where M∈ℝr×ksuperscriptℝM ^r× kM ∈ blackboard_Rr × k. The redaction problem is equivalent finding an M′∈ℝr×ksuperscript′ℝM ^r× kM′ ∈ blackboard_Rr × k such that M′⁢vi=M⁢visuperscript′subscriptsubscriptM v_i=Mv_iM′ vitalic_i = M vitalic_i for i≠ji≠ ji ≠ j and M′⁢vj=M⁢V−j⁢η−jsuperscript′subscriptsubscriptsubscriptM v_j=MV_-j _-jM′ vitalic_j = M V- j η- j for an one-hot vector η−j∈ℝk−1subscriptsuperscriptℝ1 _-j ^k-1η- j ∈ blackboard_Rk - 1, where V−j=[v1,⋯,vj−1,vj+1,⋯,vk]∈ℝk×(k−1)subscriptsubscript1⋯subscript1subscript1⋯subscriptsuperscriptℝ1V_-j=[v_1,·s,v_j-1,v_j+1,·s,v_k] ^k×(k-1)V- j = [ v1 , ⋯ , vitalic_j - 1 , vitalic_j + 1 , ⋯ , vitalic_k ] ∈ blackboard_Rk × ( k - 1 ). The first condition M′⁢vi=M⁢visuperscript′subscriptsubscriptM v_i=Mv_iM′ vitalic_i = M vitalic_i for i≠ji≠ ji ≠ j indicates every row of M′−Msuperscript′M -M′ - M is in the null space of vii≠jsubscriptsubscript\v_i\_i≠ j vitalic_i i ≠ j. The null space is a one-dimensional subspace with basis vector u. Then, M′−Msuperscript′M -M′ - M can be decomposed as ω⁢u⊤superscripttopω u ω u⊤ for some ω∈ℝrsuperscriptℝω ^rω ∈ blackboard_Rr. Then, by to the second condition M′⁢vj=M⁢V−j⁢η−jsuperscript′subscriptsubscriptsubscriptM v_j=MV_-j _-jM′ vitalic_j = M V- j η- j, we have ω=1u⊤⁢vj⁢M⁢(V−j⁢η−j−vj)1superscripttopsubscriptsubscriptsubscriptsubscriptω= 1u v_jM(V_-j _-j-v_j)ω = divide start_ARG 1 end_ARG start_ARG u⊤ vitalic_j end_ARG M ( V- j η- j - vitalic_j ). This means by replacing M with M′=M⁢(I+1u⊤⁢vj⁢(V−j⁢η−j−vj)⁢u⊤)superscript′1superscripttopsubscriptsubscriptsubscriptsubscriptsuperscripttopM =M(I+ 1u v_j(V_-j _-j-v_j)u )M′ = M ( I + divide start_ARG 1 end_ARG start_ARG u⊤ vitalic_j end_ARG ( V- j η- j - vitalic_j ) u⊤ ), we are able to redact label j. When conditioned on j, the edited model will generate another digit based on which element in η−jsubscript _-jη- j is non-zero. Redacting multiple labels. Suppose there are multiple labels 1,⋯,J1⋯\1,·s,J\ 1 , ⋯ , J (J<kJ<kJ < k) to be redacted. The M′ matrix needs to satisfy M′⁢vi=M⁢visuperscript′subscriptsubscriptM v_i=Mv_iM′ vitalic_i = M vitalic_i for i>Ji>Ji > J, and M′⁢vj=M⁢V−J⁢η−jsuperscript′subscriptsubscriptsubscriptM v_j=MV_-J _-jM′ vitalic_j = M V- J η- j for j≤Jj≤ Jj ≤ J, where V−J=[vJ+1,⋯,vk]∈ℝk×(k−J)subscriptsubscript1⋯subscriptsuperscriptℝV_-J=[v_J+1,·s,v_k] ^k×(k-J)V- J = [ vitalic_J + 1 , ⋯ , vitalic_k ] ∈ blackboard_Rk × ( k - J ). For j≤Jj≤ Jj ≤ J, let ujsubscriptu_juitalic_j be the basis vector of the null space of vii≠jsubscriptsubscript\v_i\_i≠ j vitalic_i i ≠ j. Each row of M′−Msuperscript′M -M′ - M is in the null space of vii>Jsubscriptsubscript\v_i\_i>J vitalic_i i > J, which can be written as a linear combination of ujj=1Jsuperscriptsubscriptsubscript1\u_j\_j=1^J uitalic_j j = 1J. Therefore, we can represent M′−Msuperscript′M -M′ - M as M′−M=∑j=1Jωj⁢uj⊤=W⁢U⊤,superscript′subscript1subscriptsuperscriptsubscripttopsuperscripttopM -M= _j=1^J _ju_j =WU ,M′ - M = ∑j = 1J ωitalic_j uitalic_j⊤ = W U⊤ , where the j-th column of W (U) is ωjsubscript _jωitalic_j (ujsubscriptu_juitalic_j). Let VJ=[v1,⋯,vJ]subscriptsubscript1⋯subscriptV_J=[v_1,·s,v_J]Vitalic_J = [ v1 , ⋯ , vitalic_J ] and Y−J=[η−1,⋯,η−J]subscriptsubscript1⋯subscriptY_-J=[ _-1,·s, _-J]Y- J = [ η- 1 , ⋯ , η- J ]. We have M′⁢VJ=M⁢V−J⁢Y−Jsuperscript′subscriptsubscriptsubscriptM V_J=MV_-JY_-JM′ Vitalic_J = M V- J Y- J. This simplifies to W⁢U⊤⁢VJ=M⁢(V−J⁢Y−J−VJ).superscripttopsubscriptsubscriptsubscriptsubscriptWU V_J=M(V_-JY_-J-V_J).W U⊤ Vitalic_J = M ( V- J Y- J - Vitalic_J ) . Notice that U⊤⁢VJsuperscripttopsubscriptU V_JU⊤ Vitalic_J is a diagonal matrix with j-th diagonal element uj⊤⁢vj≠0superscriptsubscripttopsubscript0u_j v_j≠ 0uitalic_j⊤ vitalic_j ≠ 0. Therefore, we have W=M⁢(V−J⁢Y−J−VJ)⁢(U⊤⁢VJ)−1.subscriptsubscriptsubscriptsuperscriptsuperscripttopsubscript1W=M(V_-JY_-J-V_J)(U V_J)^-1.W = M ( V- J Y- J - Vitalic_J ) ( U⊤ Vitalic_J )- 1 . (3) Simplified formula for one-hot embedding vectors. Let vi=isubscriptsubscriptv_i=e_ivitalic_i = eitalic_i for each i. Then, we have ui=vi=isubscriptsubscriptsubscriptu_i=v_i=e_iuitalic_i = vitalic_i = eitalic_i, and therefore U⊤⁢VJ=IsuperscripttopsubscriptU V_J=IU⊤ Vitalic_J = I. We also have U=VJ=[IJ|]⊤subscriptsuperscriptdelimited-[]conditionalsubscript0topU=V_J=[I_J|0] U = Vitalic_J = [ Iitalic_J | 0 ]⊤ and V−J=[|Ik−J]subscriptdelimited-[]conditional0subscriptV_-J=[0|I_k-J]V- J = [ 0 | Iitalic_k - J ], where IJsubscriptI_JIitalic_J is the J-dimensional identity matrix. Then, W⁢U⊤=M⁢([|Ik−J]⁢Y−J−[IJ|]⊤)⁢[IJ|]=M⁢(−IJY−J).superscripttopdelimited-[]conditional0subscriptsubscriptsuperscriptdelimited-[]conditionalsubscript0topdelimited-[]conditionalsubscript0subscript0subscript0WU =M([0|I_k-J]Y_-J-[I_J|0] )[I_J|% 0]=M ( array[]c-I_J&0\\ Y_-J&0 array ).W U⊤ = M ( [ 0 | Iitalic_k - J ] Y- J - [ Iitalic_J | 0 ]⊤ ) [ Iitalic_J | 0 ] = M ( start_ARRAY start_ROW start_CELL - Iitalic_J end_CELL start_CELL 0 end_CELL end_ROW start_ROW start_CELL Y- J end_CELL start_CELL 0 end_CELL end_ROW end_ARRAY ) . As a result, M′=M+W⁢U⊤=M⁢(Y−JIk−J).superscript′op00subscriptsubscriptM =M+WU =M ( array[]c0&0\\ Y_-J&I_k-J array ).M′ = M + W U⊤ = M ( start_ARRAY start_ROW start_CELL 0 end_CELL start_CELL 0 end_CELL end_ROW start_ROW start_CELL Y- J end_CELL start_CELL Iitalic_k - J end_CELL end_ROW end_ARRAY ) . Higher embedding dimension. Because of linear independence, the null space of vii≠jsubscriptsubscript\v_i\_i≠ j vitalic_i i ≠ j has 1 dimension higher than the null space of vii=1ksuperscriptsubscriptsubscript1\v_i\_i=1^k vitalic_i i = 1k. Therefore, we can pick uj∈⁢(vii≠j)∖⁢(vii=1k)subscriptsubscriptsubscriptsuperscriptsubscriptsubscript1u_j (\v_i\_i≠ j) (\v_i\_i=% 1^k)uitalic_j ∈ null ( vitalic_i i ≠ j ) ∖ null ( vitalic_i i = 1k ). V Redacting Models Conditioned on Continuous Representations In practice, the networks are usually complicated and highly non-linear. Therefore, there is generally no explicit formula to achieve (2) due to non-linearity and limited expressive power of the conditioning network. To approximately achieve (2), we propose to distill the conditioning network by minimizing minH′⁡L⁢(H′;λ)=c∈∖Ω⁢‖H′⁢(c)−H⁢(c)‖+λ⋅c∈Ω⁢‖H′⁢(c)−H⁢(c^)‖subscriptsuperscript′absentsubscriptsubscriptΩnormsuperscript′missing-subexpression⋅subscriptsubscriptΩnormsuperscript′ array[]rl _H ~L(H ;λ)=&E_c∈% C _ \|H (c)-H(c)\|\\ &+λ·E_c _ \|H (c)-H( c)% \| arraystart_ARRAY start_ROW start_CELL minitalic_H′ L ( H′ ; λ ) = end_CELL start_CELL blackboard_Ec ∈ C ∖ C start_POSTSUBSCRIPT Ω end_POSTSUBSCRIPT ∥ H′ ( c ) - H ( c ) ∥ end_CELL end_ROW start_ROW start_CELL end_CELL start_CELL + λ ⋅ blackboard_Ec ∈ C start_POSTSUBSCRIPT Ω end_POSTSUBSCRIPT ∥ H′ ( c ) - H ( over start_ARG c end_ARG ) ∥ end_CELL end_ROW end_ARRAY (4) for some metric ∥⋅∥\|·\|∥ ⋅ ∥ and balancing coefficient λ>00λ>0λ > 0. In the rest of this section, we study two types of common conditional generative models: image models conditioned on text prompts, and speech models conditioned on spectrogram representations. We will demonstrate specific losses and distillation techniques for each model that align with the slightly different goals in each task. V-A Redacting GAN-based Text-to-Image Models In this section, we study how to redact text prompts in text-to-image models. Modern text-to-image models can produce high-resolution images conditioned on text prompts that may be offensive, biased, malignant, or fabricated (nichol2021glide, ; birhane2021multimodal, ; schuhmann2022laion, ; ramesh2022hierarchical, ; rando2022red, ; nudenet, ; man, ). These models are usually expensive to re-train, so it is important to redact these prompts without re-training. Especially, we look at DM-GAN (zhu2019dm, ), a GAN-based text-to-image model. It is trained on pairs of text and images from the CUB dataset (CUB, ; reed2016learning, ), a dataset for various species of birds. DM-GAN is composed of three cascaded generative networks G1,G2,G3subscript1subscript2subscript3\G_1,G_2,G_3\ G1 , G2 , G3 . The first G1subscript1G_1G1 generates 64×64646464× 6464 × 64 images, the second G2subscript2G_2G2 up-samples to 128×128128128128× 128128 × 128, and the third G3subscript3G_3G3 up-samples to 256×256256256256× 256256 × 256. Each GisubscriptG_iGitalic_i has its own conditioning network HisubscriptH_iHitalic_i. For a given prompt c, the model computes a sentence embedding vs⁢(c)subscriptv_s(c)vitalic_s ( c ) and word embeddings vw⁢(c)subscriptv_w(c)vitalic_w ( c ) from a pre-trained text encoder (xu2018attngan, ). The first conditioning network H1subscript1H_1H1 performs conditioning augmentation on the sentence embedding and concatenate the output to the latent variable. H2subscript2H_2H2 and H3subscript3H_3H3 apply memory writing modules to the word embeddings and fuse the outputs with the previously generated low-resolution images via several gates. Defining c^normal- cover start_ARG c end_ARG. We assume ΩsubscriptΩC_ Croman_Ω contains prompts that have undesirable words or phrases. For these prompts, the reference prompts are defined by replacing these words with non-redacted ones. Sequential distillation. We propose to distill the conditioning networks H1,H2,H3subscript1subscript2subscript3\H_1,H_2,H_3\ H1 , H2 , H3 sequentially based on (4). This is because both G2subscript2G_2G2 and G3subscript3G_3G3 are generative super-sampling networks, which take G1subscript1G_1G1 and G2subscript2G_2G2 outputs as inputs, respectively. After G1subscript1G_1G1 is edited to G1′subscript1′G_1 G1′ for redaction, G2subscript2G_2G2 will take G1′subscript1′G_1 G1′ outputs as inputs, and similar for G3subscript3G_3G3. Formally, H1′=arg⁡minH1′c∈∖Ω∥H1′(vs(c))−H1(vs(c))∥+λ⋅c∈Ω∥H1′(vs(c))−H1(vs(c^))∥, array[]rl H_1 = _H_1 &\% E_c _ \|H_1 (v_s% (c))-H_1(v_s(c))\|\\ &+λ·E_c _ \|H_1 (v_s(c))-% H_1(v_s( [rgb]1,0,0 c))\|\, arraystart_ARRAY start_ROW start_CELL H1′ = arg minitalic_H start_POSTSUBSCRIPT 1′ end_POSTSUBSCRIPT end_CELL start_CELL blackboard_Ec ∈ C ∖ C start_POSTSUBSCRIPT Ω end_POSTSUBSCRIPT ∥ H1′ ( vitalic_s ( c ) ) - H1 ( vitalic_s ( c ) ) ∥ end_CELL end_ROW start_ROW start_CELL end_CELL start_CELL + λ ⋅ blackboard_Ec ∈ C start_POSTSUBSCRIPT Ω end_POSTSUBSCRIPT ∥ H1′ ( vitalic_s ( c ) ) - H1 ( vitalic_s ( over start_ARG c end_ARG ) ) ∥ , end_CELL end_ROW end_ARRAY (5) Hi′=arg⁡minHi′c∈∖Ω,z∥Hi′(vw(c),Gi−1′(z|c))−Hi(vw(c),Gi−1′(z|c))∥+λ⋅c∈Ω,z∥Hi′(vw(c),Gi−1′(z|c))−Hi(vw(c^),Gi−1′(z|c^))∥ array[]rll H_i = _H_i &\% E_c _ ,z&\|H_i (v% _w(c),G_i-1 (z|c))-\\ &&~~~~~~H_i(v_w(c),G_i-1 (z|c))\|\\ +&λ·E_c _ ,z&\|H_i^% (v_w(c),G_i-1 (z|c))-\\ &&~~~~~~H_i(v_w( [rgb]1,0,0 c),G_i-1 (% z| [rgb]1,0,0 c))\|\ arraystart_ARRAY start_ROW start_CELL Hitalic_i′ = arg minitalic_H start_POSTSUBSCRIPT i′ end_POSTSUBSCRIPT end_CELL start_CELL blackboard_Ec ∈ C ∖ C start_POSTSUBSCRIPT Ω , z end_POSTSUBSCRIPT end_CELL start_CELL ∥ Hitalic_i′ ( vitalic_w ( c ) , Gitalic_i - 1′ ( z | c ) ) - end_CELL end_ROW start_ROW start_CELL end_CELL start_CELL end_CELL start_CELL Hitalic_i ( vitalic_w ( c ) , Gitalic_i - 1′ ( z | c ) ) ∥ end_CELL end_ROW start_ROW start_CELL + end_CELL start_CELL λ ⋅ blackboard_Ec ∈ C start_POSTSUBSCRIPT Ω , z end_POSTSUBSCRIPT end_CELL start_CELL ∥ Hitalic_i′ ( vitalic_w ( c ) , Gitalic_i - 1′ ( z | c ) ) - end_CELL end_ROW start_ROW start_CELL end_CELL start_CELL end_CELL start_CELL Hitalic_i ( vitalic_w ( over start_ARG c end_ARG ) , Gitalic_i - 1′ ( z | over start_ARG c end_ARG ) ) ∥ end_CELL end_ROW end_ARRAY (6) for i=2,323i=2,3i = 2 , 3. Improved capacity. As H1′subscript1′H_1 H1′ needs to approximate a piecewise function that is defined differently for two sets of sentence embeddings, we need to increase the capacity of H1′subscript1′H_1 H1′ for better distillation. We append a few LSTM layers to the beginning of H1′subscript1′H_1 H1′, which directly take the sentence embeddings as inputs. The LSTM layers are followed by a convolution layer that reduces hidden dimensions to 1. We initialize this layer with zero weights for training stability. We expect these layers can project sentence embeddings of c to those of c^ cover start_ARG c end_ARG. The rest of H1′subscript1′H_1 H1′ has the same architecture as H1subscript1H_1H1 but all weights are initialized for training. We do not increase the capacity of H2′subscript2′H_2 H2′ and H3′subscript3′H_3 H3′ for two reasons. First, H1′subscript1′H_1 H1′ has more direct impact on the generated images because it directly controls the initial low-resolution image. Second, the memory writing modules of H2subscript2H_2H2 and H3subscript3H_3H3 are already very expressive. Fixing the variance prediction part in H1subscript1H_1H1. We aim to reduce the computational overhead by fixing certain variables. The conditioning augmentation module in H1subscript1H_1H1 first computes a mean and a variance vector, and then samples from the Gaussian defined by them. We fix the variance prediction part and only distill the mean prediction part. In our experiments the number of parameters to be trained in H1′subscript1′H_1 H1′ (with improved capacity) is reduced by ∼32%similar-toabsentpercent32 32\%∼ 32 % and therefore matches H1subscript1H_1H1. λ annealing. In order to make sure the distilled conditioning networks also approximate the pre-trained ones well for non-redacted prompts, we anneal the balancing coefficient λ during distillation: we initialize λ=λminsubscriptλ= _ λ = λroman_min and linearly increases to λmaxsubscript _ λroman_max in the end. V-B Redacting Diffusion-based Text-to-Speech Models Modern text-to-speech models can turn text into high-quality speech in unseen voices such as celebrity voices (kong2021diffwave, ; Betker2022TTS, ; wang2023neural, ; zhang2023speak, ). This may have unpredictable public impact if these models are used to fake celebrities. In this section, we study redacting certain voices from a pre-trained text-to-speech model. Especially, we look at DiffWave (kong2021diffwave, ), a diffusion probabilistic model that is conditioned on spectrogram and outputs waveform. It is trained on speech of a single female reading a book, which we call the pre-trained voice. There are n=3030n=30n = 30 layers or residual blocks in DiffWave, each containing one independent conditioning network HisubscriptH_iHitalic_i. The architecture of each HisubscriptH_iHitalic_i includes two up-sampling layers followed by one convolution layer. Defining c^normal- cover start_ARG c end_ARG with voice cloning. We assume ΩsubscriptΩC_ Croman_Ω contains a few clips of speech in a specific voice. We train a voice cloning model (CycleGAN-VC2 (kaneko2019cyclegan, )) between the specific and pre-trained voices, and then transform all clips in ΩsubscriptΩC_ Croman_Ω to the pre-trained voice. By doing this we obtain time-aligned pairs between c∈ΩsubscriptΩc _ c ∈ Croman_Ω and the corresponding c^ cover start_ARG c end_ARG: when we select a small duration [t,t+Δ⁢t]Δ[t,t+ t][ t , t + Δ t ], the content of ct:t+Δ⁢tsubscript:Δc_t:t+ tcitalic_t : t + Δ t is the same as c^t:t+Δ⁢tsubscript^:Δ c_t:t+ tover start_ARG c end_ARGt : t + Δ t, yet only the voices are different. Improved voice cloning. We find the voice cloning quality of CycleGAN-VC2 can be improved by making the two unpaired training sets more similar. We first use a pre-trained Whisper model (radford2022robust, ) to extract text from redacted speech. Then, we use Tortoise-TTS (Betker2022TTS, ) to turn these text into speech in the pre-trained voice. Note that this cannot be used to define c^ cover start_ARG c end_ARG directly because the generated samples are not time-aligned with the speech to be redacted. However, these generated samples are more similar to the redacted samples because they have the same text, and therefore it is easier for CycleGAN-VC2 to learn transformations between these two voices. Parallel distillation. We propose to distill all conditional layers HisubscriptH_iHitalic_i’s in parallel as they are independent. We minimize the following loss: min⁡1n⁢∑i=1nL⁢(Hi′;λ).1superscriptsubscript1superscriptsubscript′ 1n _i=1^nL(H_i ;λ).min divide start_ARG 1 end_ARG start_ARG n end_ARG ∑i = 1n L ( Hitalic_i′ ; λ ) . Fixing up-sampling layers in HisubscriptH_iHitalic_i. To reduce computation overhead we fix the two up-sampling layers in each HisubscriptH_iHitalic_i. We only distill the last convolution layer in each HisubscriptH_iHitalic_i. Improved capacity. To improve redaction quality, we increase the capacity of each Hi′subscript′H_i Hitalic_i′ by replacing its last convolution layer hconvsubscriptℎconvh_convhroman_conv with a spectrogram-rewriting module. It has two components: a gate hgatesubscriptℎgateh_gatehroman_gate consisting of a convolution with zero initialization followed by sigmoid, and a transformation block htranssubscriptℎtransh_transhroman_trans consisting of two convolution layers. The forward computation of the spectrogram-rewriting module is defined as: y=hconv⁢(v)⊙hgate⁢(v)+hconv⁢(htrans⁢(v))⊙(1−hgate⁢(v)),direct-productsubscriptℎconvsubscriptℎgatedirect-productsubscriptℎconvsubscriptℎtrans1subscriptℎgatey=h_conv(v) h_gate(v)+h_conv(h_% trans(v)) (1-h_gate(v)),y = hroman_conv ( v ) ⊙ hroman_gate ( v ) + hroman_conv ( hroman_trans ( v ) ) ⊙ ( 1 - hroman_gate ( v ) ) , where v is the up-sampled mel-spectrogram and y is the output representation at each layer. We expect this module can retain the pre-trained voice and also project redacted voices to the pre-trained voice. Non-uniform distillation losses. We conjecture the all conditioning layers are not of the same importance because of their order and different hyper-parameters specifically the dilation 2i⁢mod⁢n′2modsuperscript′2^i~mod~n 2i mod n start_POSTSUPERSCRIPT ′ end_POSTSUPERSCRIPT in the corresponding residual layer. This motivates us to use different weights and λ values for each HisubscriptH_iHitalic_i: min⁢∑i=1nwi⁢L⁢(Hi′;λi).superscriptsubscript1subscriptsuperscriptsubscript′subscript _i=1^nw_iL(H_i ; _i).min ∑i = 1n witalic_i L ( Hitalic_i′ ; λitalic_i ) . We test different schedules described in Table I. TABLE I: Schedules for the non-uniform distillation losses. name schedule wisubscriptw_iwitalic_i-order wi=1n+α⁢(i−(n+1)/2)subscript112w_i= 1n+α(i-(n+1)/2)witalic_i = divide start_ARG 1 end_ARG start_ARG n end_ARG + α ( i - ( n + 1 ) / 2 ) λisubscript _iλitalic_i-order λi=λ+β⁢(i−(n+1)/2)subscript12 _i=λ+β(i-(n+1)/2)λitalic_i = λ + β ( i - ( n + 1 ) / 2 ) wisubscriptw_iwitalic_i-dilation wi=1n+α⁢(i⁢mod⁢n′−(n′+1)/6)subscript1modsuperscript′16w_i= 1n+α(i~mod~n -(n +1)/6)witalic_i = divide start_ARG 1 end_ARG start_ARG n end_ARG + α ( i mod n′ - ( n′ + 1 ) / 6 ) λisubscript _iλitalic_i-dilation λi=λ+β⁢(i⁢mod⁢n′−(n′+1)/6)subscriptmodsuperscript′16 _i=λ+β(i~mod~n -(n +1)/6)λitalic_i = λ + β ( i mod n′ - ( n′ + 1 ) / 6 ) VI Experiments In this section, we aim to answer the following questions. (1) Is the redaction method in Section IV able to fully redact labels? And (2) do the redaction algorithms in Section V redact certain conditionals well and retain high generation quality on real-world applications? VI-A Redacting Models Conditioned on Discrete Labels We train a class-conditional GAN called cGAN (mirza2014conditional, ) on MNIST (lecun2010mnist, ). Each conditional has a 10101010-dimensional embedding vector, and is concatenated to the latent vector as the input. The affine transformation matrix M in Section IV is the last 10 rows of the weight matrix of the first fully connected layer. We redact labels 0,1,2,3 according to (3), where we let c^=9−c^9 c=9-cover start_ARG c end_ARG = 9 - c for them. Generated samples of pre-trained and redacted models are shown in Fig. 2. Figure 2: Redacting labels 0,1,2,3 in cGAN on MNIST. Upper: samples generated from the pre-trained model. Down: samples generated from the redacted model. Redacted conditionals (first two rows) are edited as expected, and other conditionals (last three rows) remain unchanged. VI-B Redacting GAN-based Text-to-Image Models Setup. We use the pre-trained DM-GAN (zhu2019dm, ) model trained on the CUB dataset (CUB, ), which contains 8855 training images and 2933 testing images of 200 subcategories belonging to birds. Each image has 10 captions (reed2016learning, ). Our distillation algorithm is trained with the caption data only. We redact prompts that contain certain words or phrases. We redact the word blue∈cabsent∈ c∈ c by defining c^ cover start_ARG c end_ARG as the prompt that replaces all blue with another word red. 555Any word other than blue can be used. Similarly, we redact blue wings and red wings by replacing these phrases to white wings. We redact long beak and white belly by replacing the first to short beak and the second to black belly. Finally, we redact yellow and red by replacing them to black, which is more challenging as many samples are redacted. Table I includes the number of training and test prompts that are redacted in each experiment. Note that when we redact blue wings and red wings, we also redact phrases wings that are blue and wings that are red. TABLE I: Number of redacted training and test prompts. There are 88550 training prompts and 29330 test prompts in total. Redaction prompts # redacted training prompts # redacted test prompts long beak, white belly 10377 3369 blue / red wings 732 303 blue 6113 2175 yellow, red 29514 9319 Architecture and optimization. The architecture of the pre-trained model and other details are in Appendix B. The architecture of student conditioning networks with improved capacity is shown in Fig. 4. For each HisubscriptH_iHitalic_i, i=1,2,3123i=1,2,3i = 1 , 2 , 3, we use the Adam optimizer (kingma2014adam, ) with a learning rate 0.0050.0050.0050.005 to optimize the mean square error loss. The redaction algorithm terminates at 1000 iterations. For H1subscript1H_1H1 we use a batch size of 128, and for H2subscript2H_2H2 and H3subscript3H_3H3 we reduce the batch size to 32 in order to fit into GPU memory. Configurations. We first use the sequential distillation (5) and (6) with λ=11λ=1λ = 1 to perform redaction, which we denote as the base configuration. We then improve the capacity by using a 3-layer bidirectional LSTM with hidden size =32absent32=32= 32 and dropout rate =0.1absent0.1=0.1= 0.1. Next, we fix the variance prediction in H1subscript1H_1H1 to reduce the number of parameters to optimize, which matches the base configuration. Finally, we apply λ annealing by setting λmin=1subscript1 _ =1λroman_min = 1 and λmax=3subscript3 _ =3λroman_max = 3. Baseline. We compare to the Rewriting algorithm (bau2020rewriting, ), a semantic editing method originally designed for unconditional generative models. We adapt their method to DM-GAN by rewriting G1subscript1G_1G1, G2subscript2G_2G2, and G3subscript3G_3G3 sequentially. For both G2subscript2G_2G2 and G3subscript3G_3G3 we rewrite the up-sampling layer before the feature output. For G1subscript1G_1G1 we have choices of rewriting the up-sampling layer at different resolutions ranging from 8×8888× 88 × 8 to 64×64646464× 6464 × 64. We test all these choices in the experiment. Evaluation metrics. To evaluate generation quality of G′, we compute Inception Scores (IS) (salimans2016improved, ) for images conditioned on redacted and valid prompts, separately. In detail, the IS scores are computed as exp⁡(x⁢(p⁢(y|x)∥p⁢(y)))subscriptconditionalconditional (E_xKL(p(y|x) p(y)))exp ( blackboard_Ex blackboard_K blackboard_L ( p ( y | x ) ∥ p ( y ) ) ), where x∼pG′(⋅|c)x p_G (·|c)x ∼ pitalic_G′ ( ⋅ | c ) for c∼Uniform⁢(Ω)similar-toUniformsubscriptΩc (C_ )c ∼ Uniform ( Croman_Ω ) or Uniform⁢(∖Ω)UniformsubscriptΩUniform(C _ )Uniform ( C ∖ Croman_Ω ), p⁢(y|x)conditionalp(y|x)p ( y | x ) is the logit from the Inception-V3 output layer (szegedy2016rethinking, ), and p⁢(y)p(y)p ( y ) is the marginal. We generate one sample for each text prompt for evaluation. To evaluate redaction quality, we compute the following three metrics where c∼Ωsimilar-tosubscriptΩc _ c ∼ Croman_Ω and z∼similar-toz ∼ N. 1. ℛG(⋅|c/c^)R_G(·|c/ c)Ritalic_G ( ⋅ | c / over start_ARG c end_ARG ) measures faithfulness of G′ on the redaction prompts. It is defined as the fraction of samples G′⁢(z|c)superscript′conditional\G (z|c)\ G′ ( z | c ) such that dist⁢(G′⁢(z|c),G⁢(z|c^))<dist⁢(G′⁢(z|c),G⁢(z|c))distsuperscript′conditionalconditional^distsuperscript′conditionalconditionaldist(G (z|c),G(z| c))<dist(G (z|c),G(z|% c))dist ( G′ ( z | c ) , G ( z | over start_ARG c end_ARG ) ) < dist ( G′ ( z | c ) , G ( z | c ) ), where distdistdistdist is ℓ2subscriptℓ2 _2ℓ2 distance in the Inception-V3 feature space (szegedy2016rethinking, ). 2. A modified R-precision score ℛrsubscriptℛR_rRitalic_r measures how well G′⁢(z|c)superscript′conditionalG (z|c)G′ ( z | c ) matches the target caption c^ cover start_ARG c end_ARG. (xu2018attngan, ) defined correlation corr⁢(x,c)corrcorr(x,c)corr ( x , c ) between sample x and caption c as cos⁡⟨EncCNN⁢(x),EncRNN⁢(c)⟩subscriptEncCNNsubscriptEncRNN _CNN(x),Enc_RNN(c) ⟨ Encroman_CNN ( x ) , Encroman_RNN ( c ) ⟩ for pretrained CNN (image) and RNN (text) encoders. We use the pretrained encoders from DM-GAN. Then, ℛrsubscriptℛR_rRitalic_r is defined as the fraction of samples G′⁢(z|c)superscript′conditionalG (z|c)G′ ( z | c ) such that corr⁢(G′⁢(z|c),c^)corrsuperscript′conditional^corr(G (z|c), c)corr ( G′ ( z | c ) , over start_ARG c end_ARG ) is larger than the correlation between G′⁢(z|c)superscript′conditionalG (z|c)G′ ( z | c ) and 100 random, mismatch captions. 3. We further introduce ℛc/c^subscriptℛ^R_c/ cRitalic_c / over start_ARG c end_ARG, which measures how much better G′⁢(z|c)superscript′conditionalG (z|c)G′ ( z | c ) matches c^ cover start_ARG c end_ARG than c. It is defined as the fraction of samples G′⁢(z|c)superscript′conditionalG (z|c)G′ ( z | c ) such that corr⁢(G′⁢(z|c),c^)>corr⁢(G′⁢(z|c),c)corrsuperscript′conditional^corrsuperscript′conditionalcorr(G (z|c), c)>corr(G (z|c),c)corr ( G′ ( z | c ) , over start_ARG c end_ARG ) > corr ( G′ ( z | c ) , c ). Results. The results for redacting yellow and red shown in Table I. The base configuration already achieves good redaction and generation quality. After improving capacity, we find all redaction quality metrics increase by 2.3∼2.7%similar-to2.3percent2.72.3 2.7\%2.3 ∼ 2.7 %, and generation quality is retained. After we fix the variance prediction in H1subscript1H_1H1, the redaction decrease by ∼1%similar-toabsentpercent1 1\%∼ 1 %, but the generation quality on valid prompts increases by 0.10.10.10.1. Finally, by performing λ annealing, all metrics improve. Notably, ℛG(⋅|c/c^)R_G(·|c/ c)Ritalic_G ( ⋅ | c / over start_ARG c end_ARG ) and ℛc/c^subscriptℛ^R_c/ cRitalic_c / over start_ARG c end_ARG increase by over 5%percent55\%5 %, indicating generated samples are more similar to c^ cover start_ARG c end_ARG rather than c. We find the Rewriting baselines achieve better IS. However, generated samples are blurred and lack sharp edges as shown in the visualization. The redaction quality of Rewriting has a significant gap with ours: all redaction metrics are less than half of ours. Especially, ℛrsubscriptℛR_rRitalic_r is worse than the pre-trained model, indicating generated samples conditioned on redacted prompts are not very correlated to c^ cover start_ARG c end_ARG. We hypothesize the main problem for Rewriting is that it is crafted for 2D convolutions and edits the main generative network, which makes it hard to handle and distinguish the information from different prompts. In terms of different choices of resolutions, we find rewriting the layer at resolution 8×8888× 88 × 8 yields the best redaction quality. TABLE I: Generation and redaction quality after redacting yellow and red. Our method achieves significantly better redaction quality than Rewriting and retains good generation quality. The effects of each component within our method are displayed. Method Inception Score (↑ ↑) Redacting quality (↑ ↑) Training time redacted valid ℛG(⋅|c/c^)R_G(·|c/ c)Ritalic_G ( ⋅ | c / over start_ARG c end_ARG ) ℛc/c^subscriptℛ^R_c/ cRitalic_c / over start_ARG c end_ARG ℛrsubscriptℛR_rRitalic_r mins Pre-trained 4.624.624.624.62 5.225.225.225.22 0%percent00\%0 % 6.0%percent6.06.0\%6.0 % 13.5%percent13.513.5\%13.5 % - Rewriting 8×8888× 88 × 8 5.575.575.575.57 5.525.525.525.52 33.0%percent33.033.0\%33.0 % 39.7%percent39.739.7\%39.7 % 5.0%percent5.05.0\%5.0 % 24.324.324.324.3 16×16161616× 1616 × 16 5.635.635.635.63 5.535.535.535.53 30.4%percent30.430.4\%30.4 % 37.2%percent37.237.2\%37.2 % 4.8%percent4.84.8\%4.8 % 25.325.325.325.3 32×32323232× 3232 × 32 5.725.725.725.72 5.715.715.715.71 28.8%percent28.828.8\%28.8 % 35.9%percent35.935.9\%35.9 % 4.7%percent4.74.7\%4.7 % 23.523.523.523.5 64×64646464× 6464 × 64 5.775.775.775.77 5.735.735.735.73 27.5%percent27.527.5\%27.5 % 35.2%percent35.235.2\%35.2 % 4.6%percent4.64.6\%4.6 % 24.124.124.124.1 Ours (base) 4.794.794.794.79 5.235.235.235.23 65.1%percent65.165.1\%65.1 % 77.0%percent77.077.0\%77.0 % 46.9%percent46.946.9\%46.9 % 27.427.427.427.4 + improved capacity 4.744.744.744.74 5.255.255.255.25 67.8%percent67.867.8\%67.8 % 79.7%percent79.779.7\%79.7 % 49.2%percent49.249.2\%49.2 % 28.428.428.428.4 + fix variance 4.794.794.794.79 5.355.355.355.35 66.5%percent66.566.5\%66.5 % 79.0%percent79.079.0\%79.0 % 48.4%percent48.448.4\%48.4 % 22.522.522.522.5 + λ annealing 4.84 5.36 72.2%percent72.272.2\%72.2 % 84.2%percent84.284.2\%84.2 % 49.2%percent49.249.2\%49.2 % 29.929.929.929.9 Table IV includes results for redacting the other prompts. The Rewriting baseline is applied to 8×8888× 88 × 8 resolution in H1subscript1H_1H1 because it yields the best redaction quality. We find the base configuration of our method is already very effective. Our method greatly outperforms Rewriting in all redaction quality metrics and keeps good generation quality. Visualization. See Appendix B-B for generated samples. The Rewriting baseline generate very blurry samples while our method generates high-quality, sharp samples which also satisfy the redaction requirements. Computation. Data redaction takes about 30 minutes to train on a single NVIDIA 3080 GPU. TABLE IV: Generation and redaction quality after redacting various words or phrases. Our method achieves significantly better redaction quality than Rewriting and retains good generation quality. Redaction prompts Method Inception Score (↑ ↑) Redacting quality (↑ ↑) Training time redacted valid ℛG(⋅|c/c^)R_G(·|c/ c)Ritalic_G ( ⋅ | c / over start_ARG c end_ARG ) ℛc/c^subscriptℛ^R_c/ cRitalic_c / over start_ARG c end_ARG ℛrsubscriptℛR_rRitalic_r mins long beak, white belly Pre-trained 4.144.144.144.14 5.615.615.615.61 0%percent00\%0 % 5.2%percent5.25.2\%5.2 % 13.1%percent13.113.1\%13.1 % - Rewriting 5.365.365.365.36 5.855.855.855.85 32.6%percent32.632.6\%32.6 % 51.4%percent51.451.4\%51.4 % 5.6%percent5.65.6\%5.6 % 23.023.023.023.0 Ours (base) 4.914.914.914.91 5.815.815.815.81 70.5%percent70.570.5\%70.5 % 83.6%percent83.683.6\%83.6 % 50.1%percent50.150.1\%50.1 % 28.328.328.328.3 blue / red wings Pre-trained 3.973.973.973.97 5.485.485.485.48 0%percent00\%0 % 4.1%percent4.14.1\%4.1 % 13.1%percent13.113.1\%13.1 % - Rewriting 5.215.215.215.21 5.855.855.855.85 27.8%percent27.827.8\%27.8 % 15.1%percent15.115.1\%15.1 % 6.9%percent6.96.9\%6.9 % 23.323.323.323.3 Ours (base) 5.045.045.045.04 5.285.285.285.28 68.6%percent68.668.6\%68.6 % 71.7%percent71.771.7\%71.7 % 58.4%percent58.458.4\%58.4 % 28.128.128.128.1 blue Pre-trained 3.653.653.653.65 5.185.185.185.18 0%percent00\%0 % 3.2%percent3.23.2\%3.2 % 7.2%percent7.27.2\%7.2 % - Rewriting 5.005.005.005.00 5.455.455.455.45 61.8%percent61.861.8\%61.8 % 60.2%percent60.260.2\%60.2 % 17.7%percent17.717.7\%17.7 % 28.728.728.728.7 Ours (base) 3.853.853.853.85 5.215.215.215.21 81.3%percent81.381.3\%81.3 % 89.7%percent89.789.7\%89.7 % 66.2%percent66.266.2\%66.2 % 34.734.734.734.7 Robustness to adversarial prompting. In order to understand whether adversarial prompts may cause the redacted model to generate content we would like to redact, we perform an adversarial prompting attack to redacted or rewritten models in this section. Specifically, we adopt the Square Attack (andriushchenko2020square, ; maus2023adversarial, ) directly to the discrete text space. For c∈ΩsubscriptΩc _ c ∈ Croman_Ω, the goal is to find an adversarial conditional cadvsubscriptadvc_advcroman_adv such that corr⁢(G′⁢(z|cadv),c)>corr⁢(G′⁢(z|cadv),c^)corrsuperscript′conditionalsubscriptadvcorrsuperscript′conditionalsubscriptadv^corr(G (z|c_adv),c)>corr(G (z|c_% adv), c)corr ( G′ ( z | croman_adv ) , c ) > corr ( G′ ( z | croman_adv ) , over start_ARG c end_ARG ). The algorithm is illustrated in Algorithm 1. See Appendix B-C for a few examples of successful attacks. We measure the success rates of the proposed attack in Table V. The success rates for our redaction method is consistently lower than the Rewriting baseline (by 31%∼45%similar-topercent31percent4531\% 45\%31 % ∼ 45 %), indicating our method is considerably more robust to adversarial prompting attacks than Rewriting. Algorithm 1 Adversarial Prompting via Square Attack (andriushchenko2020square, ; maus2023adversarial, ) 1: Initialize cadv=csubscriptadvc_adv=ccroman_adv = c. 2: for iteration = 1,⋯,161⋯161,·s,161 , ⋯ , 16 do 3: Uniformly sample a position s of the caption cadvsubscriptadvc_advcroman_adv to update. 4: Uniformly sample 32 candidate words from the token dictionary. Construct 32 candidate adversarial captions by replacing the s-th token of cadvsubscriptadvc_advcroman_adv with these words, respectively. 5: Update the adversarial caption cadvsubscriptadvc_advcroman_adv with the one with the largest sim⁢(G′⁢(z|cadv),c)simsuperscript′conditionalsubscriptadvsim(G (z|c_adv),c)sim ( G′ ( z | croman_adv ) , c ). 6: end for 7: return cadvsubscriptadvc_advcroman_adv TABLE V: Success rates of the adversarial prompting attack (Algorithm 1) to our redaction method and the Rewriting baseline. Our redaction method is more robust to such attacks than Rewriting. Redaction prompts Method Attack Success Rate (↓)↓( )( ↓ ) long beak,white belly Rewriting 92.8%percent92.892.8\%92.8 % Ours (base) 50.3%percent50.350.3\%50.3 % blue / red wings Rewriting 97.4%percent97.497.4\%97.4 % Ours (base) 65.7%percent65.765.7\%65.7 % blue Rewriting 81.1%percent81.181.1\%81.1 % Ours (base) 35.5%percent35.535.5\%35.5 % yellow, red Rewriting 95.5%percent95.595.5\%95.5 % Ours (base) 59.9%percent59.959.9\%59.9 % VI-C Redacting Diffusion-based Text-to-Speech Models Setup. We use the pre-trained DiffWave model (kong2021diffwave, ) trained on the LJSpeech dataset (Ito2017ljspeech, ), which contains 13100 utterances from a female speaker reading books in home environment. The model is conditioned on Mel-spectrogram. We redact unseen voices from the disjoint LibriTTS dataset (zen2019libritts, ). We randomly choose five voices to redact: speakers 125, 1578, 1737, 1926 (female’s voice) and 1040 (men’s voice). The training set for each voice has total lengths between 4 and 6 minutes. Table VI includes the specific train-test splits of the LibriTTS voices. Note that for speaker 1040 there is only one chapter id, so we split based on the segment id shown in columns. TABLE VI: Specific train-test splits of the LibriTTS voices, and their total lengths measured in minutes. Redaction voices training test chapter id total length chapter id total length speaker 125 121124 5.89 121342 2.30 speaker 1578 140045, 140049 4.81 6379 1.30 speaker 1737 142397, 148989, 142396 3.75 146161 2.51 speaker 1926 147979, 147987 5.44 143879 1.98 speaker 1040 133433 (0-98) 4.65 133433 (100-168) 2.35 CycleGAN-VC2, Whisper, and Tortoise-TTS details. We train CycleGAN-VC2 (kaneko2019cyclegan, ) with the following code 666https://github.com/jackaduma/CycleGAN-VC2. The training data for CycleGAN-VC2 is the training data of a LibriTTS voice and the first 100 samples of LJ003 from LJSpeech 777These equals ∼1%similar-toabsentpercent1 1\%∼ 1 % of training utterances from LJSpeech (∼11similar-toabsent11 11∼ 11 minutes).. We train CycleGAN-VC2 for 1000 iterations with a batch size of 8. We use the medium-sized English-only Whisper model 888https://github.com/openai/whisper and the Tortoise-TTS model 999https://github.com/neonbjb/tortoise-tts. To sample from Tortoise-TTS we use two 10-second utterances from LJSpeech as the reference voice. Architecture and optimization. The architecture of the pre-trained model and other details are in Appendix C. The architecture of student conditioning networks with improved capacity is shown in Fig. 28. We use the Adam optimizer with a learning rate 0.0010.0010.0010.001 to optimize the ℓ1subscriptℓ1 _1ℓ1 loss. The redaction algorithm terminates at 80000 iterations. We use a batch size of 32. Our distillation algorithm is trained with the spectrogram data only. Configurations. We first use the uniform parallel distillation loss with λ=1.51.5λ=1.5λ = 1.5. We fix all up-sampling layers and denote it as the base configuration. We then use the spectrogram-rewriting module to improve capacity. Next, we improve voice cloning with Whisper and Tortoise-TTS when training CycleGAN-VC2. Finally, we investigate non-uniform distillation losses in Table I, where we set α=0.0010.001α=0.001α = 0.001 and β=0.010.01β=0.01β = 0.01 so that all wisubscriptw_iwitalic_i’s or λisubscript _iλitalic_i’s have the same order or magnitude. Evaluation metrics. To evaluate generation quality on the training voice ∖ΩsubscriptΩC _ C ∖ Croman_Ω, we compute the following two speech quality metrics on the test set of LJSpeech: Perceptual Evaluation of Speech Quality (PESQ) (PESQ2001, ) and Short-Time Objective Intelligibility (STOI) (taal2011algorithm, ). To evaluate redaction quality, we train a speaker classifier between redacted and training voices in each experiment. We extract Mel-frequency cepstral coefficients (xu2005hmm, ), spectral contrast (jiang2002music, ), and chroma features (ellis2007chroma, ) as sample features and train a support vector classifier. We then compute the recall rate of redacted voices after we perform redaction. In contrast to the standard classification, a lower recall rate means a higher fraction of redacted voices are projected to the training voice by the edited model, which indicates better redaction quality. See Appendix C-B for details of these metrics. TABLE VII: Results of generation and redaction quality for redacting the man speaker 1040 in LibriTTS. The λisubscript _iλitalic_i-order schedule in the non-uniform distillation losses leads to the best overall performance. The effects of each component within our method are displayed. Method Speech quality (LJSpeech) Recall (ΩsubscriptΩC_ Croman_Ω) (↓ ↓) PESQ (↑ ↑) STOI (↑ ↑) Pre-trained 3.333.333.333.33 97.8%percent97.897.8\%97.8 % - base 2.852.852.852.85 95.7%percent95.795.7\%95.7 % 52%percent5252\%52 % + improved capacity 3.033.033.033.03 96.6%percent96.696.6\%96.6 % 35%percent3535\%35 % + improved voice cloning 3.023.023.023.02 96.6%percent96.696.6\%96.6 % 35%percent3535\%35 % + non-uniform λisubscript _iλitalic_i-order 3.233.233.233.23 97.4%percent97.497.4\%97.4 % 40%percent4040\%40 % λisubscript _iλitalic_i-dilation 3.213.213.213.21 97.4%percent97.497.4\%97.4 % 50%percent5050\%50 % wisubscriptw_iwitalic_i-order 3.023.023.023.02 96.6%percent96.696.6\%96.6 % %percent2929\%29 % wisubscriptw_iwitalic_i-dilation 3.023.023.023.02 96.6%percent96.696.6\%96.6 % 30%percent3030\%30 % TABLE VIII: Results of generation and redaction quality for redacting several female speakers in LibriTTS. The improved capacity configuration leads to the best overall performance in most settings, with an exception for speaker 1926 where both configurations lead to similar performance. Redaction Method Speech quality (LJSpeech) Recall (ΩsubscriptΩC_ Croman_Ω) (↓ ↓) voices PESQ (↑ ↑) STOI (↑ ↑) Pre-trained 3.333.333.333.33 97.8%percent97.897.8\%97.8 % - speaker 125 base 3.143.143.143.14 97.0%percent97.097.0\%97.0 % %percent00\%0 % + improved capacity 3.273.273.273.27 97.4%percent97.497.4\%97.4 % 3%percent33\%3 % speaker 1578 base 2.142.142.142.14 94.4%percent94.494.4\%94.4 % %percent11\%1 % + improved capacity 3.243.243.243.24 97.4%percent97.497.4\%97.4 % 3%percent33\%3 % speaker 1737 base 2.492.492.492.49 94.9%percent94.994.9\%94.9 % %percent44\%4 % + improved capacity 3.243.243.243.24 97.2%percent97.297.2\%97.2 % 9%percent99\%9 % speaker 1926 base 3.063.063.063.06 96.3%percent96.396.3\%96.3 % %percent1616\%16 % + improved capacity 3.043.043.043.04 96.6%percent96.696.6\%96.6 % %percent1616\%16 % Results. The results for redacting speaker 1040 are shown in Table VII. With the base configuration we can redact a fraction of conditionals but the generation quality is much worse than the pre-trained model. By improving capacity both generation and redaction quality are improved. Improved voice cloning does not increase the quantitative metrics, but we find the generation quality is perceptually slightly better. The non-uniform distillation losses have a huge impact on the results. The λisubscript _iλitalic_i-order and λisubscript _iλitalic_i-dilation schedules can boost generation quality by a large gap without compensating redaction quality too much. The wisubscriptw_iwitalic_i-order and wisubscriptw_iwitalic_i-dilation schedules can improve redaction quality while keeping the generation quality. As high generation quality is very important for speech synthesis (on non-redacted voices), the λisubscript _iλitalic_i-order schedule leads to the best overall performance. The results for redacting other speakers are shown in Table VIII. In most settings the improved capacity configuration leads to much better generation quality than the base configuration with very little compensation for redaction quality, except for speaker 1926 where results are similar. Computation. On a single NVIDIA 3080 GPU, it takes less than 60 minutes to distill with the base configuration, and around 100 minutes with the other configurations. It takes around 2 hours to train the CycleGAN-VC2 model. As a comparison, DiffWave takes days to train on 8 GPUs. Demo. We include audio samples in our demo website: https://dataredact2023.github.io/. VII Conclusion and Discussion In this paper, we introduce a formal statistical machine learning framework for redacting data from conditional generative models, and present a computationally efficient method that only involves the conditioning networks. We introduce explicit formula for simple models, and propose distillation-based methods for practical conditional models. Empirically, our method performs well for practical text-to-image/speech models. It is computationally efficient, and can effectively redact certain conditionals while retaining high generation quality. For redacting prompts in text-to-image models, our method redacts better and is considerably more robust than the baseline methods. For redacting voices in text-to-speech models, our method can redact both similar and different voices while retaining high speech quality and intelligibility. In the following we include discussion on guaranteed safety, adversarial robustness, limitations of our method, and future work. Guaranteed Safety and Fine-tuning We first note that complete redaction to zero probability mass may be impossible for generative models with infinite support (as most deep generative models are). Take the unconditional normalizing flow as an example. We have the following proposition: Proposition 1. Let an invertible and smooth function F:ℝd→ℝdnormal-:normal-→superscriptℝsuperscriptℝF:R^d ^dF : blackboard_Rd → blackboard_Rd be an unconditional normalizing flow on the data space ℝdsuperscriptℝR^dblackboard_Rd that converts a standard Gaussian NN to the output distribution F#⁢subscriptnormal-#F_\#NF# N. For any set XX that has non-zero measure on ℝdsuperscriptℝR^dblackboard_Rd, the probability mass on XX, (F#⁢)⁢()subscriptnormal-#(F_\#N)(X)( F# N ) ( X ), is positive. Proof. (F#⁢)⁢()subscript# (F_\#N)(X)( F# N ) ( X ) =∫x∈(F#⁢)⁢(x)⁢xabsentsubscriptsubscript#differential-d = _x (F_\#N)(x)dx= ∫x ∈ X ( F# N ) ( x ) d x =∫z∈F−1⁢()⁢(z)⁢zabsentsubscriptsuperscript1differential-d = _z∈ F^-1(X)N(z)dz= ∫z ∈ F- 1 ( X ) N ( z ) d z =⁢(F−1⁢()).absentsuperscript1 =N(F^-1(X)).= N ( F- 1 ( X ) ) . Because XX has positive measure and F is invertible and smooth, F−1⁢()superscript1F^-1(X)F- 1 ( X ) has positive measure. Because NN is positive, ⁢(F−1⁢())>0superscript10N(F^-1(X))>0N ( F- 1 ( X ) ) > 0, and therefore (F#⁢)⁢()>0subscript#0(F_\#N)(X)>0( F# N ) ( X ) > 0. ∎ Furthermore, (pham2023circumventing, ) discovered that fine-tuning on a set of in appropriate samples can break many mitigation methods for text-to-image models. We believe there is an impossibility result – if the adversary have access to a dataset of inappropriate samples and fine-tune on it, there is nothing a learner can do to prevent this. In the previous normalizing flow example, the adversary can optimize the following objective: arg⁡maxF⁡x∼⁢log⁡[(F#⁢)⁢(x)]=arg⁡maxF⁡z∼F−1⁢()⁢log⁡[⁢(z)/|det⁢∇zF⁢(z)|]=arg⁡minF⁡z∼F−1⁢()⁢(‖z‖22/2+log⁡|det⁢∇zF⁢(z)|)subscriptsubscriptsimilar-tosubscript#absentsubscriptsubscriptsimilar-tosuperscript1detsubscript∇absentsubscriptsubscriptsimilar-tosuperscript1superscriptsubscriptnorm222detsubscript∇ array[]l _FE_x [(F_\#% N)(x)]\\ = _FE_z F^-1(X) [N(z)/|% det _zF(z)|]\\ = _FE_z F^-1(X)(\|z\|_2^2/2+ |% det _zF(z)|) arraystart_ARRAY start_ROW start_CELL arg maxitalic_F blackboard_Ex ∼ X log [ ( F# N ) ( x ) ] end_CELL end_ROW start_ROW start_CELL = arg maxitalic_F blackboard_Ez ∼ F- 1 ( X ) log [ N ( z ) / | det ∇z F ( z ) | ] end_CELL end_ROW start_ROW start_CELL = arg minitalic_F blackboard_Ez ∼ F- 1 ( X ) ( ∥ z ∥22 / 2 + log | det ∇z F ( z ) | ) end_CELL end_ROW end_ARRAY and as a result having more probability mass on XX. Despite this impossibility result, our method has largely increased the barrier for users to generate undesirable contents. For example, if the adversary do not have enough data of a certain celebrity’s voice they are then not able to reverse engineering the network by fine-tuning on those data. In practice, it is usually necessary to combine different security mechanisms to ensure safety of generation. Adversarial Robustness We have shown our method is less susceptible to be attacked by an existing adversarial prompting method than the baseline method in the text-to-image experiments. However, we would like to note that a formal definition of adversarial robustness in conditional generative models (e.g. text-to-X) is a largely open problem. We think many different threat models can be defined depending on the setting and adversary’s goal, capabilities, and knowledge of the model, which is outside the scope of this paper. Limitations There are certain types of neural networks that our method cannot be easily adapted to, especially when the conditional network is not completely independent from the main generative network. Examples include StyleGAN (karras2020analyzing, ), Transformer-based architectures with complex cross-attention layers, or multi-modal networks that mix input tokens from different modalities at the beginning. Future work One important future direction is to further improve robustness against adversarial attacks. Another line of future work is to apply the proposed method to Transformer-based architectures, where the conditioning networks are based on cross-attention blocks. A third direction is to extend our method to the online setting where redacted samples come in a stream. To achieve this, we need to modify the loss in (4) by using a weighted sampling strategy that assigns higher probability to newly seen samples. Acknowledgements This work was supported by NSF under CNS 1804829 and ARO MURI W911NF2110317. References (1) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” 2021. (2) A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever, “Zero-shot text-to-image generation,” in International Conference on Machine Learning. PMLR, 2021, p. 8821–8831. (3) A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, 2022. (4) A. Sauer, T. Karras, S. Laine, A. Geiger, and T. Aila, “Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis,” arXiv preprint arXiv:2301.09515, 2023. (5) Z. Kong, W. Ping, J. Huang, K. Zhao, and B. Catanzaro, “Diffwave: A versatile diffusion model for audio synthesis,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=a-xFK8Ymz5J (6) S.-g. Lee, W. Ping, B. Ginsburg, B. Catanzaro, and S. Yoon, “Bigvgan: A universal neural vocoder with large-scale training,” in International Conference on Learning Representations, 2023. (7) OpenAI, “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023. (8) H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023. (9) A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,” arXiv preprint arXiv:2112.10741, 2021. (10) A. Birhane, V. U. Prabhu, and E. Kahembwe, “Multimodal datasets: misogyny, pornography, and malignant stereotypes,” arXiv preprint arXiv:2110.01963, 2021. (11) C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman et al., “Laion-5b: An open large-scale dataset for training next generation image-text models,” arXiv preprint arXiv:2210.08402, 2022. (12) J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tramèr, “Red-teaming the stable diffusion safety filter,” arXiv preprint arXiv:2210.04610, 2022. (13) P. Bedapudi. (2022) Nudenet: Neural nets for nudity detection and censoring. [Online]. Available: https://github.com/notAI-tech/NudeNet (14) G. Laborde, “Deep n for nsfw detection,” 2022. [Online]. Available: https://github.com/GantMan/nsfw_model (15) J. Betker. (2022, 4) TorToiSe text-to-speech. [Online]. Available: https://github.com/neonbjb/tortoise-tts (16) C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li et al., “Neural codec language models are zero-shot text to speech synthesizers,” arXiv preprint arXiv:2301.02111, 2023. (17) Z. Zhang, L. Zhou, C. Wang, S. Chen, Y. Wu, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li et al., “Speak foreign languages with your own voice: Cross-lingual neural codec language modeling,” arXiv preprint arXiv:2303.03926, 2023. (18) G. K. Pitsilis, H. Ramampiaro, and H. Langseth, “Detecting offensive language in tweets using deep learning,” arXiv preprint arXiv:1801.04433, 2018. (19) E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing nlp,” arXiv preprint arXiv:1908.07125, 2019. (20) K. McGuffie and A. Newhouse, “The radicalization risks of gpt-3 and advanced neural language models,” arXiv preprint arXiv:2009.06807, 2020. (21) S. Gehman, S. Gururangan, M. Sap, Y. Choi, and N. A. Smith, “Realtoxicityprompts: Evaluating neural toxic degeneration in language models,” arXiv preprint arXiv:2009.11462, 2020. (22) A. Abid, M. Farooqi, and J. Zou, “Persistent anti-muslim bias in large language models,” in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, 2021, p. 298–306. (23) E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving, “Red teaming language models with language models,” arXiv preprint arXiv:2202.03286, 2022. (24) P. Schramowski, C. Turan, N. Andersen, C. A. Rothkopf, and K. Kersting, “Large pre-trained language models contain human-like biases of what is right and wrong to do,” Nature Machine Intelligence, vol. 4, no. 3, p. 258–268, 2022. (25) P. Schramowski, M. Brack, B. Deiseroth, and K. Kersting, “Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models,” arXiv preprint arXiv:2211.05105, 2022. (26) Z. Kong and K. Chaudhuri, “Data redaction from pre-trained gans,” in First IEEE Conference on Secure and Trustworthy Machine Learning, 2023. (27) M. Zhu, P. Pan, W. Chen, and Y. Yang, “Dm-gan: Dynamic memory generative adversarial networks for text-to-image synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, p. 5802–5810. (28) D. Bau, S. Liu, T. Wang, J.-Y. Zhu, and A. Torralba, “Rewriting a deep generative model,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I 16. Springer, 2020, p. 351–369. (29) Y. Cao and J. Yang, “Towards making systems forget with machine unlearning,” in 2015 IEEE Symposium on Security and Privacy. IEEE, 2015, p. 463–480. (30) C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified data removal from machine learning models,” arXiv preprint arXiv:1911.03030, 2019. (31) S. Schelter, “Amnesia - machine learning models that can forget user data very fast.” in CIDR, 2020. (32) S. Neel, A. Roth, and S. Sharifi-Malvajerdi, “Descent-to-delete: Gradient-based methods for machine unlearning,” in Algorithmic Learning Theory. PMLR, 2021, p. 931–962. (33) A. Sekhari, J. Acharya, G. Kamath, and A. T. Suresh, “Remember what you want to forget: Algorithms for machine unlearning,” Advances in Neural Information Processing Systems, vol. 34, 2021. (34) Z. Izzo, M. A. Smart, K. Chaudhuri, and J. Zou, “Approximate data deletion from machine learning models,” in International Conference on Artificial Intelligence and Statistics. PMLR, 2021, p. 2008–2016. (35) E. Ullah, T. Mai, A. Rao, R. A. Rossi, and R. Arora, “Machine unlearning via algorithmic stability,” in Conference on Learning Theory. PMLR, 2021, p. 4126–4142. (36) L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, and N. Papernot, “Machine unlearning,” in 2021 IEEE Symposium on Security and Privacy (SP). IEEE, 2021, p. 141–159. (37) A. Warnecke, L. Pirch, C. Wressnegger, and K. Rieck, “Machine unlearning of features and labels,” arXiv preprint arXiv:2108.11577, 2021. (38) Z. Kong and S. Alfeld, “Approximate data deletion in generative models,” arXiv preprint arXiv:2206.14439, 2022. (39) I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” Advances in neural information processing systems, vol. 27, 2014. (40) D. Bau, H. Strobelt, W. Peebles, J. Wulff, B. Zhou, J.-Y. Zhu, and A. Torralba, “Semantic photo manipulation with a generative image prior,” arXiv preprint arXiv:2005.07727, 2020. (41) J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in Neural Information Processing Systems, vol. 33, p. 6840–6851, 2020. (42) O. Bar-Tal, D. Ofri-Amar, R. Fridman, Y. Kasten, and T. Dekel, “Text2live: Text-driven layered image and video editing,” in Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XV. Springer, 2022, p. 707–723. (43) A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or, “Prompt-to-prompt image editing with cross attention control,” arXiv preprint arXiv:2208.01626, 2022. (44) B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, I. Mosseri, and M. Irani, “Imagic: Text-based real image editing with diffusion models,” arXiv preprint arXiv:2210.09276, 2022. (45) D. Valevski, M. Kalman, Y. Matias, and Y. Leviathan, “Unitune: Text-driven image editing by fine tuning an image generation model on a single image,” arXiv preprint arXiv:2210.09477, 2022. (46) M. Brack, P. Schramowski, F. Friedrich, D. Hintersdorf, and K. Kersting, “The stable artist: Steering semantics in diffusion latent space,” arXiv preprint arXiv:2212.06013, 2022. (47) S. Asokan and C. Seelamantula, “Teaching a gan what not to learn,” Advances in Neural Information Processing Systems, vol. 33, p. 3964–3975, 2020. (48) A. Sinha, K. Ayush, J. Song, B. Uzkent, H. Jin, and S. Ermon, “Negative data augmentation,” in International Conference on Learning Representations, 2021. (49) A. Cherepkov, A. Voynov, and A. Babenko, “Navigating the gan parameter space for semantic image editing,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, p. 3671–3680. (50) S. Malnick, S. Avidan, and O. Fried, “Taming a generative model,” arXiv preprint arXiv:2211.16488, 2022. (51) S. Moon, S. Cho, and D. Kim, “Feature unlearning for generative models via implicit feedback,” arXiv preprint arXiv:2303.05699, 2023. (52) R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau, “Erasing concepts from diffusion models,” arXiv preprint arXiv:2303.07345, 2023. (53) R. Gandikota, H. Orgad, Y. Belinkov, J. Materzyńska, and D. Bau, “Unified concept editing in diffusion models,” arXiv preprint arXiv:2308.14761, 2023. (54) E. Zhang, K. Wang, X. Xu, Z. Wang, and H. Shi, “Forget-me-not: Learning to forget in text-to-image diffusion models,” arXiv preprint arXiv:2303.17591, 2023. (55) A. Heng and H. Soh, “Selective amnesia: A continual learning approach to forgetting in deep generative models,” arXiv preprint arXiv:2305.10120, 2023. (56) N. Kumari, B. Zhang, S.-Y. Wang, E. Shechtman, R. Zhang, and J.-Y. Zhu, “Ablating concepts in text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, p. 22 691–22 702. (57) J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022. (58) M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014. (59) P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona, “Caltech-ucsd birds 200,” Caltech, Tech. Rep. CNS-TR-201, 2010. [Online]. Available: http://w.vision.caltech.edu/visipedia/CUB-200.html (60) S. Reed, Z. Akata, H. Lee, and B. Schiele, “Learning deep representations of fine-grained visual descriptions,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, p. 49–58. (61) T. Xu, P. Zhang, Q. Huang, H. Zhang, Z. Gan, X. Huang, and X. He, “Attngan: Fine-grained text to image generation with attentional generative adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, p. 1316–1324. (62) T. Kaneko, H. Kameoka, K. Tanaka, and N. Hojo, “Cyclegan-vc2: Improved cyclegan-based non-parallel voice conversion,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, p. 6820–6824. (63) A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” arXiv preprint arXiv:2212.04356, 2022. (64) Y. LeCun, C. Cortes, and C. Burges, “Mnist handwritten digit database,” ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, vol. 2, 2010. (65) D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. (66) T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” Advances in neural information processing systems, vol. 29, 2016. (67) C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, p. 2818–2826. (68) M. Andriushchenko, F. Croce, N. Flammarion, and M. Hein, “Square attack: a query-efficient black-box adversarial attack via random search,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIII. Springer, 2020, p. 484–501. (69) N. Maus, P. Chao, E. Wong, and J. Gardner, “Adversarial prompting for black box foundation models,” arXiv preprint arXiv:2302.04237, 2023. (70) K. Ito, “The LJ speech dataset,” 2017. (71) H. Zen, V. Dang, R. Clark, Y. Zhang, R. J. Weiss, Y. Jia, Z. Chen, and Y. Wu, “Libritts: A corpus derived from librispeech for text-to-speech,” arXiv preprint arXiv:1904.02882, 2019. (72) I.-T. Recommendation, “Perceptual evaluation of speech quality (pesq): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs,” Rec. ITU-T P. 862, 2001. (73) C. H. Taal et al., “An algorithm for intelligibility prediction of time–frequency weighted noisy speech,” IEEE Transactions on Audio, Speech, and Language Processing, 2011. (74) M. Xu, L.-Y. Duan, J. Cai, L.-T. Chia, C. Xu, and Q. Tian, “Hmm-based audio keyword generation,” in Advances in Multimedia Information Processing-PCM 2004: 5th Pacific Rim Conference on Multimedia, Tokyo, Japan, November 30-December 3, 2004. Proceedings, Part I 5. Springer, 2005, p. 566–574. (75) D.-N. Jiang, L. Lu, H.-J. Zhang, J.-H. Tao, and L.-H. Cai, “Music type classification by spectral contrast feature,” in Proceedings. IEEE International Conference on Multimedia and Expo, vol. 1. IEEE, 2002, p. 113–116. (76) D. Ellis, “Chroma feature analysis and synthesis,” Resources of laboratory for the recognition and organization of speech and audio-LabROSA, vol. 5, 2007. (77) M. Pham, K. O. Marshall, and C. Hegde, “Circumventing concept erasure methods for text-to-image generative models,” arXiv preprint arXiv:2308.01508, 2023. (78) T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, p. 8110–8119. Appendix A Recovering Original Score in (gandikota2023erasing, ) Let ϵθ*⁢(xt,c,t)subscriptitalic-ϵsuperscriptsubscript _θ^*(x_t,c,t)ϵitalic_θ* ( xitalic_t , c , t ) be the original score and ϵθ⁢(xt,c,t)=ϵθ*⁢(xt,t)−η⁢(ϵθ*⁢(xt,c,t)−ϵθ*⁢(xt,t))subscriptitalic-ϵsubscriptsubscriptitalic-ϵsuperscriptsubscriptsubscriptitalic-ϵsuperscriptsubscriptsubscriptitalic-ϵsuperscriptsubscript _θ(x_t,c,t)= _θ^*(x_t,t)-η( _% θ^*(x_t,c,t)- _θ^*(x_t,t))ϵitalic_θ ( xitalic_t , c , t ) = ϵitalic_θ* ( xitalic_t , t ) - η ( ϵitalic_θ* ( xitalic_t , c , t ) - ϵitalic_θ* ( xitalic_t , t ) ) be the distilled score. We could recover the original score from the distilled score in the following way. First, by letting c=∅c= = ∅, we have ϵθ⁢(xt,t)=ϵθ*⁢(xt,t).subscriptitalic-ϵsubscriptsubscriptitalic-ϵsuperscriptsubscript _θ(x_t,t)= _θ^*(x_t,t).ϵitalic_θ ( xitalic_t , t ) = ϵitalic_θ* ( xitalic_t , t ) . Inserting this to the right-hand-side of the definition of distilled score, one could get ϵθ⁢(xt,c,t)=ϵθ⁢(xt,t)−η⁢(ϵθ*⁢(xt,c,t)−ϵθ⁢(xt,t)).subscriptitalic-ϵsubscriptsubscriptitalic-ϵsubscriptsubscriptitalic-ϵsuperscriptsubscriptsubscriptitalic-ϵsubscript _θ(x_t,c,t)= _θ(x_t,t)-η( _θ^% *(x_t,c,t)- _θ(x_t,t)).ϵitalic_θ ( xitalic_t , c , t ) = ϵitalic_θ ( xitalic_t , t ) - η ( ϵitalic_θ* ( xitalic_t , c , t ) - ϵitalic_θ ( xitalic_t , t ) ) . As a result, one can recover the original score as ϵθ*⁢(xt,c,t)=1η⁢((1+η)⁢ϵθ⁢(xt,t)−ϵθ⁢(xt,c,t)).subscriptitalic-ϵsuperscriptsubscript11subscriptitalic-ϵsubscriptsubscriptitalic-ϵsubscript _θ^*(x_t,c,t)= 1η((1+η) _θ(x_t% ,t)- _θ(x_t,c,t)).ϵitalic_θ* ( xitalic_t , c , t ) = divide start_ARG 1 end_ARG start_ARG η end_ARG ( ( 1 + η ) ϵitalic_θ ( xitalic_t , t ) - ϵitalic_θ ( xitalic_t , c , t ) ) . By using the original score for sampling one may be able to generate concepts that have been erased. Appendix B Additional Details and Experiments for Redaction from DM-GAN B-A Details of the Pre-trained Model and the Proposed Student Networks The high-level architecture of DM-GAN is shown in Fig. 3 and 4. The first conditioning network H1subscript1H_1H1 takes the sentence embedding vs⁢(c)subscriptv_s(c)vitalic_s ( c ) as input and outputs two vectors: a mean vector, and the square root of the variance vector. A re-parameterization similar to variational auto-encoders is applied to these two vectors, and the output is concatenated to the latent code. The other two conditioning networks H2subscript2H_2H2 and H3subscript3H_3H3, called the memory writing module, take two inputs: the word embeddings vw⁢(c)subscriptv_w(c)vitalic_w ( c ), and the image features of the previously generated low resolution images. The output of H2subscript2H_2H2 or H3subscript3H_3H3 then goes through the rest of the modules in the main generative network. We use the pre-trained model and code from https://github.com/MinfengZhu/DM-GAN under MIT license. The pre-trained model takes days to train on 1 or more GPUs. Figure 3: High-level architecture of DM-GAN. Figure 4: High-level architecture of original and higher-capacity conditioning networks of DM-GAN. B-B Visualization In Fig. 5 - Fig. 8 , we visualize examples where we redact prompts that contain long beak or white belly. In Fig. 9 - Fig. 12 , we visualize examples where we redact prompts that contain blue wings or red wings. In Fig. 13 - Fig. 16 , we visualize examples where we redact prompts that contain blue. In Fig. 17 - Fig. 20 , we visualize examples where we redact prompts that contain yellow or red. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 5: Redacted prompt: ‘this particular bird has a white belly and breasts and black head and back’. Reference prompt: ‘this particular bird has a black belly and breasts and black head and back’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 6: Redacted prompt: ‘this bird has feathers that are black and has a white belly’. Reference prompt: ‘this bird has feathers that are black and has a black belly’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 7: Redacted prompt: ‘a small bird with an orange throat and long beak’. Reference prompt: ‘a small bird with an orange throat and short beak’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 8: Redacted prompt: ‘the black and white bird has a sharp long beak’. Reference prompt: ‘the black and white bird has a sharp short beak’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 9: Redacted prompt: ‘this bird has wings that are blue and has black feet’. Reference prompt: ‘this bird has wings that are white and has black feet’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 10: Redacted prompt: ‘this is a grey bird with blue wings and a pointy beak’. Reference prompt: ‘this is a grey bird with white wings and a pointy beak’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 11: Redacted prompt: ‘this bird has wings that are red and has a white belly’. Reference prompt: ‘this bird has wings that are white and has a white belly’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 12: Redacted prompt: ‘this bird has wings that are red and has a yellow belly’. Reference prompt: ‘this bird has wings that are white and has a yellow belly’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 13: Redacted prompt: ‘this bird has wings that are blue and has black feet’. Reference prompt: ‘this bird has wings that are red and has black feet’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 14: Redacted prompt: ‘this bird has small wings and blue grey nape’. Reference prompt: ‘this bird has small wings and red grey nape’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 15: Redacted prompt: ‘the bird is blue with gray wins and tail’. Reference prompt: ‘the bird is red with gray wins and tail’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 16: Redacted prompt: ‘this bird has wings that are blue and has a white belly’. Reference prompt: ‘this bird has wings that are red and has a white belly’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 17: Redacted prompt: ‘this is a red bird with a white belly and a large beak’. Reference prompt: ‘this is a black bird with a white belly and a large beak’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 18: Redacted prompt: ‘a bird with thick short beak red crown red breast that fades into a pink and white belly and red coverts’. Reference prompt: ‘a bird with thick short beak black crown black breast that fades into a pink and white belly and black coverts’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 19: Redacted prompt: ‘this yellow breasted bird has a dark gray head and chest a thin beak and a long tail’. Reference prompt: ‘this black breasted bird has a dark gray head and chest a thin beak and a long tail’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Reference G(⋅|c^)G(·| c)G ( ⋅ | over start_ARG c end_ARG ) (c) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (d) Rewriting Baseline Figure 20: Redacted prompt: ‘the yellow and black bird has a white belly’. Reference prompt: ‘the black and black bird has a white belly’. B-C Adversarial Prompting Attack (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 21: Redacted prompt: ‘a short yellow beak on an otherwise black bird bright irises around black pupils’. Adversarial prompt: ‘a short yellow wingtip on yhellow otherwise wa bird bright irises around dramatic pupils’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 22: Redacted prompt: ‘this is a colorful bird boasting a coral red color strongest at head and fading towards tail’. Adversarial prompt: ‘wingtip is a colorful bird boasting the coral red color strongest at primary and fading towards tail’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 23: Redacted prompt: ‘this small bird has a light yellow belly and a dark yellow head with black wings and tail’. Adversarial prompt: ‘this small bird the the light yellow belly and a dark yellow head with black remaining and mahogany’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 24: Redacted prompt: ‘this bird is golden yellow with a black head’. Adversarial prompt: ‘through mahogany wingtip golden yellow bed a black head’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 25: Redacted prompt: ‘this bird has a golden yellow body a black head a pink bill that curves downward’. Adversarial prompt: ‘mahogany the has blacker golden yellow basically a black head a primary bill that through hughed’. (a) Pre-trained G(⋅|c)G(·|c)G ( ⋅ | c ) (b) Our Redaction G′(⋅|c)G (·|c)G′ ( ⋅ | c ) (c) Adversarial G′(⋅|cadv)G (·|c_adv)G′ ( ⋅ | croman_adv ) Figure 26: Redacted prompt: ‘this bird has a black beak a black crown and a belly that is golden yellow’. Adversarial prompt: ‘mahogany love has a riangular love bed black crown and a dramatic hjas breadt golden yellow’. Appendix C Additional Details and Experiments for Redaction from DiffWave C-A Details of the Pre-trained Model and the Proposed Student Networks The high-level architecture of DiffWave is shown in Fig. 27. We select the base (64 channels) version of the model. The model is conditioned on 80-band Mel-spectrogram with FFT size=1024absent1024=1024= 1024, hop size=256absent256=256= 256, and window size=1024absent1024=1024= 1024. Each conditioning network has two up-sampling layers that up-sample the spectrogram, and a one-dimensional convolution layer that maps the number of channels to 128. We use the pre-trained model and code from https://github.com/philsyn/DiffWave-Vocoder under MIT license, which is trained on all LJSpeech samples except for LJ001 and LJ002, which is used as the test set. The pre-trained model takes days to train on 8 GPUs. Figure 27: High-level architecture of DiffWave. For the additional layers in the improved capacity configuration, all convolutions are one-dimensional with kernel size =1absent1=1= 1. htranssubscriptℎtransh_transhroman_trans includes two convolutions that keep the channels (=80absent80=80= 80) and a leaky ReLU activation with negative slope =0.4absent0.4=0.4= 0.4 between. hgatesubscriptℎgateh_gatehroman_gate includes one zero-initialized convolution that changes channels from 80 to 128 followed by a sigmoid activation. The architecture of student conditioning networks with improved capacity is shown in Fig. 28. Figure 28: High-level architecture of original and higher-capacity conditioning networks of DiffWave. C-B Evaluation Metrics The metrics for speech quality are as follows. 1. Perceptual Evaluation of Speech Quality (PESQ2001, ), or PESQ, measures the quality of generated speech. It ranges between -0.5 and 4.5 and is higher for better quality. 2. Short-Time Objective Intelligibility (taal2011algorithm, ), or STOI, measures the intelligibility of generated speech. It ranges between 0% and 100% and is higher for better intelligibility. The voice classifier is trained and tested on audio clips with 0.7256 second. For each audio clip, we extract 20-dimensional Mel-frequency cepstral coefficients (xu2005hmm, ), 7-dimensional spectral contrast (jiang2002music, ), and 12-dimensional chroma features (ellis2007chroma, ). The classifier is a support vector classifier with the radial basis function kernel with regularization coefficient =1absent1=1= 1.

Instant research discovery