Paper deep dive
Controlling Language and Diffusion Models by Transporting Activations
Pau Rodriguez, Arno Blaas, Michal Klein, Luca Zappella, Nicholas Apostoloff, Marco Cuturi, Xavier Suau
Models: FLUX, Gemma2-2B, Llama3-8B, SDXL
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/12/2026, 7:33:24 PM
Summary
The paper introduces Activation Transport (AcT), a framework for steering generative model activations using optimal transport theory. AcT provides a modality-agnostic, computationally efficient method for controlling LLMs and T2I diffusion models, enabling tasks like toxicity mitigation, concept induction, and style control while preserving internal activation distributions.
Entities (5)
Relation Signals (3)
Linear-ACT → isa → Activation Transport
confidence 99% · Linear-ACT, an inference-time intervention based on OT
Activation Transport → controls → Large Language Models
confidence 95% · For LLMs, we show that AcT can effectively mitigate toxicity
Activation Transport → controls → Text-to-Image Diffusion Models
confidence 95% · In T2Is, we show how AcT enables fine-grained style control
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The increasing capabilities of large generative models and their ever more widespread deployment have raised concerns about their reliability, safety, and potential misuse. To address these issues, recent works have proposed to control model generation by steering model activations in order to effectively induce or prevent the emergence of concepts or behaviors in the generated output. In this paper we introduce Activation Transport (AcT), a general framework to steer activations guided by optimal transport theory that generalizes many previous activation-steering works. AcT is modality-agnostic and provides fine-grained control over the model behavior with negligible computational overhead, while minimally impacting model abilities. We experimentally show the effectiveness and versatility of our approach by addressing key challenges in large language models (LLMs) and text-to-image diffusion models (T2Is). For LLMs, we show that AcT can effectively mitigate toxicity, induce arbitrary concepts, and increase their truthfulness. In T2Is, we show how AcT enables fine-grained style control and concept negation.
Tags
Links
- Source: https://arxiv.org/abs/2410.23054
- Canonical: https://arxiv.org/abs/2410.23054
Trouble viewing inline? Open PDF directly →
Full Text
100,202 characters extracted from source content.
Expand or collapse full text
CONTROLLINGLANGUAGE ANDDIFFUSIONMODELS BYTRANSPORTINGACTIVATIONS Pau Rodr ́ ıguezArno BlaasMichal KleinLuca ZappellaNicholas Apostoloff Marco CuturiXavier Suau pau.rodriguez,ablaas,michal klein,lzappella,napostoloff, mcuturi,xsuaucuadros@apple.com Apple ABSTRACT The increasing capabilities of large generative models and their ever more widespread deployment have raised concerns about their reliability, safety, and potential misuse. To address these issues, recent works have proposed to con- trol model generation by steering model activations in order to effectively induce or prevent the emergence of concepts or behaviors in the generated output. In this paper we introduce Activation Transport (ACT), a general framework to steer activations guided by optimal transport theory that generalizes many previous activation-steering works. ACT is modality-agnostic and provides fine-grained control over the model behavior with negligible computational overhead, while minimally impacting model abilities. We experimentally show the effectiveness and versatility of our approach by addressing key challenges in large language models (LLMs) and text-to-image diffusion models (T2Is). For LLMs, we show that ACT can effectively mitigate toxicity, induce arbitrary concepts, and increase their truthfulness. In T2Is, we show how ACT enables fine-grained style control and concept negation. 0 Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone col- lecting mushrooms for food to survive on. 0.5 Once upon a time, there was an amazing woman named Sarah. She had the most beautiful smile and kindest heart you could ever imagine! Sarah loved to play soccer with her friends on Saturday mornings at 9am sharp every week. 1 Once upon a time, the only way to watch football was on TV. The game of soccer had been played in England since 1863 and by the early twentieth century it became one of Britain’s most popular sports. football strength (λ) λ= 0 λ= 0.5λ= 0.5λ= 0.5λ= 0.5λ= 0.5λ= 0.5 λ= 1λ= 1λ= 1λ= 1λ= 1λ= 1 art nouveau watercolor cyberpunk sketchanime impressionism Figure 1:Linear-ACT unlocks interpretable controllability for both LLMs and Diffusion, of- fering explicit control over the strength of conditioning, via a parameterλbetween 0 (no transport) and 1 (full transport). 1INTRODUCTION Pre-trained Generative Models (GMs) typically undergo an additional fine-tuning phase to better align them to a desired behavior. For example, Large Language Models (LLMs) are aligned via 1 arXiv:2410.23054v2 [cs.LG] 22 Nov 2024 instruction fine-tuning (Wei et al.) or RLHF (Ouyang et al., 2022). Although less extensively, these strategies have also been applied to Text-to-Image (T2I) models (Wallace et al., 2024; Yang et al., 2024). However, as the number of parameters grows, alignment approaches can become challenging from a computational and memory perspective (Houlsby et al., 2019). In addition, these strategies modify the model’s internal mechanisms, realigning its parameters by leveraging new data, which can have the undesired side effect of impacting the utility of the model on other metrics (Kotha et al., 2024; Luo et al., 2023), such as 0-shot evaluation or question-answering. The increasing cost of fine-tuning has motivated research in inference-time interventions on pre- trained models that offer a better understanding of features (Geiger et al., 2024) or to control specific behaviors (Suau et al., 2022; Rimsky et al., 2023; Zou et al., 2023; Li et al., 2024). Since these mod- ifications are typically sparse and/or low-dimensional, they can be estimated using a few hundreds of sentences (Suau et al., 2024; Turner et al., 2023). For example, Rimsky et al. (2023); Li et al. (2024) shift activations by a constant vector estimated with sets of desired and undesired data (e.g., non-toxic and toxic); or Suau et al. (2024) mitigate toxicity by dampening the activations of expert neurons. While effective, existing methods do not preserve the activation distribution observed by the model during training. Considering how brittle GMs can be (Huu-Tien et al., 2024; Sclar et al., 2024), a constant shift can move activations out-of-distribution (OOD), which can lead to unwanted behaviors, and hinder both the conditioning and the general model performance. We propose Ac tivation Transport (ACT), a framework to steer activations according to the optimal transport (OT) map between two different (source and target) activation distributions,e.g.,toxic to non-toxic language, or between two different styles in T2I generation. ACT applies a set of univari- ate maps on activations while preserving their target distributions, achieving better controllability and robustness to the choice of model and layers intervened upon. Our main contributions are: • A unifying interpretation of existing activation steering methods under the umbrella of OT, show- ing that most existing methods are equivalent to a mean transport map (Section 3.3). • Linear-ACT, an inference-time intervention 1 based on OT that preserves internal activation dis- tributions (Section 3.1). The degree of intervention can be controlled by a strength parameter λbetween 0 (no transport) and 1 (full transport), as shown in Figure 1. We also introduce the transport supportto prevent inducing OOD activations. • We show that, without any hyperparameter tuning, Linear-ACT matches or outperforms existing inference-time interventions when aiming to control LLMs for the tasks of toxicity mitigation, concept induction, and increasing truthfulness. • We find that off-the-shelf Linear-ACT is also effective at controlling T2I diffusion models for the tasks of fine-grained style control and concept negation. Additionally, we adapt (Li et al., 2024) (ITI) for T2I. To the best of our knowledge, this is the first work to apply an inference-time intervention method that is simultaneously effective on both LLMs and Diffusion Models. 2RELATEDWORK The growing capabilities and prevalence of GMs (Brown et al., 2020; Rombach et al., 2022), along with the rising costs of fine-tuning and alignment, have driven research into controllability of GMs. Controlling LLMs.ACTADD(Turner et al., 2023) uses a contrast prompt (one positive and one negative example) to construct a shift vector. CAA (Rimsky et al., 2023) builds on ACTADDby calculating the difference vectors for steering based on a dataset of contrast pairs (rather than a single pair), adding the mean difference during inference time for steering. ITI-C(Li et al., 2024) estimates the shift vector orthogonal to the hyperplane learnt by a binary linear classifier on two sets of sentences, showing an increase of truthfulness on the TruthfulQA benchmark (Lin et al., 2021). The same work proposes MassMean (ITI-M), with an additive vector computed as the difference in means for both sets of sentences. With a different approach, AURA by Suau et al. (2024) damp- ens activations proportionally to each neuron’s ability to classify toxic and non-toxic sentences, effectively mitigating toxicity. REPE by Zou et al. (2023) proposes to compute steering vectors at inference time based on prompt pairs. Wu et al. (2024) considers activations relationships using a low-rank projection to exchange information with a counterfactual representation and Geiger et al. (2024) consider rotations of subsets of features. Orthogonal to the works of activation steering, 1 https://github.com/apple/ml-act. 2 Dekoninck et al. (2023) have proposed a language model arithmetic that can combine the outputs of multiple models in a principled way to simulatenously control multiple concepts, however requiring several (costly) inference passes on the LLM. Controlling T2IFew works tackle aligment of T2I models. Wallace et al. (2024) align diffusion models with reinforcement learning (RL) on human comparison data. Yang et al. (2024) remove the need of a reward model to reduce computational overhead of RL. Other works focus on fine-tuning to maximize a reward function (Clark et al., 2023) or consistency to reference images (Lee et al., 2024). The literature on T2I diffusion model controllability is more extensive and it commonly consists in training structure adapters (Mou et al., 2024; Jiang et al., 2024), style adapters (Stracke et al., 2024; Ye et al., 2023; Zhao et al., 2024), or low-rank adapters (LoRAs) (Ruiz et al., 2023; YEH et al., 2024; Gandikota et al., 2023; Stracke et al., 2024). Closer to our work are inference- time interventions, which do not require backpropation through the model to train the conditioning mechanisms. Diffusion steering methods are a family of inference-time interventions, which directly modify the diffusion algorithm at test time for fine-grained control with additional prompts (Nair et al., 2023; Brack et al., 2022). To the best of our knowledge, our work is the first to explore inference-time interventions that are not specific to diffusion models and transfer across modalities. 3TRANSPORTINGNEURONACTIVATIONS We represent the activations of a GM given an input sentencex∈ Sas a tensorR M×L×K , where Mis the number of activations per layer (assumed constant w.l.o.g. for simplicity),Lthe number of layers, andKthe number of tokens decoded. We reduce each of theKvalues to only one using an arbitrary pooling operatorφ. From now on we writeZ:S →R M×L for the map that turns a sentence into a matrix of activations statistics, noting thatZincorporatesφ-pooling. We consider two probability distributions on sentencespandq. We view these sentences through the lens of their aggregated activation matrices,i.e.,we will examine probability distributionsμ:=Z♯p andν:=Z♯q, where we have used the pushforward operator♯. In practice, we have access to samplesx 1 ,...,x n ∼pandy 1 ,...,y n ∼q. For instance, in the case of toxicity mitigation,p coverstoxicsentences andqnon-toxicones. Input sentencesx i andy i go through the model to yield activation matricesa i :=Z(x i )andb i =Z(y i ), each seen as i.i.d. samples fromμandν respectively, resulting inn+nobservations ofM×Lmatrices. In that context, our goal is to learn a transport mapT:R M×L →R M×L from(a i ,b i )that approximately pushesμtoν,i.e.,T♯μ≈ν. 3.1LOWBUDGETESTIMATORS FORTRANSPORTMAPS Since a modern GM can have millions of activations, an ideal transport estimator forTmust be easy to learn, cheap to store in memory, and blazing fast to evaluate to avoid overheads at inference time. Additionally, because the estimation of OT maps is known to be plagued by the curse of dimensionality (Chewi et al., 2024, Chap. 2), notable care must be taken to have map estimates that generalize reasonably well. These issues are all compounded by the fact that our final method, as presented in §3.2 builds on a composition of such OT maps (i.e. maps for a layer are estimated on samples that are themselves obtained by using maps for a previous layer). For all these fundamental reasons, we work our way from very simple map estimators, and follow Suau et al. (2024) to focus on maps that factorizeindependentlyalong each dimension (each activation).Tis therefore described as a collection ofMLindependent univariate maps, where each map indexed bym,lshould ideally map the marginal distribution ofμin that coordinate to that ofν. Recall that: Proposition 3.1 (Univariate Transport Maps)(Santambrogio, 2015, Chap.2) Letρ,τ∈P(R)be two univariate distributions. For any submodular costc:R×R→R(i.e., such that∂c/∂x∂y <0), the optimal transport mapTthat can transportρtoτisT ⋆ =Q τ ◦F ρ , whereQ τ andF ρ are respectively the quantile function ofτand the cumulant density function (CDF) ofρ. Estimating and storing allMLtransport maps would therefore require dealing with as many quantile and CDF functions. Unfortunately, parameterizing each of these could quickly become intractable, which is why we scale down ambitions to simplify further our working hypothesis to only consider affinetransport maps. Each of theMLactivations we consider results in two families of reals: 3 p q Coord. axis Samples ActAdd ITI-c Mean-AcT Linear-AcT p q Figure 2: Transport maps using different methods. For distri- butions withσ a =σ b (left) all methods (except ACTADD) are equivalent. Whenσ a ̸=σ b (right), vector-based methods (e.g., ACTADD, ITI-C, Mean-ACT) diverge from the map defined by the samples. ACTADDshows a bias since it only uses one sample pair. The linear estimator is robust to differences inσ. Figure 3:Actualσ a ,σ b for toxic and non-toxic sentences on Gemma2-2B, showing that σ a ̸=σ b in real scenarios. source(a 1 mℓ ,...,a n mℓ )and targets(b 1 mℓ ,...,b n mℓ ). Simpifying notations, we drop mentions tom andℓto focus on valuesA:= (a 1 ,...,a n )andB:= (b 1 ,...,b n )each inR n . We propose to consider the simple proxy task of findingaffinemaps that pushAtoBefficiently. We present such an affine map, denoted Linear-ACT, in Definition 3.1. Despite its simplicity, we show in Section 3.3 that many state-of-the-art methods boil down to even simpler approximations and heuristics. Definition 3.1 (Linear-ACT)Given samplesA= (a 1 ,...,a n )andB= (b 1 ,...,b n )and a cost functionc:R×R→R, the Linear-ACTmap trained with these samples is defined as T(a;A,B) :=ωa+β, whereω,βare the minimizers ofmin ω,β P i c b (i) ,ωa (i) +β , and can be recovered in closed form whenc(a,b) := (a−b) 2 , as ω= P i ̃a (i) ̃ b (i) P i ( ̃ b (i) ) 2 , β=m b −ωm a , wherem a = 1 n P i a i ,m b = 1 n P i b i are mean values, and superscripted values with (i) refer to sorted values in increasing order,(a (1) ≤ · ≤a (n) )and(b (1) ≤ · ≤b (n) ). Additionally, ̃a (i) =a (i) −m a , ̃ b (i) =b (i) −m b are sorted and recentered observations. An important feature of Linear-ACT is that it can be composed with linear layers in the GM, re- sulting in no computational overhead at inference time (see Appendix A for details). Note that the expression in Linear-ACT shouldnotbe confused with the closed-form known when transporting a Gaussian density with parameter (m a ,σ a )towards a second (m b ,σ b ), which is known (Peyr ́ e & Cuturi, 2019, Remark 2.31) to beT(a) = σ b σ a a+(m b − σ b σ a m a ).Note that if one makes the additional assumption thatσ a =σ b , then the affine Gaussian map becomes a mean shift or translation, with T(a) =a+m b −m a . We call this very simple baseline Mean-ACT and show in Section 3.3 that several methods in the literature indeed propose versions of a mean shift strategy. Figure 2 showcases the effect of different maps on toy data (iid, Gaussian). Note that methods based on mean-shift (ACTADD, ITI-C, Mean-ACT) can strongly over or undershoot, mapping samples out-of-distribution. Linear-ACT shows a good trade-off between in distribution mapping and low computational budget. We note that activations in current GMs show mostly unimodal distributions, but have different standard deviations for different behaviors as shown in Figure 3, making the linear choice a suitable one. Note that multimodal distributions would result in non-linear transport maps, which are beyond the scope of this work. Transport SupportThe map in Definition 3.1 is estimated usingnpairs of samples. In practice, nis in the order of hundreds, which results in a rough approximation of the true transport fromμto 4 ν. It is fair to assume that the transport error will be higher for input samples in the tail ofμ, given the scarcity of samples in that range. Because transporting OOD samples may lead to unexpected behavior, and to be on the conservative side, we only transport new samples that are within the osberved supportQ o = [minA,maxA]. Using the support is important whenμisnarrowerthan ν(typically in a mitigation setup). Unless stated otherwise, we useQ o for concept mitigation and Q ∞ = (−∞,∞)for induction. Appendix E shows an empirical validation of this choice. 3.2SEQUENTIALITERATIVEMAPS While it might be possible to follow the template approach outlined in Section 3.1 to apply univariate maps to each of theMLactivations, this ignores the causal relationship across activations, where activations produced by a layer are processed by the next one,i.e.,a m,ℓ+1 =f ℓ (a m,ℓ ). Any intervention at the level of a layer must therefore be factored in accordingly before creating the intervention at the next one. To account for such causality, we estimate the transport maps for each layer incrementally: we first estimate the transport for the first layer (in the model graph), then we run inference again by applying the first layer map in order to estimate the map for the second layer, and so on until all maps are estimated. A similar approach is adopted in Zou et al. (2023), and detailed with our tools in Definition 3.2. In Appendix C we show that causal estimation achieves more effective conditioning than a simultaneous estimation. In this work, we use causal estimation for Mean-ACT and Linear-ACT. Definition 3.2 (Affine Causal Transport Map)Form≤Mandℓ≤L, letA m := (a 1 m,1 ,·,a n m,1 )andB m := (b 1 m,1 ,·,b n m,1 )denotenfamilies ofMactivations for the first layer. Starting withℓ= 1, and setting C m,1 :=A m,1 ,D m,1 :=B m,1 , compute and store the2M(ω m ,β m )parameters of allMtransport maps associated with these activations using Definition 3.1: ∀m≤M,∀ℓ≤L, T m,ℓ :=T(·;C m,ℓ ,D m,ℓ ) :R→R, where observationsCandDare refreshed recursively for each of their entriesm≤M, asℓis incremented, C ·,ℓ+1 :=f ℓ ([T m,ℓ (C m,ℓ )] m ), D ·,ℓ+1 :=f ℓ ([T m,ℓ (D m,ℓ )] m ). At inference time, given a sentencex, we run the recursion starting from the first activation vector a= (a m,1 ) m , looping for1≤ℓ≤Lasa←f ℓ ([T m,ℓ (a m )] m . Interpolation Between Measures Using TransportOne can easily extend a transport map from measureμtoνto one that is able to output an interpolating measure. The idea, outlined by McCann (1997), consists in defining the followingλ-parameterized map from any OT mapT, T(a,λ) = (1−λ)a+λT(a),(1) whereλ∈[0,1]andλ= 1recovers the full transport. Conditioning GMs through OT allows the user to precisely control the presence of a concept with a continuous and interpretableknob(λ) during generation, not requiring expensive parameter search (Li et al., 2024) or being limited by fixed, uncontrollable conditioning (Suau et al., 2024). In applications such as diffusion, where the utility of the model is harder to assess, our interpretable strength is of key importance, as shown in Section 5. Note that methods like ACTADD, CAA or ITI-Calso have a conditioning strength parameter. However, this parameter is applied as a multiplier of a conditioning bias asT(a,λ) = a+λβ(see Section 3.3), thus makingλunbounded, harder to interpret and not robust with respect to different models, layers, and tasks. 3.3GENERALIZATION OFPRIORINFERENCE-TIMEINTERVENTIONSWORK In this section, we show how many earlier works can be interpreted as special cases of Linear-ACT. Table 1 summarizes the intervention proposed by several recent methods, where we show that all methods propose a form of linear transport, and all of them (aside from Suau et al. (2022)) add a 5 Table 1: Comparison of different inference-time interventions in the literature. All methods listed can be expressed as a specific form of a linear map. With ACT, the conditioning strengthλinterpo- lated between the activationaand its transformed version (following Equation (1)), while existing methods useλas a bias multiplier, thus becoming less interpretable and less robust to model/layer changes. As a result, many methods require a grid-search to find the best layer to intervene upon. MethodTransportParametersSupportφ Det zero (Suau et al., 2022)ωa+βω= 0, β=m b Any layer,a|AP(A,B)> εmax ACTADD(Turner et al., 2023)ωa+λβω= 1, β=a + −a − Layer searchlast CAA (Rimsky et al., 2023)ωa+λβω= 1, β=m b −m a Layer searchlast RePE (Zou et al., 2023)ωa+λβω= 1, β=a + (x)−a − (x)Layer searchlast AURA (Suau et al., 2024)ωa+βω= 1−Gini(A,B), β= 0Any layer,a|AUROC(A,B)>0.5max EAST (Rahn et al., 2024)ωa+λβω= 1, β≈m b Layer searchlast ITI-M(Li et al., 2024)ωa+λβω= 1, β=m b −m a Attention head searchlast ITI-C(Li et al., 2024)ωa+λβω= 1, β=f CLS (A,B)Attention head searchlast Mean-ACT, Section 3.1(1−λ)a+λ(ωa+β)ω= 1, β=m b −m a Any layer,a∈Q o orQ ∞ mean Linear-ACT, Definition 3.1(1−λ)a+λ(ωa+β)ω,β= arg min ω,β P i (b (i) −(ωa (i) +β)) 2 Any layer,a∈Q o orQ ∞ mean bias to the activations. The way this bias is pre-computed is what differentiates each method. Note that the parameterλtypically multiplies the bias, thus becoming unbounded and non-interpretable. ACT applies a linear transformation on activations that maximally preserves internal distributions (Section 3.1, and distribution plots in Appendix F). Moreover, ACT interpolates between the current and transformed activations, makingλbounded between[0,1]and interpretable. An additional aspect is that other methods propose various heuristics to choose the support, while ACT uses all activations or the observed input range (Q o ). Note that CAA, ITI-Mand Mean-ACT use a difference in means. We subsume this family of methods reporting results for Mean-ACT, which has the additional advantage of an interpretableλ. An additional difference is that many methods use the last token only (in pseudocode,φ(z) =z[...,−1]). Det zero and AURA use max-pooling (φ(z) = z.max(−1)) whileACT uses an average across tokens (φ(z) =z.mean(−1)), which we have found to be more robust (see Appendix D). 4EXPERIMENTS ONLLMS We empirically verify the performance of ACT on pre-trained LLMs on toxicity mitigation (Sec- tion 4.1), general concept induction (Section 4.2), and truthfulness induction in particular (Sec- tion 4.3), showing the efficacy and robustness of ACT in different scenarios related to LLMs. 4.1TOXICITYMITIGATION INLLMS It is known that LLMs are prone to generate toxic language (Wen et al., 2023), especially when prompts are designed to elicit toxic behavior. In this section, we study how ACT is effective at toxic language mitigation compared to some recents methods such as AURA, ACTADDand ITI- C, on Gemma2-2B (Team et al., 2024) and Llama3-8B Dubey et al. (2024). To do so, we prompt each LLM with 1000 randomly chosen prompts from RealToxicityPrompts (RTP) (Gehman et al., 2020),known to induce toxic language generation. Then, we collect the generated continuation to each prompt and we evaluate toxicity with a ROBERTA-based classifier 2 , as in Suau et al. (2024). In addition, we also measure toxicity in a 0-shot manner by querying Llama3-8B-instruct as LLM-as- a-judge (Zheng et al., 2023) (more details on Appendix H). As a measure of general LLM utility we report in Table 2: (i) perplexity (PPL) on a fixed set of 20k Wikipedia sentences measured with the intervened model, (i) PPL of the generated sentences measured with Mistral-7B (Jiang et al., 2023) and (i) MMLU (Hendrycks et al., 2021) 5-shot accuracy using the intervened model. Besides, we report generation diversity results in Appendix G. Linear-ACT reduces toxicity up to7.5×and is robust toλ, layer, and model choiceWe ob- serve that Linear-ACT achieves up to7.5×reduction in toxicity on Gemma2-2B and4.3×on Llama3-8B, with minimal impact on PPL and MMLU. Most importantly, ACT obtains the best results atλ= 1, which is in line with our OT formulation, sinceλ= 1means full transport. Linear-ACT and Mean-ACT obtain similar toxicity mitigation results. ITI-Cachieves5.6×and 3.6×toxicity reduction on Gemma2-2B and Llama3-8B respectively. In line with the ITI-Cpaper 2 https://huggingface.co/s-nlp/roberta_toxicity_classifier 6 Table 2: Toxicity mitigation for Gemma2-2B and Llama3-8B, results over 5 runs. We intervene upon different layer types (layer column) and show the best layer per method. ITI-C, ACTADDand ACT have astrengthparameterλwhich we sweep. For each method, we report results for theλthat attained the best CLS toxicity that incurs less than+1increase in PPL Wikipedia. ACT methods and provide best results forλ= 1, achieving up to7.5×(Gemma2-2B) and4.3×(Llama3-8B) CLS toxicity mitigation with Linear-ACT. ITI-Cis very sensitive toλas well as layer choice (see full results in Appendix J), and AURA reaches up to3.1×reduction. LayerBestλCLS Tox. (%)↓0-shot Tox. (%)↓PPL Wikipedia↓PPL Mistral-7B↓MMLU↑ Original--4.17±0.3213.42±1.0813.986.6853.1 ACTADDMLP0.53.96±0.24 (1.1×) 13.43±1.4214.69(+0.72)6.67(+0.05)53.0(-0.1) AURAMLP-2.12±0.27(2.0×)9.04±0.6614.18(+0.21)7.04(+0.36)53.0(-0.1) ITI-CAttention8.00.74±0.18 (5.6×) 5.36±0.9114.90(+0.92)7.44(+0.76)52.6(-0.5) Mean-ACTPost-LN1.00.54±0.44(7.7×)4.10±0.4114.21(+0.23)7.59(+0.90)51.6(-1.5) Linear-ACTPost-LN1.00.56±0.21 (7.5×) 4.14±0.5514.79(+0.81)7.99(+1.31)51.3(-1.8) Original--5.8015.009.065.6865.3 ACTADDAttention0.35.57±0.45(1.0×)15.73±0.219.71(+0.65)5.85(+0.16)65.5(+0.2) AURAMLP-1.90±0.61(3.1×)8.12±0.859.52(+0.45)6.05(+0.37)65.5(+0.2) ITI-CAttention3.01.60±0.22(3.6×)6.53±0.669.48(+0.42)6.17(+0.49)64.7(-0.6) Mean-ACTAttention1.01.38±0.17(4.2×)5.60±0.349.56(+0.49)6.36(+0.68)64.7(-0.7) Linear-ACTAttention1.01.35±0.39(4.3×)6.68±0.819.56(+0.49)6.28(+0.60)64.5(-0.8) Gemma2-2B Llama3-8B findings, ITI-Cperforms well on attention, but is very sensitive to models and layers, as well as to the choice ofλ(see a layer diagram in Appendix B and full tables and plots in Appendix J). AURA achieves2.0×and3.1×toxicity reduction per model and ACTADDinduces the mildest mitigation. 4.2INDUCINGCONCEPTS INLLMS WITHACT λ= 1λ= 10 Intervention strengthλ 0.0 0.5 1.0 0-shot p ( yes ) ITI-c Linear-AcT Mean-AcT λ= 1λ= 10 Intervention strengthλ 5 10 15 20 PPL Mistral-7B ITI-c Linear-AcT Mean-AcT Figure 4: Concept induction using ACT (post-LN layers) and ITI-C(attention layers) on Gemma2- 2B. We aggregate results over 7 WordNet concepts, generating 500 sentences at different interven- tion strength levels. We report concept presence with LLM-as-a-judge (p(yes)), and the PPL of the generated sentences using Mistral-7B. We plot the median (and 25/75 quantile band) across concepts and generations per level, showing that Linear-ACT achieves a peak of concept induction atλ≈1, which is inline with our OT formulation. Other methods show different maxima. ACT allows transporting activations from distributionμtoν(derived from sentence distributionsp andqrespectively). In an induction setting,pcovers generic content, whileqa specific concept that we want to induce. We mine the OneSec dataset (Scarlini et al., 2019), collecting 700 sentences that contain a specific concept (q) and 700 sentences randomly sampled from other concepts (p). We do so for seven different concepts (football, cloud, baby, church, book, flower, balloon) and we estimate an intervention for each of them. We assess the presence of a concept in the generated text in a LLM- as-a-judge manner by querying Llama3-8B-instruct (LLM-as-a-judge details in Appendix I). Linear-ACT can induce arbitrary concepts with consistentλ= 1Figure 4 shows the effect of increasingλboth on the presence of the concept,p(yes), and the PPL measured with Mistral-7B on the generated text. We intervene upon the most effective layers for each method according to the toxicity results: attention for ITI-C, and Post-LN for ACT. In general, we found that LN layers were the most suited for ACT, across models and tasks. A naive explanation is that centering and scaling activations keeps the source and target activation distributions within a reasonable range, which makes the transport map more reliable. We do not include AURA because it is designed for mitigation, and ACTADDgives lower performance on this task. For Linear-ACT, we observe a peak of concept presence atλ≈1, with a medianp(yes) = 0.87(i.e.,87% of the generated sentences 7 Table 3: TruthfulQA results for Gemma2-2B and Llama3-8B, results over 5 runs. We intervene upon different layers (layer column) and show the best per model. ITI-C, ACTADDand ACT have a strengthparameterλwhich we sweep, reporting the bestλresult per model (MC1 Accuracy so that MMLU is within the best ACT MMLU±0.1). LayerBestλMC1 Accuracy (%)↑MC2 Accuracy (%)↑MMLU Accuracy (%)↑ Original--21.0532.8053.10 ACTADDMLP3.023.01±0.00(+1.96)34.76±0.00(+1.96)52.83±0.00(−0.27) AURAMLP-21.20±0.10(+0.15)32.88±0.22(+0.08)52.73±0.07(−0.37) ITI-CMLP2.024.53±0.11(+3.48)37.06±0.38(+4.26)51.39±0.41(−1.71) Mean-ACTAll-LN1.025.07±0.20(+4.02)38.68±0.30(+5.88)51.81±0.12(−1.29) Linear-ACTAll-LN1.026.00±0.32(+4.95)40.17±0.24(+7.37)51.47±0.27(−1.63) Original--25.4640.2765.35 ACTADDAttention0.726.19±0.00(+0.73)40.88±0.00(+0.61)65.42±0.00(+0.07) AURAMLP-25.34±0.15(−0.12)40.47±0.20(+0.20)65.37±0.06(+0.02) ITI-CMLP2.030.11±0.60(+4.65)45.41±0.24(+5.14)64.71±0.14(−0.64) Mean-ACTAll-LN1.032.88±0.54(+7.42)48.23±0.64(+7.96)64.83±0.14(−0.52) Linear-ACTAll-LN1.033.22±0.22(+7.76)48.69±0.34(+8.42)64.78±0.15(−0.57) Gemma2-2B Llama3-8B are classified as containing the induced concept) and an acceptable PPL= 8.5. Forλ >1, the PPL quickly degrades and the presence of the concept diminishes. This is also consistent with the toxicity mitigation experiments in Section 4.1. Interestingly, the peak for Mean-ACT is atλ≈2.5, also highlighting that Mean-ACT is a poorer approximation of the OT transport. Notably, ITI-C achieves a similarp(yes)and PPL as Linear-ACT forλ≈5. However, note that ITI-C’s bestλis different than the ones for toxicity. Appendix K contains generation examples. 4.3INDUCING TRUTHFULNESS INLLMS WITHACT One particular concept that has gained attention in previous activation steering works is “truthful- ness” (Li et al., 2024). We study how ACT can increase truthfulness on Gemma2-2B and Llama3- 8B, compared to the original model. Again, we compare to AURA, ACTADDand ITI-C. We evalu- ate all methods on the TruthfulQA multiple choice part that has been used in prior work (Lin et al., 2021; Li et al., 2024). We report both MC1 and MC2 of TruthfulQA, and control for overfitting on the TruthfulQA task by also evaluating MMLU 5-shot accuracy (Hendrycks et al., 2021). ACT can induce truthfulness with consistentλ= 1.The results of our experiments are summa- rized in Table 3. As we can see, ACT can successfully induce truthfulness in both models in its default settingλ= 1(corresponding to full transport). Both Linear-ACT and Mean-ACT achieve the best and second-best MC1 and MC2 accuracy improvements among all methods investigated. Linear-ACT increases MC1 by roughly5%for Gemma2-2B and by almost8%for Llama3-8B, which is about1.5%and3%more than the closest non-ACT baseline (ITI-C), while incurring even slightly less decrease in MMLU performance. Full results and experimental setup in Appendix L. 5CONTROLLINGIMAGEDIFFUSIONMODELS In this section, we show that ACT improves the controllability of text-to-image diffusion models (T2Is), a well-known challenge (Cao et al., 2024). We address two open problems in T2I generation: fine-grained style control (Section 5.1) and concept negation (Section 5.2). We show that off-the- shelf ACT succeeds at both tasks. In line with optimal transport theory and experimental results on LLMs (Section 4), ACT consistently achieves the strongest conditioning with full strength (i.e., λ= 1). We also adapt ITI-Cto the topology of images by training it on the spatial average pooling of activations (as we do by default for ACT), and applying it to each spatial position independently. Remarkably, ITI-Csucceeds at the fine-grained control task with our adaptation, but requires tuning λ, and it fails with concept negation. Setup.We apply ACT on the denoising convolutional UNet of Stable Diffusion XL (SDXL) (Podell et al.) and the denoising transformer of FLUX.1.Schnell 3 . For FLUX, we use 3 https://blackforestlabs.ai/announcing-black-forest-labs/ 8 0.00.40.60.81.0 2.0best 0.00.40.60.81.0 1.0best 0.00.40.60.81.0 1.0best 0.00.40.60.81.0 1.0best 0.00.40.60.81.0 1.0best 0.00.40.60.81.0 1.0best Figure 5:Linear-ACT allows controlled conditioning of SDXL and FLUX.“A cat resting on a laptop keyboard in a bedroom.” SDXL (left) and FLUX (right) intervened with ITI-C(top), Mean- ACT (middle) and Linear-ACT (bottom) for the conceptcyberpunk, with aλstrength in[0,1]. The image with the bestλ(according to the highest 0-shot score in Figure 6) is shown right. Qualitatively, Linear-ACT balances better acyberpunkstyle increase with prompt semantics preservation. the T5-XXL text encoding modality (Raffel et al., 2020) instead of CLIP (Radford et al., 2017) to account for the effects of language modelling. We use a distilled version of SDXL, which only re- quires 4 diffusion steps (Lin et al., 2024) like FLUX. We intervene upon all normalization layers in SDXL’s UNET and the output of most residual layers in FLUX (details in Appendix M.8). We only show results for ACT and ITI-Csince ACTADDis not applicable to images and AURA resulted in noisy images. To measure the presence of a style or a concept, we use a CLIP zero-shot classifier with the classes (+) “A picture of astyle or concept” and (-) “A picture ofsomething”. We also track whether the content from the original prompt (with no style or concept modifiers) is preserved using the CLIPScore (cosine similarity of CLIP embeddings, Hessel et al. (2021)) between the images generated after the intervention and the original prompt. 5.1STYLE CONTROL A major challenge in T2I generation is fine-grained control. For example, while one can prompt SDXL to create a sketch of an object, it is hard to control the level of “sketchiness”. Models such as SDXL have a guidance parameter, but its use is limited since low guidance values tend to remove image semantics (see example in Appendix M.1). To showcase the ability of ACT to achieve such a fine-grained control, we sample 2048 prompts from the COCO Captions (Chen et al., 2015) training set and append a series of tags generated with Llama-8B-instruct to induce the following styles: anime, art nouveau, cyberpunk, impressionism, sketch, watercolor(see Table 15 for details). Then we use the original prompt as the source distribution (p) and the style-modified prompt as the target distribution (q) to learn transport maps for style. To evaluate, we sample 512 prompts from the COCO Captions validation set and generate images with different intervention strengths. Linear-ACT is a robust method for fine-grained control in text-to-image generation.Fig- ure 6a shows that Linear-ACT on SDXL and FLUX increases the presence of a desired style,e.g., on SDXL from∼12%to∼95%of the generated images while keeping∼80%of the similarity to the original prompt (λ= 1). In accordance to the theory and experiments on LLMs, the maximum conditioning (i.e.,highest 0-shot score) for ACT is achieved atλ= 1for both models. ITI-C can also accomplish fine-grained control, but its best performance is achieved at differentλs, equal to 2 and 1 for SDXL and FLUX respectively, which is in turn not consistent with the bestλ found in LLM experiments. A closer look at images generated with ITI-Cfor bestλin Figure 5 and appendix M.3 reveals that ITI tends to exaggerate style traits while distorting the semantics. This further highlights the reliability of ACT across different modalities, tasks, and models. While quantitatively ACT and ITI-Cperform well, we invite the reader to compare the quality of the generated images and styles in Figures 1 and 5, and in more examples in Appendix M.3. 5.2CONCEPTNEGATION T2I diffusion models are known for struggling with concept negation (Li et al.; Hwang et al., 2024). For example, Hwang et al. (2024) showed that recent models such as Stable Diffusion (Rombach et al., 2022) and DALL-E 3 (Betker et al., 2023) are prone to generate a pink elephant when in- structed not to generate one. To improve controllability, some T2I generators like SDXL include a 9 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (a) Style control 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(concept) ( ← ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (b) Concept Negation Figure 6: Style control (a) and concept negation (b) on SDXL and FLUX. Top row shows the fraction of generated images classified (CLIP 0-shot) as containing a given concept or style. Bottom row shows how much the intervened model deviates from the unmodified one in terms of ClipScore between the image and the original unconditional prompt. Points inside the gray area represent images that have lost their semantic content. Figure 7:Concept Negationfor “A plate of food with rice and beans, broccoli and meat. And a pink elephant is missing.”. (a) Linear-ACT on SDXL with transport strengthλlinearly increasing from 0 to 1. Note how the presence of the pink elephant is prominent for the original model (leftmost image) and gradually disappears asλincreases. negative promptmechanism to remove concepts from the generated images. However, we found that both SDXL (CLIP encoder + negative prompt) and FLUX (T5-XXL encoder) still tend to generate unwanted concepts (see some examples in Appendix M.2). We use the COCO Captions (Chen et al., 2015) training set to sample 2048 prompts used to gen- erate the images. To create a source and target activation distribution to estimate ACT, we ask Llama3-8B-instruct to generate a diverse set of prompt modifiers requiring the model to include the following concepts:pink elephant, white bear,andgorilla. The exact phrasing of the modifiers is provided in Table 16. We estimate our transport maps from the modified prompts (p, with concept) to the unmodified prompts (q). To evaluate the model, we sample 512 captions the COCO Captions validation set and ask Llama-3B-instruct to negate each of the modifiers used before (e.g., “without a pink elephant”, “a gorilla cannot be seen anywhere”) to generate images with unintended concept spillage such as the leftmost image in Figure 7 or the examples in Figures 18 and 19. Linear-ACT is a robust method for concept negation in text-to-image generation. InFig- ure 6b, we observe that ACT is more effective at concept negation than ITI-Cwhile better preserving the original semantics of the image, as indicated by the drop in 0-shot concept score for higher CLIPScore than ITI-C. ITI requires a stronger intervention to reduce the presence of the undesired concept, at the cost of losing the whole semantic content, hence the drop in the Relative ClipScore. Additional examples and images for each concept can be found in Appendix M.4. 6LIMITATIONS ANDDISCUSSION In this work, we introduce Activation Transport (ACT), a general framework to achieve intuitive and fine-grained control of GMs. Our approach is based on optimal transport theory, effectively mapping activations from a source to a target distribution by preserving the latter, and unifies many previous activation steering works. We show experimentally that our Linear-ACT approach generalizes well across models and tasks, for both LLMs and T2I architectures. Moreover, ACT provides a robust parameter to control the amount of conditioning, bounded between 0 and 1, which makes it user- 10 friendly and interpretable. While effective, Linear-ACT assumes a linear transport between i.i.d. activations, which are simplifications adopted for compute and memory reasons. Additionally, the map estimation purely depends on the samples used, thus being limited by their expressiveness. In future work, we plan on exploring non-linear maps and joint activations distributions. 11 ETHICSSTATEMENT Our method could theoretically be used to mitigate or induce the presence of any concept. Therefore, it could eventually lead to the development of censorship or misinformation tools. While our work can be used to align in pre-trained GMs, it should not be taken as a reason not to pursue the adoption of clean data and additional alignment strategies during the pre-training phase. REPRODUCIBILITYSTATEMENT We will make our code and data publicly available on github. To aid reproducibility, all tables contain the bestλfound through grid-search and results are averaged over 5 runs. We include additional details on the intervened layers in Appendix B, ablations on the effect of transport support in Appendix E, pooling operation ablations in Appendix D, the exact prompt templates of LLM as a judge in Appendices H and I, experimental details on TruthfulQA in Appendix L, as well as experimental details for T2I models in Appendix M. ACKNOWLEDGEMENTS We thank Miguel A. Bautista, Federico Danieli, Gerard G ́ allego, Yu-Guan Hsieh, Miguel Sarabia, Federico Scozzafava, and Barry Theobald (in alphabetical order) for their helpful feedback and crit- ical discussions throughout the process of writing this paper. We would also like to thank Aswathy Balagopalan for contributing to the codebase, and Jerremy Holland for supporting this work. REFERENCES James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions.Computer Science., 2(3):8, 2023. Manuel Brack, Patrick Schramowski, Felix Friedrich, Dominik Hintersdorf, and Kristian Kersting. The stable artist: Steering semantics in diffusion latent space.arXiv preprint arXiv:2212.06013, 2022. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020. Pu Cao, Feng Zhou, Qing Song, and Lu Yang. Controllable generation with text-to-image diffusion models: A survey.arXiv preprint arXiv:2403.04279, 2024. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Doll ́ ar, and C Lawrence Zitnick. Microsoft coco captions: Data collection and evaluation server.arXiv preprint arXiv:1504.00325, 2015. Sinho Chewi, Jonathan Niles-Weed, and Philippe Rigollet. Statistical optimal transport.arXiv preprint arXiv:2407.18163, 2024. Kevin Clark, Paul Vicol, Kevin Swersky, and David J Fleet. Directly fine-tuning diffusion models on differentiable rewards.arXiv preprint arXiv:2309.17400, 2023. Jasper Dekoninck, Marc Fischer, Luca Beurer-Kellner, and Martin Vechev. Controlled text genera- tion via language model arithmetic.arXiv preprint arXiv:2311.14479, 2023. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 12 Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ̈ uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. InForty-first International Conference on Machine Learning, 2024. C. Fellbaum.WordNet: An Electronic Lexical Database. Language, Speech and Communication. Mit Press, 1998. Rohit Gandikota, Joanna Materzynska, Tingrui Zhou, Antonio Torralba, and David Bau. Concept sliders: Lora adaptors for precise control in diffusion models.arXiv preprint arXiv:2311.12092, 2023. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith.Real- toxicityprompts: Evaluating neural toxic degeneration in language models.arXiv preprint arXiv:2009.11462, 2020. Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, and Noah Goodman. Find- ing alignments between interpretable causal variables and distributed neural representations. In Causal Learning and Reasoning, p. 160–187. PMLR, 2024. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding.Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2021. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 7514–7528, 2021. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. InInternational conference on machine learning, p. 2790–2799. PMLR, 2019. Dang Huu-Tien, Trung-Tin Pham, Hoang Thanh-Tung, and Naoya Inoue. On effects of steering la- tent representation for large language model unlearning.arXiv preprint arXiv:2408.06223, 2024. Kyomin Hwang, Suyoung Kim, JunHoo Lee, and Nojun Kwak. Do not think pink elephant!arXiv preprint arXiv:2404.15154, 2024. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023. Zeyinzi Jiang, Chaojie Mao, Yulin Pan, Zhen Han, and Jingfeng Zhang. Scedit: Efficient and con- trollable image diffusion generation via skip connection editing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8995–9004, 2024. Suhas Kotha, Jacob Mitchell Springer, and Aditi Raghunathan. Understanding catastrophic forget- ting in language models via implicit inference. 2024. Kyungmin Lee, Sangkyung Kwak, Kihyuk Sohn, and Jinwoo Shin. Direct consistency optimization for compositional text-to-image personalization.arXiv preprint arXiv:2402.12004, 2024. Kenneth Li, Oam Patel, Fernanda Vi ́ egas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model.Advances in Neural Information Processing Systems, 36, 2024. Senmao Li, Joost van de Weijer, Fahad Khan, Qibin Hou, Yaxing Wang, et al. Get what you want, not what you don’t: Image content suppression for text-to-image diffusion models. InThe Twelfth International Conference on Learning Representations. Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation.arXiv e-prints, p. arXiv–2402, 2024. 13 Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods.arXiv preprint arXiv:2109.07958, 2021. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning, 2023. Robert J McCann. A convexity principle for interacting gases.Advances in mathematics, 128(1): 153–179, 1997. Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, p. 4296– 4304, 2024. Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal M Patel, and Tim K Marks. Steered diffusion: A generalized framework for plug-and- play conditional image synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, p. 20850–20860, 2023. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in Neural Information Processing Systems, 35: 27730–27744, 2022. Gabriel Peyr ́ e and Marco Cuturi. Computational Optimal Transport.Foundations and Trends in Machine Learning, 11(5-6), 2019. ISSN 1935-8245. Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ̈ uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. InThe Twelfth International Conference on Learning Representations. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. Learning to generate reviews and discovering sentiment.arXiv preprint arXiv:1704.01444, 2017. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer.The Journal of Machine Learning Research, 21(1):5485–5551, 2020. Nate Rahn, Pierluca D’Oro, and Marc G Bellemare. Controlling large language model agents with entropic activation steering.arXiv preprint arXiv:2406.00244, 2024. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ̈ orn Ommer. High- resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, p. 10684–10695, 2022. Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 22500– 22510, 2023. Filippo Santambrogio. Optimal transport for applied mathematicians.Birk ̈ auser, NY, 55(58-63):94, 2015. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. Just “onesec” for producing multilingual sense-annotated data. p. 699–709, 01 2019. doi: 10.18653/v1/P19-1069. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models’ sen- sitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting.ICLR, 2024. 14 Nick Stracke, Stefan Andreas Baumann, Joshua M Susskind, Miguel Angel Bautista, and Bj ̈ orn Ommer. Ctrloralter: Conditional loradapter for efficient 0-shot control & altering of t2i models. arXiv preprint arXiv:2405.07913, 2024. Xavier Suau, Luca Zappella, and Nicholas Apostoloff. Self-conditioning pre-trained language mod- els. InInternational Conference on Machine Learning, p. 4455–4473. PMLR, 2022. Xavier Suau, Pieter Delobelle, Katherine Metcalf, Armand Joulin, Nicholas Apostoloff, Luca Zap- pella, and Pau Rodriguez. Whispering experts: Neural interventions for toxicity mitigation in language models. InForty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=2P6GVfSrfZ. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhu- patiraju, L ́ eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram ́ e, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024. Alex Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDi- armid. Activation addition: Steering language models without optimization.arXiv preprint arXiv:2308.10248, 2023. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural informa- tion processing systems, 30, 2017. Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8228–8238, 2024. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, An- drew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. InInternational Conference on Learning Representations. Jiaxin Wen, Pei Ke, Hao Sun, Zhexin Zhang, Chengfei Li, Jinfeng Bai, and Minlie Huang. Unveiling the implicit toxicity in large language models. p. 1322–1338. Association for Computational Linguistics, December 2023. Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Man- ning, and Christopher Potts. ReFT: Representation finetuning for language models. 2024. URL arxiv.org/abs/2404.03592. Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8941–8951, 2024. Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.arXiv preprint arXiv:2308.06721, 2023. SHIH-YING YEH, Yu-Guan Hsieh, Zhidong Gao, Bernard B W Yang, Giyeong Oh, and Yanmin Gong. Navigating text-to-image customization: From lyCORIS fine-tuning to model evaluation. InThe Twelfth International Conference on Learning Representations, 2024. URLhttps: //openreview.net/forum?id=wfzXa8e783. Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models.Advances in Neural Information Processing Systems, 36, 2024. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena.Advances in Neural Information Processing Systems, 36:46595–46623, 2023. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. InThe 41st international ACM SIGIR conference on research & development in information retrieval, p. 1097–1100, 2018. 15 Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405, 2023. 16 AMEMORY ANDCOMPUTATIONALASPECTS Linear-ACT requires storing 2 floats (ω,β) per activation intervened. For example, Linear-ACT on post-LN layers of Gemma2-2B requires(2×52layers×2304activations×4bytes) = 0.91Mb. If we choose to use the support transport, 2 more floats per activation are storedQ o = [minA,maxA], which means an extra0.91Mb for the Gemma2-2B example. In terms of compute, Linear-ACT requires an extra element-wise product and sum per intervened layer. However, the inference cost of such operations is of second order compared to the overall LLM inference cost. One has the option to fixλ. If so, our Linear-ACT formulation in Definition 3.1 becomesT lin (a) = λ(ω−1) + 1 a+λβ= ̃ωa+λβ. Assuming we intervene after a linear layerγa+δ, we compose both functions as(T lin ◦f)(a) = ̃ωγa+ ( ̃ωδ+λβ), which is also a linear map whose parameters can replace those offin the computational graph, without any extra cost at inference time. The memory cost is0if we fixλand compose Linear-ACT with the model linear layers. A.1DETAILS ON COMPUTATIONAL COMPLEXITY The computational cost of Linear-ACT can be divided in two main parts: estimation and inference. Estimation.The estimation cost is the cost related to extracting activations from a model and estimating a transport map on top. Let us assume the cost for running an inference step with a model up to the latest layer where an intervention is placedLisM L ,Nthe number of samples upon which we learn the transport, andDthe dimensionality of each activation vector. We also assume batch size= 1. • Extracting activations: –Assuming non-sequential iterative maps (see Section 3.2): the cost for extracting ac- tivations isO(NM L ). –Assuming sequential iterative maps, we need two forward passes per layer: the first is used to estimate a transport map, and the second to produce responses after applying the map. Since the cost of applying a map with fixed strength is 0, the cost of extracting activations with iterative maps isO(2NM L ). • Estimating a linear transport map involves sortingNLDactivations for the source and target distribution and computing the affine transport params analytically (see Definition 3.1). Assuming half of theNsamples belong to the source and the target distributions re- spectively, the cost is dominated by the sorting operationO(NLDlog(NLD))(assuming quicksort is used), which is also smaller than the cost of a forward pass through the model. Inference.The inference cost is the cost related to generating an output with an intervened model. As explained at the beginning of the section, assuming a fixed transport map strength (λ), the affine transport map can be directly fused into the model weights and thus the additional cost of Linear- ACT isO(0). If we need to be able to tune the intervention strength, then we cannot fuse it into the weights and the cost is that of a 1-d affine map on all the transported activations, which is significantly smaller than the cost of a forward pass on the model, which involves expensive matrix multiplication:O(LD)<< O(M). Summarizing, estimation is only done once, has costO(NM L ), and it is amortized during inference. During inference, the transport cost isO(0)with fixedλandO(LD)with variableλ. In plain words, estimating a transport map is much cheaper than training a model and has no impact at inference time unless one needs control overλ, in which case the additional cost is significantly smaller than the cost of a forward pass with the model. 17 BINTERVENEDLAYERS input Pre-LN Attention Post-LN + Pre-LN MLP Post-LN + output Gemma2-2B Figure 8: Schema of a Transformer block of Gemma2-2B with the layer names as referenced in this work. Note that Llama3-8B has a similar structure without the Post-LN layers. CCAUSAL VS. SIMULTANEOUSESTIMATION OFACT In Table 4 and Table 5 we compare the estimation of ACT interventions in a causal and simultaneous way (see Section 3.1). We observe that causal estimations show better toxicity mitigation than its simultaneous counterparts. Table 4: Causal (gray background) vs. simultaneous estimation of ACT on Gemma2-2B in a toxicity mitigation setting (explained in Section 4.1). Causal estimation provides better conditioning (lower toxicity). CausalLayerBestλPPL Wikipedia↓PPL Mistral-7B↓CLS Toxicity (%)↓0-shot Toxicity (%)↓ Original---13.986.624.08±0.3613.25±0.88 Mean-ACTAttention1.013.907.23(+0.61)1.12±0.355.60±1.01 Mean-ACT✓Attention1.014.08(+0.11)7.23(+0.61)1.06±0.175.14±0.50 Linear-ACTAttention1.014.04(+0.06)7.26(+0.64)0.97±0.395.75±0.90 Linear-ACT✓Attention1.014.21(+0.23)7.24(+0.62)0.90±0.335.06±0.63 Mean-ACTPost-LN1.014.11(+0.13)7.71(+1.09)0.62±0.054.47±0.65 Mean-ACT✓Post-LN1.014.21(+0.23)7.59(+0.97)0.54±0.444.10±0.41 Linear-ACTPost-LN0.914.54(+0.57)7.87(+1.25)0.65±0.174.40±0.39 Linear-ACT✓Post-LN1.014.79(+0.81)7.99(+1.37)0.56±0.214.14±0.55 Table 5: Causal (gray background) vs. simultaneous estimation of ACT on Llama3-8B in a toxicity mitigation setting (see Section 4.1). Causal estimation provides better conditioning (lower toxicity). CausalLayerBestλPPL Wikipedia↓PPL Mistral-7B↓CLS Toxicity (%)↓0-shot Toxicity (%)↓ Original---9.065.685.8015.00 Mean-ACTAttention1.09.35(+0.28)6.33(+0.65)1.40±0.296.73±1.13 Mean-ACT✓Attention1.09.56(+0.49)6.36(+0.68)1.38±0.175.60±0.34 Linear-ACTAttention1.09.38(+0.32)6.27(+0.58)1.38±0.246.55±0.75 Linear-ACT✓Attention1.09.56(+0.49)6.28(+0.60)1.35±0.396.68±0.81 DTHE EFFECT OF THE POOLING OPERATION The number of activations to store to compute a transport map isO(NMLK), whereNis the number of samples used to estimate the transport,Mis the number of activations per layer,Lis the number of layers, andKthe number of tokens decoded. This number can easily become intractable so most methods perform a pooling operationφoverK. We run an ablation on the pooling operation for ACT on Gemma2-2B, in the toxicity mitigation setup. We find that mean pooling achieves a better trade-off between toxicity mitigation and utility, measured as MMLU (Table 6). 18 Table 6: Ablation on the choice of pooling operation (see Section 3) on Gemma2-2B. MethodPoolingφStrengthλCLS Tox. (↓)MMLU (↑) Original--4.17±0.3253.06 Linear-ACTmin10.77±0.1245.85±0.09 Linear-ACTmax11.80±0.1247.01±0.30 Linear-ACTlast10.47±0.1748.49±0.25 Linear-ACTmean10.70±0.1051.87±0.06 ETHEEFFECT OF THETRANSPORTSUPPORT In this section we validate the choice oftransport support, as a way to make the pro- posed intervention more robust.In this experiment, we sweep different supports by narrowing the quantiles (qt) of the input data setA, in the setting of toxicity mit- igation (as in Section 4.1), both for Mean-ACT and Linear-ACT. The supports tested are:[qt 40 ,qt 60 ],[qt 30 ,qt 70 ],[qt 20 ,qt 80 ],[qt 10 ,qt 90 ],[qt 5 ,qt 95 ],[qt 3 ,qt 97 ],[qt 1 ,qt 99 ],[qt 0 ,qt 100 ]and (−∞,∞). Note that[qt 0 ,qt 100 ] =Q o , as defined in Section 3.1. We show the results of this sweep in Fig- ure 9, where we observe that[qt 0 ,qt 100 ]offers a good trade-off between conditioning strength and acceptable increase in PPL (below +1 points with respect to the original model). 0.000.020.04 CLS Toxicity 13.5 14.0 14.5 15.0 15.5 16.0 PPL Wikipedia −∞,∞ qt 0 , qt 100 qt 1 , qt 99 qt 40 , qt 60 −∞,∞ qt 0 , qt 100 qt 1 , qt 99 qt 40 , qt 60 Original Mean-AcT Linear-AcT Figure 9: We measure toxicity mitigation on Gemma2-2B by increasingly expanding the transport support from[qt 40 ,qt 60 ]on the farther right of the plots to[qt 0 ,qt 100 ] = [minA,maxA], which means the support spanned by all the samples inA. For completeness, we add the full real support (−∞,∞). For Linear-ACT, using[qt 0 ,qt 100 ]achieve the best toxicity mitigation by incurring less than+1increase in PPL. Note that(−∞,∞)results in higher PPL. FHOWDODIFFERENTINTERVENTIONSAFFECTDISTRIBUTIONS? We show in this experiment how activation distributions are modified by the effect of different interventions. For that, we plot in Figure 10 the distribution of source activationsμ(toxic), that of target activationsν(non-toxic) and also the distribution obtained when mapping samples with a map T,i.e.,T♯μ. Ideally, we would like to observe thatν≈T♯μ. We show the distributions of those 19 activations with highestnormalized cost ̄wcomputed as ̄c= 1 N P N i=0 b (i) −ωa (i) −β 2 |m b −m a |+σ b +σ a ,(2) so that we pick activations withμ̸=νfor the sake of illustration. We observe that Linear-ACT obtains a very good overlap of distributions (first row) while ITI-Cdoes not in many cases (this result extends to any bias-based method, we show ITI-Cas an example of such family of methods). The latter is onlyshiftingactivations with a bias, thus becoming impossible to adapt the shape of distributions. Moreover, we can observe that with ITI-Csome activations are mildly shifted (4th column), and some others are strongly shifted (2nd, 3rd, 5th columns). This makes it evident that it is very hard to set a robustλfor bias-based steering methods. −3−2−10123 0.0 0.1 0.2 0.3 0.4 0.5 0.6 7.post feedforward layernorm@1159, ̄ c= 0.234 μ(Original) ν(Original) T]μ(Linear-AcT) −2.0−1.5−1.0−0.50.00.51.01.5 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 8.post feedforward layernorm@771, ̄ c= 0.128 μ(Original) ν(Original) T]μ(Linear-AcT) −15−10−50 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 10.post feedforward layernorm@1170, ̄ c= 0.265 μ(Original) ν(Original) T]μ(Linear-AcT) −4−3−2−1012 0.0 0.2 0.4 0.6 0.8 14.post feedforward layernorm@1939, ̄ c= 0.273 μ(Original) ν(Original) T]μ(Linear-AcT) −1001020304050 0.00 0.01 0.02 0.03 0.04 0.05 0.06 17.post feedforward layernorm@334, ̄ c= 0.432 μ(Original) ν(Original) T]μ(Linear-AcT) −10−50510 0.00 0.05 0.10 0.15 0.20 0.25 7.post feedforward layernorm@1159, ̄ c= 0.289 μ(Original) ν(Original) T]μ(ITI-c) −4−2024 0.0 0.1 0.2 0.3 0.4 8.post feedforward layernorm@1453, ̄ c= 0.181 μ(Original) ν(Original) T]μ(ITI-c) −4−20246 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 10.post feedforward layernorm@914, ̄ c= 0.165 μ(Original) ν(Original) T]μ(ITI-c) −7.5−5.0−2.50.02.55.07.5 0.00 0.05 0.10 0.15 0.20 14.post feedforward layernorm@2193, ̄ c= 0.287 μ(Original) ν(Original) T]μ(ITI-c) −10−50510 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 17.post feedforward layernorm@509, ̄ c= 0.522 μ(Original) ν(Original) T]μ(ITI-c) Figure 10:Transport of distributions. We show how different interventionstransportthe internal distributions. In gray the source distributionμ(toxic), in blue the target distributionν(non-toxic) and in red the distributionT♯μobtained when pushing-forwardμthrough a given interventionT. Each column contains the distributions for the activation with highest ̄c(see Equation (2)) in a given layer. In the first row we show Linear-ACT, observing a good overlap betweenνandT♯μ. The second row shows ITI-C, with a poorer distribution overlap. We useλ= 1for Linear-ACT and λ= 8for ITI-C(optimalλs from Table 2). GASSESSINGTEXTGENERATIONDIVERSITY One important question is whether the generated text after a model intervention still shows diversity. To answer this question, in Table 7 we measure the Self-BLEU score (Zhu et al., 2018) for the sets of generated sentences after RTP prompts. Note that smaller Self-BLEU scores indicate higher diversity in the set, while large Self-BLEU shows repeatedness in the sentences. For example, a set of identical sentences will return a Self-BLEU of 1. We evaluate the best configuration for each method (layer,λchoice) according to Table 2. From the results in Table 7, we observe that Linear-ACT (Self-BLEU= 0.134) better preserves the diversity shown by the non-intervened model (Self-BLEU= 0.130). In this setting, ITI-Cachieves 0.144 and our Mean-ACT a Self-BLEU of 0.140. We obtain these results averaging over 4 runs of 1000 generations each, and the standard deviations show that the results are significant. 20 Table 7: ACT obtains better diversity. We measure the Self-BLEU score as a proxy for diversity, on 4 randomly chosen sets of 1000 generations after RTP prompts. Linear-ACT obtains the closest result to the original model, showing higher diversity. MethodSelf-BLEU (↓) ITI-C0.144±0.003 Mean-ACT0.140±0.002 Linear-ACT0.134±0.002 Original0.130±0.003 H0-SHOT TOXICITY CLASSIFICATION We query Llama3-8B-instruct with the system prompt below, where[SENTENCE]contains the sentence to be assessed in 0-shot manner. "You are a highly accurate toxicity classification model. Your task is to determine if a given sentence contains toxic content. You will output ’yes’ only if the sentence is toxic and ’no’ if the sentence is neutral or non toxic. [SENTENCE]" Then, we measure the toxicity probability as count(yes) count(yes)+count(no) , reported as 0-shot toxicity in Tables 2, 8 and 9. Note that we use the logits for tokensyesandno, so we consider the answer to be positive whenlogit(yes)> logit(no). I0-SHOT CONCEPT PRESENCE CLASSIFICATION We query Llama3-8B-instruct with the system prompt template below. "You are a chatbot who answers whether the provided sentence is referring to [CONCEPT] defined as [WORDNETDEF]. Note that the sentence might not contain the word [CONCEPT], but may just be referencing concept as defined. [SENTENCE]". Where: •[CONCEPT]can befootball, cloud, baby, church, book, flower, balloon. •[WORDNET DEF]are taken from WordNet Fellbaum (1998): – football: Any of various games played with a ball (round or oval) in which two teams try to kick or carry the ball into each other’s goal. – cloud: A visible mass of water or ice particles suspended at a considerable altitude. – baby: A very young child (birth to 1 year) who has not yet begun to walk or talk. – church: A place for public (especially Christian) worship. – book: A written work or composition that has been published (printed on pages bound together). – flower: A plant cultivated for its blooms or blossoms. – balloon: Large tough nonrigid bag filled with gas or heated air. •[SENTENCE]Contains the sentence to be assessed in 0-shot manner. We measure the probability of a concept being present as we do with toxicity, explained in Ap- pendix H. 21 JEXTENDEDRESULTS ONTOXICITYMITIGATION We report here the full experimental results for toxicity mitigation, which have been summarized in Section 4.1. Note the variability in the optimal strengthλfor ITI-Cand ACTADD, which compli- cates the applicability of these methods on different models and layers. Table 8: Toxicity mitigation for Gemma2-2B, results over 5 runs. We show results intervening different layers in the model (layer column). ITI-C, ACTADDand ACT have astrengthparameterλ which we sweep, reporting for each method the best result (bestλ) in CLS toxicity that incurs less than+1increase in PPL Wikipedia. ACT methods are robust to the choice of layer and provide best results forλ= 1, achieving up to7.5×toxicity mitigation with Linear-ACT. ITI-Cis very sensitive toλas well as layer choice, and AURA does not provide a strength control. LayerBestλPPL Wikipedia↓PPL Mistral-7B↓MMLU↑CLS Toxicity (%)↓0-shot Toxicity (%)↓ Original--13.986.6853.14.17±0.3213.42±1.08 ACTADDAtention0.513.99(+0.02)6.5853.2(+0.2)4.17±0.1513.25±1.63 ITI-CAtention8.014.90(+0.92)7.44(+0.76)52.6(-0.5)0.74±0.185.36±0.91 Mean-ACTAtention1.014.08(+0.11)7.23(+0.55)52.5(-0.6)1.06±0.175.14±0.50 Linear-ACTAtention1.0 14.21(+0.23)7.24(+0.56)52.2(-0.9)0.90±0.335.06±0.63 ACTADDPost-LN0.114.04(+0.06)6.6153.2(+0.2)4.08±0.4313.50 ITI-CPost-LN13.014.89(+0.92)7.34(+0.66)52.8(-0.3)3.08±0.6112.24±0.69 Mean-ACTPost-LN1.014.21(+0.23)7.59(+0.90)51.6(-1.5)0.54±0.444.10±0.41 Linear-ACTPost-LN1.014.79(+0.81)7.99(+1.31)51.3(-1.8)0.56±0.214.14±0.55 AURAMLP-14.18(+0.21)7.04(+0.36)53.0(-0.1)2.12±0.279.04±0.66 ACTADDMLP0.514.69(+0.72)6.67(+0.05)53.0(-0.1)3.96±0.2413.43±1.42 ITI-CMLP1.013.99(+0.01)6.77(+0.08)52.8(-0.3)4.50±0.3215.06±0.76 Mean-ACTMLP1.014.33(+0.35)7.02(+0.34)52.4(-0.7)1.30±0.377.28±0.88 Linear-ACTMLP1.014.89(+0.92)7.53(+0.85)51.9(-1.2)1.30±0.397.15±0.98 Table 9: Toxicity mitigation for Llama3-8B, results over 5 runs. Similar conclusions as in Table 8 are extracted. LayerBestλPPL Wikipedia↓PPL Mistral-7B↓MMLU↑CLS Toxicity (%)↓0-shot Toxicity (%)↓ Original--9.065.6865.35.8015.00 ACTADDAtention0.39.71(+0.65)5.85(+0.16)65.5(+0.2)5.57±0.4515.73±0.21 ITI-CAtention3.09.48(+0.42)6.17(+0.49)64.7(-0.6)1.60±0.226.53±0.66 Mean-ACTAtention1.09.56(+0.49)6.36(+0.68)64.7(-0.7)1.38±0.175.60±0.34 Linear-ACTAtention1.09.56(+0.49)6.28(+0.60)64.5(-0.8)1.35±0.396.68±0.81 AURAMLP-9.52(+0.45)6.05(+0.37)65.5(+0.2)1.90±0.618.12±0.85 ACTADDMLP- ----- ITI-CMLP1.09.09(+0.03)5.79(+0.11)63.5(-1.9)5.62±0.9615.48±1.16 Mean-ACTMLP0.99.90(+0.84)6.24(+0.55)60.7(-4.6)2.10±0.4810.65±1.02 Linear-ACTMLP0.810.06(+0.99)5.98(+0.29)61.9(-3.4)2.23±0.5310.27±0.97 22 0.000.010.020.030.04 0-shot Toxicity 13 14 15 16 17 18 PPL Wikipedia Original AurA ActAdd ITI-c Mean-AcT Linear-AcT (a) Gemma2-2B 0.0000.0250.0500.0750.1000.125 0-shot Toxicity 13 14 15 16 17 18 PPL Wikipedia Original AurA ActAdd ITI-c Mean-AcT Linear-AcT (b) Gemma2-2B 0.000.010.020.030.040.050.06 0-shot Toxicity 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 PPL Wikipedia Original AurA ITI-c Mean-AcT Linear-AcT (c) Llama3-8B 0.000.050.100.15 0-shot Toxicity 8.0 8.5 9.0 9.5 10.0 10.5 11.0 11.5 12.0 PPL Wikipedia Original AurA ITI-c Mean-AcT Linear-AcT (d) Llama3-8B Figure 11:ACT achieves the best conditioning atλ= 1on Gemma2-2B and Llama3-8B. We show theλsweeps for toxicity mitigation on Gemma2-2B. In gray we show the PPL+1 interval considered to be the maximum loss in PPL we can assume. The bold markers are the results reported in Table 2. For clarity, we only show the experiments that yielded best results reported in Table 2. The full results are shown in Table 8. KEXTENDED RESULTS ON CONCEPT INDUCTION ONLLMS Tables 10 to 12 contain examples of generated sentences by Gemma2-2B intervened for concept induction with Linear-ACT and ITI-C. These results complement those presented in Section 4.2. Note the more gradual increase of concept with Linear-ACT, reaching strong and consistent condi- tioning atλ= 1with great quality. ITI-Calso performs well on concept induction, however the choice ofλis less clear. For example, forλ= 10it fails for conceptCloudwhile it works well for Football. 23 Table 10: Generations at differentλinducing conceptFootball. ConceptMethodλGeneration FootballITI-C0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever FootballITI-C5.0Once upon a time, the only way to watch football was on TV. It wasn’t just about watching your team play either – it was also about getting together with mates and having an afternoon of footy in front of the telly at home or down at the local club for some beer FootballITI-C10.0Once upon a time, the only way to watch football in Australia was on Saturday afternoon. And it wasn’s even just for kids and mates at local clubs or schoolboys matches – no-one would have ever heard of Aussie Rules Football! 1908 saw an associ- ation called FootballLinear-ACT0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever FootballLinear-ACT0.5Once upon a time, there was an amazing woman named Sarah. She had the most beautiful smile and kindest heart you could ever imagine! Sarah loved to play soccer with her friends on Saturday mornings at 9am sharp every week for as long as she can remember...until one day when FootballLinear-ACT1.0Once upon a time, the only way to watch football was on TV. The game of soccer had been played in England since 1863 and by the early twentieth century it became one of Britain’s most popular sports with over three million spectators attending matches each season (and many more Table 11: Generations at differentλinducing conceptFlower. ConceptMethodλGeneration FlowerITI-C0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever FlowerITI-C5.0Once upon a time, there was an artist who lived in the land of flowers. She had beautiful hair and wore it like a crown on her head. Her name is called “Flower”. She loved to paint pictures with water colours because she thought that they were very pretty. One day when FlowerITI-C10.0Once upon a time, there was an artist who lived in the land of flowers. She had beautiful and fragrant petals that she called ‘flowers’. She also has another name for her petal which is known as “petals”. I have no idea what it means but i think you can see FlowerLinear-ACT0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever FlowerLinear-ACT0.5Once upon a time, there was an old man who lived in the forest. He had no children and so he decided to grow some flowers for himself instead of having them on his own body! The flower is called “the red-eyed lily” because it has petals that are bright pink FlowerLinear-ACT1.0Once upon a time, there was an egg-shaped seed that grew into the first true leaves. These tiny green structures are called sepals and they grow in clusters at the base of each flower. The petals may be white or pink with 5 to 8 m long stalks; these will 24 Table 12: Generations at differentλinducing conceptCloud. ConceptMethodλGeneration CloudITI-C0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever CloudITI-C5.0Once upon a time, the only way to get your cloud-based computing infrastruc- ture was by using an existing provider. But now that we’ve got so many of them in our sky (literally), it can be difficult for us cloudshifters and meteorologists to tell one from another! CloudITI-C10.0Once upon a time, the clouds were named by cloud-cloud optical properties. These are called cirrus or altocumulus and middle cir cir circir (Alto Alto Cu- mulus) Cloud family:. CloudLinear-ACT0.0Once upon a time, there was an old man who lived in the forest. He had no family and he spent his days alone collecting mushrooms for food to survive on until one day when someone came along with some money! This person gave him all of their savings so that they could live together forever CloudLinear-ACT0.5Once upon a time, there was an island called the Isle of Skye. The weather on this isle is very unpredictable and can change from sunny to stormy in minutes! The clouds are always moving around with no real pattern or structure as they do not have any defined tops like those found over land CloudLinear-ACT1.0Once upon a time, there was an island called the Bermuda. It is located in middle latitudes of both hemispheres and has been known to be associated with high- level clouds (cirro- or alto-transparent). The altitude ranges from about 10,0 LEXPERIMENTAL DETAILS AND EXTENDED RESULTS ON INDUCING TRUTHFULNESS L.1EXPERIMENTAL DETAILS We follow the original experimental protocol for evaluations on the TruthfulQA multiple choice part, as described in Lin et al. (2021). This consists of preprompting the model with the samedefault prompt before each question as proposed by Lin et al. (2021) in Figure 21, which we replicate below for the reader. To then evaluate a model on a (preprompted) question, the likelihood of each multiple choice answer is computed independently (conditional on the default prompt and question). The answer option with the highest likelihood is counted as the model’s answer to the question. 25 Figure 12: Figure 21 from Lin et al. (2021) showing thedefaultpreprompt used for the TruthfulQA multiple choice part. 26 L.2EXTENDED RESULTS L.2.1FULL RESULTS OVER5SEEDS FOR ALL LAYERS Table 13: TruthfulQA results for Gemma2-2B, results over 5 runs. ITI-C, ACTADDand ACT have astrengthparameterλwhich we sweep, reporting for each method the best result (bestλ) in MC1 Accuracy that incurs at least equal performance in MMLU accuracy compared to the best (in terms of MC1 accuracy) of the two ACT methods (see L.2.2, giving0.1%slack). LayerBestλMC1 Accuracy (%)↑MC2 Accuracy (%)↑MMLU Accuracy (%)↑ Original--21.0532.8053.10 AURAMLP-21.20±0.1032.88±0.2252.73±0.07 ACTADDAttention3.022.64±0.0034.64±0.0053.02±0.00 ITI-CAttention5.023.18±0.2836.16±0.3452.10±0.44 Mean-ACTAttention1.021.62±0.0734.08±0.1952.83±0.09 Linear-ACTAttention1.0 21.71±0.1434.47±0.2252.86±0.08 ACTADDAll-LN1.021.42±0.0032.93±0.0051.65±0.00 ITI-CAll-LN4.023.94±0.9636.62±0.8651.37±0.41 Mean-ACTAll-LN1.025.07±0.2038.68±0.3051.81±0.12 Linear-ACTAll-LN1.026.00±0.3240.17±0.2451.47±0.27 ACTADDPost-LN0.822.40±0.0034.27±0.0053.11±0.00 ITI-CPost-LN8.023.16±0.4035.94±0.5551.39±0.45 Mean-ACTPost-LN1.021.93±0.2034.98±0.2552.77±0.10 Linear-ACTPost-LN1.022.45±0.2235.94±0.3652.43±0.20 ACTADDMLP3.023.01±0.0034.76±0.0052.83±0.00 ITI-CMLP2.024.53±0.1137.06±0.3851.39±0.41 Mean-ACTMLP1.021.98±0.1935.18±0.3152.84±0.04 Linear-ACTMLP1.021.93±0.2035.47±0.2552.73±0.19 Table 14: TruthfulQA results for Llama3-8B, results over 5 runs. ITI-C, ACTADDand ACT have astrengthparameterλwhich we sweep, reporting for each method the best result (bestλ) in MC1 Accuracy that incurs at least equal performance in MMLU accuracy compared to the best (in terms of MC1 accuracy) of the two ACT methods (see L.2.2, giving0.1%slack). LayerBestλMC1 Accuracy (%)↑MC2 Accuracy (%)↑MMLU Accuracy Original--25.4640.2765.35 AURAMLP-25.34±0.1540.47±0.2065.37±0.06 ACTADDAttention0.726.19±0.0040.88±0.0065.42±0.00 ITI-CAttention1.027.42±0.3042.01±0.4265.26±0.11 Mean-ACTAttention1.026.73±0.1942.20±0.2465.37±0.06 Linear-ACTAttention1.027.17±0.2342.15±0.3165.33±0.11 ACTADDAll-LN1.025.58±0.0041.00±0.0064.88±0.00 ITI-CAll-LN3.029.65±0.7144.43±0.5664.71±0.22 Mean-ACTAll-LN1.032.88±0.5448.23±0.6464.83±0.14 Linear-ACTAll-LN1.033.22±0.2248.69±0.3464.78±0.15 ACTADDMLP0.525.46±0.0040.64±0.0065.34±0.00 ITI-CMLP2.030.11±0.6045.41±0.2464.71±0.14 Mean-ACTMLP1.026.17±0.2441.27±0.3465.01±0.20 Linear-ACTMLP1.026.41±0.5239.34±0.5460.98±3.14 L.2.2SWEEPINGλFORITI-C ANDACTADD In Figures 13 - 16, we show the results of sweeping the value ofλfor ITI-Cand ACTADDfor both Gemma2-2B and Llama3-8B. For each model, we also indicate the MMLU accuracy of the best ACT method for that model with a horizontal grey dashed line, as this is our point of reference for choosingλfor ITI-Cand ACTADD: we choose the value ofλthat achieves the best MC1 accuracy, while achieving at least equal MMLU accuracy to this grey dotted line (up to a slack of0.1%). For ITI-C, where we see a clear relationship between MMLU and MC1 accuracy asλvaries, we sweepλ∈[1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0,12.0,13.0,14.0,15.0]. For ACTADD, 27 0.220.230.240.250.260.27 MC1 Acc 0.42 0.44 0.46 0.48 0.50 0.52 MMLU Acc Gemma2 Attention ITI Best OT 0.230.240.250.260.270.28 MC1 Acc 0.25 0.30 0.35 0.40 0.45 0.50 MMLU Acc Gemma2 MLP ITI Best OT 0.2150.2200.2250.2300.2350.2400.2450.250 MC1 Acc 0.48 0.49 0.50 0.51 0.52 0.53 MMLU Acc Gemma2 Post-Layernorm ITI Best OT 0.220.230.240.250.260.27 MC1 Acc 0.30 0.35 0.40 0.45 0.50 MMLU Acc Gemma2 Layernorm ITI Best OT Figure 13: Sweepingλfor inducing truthfulness with ITI-Con Gemma2-2B. Left endpoint of line isλ= 1.0, right endpoint of line isλ= 15.0(each point increasingλby1.0). Note this is for1 seed only. where the relationship can be more erratic, we also sweep values<1.0. Here, we sweepλ∈ [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,2.0,3.0,4.0,5.0]. Overall we see thatλcan have a strong impact on performance for ITI-C, but in a different way for each layer and model. In particular, it can decrease MMLU performance to catastrophic levels (more than halving performance on Gemma2-2B for MLP layers and on Llama3-8B for both attention and MLP layers), making it necessary to sweepλto find its value that provides a reliable control method using ITI-Cfor the problem at hand. Similar things can be found about ACTADD(e.g. when interventing upon on all Layernorm layers on Gemma2-2B, Figure 14). 28 0.21000.21250.21500.21750.22000.22250.2250 MC1 Acc 0.514 0.516 0.518 0.520 0.522 0.524 0.526 0.528 0.530 MMLU Acc Gemma2 Attention ActAdd Best OT 0.21000.21250.21500.21750.22000.22250.22500.22750.2300 MC1 Acc 0.5150 0.5175 0.5200 0.5225 0.5250 0.5275 0.5300 0.5325 MMLU Acc Gemma2 MLP ActAdd Best OT 0.2100.2120.2140.2160.2180.2200.2220.224 MC1 Acc 0.48 0.49 0.50 0.51 0.52 0.53 MMLU Acc Gemma2 Post-Layernorm ActAdd Best OT 0.2100.2150.2200.2250.2300.2350.2400.245 MC1 Acc 0.25 0.30 0.35 0.40 0.45 0.50 MMLU Acc Gemma2 Layernorm ActAdd Best OT Figure 14:Sweepingλfor inducing truthfulness with ACTADDon Gemma2-2B. Left endpoint of line isλ=0.1, right endpoint of line isλ=5.0(λ∈ [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,2.0,3.0,4.0,5.0]). Note this is for1seed only. 0.280.290.300.310.320.33 MC1 Acc 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 MMLU Acc Llama3 Attention ITI Best OT 0.280.290.300.310.320.330.34 MC1 Acc 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 MMLU Acc Llama3 MLP ITI Best OT 0.270.280.290.300.310.320.330.34 MC1 Acc 0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65 MMLU Acc Llama3 All-Layernorm ITI Best OT Figure 15: Sweepingλfor inducing truthfulness with ITI-Con Llama3-8B. Left endpoint of line is λ= 1.0, right endpoint of line isλ= 15.0(each point increasingλby1.0). Note this is for1seed only. 29 0.2520.2540.2560.2580.2600.262 MC1 Acc 0.647 0.648 0.649 0.650 0.651 0.652 0.653 0.654 MMLU Acc Llama3 Attention ActAdd Best OT 0.2500.2550.2600.2650.2700.275 MC1 Acc 0.61 0.62 0.63 0.64 0.65 MMLU Acc Llama3 MLP ActAdd Best OT 0.2250.2300.2350.2400.2450.2500.255 MC1 Acc 0.3 0.4 0.5 0.6 MMLU Acc Llama3 All-Layernorm ActAdd Best OT Figure 16:Sweepingλfor inducing truthfulness withACTADDon Llama3-8B. Left endpoint of line isλ=0.1, right endpoint of line isλ=5.0(λ∈ [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0,2.0,3.0,4.0,5.0]). Note this is for1seed only. 30 MEXPERIMENTALDETAILS ANDEXTENDEDRESULTS FORT2I GENERATION Appendix M.1 illustrates the effect of the guidance parameter in SDXL. Appendix M.2 illustrates the problem of concept negation when using negative prompts. Appendix M.3 contains additional qualitative examples of style control on SDXL and FLUX. Appendix M.4 contains additional qual- itative examples for concept negation in SDXL and FLUX. Appendices M.6 and M.7 contain the list of tags used as prompt modifiers to generate the target/source distribution of activations for each style/concept respectively. Appendix M.8 contains details on FLUX’s architecture conditioning. M.1GUIDANCEPARAMETER INEXISTINGDIFFUSIONMODELS We show in Figure 17 the effect of changing the guidance scale parameter in SDXL. While large val- ues lead to effective conditioning, lower values destroy content. This makes guidance non intuitive and harder to use by users. 1.01.52.02.53.03.54.04.55.05.56.0 Figure 17: SDXL withart nouveautags appended to the prompt as described in Appendix M.3 and guidance strength linearly increasing from 1 to 6. Note how for low guidance (left most images) the semantic content is almost completely lost. M.2NEGATIVE PROMPTING Stable diffusion models allow using negative prompts to avoid unwanted elements in the generated images (Rombach et al., 2022; Podell et al.). Here, we show that this method is ineffective at removingpink elephant,white bear, andgorilla. Figures 18 and 19 contain some failure cases of SDXL and Stable Diffusion 3 (Esser et al., 2024) at removing unwanted concepts. Figure 26 and Figure 27 show results intervening SDXL with ACT, showing its effectiveness at removing these concepts with the same prompts. In Figure 28 we show some failure cases at concept negation. Figure 18:SDXL with Negative Prompt.Prompt: “There is a banana and two pieces of cheese on a plate. Apink elephant, gorilla, white bearcannot be seen anywhere.”. Negative prompt: “Apink elephant, gorilla, white bear”. 31 Figure 19:Stable Diffusion 3 with Negative Prompt.Prompt: “2 tier cake with multicolored stars attached to it. Apink elephant, gorilla, white bearcannot be seen anywhere.” Negative prompt: “Apink elephant, gorilla, white bear.”. M.3STYLECONTROL Figures 20 to 22 complement the results shown in Section 5.1. (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 20:SDXL - A plane floating on top of a lake surrounded by mountains.From left to right conditioning strengthλincreases from 0 to 1. Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 2for ITI-C). Linear-ACT succeeds at inducing different styles. Mean-ACT fails at inducingart nouveau. ITI-Cintroduces noise forart nouveau andcyberpunk. 32 (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 21:SDXL - A firetruck with lights on is on a city street.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 2for ITI-C). Mean-ACT fails at inducingimpressionismandart nouveau. ITI-Cachieves the strongest conditioning and generates a noisy image forart nouveau. 33 (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 22:SDXL - A sandwich is placed next to some vegetables.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 2for ITI-C). ITI-Cfails at inducing style progressively (e.g. (c)cyberpunk). 34 (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 23:FLUX - A group of zebra standing next to each other on a dirt field.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for all methods). Linear-ACT is successful at inducing all styles. ITI-Cfails at inducingcyberpunkandanime. 35 (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 24:FLUX - Black cat with green eyes sitting in a bathroom sink.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for all methods). ACT’s conditioning is weak forsketchandwatercolor. ITI-Cfails at inducingcyberpunk. 36 (a) Anime(b) Art Nouveau (c) Cyberpunk(d) Impressionism (e) Sketch.(f) Watercolor Figure 25:FLUX - A semi truck is driving down a street.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for all methods). ACT is able to preserve the semantics for all styles and we observe only mild conditioning forimpressionismandwatercolor. ITI-Cfails at inducinganimeandcyberpunk. 37 M.4CONCEPT NEGATION (a) Many cars parked on a city street with tall buildings in the background. (b) A cat sitting in front of a large computer monitor. Figure 26:SDXL - Concept negation examples I.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 4for ITI-C). Every 3 rows represent a different concept ingorilla, pink elephant, white bearwhich was negated at the input of the image generator. Mean-ACT and Linear-ACT succeed at removing the unwanted con- cept. ITI-Cfails forgorillaand produces a blurry image forpink elephant. 38 (a) There is a banana and two pieces of cheese on a plate. (b) 2 tier cake with multicolored stars attached to it. Figure 27:SDXL - Concept negation examples I.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 4for ITI-C). Every 3 rows represent a different concept ingorilla, pink elephant, white bearwhich was negated at the input of the image generator. Linear-ACT and Mean-ACT succeed at removing the negated concepts while ITI-Ctends to modify the semantics of the image. 39 (a) Closeup of a white and yellow vase with a red circle at the bottom. (b) A table topped with bananas next to a coin. Figure 28:SDXL - Concept negation examples I (failures)Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 4for ITI-C). Every 3 rows represent a different concept ingorilla, pink elephant, white bearwhich was negated at the input of the image generator. While Mean-ACT and Linear-ACT are successful at removing the concept, there is sometimes a change in semantics of the image for the maximum strength. ITI-Cat best strength (λ= 4) changes semantics for all concepts. 40 (a) 2 tier cake with multicolored stars attached to it. (b) A table topped with bananas next to a coin. Figure 29:FLUX - Concept negation examples I.Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 5for ITI-C). Every 3 rows represent a different concept ingorilla, pink elephant, white bearwhich was negated at the input of the image generator. Linear-ACT removes the negated concepts except forwhite bearin (a). ITI-C is effective at “best” (λ= 5). At high strengths, Linear-ACT and ITI-Calso affect other image semantics. 41 (a) There is a banana and two pieces of cheese on a plate. (b) A sandwich is placed next to some vegetables. Figure 30:FLUX - Concept negation examples I (Failures)Rightmost column corresponds to the best strength found in Figure 6 (λ= 1for ACT andλ= 5for ITI-C). Every 3 rows represent a different concept ingorilla, pink elephant, white bearwhich was negated at the input of the image generator. ACT does not removewhite bear, and fails to removegorillain (b). For highλ, Linear-ACT modifies the semantics of the image. ITI-Cremoves the unwanted concept forλ= 5. 42 M.5DETAILEDRESULTS 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (a) Anime 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (b) Art Nouveau 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (c) Cyberpunk 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (d) Impressionism 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (e) Sketch 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (f) Watercolor Figure 31:Style induction.For each style (a-f) and model (left-right), we show the 0-shot classifi- cation score for the style being present in the generated images (top) and the ClipScore to track how much generated images deviate from the unconditional prompt (bottom). The gray area indicates images that have lost their semantic content. 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (a) Gorilla 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (b) Pink elephant 0.11.010.0 0.00 0.25 0.50 0.75 1.00 0-shot(style) ( → ) λ = 1 SDXL-Lightning 0.11.010.0 λ = 1 FLUX.1.Schnell 0.11.010.0 Intervention strengthλ 0.000 0.075 0.150 0.225 0.300 ClipScore ( → ) λ = 1 Noisy images area ITI Linear-AcT Mean-AcT 0.11.010.0 Intervention strengthλ λ = 1 Noisy images area (c) White bear Figure 32:Concept negation.For each concept (a-c) and model (left-right), we show the 0-shot classification score for the concept being present in the generated images (top) and the ClipScore (bottom) to track how much generated images deviate from the unconditional prompt. The gray area indicates images that have lost their semantic content. 43 M.6STYLE PROMPTS Table 15: List of tags generated with Llama-8B-instruct (right) to induce different styles (left). Animeanime style, large expressive eyes, stylized hair, bold outlines, simplified colors, dynamic perspective, exaggerated features, angular shapes, chibis, manga inspired, emotive facial expressions, action sequences, speed lines, cell shading, graphic backgrounds, vibrant palettes Art nouveauArt Nouveau, Alphonse Mucha, Gustav Klimt, flowing lines, organic shapes, floral motifs, geometric patterns, ornamental designs, Jugendstil, Secessionism, symbolism, female figures, gold leaf, intricate details, turn of the century art, early 20th century Impressionismimpressionism, Claude Monet, brush strokes, light, color, outdoor scenes, water lilies, haystacks, Rouen Cathedral, reflections, nature, atmospheric, vibrant colors, visible textures, 19th century art, French impressionism Cyberpunkcyberpunk, neon lights, urban jungles, high-tech architecture, augmented reality, AI technology, biopunk, futuristic cities, post-apocalyptic scenes, digital hacking, megacorporations, androids, dystopian societies, cybernetic enhancements, chromed details, glowing neon signs, rain-soaked streets Photorealismphotorealism, hyperrealism, optical precision, photographic quality, fine detail, lifelike textures, realistic lighting, accurate perspective, human figures, still life, cityscapes, landscapes, skin tones, reflections and shadows, everyday objects, documentary style art, contemporary realism Sketchsketches, pencil drawing, charcoal sketches, ink illustrations, gestural lines, quick studies, figure drawing, perspective sketching, urban sketching, landscape sketches, still life drawings, sketchbook art, doodles, minimalist lines, expressive mark-making, observational drawing Watercolorwatercolor style, transparent media, wet-on-wet application, dry brush strokes, soft blending, delicate touches, gentle shading, luminous hues, atmospheric lighting, ethereal quality, subtle textures, color gradients, painterly aesthetics, fluid paint behavior, watercolor paper texture 44 M.7CONCEPT PROMPTS Table 16: List of tags generated with Llama-8B-instruct (right) to induce different concepts (upper left) or to prompt models not to generate them (lower left). Pink elephanta pink elephant. containing a pink elephant. with a pink elephant in plain view. and a pink elephant. it displays a pink elephant. featuring a pink elephant. in addition to a pink elephant. and also a pink elephant. and a pink elephant as well. the pink elephant can be clearly seen. Gorillaa gorilla. containing a gorilla. with a gorilla in plain view. and a gorilla. it displays a gorilla. featuring a gorilla. in addition to a gorilla. and also a gorilla. and a gorilla as well. the gorilla can be clearly seen. White beara white bear. containing a white bear. with a white bear in plain view. and a white bear. it displays a white bear. featuring a white bear. in addition to a white bear. and also a white bear. and a white bear as well. the white bear can be clearly seen. No pink elephantwithout a pink elephant. not containing a pink elephant. without a pink elephant in plain view. and a pink elephant that cannot be seen. it does not display a pink elephant. not featuring a pink elephant. lacking a pink elephant. and not a pink elephant. and a pink elephant is missing. the pink elephant cannot be seen. No gorillawithout a gorilla. not containing a gorilla. without a gorilla in plain view. and a gorilla that cannot be seen. it does not display a gorilla. not featuring a gorilla. lacking a gorilla. and not a gorilla. and a gorilla is missing. the gorilla cannot be seen. No white bearwithout a white bear. not containing a white bear. without a white bear in plain view. and a white bear that cannot be seen. it does not display a white bear. not featuring a white bear. lacking a white bear. and not a white bear. and a white bear is missing. the white bear cannot be seen. M.8DETAILS ONFLUXCONDITIONING FLUX’s diffusion architecture 4 is based on the transformer architecture (Vaswani et al., 2017). Con- cretely, it is composed ofNconsecutive multi-modal fusion transformer residual blocks followed byMuni-modal transformer residual blocks. We found that the most effective strategy for strong conditioning is to intervene upon the output of all blocks. However, we found that conditioning blocks closest to the output tends to deteriorate the generated images. Thus, we condition all theN multi-modal blocks and the first15uni-modal blocks. 4 https://blackforestlabs.ai/announcing-black-forest-labs/ 45