Paper deep dive

LEACE: Perfect Linear Concept Erasure in Closed Form

Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, Stella Biderman

Year: 2023Venue: NeurIPS 2023Area: Representation AnalysisType: EmpiricalEmbeddings: 166

Models: BERT, GPT-2, LLaMA, Pythia

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/12/2026, 8:22:06 PM

Summary

The paper introduces LEAst-squares Concept Erasure (LEACE), a closed-form method for removing specific features from embeddings to improve fairness and interpretability in machine learning models. By proving that linear guardedness is equivalent to the equality of class-conditional means, the authors derive a surgical transformation that minimizes the mean squared distance from original features. They also propose 'concept scrubbing' to apply LEACE across layers of deep neural networks, demonstrating its effectiveness in reducing gender bias in BERT and measuring part-of-speech reliance in LLMs.

Entities (4)

LEACE · method · 100%BERT · model · 99%Concept Scrubbing · procedure · 95%Linear Guardedness · concept · 95%

Relation Signals (3)

Concept Scrubbing → utilizes → LEACE

confidence 98% · concept scrubbing, which sequentially applies LEACE to the activations at each layer of a deep network.

LEACE → erases → Concept

confidence 95% · LEACE, a closed-form method which provably prevents all linear classifiers from detecting a concept

LEACE → reducesbiasin → BERT

confidence 95% · demonstrating the superiority of LEACE for erasing gender bias from BERT embeddings

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Concept erasure aims to remove specified features from an embedding. It can improve fairness (e.g. preventing a classifier from using gender or race) and interpretability (e.g. removing a concept to observe changes in model behavior). We introduce LEAst-squares Concept Erasure (LEACE), a closed-form method which provably prevents all linear classifiers from detecting a concept while changing the embedding as little as possible, as measured by a broad class of norms. We apply LEACE to large language models with a novel procedure called "concept scrubbing," which erases target concept information from every layer in the network. We demonstrate our method on two tasks: measuring the reliance of language models on part-of-speech information, and reducing gender bias in BERT embeddings. Code is available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

165,839 characters extracted from source content.

Expand or collapse full text

[]algorithmic LEACE: Perfect linear concept erasure in closed form Nora Belrose1 David Schneider-Joseph1 Shauli Ravfogel2 Ryan Cotterell3 Edward Raff4 Stella Biderman1,4 1EleutherAI 2Bar-Ilan University 3ETH Zürich 4Booz Allen Hamilton nora,stella@eleuther.ai david@davidsj.com Abstract Concept erasure aims to remove specified features from an embedding. It can improve fairness (e.g. preventing a classifier from using gender or race) and interpretability (e.g. removing a concept to observe changes in model behavior). We introduce LEAst-squares Concept Erasure (LEACE), a closed-form method which provably prevents all linear classifiers from detecting a concept while changing the embedding as little as possible, as measured by a broad class of norms. We apply LEACE to large language models with a novel procedure called concept scrubbing, which erases target concept information from every layer in the network. We demonstrate our method on two tasks: measuring the reliance of language models on part-of-speech information, and reducing gender bias in BERT embeddings. Our code is available at https://github.com/EleutherAI/concept-erasure. 1 Introduction The ability to prevent a machine learning system from using a specified concept is important for fairness and interpretability. Popular notions of fairness require that protected attributes should not causally affect predictions [22, 26], and interpretability research often estimates the causal effect of a concept by attempting to remove it from a model’s internal activations [10, 30, 25, 5, 18]. What it means for a model ℳMM to “use” a concept ZZZ is often vague and application-specific, but a necessary condition is that its outputs—and therefore its inputs and hidden states—should have significant mutual information with ZZZ.111This follows from the fact that causal dependence is a special kind of statistical dependence [28]. By the data processing inequality, ℳMM’s output can’t have any more information about ZZZ than its input or hidden states. Concept erasure leverages this fact to limit ℳMM’s use of ZZZ without finetuning or inspecting its parameters. Instead, we edit the input or hidden states XXX used by ℳMM to minimize the predictive VV-information I⁢(X→Z)subscript→XZI_V(X→Z)Icaligraphic_V ( X → Z ) [43], a tractable lower bound on the mutual information I⁢(X;Z)XZI(X;Z)I ( X ; Z ) which measures the degree to which classifiers from the family VV can predict ZZZ. Intuitively, if no classifier in VV can outperform a constant function at predicting ZZZ—a condition known as guardedness—then ℳMM can’t use ZZZ either, at least if VV is expressive enough relative to ℳMM. In this work, we improve upon existing concept erasure techniques using a theory-driven approach. We focus on the case where VV is the set of linear classifiers, and prove a previously unnoticed equivalence: a classification task is linearly guarded if and only if every class has exactly the same mean feature vector (§ 3). Leveraging this equivalence, we derive a simple necessary and sufficient condition for an affine transformation to produce linearly guarded features. We then identify the unique surgical transformation in this family—the one that minimizes the mean squared distance from the original features with respect to all norms induced by inner products, including the popular Euclidean and Mahalanobis norms. We name it LEAst-squares Concept Erasure (LEACE) (§ 4). While prior work has focused on preventing linear models from leveraging ZZZ, we aim to erase concepts from deep neural networks as well. Interpretability research has shown that networks can be usefully described as encoding features in linear subspaces [11, 24, 41], suggesting that fundamentally nonlinear methods may not be necessary for successful erasure in DNNs. In light of this, we introduce a simple procedure called concept scrubbing (§ 6), which sequentially applies LEACE to the activations at each layer of a deep network. We empirically validate our proposals, demonstrating the superiority of LEACE for erasing gender bias from BERT embeddings (§ 5.2), and using concept scrubbing to measure the extent to which large language models use part-of-speech information (§ 6). 2 Preliminaries Consider a k-class classification task over jointly defined random vectors XXX (the input data) and ZZZ (the one-hot labels), with XXX of finite first moment and taking values in ℝdsuperscriptℝR^dblackboard_Rd, and ZZZ taking values in =∈0,1k|‖1=1conditional-setsuperscript01subscriptnorm11Z=\z∈\0,1\^k\ |\ \|z\|_1=1\Z = z ∈ 0 , 1 k | ∥ z ∥1 = 1 222We frequently use the integer j≤kj≤ kj ≤ k to refer to the element of ZZ which is 1111 at the jthsuperscriptthj^thjth index and 00 elsewhere. with each ℙ⁢(Z=j)>0ℙZ0P(Z=j)>0blackboard_P ( Z = j ) > 0. Let η⁢(⋅;):ℝd→ℝk:⋅→superscriptℝsuperscriptℝη(·; θ):R^d ^kη ( ⋅ ; italic_θ ) : blackboard_Rd → blackboard_Rk be a predictor chosen from a function class =η⁢(⋅;)|∈Θconditional-set⋅ΘV=\η(·; θ)\ |\ θ∈ \V = η ( ⋅ ; italic_θ ) | italic_θ ∈ Θ (presumed to contain all constant functions) so as to minimize the expectation ⁢[ℒ⁢(η⁢(X),Z)]delimited-[]ℒXZE [ [rgb]0,0,0 [named]pgfstrokecolorrgb% 0,0,0 @color@gray@stroke0 @color@gray@fill0L(η(% X),Z) ]blackboard_E [ L ( η ( X ) , Z ) ] of some ℒ:ℝk×→[0,∞):ℒ→superscriptℝ0 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L:R^% k×Z→[0,∞)L : blackboard_Rk × Z → [ 0 , ∞ ) in a class LL of loss functions. We borrow the concept of guardedness from Ravfogel et al. [33], who define it in terms of VV-information [43]. We opt for a slightly more general definition here, which is equivalent to theirs in the case of cross-entropy loss (see Appendix G). Definition 2.1 (Guardedness). Let XXX, ZZZ, VV, and LL be as defined above, and let χ be the set of all random vectors of finite first moment taking values in ℝdsuperscriptℝR^dblackboard_Rd, jointly defined with ZZZ. We say XXX (,)−limit-from(V, L)-( V , L ) -guards ZZZ if, for all losses ℒ∈ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L∈ LL ∈ L, it maximizes the minimum expected loss: X∈argmaxX′∈χinf∈Θ⁢[ℒ⁢(η⁢(X′;),Z)].XsubscriptargmaxsuperscriptX′subscriptinfimumΘdelimited-[]ℒsuperscriptX′ZX∈ argmax\>_ subarraycX^% ∈χ subarray _ θ∈ \>E% [ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η(% X ; θ),Z) ].X ∈ start_BIGOP argmax end_BIGOPstart_ARG start_ROW start_CELL X′ ∈ χ end_CELL end_ROW end_ARG infbold_italic_θ ∈ Θ blackboard_E [ L ( η ( X′ ; italic_θ ) , Z ) ] . In other words, its conditional distribution ℙ⁢(X∣Z=⋅)ℙconditionalXZ⋅P(X Z=·)blackboard_P ( X ∣ Z = ⋅ ) is among the worst possible distributions for predicting ZZZ from XXX using a predictor of the form η⁢(⋅;)∈⋅η(·; θ) η ( ⋅ ; italic_θ ) ∈ V and a loss function in LL. Definition 2.2 (Trivially Attainable Loss). The trivially attainable loss for labels ZZZ and loss ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL is the lowest possible expected loss available to a constant predictor η⁢()=η(x)=bη ( x ) = b: Lτ=inf∈ℝk⁢[ℒ⁢(,Z)]subscriptsubscriptinfimumsuperscriptℝdelimited-[]ℒZL_τ= _b ^kE[ [rgb]0,0,0% [named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(b,Z)]Litalic_τ = infbold_b ∈ blackboard_Rk blackboard_E [ L ( b , Z ) ] We will sometimes write it Lτ(Z,ℒ)superscriptsubscriptZℒL_τ^(Z, [rgb]0,0,0 [named]pgfstrokecolor% rgb0,0,0 @color@gray@stroke0 @color@gray@fill0L)Litalic_τ( Z , L ) in cases of possible ambiguity. If there is a specific constant predictor actually achieving this loss, we call it the trivial predictor ητ=ητ(Z,ℒ)subscriptsuperscriptsubscriptZℒ _τ= _τ^(Z,L)ηitalic_τ = ηitalic_τ( Z , L ). We examine this problem in the important case of loss functions ℒ:ℝk×→[0,∞):ℒ→superscriptℝ0 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L:R^% k×Z→[0,∞)L : blackboard_Rk × Z → [ 0 , ∞ ) which are convex in the prediction η⁢()η(x)η ( x ), and linear predictors that take the functional form η⁢(;,)=+η(x;b,W)=b+Wxη ( x ; b , W ) = b + Wx, for some bias ∈ℝksuperscriptℝb ^kb ∈ blackboard_Rk and weight matrix ∈ℝk×dsuperscriptℝW ^k× dW ∈ blackboard_Rk × d. Definition 2.3 (Linear Guardedness). If XXX (,)(V, L)( V , L )-guards ZZZ, where LL is the class of nonnegative loss functions which are convex in their first argument, and VV is the class of linear predictors η⁢()=+η(x)=b+Wxη ( x ) = b + Wx, we say that XXX linearly guards ZZZ. 3 Theoretical Results Our primary theoretical result is that the following conditions are all equivalent: 1. The data XXX linearly guards the labels ZZZ. (Definition 2.3) 2. For all convex losses ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL, the trivially attainable loss is optimal on (X,Z)XZ(X,Z)( X , Z ). (Definition 2.2) 3. The class-conditional mean vectors ⁢[X∣Z=i]delimited-[]conditionalXZE[X Z=i]blackboard_E [ X ∣ Z = i ] are equal to the unconditional mean ⁢[X]delimited-[]XE[X]blackboard_E [ X ]. 4. Every component of XXX has zero covariance with every component of ZZZ. 5. Every linear classifier evaluated on XXX exhibits statistical parity w.r.t. ZZZ. (App. C) The equivalence of conditions 1, 2, and 5 is relatively straightforward to show, and the relevant theorems can be found in Appendices B and C. The other equivalences are proven below (cond. 3 ↔ ↔ cond. 2 in § 3.1 and § 3.2); cond. 3 ↔ ↔ 4 in § 3.3). 3.1 Equality of Class Centroids Implies Linear Guardedness The following result establishes the implication from condition 3 to condition 2. Theorem 3.1. Suppose ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL is convex in the linear prediction η. Then if each class-conditional mean ⁢[X∣Z=i]delimited-[]conditionalXZE [X Z=i ]blackboard_E [ X ∣ Z = i ] is equal to ⁢[X]delimited-[]XE [X ]blackboard_E [ X ], the trivially attainable loss cannot be improved upon. Proof. Let η⁢()=+η(x)=b+Wxη ( x ) = b + Wx be any linear predictor. By Jensen’s inequality,333Specifically, its generalization to convex functions over ℝksuperscriptℝR^kblackboard_Rk. See [12] p. 76. the loss with η evaluated on XXX is lower bounded by the loss with η evaluated on the unconditional mean of the data ⁢[X]delimited-[]XE [X ]blackboard_E [ X ]: ⁢[ℒ⁢(η,Z)]=Zdelimited-[]ℒZsubscriptZ [ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,Z) ]=E_Zblackboard_E [ L ( η , Z ) ] = blackboard_EZ [⁢[ℒ⁢(η,Z)|Z]]delimited-[]delimited-[]conditionalℒZ [E [ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,Z) |Z ] ][ blackboard_E [ L ( η , Z ) | Z ] ] ≥ZabsentsubscriptZ _Z≥ blackboard_EZ [ℒ⁢(⁢[η|Z],Z)]delimited-[]ℒdelimited-[]conditionalZZ [ [rgb]0,0,0 [named]pgfstrokecolorrgb% 0,0,0 @color@gray@stroke0 @color@gray@fill0L (% E [η |Z ],Z ) ][ L ( blackboard_E [ η | Z ] , Z ) ] (Jensen’s inequality) =ZabsentsubscriptZ =E_Z= blackboard_EZ [ℒ⁢(+⁢[X|Z],Z)]delimited-[]ℒdelimited-[]conditionalXZZ [ [rgb]0,0,0 [named]pgfstrokecolorrgb% 0,0,0 @color@gray@stroke0 @color@gray@fill0L (% b+WE [X |Z ],% Z ) ][ L ( b + W blackboard_E [ X | Z ] , Z ) ] (linearity of η) =ZabsentsubscriptZ =E_Z= blackboard_EZ [ℒ⁢(+⁢[X],Z)].delimited-[]ℒdelimited-[]XZ [ [rgb]0,0,0 [named]pgfstrokecolorrgb% 0,0,0 @color@gray@stroke0 @color@gray@fill0L (% b+WE [X ],Z )% ].[ L ( b + W blackboard_E [ X ] , Z ) ] . (by assumption) This in turn is the loss of the constant predictor η′⁢()=+⁢[X]superscript′delimited-[]Xη (x)=b+WE [X% ]η′ ( x ) = b + W blackboard_E [ X ]. Since the trivially attainable loss is the best that can be achieved by a constant predictor, and every predictor’s loss is lower bounded by that of some constant predictor, we cannot improve upon the trivially attainable loss. ∎ Intuitively, this shows that the classifier’s expected loss is lower-bounded by the loss it would receive if each data point were replaced with the centroid of its class. But, if these centroids are all equal, the loss can’t be any lower than what we’d get if every data point were replaced with the global mean ⁢[X]delimited-[]XE[X]blackboard_E [ X ]. In that case, the data points are indistinguishable and we can’t do better than =0W=0W = 0. 3.2 Linear Guardedness Implies Equality of Class Centroids We now prove the implication from condition 2 to condition 3. Condition 2 applies when the trivially attainable loss is optimal for all convex losses, including cross-entropy loss in particular. And if it holds for cross-entropy loss, we now show that condition 3—the class centroids are equal—must follow. First a more general lemma: Lemma 3.2. Suppose ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL has bounded partial derivatives, which when off-category never vanish and do not depend on the category, i.e. ∂ℒ⁢(η,z1)/∂ηi=∂ℒ⁢(η,z2)/∂ηi≠0ℒsubscript1subscriptℒsubscript2subscript0∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,z_1)% /∂ _i=∂ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,z_2)/∂ _i≠ 0∂ L ( η , z1 ) / ∂ ηitalic_i = ∂ L ( η , z2 ) / ∂ ηitalic_i ≠ 0 for all categories z1,z2≠isubscript1subscript2z_1,z_2≠ iz1 , z2 ≠ i. If ⁢[ℒ⁢(η,Z)]delimited-[]ℒZE [ [rgb]0,0,0 [named]pgfstrokecolorrgb% 0,0,0 @color@gray@stroke0 @color@gray@fill0L(η,% Z) ]blackboard_E [ L ( η , Z ) ] is minimized among linear predictors by the constant predictor η⁢()=∗+∗⁢superscriptsuperscriptη(x)=b^*+W^*xη ( x ) = b∗ + W∗ x with ∗=superscript0W^*=0W∗ = 0, then each class-conditional mean ⁢[X|Z=i]delimited-[]conditionalXZE [X|Z=i ]blackboard_E [ X | Z = i ] is equal to ⁢[X]delimited-[]XE [X ]blackboard_E [ X ]. Proof. The first-order optimality condition on the ithsuperscriptthi^thith component of our parameters bb and WW yields the equations: ⁢[∂ℒ⁢(η,Z)∂ηi⋅∂ηi∂bi]=0and⁢[∂ℒ⁢(η,Z)∂ηi⋅∂ηi∂]=,formulae-sequencedelimited-[]⋅ℒZsubscriptsubscriptsubscript0anddelimited-[]⋅ℒZsubscriptsubscriptsubscript0E [ ∂ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,Z)∂ _i· ∂ _i% ∂ b_i ]=0 [ % ∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,% Z)∂ _i· ∂ _i _% i ]=0,blackboard_E [ divide start_ARG ∂ L ( η , Z ) end_ARG start_ARG ∂ ηitalic_i end_ARG ⋅ divide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ bitalic_i end_ARG ] = 0 and blackboard_E [ divide start_ARG ∂ L ( η , Z ) end_ARG start_ARG ∂ ηitalic_i end_ARG ⋅ divide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ Wbold_i end_ARG ] = 0 , (1) where we have used the boundedness of ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL’s partial derivative and the finite first moment of ∂ηi∂bi=1subscriptsubscript1 ∂ _i∂ b_i=1divide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ bitalic_i end_ARG = 1 and ∂ηi∂=XsubscriptsubscriptX ∂ _i _i=Xdivide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ Wbold_i end_ARG = X to justify (via the Dominated Convergence Theorem) interchanging the derivative with the expectation. Since η is constant over all values of XXX, and ∂ηi∂bi=1subscriptsubscript1 ∂ _i∂ b_i=1divide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ bitalic_i end_ARG = 1, the first equation in (1) reduces to: ℙ⁢(Z=i)⁢∂ℒ⁢(η,i)∂ηi+ℙ⁢(Z≠i)⁢∂ℒ(η,≠i)∂ηi=0,P(Z=i) ∂ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,i)∂ _i+P(Z≠ i) % ∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,≠ i)% ∂ _i=0,blackboard_P ( Z = i ) divide start_ARG ∂ L ( η , i ) end_ARG start_ARG ∂ ηitalic_i end_ARG + blackboard_P ( Z ≠ i ) divide start_ARG ∂ L ( η , ≠ i ) end_ARG start_ARG ∂ ηitalic_i end_ARG = 0 , (2) where ∂ℒ(η,≠i)∂ηi ∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,≠ i% )∂ _idivide start_ARG ∂ L ( η , ≠ i ) end_ARG start_ARG ∂ ηitalic_i end_ARG is an abuse of notation denoting the off-category partial derivative, emphasizing its independence of the category ZZZ. Similarly, the constancy of η and the fact that ∂ηi∂=XsubscriptsubscriptX ∂ _i _i=Xdivide start_ARG ∂ ηitalic_i end_ARG start_ARG ∂ Wbold_i end_ARG = X reduces the second equation in (1) to: ℙ⁢(Z=i)⁢∂ℒ⁢(η,i)∂ηi⋅⁢[X|Z=i]+ℙ⁢(Z≠i)⁢∂ℒ(η,≠i)∂ηi⋅⁢[X|Z≠i]=.P(Z=i) ∂ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,i)∂ _i·E [X% |Z=i ]+\\ P(Z≠ i) ∂ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η,≠ i)∂ _i·% E [X |Z≠ i ]=0.blackboard_P ( Z = i ) divide start_ARG ∂ L ( η , i ) end_ARG start_ARG ∂ ηitalic_i end_ARG ⋅ blackboard_E [ X | Z = i ] + blackboard_P ( Z ≠ i ) divide start_ARG ∂ L ( η , ≠ i ) end_ARG start_ARG ∂ ηitalic_i end_ARG ⋅ blackboard_E [ X | Z ≠ i ] = 0 . (3) Solving for ℙ⁢(Z=i)⁢∂ℒ⁢(η,i)∂ηiℙZℒsubscriptP(Z=i) ∂ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η,i)∂ _iblackboard_P ( Z = i ) divide start_ARG ∂ L ( η , i ) end_ARG start_ARG ∂ ηitalic_i end_ARG in (2) and substituting in (3) gives us: ℙ⁢(Z≠i)⁢∂ℒ(η,≠i)∂ηi⋅(⁢[X|Z≠i]−⁢[X|Z=i])=.P(Z≠ i) ∂ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η,≠ i)∂ _i·% (E [X |Z≠ i ]-% E [X |Z=i ] )=0.blackboard_P ( Z ≠ i ) divide start_ARG ∂ L ( η , ≠ i ) end_ARG start_ARG ∂ ηitalic_i end_ARG ⋅ ( blackboard_E [ X | Z ≠ i ] - blackboard_E [ X | Z = i ] ) = 0 . If ℙ⁢(Z≠i)=0ℙZ0P(Z≠ i)=0blackboard_P ( Z ≠ i ) = 0, then ⁢[X]=⁢[X|Z=i]delimited-[]Xdelimited-[]conditionalXZE[X]=E[X|Z=i]blackboard_E [ X ] = blackboard_E [ X | Z = i ] is trivially true. Otherwise, using the non-vanishingness of the off-category partial derivative ∂ℒ(η,≠i)∂ηi ∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,≠ i% )∂ _idivide start_ARG ∂ L ( η , ≠ i ) end_ARG start_ARG ∂ ηitalic_i end_ARG, division yields the equivalence of ⁢[X|Z=i]delimited-[]conditionalXZE [X |Z=i ]blackboard_E [ X | Z = i ] to ⁢[X|Z≠i]delimited-[]conditionalXZE [X |Z≠ i ]blackboard_E [ X | Z ≠ i ], and hence to the unconditional mean ⁢[X]delimited-[]XE [X ]blackboard_E [ X ]. ∎ We now show that Lemma 3.2 applies to the widely used cross entropy loss: Theorem 3.3. If the class probabilities ℙ⁢(Z=j)ℙZP(Z=j)blackboard_P ( Z = j ) are all nonzero, and the trivially obtainable loss is optimal when ℒ⁢(η,z)=−log⁡exp⁡(ηz)∑i=1kexp⁡(ηi)ℒsubscriptsuperscriptsubscript1subscript [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,z)=-% ( _z) _i=1^k ( _i)L ( η , z ) = - log divide start_ARG exp ( ηitalic_z ) end_ARG start_ARG ∑i = 1k exp ( ηitalic_i ) end_ARG, then each class has the same mean ⁢[X|Z=z]delimited-[]conditionalXZE [X |Z=z ]blackboard_E [ X | Z = z ]. Proof. In this case, the trivial predictor ητ⁢(Z)j=log⁡(ℙ⁢(Z=j))subscriptsubscriptZℙZ _τ(Z)_j= (P(Z=j))ηitalic_τ ( Z )j = log ( blackboard_P ( Z = j ) ) exists, achieving the trivially obtainable loss, which we have assumed optimal. Furthermore, ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0LL has on-category partial derivative ∂ℒ⁢(η,i)/∂ηi=exp⁡(ηi)/∑j=1kexp⁡(ηj)−1∈(−1,0]ℒsubscriptsubscriptsuperscriptsubscript1subscript110∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,i)/% ∂ _i= ( _i)/ _j=1^k ( _j)-1∈(-1,0]∂ L ( η , i ) / ∂ ηitalic_i = exp ( ηitalic_i ) / ∑j = 1k exp ( ηitalic_j ) - 1 ∈ ( - 1 , 0 ], and nonvanishing off-category partial derivative ∂ℒ(η,≠i)/∂ηi=exp(ηi)/∑j=1kexp(ηj)∈(0,1)∂ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,≠ i)% /∂ _i= ( _i)/ _j=1^k ( _j)∈(0,1)∂ L ( η , ≠ i ) / ∂ ηitalic_i = exp ( ηitalic_i ) / ∑j = 1k exp ( ηitalic_j ) ∈ ( 0 , 1 ), both bounded, so the conditions of Lemma 3.2 apply. ∎ 3.3 Linearly Guarded Labels Have Zero Covariance with the Features The next theorem establishes the equivalence of conditions 3 and 4. Theorem 3.4. Let XXX be a random vector taking values in ℝdsuperscriptℝR^dblackboard_Rd with finite first moment, and ZZZ a random vector taking values in 0,1ksuperscript01\0,1\^k 0 , 1 k with one-hot encoding, with each class probability ℙ⁢(Z=j)ℙZP(Z=j)blackboard_P ( Z = j ) being nonzero. Then the class-conditional means ⁢[X|Z=j]delimited-[]conditionalXZE[X|Z=j]blackboard_E [ X | Z = j ] are all equal to the unconditional mean ⁢[X]delimited-[]XE[X]blackboard_E [ X ] if and only if every component of XXX has zero covariance with every component of ZZZ, i.e. the cross-covariance matrix XZsubscriptXZ _XZΣroman_XZ, whose (i,j)thsuperscriptth(i,j)^th( i , j )th entry is Cov⁢(Xi,Zj)CovsubscriptXsubscriptZCov(X_i,Z_j)Cov ( Xitalic_i , Zitalic_j ), is the zero matrix. Proof. Since ZZZ is one-hot, we can rewrite the (i,j)thsuperscriptth(i,j)^th( i , j )th entry of XZsubscriptXZ _XZΣroman_XZ as: ⁢[Xi⁢Zj]−⁢[Xi]⁢[Zj]=ℙ⁢(Z=j)⁢(⁢[Xi|Z=j]−⁢[Xi]).delimited-[]subscriptXsubscriptZdelimited-[]subscriptXdelimited-[]subscriptZℙZdelimited-[]conditionalsubscriptXZdelimited-[]subscriptX [X_iZ_j]-E[% X_i]E[Z_j]=P(Z=j) (% E[X_i|Z=j]-E[X_i] ).blackboard_E [ Xitalic_i Zitalic_j ] - blackboard_E [ Xitalic_i ] blackboard_E [ Zitalic_j ] = blackboard_P ( Z = j ) ( blackboard_E [ Xitalic_i | Z = j ] - blackboard_E [ Xitalic_i ] ) . As ℙ⁢(Z=j)>0ℙZ0P(Z=j)>0blackboard_P ( Z = j ) > 0, it follows that ⁢[Xi|Z=j]=⁢[Xi]delimited-[]conditionalsubscriptXZdelimited-[]subscriptXE[X_i|Z=j]=E[X_i]blackboard_E [ Xitalic_i | Z = j ] = blackboard_E [ Xitalic_i ] if and only if Cov⁢(Xi,Zj)=0CovsubscriptXsubscriptZ0Cov(X_i,Z_j)=0Cov ( Xitalic_i , Zitalic_j ) = 0. ∎ We have thus established the equivalence of the first four conditions stated earlier. See Appendix C for the last one, on statistical parity. 4 Least-Squares Concept Erasure In Section 3 we saw that XXX linearly guards ZZZ if and only if each component of XXX has zero covariance with each component of ZZZ. We will now characterize the set of affine transformations r⁢()=⁢+r(x)=P x+br ( x ) = P italic_x + b such that r⁢(X)Xr(X)r ( X ) linearly guards ZZZ. Theorem 4.1. Let XXX and ZZZ be random vectors taking values in ℝdsuperscriptℝR^dblackboard_Rd and ℝksuperscriptℝR^kblackboard_Rk respectively, with XXX of finite first moment. Then given some affine function r⁢()=⁢+r( x)=P x+br ( italic_x ) = P italic_x + b, the modified random vector r⁢(X)Xr(X)r ( X ) linearly guards ZZZ if and only if the columns of the cross-covariance matrix XZsubscriptXZ _XZΣroman_XZ are contained in the null space of PP. Proof. From Theorem 3.4 we know that r⁢(X)Xr(X)r ( X ) linearly guards ZZZ if and only if Cov⁢(r⁢(X),Z)CovXZCov(r(X),Z)Cov ( r ( X ) , Z ) is the zero matrix. By the linearity property of cross-covariance, we have: Cov⁢(r⁢(X),Z)=Cov⁢(⁢X+,Z)=⁢Cov⁢(X,Z)=⁢XZ.CovXZCovXZCovXZsubscriptXZ (r(X),Z)=Cov(P% X+b,Z)=PCov(X,% Z)=P _XZ.Cov ( r ( X ) , Z ) = Cov ( P X + b , Z ) = P Cov ( X , Z ) = P Σroman_XZ . Therefore, r⁢(X)Xr(X)r ( X ) linearly guards ZZZ if and only if ker⁡()⊇colsp⁢(XZ)colspsubscriptXZkernel (P) ( _X% Z)ker ( P ) ⊇ colsp ( Σroman_XZ ). ∎ Implications for prior work. Notably, the above theorems imply that three previously proposed methods in the literature, Spectral Attribute Removal (SAL) [36], Mean Projection [17], and Fair PCA [20], are guaranteed to achieve linear guardedness given suitable hyperparameters. See Appendix D for further discussion. 4.1 Derivation of LEACE Theorem 4.1 is a very weak condition, which is far from identifying unique values for PP and bb. In most applications, however, we’d like to make a “small” edit to XXX so that useful information contained in XXX is maximally preserved. We operationalize the notion of a small edit in terms of the mean squared norm ⁢‖r⁢(X)−X‖2subscriptsuperscriptnormXX2E\|r(X)-X\|^2_Mblackboard_E ∥ r ( X ) - X ∥2M defined by some positive-definite inner product MM,444Our proofs also include degenerate “inner products” where MM is singular, and the associated seminorms. which can be thought of as a local quadratic approximation to any measure of divergence between XXX and r⁢(X)Xr(X)r ( X ) (such as Kullback–Leibler divergence, for example). While we are primarily interested in the Euclidean (=M=IM = I) and Mahalanobis (=X+superscriptsubscriptXXM= _XX^+M = Σroman_X+) norms, it will turn out that there is a single erasure function that minimizes all such norms simultaneously. We will see in Section 6 that ensuring edits are small in this sense provides substantial benefit to downstream task performance as compared to other methods which also guard the labels ZZZ. Below, we derive the optimal eraser under the assumption that XXX and ZZZ are centered. Theorem 4.2. Let XXX and ZZZ be centered random vectors taking values in ℝdsuperscriptℝR^dblackboard_Rd and ℝksuperscriptℝR^kblackboard_Rk respectively, each of finite second moment. Let ∈ℝd×dsuperscriptℝM∈ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^d× dM ∈ blackboard_Rd × d be a p.s.d. matrix defining a (possibly degenerate) inner product on ℝdsuperscriptℝR^dblackboard_Rd: ⟨,⟩=T⁢subscriptsuperscript ,y _M=x^TMy⟨ x , y ⟩M = xitalic_T My. Let X∈ℝd×dsubscriptXXsuperscriptℝ _XX ^d× dΣroman_X ∈ blackboard_Rd × d be XXX’s covariance matrix, and XZ∈ℝd×ksubscriptXZsuperscriptℝ _XZ ^d× kΣroman_XZ ∈ blackboard_Rd × k be the cross-covariance matrix of XXX and ZZZ. Let +superscriptA^+A+ denote the Moore-Penrose pseudoinverse of a matrix AA, and let 1/2superscript12A^1/2A1 / 2 be the p.s.d. square root of a p.s.d. matrix AA. Then the objective argmin∈ℝd×d⁢[‖⁢X−X‖2]subject⁢to⁢Cov⁢(⁢X,Z)=subscriptargminsuperscriptℝdelimited-[]subscriptsuperscriptnormXX2subjecttoCovXZ0 argmin\>_ subarraycP∈% R^d× d subarrayE [ \|P% X-X \|^2_M ] \>% to\>\>Cov(PX,Z)=0start_BIGOP argmin end_BIGOPstart_ARG start_ROW start_CELL P ∈ blackboard_Rd × d end_CELL end_ROW end_ARG blackboard_E [ ∥ P X - X ∥2M ] subject to Cov ( P X , Z ) = 0 has the following solution: ∗=−+⁢XZ⁢,superscriptsuperscriptsubscriptsubscriptXZ ^*=I-W^+P_W% _XZW,P∗ = I - W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W , where WW is the whitening transformation (X1/2)+superscriptsuperscriptsubscriptXX12( _XX^1/2)^+( Σroman_X1 / 2 )+ and ⁢XZ=(⁢XZ)⁢(⁢XZ)+subscriptsubscriptXZsubscriptXZsuperscriptsubscriptXZP_W _XZ=(W% _XZ)(W _% XZ)^+Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT = ( W Σroman_XZ ) ( W Σroman_XZ )+ is the orthogonal projection matrix onto colsp⁢(⁢XZ)colspsubscriptXZcolsp(W _XZ)colsp ( W Σroman_XZ ). Proof. See Appendices E.1 and E.2 for two independent proofs of Theorem 4.2. ∎ The above theorem assumes that the random vectors XXX and ZZZ are centered, and does not include a bias term. Below we extend our results to the uncentered case, and derive the optimal bias ∗superscriptb^*b∗. Theorem 4.3. Let XXX and ZZZ be random vectors taking values in ℝdsuperscriptℝR^dblackboard_Rd and ℝksuperscriptℝR^kblackboard_Rk respectively, each of finite second moment. Define MM and ∗superscriptP^*P∗ as in Theorem 4.2 and ∗=⁢[X]−∗⁢[X]superscriptdelimited-[]Xsuperscriptdelimited-[]Xb^*=E[X]-P^*E[X]b∗ = blackboard_E [ X ] - P∗ blackboard_E [ X ]. Then (∗,∗)superscriptsuperscript(P^*,b^*)( P∗ , b∗ ) minimizes ⁢‖⁢X+−X‖2superscriptnormXX2E \|PX+b-X \|^2blackboard_E ∥ P X + b - X ∥2, subject to Cov⁢(⁢X+,Z)=CovXZ0Cov(PX+b,Z)=0Cov ( P X + b , Z ) = 0. Proof. Let ∈ℝd×dsuperscriptℝP ^d× dP ∈ blackboard_Rd × d and define X~=X−⁢[X]~XXdelimited-[]X X=X-E[X]over~ start_ARG X end_ARG = X - blackboard_E [ X ] and =⁢[X]+−⁢[X]delimited-[]Xdelimited-[]Xc=PE[X]+b-E[X% ]c = P blackboard_E [ X ] + b - blackboard_E [ X ]. Then, ⁢‖⁢X+−X‖2subscriptsuperscriptnormXX2 \|PX+b-X% \|^2_Mblackboard_E ∥ P X + b - X ∥2M =⁢‖(⁢X~−X~)+‖2absentsubscriptsuperscriptnorm~X~X2 =E \|(P X- % X)+c \|^2_M= blackboard_E ∥ ( P over~ start_ARG X end_ARG - over~ start_ARG X end_ARG ) + c ∥2M =⁢‖⁢X~−X~‖2+2⁢[⁢X~−X~]T⁢+T⁢absentsubscriptsuperscriptnorm~X~X22superscriptdelimited-[]~X~Xsuperscript =E \|P X- % X \|^2_M+2E [P X% - X ]^TMc+c^TM% c= blackboard_E ∥ P over~ start_ARG X end_ARG - over~ start_ARG X end_ARG ∥2M + 2 blackboard_E [ P over~ start_ARG X end_ARG - over~ start_ARG X end_ARG ]T Mc + citalic_T Mc =⁢‖⁢X~−X~‖2+T⁢,absentsubscriptsuperscriptnorm~X~X2superscript =E \|P X- % X \|^2_M+c^TMc,= blackboard_E ∥ P over~ start_ARG X end_ARG - over~ start_ARG X end_ARG ∥2M + citalic_T Mc , where we have eliminated the middle term because PP is linear and ⁢[X~]=0delimited-[]~X0E[ X]=0blackboard_E [ over~ start_ARG X end_ARG ] = 0. Since MM is p.s.d., our objective is minimized for =0c=0c = 0, i.e. =⁢[X]−⁢[X]delimited-[]Xdelimited-[]Xb=E[X]-PE[X]b = blackboard_E [ X ] - P blackboard_E [ X ]. The problem thus reduces to choosing PP so as to minimize ⁢‖⁢X~−X~‖2subscriptsuperscriptnorm~X~X2E \|P X- X \|^% 2_Mblackboard_E ∥ P over~ start_ARG X end_ARG - over~ start_ARG X end_ARG ∥2M subject to Cov⁢(⁢X+,Z)=Cov⁢(⁢X~,Z)=CovXZCov~XZ0Cov(PX+b,Z)=Cov(% P X,Z)=0Cov ( P X + b , Z ) = Cov ( P over~ start_ARG X end_ARG , Z ) = 0, which Theorem 4.2 shows occurs when =∗superscriptP=P^*P = P∗. ∎ Putting together Theorems 4.2 and 4.3 and rearranging, we arrive at the LEACE formula: rLEACE⁢()subscriptLEACE r_LEACE( x)rroman_LEACE ( italic_x ) =−+⁢XZ⁢(−⁢[X])absentsuperscriptsubscriptsubscriptXZdelimited-[]X = x-W^+P_W% _XZW ( x-E[% X] )= italic_x - W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W ( italic_x - blackboard_E [ X ] ) (1) Intuitively, LEACE de-means and whitens xitalic_x, projects onto the subspace responsible for correlations between XXX and ZZZ, then unwhitens the result. Finally, it subtracts this value from xitalic_x, thereby surgically removing the linearly available information about ZZZ. Figure 1: LEACE projection in 3 steps. First the data is whitened, ensuring equal variance in all directions. It is then orthogonally projected onto colsp⁢(⁢XZ)⟂colspsuperscriptsubscriptXZperpendicular-tocolsp(W _XZ) colsp ( W Σroman_XZ )⟂, guaranteeing linear guardedness. Finally, we unwhiten the data so that its covariance structure mimics the original. 4.2 Oblique Projections are Least-Squares Optimal Prior work on linear concept erasure has assumed that erasure functions should be orthogonal projections [29, 32, 36], appealing to the well-known fact that an orthogonal projection of a point xitalic_x onto a subspace U yields the nearest point in U to xitalic_x. But even in the case where XXX is centered, rLEACEsubscriptLEACEr_LEACErroman_LEACE is not an orthogonal projection in general. Orthogonal projection matrices are symmetric, and −+⁢XZ⁢superscriptsubscriptsubscriptXZI-W^+P_W _X% ZWI - W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W is only symmetric in the special case where ⁢XZsubscriptsubscriptXZP_W _XZPbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT and WW commute. It is an oblique projection however, since applying ∗superscriptP^*P∗ twice yields the same result as applying it once: (∗)2=−2⁢XZ⁢+++⁢XZ⁢+⁢XZ⁢=∗superscriptsuperscript22subscriptsubscriptXZsuperscriptsuperscriptsubscriptsubscriptXZcancelsuperscriptsubscriptsubscriptXZsuperscript(P^*)^2=I-2WP_W% _XZW^++W^+P_% W _XZ W % W^+P_W _XZ% W=P^*( P∗ )2 = I - 2 WPbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W+ + W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT cancel W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W = P∗. Orthogonal projections are generally not least-squares optimal for concept erasure because the necessary and sufficient condition for linear guardedness, ⁢XZ=subscriptXZ0P _XZ=0P Σroman_XZ = 0, is a constraint on the nullspace of PP, and not on its range. We may freely choose the range of the projection to minimize the mean squared distance, as long as we zero out colsp⁢(XZ)colspsubscriptXZcolsp( _XZ)colsp ( Σroman_XZ ). In Figure 1, an orthogonal projection would map all points onto the the dashed line, thereby preserving less of the variance of the original data than LEACE does (green line). See Appendix F for a concrete example. 4.3 Extension to Continuous ZZZ While not a focus of this work, it’s worth noting that LEACE can also be applied to the setting where ZZZ takes arbitrary values in ℝksuperscriptℝR^kblackboard_Rk, as long as we restrict ourselves to the ordinary least squares regression loss ℒ⁢(η,)=‖η−‖22ℒsubscriptsuperscriptnorm22 [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η, % z)=\|η-z\|^2_2L ( η , z ) = ∥ η - z ∥22. In particular, the proofs of equivalence between conditions 1 and 2 given in Appendix B make no categorical assumption on ZZZ, and the equivalence between the optimality of a zero weight matrix (condition 2) and zero cross-covariance (condition 4) is well known in the OLS setting. We can then apply Theorems 4.2 and 4.3, which also make no categorical assumption, to derive the same optimal affine eraser as in the categorical case. 5 Evaluation 5.1 Intrinsic Evaluation Following Ravfogel et al. [31] we evaluate the ability of our method to remove gender information from the last hidden layer of a frozen BERT model. We use the biographies dataset of De-Arteaga et al. [6], composed of short biographies annotated by both binary gender and profession. We embed each biography with the [CLS] embedding in the last layer of BERT, enforce the same-conditional-mean constraint to remove gender information from the [CLS] , and then evaluate the performance of the model, after the intervention, on the main task of profession prediction. We compare our intervention with RLACE [31], which uses gradient-based optimization to solve a linear concept-erasure adversarial game. Concept erasure results. First, we evaluate the ability of logistic regression classifiers to recover the removed information. The results, presented in Fig. 2, show that our method is the only to achieve random accuracy (perfect erasure) with a small edit, although RLACE (but not INLP) comes close. At the same time, our method is around 2 orders of magnitude faster, and does not require gradient-based optimization. Figure 2: Gender prediction accuracy after bias-removal projection versus the mean squared distance from the original embedding for INLP, RLACE, and LEACE on BERT embeddings. 5.2 Downstream Fairness (a) (b) Figure 3: The correlation between G⁢A⁢Pf⁢e⁢m⁢a⁢l⁢e,yT⁢P⁢RsuperscriptsubscriptGAP_female,y^TPRG A Pitalic_f e m a l e , yitalic_T P R and the relative proportion of women in profession y, for BERT embedding, before (left; R=0.867) and after (right; R=0.392) the projection. How does our intervention affect the behavior of the model on the main classification task of profession prediction? We fit a logistic regression profession-prediction classifier over the projected [CLS] embeddings. To measure the bias in a classifier, we follow De-Arteaga et al. [6] and use the TPR-GAP measure, which quantifies the bias in a classifier by considering the difference (GAP) in the true positive rate (TPR) between individuals with different protected attributes (e.g. race or gender). We use the notation GAPz,yTPRsuperscriptsubscriptGAPTPRGAP_z,y^TPRGAPitalic_z , yroman_TPR to denote the TPR-gap in some main-class label y (e.g. “nurse” prediction) for some protected group z (e.g. “female”), we also consider GAPzTPR,RMSsuperscriptsubscriptGAPTPRRMSGAP_z^TPR,RMSGAPitalic_zroman_TPR , RMS, the RMS of the TPR-gap across all professions for a protected group z: GAPzTPR,RMS=1|C|⁢∑y∈C(GAPz,yTPR)2superscriptsubscriptGAPTPRRMS1subscriptsuperscriptsuperscriptsubscriptGAPTPR2 _z^TPR,RMS= 1|C|% _y∈ C(GAP_z,y^TPR)^2GAPitalic_zroman_TPR , RMS = square-root start_ARG divide start_ARG 1 end_ARG start_ARG | C | end_ARG ∑y ∈ C ( GAPitalic_z , yroman_TPR )2 end_ARG To calculate the relation between the bias the model exhibits and the bias in the data, we also calculate σ(GAPTPR,%Women) _(GAP^TPR,\%Women)σ( GAProman_TPR , % Women ), the correlation between the TPR gap in a given profession and the percentage of women in that profession. Results. The main-task classifier achieves profession-prediction accuracy of 77.3% on the projected embeddings (compared with 79.3% over the original embeddings), indicating that the intervention minimally affects the ability to predict the profession of a person from the embedding of their biography. At the same time, the TPR gap drops significantly from 0.198 to 0.084, indicating a sharp drop in the biased behavior of the profession classifier. Indeed, inspecting the correlation σ(GAPTPR,%Women) _(GAP^TPR,\%Women)σ( GAProman_TPR , % Women ) between the gap (per profession) and the embedding of women in this profession, we see that this correlation plummets from 0.867 to 0.392 after erasure. Re-fitting the main-task logistic regression classifier over the projected embeddings yields a slightly higher main-task accuracy of 78.1%, at the price of significantly increasing the TPR gap to 0.158.555The softmax probabilities of a multiclass logistic regression classifier can leak the removed information if another classifier is stacked on top of it [33], though this setup is not linear. 5.3 Revisiting Amnesic Probing (a) (b) Figure 4: Amnesic probing results on bert-base-uncased. Elazar et al. [10] have introduced the idea of amnesic probing as a causal intervention that aims to test the importance of a given concept (e.g. part-of-speech tag) to some main task (e.g. language modeling). They applied Iterative Nullspace Projection (INLP) to remove different concepts from the activations of the model, and assessed the degree to which its behavior changed when performing masked language modeling. Since INLP often requires dozens of iterations to completely erase the concept, its usage in this context raises concerns of collateral damage due to magnitude of the intervention and the non-exhaustive nature of INLP removal. Here, we replicate their experiments on the bert-base-uncased model with our interventions. Experimental setup. We use part-of-speech (POS) tags as our concept of interest. We collect sentences and their coarse POS tags (“Noun”, “Verb” etc.; 18 in total) from the English Universal Dependencies dataset [27]. We tokenize the sentences with the BERT tokenizer and map each word-piece to the POS tag of the word to which it belongs. We collect the unmasked BERT embeddings for each layer, intervene to linearly erase the POS concept from that layer, and continue the forward pass until the last layer, from which we compute the distribution of the MLM over the vocabulary. Note that in each experiment we intervene on a single layer. We quantify the decrease in accuracy following the intervention, as well as the increase in the loss. We compare with a baseline intervention of a random orthogonal projection whose null space has the same rank as the label space (18). For INLP, we perform 20 iterations. This is needed because INLP does not effectively remove the concept; even after 20 iterations, classification accuracy is above majority accuracy. As a result, INLP reduces the rank of the embedding by 360. By contrast, our method decreases the rank just by 17. Results. The results are shown in Fig. 4b. Our intervention only mildly changes BERT LM accuracy and loss until layer 8, with the highest drop recorded in layer 11. INLP, in contrast, shows maximum effect at layer 6. Since it removes hundreds of dimensions, it is difficult to attribute this effect to the erasure of the concept. These results suggest that the causal effect of the POS concept on the language model is concentrated in layer 11. Interestingly, this stands in contrast with POS linear probing results, which are optimal at earlier layers [38]. As Elazar et al. [10] have noted, probing does not generally correlate with intervention-based analysis techniques. 6 Concept Scrubbing Algorithm 1 Concept scrubbing 1:Model with ℓ ℓ layers f=fℓ∘…∘f1subscriptℓ…subscript1f=f_ … f_1f = froman_ℓ ∘ … ∘ f1 2:Design matrix ∈ℝn×dsuperscriptℝX ^n× dX ∈ blackboard_Rn × d 3:Label matrix ∈ℝn×ksuperscriptℝZ ^n× kZ ∈ blackboard_Rn × k 4:LEACE parameters for each layer in f 5:1←Embed⁢()←subscript1EmbedH_1 (X)H1 ← Embed ( X ) 6:L←absentL ←list() 7:for l∈1⁢…⁢ℓ1…ℓl∈ 1… ∈ 1 … ℓ do 8: Fit (,)(P,b)( P , b ) on lsubscriptH_lHitalic_l and ZZ 9: Append (,)(P,b)( P , b ) to L 10: l←⁢(l−μl)+μl←subscriptsubscriptsubscriptsubscriptsubscriptsubscriptH_l (H_l- _H_l)+ _% H_lHitalic_l ← P ( Hitalic_l - μbold_H start_POSTSUBSCRIPT l end_POSTSUBSCRIPT ) + μbold_H start_POSTSUBSCRIPT l end_POSTSUBSCRIPT (Eq. 1) 11: l+1←fl⁢(l)←subscript1subscriptsubscriptH_l+1← f_l(H_l)Hitalic_l + 1 ← fitalic_l ( Hitalic_l ) 12:return L Unfortunately, Elazar et al. [10] were forced to limit their interventions to a single layer due to the limitations of INLP. INLP often requires the deletion of several dozen dimensions before linear guarding is achieved—as demonstrated in Figure 2. Kumar et al. [21] show empirically and theoretically that INLP causes needless “collateral damage” to useful parts of the embedding that are orthogonal to the concept being erased. Because of this collateral damage, it’s impossible to apply INLP to multiple layers of a transformer without causing its outputs to collapse into gibberish. Instead, we would like to erase all linear information about a concept in the activations at every layer, which we term concept scrubbing. LEACE makes concept scrubbing possible and eminently practical. It causes minimal collateral damage, induces little computational overhead, and the covariance statistics it relies on can be computed in a streaming fashion, without ever storing all the hidden states in memory or on disk. Algorithm. Any intervention on the model at layer ℓ ℓ changes the distribution of hidden states at layers ℓ′>ℓsuperscriptℓ′ℓ > ℓ′ > ℓ. Because of this, the naive approach of independently fitting LEACE parameters (,)(P,b)( P , b ) for all layers of the clean model, then applying them all at once, may fail to fully erase the target concept. Instead, we fit LEACE parameters sequentially, starting from the first layer and proceeding to the final layer. After we compute (,)(P,b)( P , b ) for a layer, we immediately use them to scrub the hidden states for that layer, then feed these scrubbed embeddings to the next layer (Algorithm 1). LLaMA Pythia Condition 7B 13B 30B 160M 1.4B 6.9B 12B No intervention 0.69 0.66 0.62 0.90 0.70 0.64 0.62 Random erasure 0.69 0.66 0.62 0.99 0.72 0.66 0.63 LEACE 1.73 1.84 1.96 2.79 2.25 3.57 3.20 SAL 3.24 3.26 3.16 3.53 3.44 4.17 4.69 unigram entropy 2.90 2.90 2.90 2.66 2.66 2.66 2.66 Table 1: Perplexity in autoregressive language models when removing linearly available part-of-speech information from the input to each transformer layer. Units are bits per UTF-8 byte. The unigram baseline assigns probabilities to tokens based only on their frequency and not on the context. 6.1 Experimental Details Dataset. For each model family, we use a sample from the respective pretraining distribution: the validation split of the Pile [13] for the Pythia models [2], and the RedPajama replication of the LLaMA pretraining corpus for the LLaMA family [39]. sampling a slice of 222superscript2222^22222 tokens for fitting the LEACE parameters and another slice of 222superscript2222^22222 tokens for evaluation. Since neither corpus comes with part-of-speech tags, we use the model from the SpaCy library [19] to automatically generate Universal Dependency tags [23]. Baseline method. We also run concept scrubbing using full-rank SAL [36], which is similar to our method but lacks a bias term and does not adjust for correlations between features (Appendix D). Architecture. We focus on autoregressive language models. We evaluate our method on EleutherAI’s Pythia 160M, 1.4B, 6.9B, and 12B models [2], and Meta’s LLaMA 7B, 13B, and 30B [39]. We apply concept erasure to the input of each transformer block, immediately after normalization is applied (LayerNorm or RMSNorm). Randomized erasure. Almost any intervention on a neural network will cause its performance to degrade to some extent. Following Elazar et al. [10], we isolate the effect of the concept erasure by comparing it to a control condition in which we orthogonally project onto a random linear subspace of the same rank as the cross-covariance matrix. To reduce the variance of our results, we sample a fresh subspace for each minibatch, and erase that subspace at each layer, reporting the cross-entropy loss averaged over subspaces. Training efficiency. Algorithm 1 avoids redundant computation by caching the layer i hidden states for every data point, then using them to run layer i+11i+1i + 1. This approach has the downside of requiring a large amount of memory or disk space during training (up to 500GB in our experiments). It’s possible to avoid caching any hidden states and instead recompute them as needed, at the expense of increasing the total compute cost from O⁢(ℓ)ℓO( )O ( ℓ ) to O⁢(ℓ2)superscriptℓ2O( ^2)O ( ℓ2 ). 6.2 Results We find strong evidence that autoregressive language models heavily rely on linearly encoded part-of-speech information. While erasing a randomly selected subspace has little to no effect on language modeling performance, scrubbing away part-of-speech information induces a large increase in perplexity across all models (Table 1). The specific numbers, however, depend on the erasure method used: SAL induces significantly larger increases in perplexity for all models we tested. We take this to mean that SAL inflicts more collateral damage on other useful features in the embedding than LEACE does. In other words, interventions made with LEACE are more surgical than those made with prior work; they more closely approximate the ideal of a perfect intervention which only erases the target concept and keeps everything else fixed [40, 15]. If this experiment were conducted with SAL alone, we would have overestimated the causal effect of part-of-speech. 7 Limitations and Future Work Much work remains to be done to validate concept scrubbing. Specifically, we’d like to see experiments that target concepts much narrower than part-of-speech, and use behavioral metrics to determine whether scrubbing changes the network in the ways we’d intuitively expect. If these experiments succeed, an exciting next step would be the incorporation of concept scrubbing into the pretraining and/or finetuning process. This may make it possible to train deep neural networks subject to conceptual constraints. It remains to be seen if gradient-based optimizers will be able to “circumvent” such constraints by encoding protected attributes in completely nonlinear ways. In this work, we focused exclusively on linear concept erasure due to its simplicity and tractability. Some authors have proposed nonlinear concept erasure techniques based on kernel methods, but have found that erasure functions fit using one kernel do not generalize well to other kernels [32, 36]. We conjecture that it is intractable to nondestructively edit XXX so as to prevent a general nonlinear adversary from recovering ZZZ, unless the data generating process for XXX is known in detail.666We suspect erasing a concept is at least as hard as extracting it from the original embedding. But in the worst case, information about ZZZ could be encoded cryptographically in XXX, which would be intractable to decode given standard computational complexity assumptions. If the data is generated by a known algorithm, however, it may be possible to efficiently eliminate mutual information between ZZZ and XXX by simply breaking the links in the causal graph that connect them. A major motivation of concept erasure is that it promises to prevent models from using a concept in a post hoc, model-agnostic fashion. But if our concept scrubbing procedure turns out to yield unsatisfactory results in practical use cases, the most promising research direction might then be to improve model-specific techniques, such as those that modify the training procedure [8, 9, 14]. 8 Acknowledgements We are grateful to CoreWeave for providing the compute resources used in Section 6. Shauli Ravfogel is grateful to be supported by the Bloomberg Data Science PhD Fellowship. References Berkeley [2018] UC Berkeley. The Hilbert space of random variables. Lecture Notes Electrical Engineering 126, 2018. URL https://inst.eecs.berkeley.edu/~e126/sp18/projection.pdf. Biderman et al. [2023] Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. arXiv preprint arXiv:2304.01373, 2023. Bolukbasi et al. [2016] Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Advances in Neural Information Processing Systems, 29:4349–4357, 2016. URL https://proceedings.neurips.c/paper/2016/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf. Chen et al. [2018] Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinberger. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics, 6:557–570, 2018. URL https://aclanthology.org/Q18-1039. Dankers et al. [2022] Verna Dankers, Christopher Lucas, and Ivan Titov. Can transformer be too compositional? Analysing idiom processing in neural machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3608–3626, 2022. De-Arteaga et al. [2019] Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, page 120–128, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450361255. doi: 10.1145/3287560.3287572. URL https://doi.org/10.1145/3287560.3287572. Dev et al. [2021] Sunipa Dev, Tao Li, Jeff M. Phillips, and Vivek Srikumar. OSCaR: Orthogonal subspace correction and rectification of biases in word embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5034–5050, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.411. URL https://aclanthology.org/2021.emnlp-main.411. Edwards and Storkey [2016] Harrison Edwards and Amos Storkey. Censoring representations with an adversary. In International Conference in Learning Representations, pages 1–14, May 2016. URL https://arxiv.org/abs/1511.05897. Elazar and Goldberg [2018] Yanai Elazar and Yoav Goldberg. Adversarial removal of demographic attributes from text data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 11–21, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1002. URL https://aclanthology.org/D18-1002. Elazar et al. [2021] Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics, 9:160–175, 2021. doi: 10.1162/tacl_a_00359. URL https://aclanthology.org/2021.tacl-1.10. Elhage et al. [2021] Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Hatfield Zac Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jack Clark, Jared Kaplan, Sam McCandlish, and Chris Olah. A mathematical framework for transformer circuits. Transformer Circuits Thread, 2021. Ferguson [1967] Thomas S. Ferguson. Mathematical Statistics. Academic Press, Cambridge, MA, 1967. Gao et al. [2020] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. Geiger et al. [2022] Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, pages 7324–7338. PMLR, 2022. Grimsley et al. [2020] Christopher Grimsley, Elijah Mayfield, and Julia R.S. Bursten. Why attention is not explanation: Surgical intervention and causal reasoning about neural models. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1780–1790, Marseille, France, May 2020. European Language Resources Association. URL https://aclanthology.org/2020.lrec-1.220. Haghighatkhah et al. [2021] Pantea Haghighatkhah, Wouter Meulemans, Bettina Speckmann, Jérôme Urhausen, and Kevin Verbeek. Obstructing classification via projection. In Filippo Bonchi and Simon J. Puglisi, editors, 46th International Symposium on Mathematical Foundations of Computer Science, Leibniz International Proceedings in Informatics, LIPIcs. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021. Haghighatkhah et al. [2022] Pantea Haghighatkhah, Antske Fokkens, Pia Sommerauer, Bettina Speckmann, and Kevin Verbeek. Better hit the nail on the head than beat around the bush: Removing protected attributes with a single projection. pages 8395–8416, December 2022. doi: 10.18653/v1/2022.emnlp-main.575. URL https://aclanthology.org/2022.emnlp-main.575. Hernandez and Andreas [2021] Evan Hernandez and Jacob Andreas. The low-dimensional linear geometry of contextualized word representations. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 82–93, 2021. Honnibal et al. [2020] Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python, 2020. Kleindessner et al. [2023] Matthäus Kleindessner, Michele Donini, Chris Russell, and Muhammad Bilal Zafar. Efficient fair PCA for fair representation learning. In International Conference on Artificial Intelligence and Statistics, pages 5250–5270. PMLR, 2023. Kumar et al. [2022] Abhinav Kumar, Chenhao Tan, and Amit Sharma. Probing classifiers are unreliable for concept removal and detection. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 17994–18008. Curran Associates, Inc., 2022. URL https://proceedings.neurips.c/paper_files/paper/2022/file/725f5e8036c08adeba4a7c3bcbc6f2c-Paper-Conference.pdf. Kusner et al. [2017] Matt J. Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. Counterfactual fairness. Advances in Neural Information Processing Systems, 30, 2017. McDonald et al. [2013] Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, et al. Universal dependency annotation for multilingual parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 92–97, 2013. Nanda [2023] Neel Nanda. Actually, Othello-GPT has a linear emergent world model, Mar 2023. URL <https://neelnanda.io/mechanistic-interpretability/othello>. Nikoulina et al. [2021] Vassilina Nikoulina, Maxat Tezekbayev, Nuradil Kozhakhmet, Madina Babazhanova, Matthias Gallé, and Zhenisbek Assylbekov. The rediscovery hypothesis: Language models need to meet linguistics. Journal of Artificial Intelligence Research, 72:1343–1384, 2021. Nilforoshan et al. [2022] Hamed Nilforoshan, Johann D. Gaebler, Ravi Shroff, and Sharad Goel. Causal conceptions of fairness and their consequences. In International Conference on Machine Learning, pages 16848–16887. PMLR, 2022. Nivre et al. [2020] Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajic, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4034–4043, 2020. Pearl [2009] Judea Pearl. Causality. Cambridge University Press, Cambridge, UK, 2 edition, 2009. ISBN 978-0-521-89560-6. doi: 10.1017/CBO9780511803161. Ravfogel et al. [2020] Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7237–7256, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.647. URL https://aclanthology.org/2020.acl-main.647. Ravfogel et al. [2021] Shauli Ravfogel, Grusha Prasad, Tal Linzen, and Yoav Goldberg. Counterfactual interventions reveal the causal effect of relative clause representations on agreement prediction. In Proceedings of the 25th Conference on Computational Natural Language Learning, pages 194–209, 2021. Ravfogel et al. [2022a] Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. Linear adversarial concept erasure. In International Conference on Machine Learning, pages 18400–18421. PMLR, 2022a. Ravfogel et al. [2022b] Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, and Ryan Cotterell. Adversarial concept erasure in kernel space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6034–6055, 2022b. Ravfogel et al. [2023] Shauli Ravfogel, Yoav Goldberg, and Ryan Cotterell. Log-linear guardedness and its implications. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9413–9431, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.523. URL https://aclanthology.org/2023.acl-long.523. Sadeghi and Boddeti [2021] Bashir Sadeghi and Vishnu Boddeti. On the fundamental trade-offs in learning invariant representations. arXiv preprint arXiv:2109.03386, 2021. URL https://openreview.net/pdf?id=KOk7mUGspN9. Sadeghi et al. [2019] Bashir Sadeghi, Runyi Yu, and Vishnu Boddeti. On the global optima of kernelized adversarial representation learning. In 2019 IEEE/CVF International Conference on Computer Vision, pages 7970–7978. IEEE, 2019. URL http://hal.cse.msu.edu/assets/pdfs/papers/2019-iccv-kernel-adversarial-representation-learning.pdf. Shao et al. [2023a] Shun Shao, Yftah Ziser, and Shay B. Cohen. Gold doesn’t always glitter: Spectral removal of linear and nonlinear guarded attribute information. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1611–1622, Dubrovnik, Croatia, May 2023a. Association for Computational Linguistics. URL https://aclanthology.org/2023.eacl-main.118. Shao et al. [2023b] Shun Shao, Yftah Ziser, and Shay B. Cohen. Erasure of unaligned attributes from neural representations. Transactions of the Association for Computational Linguistics, 11:488–510, 2023b. doi: 10.1162/tacl_a_00558. URL https://aclanthology.org/2023.tacl-1.29. Tenney et al. [2019] Ian Tenney, Dipanjan Das, and Ellie Pavlick. BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593–4601, 2019. Touvron et al. [2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and efficient foundation language models, 2023. URL http://arxiv.org/abs/2302.13971. Woodward [2005] James Francis Woodward. Making Things Happen: A Theory of Causal Explanation explanation. Oxford University Press, 2005. Wu et al. [2023] Zhengxuan Wu, Atticus Geiger, Christopher Potts, and Noah Goodman. Interpretability at scale: Identifying causal mechanisms in Alpaca. 2023. Xie et al. [2017] Qizhe Xie, Zihang Dai, Yulun Du, Eduard Hovy, and Graham Neubig. Controllable invariance through adversarial feature learning. In Advances in Neural Information Processing Systems, volume 30, pages 585–596, 2017. URL https://dl.acm.org/doi/10.5555/3294771.3294827. Xu et al. [2020] Yilun Xu, Shengjia Zhao, Jiaming Song, Russell Stewart, and Stefano Ermon. A theory of usable information under computational constraints. In 8th International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=r1eBeyHFDH. Zhang et al. [2018] Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, page 335–340, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450360128. URL https://doi.org/10.1145/3278721.3278779. Appendix A Additional Related Work The problem of linear concept erasure is an instance of the general problem of information removal. Information removal methods generally divide into adversarial methods, which are applied during training, and the post-hoc linear methods considered in this paper. Adversarial methods [8, 42, 4, 9, 44] use a gradient-reversal layer during training to induce embeddings that do not encode the protected attribute. However, Elazar and Goldberg [9] have shown that these methods fail in exhaustively removing all the information associated with the protected attribute: it is often possible to train new adversaries that successfully recover the removed information. Linear methods have been proposed as a tractable alternative, where one identifies a linear subspace that captures the concept of interest, and neutralizes it using algebraic techniques. Different methods have been proposed for the identification of the subspace, e.g. PCA and variants thereof [3, 20], orthogonal-rotation [7], classification-based [29], spectral [36, 37] and adversarial approaches [31]. Few works theoretically characterize the condition of linear guardedness. Haghighatkhah et al. [16] extensively analyzed the problem of preventing linear classification, with the focus on decreasing accuracy. They provide a constructive proof of an optimal intervention for an SVM classifier. Ravfogel et al. [33] have proposed a formal definition of linear guardedness based on VV information, and characterized the fairness implications of guardedness; we show the relations with our definition above. Ravfogel et al. [31] provide an adversarial formulation of the problem, derive a closed-formed solution to certain cases, and propose an SGD-based optimization for others. While they seek an orthogonal projection, we empirically showed that their solution is very close to ours. Sadeghi et al. [35] and Sadeghi and Boddeti [34] both study an adversarial formulation of concept erasure for linear regression, and they trade-off with main-task performance. In contrast to Ravfogel et al. [31], they consider a general linear adversary, i.e. not necessarily a projection matrix. Closest to our work are Kleindessner et al. [20], Haghighatkhah et al. [17], Shao et al. [36]. As we showed above (§ 4), those methods do achieve the goal of linear guardedness though they are unable to prove this fact. At the same time, they are not optimal in terms of damage to the original embedding space. Appendix B Equivalence of Guardedness with the Optimality of Constant Predictors The following two theorems establish the equivalence of conditions 1 and 2 (indeed, they do so in the general setting, with no assumption of convex loss or linear predictors). Theorem B.1. Suppose XXX (,)(V, L)( V , L )-guards ZZZ. Then for every loss ℒ∈ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L∈ LL ∈ L, the corresponding trivially attainable loss Lτ(Z,ℒ)superscriptsubscriptZℒL_τ^(Z, [rgb]0,0,0 [named]pgfstrokecolor% rgb0,0,0 @color@gray@stroke0 @color@gray@fill0L)Litalic_τ( Z , L ) cannot be improved upon by any predictor η⁢(⋅;)∈⋅η(·; θ) η ( ⋅ ; italic_θ ) ∈ V, i.e. Lτ=inf⁢[ℒ⁢(η⁢(X;),Z)]subscriptsubscriptinfimumdelimited-[]ℒXZL_τ= _ θE[ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X; θ)% ,Z)]Litalic_τ = infbold_italic_θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ]. Proof. Consider the null random vector X′⁢(ω)=superscriptX′0X (ω)=0X′ ( ω ) = 0. Since all predictors are constant on X′X X′, and the trivially attainable loss gives the best available expected loss among constant predictors, we must have: Lτ=inf⁢[ℒ⁢(η⁢(X′;),Z)]subscriptsubscriptinfimumdelimited-[]ℒsuperscriptX′ZL_τ= _ θE[ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X ; % θ),Z)]Litalic_τ = infbold_italic_θ blackboard_E [ L ( η ( X′ ; italic_θ ) , Z ) ] (4) The right side of equation (4) is the best possible loss achievable by a function η⁢(⋅;)⋅η(·; θ)η ( ⋅ ; italic_θ ) on the joint distribution of (X′,Z)superscriptX′Z(X ,Z)( X′ , Z ), which by the definition of guardedness is upper bounded by the best possible loss achievable on the joint distribution of (X,Z)XZ(X,Z)( X , Z ): inf⁢[ℒ⁢(η⁢(X′;),Z)]≤inf⁢[ℒ⁢(η⁢(X;),Z)]subscriptinfimumdelimited-[]ℒsuperscriptX′Zsubscriptinfimumdelimited-[]ℒXZ _ θE[ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η(X ; θ),Z)% ]≤ _ θE[ [rgb]0,0,0 [named% ]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X; θ)% ,Z)]infbold_italic_θ blackboard_E [ L ( η ( X′ ; italic_θ ) , Z ) ] ≤ infbold_italic_θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ] (5) Combining equations (4) and (5), and the fact that all constant functions exist in our function class =η⁢(⋅;)⋅V=\η(·; θ)\V = η ( ⋅ ; italic_θ ) , we arrive at our desired result: Lτ=inf⁢[ℒ⁢(η⁢(X;),Z)]subscriptsubscriptinfimumdelimited-[]ℒXZL_τ= _ θE[ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X; θ)% ,Z)]Litalic_τ = infbold_italic_θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ] ∎ Theorem B.2. Suppose that for every loss ℒ∈ℒ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L∈ LL ∈ L, the corresponding trivially attainable loss Lτ(Z,ℒ)superscriptsubscriptZℒL_τ^(Z, [rgb]0,0,0 [named]pgfstrokecolor% rgb0,0,0 @color@gray@stroke0 @color@gray@fill0L)Litalic_τ( Z , L ) cannot be improved upon by any predictor η⁢(⋅;)∈⋅η(·; θ) η ( ⋅ ; italic_θ ) ∈ V, i.e. Lτ=inf⁢[ℒ⁢(η⁢(X;),Z)]subscriptsubscriptinfimumdelimited-[]ℒXZL_τ= _ θE[ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X; θ)% ,Z)]Litalic_τ = infbold_italic_θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ]. Then XXX (,)(V, L)( V , L )-guards ZZZ. Proof. Let X′:Ω→ℝd:superscriptX′→ΩsuperscriptℝX : ^dX′ : Ω → blackboard_Rd be any other random data vector with finite first moment. Since all constant predictors exist in our predictor class =η⁢(⋅;)⋅V=\η(·; θ)\V = η ( ⋅ ; italic_θ ) , the best loss achievable on (X′,Z)superscriptX′Z(X ,Z)( X′ , Z ) by functions in VV must be at least as good as the trivially attainable loss (the best loss available by such constant predictors): inf⁢[ℒ⁢(η⁢(X′;),Z)]≤Lτsubscriptinfimumdelimited-[]ℒsuperscriptX′Zsubscript _ θE[ [rgb]0,0,0 [named]% pgfstrokecolorrgb0,0,0 @color@gray@stroke0 @color@gray@fill% 0L(η(X ; θ),Z)% ]≤ L_τinfbold_italic_θ blackboard_E [ L ( η ( X′ ; italic_θ ) , Z ) ] ≤ Litalic_τ By assumption, the trivially attainable loss cannot be improved upon over (X,Z)XZ(X,Z)( X , Z ) by predictors in VV: Lτ=inf⁢[ℒ⁢(η⁢(X;),Z)]subscriptsubscriptinfimumdelimited-[]ℒXZL_τ= _ θE[ [rgb]0,0,0 [% named]pgfstrokecolorrgb0,0,0 @color@gray@stroke0% @color@gray@fill0L(η(X; θ)% ,Z)]Litalic_τ = infbold_italic_θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ] Since our choice of X′X X′ was arbitrary, this shows that XXX maximizes the minimal achievable loss, so XXX (,)(V, L)( V , L )-guards ZZZ. ∎ Appendix C Linear Guardedness is Equivalent to Linear Statistical Parity To measure the effect of linear guardedness on main-task classifiers, we use the following minimal definition of “fairness” with respect to an attribute, adapted from Edwards and Storkey [8]. Definition C.1 (Statistical Parity). Let XXX and ZZZ be defined as above, and let f be a function with domain ℝdsuperscriptℝR^dblackboard_Rd. Then f exhibits statistical parity with respect to ZZZ when evaluated on XXX if ∀z∈:⁢[f⁢(X)|Z=z]=⁢[f⁢(X)].:for-alldelimited-[]conditionalXZdelimited-[]X∀ z :E[f(X)|Z=z]=E[% f(X)].∀ z ∈ Z : blackboard_E [ f ( X ) | Z = z ] = blackboard_E [ f ( X ) ] . We now prove the equivalence of conditions 3 and 5. Theorem C.2. Let XXX and ZZZ be defined as above. Then every linear predictor f⁢()=+f(x)=b+Wxf ( x ) = b + Wx exhibits statistical parity w.r.t. ZZZ when evaluated on XXX if and only if each class-conditional mean ⁢[X|Z=z]delimited-[]conditionalXZE [X|Z=z ]blackboard_E [ X | Z = z ] is equal to ⁢[X]delimited-[]XE [X ]blackboard_E [ X ]. Proof. Suppose each class-conditional mean ⁢[X|Z=z]delimited-[]conditionalXZE [X|Z=z ]blackboard_E [ X | Z = z ] is equal to ⁢[X]delimited-[]XE [X ]blackboard_E [ X ]. Then by the linearity of expectation, we have for all z∈z ∈ Z: ⁢[f⁢(X)|Z=z]=⁢[⁢X+|Z=z]=⁢[X|Z=z]+=⁢[X]+=⁢[f⁢(X)].delimited-[]conditionalXZdelimited-[]XconditionalZdelimited-[]conditionalXZdelimited-[]Xdelimited-[]X [f(X)|Z=z]=E[W% X+b|Z=z]=WE[X|% Z=z]+b=WE[X]+b=% E[f(X)].blackboard_E [ f ( X ) | Z = z ] = blackboard_E [ W X + b | Z = z ] = W blackboard_E [ X | Z = z ] + b = W blackboard_E [ X ] + b = blackboard_E [ f ( X ) ] . This matches the definition of statistical parity provided in Definition C.1. Conversely, suppose every linear predictor f⁢()=+f(x)=b+Wxf ( x ) = b + Wx exhibits statistical parity w.r.t. ZZZ when evaluated on XXX. Then this holds for the identity function id⁢()=idid(x)=xid ( x ) = x, and thus for all z∈z ∈ Z: ⁢[X|Z=z]=⁢[id⁢(X)|Z=z]=⁢[id⁢(X)]=⁢[X].delimited-[]conditionalXZdelimited-[]conditionalidXZdelimited-[]idXdelimited-[]X [X|Z=z]=E[id(% X)|Z=z]=E[id(X)]=E[% X].blackboard_E [ X | Z = z ] = blackboard_E [ id ( X ) | Z = z ] = blackboard_E [ id ( X ) ] = blackboard_E [ X ] . ∎ Appendix D Implications for Prior Work In this section we discuss the implications of Theorem 4.1, which characterizes the necessary and sufficient conditions for an affine erasure function to yield a perfectly linearly guarded dataset, for methods proposed in prior work. Spectral Attribute RemovaL (SAL) [36] uses the top n left singular vectors of XZsubscriptXZ _XZΣroman_XZ to construct an orthogonal projection matrix SAL=−:n⁢:nTsubscriptSALsubscript:absentsuperscriptsubscript:absentQ_SAL=I-U_:nU_:n^TQroman_SAL = I - U: n U: nitalic_T which is then applied to XXX. Notably, while n is presented as a free parameter in their method, all of their experiments involve binary classification problems where ZZZ is a one-hot vector, and n is set to a value no greater than 2. We’l call the version of SAL where n=rank⁢(XZ)ranksubscriptXZn=rank( _XZ)n = rank ( Σroman_XZ ), “full-rank SAL.” Since these left singular vectors are an orthonormal basis for XZsubscriptXZ _XZΣroman_XZ’s column space, Theorem 4.1 implies that full-rank SAL guarantees linear guardedness. Mean Projection (MP) [17] orthogonally projects XXX onto the orthogonal complement of the span of the difference in class centroids ⁢[X|Z=1]−⁢[X|Z=0]delimited-[]conditionalXZ1delimited-[]conditionalXZ0E[X|Z=1]-E[X|Z=0]blackboard_E [ X | Z = 1 ] - blackboard_E [ X | Z = 0 ], where ZZZ is assumed to be binary. Since the centroids are equal after the projection, this method guarantees linear guardedness by Theorem 3.1. In fact, by Theorem 3.4, MP is mathematically equivalent to SAL when ZZZ is a one-dimensional random vector taking one of two possible values. Appendix E Derivation of LEACE See 4.2 Below are two independent proofs of Theorem 4.2. E.1 Algebraic Proof Proof. We shall first show that, in any orthonormal basis,777Throughout this proof, we abuse the notations Xi,subscriptXsubscriptX_i,P_iXitalic_i , Pbold_i, etc. to refer to the ithsuperscriptthi^thith component in the specified basis, not necessarily the standard one. each row subscriptP_iPbold_i constitutes an independent optimization problem, and then select a basis in which we can easily show that the corresponding component XisubscriptXX_iXitalic_i of XXX can be almost surely decomposed into a linear combination of mutually uncorrelated components in the whitened random vector ⁢XWXW X, some of which correlate with ZZZ and some of which do not. The solution (⁢X)isubscriptX(PX)_i( P X )i is then that same linear combination, restricted to those components which do not correlate with ZZZ. Consider first an orthonormal basis diagonalizing the inner product MM, so that ⟨,⟩=∑i=1dαi⁢xi⁢yisubscriptsuperscriptsubscript1subscriptsubscriptsubscript ,y _M= _i=1^d _ix_i% y_i⟨ x , y ⟩M = ∑i = 1d αitalic_i xitalic_i yitalic_i for fixed α1,…,αd≥0subscript1…subscript0 _1,…, _d≥ 0α1 , … , αitalic_d ≥ 0. This allows us to treat each row ∈ℝdsubscriptsuperscriptℝP_i ^dPbold_i ∈ blackboard_Rd of PP as a separate optimization problem, argmin∈ℝd⁢[αi⁢(T⁢X−Xi)2]subject⁢to⁢Cov⁢(T⁢X,Z)=,subscriptargminsubscriptsuperscriptℝdelimited-[]subscriptsuperscriptsuperscriptsubscriptXsubscriptX2subjecttoCovsuperscriptsubscriptXZ0 argmin\>_ subarraycP_i ^d% subarrayE [ _i (P_i^TX% -X_i )^2 ] \>to\>\>Cov(% P_i^TX,Z)=0,start_BIGOP argmin end_BIGOPstart_ARG start_ROW start_CELL P start_POSTSUBSCRIPT i ∈ blackboard_Rd end_CELL end_ROW end_ARG end_POSTSUBSCRIPT blackboard_E [ αitalic_i ( Pbold_iitalic_T X - Xitalic_i )2 ] subject to Cov ( Pbold_iitalic_T X , Z ) = 0 , at which point the weights αisubscript _iαitalic_i of each subproblem become irrelevant, and our objective may as well be Euclidean, allowing us to view each row as an independent optimization problem not just in this basis, but from any convenient one. So now let ℓ=rank⁢(XZ)=rank⁢(⁢X,Z)ℓranksubscriptXZranksubscriptXZ =rank( _XZ)=rank(% _WX,Z)ℓ = rank ( Σroman_XZ ) = rank ( Σbold_W X , Z ) and m=rank⁢(X)=rank⁢(⁢X,⁢X)ranksubscriptXXranksubscriptXXm=rank( _XX)=rank(% _WX,WX)m = rank ( Σroman_X ) = rank ( Σbold_W X , W X ), and consider a (new) orthonormal basis whose first m coordinates span the column (and row) space of WW (i.e. the subspace of ℝdsuperscriptℝR^dblackboard_Rd in which XXX and ⁢XWXW X have nonzero variance), and whose first ℓ≤mℓ ≤ mℓ ≤ m coordinates span the column space of ⁢X,ZsubscriptXZ _WX,ZΣbold_W X , Z (i.e. the subspace of ℝdsuperscriptℝR^dblackboard_Rd in which ⁢XWXW X has nonzero covariance with ZZZ). Any component of XXX can be (almost surely) written as a fixed linear combination of the nontrivial components of its whitening ⁢XWXW X: Xi=(+⁢X)i=∑j=1mWi⁢j+⁢(⁢X)j.subscriptXsubscriptsuperscriptXsuperscriptsubscript1superscriptsubscriptsubscriptX X_i=(W^+WX)_i= _% j=1^mW_ij^+(WX)_j.Xitalic_i = ( W+ W X )i = ∑j = 1m Witalic_i j+ ( W X )j . (almost surely) Meanwhile, any component of ⁢XPXP X can be (always) written as a fixed linear combination of the nontrivial components of ⁢XWXW X and the almost surely zero components of XXX: (⁢X)i=∑j=1mAi⁢j⁢(⁢X)j+∑j=m+1dBi⁢j⁢Xj,subscriptXsuperscriptsubscript1subscriptsubscriptXsuperscriptsubscript1subscriptsubscriptX (PX)_i= _j=1^mA_ij(W% X)_j+ _j=m+1^dB_ijX_j,( P X )i = ∑j = 1m Aitalic_i j ( W X )j + ∑j = m + 1d Bitalic_i j Xitalic_j , i.e. =+P=AW+BVP = AW + BV, where =−+⁢superscriptV=I-W^+WV = I - W+ W is the orthogonal projection onto XXX’s almost surely zero components. The ithsuperscriptthi^thith sub-objective is then: ⁢(T⁢X−Xi)2=⁢[∑j=1m(Ai⁢j−Wi⁢j+)⁢(⁢X)j]2=∑j=1m(Ai⁢j−Wi⁢j+)2,superscriptsuperscriptsubscriptXsubscriptX2superscriptdelimited-[]superscriptsubscript1subscriptsuperscriptsubscriptsubscriptX2superscriptsubscript1superscriptsubscriptsuperscriptsubscript2 (P_i^TX-X_i% )^2=E [ _j=1^m(A_ij-W_ij^+)(W% X)_j ]^2= _j=1^m(A_ij-W_ij^+)^2,blackboard_E ( Pbold_iitalic_T X - Xitalic_i )2 = blackboard_E [ ∑j = 1m ( Aitalic_i j - Witalic_i j+ ) ( W X )j ]2 = ∑j = 1m ( Aitalic_i j - Witalic_i j+ )2 , where we have safely ignored the almost surely zero terms Bi⁢j⁢XjsubscriptsubscriptXB_ijX_jBitalic_i j Xitalic_j (j>mj>mj > m), and used the fact that the first m components of ⁢XWXW X have identity covariance matrix. ⁢XPXP X is almost surely equal to ⁢X AWXAW X, so our constraint Cov⁢(⁢X,Z)=CovXZ0Cov(PX,Z)=0Cov ( P X , Z ) = 0 is equivalent to ⁢X,Z=Cov⁢(⁢X,Z)=subscriptXZCovXZ0A _WX,Z=Cov(% AWX,Z)=0A Σbold_W X , Z = Cov ( AW X , Z ) = 0, i.e. Ai⁢j=0subscript0A_ij=0Aitalic_i j = 0 when j≤ℓj≤ ≤ ℓ, since the first ℓ ℓ components are those for which ⁢XWXW X correlates with ZZZ. Subject to this, the objective is minimized for Ai⁢j=Wi⁢j+subscriptsuperscriptsubscriptA_ij=W_ij^+Aitalic_i j = Witalic_i j+ when j>ℓj> > ℓ, i.e. =+⁢(−⁢XZ)superscriptsubscriptsubscriptXZA=W^+(I-P_W _% XZ)A = W+ ( I - Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT ). The particular choice =B=IB = I gives our solution ∗=−+⁢XZ⁢superscriptsuperscriptsubscriptsubscriptXZP^*=I-W^+P_W _% XZWP∗ = I - W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W, leaving the non-varying components of XXX intact (see Fig. 1 for a visualization). ∎ The solution is unique except for columns corresponding to the components of XXX with zero variance, and rows corresponding to the zero-weighted components of the (pseudo) inner product MM. E.2 Covector Proof Proof. We assume without loss of generality that vectors in ℝdsuperscriptℝR^dblackboard_Rd are represented in a basis diagonalizing the inner product MM, so that ⟨,⟩=∑i=1dmi⁢xi⁢yisubscriptsuperscriptsubscript1subscriptsubscriptsubscript ,y _M= _i=1^dm_ix_iy_i⟨ x , y ⟩M = ∑i = 1d mitalic_i xitalic_i yitalic_i for fixed m1,…,md≥0subscript1…subscript0m_1,…,m_d≥ 0m1 , … , mitalic_d ≥ 0. This allows us to treat each row ∈ℝdsubscriptsuperscriptℝP_i ^dPbold_i ∈ blackboard_Rd of PP as a separate optimization problem, argmin∈ℝd⁢[mi⁢(T⁢X−Xi)2]subject⁢to⁢Cov⁢(T⁢X,Z)=.subscriptargminsubscriptsuperscriptℝdelimited-[]subscriptsuperscriptsuperscriptsubscriptXsubscriptX2subjecttoCovsuperscriptsubscriptXZ0 argmin\>_ subarraycP_i ^d% subarrayE [m_i (P_i^TX-% X_i )^2 ] \>to\>\>Cov(% P_i^TX,Z)=0.start_BIGOP argmin end_BIGOPstart_ARG start_ROW start_CELL P start_POSTSUBSCRIPT i ∈ blackboard_Rd end_CELL end_ROW end_ARG end_POSTSUBSCRIPT blackboard_E [ mitalic_i ( Pbold_iitalic_T X - Xitalic_i )2 ] subject to Cov ( Pbold_iitalic_T X , Z ) = 0 . Our objective only depends on subscriptP_iPbold_i through its effect on the scalar random variable ξ=T⁢XsuperscriptsubscriptXξ=P_i^TXξ = Pbold_iitalic_T X. All random variables888Strictly speaking, equivalence classes of almost surely equal random variables. of the form ζ=ζT⁢XsuperscriptsubscriptXζ=u_ζ^TXζ = uitalic_ζitalic_T X for some covector ζT∈ℝdsuperscriptsubscriptsuperscriptℝu_ζ^T∈ [rgb]0,0,0 [named]pgfstrokecolor% rgb0,0,0 @color@gray@stroke0 @color@gray@fill0R% ^duitalic_ζitalic_T ∈ blackboard_Rd form a vector space U, which we equip with the covariance inner product ⟨ξ,ζ⟩Cov=Cov⁢(ξ,ζ)=⁢[ξ⁢ζ]=ξT⁢X⁢ζsubscriptCovCovdelimited-[]superscriptsubscriptsubscriptXXsubscript ξ,ζ _Cov=Cov(ξ,ζ)=E[ξ% ζ]=u_ξ^T _XXu% _ζ⟨ ξ , ζ ⟩Cov = Cov ( ξ , ζ ) = blackboard_E [ ξ ζ ] = uitalic_ξitalic_T Σroman_X uitalic_ζ. By the linearity of covariance, the elements of U uncorrelated with ZZZ form a subspace Z⟂⊆Usuperscriptperpendicular-toZ UZ⟂ ⊆ U. Note also that ξ∈Z⟂superscriptperpendicular-toξ∈ Z ξ ∈ Z⟂ if and only if ξ’s covector ξTsuperscriptsubscriptu_ξ^Tuitalic_ξitalic_T satisfies Cov⁢(ξT⁢X,Z)=ξT⁢XZ=kCovsuperscriptsubscriptXZsuperscriptsubscriptsubscriptXZsubscript0Cov(u_ξ^TX,Z)=u_ξ^% T _XZ=0_kCov ( uitalic_ξitalic_T X , Z ) = uitalic_ξitalic_T Σroman_XZ = 0italic_k, and that these covectors themselves form the subspace colsp⁢(XZ)⟂colspsuperscriptsubscriptXZperpendicular-tocolsp( _XZ) colsp ( Σroman_XZ )⟂ of ℝdsuperscriptℝ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^dblackboard_Rd. Our objective now reduces to finding a covector iTsuperscriptsubscriptP_i^TPitalic_iitalic_T that defines the orthogonal projection of XisubscriptXX_iXitalic_i onto Z⟂superscriptperpendicular-toZ Z⟂. The difficulty is that orthogonality of elements in U is not equivalent to orthogonality of the corresponding covectors. We can fix this by changing the basis in which covectors are represented. Since X∈colsp⁢()XcolspX (W)X ∈ colsp ( W ) a.s., we can write any element of U as a linear form in ⁢XWXW X rather than XXX by applying the change-of-basis ξ′=+⁢ξsuperscriptsubscript′subscriptu_ξ =W^+u_ξuitalic_ξ′ = W+ uitalic_ξ to every covector: ξ=(ξ′)T⁢X=ξT⁢+⁢Xsuperscriptsuperscriptsubscript′XsuperscriptsubscriptcancelsuperscriptXξ=(u_ξ )^TWX=u_ξ^T% W^+WXξ = ( uitalic_ξ′ )T W X = uitalic_ξitalic_T cancel W+ W X a.s. In this new basis, which is orthonormal under our covariance inner product, each component of XXX is written Xi=(+)iT⁢XsubscriptXsuperscriptsubscriptsuperscriptXX_i=(W^+)_i^TWXXitalic_i = ( W+ )iitalic_T W X and the inner product of any two elements of U is simply the Euclidean inner product of the corresponding covectors:999If XXsubscriptXX _XXΣroman_X is full rank, there is a one-to-one correspondence between random variables in U and covectors. In the singular case, we may choose the component of the covector inside ker⁡(X)kernelsubscriptXX ( _XX)ker ( Σroman_X ) arbitrarily, since it will make no difference to the inner product. ⟨ξ,ζ⟩Cov=Cov⁢(ξ′⁣T⁢X,ζ′⁣T⁢X)=ξ′⁣T⁢X⁢ζ′=ξ′⁣T⁢ζ′.subscriptCovCovsuperscriptsubscript′Xsuperscriptsubscript′Xsuperscriptsubscript′cancelsubscriptXXsuperscriptsubscript′subscript′subscript′ ξ,ζ _Cov=Cov(u_ξ T% WX,u_ζ TWX)=% u_ξ T W _X% XWu_ζ =u_ξ T% u_ζ .⟨ ξ , ζ ⟩Cov = Cov ( uitalic_ξ′ T W X , uitalic_ζ′ T W X ) = uitalic_ξ′ T cancel W Σroman_X W uitalic_ζ′ = uitalic_ξ′ T uitalic_ζ′ . Since the two inner products are now equivalent, and Z⟂superscriptperpendicular-toZ Z⟂ is precisely those random variables with covector ′∈colsp⁢(⁢XZ)⟂superscript′colspsuperscriptsubscriptXZperpendicular-tou (W _X% Z) u′ ∈ colsp ( W Σroman_XZ )⟂, the orthogonal projection of XisubscriptXX_iXitalic_i onto Z⟂superscriptperpendicular-toZ Z⟂ is also an orthogonal projection of its covector (+)iTsuperscriptsubscriptsuperscript(W^+)_i^T( W+ )iitalic_T onto colsp⁢(⁢XZ)⟂colspsuperscriptsubscriptXZperpendicular-tocolsp(W _XZ) colsp ( W Σroman_XZ )⟂: X^i=(+)iT⁢(−⁢XZ)⁢(⁢X)subscript^XsuperscriptsubscriptsuperscriptsubscriptsubscriptXZX X_i=(W^+)_i^T(I-P_ % W _XZ)(WX)over start_ARG X end_ARGi = ( W+ )iitalic_T ( I - Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT ) ( W X ) (6) Putting all the components of XXX together, we have our final solution, X^=(−+⁢XZ⁢)⁢X,^XsuperscriptsubscriptsubscriptXZX X=(I-W^+P_W% _XZW)X,over start_ARG X end_ARG = ( I - W+ Pbold_W Σ start_POSTSUBSCRIPT XZ end_POSTSUBSCRIPT W ) X , which is almost surely equivalent to Eq. 6, but keeps the non-varying components of XXX intact. ∎ Appendix F The Optimality of Oblique Projections As noted in subsection 4.2, the optimal affine erasure function r⁢()=+r(x)=b+Pxr ( x ) = b + Px does not in general use an orthogonal projection for the matrix PP. A simple example illustrates why. Let d=2,k=1formulae-sequence21d=2,k=1d = 2 , k = 1 so that XXX takes values in ℝ2superscriptℝ2R^2blackboard_R2 and ZZZ takes values in ℝRblackboard_R, with the first feature X1subscriptX1X_1X1 and the label ZZZ each independently and uniformly distributed in −1,+111\-1,+1\ - 1 , + 1 , and the second feature X2subscriptX2X_2X2 simply equal to the sum X2=X1+ZsubscriptX2subscriptX1ZX_2=X_1+ZX2 = X1 + Z. A dataset reflecting such a distribution has four (,)(x,y)( x , y ) pairs: ([1,2]T,1),([1,0]T,−1),([−1,0]T,1),([−1,−2]T,−1)superscript121superscript101superscript101superscript121([1,2]^T,1), ([1,0]^T,-1), ([-1,0]^T,1), ([-1,-2]^T,-1)( [ 1 , 2 ]T , 1 ) , ( [ 1 , 0 ]T , - 1 ) , ( [ - 1 , 0 ]T , 1 ) , ( [ - 1 , - 2 ]T , - 1 ) In this case, all of the information XXX has about ZZZ resides in X2subscriptX2X_2X2, so the minimally disruptive orthogonal projection which guards ZZZ will nullify that component: ortho=[1000]subscriptorthomatrix1000P_ortho= bmatrix1&0\\ 0&0 bmatrixPortho = [ start_ARG start_ROW start_CELL 1 end_CELL start_CELL 0 end_CELL end_ROW start_ROW start_CELL 0 end_CELL start_CELL 0 end_CELL end_ROW end_ARG ] On the other hand, X1subscriptX1X_1X1 contains some information about X2subscriptX2X_2X2 (despite having no information about ZZZ), allowing a partial reconstruction of X2subscriptX2X_2X2 while preserving full concept erasure: oblique=[1010]subscriptobliquematrix1010P_oblique= bmatrix1&0\\ 1&0 bmatrixPoblique = [ start_ARG start_ROW start_CELL 1 end_CELL start_CELL 0 end_CELL end_ROW start_ROW start_CELL 1 end_CELL start_CELL 0 end_CELL end_ROW end_ARG ] Both methods fully erase the ability to predict ZZZ from the data, however a simple calculation shows the second, oblique method to perform better as measured by mean squared edit distance: ⁢‖ortho⁢X−X‖2=2,⁢‖oblique⁢X−X‖2=1formulae-sequencesuperscriptnormsubscriptorthoXX22superscriptnormsubscriptobliqueXX21 \|P_orthoX-X\|^% 2=2, \|P_obliqueX-X\|^% 2=1blackboard_E ∥ Portho X - X ∥2 = 2 , blackboard_E ∥ Poblique X - X ∥2 = 1 Appendix G Equivalence of Guardedness Definitions Xu et al. [43] define the conditional VV-entropy of ZZZ given XXX as the lowest achievable cross-entropy loss predicting ZZZ with a function of XXX in the predictor class VV. In our notation: H⁢(Z|X)=infθ∈Θ⁢[ℒ⁢(η⁢(X;),Z)],subscriptconditionalZXsubscriptinfimumΘdelimited-[]ℒXZH_V(Z\ |\ X)= _θ∈ E% [ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η(% X; θ),Z)],Hcaligraphic_V ( Z | X ) = infitalic_θ ∈ Θ blackboard_E [ L ( η ( X ; italic_θ ) , Z ) ] , where ℒ⁢(η,z)=−log⁡exp⁡(ηz)∑i=1kexp⁡(ηi)ℒsubscriptsuperscriptsubscript1subscript [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L(η,z)=-% ( _z) _i=1^k ( _i)L ( η , z ) = - log divide start_ARG exp ( ηitalic_z ) end_ARG start_ARG ∑i = 1k exp ( ηitalic_i ) end_ARG is the cross-entropy loss function. They then define the (unconditional) VV-entropy H⁢(Z)=H⁢(Z| 0)subscriptZsubscriptconditionalZ 0H_V(Z)=H_V(Z\ |\ 0)Hcaligraphic_V ( Z ) = Hcaligraphic_V ( Z | 0 ) to be the lowest achievable cross-entropy loss in the case of a constantly null random data variable. This is exactly our trivially attainable loss LτsubscriptL_τLitalic_τ (Definition 2.2). Finally, they define the VV-information from XXX to ZZZ as the reduction in VV-entropy as compared to using such a null random data variable: I⁢(X→Z)=H⁢(Z)−H⁢(Z|X).subscript→XZsubscriptZsubscriptconditionalZXI_V(X→Z)=H_V(Z)-H_% V(Z\ |\ X).Icaligraphic_V ( X → Z ) = Hcaligraphic_V ( Z ) - Hcaligraphic_V ( Z | X ) . Using these notions, Ravfogel et al. [33] say that XXX is ϵitalic-ϵεϵ-guarded with respect to VV if I⁢(X→Z)<ϵsubscript→XZitalic-ϵI_V(X~→~Z)< _V ( X → Z ) < ϵ. In Appendix B, we showed the equivalence of guardedness (as we have defined it in Definition 2.1) to the optimality of the trivially attainable loss. That is, XXX (,)(V, L)( V , L )-guards ZZZ when H⁢(Z|X)=Lτ=H⁢(Z)subscriptconditionalZXsubscriptsubscriptZH_V(Z\ |\ X)=L_τ=H_V(% Z)Hcaligraphic_V ( Z | X ) = Litalic_τ = Hcaligraphic_V ( Z ), in the case where LL is the singleton class consisting solely of the cross-entropy loss function. In the language of [33], XXX is ϵitalic-ϵεϵ-guarded with respect to VV for all ϵ>0italic-ϵ0ε>0ϵ > 0. Appendix H Constraining Norm Growth In early concept scrubbing experiments (Sec. 6), we found that at specific layers in some models, concept scrubbing with LEACE would cause the norm of the embedding to diverge, leading to NaN outputs. By contrast, SAL never caused divergence, even though it causes a larger disruption to model performance on average (Table 1). This is because SAL uses an orthogonal projection QQ, whose eigenvalues are thus all in 0,101\0,1\ 0 , 1 , so the norm of the hidden state can never increase after erasure, while LEACE’s oblique projection matrix PP does generally have singular values greater than 1. To combine the superior average-case MSE of LEACE with the stability of SAL, we adopt a simple regularization heuristic. After constructing PP, we analytically compute the trace of the covariance matrix of the hidden states after applying PP. If tr⁢(⁢X⁢)>tr⁢(X)trsubscriptXXsuperscripttrsubscriptXXtr(P _XXP^T)> % tr( _XX)tr ( P Σroman_X Pbold_T ) > tr ( Σroman_X ), we solve a quadratic equation to find the convex combination ′=α⁢+(1−α)⁢superscript′1P = +(1-α)QP′ = α P + ( 1 - α ) Q such that tr⁢(X)=tr⁢(′⁢X⁢(′))trsubscriptXXtrsuperscript′subscriptXXsuperscriptsuperscript′tr( _XX)=tr(P^% _XX(P )^T)tr ( Σroman_X ) = tr ( P′ Σroman_X ( P′ )T ). By Theorem 4.1, the set of matrices which ensure linear guardedness is convex,101010In fact, it is a subspace of ℝd×dsuperscriptℝ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^d× dblackboard_Rd × d. For any matrices ,∈ℝd×dsuperscriptℝA,B∈ [rgb]0,0,0 [named]pgfstrokecolor% rgb0,0,0 @color@gray@stroke0 @color@gray@fill0R^% d× dA , B ∈ blackboard_Rd × d such that ⁢XZ=subscriptXZ0A _XZ=0A Σroman_XZ = 0 and ⁢XZ=subscriptXZ0B _XZ=0B Σroman_XZ = 0, we have by linearity (α⁢+β⁢)⁢XZ=α⁢XZ+β⁢XZ=α⁢+β⁢=subscriptXZsubscriptXZsubscriptXZ000( + ) _XZ=% _XZ+ % _XZ= 0+ 0=% 0( α A + β B ) Σroman_XZ = α A Σroman_XZ + β B Σroman_XZ = α 0 + β 0 = 0 for any scalars α and β. so ′superscript′P P′ is guaranteed to be in the feasible set. Furthermore, since our mean squared error objective is convex, ′superscript′P P′ is guaranteed to have no worse MSE than QQ. We find this solves the divergence issue in practice. Appendix I Oracle LEACE Figure 5: Orthogonal projection of iiith component of XXX, itself a vector in the random variable Hilbert space ℋHH, onto the span of the components of ZZZ. The residual Xi−proj⁢XisubscriptXsubscriptprojsubscriptXX_i-proj_ZX_iXitalic_i - projcaligraphic_Z Xitalic_i is the closest vector to XisubscriptXX_iXitalic_i orthogonal to, and hence uncorrelated with, =span⁢(Z1,Z2)spansubscriptZ1subscriptZ2Z=span(\Z_1,Z_2\)Z = span ( Z1 , Z2 ). The concept erasure method derived in Section 4 does not require access to concept labels at inference time. That is, we can fit an erasure function on a labeled training dataset, then apply the function to unlabeled datapoints. If we have oracle access to the label zitalic_z for each xitalic_x, we can achieve an even more surgical edit. In Theorem I.1 below, we derive Oracle LEACE, a closed-form formula for the the nearest X′X X′ to any XXX such that Cov⁢(X′,Z)=0CovsuperscriptX′Z0Cov(X ,Z)=0Cov ( X′ , Z ) = 0. Like in Sec. 4, the resulting XLEACE′subscriptsuperscriptX′LEACEX _LEACEX′roman_LEACE is “nearest” to XXX with respect to all p.s.d. inner products T⁢superscripta^TMbaitalic_T Mb defined on ℝdsuperscriptℝ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^dblackboard_Rd simultaneously. This is because, by expressing XXX in a basis that diagonalizes MM, we can decompose the problem into d independent subproblems, one for each component of XisubscriptXX_iXitalic_i. Each subproblem can then be viewed as an orthogonal projection, not in ℝdsuperscriptℝ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^dblackboard_Rd, but in an abstract vector space of real-valued random variables. For geometric intuition, see Figure 5. Prior work has noted that computing an orthogonal projection in a random variable Hilbert space is equivalent to solving an ordinary least squares regression problem [1]. Our theorem is a natural extension of this work: we find that XLEACE′subscriptsuperscriptX′LEACEX _LEACEX′roman_LEACE is equal to the OLS residual from regressing XXX on ZZZ, plus a constant shift needed to ensure that erasing ZZZ does not change the mean of XXX. Theorem I.1 (Oracle Concept Erasure). Let ℋHH be the Hilbert space of square-integrable real-valued random variables equipped with the inner product ⟨ξ,ζ⟩ℋ:=⁢[ξ⁢ζ]assignsubscriptℋdelimited-[] ξ,ζ _H:=E[ξζ]⟨ ξ , ζ ⟩H := blackboard_E [ ξ ζ ]. Let (X,Z)XZ(X,Z)( X , Z ) be random vectors in ℋdsuperscriptℋH^dHitalic_d and ℋksuperscriptℋH^kHitalic_k respectively. Then for every p.s.d. inner product ⟨,⟩=T⁢subscriptsuperscript ,b _M=a^TM% b⟨ a , b ⟩M = aitalic_T Mb on ℝdsuperscriptℝ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0R^dblackboard_Rd, the objective argminX′∈ℋd⁢‖X′−X‖2subject⁢to⁢Cov⁢(X′,Z)=subscriptargminsuperscriptX′ℋsubscriptsuperscriptnormsuperscriptX′X2subjecttoCovsuperscriptX′Z0 argmin\>_ subarraycX^% ^d subarrayE \|X % -X \|^2_M \>to\>\>% Cov(X ,Z)=0start_BIGOP argmin end_BIGOPstart_ARG start_ROW start_CELL X′ ∈ Hitalic_d end_CELL end_ROW end_ARG blackboard_E ∥ X′ - X ∥2M subject to Cov ( X′ , Z ) = 0 is minimized by the (appropriately shifted) ordinary least squares residuals from regressing XXX on ZZZ: XLEACE′=X−XZ⁢Z+⁢(Z−⁢[Z]).subscriptsuperscriptX′LEACEXsubscriptXZsuperscriptsubscriptZZZdelimited-[]Z X _LEACE=X-% _XZ _ZZ^% + (Z-E[Z] ).X′roman_LEACE = X - Σroman_XZ Σroman_Z+ ( Z - blackboard_E [ Z ] ) . Proof. Assume w.l.o.g. that XXX and X′X X′ are represented in a basis diagonalizing MM, so we may write ⁢‖X′−X‖2=∑i=1dmi⁢[(Xi′−Xi)2],subscriptsuperscriptnormsuperscriptX′X2superscriptsubscript1subscriptdelimited-[]superscriptsubscriptsuperscriptX′subscriptX2 \|X -X \|^2_% M= _i=1^dm_i\>E [(X _i-% X_i)^2 ],blackboard_E ∥ X′ - X ∥2M = ∑i = 1d mitalic_i blackboard_E [ ( X′italic_i - Xitalic_i )2 ] , where m1,…,md≥0subscript1…subscript0m_1,…,m_d≥ 0m1 , … , mitalic_d ≥ 0 are eigenvalues of MM. Crucially, each term in this sum is independent from the others, allowing us to decompose the primal problem into d separate subproblems of the form ‖Xi′−Xi‖ℋ2subscriptsuperscriptnormsuperscriptsubscriptX′subscriptX2ℋ\|X_i -X_i\|^2_H∥ Xitalic_i′ - Xitalic_i ∥2H, one for each component i of (X,X′)XsuperscriptX′(X,X )( X , X′ ). Factoring out constants. Now consider the subspace =span⁢(1)⊂ℋspan1ℋC=span(1) = span ( 1 ) ⊂ H consisting of all constant (i.e. zero variance) random variables. Orthogonally decomposing XisubscriptXX_iXitalic_i along CC yields Xi=X~i+μisubscriptXsubscript~XsubscriptX_i= X_i+ _iXitalic_i = over~ start_ARG X end_ARGi + μitalic_i, where μi=⁢[Xi]∈subscriptdelimited-[]subscriptX _i=E[X_i] μitalic_i = blackboard_E [ Xitalic_i ] ∈ C and X~i=X−⁢[X]i∈⟂subscript~XXsubscriptdelimited-[]Xsuperscriptperpendicular-to X_i=X-E[X]_i∈% C over~ start_ARG X end_ARGi = X - blackboard_E [ X ]i ∈ C⟂, and likewise for Xi′subscriptX′X_i Xitalic_i′. Our objective is now ‖Xi′−Xi‖ℋ2=‖μi′−μi‖ℋ2+‖X~i′−X~i‖ℋ2.subscriptsuperscriptnormsuperscriptsubscriptX′subscriptX2ℋsubscriptsuperscriptnormsuperscriptsubscript′subscript2ℋsubscriptsuperscriptnormsuperscriptsubscript~X′subscript~X2ℋ \|X_i -X_i \|^2_H=% \| _i - _i \|^2_H+ \| % X_i - X_i \|^2_H.∥ Xitalic_i′ - Xitalic_i ∥2H = ∥ μitalic_i′ - μitalic_i ∥2H + ∥ over~ start_ARG X end_ARGi′ - over~ start_ARG X end_ARGi ∥2H . (7) Since μi′subscript′ _i μitalic_i′ and μisubscript _iμitalic_i are orthogonal to X~i′subscript~X′ X_i over~ start_ARG X end_ARGi′ and X~isubscript~X X_iover~ start_ARG X end_ARGi, and the constraint Cov⁢(X′,Z)=CovsuperscriptX′Z0Cov(X ,Z)=0Cov ( X′ , Z ) = 0 is invariant to constant shifts, we can optimize the two terms in Eq. 7 independently. The first term is trivial: it is minimized when μi′=μisuperscriptsubscript′subscript _i = _iμitalic_i′ = μitalic_i, and hence Xi′=X~i′+⁢[Xi]superscriptsubscriptX′subscript~X′delimited-[]subscriptXX_i = X_i +E[% X_i]Xitalic_i′ = over~ start_ARG X end_ARGi′ + blackboard_E [ Xitalic_i ]. Orthogonal projection. We can now rewrite the zero covariance condition as an orthogonality constraint on X~isubscript~X X_iover~ start_ARG X end_ARGi. Specifically, for every i∈1⁢…⁢d1…i∈\1… d\i ∈ 1 … d we have argminX~i′∈ℋ∥X~i′−X~i∥ℋ2s.t.∀j∈1…k:⟨X~i′,Z~j⟩ℋ=0, argmin\>_ subarrayc X_i^% subarray \| X_i -% X_i \|^2_H .t.\>\>% ∀ j∈\1… k\: X_i , % Z_j _H=0,start_BIGOP argmin end_BIGOPstart_ARG start_ROW start_CELL over~ start_ARG X end_ARG start_POSTSUBSCRIPT i′ ∈ H end_CELL end_ROW end_ARG end_POSTSUBSCRIPT ∥ over~ start_ARG X end_ARGi′ - over~ start_ARG X end_ARGi ∥2H s . t . ∀ j ∈ 1 … k : ⟨ over~ start_ARG X end_ARGi′ , over~ start_ARG Z end_ARGj ⟩H = 0 , (8) where Z~=Z−⁢[Z]~ZZdelimited-[]Z Z=Z-E[Z]over~ start_ARG Z end_ARG = Z - blackboard_E [ Z ]. In other words, we seek the nearest X~i′subscript~X′ X_i over~ start_ARG X end_ARGi′ to X~isubscript~X X_iover~ start_ARG X end_ARGi orthogonal to =span⁢(Z~1,…,Z~k)spansubscript~Z1…subscript~ZZ=span(\ Z_1,…, Z% _k\)Z = span ( over~ start_ARG Z end_ARG1 , … , over~ start_ARG Z end_ARGk ), which is simply the orthogonal projection of X~isubscript~X X_iover~ start_ARG X end_ARGi onto ⟂superscriptperpendicular-toZ Z⟂. This in turn is equal to the ordinary least squares residual from regressing X~~X Xover~ start_ARG X end_ARG on Z~~Z Zover~ start_ARG Z end_ARG: X~i′=X~i−proj⁢(X~i,)=Xi−(XZ)i⁢Z+⁢(Z−⁢[Z])−⁢[Xi].superscriptsubscript~X′subscript~Xprojsubscript~XsubscriptXsubscriptsubscriptXZsuperscriptsubscriptZZZdelimited-[]Zdelimited-[]subscriptX X_i = X_i-proj (% X_i,Z )=X_i-( % _XZ)_i _ZZ^+% (Z-E[Z])-E[X_i].over~ start_ARG X end_ARGi′ = over~ start_ARG X end_ARGi - proj ( over~ start_ARG X end_ARGi , Z ) = Xitalic_i - ( Σroman_XZ )i Σroman_Z+ ( Z - blackboard_E [ Z ] ) - blackboard_E [ Xitalic_i ] . (9) Putting it all together. Plugging Eq. 9 into Xi′=X~i′+⁢[Xi]superscriptsubscriptX′subscript~X′delimited-[]subscriptXX_i = X_i +E[% X_i]Xitalic_i′ = over~ start_ARG X end_ARGi′ + blackboard_E [ Xitalic_i ] and combining all components into vector form yields XLEACE′=X−XZ⁢Z+⁢(Z−⁢[Z]),subscriptsuperscriptX′LEACEXsubscriptXZsuperscriptsubscriptZZZdelimited-[]ZX _LEACE=X- _% XZ _ZZ^+(Z-% E[Z]),X′roman_LEACE = X - Σroman_XZ Σroman_Z+ ( Z - blackboard_E [ Z ] ) , (10) which completes the proof. ∎ Appendix J Notation Key ZZ The space of one-hot labels (z1,…zk)∈0,1k|∑j=1kzj=1\(z_1,… z_k)∈\0,1\^k\ |\ _j=1^kz_j=1\\ ( z1 , … zitalic_k ) ∈ 0 , 1 k | ∑j = 1k zitalic_j = 1 (treated interchangeably with the integers 1,…,k1…\1,…,k\ 1 , … , k when convenient). X,ZXZX,ZX , Z Integrable (i.e. finite first moment) random vectors taking values in ℝdsuperscriptℝR^dblackboard_Rd and ℝksuperscriptℝR^kblackboard_Rk respectively (or their realized values inside an expectation, e.g. in ⁢[f⁢(X)]delimited-[]XE[f(X)]blackboard_E [ f ( X ) ]). ZZZ is sometimes restricted to the one-hot labels ZZ, in which case we assume each ℙ⁢(Z=j)>0ℙZ0P(Z=j)>0blackboard_P ( Z = j ) > 0. Xi,ZjsubscriptXsubscriptZX_i,Z_jXitalic_i , Zitalic_j The ithsuperscriptthi^thith and jthsuperscriptthj^thjth components thereof, themselves scalar random variables (or their realized values inside an expectation). ξ,ζξ,ζξ , ζ Scalar random variables taking values in ℝRblackboard_R. η A predictor function ℝd→superscriptℝR^d _Rd → Z (or its value η⁢(X)Xη(X)η ( X ) when inside an expectation). VV A space of predictor functions η⁢(⋅;):ℝd→ℝk|∈Θconditional-set⋅→superscriptℝconditionalsuperscriptℝΘ\η(·; θ):R^d ^k\ |\ % θ∈ \ η ( ⋅ ; italic_θ ) : blackboard_Rd → blackboard_Rk | italic_θ ∈ Θ , parameterized by θitalic_θ and containing all constant functions. LL A space of loss functions ℒ:ℝk×→[0,∞)conditional-setℒ→superscriptℝ0\ [rgb]0,0,0 [named]pgfstrokecolorrgb0,0,0% @color@gray@stroke0 @color@gray@fill0L:R^% k×Z→[0,∞)\ L : blackboard_Rk × Z → [ 0 , ∞ ) . r An erasure function ℝd→ℝd→superscriptℝsuperscriptℝR^d ^dblackboard_Rd → blackboard_Rd, hopefully making a minimal edit to XXX that eliminates the ability to predict labels ZZZ with predictors in VV. AA A matrix with entries in ℝRblackboard_R. Ai⁢jsubscriptA_ijAitalic_i j The entry thereof at the ithsuperscriptthi^thith row and jthsuperscriptthj^thjth column. +superscriptA^+A+ The Moore-Penrose pseudoinverse of AA. vv A column vector with entries in ℝRblackboard_R. visubscriptv_ivitalic_i The ithsuperscriptthi^thith component thereof.

Instant research discovery