Paper deep dive
Operational Abstractions of Neural Network Concepts via Topological Representations
Mathieu Pont, Christoph Garth
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Concept-based methods provide a semantic level for interpreting and manipulating learned representations, but existing editing approaches are typically specialized to particular interventions and do not provide a common and editable representation of concept organization. To achieve this, we introduce Topological Concept Representations (TCR), a post-hoc operational abstraction that jointly characterizes the concepts encoded in a learned representation and their relationships. TCR constructs an intermediate concept space from concept recoverability and interaction scores, and compactly encodes its organization through topology. Interventions are expressed through modifications of this abstraction that are propagated back to the underlying learned representation. This allows different concept-level operations to share the same optimization framework and separates the desired concept organization from the mechanism to achieve it. We establish stability and reparameterization-invariance properties of TCR and its connections to existing concept-editing formulations. We use TCR to disentangle concepts as a preprocessing step for existing erasure methods, improving worst-group accuracy by 21.89 on average at comparable concept leakage. We further use TCR to transfer concepts from teacher to student models, improving concept recoverability by up to 5.54 while also improving or maintaining competitive test top-1 accuracy.
Tags
Links
- Source: https://arxiv.org/abs/2610.05017v1
- Canonical: https://arxiv.org/abs/2610.05017v1
Trouble viewing inline? Open PDF directly →
Full Text
152,459 characters extracted from source content.
Expand or collapse full text
Operational Abstractions of Neural Network Concepts via Topological Representations Mathieu Pont & Christoph Garth Affiliation: RPTU Kaiserslautern-Landau Affiliation: Kaiserslautern, Germany Email: mathieu.pont,garth@rptu.de Affiliation: Ji Q. Ren & Yevgeny LeNet Affiliation: Department of Computational Neuroscience Affiliation: University of the Witwatersrand Affiliation: Joburg, South Africa Email: robot,net@wits.ac.za Coauthor Affiliation: Affiliation Affiliation: Address Affiliation: email Abstract Concept-based methods provide a semantic level for interpreting and manipulating learned representations, but existing editing approaches are typically specialized to particular interventions and do not provide a common and editable representation of concept organization. To achieve this, we introduce Topological Concept Representations (TCR), a post-hoc operational abstraction that jointly characterizes the concepts encoded in a learned representation and their relationships. TCR constructs an intermediate concept space from concept recoverability and interaction scores, and compactly encodes its organization through topology. Interventions are expressed through modifications of this abstraction that are propagated back to the underlying learned representation. This allows different concept-level operations to share the same optimization framework and separates the desired concept organization from the mechanism to achieve it. We establish stability and reparameterization-invariance properties of TCR and its connections to existing concept-editing formulations. We use TCR to disentangle concepts as a preprocessing step for existing erasure methods, improving worst-group accuracy by 21.8921.89 on average at comparable concept leakage. We further use TCR to transfer concepts from teacher to student models, improving concept recoverability by up to 5.545.54 while also improving or maintaining competitive test top-1 accuracy. Figure 1: TCR provides a common post-hoc abstraction for concept organization in learned representations. It maps encoded concepts and their interactions to a concept mesh and summarizes its topological structure. Desired changes are specified on this abstraction and propagated back to the representation, enabling diverse interventions such as concept erasure, disentanglement and transfer. 1 Introduction As computational resources and learning methods continue to advance, increasingly large and complex models have become trainable (Kaplan et al., 2020), giving rise to rich and complex learned representations (Ansuini et al., 2019). Their internal organization is often not directly interpretable, making it difficult to understand and manipulate them in terms of semantically meaningful properties we wish to reason about. A natural way to address this is to seek higher-level abstractions of these representations that would capture such properties in a compact way. This notably raises the question of which operational abstractions can be defined for such representations, that, in addition of capturing these properties, would offer a principled and unified interface to manipulate them. The concept-based literature provides a natural semantic level for reasoning about learned representations by identifying human-interpretable properties encoded within them (Bau et al., 2017; Kim et al., 2018; Fel et al., 2023a). Beyond interpretation, such concepts can also be manipulated, notably by erasing unwanted information from a representation (Ravfogel et al., 2020; Ravfogel et al., 2022; Belrose et al., 2023), preserving downstream task information during erasure (Holstege et al., 2024; Holstege et al., 2025; Akbari et al., 2025) or transferring concept-related information across models (Gupta et al., 2023). However, these methods are typically developed as independent frameworks tailored to a particular intervention and optimization algorithm. Consequently, changing the intended operation or introducing additional requirements often requires deriving a new formulation and optimization procedure rather than simply specifying a different desired outcome. Specifically, they are not formulated within a common framework in which concepts and their interactions are jointly characterized and different concept-level objectives can be expressed, composed and optimized. Moreover, existing methods generally do not treat concept relationships as quantities that can be adjusted to influence the outcome of an intervention. This fragmentation motivates a common post-hoc representation in which concept organization itself can be described, compared and manipulated in a unified way. Topological Data Analysis (TDA) provides natural tools for constructing such abstractions (Edelsbrunner & Harer, 2009; Zomorodian, 2010). The resulting objects compactly characterize the global organization of their input, while their multiscale nature accounts for relationships of varying strengths without committing to a single arbitrary threshold. They can also be compared in a principled manner and, beyond their descriptive role, can serve as operational proxies: when coupled with appropriate optimization procedures, modifications expressed on the abstraction can be translated into controlled modifications of the underlying input (Carriere et al., 2021). We build on these properties to introduce Topological Concept Representations (TCR), a post-hoc operational abstraction of the concept organization encoded in learned representations. TCR first constructs an intermediate concept space in which individual concepts are represented together with their relationships, characterized by scalar scores derived from the learned representation. This space is then compactly encoded through a topological abstraction, which provides an interface for acting on the concept organization and, through it, on the underlying learned representation. Concretely, an intervention is specified by modifying this abstraction to define a target TCR, and optimizing a transformation of the learned representation to realize it. In this way, different operations are expressed through different target topologies within a shared optimization framework, thereby separating what concept organization is desired from how the representation is modified to achieve it. We evaluate TCR on complementary concept editing and transfer tasks. For concept erasure, we consider both direct erasure with TCR and its use as a preprocessing step that disentangles unwanted concepts from task-relevant information before applying existing erasure methods; across datasets and baselines, this improves test WGA by 21.8921.89 on average at comparable levels of concept leakage. For concept transfer, we use the TCR of a teacher representation as a target for a student and show that the transfer generalizes beyond the concepts explicitly included in the optimization, improving concept recoverability by up to 5.545.54 while improving or maintaining competitive test top-1 accuracy. Contributions. Our main contributions are as follows: • We introduce TCR, a post-hoc topological abstraction that jointly represents concepts and their relationships and provides a common interface for concept-level interventions. • We establish theoretical properties of TCR such as stability, invariance to representation reparameterization and its connections to existing concept-editing formulations. • We evaluate TCR across concept erasure, disentanglement and transfer, showing improved task preservation during erasure and improved transfer of concept recoverability. Related work. We provide an extended discussion of concept representation and intervention methods as well as topological abstractions and persistence-based optimization in Appendix A. 2 Preliminaries Concepts and probes. Given a representation function h:→ℝdh:X ^d and a collection of N samples x1,…,xN\x_1,…,x_N\, we denote their representations by H=[h(x1),…,h(xN)]⊤∈ℝN×dH=[h(x_1),…,h(x_N)] ^N× d. A concept CiC_i is described by a vector i=(ci,1,…,ci,N)⊤c_i=(c_i,1,…,c_i,N) , indicating its presence or strength for each sample. Such concept annotations may be provided explicitly or obtained through a concept discovery procedure. A common way to characterize how a concept is encoded in a representation is through a probe qi:ℝd→ℝq_i:R^d predicting ci,nc_i,n from h(xn)h(x_n). For instance, a linear probe qi(h)=i⊤h+biq_i(h)=w_i h+b_i associates the concept with a direction iw_i in the representation space. The ability of such a probe to predict ic_i can then be used to quantify how strongly the concept is encoded in the representation. Persistence diagrams. Let a scalar field f:V→ℝf:V defined on the vertex set V of a simplicial complex K. The topological features of f can be identified using the lower-star filtration and encoded in the persistence diagram (Edelsbrunner & Harer, 2009). Let =(f(v1),…,f(vnv))∈ℝ|V|f= (f(v_1),…,f(v_n_v) ) ^|V| be the vector of vertex values of f and let the filtration map ℱ:ℝ|V|→ℝ||F:R^|V| ^|K| mapping a vector of vertex values to a ℝ||R^|K|-dimensional vector of simplex values where ℱσ()=maxv∈σf(v)F_σ(f)= _v∈σf(v) for each simplex σ∈σ , ensuring that ℱτ()≤ℱσ()F_τ(f) _σ(f) for any τ⊆στ σ. The lower-star filtration is the nested family of subcomplexes a=σ∈|ℱσ()≤aK_a=\σ ~|~F_σ(f)≤ a\ with a⊆a′K_a _a for any a≤a′a≤a . Assuming f injective on the vertices of K, obtained easily with symbolic perturbations (Edelsbrunner & Mucke, 1990), the filtration changes only at the nv=|V|n_v=|V| distinct vertex values, represented by the finite sequence ∅⊂f(v1)⊂…⊂f(vnv)= _f(v_1)⊂… _f(v_n_v)=K (such that f(v1)<⋯<f(vnv)f(v_1)<…<f(v_n_v)) where each step adds a vertex v as well as its lower star (i.e. each simplex σ containing v such that ℱσ()=f(v)F_σ(f)=f(v)). Topological changes, such as the appearance and merging of connected components, occur at specific filtration values. When two components merge, the one created later dies according to the Elder rule (Edelsbrunner & Harer, 2009) and defines a persistence pair (v,v′)(v,v ), where v creates the component and v′v merges it; f(v)f(v) and f(v′)f(v ) are respectively its birth and death, and f(v′)−f(v)f(v )-f(v) its persistence, commonly interpreted as a measure of importance. The component born at the global minimum never dies, and we conventionally assign it the global maximum as death value. The persistence diagram (f)D(f) is the multiset of points (f(v),f(v′)) (f(v),f(v ) ) associated with all persistence pairs. Wasserstein distances. Two persistence diagrams can be compared using the Wasserstein distance, which seeks a minimum-cost matching between their points. Each off-diagonal point can either be matched to exactly one point of the other diagram, associating two topological features, or to the diagonal, corresponding to the insertion or deletion of a feature. In practice, for diagrams (fi)D(f_i) and (fj)D(f_j), each is augmented with the diagonal projections of the off-diagonal points of the other, e.g. ′(fi)=(fi)∪Δ(pj)∣pj∈(fj)D (f_i)=D(f_i)∪\ (p_j) p_j (f_j)\, where Δ(x,y)=((x+y)/2,(x+y)/2) (x,y)=((x+y)/2,(x+y)/2). This accounts for the infinite multiplicity of the diagonal and ensures |′(fi)|=|′(fj)||D (f_i)|=|D (f_j)|, so that the distance can be computed as a bijective matching between the augmented diagrams. Two points pi=(xi,yi)∈′(fi)p_i=(x_i,y_i) (f_i) and pj=(xj,yj)∈′(fj)p_j=(x_j,y_j) (f_j) are mapped with a cost of dq(pi,pj)q=|xi−xj|q+|yi−yj|q=‖pi−pj‖qqd_q(p_i,p_j)^q=|x_i-x_j|^q+|y_i-y_j|^q=\|p_i-p_j\|_q^q, with q≥1q≥ 1 and where dq(pi,pj)d_q(p_i,p_j) is set to zero if both pip_i and pjp_j are diagonal points. The LqL^q-Wasserstein distance WqW_q is then defined as: Wq((fi),(fj))= W_q (D(f_i),D(f_j) )= minϕ∈Φ -4.26773pt φ∈ -4.26773pt (∑pi∈′(fi)dq(pi,ϕ(pi))q)1/q, ( _p_i (f_i)d_q (p_i,φ(p_i) )^q )^1/q, (1) where Φ is the set of all possible assignments ϕφ mapping each point pi∈′(fi)p_i (f_i) to exactly one point pj∈′(fj)p_j (f_j) being either an off-diagonal point or the diagonal projection of pip_i (i.e. ϕ(pi)=Δ(pi)=pj∈′(fj)φ(p_i)= (p_i)=p_j (f_j)) indicating its deletion. WqW_q can be computed using traditional assignment problem solvers (Munkres, 1957; Bertsekas, 1981; Cuturi, 2013; Kerber et al., 2017). Persistence-based topological optimization. Persistence diagrams can be incorporated into differentiable optimization objectives (Carriere et al., 2021). Let :ℝ||→ℝ||P:R^|K| ^|K| denote the persistence map, being a permutation of the simplex filtration values returned by ℱF according to the persistence pairing, such that the birth and death values of each pair are placed consecutively (including zero-persistence pairs). Gradients with respect to birth and death values can therefore be propagated back to the simplices inducing them and, through ℱF, to the vertex values defining the filtration. A persistence-based objective can then be written as ℒ=ℰ∘ℱL=E , where ℰE is a differentiable energy defined on the persistence representation, such as one derived from the Wasserstein distance of Equation 1. Under mild regularity conditions, such objectives are differentiable almost everywhere, admit well-defined generalized subgradients at non-smooth points and can be optimized with stochastic subgradient methods with convergence guarantees to critical points (Carriere et al., 2021). 3 Topological Concept Representations 3.1 Overview and Problem Formulation Given N samples x1,…,xN\x_1,…,x_N\, let H=[h(x1),…,h(xN)]⊤∈ℝN×dH=[h(x_1),…,h(x_N)] ^N× d denote their representations, such as those extracted at a selected model layer, and let =C1,…,CnCC=\C_1,…,C_n_C\ be a collection of concepts defined on these samples. Our objective is to construct an abstraction of H that jointly characterizes which concepts are encoded in the representation and how these concepts interact with each other, and that can subsequently be used as an operational proxy to modify H. To this end, we introduce Topological Concept Representations (TCR). We associate to each concept CiC_i a scalar value mim_i characterizing how strongly CiC_i is encoded in H, and to each pair of concepts (Ci,Cj)(C_i,C_j) an interaction value sijs_ij. These values define a scalar field on a concept mesh in which concepts form maxima and their interactions form saddle vertices. The persistence diagram of this scalar field, the TCR of H, summarizes both the presence of individual concepts and their organization with respect to the other concepts. Beyond being descriptive, the resulting abstraction permits to express a desired intervention by defining a target TCR, e.g., by reducing or increasing a concept maximum or a saddle connecting two concepts. Considering a parameterized transformation Tθ:ℝd→ℝd′T_θ:R^d ^d , applied row-wise to H as Tθ(H):=[Tθ(h1),…,Tθ(hN)]⊤∈ℝN×d′T_θ(H):=[T_θ(h_1),…,T_θ(h_N)] ^N× d , we optimize θ such that the TCR of Tθ(H)T_θ(H) matches this target. Hence, different concept manipulations can be formulated as modifications of a common abstraction rather than through operation-specific objectives. 3.2 Constructing Topological Concept Representations Given a representation Z∈ℝN×d′Z ^N× d , we first associate scalar values with individual concepts, then measure their pairwise interactions and combine them into a topological abstraction. Concept scores. At the framework level, TCR represents simply a concept score by a scalar mi=S(Z,Ci)m_i=S(Z,C_i) whose value increases with how much the concept CiC_i is encoded in the representation. While different choices of S can instantiate different notions of concept encoding, in this work we primarily quantify it through recoverability. Let i=(ci,1,…,ci,N)⊤c_i=(c_i,1,…,c_i,N) denote the sample-level scores of CiC_i, and let Z=[z1,…,zN]⊤∈ℝN×d′Z=[z_1,…,z_N] ^N× d denote the representation on which the concept score is evaluated. Given a probe qϕiq_ _i and a prediction loss ℓ , we define the concept score as: mi=ℒi0−ℒi⋆,ℒi0=mina1N∑n=1Nℓ(ci,n,a),ℒi⋆=minψi1N∑n=1Nℓ(ci,n,qψi(zn)),m_i=L_i^0-L_i , _i^0= _a 1N _n=1^N (c_i,n,a), _i = _ _i 1N _n=1^N \! (c_i,n,q_ _i(z_n) ), (2) where ψi _i denotes the fitted probe parameters and ℒi0L_i^0 is the loss of the optimal constant predictor, which does not use Z and thus provides a baseline for prediction without information from the representation. Thus, mim_i is the improvement of the fitted probe over the optimal constant predictor where mi=0m_i=0 indicates no improvement over the null predictor, while larger values indicate that CiC_i is more readily predicted from Z. We consider binary cross-entropy (BCE) with a logistic probe for binary concepts, and mean squared error (MSE) with a least-squares probe for binary or continuous concepts. The score mim_i can optionally be normalized by ℒi0L_i^0, yielding a dimensionless score facilitating the joint treatment of concepts evaluated with different prediction losses, e.g., BCE for binary and MSE for continuous concepts. For MSE, it corresponds to an R2R^2-type explained-variance score. The MSE formulation additionally admits a convenient analytical form described in Appendix B. Concept interactions. Individual concept scores characterize how much a concept is encoded in a representation, but not how concepts relate to one another. We therefore associate every pair (Ci,Cj)(C_i,C_j) with a normalized interaction score ρij∈[0,1] _ij∈[0,1], where ρij=0 _ij=0 indicates no interaction according to the chosen criterion and ρij=1 _ij=1 indicates maximal interaction. While some interaction criteria are naturally symmetric, others characterize a directional relationship ri→j∈[0,1]r_i→ j∈[0,1]. Directional scores can be combined to associate each concept pair with a single interaction value: ρij=Ψ(ri→j,rj→i) _ij= \! (r_i→ j,r_j→ i ), where Ψ:[0,1]2→[0,1] :[0,1]^2→[0,1] is a symmetric aggregation function, such as Ψ(a,b)=(a+b)/2 (a,b)=(a+b)/2. We principally instantiate this construction through interference. For instance, a representation may strongly encode both an undesired concept and the task while relying on shared information to encode them, such that removing the former also reduces the score of the latter. Inspired by Naowarat et al. (2026), who characterize interference through the prediction error of one concept after erasing another, we measure directional interference through the relative decrease in the concept score of CjC_j after removing CiC_i. Let ℰi(Z)E_i(Z) denote the representation obtained by erasing concept CiC_i from Z, and let mj(Z)m_j(Z) denote the score of CjC_j evaluated on Z. We define the directional interference as: ri→j=1−mj(ℰi(Z))mj(Z).r_i→ j=1- m_j (E_i(Z) )m_j(Z). (3) Intuitively, rijr_ij is high when removing CiC_i substantially reduces the recoverability score of CjC_j, and low when CjC_j remains largely unaffected by this intervention. We instantiate ℰiE_i with a differentiable implementation of LEACE (Belrose et al., 2023). Consequently, Equation 3 characterizes interactions through the effect of an explicit intervention, rather than through their correlation alone. For MSE concept scores, interference admits an equivalent closed-form expression, described in Appendix B. From concept scores and interactions to topology. To jointly represent concept scores and interactions, we first express both quantities on a common scalar scale. While mim_i directly quantifies concept CiC_i, the interaction score ρij∈[0,1] _ij∈[0,1] is normalized and therefore only specifies the relative interaction between CiC_i and CjC_j. We seek an absolute interaction level that increases with ρij _ij, vanishes when ρij=0 _ij=0, and cannot exceed the score of either concept: an interaction between two concepts should not be assigned a larger value than the extent to which either of them is itself represented. Accordingly, we rescale ρij _ij by the largest admissible value min(mi,mj) (m_i,m_j) and define: sij=min(mi,mj)ρij.s_ij= (m_i,m_j) _ij. (4) Thus, sijs_ij varies continuously from 00 for no interaction to min(mi,mj) (m_i,m_j) for maximal interaction. Intuitively, the difference mi−sijm_i-s_ij represents how much CiC_i is independent from CjC_j. We use these quantities to define a scalar field on a graph ℳM that we refer to as the concept mesh. The graph contains one concept vertex viv_i of value mim_i for each CiC_i, together with a connector vertex vijv_ij of value sijs_ij connected to viv_i and vjv_j. Since sij≤min(mi,mj)s_ij≤ (m_i,m_j), concept vertices form maxima of this scalar field, while connector vertices encode their pairwise interactions below the corresponding concept values. We additionally introduce a common vertex with value below all other vertices to assign a finite reference level to the component that survives all pairwise merges. We apply the upper-star counterpart of the filtration introduced in section 2, processing vertices from highest to lowest values. Concept maxima create connected components and connector vertices merge them; following the Elder rule, we associate CiC_i with pi=(si,mi)p_i=(s_i,m_i), where sis_i is the connector value at which its component merges with an older one. Finally, TCR(Z)=pii=1nCTCR(Z)=\p_i\_i=1^n_C is the persistence diagram defining the Topological Concept Representation of Z. This representation enables Wasserstein comparisons and differentiable topological optimization, which we exploit next. 3.3 Editing Topological Concept Representations Having constructed a topological description of the concept organization, we now turn it into an operational interface to specify interventions on a representation H. Let Zθ=Tθ(H)Z_θ=T_θ(H) denote the representation obtained by applying a differentiable transformation Tθ:ℝd→ℝd′T_θ:R^d ^d row-wise to H. Starting from TCR(H)TCR(H), an intervention is specified by defining a target TCR⋆TCR through changes to its concept maxima and interaction structure. Given a pair (sℬ⋆,mi⋆)(s _AB,m_i ) in TCR⋆TCR , examples include: • mi⋆↓m_i / mi⋆↑m_i : weaken / strengthen concept CiC_i; while mi⋆=mim_i =m_i preserves it; • sℬ⋆↓s_AB / sℬ⋆↑s_AB : disentangle / entangle the sets of concepts A and ℬB that merged at sℬ⋆s _AB; • TCR=⋆TCR(Z⋆)TCR =TCR(Z ): transfer the entire concept organization from a representation Z⋆Z . These operations can be combined within the same target, e.g., suppressing an undesired concept while preserving the task and selected concepts. Importantly, the target specifies an editing criterion, rather than an editing algorithm. Concept erasure, for instance, can be expressed by setting mi⋆=0m_i =0 while preserving protected concepts. More generally, varying the concept score S and target constraints allows TCR to capture a broad class of existing erasure criteria at the objective level, independently of the optimization used to enforce them; we further examine this relation in subsection 3.4. Topological objective. Given TCR(Zθ)TCR(Z_θ) and TCR⋆TCR , we measure how closely the transformed representation ZθZ_θ realizes the desired concept organization using the LqL^q-Wasserstein distance (section 2): ℒTCR(θ)=Wq(TCR(Zθ),TCR)⋆q.L_TCR(θ)=W_q (TCR(Z_θ),TCR )^q. (5) When concept correspondences are known (possibly only on a subset) this semantic information can be exploited by constraining the Wasserstein assignment to match the corresponding features. Topological optimization. To realize the target, we optimize θ through persistence-based topological optimization (section 2). For a given ZθZ_θ, we refit the concept probes, reconstruct the concept mesh and its TCR. The gradient of Eq. 5 is propagated through the persistence map P to the filtration values of the concept mesh, and through the filtration map ℱF to the concept maxima and interaction saddles involved in the matched persistence features. Since these values are differentiable functions of ZθZ_θ, through the differentiable concept scores and interactions, the gradient further propagates through Zθ=Tθ(H)Z_θ=T_θ(H) to update θ. The framework is agnostic to the parameterization of TθT_θ, we consider post-hoc linear or nonlinear transformations, or directly the network producing H. Concept disentanglement. Given two sets of concepts A and ℬB, we define their structural disentanglement by lowering the persistence saddles at which components containing concepts from respectively each set merge. Several such merges may occur at different locations of the hierarchy, involving different subsets of A and ℬB; their saddle coordinates are jointly targeted while selected concept maxima can be preserved. As the TCR is recomputed during optimization, the selected saddles adapt to the current concept organization. This operation therefore edits the hierarchical connectivity between the two concept sets, rather than a prescribed individual pairwise interaction. Concept transfer. Given two representations ZAZ^A and ZBZ^B, possibly produced by different models or having different dimensionalities, TCR(ZA)TCR(Z^A) can serve as the target for ZBZ^B. Thus, concept transfer replaces a manually specified target by the concept organization from another representation. 3.4 Properties of Topological Concept Representations (1) Benefits of the persistence reduction. Rather than reproducing every pairwise interaction of the concept mesh, TCR summarizes how such local interactions combine globally to form groups of concepts and how they organize across interaction levels, beyond what individual interaction scores reveal in isolation. Among the (nC2) n_C2 interactions, the ones being responsible for the concept hierarchy correspond exactly to the nC−1n_C-1 saddles computed by persistence. TCR therefore provides a compact multiscale description of the concept organization by identifying the minimum sparse set of interactions through which this organization can be modified. Consequently, it is invariant to changes not affecting the concept hierarchy in contrast to concept meshes that can differ substantially entry-wise while still inducing the same concept hierarchy. Representing this organization as a persistence diagram provides established tools for Wasserstein comparison and differentiable optimization, together with stability and convergence properties, whereas operating directly on concept meshes instead requires additional choices of correspondence, comparison and optimization criteria. (2) Relation to existing erasure objectives. With appropriate choices of the concept scores, several existing erasure criteria, such as those of the baselines considered in our experiments, arise as special cases of TCR at the objective or constraint level. Details and proofs are given in Appendix E. (3) Affine reparameterization invariance. For the MSE–overlap instantiation, TCR depends on the intrinsic concept organization represented rather than on a particular affine parameterization of the feature space. Hence, TCR is invariant to invertible affine feature changes, see Appendix F. Proposition 1 (Affine reparameterization invariance). Let Z∈ℝN×dZ ^N× d be a representation of N samples and let Z~=ZB⊤+a⊤ Z=ZB +1a with B∈ℝd×dB ^d× d invertible and a∈ℝda ^d. Assume that TCR is constructed using the MSE concept scores of Equation 7 (Appendix B) and the predictive-overlap interactions of Equation 9. Then TCR(Z~)=TCR(Z)TCR( Z)=TCR(Z). The same result holds for normalized MSE concept scores. (4) Stability. Finally, small perturbations of the representation induce proportionally small changes in its TCR under mild local regularity conditions. Proofs and discussions are in Appendix G. Theorem 1 (Representation-to-TCR stability). Given H∈ℝN×dH ^N× d. Assume that Σ(H)≻0 (H) 0 and that every concept represented in the TCR has strictly positive MSE recoverability, mi(H)>0m_i(H)>0 for every i=1,…,nCi=1,…,n_C. Then there exist ε>0 >0 and C>0C>0 such that, for every centered representation H′H satisfying ‖H−H′‖F<ε\|H-H \|_F< , dB(TCR(H),TCR(H′))≤C‖H−H′‖Fd_B (TCR(H),TCR(H ) )≤ C\,\|H-H \|_F with dB=W∞d_B=W_∞. 4 Experiments & Results We evaluate TCR on two applications: concept disentanglement before erasure, studying whether modifying concept interactions can facilitate the removal of undesired information while preserving task utility, and concept transfer, investigating whether the concept organization of a teacher model can be transferred to a student. We organize our experiments around the following questions: (Q1) How does TCR disentanglement preprocessing improves existing erasure methods at similar linear concept leakage and can TCR directly realize effective concept erasure? (Q2) How do different experimental settings and data regimes, such as the level of erasure, spurious correlation and label imbalance, affect the task utility preservation TCR? (Q3) Can TCR transfer concept information across models while maintaining task accuracy compared with standard knowledge-distillation methods? (Q4) How does TCR transfer generalize beyond the concepts explicitly optimized and how is this transfer affected by concepts discovered at increasingly local and fine spatial scales? 4.1 Concept Disentanglement Before Erasure Experimental setup. We consider as baselines INLP (Ravfogel et al., 2020), LEACE (Belrose et al., 2023), RLACE (Ravfogel et al., 2022), SPLINCE (Holstege et al., 2025), LEOPARD (Saillenfest & Lemberger, 2025), KRaM (Chowdhury et al., 2023) and Obliviator (Akbari et al., 2025). We evaluate on Waterbirds (Sagawa et al., 2020), Bias in Bios (De-Arteaga et al., 2019), DIAL-Mention (Blodgett et al., 2016; Elazar & Goldberg, 2018) and Jigsaw (Borkan et al., 2019). We use frozen ImageNet-pretrained ResNet-50 (He et al., 2016) average-pooling features for Waterbirds and BERT-base (Devlin et al., 2019) final-layer [CLS] representations for the three text datasets. Each dataset defines a classification task together with an unwanted concept to erase. All methods operate post-hoc on these fixed pretrained representations reduced to 128 dimensions using PCA fitted on the training set before intervention. More details are provided in Appendix H.1. Table 1: Test task worst-group accuracy with an adjusted balanced linear concept leakage <10%<10\%. Method Waterbirds Bias in Bios SN Dial Mention Jigsaw Raw (WGA / Leakage) 56.54 / 80.22 78.77 / 98.48 39.20 / 54.57 53.29 / 41.57 INLP 27.8827.88±0.00 59.4759.47±0.00 52.0552.05±0.00 55.6455.64±0.00 LEACE 13.7113.71±0.00 59.4759.47±0.00 50.4550.45±0.00 52.5952.59±0.00 RLACE 14.5514.55±0.95 59.5359.53±0.92 50.8050.80±0.29 53.2753.27±0.37 SPLINCE 13.7113.71±0.00 58.9858.98±0.00 50.4550.45±0.00 52.5952.59±0.00 LEOPARD 13.7113.71±0.00 59.4759.47±0.00 49.9549.95±0.19 52.5952.59±0.00 KRaM 9.089.08±4.01 44.0244.02±3.17 52.2552.25±1.40 53.1253.12±1.41 Obliviator 18.6818.68±3.46 58.1858.18±1.29 50.5650.56±1.48 52.5652.56±1.07 TCR-DIS + INLP 66.0166.01±8.80 81.1281.12±3.85 54.6654.66±1.15 56.7156.71±1.12 TCR-DIS + LEACE 74.4574.45±1.31 81.6181.61±1.26 58.0458.04±0.7458.04 ±0.74 60.5860.58±0.92 TCR-DIS + RLACE 74.66 ±0.96 81.5981.59±1.36 57.7857.78±0.73 61.05 ±0.72 TCR-DIS + SPLINCE 74.1774.17±1.24 81.5381.53±0.97 58.0358.03±0.70 60.5660.56±0.99 TCR-DIS + LEOPARD 74.4774.47±0.8974.47 ±0.89 81.4781.47±1.43 57.8357.83±0.83 60.6160.61±0.8760.61 ±0.87 TCR-DIS + KRaM 55.6455.64±11.32 57.4857.48±5.85 54.2354.23±1.56 56.9256.92±1.27 TCR-DIS + Obliviator 69.3069.30±7.99 83.5083.50±1.1783.50 ±1.17 58.20 ±1.73 60.0160.01±1.21 TCR-ERA 71.1771.17±1.42 86.52 ±1.01 55.7055.70±0.89 57.3757.37±1.37 (Q1) To compare task preservation after concept removal, we report the test worst-group accuracy (WGA), averaged over 10 seeds, of methods with an adjusted balanced linear concept leakage below 10%10\% in Table 1 (metrics described formally in Appendix H.1). Disentangling the task and concept with TCR before erasure consistently improves task utility: TCR-DIS increases WGA for every erasure method on all four datasets. The improvement is particularly pronounced on Waterbirds and Bias in Bios, where the best baseline WGA increases from 27.8827.88 to 74.6674.66 and from 59.5359.53 to 83.5083.50, respectively. This also shows that the interference saddle generalizes beyond the erasure operator used to define it, although constructed from LEACE, it facilitates erasure with all considered methods. Moreover, directly optimizing TCR for concept erasure (TCR-ERA) outperforms every standalone erasure baseline across the four datasets and achieves the highest WGA on Bias in Bios. Across all datasets, TCR-DIS improves the WGA of each baseline by 21.89 on average, with large gains on Waterbirds (+53.91) and Bias in Bios (+21.31), while TCR-ERA improves WGA by 23.43. Note that, following prior work on linear-guardedness, e.g., LEACE (Belrose et al., 2023), concept leakage is measured through a linear probe. We therefore additionally evaluate stronger leakage probes, including oracle-conditioned linear and non-linear probes, in Appendix I.1.1. Additional analysis of global representational similarity before and after erasure is provided in Appendix I.1.3. Takeaway. TCR disentanglement can effectively serve as preprocessing for a diverse set of erasure methods and systematically facilitates utility-preserving concept erasure. Moreover, TCR can also perform effective task-preserving concept erasure directly within the same framework. (a) (b) (c) Figure 2: Erasure-utility trade-off showing the test WGA given the adjusted balanced linear leakage (a) and impact on test WGA of varying label imbalance (b) and spurious correlation (c). (Q2) To assess utility preservation under different erasure strengths and data regimes, we vary the erasure constraint, label imbalance and spurious correlation. We first examine the erasure-utility trade-off across different erasure strengths. On Waterbirds (Figure 2a), TCR retains 71.1771.17 WGA at 4.26%4.26\% adjusted balanced linear leakage, while INLP reaches 27.8827.88 WGA at 6.88%6.88\%. As the erasure constraint is relaxed, several baselines recover substantial task utility and the gap to TCR narrows, whereas TCR retains high WGA already under strong erasure. We then vary properties of the data distribution while maintaining strong concept erasure. The effect of increasing label imbalance is illustrated on Dial Mention (Figure 2b), where the margin of TCR over the best baseline at each setting grows from 3.463.46 WGA points at P(y=0)=0.50P(y=0)=0.50 to 26.2726.27 at P(y=0)=0.75P(y=0)=0.75. For increasing spurious correlation, on Jigsaw (Figure 2c), TCR is comparable to the best baseline at P(y=c)=0.5P(y=c)=0.5 and achieves the highest WGA at every stronger correlation, with a gain of up to 7.257.25 points at P(y=c)=0.8P(y=c)=0.8. Further erasure-utility and data-regime analyses are reported in Appendix I.1.2. Takeaway. TCR maintains a favorable erasure-utility trade-off across erasure strengths and consistently strong utility preservation relative to the baselines across data regimes, with its advantage particularly pronounced under strong erasure, label imbalance and increased spurious correlation. 4.2 Concept Transfer Experimental setup. Following the distillation experimental protocol of Tian et al. (2020), we evaluate concept transfer on CIFAR-100 (Krizhevsky & Hinton, 2009), using a ResNet32×4 teacher and ResNet8×4 and ShuffleNetV2 students. We consider as baselines a student trained without distillation (Student), KD (Hinton et al., 2015), FitNet (Romero et al., 2015), SPKD (Tung & Mori, 2019) and TopKD (Kim et al., 2024). For TCR, concepts are discovered from the teacher penultimate representation, from the full images or from 2×2, 3×3 or 4×4 crops (Fel et al., 2023b), and their organization is transferred to the student during distillation. Further details are given in Appendix H.2. (Q3) To evaluate how TCR transfers concept information while preserving task performance, we report in Table 2 test top-1 accuracy and concept recoverability (CR k20k_20), being the mean held-out R2R^2 of linear probes predicting 20 teacher-derived concept coefficients from the student representation, that can be seen as their preservation (detailed in Appendix H.2). For ResNet8×4, both TCR configurations outperform all baselines on both metrics: optimizing 10 concepts (k10k_10) achieves the highest Table 2: Distillation test top-1 accuracy and concept recoverability. Method ResNet8×4 ShuffleNetV2 Top-1 CR k20k_20 Top-1 CR k20k_20 Student 72.3572.35±0.24 41.5041.50±0.18 72.5872.58±0.23 52.6552.65±0.23 KD 73.3573.35±0.24 54.0754.07±0.17 74.6274.62±0.19 66.6466.64±0.19 FitNet 73.1973.19±0.23 41.0541.05±0.25 74.2574.25±0.15 53.5253.52±0.45 SPKD 73.0073.00±0.24 54.5154.51±0.19 76.04 ±0.15 67.62 ±0.23 TopKD 73.2073.20±0.28 54.1854.18±0.25 74.9874.98±0.28 66.5866.58±0.14 TCR k10k_10 73.68 ±0.47 57.1857.18±0.0657.18 ±0.06 74.9774.97±0.70 66.8366.83±0.3766.83 ±0.37 TCR k20k_20 73.5573.55±0.2873.55 ±0.28 60.05 ±0.35 75.0275.02±0.5475.02 ±0.54 66.6766.67±0.28 top-1 accuracy (73.6873.68) while k20k_20 achieves the highest CR (60.0560.05). Compared with the strongest baseline for each metric, this corresponds to gains of 0.330.33 for top-1 over KD and 5.545.54 for CR over SPKD, respectively. Averaged over all baselines, gains are respectively 0.600.60 and 9.559.55 points. For ShuffleNetV2, SPKD achieves the strongest results on both metrics while TCR remains close in CR. Takeaway. TCR can transfer teacher-derived concept information to a student while preserving task performance, simultaneously improving concept recoverability and task accuracy (or maintaining it). (Q4) To assess whether TCR transfers concept information beyond the concept bank used during optimization, we train TCR with ktrain∈2,5,10,20k_train∈\2,5,10,20\ concepts and evaluate CR using all keval∈2,5,10,20k_eval∈\2,5,10,20\ concept banks in subsection 4.2. We report the CR gain over SPKD, the best baseline regarding CR, with 00 indicating equal recoverability. Since concept banks for different values of k are independently discovered, settings with ktrain≠kevalk_train≠ k_eval measure transfer to concepts that were not explicitly optimized. For full-image concepts (subsection 4.2a), TCR improves over SPKD for every matched ktrain=kevalk_train=k_eval setting, but not when evaluated on different bank sizes. This behavior changes as concepts become increasingly local: the mean gain when ktrain≠kevalk_train≠ k_eval becomes +0.48+0.48, +1.16+1.16 and +1.39+1.39 for the 2×2, 3×3 and 4×4 crop grids, respectively. Likewise, the mean gain across all training and evaluation banks increases from −0.35-0.35 for full-image concepts to +1.05+1.05, +1.78+1.78 and +1.88+1.88 at finer spatial scales. At the two finest scales, larger training banks yield particularly broad transfer; for example, training with k20k_20 improves CR on the independently discovered k10k_10 bank by 3.483.48 for the 3×3 grid and 4.944.94 for the 4×4 grid. These results indicate a transition from primarily bank-specific transfer for global concepts to broader cross-bank transfer as concepts become more spatially localized. One possible interpretation is that global concepts may induce more bank-specific structure, whereas local concepts expose more reusable structure that are shared across independently discovered banks. Part of this effect may also arise because localized concepts are less recoverable for the baselines, leaving more room for improvement. Takeaway. For global concepts, TCR primarily improves recoverability of the concept bank used during optimization. As concepts become increasingly local, its advantage over SPKD grows and extends to independently discovered concept banks of different sizes. (a) (b) (c) (d) Figure 3: Impact of concept spatial scale on recoverability. 5 Conclusion We introduced TCR, a topological abstraction that jointly represents concepts and their relationships and provides a common interface for concept-level interventions. Across concept erasure, disentanglement and transfer, our results demonstrate its ability to manipulate concept organization while preserving task-relevant information. For disentanglement, it improves WGA by 21.8921.89 on average at comparable concept leakage and, for transfer, it improves concept recoverability by up to 5.545.54. AI use statement In this work, we used generative AI tools for assisting in the writing of proofs, designing or providing feedback on research methodology or experiments, implementing methods, supporting qualitative and thematic data analysis and interpreting results. We have not used generative AI tools for generating synthetic data sets, helping developing theoretical models or conceptual frameworks, formulating mathematical claims, providing critical ingredients for proving mathematical claims, proposing or refining hypotheses, assisting with translation and cleaning and reformatting dataset. Additionally, we used generative AI tools for suggesting experimental parameters, creating or editing software code, summarize or analyse existing literature, brainstorming, sourcing/searching for information, editing a research paper to improve readability and identifying relevant literature. We have reviewed all AI-assisted work. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. Ethics Statement Our experiments use existing benchmark datasets, including human-generated text with demographic, dialectal and identity-related annotations or proxies. We do not collect new human-subject data. Such annotations and proxies may be imperfect and should not be interpreted as definitive characterizations of individuals or groups. While concept erasure can help reduce unwanted information in learned representations, removing a selected concept does not by itself guarantee fairness or complete removal of the corresponding information; we therefore evaluate leakage using complementary diagnostics. More generally, representation interventions can be applied to arbitrary concepts, and their use in high-stakes applications requires careful validation of both the information removed and the utility preserved. Reproducibility Statement We provide the code used to produce the results of this paper as supplementary material with the anonymous submission. Detailed experimental protocols, including datasets and splits, model architectures, hyperparameter selection and evaluation metrics, are provided in the appendix. Upon de-anonymization, we will make the code publicly available on GitHub. References Akbari et al. (2025) Ramin Akbari, Milad Afshari, and Vishnu Boddeti. Obliviator reveals the cost of nonlinear guardedness in concept erasure. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference, p. 144860–144892. Curran Associates, Inc., 2025. doi: 10.52202/085713-4847. URL https://proceedings.neurips.c/paper_files/paper/2025/file/d55f39791f04745b2e0c8abebf3d5d7-Paper-Conference.pdf. Ansuini et al. (2019) Alessio Ansuini, Alessandro Laio, Jakob H. Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. Curran Associates Inc., Red Hook, NY, USA, 2019. Bau et al. (2017) David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), p. 3319–3327, 2017. doi: 10.1109/CVPR.2017.354. Belrose et al. (2023) Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, p. 66044–66063. Curran Associates, Inc., 2023. doi: 10.52202/075280-2884. URL https://proceedings.neurips.c/paper_files/paper/2023/file/d066d21c619d0a78c5b557fa3291a8f4-Paper-Conference.pdf. Bertsekas (1981) Dimitri P. Bertsekas. A new algorithm for the assignment problem. Mathematical Programming, 21(1):152–171, 1981. doi: 10.1007/BF01584237. Blodgett et al. (2016) Su Lin Blodgett, Lisa Green, and Brendan T. O’Connor. Demographic dialectal variation in social media: A case study of african-american english. In Jian Su, Xavier Carreras, and Kevin Duh (eds.), EMNLP, p. 1119–1130. The Association for Computational Linguistics, 2016. ISBN 978-1-945626-25-8. URL http://dblp.uni-trier.de/db/conf/emnlp/emnlp2016.html#BlodgettGO16. Borkan et al. (2019) Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Sihem Amer-Yahia, Mohammad Mahdian, Ashish Goel, Geert-Jan Houben, Kristina Lerman, Julian J. McAuley, Ricardo Baeza-Yates, and Leila Zia (eds.), W (Companion Volume), p. 491–500. ACM, 2019. ISBN 978-1-4503-6675-5. URL http://dblp.uni-trier.de/db/conf/w/w2019c.html#BorkanDSTV19. Bubenik & Elchesen (2022) Peter Bubenik and Alex Elchesen. Universality of persistence diagrams and the bottleneck and wasserstein distances. Comput. Geom. Theory Appl., 105(C), August 2022. ISSN 0925-7721. doi: 10.1016/j.comgeo.2022.101882. URL https://doi.org/10.1016/j.comgeo.2022.101882. Carriere et al. (2021) Mathieu Carriere, Frederic Chazal, Marc Glisse, Yuichi Ike, Hariprasad Kannan, and Yuhei Umeda. Optimizing persistent homology based functions. In Proceedings of the 38th International Conference on Machine Learning, volume 139, p. 1294–1303. PMLR, 18–24 Jul 2021. URL https://proceedings.mlr.press/v139/carriere21a.html. Carrière et al. (2026) Mathieu Carrière, Yuichi Ike, Théo Lacombe, and Naoki Nishikawa. Persistence-based topological optimization: a survey, 03 2026. Che et al. (2024) Mauricio Che, Fernando Galaz-García, Luis Guijarro, and Ingrid Amaranta Membrillo Solis. Metric geometry of spaces of persistence diagrams. Journal of Applied and Computational Topology, 8(8):2197–2246, Dec 2024. ISSN 2367-1734. doi: 10.1007/s41468-024-00189-2. URL https://doi.org/10.1007/s41468-024-00189-2. Chen et al. (2019) Chao Chen, Xiuyan Ni, Qinxun Bai, and Yusu Wang. A topological regularizer for classifiers via persistent homology. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, volume 89, p. 2573–2582. PMLR, 16–18 Apr 2019. URL https://proceedings.mlr.press/v89/chen19g.html. Chowdhury et al. (2023) Somnath Basu Roy Chowdhury, Nicholas Monath, Avinava Dubey, Amr Ahmed, and Snigdha Chaturvedi. Robust concept erasure via kernelized rate-distortion maximization. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA, 2023. Curran Associates Inc. Clémot et al. (2026) Mattéo Clémot, Julie Digne, and Julien Tierny. Topological autoencoders++: Fast and accurate cycle-aware dimensionality reduction. IEEE Transactions on Visualization and Computer Graphics, 32(2):1622–1639, 2026. doi: 10.1109/TVCG.2025.3644671. Cuturi (2013) Marco Cuturi. Sinkhorn distances: lightspeed computation of optimal transport. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, p. 2292–2300, Red Hook, NY, USA, 2013. Curran Associates Inc. De-Arteaga et al. (2019) Maria De-Arteaga, Alexey Romanov, Hanna M. Wallach, Jennifer T. Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Cem Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias in a high-stakes setting. In danah boyd and Jamie H. Morgenstern (eds.), FAT, p. 120–128. ACM, 2019. URL http://dblp.uni-trier.de/db/conf/fat/fat2019.html#De-ArteagaRWCBC19. Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), p. 4171–4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1423. URL https://aclanthology.org/N19-1423/. Edelsbrunner & Harer (2009) H. Edelsbrunner and J. Harer. Computational Topology: An Introduction. American Mathematical Society, 2009. Edelsbrunner & Mucke (1990) Herbert Edelsbrunner and Ernst P Mucke. Simulation of simplicity: a technique to cope with degenerate cases in geometric algorithms. ACM Transactions on Graphics, 9(1):66–104, 1990. doi: 10.1145/77635.77639. Edelsbrunner et al. (2002) Herbert Edelsbrunner, David Letscher, and Afra Zomorodian. Topological Persistence and Simplification. 28(4):511–533, 2002. doi: 10.1007/S00454-002-2885-2. Elazar & Goldberg (2018) Yanai Elazar and Yoav Goldberg. Adversarial removal of demographic attributes from text data. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun’ichi Tsujii (eds.), EMNLP, p. 11–21. Association for Computational Linguistics, 2018. ISBN 978-1-948087-84-1. URL http://dblp.uni-trier.de/db/conf/emnlp/emnlp2018.html#ElazarG18. Fel et al. (2023a) Thomas Fel, Victor Boutin, Mazda Moayeri, Rémi Cadène, Louis Bethune, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA, 2023a. Curran Associates Inc. Fel et al. (2023b) Thomas Fel, Agustin Picard, Louis Béthune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 2711–2721, June 2023b. Ghorbani et al. (2019) Amirata Ghorbani, James Wexler, James Zou, and Been Kim. Towards automatic concept-based explanations. Curran Associates Inc., Red Hook, NY, USA, 2019. Gupta et al. (2023) Avani Gupta, Saurabh Saini, and P J Narayanan. Concept distillation: Leveraging human-centered explanations for model improvement. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, p. 63724–63737. Curran Associates, Inc., 2023. doi: 10.52202/075280-2783. URL https://proceedings.neurips.c/paper_files/paper/2023/file/c9450295fd667740a39a68148fc17f6e-Paper-Conference.pdf. He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’16, p. 770–778. IEEE, June 2016. doi: 10.1109/CVPR.2016.90. URL http://ieeexplore.ieee.org/document/7780459. Hensel et al. (2021) Felix Hensel, Michael Moor, and Bastian Rieck. A survey of topological machine learning methods. Frontiers in Artificial Intelligence, Volume 4 - 2021, 2021. ISSN 2624-8212. doi: 10.3389/frai.2021.681108. URL https://w.frontiersin.org/journals/artificial-intelligence/articles/10.3389/frai.2021.681108. Hinton et al. (2015) Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. Distilling the knowledge in a neural network. CoRR, abs/1503.02531, 2015. URL http://dblp.uni-trier.de/db/journals/corr/corr1503.html#HintonVD15. Hofer et al. (2019) Christoph Hofer, Roland Kwitt, Marc Niethammer, and Mandar Dixit. Connectivity-optimized representation learning via persistent homology. In Proceedings of the 36th International Conference on Machine Learning, volume 97, p. 2751–2760. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/hofer19a.html. Hofer et al. (2020) Christoph Hofer, Florian Graf, Marc Niethammer, and Roland Kwitt. Topologically densified distributions. In Proceedings of the 37th International Conference on Machine Learning, volume 119, p. 4304–4313. PMLR, 13–18 Jul 2020. URL https://proceedings.mlr.press/v119/hofer20a.html. Holstege et al. (2024) Floris Holstege, Bram Wouters, Noud Van Giersbergen, and Cees Diks. Removing spurious concepts from neural network representations via joint subspace estimation. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, p. 18568–18610. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/holstege24a.html. Holstege et al. (2025) Floris Holstege, Shauli Ravfogel, and Bram Wouters. Preserving task-relevant information under linear concept removal. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=t79rMQGk4S. Huben et al. (2024) Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=F76bwRSLeK. Kaplan et al. (2020) Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. CoRR, abs/2001.08361, 2020. URL https://arxiv.org/pdf/2001.08361.pdf. Kerber et al. (2017) Michael Kerber, Dmitriy Morozov, and Arnur Nigmetov. Geometry helps to compare persistence diagrams. ACM J. of Experimental Algorithmics, 22:1–20, 2017. doi: 10.1145/3064175. Kim et al. (2018) Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory sayres. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, p. 2668–2677. PMLR, 10–15 Jul 2018. URL https://proceedings.mlr.press/v80/kim18d.html. Kim et al. (2024) Jungeun Kim, Junwon You, Dongjin Lee, Ha Young Kim, and Jae-Hun Jung. Do topological characteristics help in knowledge distillation? In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, p. 24674–24693. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/kim24aj.html. Kissi et al. (2025) Mohamed Kissi, Mathieu Pont, Joshua A. Levine, and Julien Tierny. A practical solver for scalar data topological simplification. IEEE Transactions on Visualization and Computer Graphics, 31(1):97–107, 2025. doi: 10.1109/TVCG.2024.3456345. Kornblith et al. (2019) Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, p. 3519–3529. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/kornblith19a.html. Krizhevsky & Hinton (2009) Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical Report 0, University of Toronto, Toronto, Ontario, 2009. URL https://w.cs.toronto.edu/~kriz/learning-features-2009-TR.pdf. Moor et al. (2020) Michael Moor, Max Horn, Bastian Rieck, and Karsten M. Borgwardt. Topological autoencoders. In ICML, 2020. Munkres (1957) James Munkres. Algorithms for the assignment and transportation problems. J. of SIAM, 5(1):32–38, 1957. doi: https://doi.org/10.1137/0105003. Naowarat et al. (2026) Burin Naowarat, Hao Tang, and Sharon Goldwater. A framework for analyzing concept representations in neural models. In Claire Bonial and Yevgeni Berzak (eds.), Proceedings of the 30th Conference on Computational Natural Language Learning, p. 574–587, San Diego, California, USA, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-410-1. doi: 10.18653/v1/2026.conll-main.34. URL https://aclanthology.org/2026.conll-main.34/. Nigmetov & Morozov (2024) Arnur Nigmetov and Dmitriy Morozov. Topological optimization with big steps. Discrete & Computational Geometry, 1 2024. ISSN 0179-5376. doi: 10.1007/s00454-023-00613-x. URL https://w.osti.gov/biblio/2337698. Pont & Garth (2026) Mathieu Pont and Christoph Garth. Region-aware wasserstein distances of persistence diagrams and merge trees. IEEE Transactions on Visualization and Computer Graphics, 32(9):7756–7773, 2026. doi: 10.1109/TVCG.2026.3703539. Ravfogel et al. (2020) Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, p. 7237–7256, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.647. URL https://aclanthology.org/2020.acl-main.647/. Ravfogel et al. (2022) Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan D Cotterell. Linear adversarial concept erasure. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, p. 18400–18421. PMLR, 17–23 Jul 2022. URL https://proceedings.mlr.press/v162/ravfogel22a.html. Ravfogel et al. (2023) Shauli Ravfogel, Yoav Goldberg, and Ryan Cotterell. Log-linear guardedness and its implications. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9413–9431, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.acl-long.523. URL https://aclanthology.org/2023.acl-long.523/. Romero et al. (2015) Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. Fitnets: Hints for thin deep nets. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6550. Sagawa et al. (2020) Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations (ICLR), 2020. Saillenfest & Lemberger (2025) Antoine Saillenfest and Pirmin Lemberger. Nonlinear concept erasure: A density matching approach. In Inês Lynce, Nello Murano, Mauro Vallati, Serena Villata, Federico Chesani, Michela Milano, Andrea Omicini, and Mehdi Dastani (eds.), ECAI 2025 - 28th European Conference on Artificial Intelligence, 25-30 October 2025, Bologna, Italy - Including 14th Conference on Prestigious Applications of Intelligent Systems (PAIS 2025), volume 413 of Frontiers in Artificial Intelligence and Applications, p. 2138–2145. IOS Press, 2025. doi: 10.3233/FAIA251053. URL https://doi.org/10.3233/FAIA251053. Soler et al. (2018) Maxime Soler, Melanie Plainchault, Bruno Conche, and Julien Tierny. Lifted Wasserstein Matcher for Fast and Robust Topology Tracking . In 2018 IEEE 8th Symposium on Large Data Analysis and Visualization (LDAV), p. 23–33, October 2018. doi: 10.1109/LDAV.2018.8739196. URL https://doi.ieeecomputersociety.org/10.1109/LDAV.2018.8739196. Tian et al. (2020) Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive representation distillation. In ICLR. OpenReview.net, 2020. URL http://dblp.uni-trier.de/db/conf/iclr/iclr2020.html#TianKI20. Tung & Mori (2019) Fred Tung and Greg Mori. Similarity-Preserving Knowledge Distillation . In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), p. 1365–1374, Los Alamitos, CA, USA, November 2019. IEEE Computer Society. doi: 10.1109/ICCV.2019.00145. URL https://doi.ieeecomputersociety.org/10.1109/ICCV.2019.00145. Vielhaben et al. (2023) Johanna Vielhaben, Stefan Bluecher, and Nils Strodthoff. Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=KxBQPz7HKh. Zomorodian (2010) Afra J. Zomorodian. Topology for computing. In Mikhail J. Atallah and Marina Blanton (eds.), Algorithms and Theory of Computation Handbook (Second Edition), chapter 3, p. 82–112. CRC Press, Boca Raton, 2010. doi: https://doi.org/10.1017/CBO9780511546945. Appendices These appendices provide additional details including extended related work, method details, mathematical proofs, detailed experimental setup and additional experiments and results to further illustrate and validate the effectiveness of TCR. The contents are organized as follows: • Appendix A: Extended Related Work • Additional Method Details – Appendix B: MSE Formulation – Appendix C: TCR Implementation Details and Computational Cost • Mathematical Proofs – Appendix Dand E: Existing Concept Erasure Criteria as Special Cases of TCR – Appendix F: Affine Reparameterization Invariance of TCR – Appendix G: Stability of TCR • Appendix H: Detailed Experimental Setup – Appendix H.1: Setup for Concept Disentanglement Before Erasure – Appendix H.2: Setup for Concept Transfer • Appendix I: Additional Experiments & Results – Appendix I.1: Concept Disentanglement Before Erasure * Appendix I.1.1: Beyond-linear Concept Leakage * Appendix I.1.2: Utility Preservation Across Erasure Levels and Data Regimes * Appendix I.1.3: Representation Similarity After Intervention • Appendix J: Limitations Appendix A Extended Related Work Concept discovery. Concept-based interpretability aims to characterize learned representations through high-level semantic properties that are more meaningful to humans than individual activation dimensions (Kim et al., 2018; Ghorbani et al., 2019). A concept is often represented as a direction in the activation space of a selected layer, for instance obtained from a linear classifier trained to distinguish examples associated with the concept (Kim et al., 2018) or from multidimensional concept subspaces (Vielhaben et al., 2023). Concepts can also be discovered in an unsupervised manner, through clustering (Ghorbani et al., 2019), matrix factorizations such as NMF (Fel et al., 2023b) or sparse autoencoders (Huben et al., 2024). These approaches provide complementary mechanisms for extracting semantically meaningful properties encoded by neural representations. Concept-based interventions. Beyond their use for interpretation, concepts can also serve as targets for post-hoc interventions on learned representations. A prominent line of work considers concept erasure, which aims to remove information associated with an undesired concept from a fixed representation through linear (Ravfogel et al., 2020; Ravfogel et al., 2022; Belrose et al., 2023) or non-linear formulations (Chowdhury et al., 2023; Saillenfest & Lemberger, 2025). Other methods additionally seek to preserve information relevant to a downstream task (Holstege et al., 2024; Holstege et al., 2025; Akbari et al., 2025). While effective for their respective objectives, these approaches are generally tailored to specific interventions, with new edits often requiring new objectives or optimization procedures. More fundamentally, they do not provide a common representation in which concept organization can be described and manipulated independently of the editing algorithm. Topological abstractions. Topological Data Analysis (TDA) provides concise abstractions of the global organization of the features present within complex data (Edelsbrunner & Harer, 2009; Hensel et al., 2021). In particular, such methods track how topological features appear and disappear across a filtration of the input, thereby characterizing structure over multiple scales (Edelsbrunner et al., 2002). These features can be summarized compactly through persistence diagrams, while richer abstractions such as merge trees additionally encode how connected components progressively merge. Their ability to jointly represent individual features and their global organization makes such abstractions a natural candidate for describing structured relationships between concepts. Persistence-based topological optimization. Beyond their descriptive role, topological abstractions can be incorporated into differentiable objectives to constrain learned representations. Such losses have been used, for instance, to regularize decision boundaries (Chen et al., 2019; Hofer et al., 2020) or preserve input topology during dimensionality reduction (Hofer et al., 2019; Moor et al., 2020; Clémot et al., 2026). Carriere et al. (2021) introduced a generic framework for optimizing persistence-based objectives, later extended to improve gradient density and scalability (Nigmetov & Morozov, 2024; Kissi et al., 2025; Carrière et al., 2026). TCR differs from these approaches in that it is not used to regularize the representation itself, but to encode its concept organization and to provide a common way to specify desired changes to the concepts of the underlying representation. Appendix B Analytical MSE Formulation This section details the closed-form MSE concept scores and predictive-overlap interactions used as an efficient analytical construction of TCR. Concept scores. Assume that the representation and concept scores are centered, and let ΣZ=Cov(Z) _Z=Cov(Z) and i=Cov(Z,i)g_i=Cov(Z,c_i). For a linear least-squares probe with coefficient vector i β_i, the population MSE is: [(ci−i⊤Z)2]=Var(i)−2i⊤i+i⊤ΣZi.E\! [ (c_i- β_i Z )^2 ]=Var(c_i)-2 β_i g_i+ β_i _Z β_i. (6) Its minimum-norm solution is i⋆=ΣZ+i β_i = _Z^+g_i, where (⋅)+(·)^+ denotes the Moore–Penrose pseudoinverse. Substitution into Equation 6 shows that the improvement over the optimal constant predictor is: mi=i⊤ΣZ+i=Aii,Aij:=i⊤ΣZ+j.m_i=g_i _Z^+g_i=A_i, A_ij:=g_i _Z^+g_j. (7) When the score is normalized by the constant-predictor loss, it is instead Aii/Var(i)A_i/Var(c_i), i.e., the explained-variance fraction. Thus, the diagonal of the matrix A contains all MSE concept scores. Predictive overlap. As an efficient analytical alternative, we additionally consider predictive overlap as concept interactions. The off-diagonal entries of the same matrix quantify how strongly two concepts rely on aligned predictive directions. We define their normalized predictive overlap as: ρijoverlap=Aij2AiiAjj. _ij^overlap= A_ij^2A_iA_j. (8) To interpret this quantity, define the whitened predictive direction i=(ΣZ1/2)+iv_i=( _Z^1/2)^+g_i. Then Aij=i⊤jA_ij=v_i v_j and Aii=‖i‖22A_i=\|v_i\|_2^2, so Equation 8 becomes: ρijoverlap=(i⊤j)2‖i‖22‖j‖22. _ij^overlap= (v_i v_j)^2\|v_i\|_2^2\|v_j\|_2^2. (9) It is therefore the squared cosine between the predictive directions of the two concepts and belongs to [0,1][0,1] whenever AiiA_i and AjjA_j are nonzero. A value of zero indicates orthogonal predictive directions, whereas a value of one indicates collinear directions, irrespective of their sign. Consequently, a single matrix A yields both the concept scores through its diagonal and every pairwise predictive overlap through its normalized off-diagonal entries. Appendix C Implementation Details and Computational Cost Differentiable construction. When optimizing a representation, we reconstruct TCR from the current transformed features at each optimization step. For BCE recoverability, the linear concept probes are therefore refitted differentiably, such that gradients propagate through the probe optimization. For interference interactions, removing a source concept is performed with differentiable LEACE and the probe of the target concept is subsequently refitted on the erased representation. Hence, interference measures the recoverability remaining after erasure, rather than the degradation of a fixed pre-erasure predictor. When differentiating a saddle sij=min(mi,mj)ρijs_ij= (m_i,m_j) _ij, we stop gradients through the min(mi,mj) (m_i,m_j) factor while retaining its value in the forward pass. This prevents interaction optimization from trivially decreasing a saddle by decreasing one of its incident concept scores instead of modifying their interaction. Analytical least-squares construction. The MSE formulation of Appendix B allows all concept scores to be computed jointly. We first accumulate the sufficient statistics required to compute the representation covariance and concept cross-covariances, namely sums of features, feature outer products and feature–concept products. Given nCn_C concepts, we collect the resulting cross-covariances in G=[1,…,nC]G=[g_1,…,g_n_C] and perform a single eigendecomposition of the d×d× d representation covariance ΣZ _Z. The same decomposition yields all least-squares predictive directions and concept scores simultaneously. When predictive overlap is used, all pairwise interactions are subsequently obtained from their nC×nCn_C× n_C predictive Gram matrix. The covariance decomposition is performed in double precision while numerically null eigendirections are discarded. Computational cost. Computing the sufficient statistics requires O(Nd2+NdnC)O(Nd^2+Ndn_C) operations, followed by one O(d3)O(d^3) covariance decomposition. Computing only the concept scores then scales linearly with nCn_C up to matrix products, whereas forming all predictive-overlap interactions additionally costs O(dnC2)O(dn_C^2). Thus, although a complete concept mesh contains O(nC2)O(n_C^2) pairwise interactions, the analytical construction does not require O(nC2)O(n_C^2) matrix decompositions. By contrast, interference explicitly evaluates both directional effects for each concept pair and therefore requires O(nC2)O(n_C^2) erased-recoverability evaluations. In our disentanglement experiments this construction involves only the task and concept nodes, while the larger concept sets used for transfer use the analytical least-squares construction. We further evaluate computation of TCR in Appendix I.1. Online construction for representation transfer. When TCR is optimized jointly with a network, constructing it from the complete training set after every minibatch would be unnecessarily expensive. We instead maintain a bounded feature bank and update the statistics required by the analytical formulation as new features are produced. The resulting TCR therefore uses a larger sample of the current representation than the instantaneous minibatch while keeping its memory footprint bounded. The precise bank size and TCR instantiation used for concept transfer are reported in the corresponding experimental setup. Choice of recoverability. BCE and MSE are different instantiations of concept recoverability rather than numerical approximations of one another. BCE is natural for the binary supervised concepts considered in our post-hoc erasure experiments and supports the intervention-based interference measure. MSE instead measures explained variance under linear least squares, naturally accommodates continuous concept coefficients and admits the shared analytical construction above. Likewise, interference and predictive overlap provide complementary interaction measures: the former measures the effect of explicitly erasing one concept on the recoverability of another, whereas the latter measures the alignment of their whitened predictive directions. We therefore use BCE with interference for concept disentanglement and normalized MSE for concept transfer. Appendix D Concept Signatures The persistence coordinates of a TCR feature describe the score of a concept and the interaction level at which it merges with an older component, but they do not encode additional concept-specific information. We therefore optionally associate each feature with a concept signature and extend the Wasserstein ground metric to jointly compare persistence coordinates and signatures. This formalism is distinct from the hard semantic matching used in the main method (subsection 3.3) and is introduced primarily to express the objective-level specializations studied in Appendix E. Related constructions have previously augmented persistence-diagram comparisons with information beyond birth and death. This has been done by incorporating spatial information associated with persistence pairs (Soler et al., 2018), properties of their associated regions (Pont & Garth, 2026) or, more generally, by allowing persistence-diagram points to live in general metric spaces (Bubenik & Elchesen, 2022; Che et al., 2024). Concept signatures follow a similar principle, but use auxiliary information describing the represented concepts rather than spatial or regional information. D.1 Signature-Augmented Ground Metric Recall that a TCR feature associated with concept CiC_i is represented by the persistence point pi=(si,mi)∈ℝ2p_i=(s_i,m_i) ^2. We additionally associate it with a signature i∈ σ_i , where (,dσ)(S,d_σ) is a metric space describing the information carried by the signatures. Given two signature-augmented persistence features (pi,i)(p_i, σ_i) and (pj,j)(p_j, σ_j), we define their signature-augmented ground metric as: d~q,λσ((pi,i),(pj,j))=(|si−sj|q+|mi−mj|q+λσqdσ(i,j)q)1/q, d_q, _σ ((p_i, σ_i),(p_j, σ_j) )= (|s_i-s_j|^q+|m_i-m_j|^q+ _σ^qd_σ( σ_i, σ_j)^q )^1/q, (10) where q≥1q≥ 1 and λσ≥0 _σ≥ 0 controls the relative importance of the signature information. Setting λσ=0 _σ=0 recovers the standard ground metric used in section 2. When signatures are vectors in a common Euclidean space, a direct choice is =ℝrS=R^r with dσ(i,j)=‖i−j‖qd_σ( σ_i, σ_j)=\| σ_i- σ_j\|_q. In this case, Equation 10 is simply the ℓq ^q distance between the augmented vectors (si,mi,λσi)(s_i,m_i, _σ σ_i) and (sj,mj,λσj)(s_j,m_j, _σ σ_j). Proposition 2. Let q≥1q≥ 1, λσ>0 _σ>0 and let dσd_σ be a metric on S. Then d~q,λσ d_q, _σ is a metric on ℝ2×R^2×S. Proof. Non-negativity, symmetry and identity of indiscernibles follow directly from the corresponding properties of the absolute value and dσd_σ. For the triangle inequality, let u,v,w∈ℝ2×u,v,w ^2×S. The two persistence-coordinate differences and the signature distance each satisfy their corresponding triangle inequalities. Applying these inequalities component-wise, followed by Minkowski’s inequality to the resulting three non-negative terms, gives d~q,λσ(u,w)≤d~q,λσ(u,v)+d~q,λσ(v,w) d_q, _σ(u,w)≤ d_q, _σ(u,v)+ d_q, _σ(v,w). Hence d~q,λσ d_q, _σ is a metric. ∎ The proposition allows signatures to be more general than vectors, although vector-valued signatures are sufficient for the constructions considered here. The corresponding signature-aware Wasserstein distance is obtained by replacing the ground metric dqd_q by d~q,λσ d_q, _σ in Equation 1. Thus, signatures modify the cost of matching two off-diagonal features but leave the assignment formulation itself unchanged. Matching to the diagonal. A signature is intended to characterize a persistence feature, not to prevent that feature from being removed or inserted through a diagonal match. We therefore let the diagonal projection of a signature-augmented point inherit its signature. For pi=(si,mi)p_i=(s_i,m_i), let Δ(pi)=((si+mi)/2,(si+mi)/2) (p_i)=((s_i+m_i)/2,(s_i+m_i)/2); its signature-augmented projection is (Δ(pi),i)( (p_i), σ_i). Consequently, matching a feature to its diagonal projection has no signature contribution and retains the usual persistence deletion cost: d~q,λσ((pi,i),(Δ(pi),i))=dq(pi,Δ(pi)). d_q, _σ ((p_i, σ_i),( (p_i), σ_i) )=d_q (p_i, (p_i) ). (11) As in the standard augmented-diagram formulation, the cost between two diagonal dummy points is set to zero. Hence signatures only affect correspondences between actual persistence features, while feature insertion and deletion retain their standard topological interpretation. Appendix E Existing Concept Erasure Criteria as Special Cases of TCR The editing formulation of TCR separates the criterion used to characterize a concept from the procedure used to modify the representation. In the main manuscript, this criterion is represented by the concept score mi=S(Z,Ci)m_i=S(Z,C_i) and an intervention is specified by a target value mi⋆m_i , possibly together with targets on the interaction structure. Concept signatures from Appendix D further allow the target to constrain information associated with a concept beyond its persistence coordinates. Together, these ingredients allow several existing concept-erasure criteria to be recovered as particular TCR editing objectives. Our goal is not to reproduce their optimization algorithms, which may rely on closed-form projections, adversarial optimization, sequential subspace construction or eigendecompositions, but rather the objectives or constraints that they seek to enforce. We group these connections according to the quantity used to characterize concept information: covariance and predictive recoverability for linear erasure, distributional and kernel dependence for nonlinear erasure and rate-distortion criteria. Most of these connections arise in a feature-wise specialization of TCR, in which interaction coordinates are fixed and only the concept maxima and, when needed, their signatures are edited. This is sufficient to recover the corresponding erasure criteria, while the saddle structure of TCR provides additional information that is not required by these special cases. E.1 Objective-Level Specialization of TCR Proposition 3 (Feature-wise specialization). Consider n semantically matched signature-augmented persistence features of the current and target TCR respectively given by (pi(Z),i(Z))(p_i(Z), σ_i(Z)) and (pi⋆,i⋆)(p_i , σ_i ), with pi(Z)=(si,mi(Z))p_i(Z)=(s_i,m_i(Z)) and pi⋆=(si,mi⋆)p_i =(s_i,m_i ). Assume that the interaction coordinate sis_i is identical in both features and does not contribute to the intervention. Under the semantic matching introduced in subsection 3.3 and the signature-augmented ground metric of Equation 10, the TCR objective reduces to: ℒTCR(Z)=∑i=1n(|mi(Z)−mi⋆|q+λσqdσ(i(Z),i⋆)q).L_TCR(Z)= _i=1^n ( |m_i(Z)-m_i |^q+ _σ^qd_σ\! ( σ_i(Z), σ_i )^q ). (12) Proof. Since (pi(Z),i(Z))(p_i(Z), σ_i(Z)) is constrained to match (pi⋆,i⋆)(p_i , σ_i ), its contribution to the LqL^q-Wasserstein objective is the q-th power of their signature-augmented ground distance. The interaction coordinates being equal, the term |si−si|q|s_i-s_i|^q vanishes and the contribution reduces to |mi(Z)−mi⋆|q+λσqdσ(i(Z),i⋆)q|m_i(Z)-m_i |^q+ _σ^qd_σ( σ_i(Z), σ_i )^q. Summing over the matched features gives Equation 12. ∎ Setting λσ=0 _σ=0 recovers the extrema-only specialization, for which Equation 12 reduces to ∑i|mi(Z)−mi⋆|q _i|m_i(Z)-m_i |^q. Consequently, once a meaningful score and, when needed, a meaningful concept signature have been chosen, their target values directly specify the corresponding TCR editing objective. In the following, we instantiate this construction with quantities having independent statistical interpretations, rather than defining features from arbitrary pre-existing losses. The resulting connections therefore arise from expressing covariance, predictive information, distributional discrepancy, statistical dependence and rate-distortion within the same editing interface. The specializations considered below are summarized compactly as follows. Each row specifies the TCR scores and target values associated with an existing erasure criterion: mc(P)=‖Pc‖2,c(P)=Pcmc⋆=0,c⋆=≡LEACEma(P)=‖Pa‖2,a(P)=Pa,a∈c,ymc⋆=0,c⋆=,≡SPLINCEmy⋆=‖y‖2,y⋆=ymc(P)=ℒc0−minψℒBCE(c,qψ(PH))mc⋆=0≡RLACE∼INLPmc(P)=(∑a<bMMD 2(ℙz|c=a,ℙz|c=b))1/2mc⋆=0≡LEOPARDmS(Z)=HSIC(Z,S),mA(Z)=HSIC(Z,A)mS⋆=0,≈ObliviatormA⋆>mA(H),A∈X,Xi,Ymc(Z)=U−R(Z∣K),mR(Z)=λR(Z)mc⋆=0,mR⋆=λR(H)≡KRaM \ array[]@l@ l@ l@ c@\;l@m_c(P)=\|Pg_c\|_2,\; σ_c(P)=Pg_c&m_c =0,& σ_c =0&≡&LEACE\\ m_a(P)=\|Pg_a\|_2,\; σ_a(P)=Pg_a,\;a∈\c,y\&m_c =0,& σ_c =0,&≡&SPLINCE\\[-2.84526pt] &m_y =\|g_y\|_2,& σ_y =g_y&&\\ m_c(P)=L_c^0- _ψL_BCE\! (c,q_ψ(PH) )&m_c =0&&≡&RLACE\\[-2.84526pt] &&& &INLP\\ m_c(P)= ( _a<b MMD^\,2\! (P_z c=a,P_z c=b ) )^1/2&m_c =0&&≡&LEOPARD\\ m_S(Z)=HSIC(Z,S),\;m_A(Z)=HSIC(Z,A)&m_S =0,&&≈&Obliviator\\[-2.84526pt] & @intercol m_A >m_A(H),\;A∈\X,X_i,Y\ &&\\ m_c(Z)=U-R(Z K),\;m_R(Z)=λ R(Z)&m_c =0,&m_R =λ R(H)&≡&KRaM array . Here, ≡ denotes an exact correspondence at the stated criterion or objective level under the specified restrictions, ≈ denotes a correspondence in the optimized quantities and direction of intervention without equality of the complete objective, and ∼ denotes a structural or procedural correspondence. These relations do not imply equivalence of the complete optimization algorithms or of the transformations they select. The following subsections detail each case. E.2 Linear and Predictive Erasure Covariance-based erasure: LEACE and SPLINCE. LEACE (Belrose et al., 2023) characterizes perfect linear erasure through the absence of cross-covariance between the edited representation and the concept, while selecting among valid erasers by minimizing representation distortion. SPLINCE (Holstege et al., 2025) additionally requires the cross-covariance with a target task to remain unchanged. These constraints can be recovered from covariance-based concept features and signatures. Let h∈ℝdh ^d be a centered random representation, let c and y denote centered scalar concept and task variables and define c=Cov(h,c)g_c=Cov(h,c) and y=Cov(h,y)g_y=Cov(h,y). For a linear edit z=Phz=Ph, their cross-covariances are: Cov(z,c)=Pc,Cov(z,y)=Py.Cov(z,c)=Pg_c, (z,y)=Pg_y. (13) For this covariance-based specialization, we consider the TCR concept score ma(P)=‖Pa‖2m_a(P)=\|Pg_a\|_2 for both the concept a=ca=c and the task a=ya=y. We associate each score with the covariance signature a(P)=Pa σ_a(P)=Pg_a and use the Euclidean signature distance. Proposition 4 (LEACE and SPLINCE as covariance-based TCR specializations). Consider the feature-wise specialization of Prop. 3 with the covariance scores and signatures defined above and λσ>0 _σ>0. For concept erasure, let the target concept feature satisfy mc⋆=0m_c =0 and c⋆= σ_c =0. Then its TCR contribution is zero if and only if the LEACE linear-guardedness constraint Pc=0Pg_c=0 holds. If a task feature is additionally assigned the targets my⋆=‖y‖2m_y =\|g_y\|_2 and y⋆=y σ_y =g_y, the joint TCR contribution is zero if and only if: Pc=0,Py=y,Pg_c=0, Pg_y=g_y, (14) which are the concept-erasure and task-covariance-preservation constraints of SPLINCE. The same result holds for vector-valued concepts or tasks by replacing the covariance vectors with the corresponding cross-covariance matrices and the Euclidean norm with the Frobenius norm. Proof. For the concept feature, mc(P)=0m_c(P)=0 and c(P)= σ_c(P)=0 are both equivalent to Pc=Pg_c=0. Since every term in Equation 12 is non-negative, its concept contribution is therefore zero if and only if the LEACE linear-erasure constraint holds. For the task feature, the signature contribution vanishes if and only if Py=yPg_y=g_y. This equality also implies my(P)=‖Py‖2=‖y‖2=my⋆m_y(P)=\|Pg_y\|_2=\|g_y\|_2=m_y , so the persistence-coordinate contribution vanishes as well. Hence, the complete TCR contribution is zero if and only if both conditions in Eq. equation 14 hold. The matrix-valued case follows identically using the Frobenius norm. ∎ Preserving only the scalar covariance norm of the task is in general weaker than the SPLINCE requirement Py=yPg_y=g_y. The covariance signature makes this distinction explicit: preserving y σ_y imposes the covariance-vector equality itself, while its associated covariance score is then preserved automatically. Thus, concept signatures allow the complete SPLINCE constraint to be expressed without replacing vector preservation by a weaker scalar constraint. LEACE and SPLINCE additionally select a particular transformation among those satisfying their respective constraints through a minimum-distortion criterion. The propositions above establish the erasure and preservation constraints represented by the TCR editing objective; reproducing the selected transformation also requires imposing the same distortion criterion and transformation family. Predictive erasure: RLACE and INLP. RLACE (Ravfogel et al., 2022) formulates linear concept erasure as an adversarial game in which a linear predictor seeks to recover the concept while a rank-constrained linear transformation seeks to prevent this prediction. This criterion is directly related to the BCE recoverability score used by TCR. Let c∈0,1c∈\0,1\ be a binary concept and, for a linear edit z=Phz=Ph, let: mcBCE(P)=ℒc0−minψℒBCE(c,qψ(Ph)),m_c^BCE(P)=L_c^0- _ψL_BCE(c,q_ψ(Ph)), (15) where qψq_ψ is a linear logistic probe and ℒc0L_c^0 is the BCE of the optimal constant predictor, as in Equation 2. Assuming that the probe family contains this constant predictor, mcBCE(P)≥0m_c^BCE(P)≥ 0. Proposition 5 (RLACE as a predictive-recoverability TCR specialization). Let A be a common feasible family of linear edits for TCR and the adversarial erasure problem. Consider a feature-wise TCR specialization with the score of Equation 15, target mc⋆=0m_c =0 and no signature contribution. For any q>0q>0, minimizing the corresponding TCR contribution over P∈P is equivalent to maximizing the loss of the optimal linear logistic predictor: argminP∈|mcBCE(P)|q=argmaxP∈minψℒBCE(c,qψ(Ph)). _P |m_c^BCE(P) |^q= _P _ψL_BCE(c,q_ψ(Ph)). (16) Consequently, choosing the same relaxed rank-constrained transformation family as RLACE recovers its concept-erasure criterion at the objective level. Proof. Since mcBCE(P)≥0m_c^BCE(P)≥ 0 and q>0q>0, minimizing |mcBCE(P)|q|m_c^BCE(P)|^q is equivalent to minimizing mcBCE(P)m_c^BCE(P). The null loss ℒc0L_c^0 does not depend on P, hence from Equation 15 this is equivalent to maximizing minψℒBCE(c,qψ(Ph)) _ψL_BCE(c,q_ψ(Ph)), which gives Equation 16. ∎ The connection also appears directly in the optimization procedure. At an TCR iteration, the probe is fitted on the current representation, approximately minimizing ℒBCE(c,qψ(Ph))L_BCE(c,q_ψ(Ph)). The subsequent gradient update of P toward mc⋆=0m_c =0 reduces the recoverability in Equation 15, and therefore increases the loss attained by this fitted predictor. After updating the representation, rebuilding the TCR refits the probe and provides a new best response to the current transformation. Thus, the repeated probe-fitting and representation-update steps can be viewed as the two sides of the same predictor–eraser adversarial game as RLACE. The two methods differ in how this game is optimized and in the constraints imposed on the transformation. This correspondence concerns a restricted specialization of TCR. RLACE restricts the intervention to a fixed-rank orthogonal projection and considers a single adversarial predictive-erasure objective. In contrast, TCR can use different concept scores and transformation families, jointly represent several concept and task features and additionally optimize their interactions through the target topology. The RLACE objective is therefore recovered when restricting TCR to a single BCE-based concept feature with a zero target, no interaction constraints and the same projection family; these restrictions are not inherent to the general TCR formulation. INLP (Ravfogel et al., 2020) targets the same general notion of linear guardedness through a different sequential construction. At each iteration, it fits a linear concept classifier on the current representation, projects the representation onto the classifier nullspace and refits a new classifier on the projected representation. Each projection therefore exactly removes the direction used by the current classifier, while the repetition searches for additional predictive directions that remain after the previous ones have been removed. In contrast, BCE-based TCR uses the recoverability score of Equation 15 as its editing criterion and replaces each hard nullspace projection by a gradient update that reduces the current concept recoverability before refitting the probe. If guardedness is evaluated with the same linear probe family and BCE criterion, both procedures target the condition mcBCE=0m_c^BCE=0. INLP can therefore be viewed as a hard sequential counterpart of this predictive-erasure specialization of TCR, rather than as a distinct scalar objective reproduced by TCR. The original INLP formulation uses classification accuracy to operationalize guardedness, so this correspondence concerns the shared linear-predictive constraint rather than their optimization procedures. E.3 Distributional and Nonlinear Dependence The previous specializations characterize concept information through covariance or linear predictability. Nonlinear erasure methods instead consider stronger distributional or statistical dependencies between the edited representation and the concept. We next show that such criteria can similarly be expressed through TCR concept scores. Distribution matching: LEOPARD. LEOPARD (Saillenfest & Lemberger, 2025) seeks a rank-constrained orthogonal projection for which the class-conditional distributions of a discrete concept become indistinguishable. Let c∈1,…,Kc∈\1,…,K\ and z=Phz=Ph. We define the following distributional TCR concept score: mcMMD(P)=(∑a<bMMD^2(ℙz|c=a,ℙz|c=b))1/2.m_c^MMD(P)= ( _a<b MMD^2 (P_z c=a,P_z c=b ) )^1/2. (17) For a binary concept, this is the empirical MMD between its two class-conditional distributions. Proposition 6 (LEOPARD as a distribution-matching TCR specialization). Consider the feature-wise specialization of Prop. 3 with the score of Equation 17, target mc⋆=0m_c =0, no signature contribution and q=2q=2. Then the corresponding TCR contribution is exactly the LEOPARD erasure loss: |mcMMD(P)|2=∑a<bMMD^2(ℙz|c=a,ℙz|c=b). |m_c^MMD(P) |^2= _a<b MMD^2 (P_z c=a,P_z c=b ). (18) Restricting P=UU⊤P=U to the same rank-r parameterization and adding the LEOPARD projection penalty γ‖U⊤U−Ir‖F2γ\|U U-I_r\|_F^2 therefore recovers its complete training objective. Proof. With mc⋆=0m_c =0, no signature contribution and q=2q=2, Equation 12 gives the contribution |mcMMD(P)|2|m_c^MMD(P)|^2. Substituting Equation 17 yields Equation 18. The remaining LEOPARD term depends only on the transformation parameterization and is recovered by imposing its orthogonality penalty. ∎ Thus, LEOPARD corresponds to a distributional choice of the TCR concept score: instead of measuring whether the concept is linearly predictable, it measures whether its class-conditional representation distributions can still be distinguished. With a characteristic kernel, a zero population MMD implies equality of these distributions. The adjustable projection rank and the orthogonalization used by LEOPARD concern how this criterion is optimized, not the criterion itself. Kernel dependence: Obliviator. Obliviator (Akbari et al., 2025) extends the erasure criterion from linear predictability to nonlinear statistical dependence. Let S denote the unwanted concept, Y the task, X the original representation and XiX_i the representation provided as input to the encoder at iteration i. For its output ZiZ_i, Obliviator minimizes an empirical RKHS objective that, up to the common HSIC normalization, can be written as: HSIC(Zi,S)−τxHSIC(Zi,X)−τxiHSIC(Zi,Xi)−τyHSIC(Zi,Y),HSIC(Z_i,S)- _xHSIC(Z_i,X)- _x_iHSIC(Z_i,X_i)- _yHSIC(Z_i,Y), (19) where the task term is omitted in the unsupervised setting. The first term directly defines a nonlinear TCR concept score mS(Z)=HSIC(Z,S)m_S(Z)=HSIC(Z,S). Targeting mS⋆=0m_S =0 therefore recovers Obliviator’s erasure criterion: with characteristic kernels, a vanishing population HSIC corresponds to statistical independence between the edited representation and the unwanted concept. The remaining terms follow the same dependence measure but encourage information about X, XiX_i and Y to remain observable in the edited representation. They can consequently be represented in TCR as additional features with scores mA(Z)=HSIC(Z,A)m_A(Z)=HSIC(Z,A), for A∈X,Xi,YA∈\X,X_i,Y\, whose target values specify the desired level of preservation. This recovers the same erasure–utility quantities used by Obliviator, although its encoder objective maximizes the utility HSIC terms directly rather than matching them to fixed target values. The correspondence concerns the criterion encoded by these dependence quantities rather than the complete Obliviator procedure. Obliviator alternates encoder optimization with an additional RKHS disentanglement step obtained from a constrained eigenvalue problem, which exposes utility-related information while remaining insensitive to S before training the next encoder. In contrast, TCR expresses the desired decrease and preservation of these concept scores through its target representation and optimizes the chosen transformation accordingly. E.4 Rate-Distortion Erasure Kernelized rate-distortion: KRaM. KRaM (Chowdhury et al., 2023) formulates concept erasure through a kernelized rate-distortion criterion. Let Z=Tθ(H)∈ℝN×dZ=T_θ(H) ^N× d denote the edited representation and let K∈ℝN×NK ^N× N encode similarities between the concept labels of the samples. KRaM considers the standard rate-distortion R(Z)R(Z) and its kernelized counterpart: R(Z∣K)=12logdet2(I+dNϵ2(ZZ⊤⊙K)).R(Z K)= 12 _2 (I+ dNε^2(Z K) ). (20) Maximizing R(Z∣K)R(Z K) encourages samples that are similar according to the concept kernel to become dissimilar in the edited representation. Since this can simultaneously expand the overall representation space, KRaM additionally constrains its standard coding rate around the original value b=R(H)b=R(H), leading to the relaxed objective: maxθR(Z∣K)−λ|R(Z)−b|. _θ\;R(Z K)-λ |R(Z)-b |. (21) Both terms can be expressed through TCR features. Under the normalization assumed by KRaM, let U denote the constant upper bound of R(Z∣K)R(Z K). We define a concept feature and a rate-preservation feature with scores and targets: mc(Z)=U−R(Z∣K),mc⋆=0,mR(Z)=λR(Z),mR⋆=λb.m_c(Z)=U-R(Z K), m_c =0, m_R(Z)=λ R(Z), m_R =λ b. (22) The first feature measures the gap to maximal kernelized rate-distortion and therefore vanishes as the concept-erasure criterion of KRaM is optimized. The second preserves the coding rate of the original representation. With q=1q=1 and no signature contribution, their joint TCR contribution is: ℒTCR(Z) _TCR(Z) =|mc(Z)−mc⋆|+|mR(Z)−mR⋆| = |m_c(Z)-m_c |+ |m_R(Z)-m_R | =U−R(Z∣K)+λ|R(Z)−b|. =U-R(Z K)+λ |R(Z)-b |. (23) Since U is independent of the transformation, minimizing Equation 23 is equivalent to maximizing the KRaM objective of Equation 21. KRaM can therefore be expressed as a two-feature rate-distortion specialization of TCR, with one feature driven toward concept erasure and the other preserving the global representation rate. The correspondence concerns the optimized criterion: KRaM additionally normalizes the transformed representations because the rate-distortion quantities are scale-sensitive and uses a nonlinear erasure function, whereas TCR is not restricted to this transformation or optimization procedure. Appendix F Affine Reparameterization Invariance of TCR We show that the analytical MSE–overlap instantiation of TCR is invariant to any invertible affine change of coordinates of the representation. This includes, for example, permutations, rotations and nonsingular rescalings of the representation dimensions. Proof of Prop. 1. Let ZcZ_c and Z~c Z_c denote the centered representations. Centering removes the translation a, so Z~c=ZcB⊤ Z_c=Z_cB . Since B is invertible, right multiplication by B⊤B does not change the column space and therefore Col(Z~c)=Col(Zc)Col( Z_c)=Col(Z_c). Let ΠZ _Z denote the orthogonal projector onto this common column space. For concept CiC_i, let ¯i c_i be its centered annotation vector. Since the annotations are centered, the optimal constant predictor is zero, with mean-squared error: MSEnull=1N‖¯i‖22.MSE_null= 1N \| c_i \|_2^2. The predictions of a linear probe belong to Col(Zc)Col(Z_c). Hence, the fitted values of the optimal least-squares probe are given by the orthogonal projection ^i=ΠZ¯i c_i= _Z c_i, and its mean-squared error is: MSEprobe=1N‖¯i−ΠZ¯i‖22.MSE_probe= 1N \| c_i- _Z c_i \|_2^2. By definition of an orthogonal projection, ΠZ¯i _Z c_i and the residual ¯i−ΠZ¯i c_i- _Z c_i are orthogonal. Let p:=ΠZ¯ip:= _Z c_i and r:=¯i−ΠZ¯ir:= c_i- _Z c_i, so that ¯i=p+r c_i=p+r. Since p and r are orthogonal, p⊤r=0p r=0. Therefore, ‖¯i‖22=‖p+r‖22=(p+r)⊤(p+r)=p⊤p+2p⊤r+r⊤r=‖p‖22+‖r‖22\| c_i\|_2^2=\|p+r\|_2^2=(p+r) (p+r)=p p+2p r+r r=\|p\|_2^2+\|r\|_2^2. Hence: ‖¯i‖22=‖ΠZ¯i‖22+‖¯i−ΠZ¯i‖22. \| c_i \|_2^2= \| _Z c_i \|_2^2+ \| c_i- _Z c_i \|_2^2. The MSE concept score is the gain of the best linear probe over the best constant predictor. Thus: mi=MSEnull−MSEprobe=1N‖ΠZ¯i‖22.m_i=MSE_null-MSE_probe= 1N \| _Z c_i \|_2^2. Since ΠZ _Z is the same for Z and Z~ Z, mim_i is identical for both representations Z and Z~ Z for every concept. Likewise, the predictive Gram matrix can be expressed using the same projected annotations. Using: ΣZ=1NZc⊤Zc,gi=1NZc⊤¯i, _Z= 1NZ_c Z_c, g_i= 1NZ_c c_i, and the original definition Aij=gi⊤ΣZ+gjA_ij=g_i _Z^+g_j, we obtain: Aij A_ij =(1NZc⊤¯i)⊤(1NZc⊤Zc)+(1NZc⊤¯j) = ( 1NZ_c c_i ) ( 1NZ_c Z_c )^+ ( 1NZ_c c_j ) =1N¯i⊤Zc(Zc⊤Zc)+Zc⊤¯j. = 1N c_i Z_c (Z_c Z_c )^+Z_c c_j. The matrix ΠZ=Zc(Zc⊤Zc)+Zc⊤ _Z=Z_c(Z_c Z_c)^+Z_c is the orthogonal projector onto Col(Zc)Col(Z_c). Hence: Aij=1N¯i⊤ΠZ¯j.A_ij= 1N c_i _Z c_j. We have ¯i⊤ΠZ¯j=¯i⊤ΠZ⊤ΠZ¯j=(ΠZ¯i)⊤(ΠZ¯j)=⟨ΠZ¯i,ΠZ¯j⟩ c_i _Z c_j= c_i _Z _Z c_j=( _Z c_i) ( _Z c_j)= _Z c_i, _Z c_j since ΠZ _Z is an orthogonal projector making ΠZ⊤=ΠZ _Z = _Z and ΠZ2=ΠZ _Z^2= _Z. Hence: Aij=1N⟨ΠZ¯i,ΠZ¯j⟩.A_ij= 1N _Z c_i, _Z c_j . Thus every AijA_ij is unchanged. Consequently, whenever the overlap interaction is defined: ρijoverlap=Aij2AiiAjj, _ij^overlap= A_ij^2A_iA_j, it is also unchanged. Since the saddle value is sij=min(mi,mj)ρijoverlaps_ij= (m_i,m_j) _ij^overlap, all concept and saddle values of the scalar field are identical for Z and Z~ Z. The two representations therefore induce the same scalar field on the same concept mesh and hence the same filtration, persistence pairs and persistence diagram. This proves TCR(Z~)=TCR(Z)TCR( Z)=TCR(Z). If the scores are normalized by Var(i)Var(c_i), the conclusion is unchanged because this quantity depends only on the concept annotations. ∎ Remark 1 (Scope of the result). The proposition concerns the MSE–overlap instantiation of TCR, for which the construction is determined by least-squares predictions. It does not require Z or its covariance matrix to have full rank. It establishes invariance to invertible affine changes of representation coordinates, not to arbitrary non-invertible or nonlinear transformations. Appendix G Stability of TCR We show that, away from degenerate cases, small perturbations of a representation induce small perturbations of its TCR. We consider the normalized MSE formulation with predictive-overlap interactions from subsection 3.2. The result is local: this is sufficient to state that TCR varies continuously and at most linearly under sufficiently small representation perturbations. G.1 Representation-to-TCR Stability Let H∈ℝN×dH ^N× d be a centered representation of N samples, and let 1,…,nC∈ℝNc_1,…,c_n_C ^N be fixed, centered, non-constant concept annotation vectors. Set n:=N−1n:=N-1. For any centered representation Z∈ℝN×dZ ^N× d, define the empirical covariance and concept cross-covariances as: Σ(Z)=1nZ⊤Z,i(Z)=1nZ⊤i. (Z)= 1nZ Z, _i(Z)= 1nZ c_i. Whenever Σ(Z) (Z) is invertible, define the predictive Gram entries as: Aij(Z)=i(Z)⊤Σ(Z)−1j(Z).A_ij(Z)=g_i(Z) (Z)^-1g_j(Z). For each concept, define its variance as: σi2=1n‖i‖22>0. _i^2= 1n\|c_i\|_2^2>0. The normalized MSE concept scores, predictive-overlap interactions and corresponding saddle values are respectively defined as: mi(Z)=Aii(Z)σi2,ρij(Z)=Aij(Z)2Aii(Z)Ajj(Z),sij(Z)=min(mi(Z),mj(Z))ρij(Z).m_i(Z)= A_i(Z) _i^2, _ij(Z)= A_ij(Z)^2A_i(Z)A_j(Z), s_ij(Z)= (m_i(Z),m_j(Z)) _ij(Z). We denote by fZf_Z the resulting scalar field on the fixed concept mesh ℳM, by TCR(Z)TCR(Z) its persistence diagram and by dBd_B the Bottleneck distance, being the q→∞q→∞ limit of the LqL^q-Wasserstein distance, measuring the largest cost of a matched pair instead of accumulating all matching costs. Proof of Theorem 1. The proof follows from the fact that the scalar values defining the concept mesh are locally Lipschitz functions of the representation, followed by stability of persistence diagrams. First, the covariance and cross-covariance maps defined above are polynomial and linear in Z, respectively, and are therefore smooth. Since Σ(H) (H) is positive definite, continuity implies that there exists ε1>0 _1>0 such that Σ(Z) (Z) remains positive definite whenever ‖Z−H‖F<ε1\|Z-H\|_F< _1. The matrix inverse is smooth on the set of invertible matrices. Hence every Aij(Z)A_ij(Z) is smooth, and therefore locally Lipschitz, around H. Because each σi2 _i^2 is a fixed positive constant, every maximum mi(Z)m_i(Z) is also locally Lipschitz. Moreover, mi(H)>0m_i(H)>0 implies Aii(H)>0A_i(H)>0. Since there are finitely many concepts and every Aii(Z)A_i(Z) is continuous, there exists ε2>0 _2>0 such that all Aii(Z)A_i(Z) remain strictly positive whenever ‖Z−H‖F<ε2\|Z-H\|_F< _2. We may therefore choose ε≤min(ε1,ε2) ≤ ( _1, _2), so that both the covariance inverse and every denominator appearing in ρij _ij are well-defined throughout the ball. Consequently, every ρij(Z) _ij(Z) is locally Lipschitz as well. Finally, the map (a,b)↦min(a,b)(a,b) (a,b) is Lipschitz, so every sij(Z)s_ij(Z) is locally Lipschitz. Since the concept mesh contains finitely many maxima and saddles, their local Lipschitz constants can be bounded by a single constant C>0C>0 on a possibly smaller ball of radius ε . Hence: ‖fH−fH′‖∞≤C‖H−H′‖Fwhenever ‖H−H′‖F<ε.\|f_H-f_H \|_∞≤ C\,\|H-H \|_F \|H-H \|_F< . (24) The two scalar fields are defined on the same concept mesh. Although TCR uses an upper-star filtration, applying the usual persistence stability theorem to the negated scalar fields (Edelsbrunner & Harer, 2009) gives: dB(TCR(H),TCR(H′))≤‖fH−fH′‖∞.d_B\! (TCR(H),TCR(H ) )≤\|f_H-f_H \|_∞. Combining this inequality with Equation 24 proves Theorem 1. ∎ Remark 2 (Scope of the result). The result is local and deliberately excludes degenerate representations. The radius ε is chosen small enough such that the covariance remains invertible and all concept recoverabilities remain strictly positive whenever |H−H′|F<ε|H-H |_F< . Within this neighborhood, all quantities defining the TCR scalar field are smooth or Lipschitz functions of the representation. Handling rank changes requires controlling the Moore–Penrose pseudoinverse and is not needed for the local stability statement above. A more quantitative version could replace this neighborhood by explicit lower bounds on the smallest eigenvalue and on concept recoverability, together with an explicit upper bound on |H−H′|F|H-H |_F. Importantly, the proof does not assume that the persistence pairing remains fixed, since the persistence stability theorem directly controls such changes. G.2 Network-to-TCR Stability The representation-level result immediately yields a network-level statement whenever the selected representation is locally Lipschitz with respect to the network parameters. Let HθH_θ denote the matrix collecting the selected representations of the same N samples for a network with parameters θ. Corollary 1 (Network-to-TCR stability). Assume the hypotheses of Theorem 1 hold at HθH_θ, with stability radius ε , and assume that the representation map is locally Lipschitz in the parameters: there exist Lθ>0L_θ>0 and η>0η>0 such that ‖Hθ−Hθ′‖F≤Lθ‖θ−θ′‖\|H_θ-H_θ \|_F≤ L_θ\|θ-θ \| whenever ‖θ−θ′‖<η\|θ-θ \|<η. Then there exists C>0C>0 such that, for every θ′θ satisfying: ‖θ−θ′‖<min(η,εLθ),\|θ-θ \|< \! (η, L_θ ), we have: dB(TCR(Hθ),TCR(Hθ′))≤CLθ‖θ−θ′‖.d_B\! (TCR(H_θ),TCR(H_θ ) )≤ CL_θ\|θ-θ \|. Proof. The parameter restriction ensures that ‖Hθ−Hθ′‖F<ε\|H_θ-H_θ \|_F< , so Hθ′H_θ lies inside the representation-level stability ball. Applying Theorem 1 and then the assumed local Lipschitz bound for the representation map proves the result. ∎ Appendix H Detailed Experimental Setup H.1 Setup for Concept Disentanglement Before Erasure Datasets and representations. We evaluate four binary classification datasets spanning image and text modalities. Table 3 summarizes their task and concept labels, split sizes, fixed representations, and dimensions before and after PCA for the erasure experiments. Table 4 summarizes their group size, task-concept agreement P(y=c)P(y=c) and label imbalance P(y=0)P(y=0). Table 3: Datasets and fixed representations used for concept disentanglement before erasure. Dataset Task y Concept c Train Val Test Representation Original dim. PCA dim. Waterbirds Bird type Background 4,795 1,199 5,794 ResNet-50 avgpool 2,048 128 Bias in Bios SN Profession (surgeon/nurse) Gender 21,145 3,256 8,136 BERT [CLS] 768 128 Dial Mention Mention detection Dialect 56,140 8,000 8,000 BERT [CLS] 768 128 Jigsaw Toxicity classification Gender 24,508 2,724 6,808 BERT [CLS] 768 128 Table 4: Group composition of each dataset split. The four group sizes are indexed by the task and concept labels (y,c)(y,c), with their percentage of the split shown in parentheses. Agreement is P(y=c)P(y=c) and label imbalance is reported as P(y=0)P(y=0). Dataset Split Group size nycn_yc Agreement P(y=c)P(y=c) Label imbalance P(y=0)P(y=0) n00n_00 n01n_01 n10n_10 n11n_11 Waterbirds Train 3,498 (72.95%) 184 (3.84%) 56 (1.17%) 1,057 (22.04%) 94.99% 76.79% Val 467 (38.95%) 466 (38.87%) 133 (11.09%) 133 (11.09%) 50.04% 77.81% Test 2,255 (38.92%) 2,255 (38.92%) 642 (11.08%) 642 (11.08%) 50.00% 77.84% Bias in Bios SN Train 7,521 (35.57%) 1,308 (6.19%) 1,127 (5.33%) 11,189 (52.92%) 88.48% 41.75% Val 1,158 (35.57%) 202 (6.20%) 174 (5.34%) 1,722 (52.89%) 88.45% 41.77% Test 2,894 (35.57%) 504 (6.19%) 434 (5.33%) 4,304 (52.90%) 88.47% 41.76% Dial Mention Train 25,263 (45.00%) 2,807 (5.00%) 2,807 (5.00%) 25,263 (45.00%) 90.00% 50.00% Val 2,000 (25.00%) 2,000 (25.00%) 2,000 (25.00%) 2,000 (25.00%) 50.00% 50.00% Test 2,000 (25.00%) 2,000 (25.00%) 2,000 (25.00%) 2,000 (25.00%) 50.00% 50.00% Jigsaw Train 11,029 (45.00%) 1,225 (5.00%) 1,225 (5.00%) 11,029 (45.00%) 90.00% 50.00% Val 681 (25.00%) 681 (25.00%) 681 (25.00%) 681 (25.00%) 50.00% 50.00% Test 1,702 (25.00%) 1,702 (25.00%) 1,702 (25.00%) 1,702 (25.00%) 50.00% 50.00% Waterbirds (Sagawa et al., 2020) is constructed by segmenting bird foregrounds from the CUB dataset and compositing them onto land or water scenes from Places. The task label distinguishes landbirds from waterbirds, while the concept records the background type. In the training set, 95% of each bird type appears on its associated background; validation and test instead allocate each bird type equally across both backgrounds. The resulting shift makes background strongly predictive during training while retaining rare landbird-on-water and waterbird-on-land groups, which expose reliance on the spurious background cue during evaluation. Bias in Bios SN (De-Arteaga et al., 2019) is obtained by retaining the surgeon and nurse examples from Bias in Bios, a corpus of 397,340 English online biographies collected from Common Crawl and labeled with one of 28 occupations and a binary gender inferred from gendered pronouns. We preserve the original train, validation, and test splits after filtering. The task is to predict which of the two professions is associated with a biography, while the concept is gender. This subset preserves the pronounced profession–gender imbalance in the source corpus and therefore tests whether gender information can be erased without removing profession-relevant content. Dial Mention (Blodgett et al., 2016; Elazar & Goldberg, 2018) is built from Twitter messages in the DIAL corpus. Its task is to determine whether a tweet originally contained an @mention, thereby distinguishing conversational from non-conversational tweets; all mention tokens are removed before the tweet is encoded, so they cannot directly reveal the label. The concept distinguishes the African-American English (AAE) and Standard American English (SAE) dialect categories, which the original benchmark uses as proxies for non-Hispanic Black and non-Hispanic White authors, respectively. The source construction retains only dialect assignments whose estimated confidence exceeds 80%, selecting tweets for which the dialect proxy is comparatively unambiguous while preserving dialect-linked lexical variation in the encoded text. Jigsaw (Borkan et al., 2019) contains public online comments from the Civil Comments platform, crowd-annotated for both toxicity and mentioned identities. We assign the positive task label when the aggregated toxicity score is greater than 0.50.5. For the concept, we retain comments for which at least one selected identity annotation is available and aggregate eight source attributes: female, male, transgender, other gender, heterosexual, homosexual/gay/lesbian, bisexual, and other sexual orientation. The concept is positive when any of these attributes has a nonzero annotation and negative otherwise; we use the compact name “Gender” in tables and results. We extract one fixed representation per example: the average-pooling output of an ImageNet-pretrained ResNet-50 for Waterbirds, and the final-layer [CLS] representation of BERT-base for the text datasets. Before applying either TCR or an erasure baseline, we fit PCA on the training representations and use the same transformation for validation and test data. Each dataset is therefore represented in its own 128-dimensional PCA space. The feature extractors remain frozen throughout, so methods differ only in how they transform these representations post hoc. Evaluation protocol. After each representation intervention, we independently fit linear logistic-regression probes for the task y and concept c on the transformed training representations. Inputs are standardized using training statistics and the ℓ2 _2 regularization parameter is selected from C∈0.01,0.1,1,10,100C∈\0.01,0.1,1,10,100\ by validation accuracy. The selected probes are kept fixed when evaluating the test set. Task utility is measured by accuracy and worst-group accuracy (WGA), where the four groups correspond to the joint task–concept labels (y,c)(y,c) and WGA is the minimum task accuracy over these groups. Our primary concept-leakage metric is the adjusted balanced accuracy of the linear concept probe, defined as Leak=2BalAcc−1Leak=2\,BalAcc-1, where BalAccBalAcc averages the concept-class recalls. Thus, 00 corresponds to chance-level balanced prediction and 11 to perfect concept prediction, independently of concept-class imbalance. The leakage probe itself is unweighted; “balanced” refers to the evaluation metric. Unless stated otherwise, the strong-erasure setting requires Leak<0.1Leak<0.1. Additional oracle-conditioned and non-linear leakage diagnostics are described in Appendix I.1.1. Model selection. For methods with tunable erasure hyperparameters, candidates under a leakage constraint t are considered feasible only when both their training and validation adjusted balanced linear leakage are at most t. Among feasible candidates, we maximize the mean of validation task accuracy and validation WGA. Test metrics are used only after model selection. TCR configuration. For both TCR-DIS and TCR-ERA, we use BCE recoverability with differentiable linear logistic probes, with C=0.01C=0.01, Newton updates and convergence-based stopping. The representation transformation is a two-layer ReLU MLP mapping the 128-dimensional PCA representation through a hidden dimension of 256 back to 128 dimensions. TCR-DIS additionally uses interference saddles between the concept and task nodes and is optimized for 200 iterations. Its learning rate is selected on validation after applying LEACE to the transformed representation, after which the same learned disentangling transformation is evaluated with all considered erasure methods. In contrast, TCR-ERA directly suppresses the concept score while preserving the task score and does not use a saddle interaction. Its learning rate, task-preservation weight and number of optimization iterations are selected jointly using the same leakage-constrained validation criterion used for model selection, considering both SGD and Adam candidates. H.2 Setup for Concept Transfer Dataset and architectures. We use the standard CIFAR-100 (Krizhevsky & Hinton, 2009) split, containing 50,000 training and 10,000 test images from 100 classes. For hyperparameter selection, we hold out a deterministic class-stratified 10% of the training set, resulting in 45,000 training and 5,000 validation images. After selecting the hyperparameters, we discard this validation split, retrain on all 50,000 training images, and report performance on the 10,000 test images. Following the protocol of CKD Tian et al. (2020), we use their pretrained ResNet32×4 teacher and consider ResNet8×4 and ShuffleNetV2 students. The teacher is frozen throughout distillation, while the students are trained from scratch. Their architectures are summarized in Table 5. Table 5: Teacher and student architectures used for concept transfer. Role Architecture Penultimate dim. Parameters Training Teacher ResNet32×4 256 7,433,860 CKD pretrained checkpoint, frozen Student ResNet8×4 256 1,233,540 From scratch Student ShuffleNetV2 1,024 1,355,528 From scratch Distillation training protocol and losses. We follow the CIFAR-100 training protocol of CKD (Tian et al., 2020): all students are trained for 240 epochs with a batch size of 64 using SGD with momentum 0.90.9 and weight decay 5×10−45× 10^-4. The initial learning rate is 0.050.05 for ResNet8×4 and 0.010.01 for ShuffleNetV2 and is multiplied by 0.10.1 after epochs 150, 180 and 210. We use random cropping and horizontal flipping during training and the standard CIFAR-100 normalization. For logit distillation, the temperature is fixed to T=4T=4. The weights used by each method are summarized in Table 7, where the total objective is written as γℒCE+αℒKD+βℒaux _CE+ _KD+ _aux. FitNet uses the CKD hint layer 2 with a learned convolutional regressor and a mean-squared hint loss, while SPKD matches pairwise similarities of the last convolutional feature maps. Table 6: Weights used in the concept transfer experiments. A dash indicates that no auxiliary loss is used. Method γ α β Auxiliary loss Student 1.0 0 – – KD 0.1 0.9 – – FitNet 1.0 0 100 Hint regression SPKD 1.0 0 3000 Similarity preservation TopKD 0.1 0.9 4.0 Topology distillation TCR k10k_10 0.1 0.9 1.0 TCR matching TCR k20k_20 0.1 0.9 0.8 TCR matching Table 7: Validation-selected auxiliary-loss weights β for TCR. k2k_2 k5k_5 k10k_10 k20k_20 Full image 30.0 20.0 10.0 1.0 4 crops 5.0 2.0 0.2 0.9 9 crops 6.0 1.3 1.0 0.8 16 crops 0.9 1.0 2.0 2.0 For TCR, the auxiliary-loss weight is selected using the held-out CIFAR-100 validation split described above; the selected configuration is then retrained from scratch using the full training set before test evaluation. We apply the same procedure to TopKD (Kim et al., 2024). TopKD is not part of the CKD benchmark and its loss coefficients are tuned in the original work for a different teacher–student configuration. We therefore retain the same γ=0.1γ=0.1 and α=0.9α=0.9 CE/KD weighting as our KD baseline and select only the weight of its topology loss on validation data. This ensures that improvements are compared under the same underlying CE/KD objective rather than being confounded by different loss weighting. Final results are averaged over five random seeds. Concept Distillation (Gupta et al., 2023) leverages teacher-defined concepts to (de)sensitize a student to selected concepts rather than transferring their organization between representations and is therefore not directly applicable to our setting. TCR construction. For concept transfer, we use the analytical normalized-MSE formulation of Appendix B with least-squares probes and the continuous Dictionary Learning coefficients as concept targets. During training, the student TCR is estimated from a rolling bank of 2,048 augmented training representations, whose least-squares sufficient statistics are updated incrementally. The first epoch initializes this bank before TCR matching is applied. Concept identities and coefficients are obtained once from the deterministic frozen-teacher concept discovery, and the resulting teacher TCR remains a fixed reference throughout student training. Concept discovery and recoverability. We discover concepts from the frozen teacher penultimate representation using Dictionary Learning with K∈2,5,10,20K∈\2,5,10,20\ components, non-negative sparse codes and a fixed discovery seed. Each value of K defines an independently fitted dictionary: the K=2,5,10K=2,5,10 and 2020 concept banks are therefore not nested subsets of a common dictionary. This distinction is important when evaluating generalization across concept banks in subsection 4.2. The same frozen teacher-defined dictionaries are used when evaluating all distillation methods, so concept recoverability always refers to an identical target concept representation. In addition to concepts discovered from full images, we consider concepts discovered at progressively finer spatial scales. Following the crop-based concept discovery strategy of CRAFT (Fel et al., 2023b), neighboring crops use a stride approximately equal to 80%80\% of their width. We construct deterministic 2×22× 2, 3×33× 3 and 4×44× 4 grids, corresponding respectively to 4, 9 and 16 crops per image. For the 32×3232× 32 CIFAR-100 images, the resulting crop sizes are 18×1818× 18, 12×1212× 12 and 9×99× 9 pixels, with strides 14, 9 and 7 pixels, respectively. Each crop is resized to 32×3232× 32 before extracting its teacher representation. The dictionary is fitted on these crop representations and subsequently used to encode the full-image teacher representations, providing concept coefficients at increasingly local spatial scales. Together with the full-image setting, this yields four concept-discovery scales. The main transfer comparison in Table 2 uses the 3×33× 3 (9-crop) concept discovery, while the other scales are used to evaluate the effect of concept locality and generalization across independently discovered concept banks. We quantify concept recoverability (CR) by measuring how accurately the teacher-derived concept coefficients can be linearly decoded from a model representation on unseen images. For a dictionary containing K concepts, let C∈ℝN×KC ^N× K denote the corresponding teacher concept coefficients and H∈ℝN×dH ^N× d the representation whose recoverability we evaluate. Using the CIFAR-100 training set, we fit a linear least-squares decoder from HtrainH_train to CtrainC_train: C^train=HtrainW+b C_train=H_trainW+b, where W and b are estimated exclusively from the training samples. The same frozen decoder is then applied to the test representation, C^test=HtestW+b C_test=H_testW+b. For a test image i and concept j, c^ij c_ij therefore denotes the coefficient predicted by this linear decoder, while cijc_ij denotes the corresponding coefficient obtained by encoding the teacher representation with the fixed concept dictionary. For each concept j, we compute the test coefficient of determination: Rj2=1−∑i∈test(cij−c^ij)2∑i∈test(cij−c¯j,train)2,R_j^2=1- _i (c_ij- c_ij)^2 _i (c_ij- c_j,train)^2, where c¯j,train c_j,train is the mean teacher coefficient of concept j over the training set and therefore defines a constant predictor that uses no test information. An Rj2R_j^2 of 11 indicates perfect linear recovery of the teacher coefficient, while 00 corresponds to the performance of this training-mean predictor; negative values indicate worse generalization than the constant predictor. We finally define CR kKk_K as the mean test R2R^2 over the K concepts and report it as a percentage. Thus, CR measures how much of the teacher-defined concept representation remains linearly accessible from the evaluated representation on unseen data: a high CR indicates that a linear decoder fitted to predict the teacher’s concept coefficients on the training set continues to recover those same concepts accurately on previously unseen images, independently of the model’s task accuracy. Appendix I Additional Experiments & Results I.1 Concept Disentanglement Before Erasure Table 8: Execution time (seconds) for standalone erasure methods, TCR-DIS followed by erasure, and direct TCR-ERA. Method Waterbirds (s) Bias in Bios SN (s) Dial Mention (s) Jigsaw (s) INLP 1.351.35±0.41 5.575.57±1.01 28.0528.05±1.90 3.073.07±0.65 LEACE 0.030.03±0.060.03 ±0.06 0.03 ±0.06 0.04 ±0.07 0.02 ±0.05 RLACE 38.9138.91±1.13 65.2865.28±2.56 208.60208.60±3.70 96.7796.77±6.67 SPLINCE 0.03 ±0.02 0.150.15±0.200.15 ±0.20 0.300.30±0.330.30 ±0.33 0.100.10±0.030.10 ±0.03 LEOPARD 10.6510.65±0.61 47.2847.28±0.36 132.32132.32±0.56 53.1853.18±0.21 KRaM 5.395.39±1.30 36.5836.58±4.44 59.5859.58±28.59 45.2645.26±5.67 Obliviator 1.671.67±0.14 93.1493.14±0.29 8.628.62±0.27 11.9511.95±0.32 TCR-DIS + INLP 48.1848.18±4.60 42.3642.36±1.95 277.02277.02±34.47 35.5235.52±1.55 TCR-DIS + LEACE 47.3347.33±4.36 41.0641.06±2.05 88.4688.46±16.05 32.7432.74±1.54 TCR-DIS + RLACE 84.5184.51±5.65 188.65188.65±7.93 1001.341001.34±679.06 113.47113.47±2.43 TCR-DIS + SPLINCE 47.4747.47±4.56 41.1241.12±2.05 88.8288.82±16.22 32.8132.81±1.53 TCR-DIS + LEOPARD 57.3957.39±4.37 88.7088.70±2.02 226.28226.28±20.29 86.5386.53±1.52 TCR-DIS + KRaM 51.9251.92±4.24 56.0056.00±1.99 127.32127.32±16.13 52.2552.25±4.41 TCR-DIS + Obliviator 49.0149.01±4.49 53.0153.01±1.95 118.78118.78±15.91 54.0654.06±1.43 TCR-ERA 4.054.05±0.47 10.4210.42±1.14 142.63142.63±1.55 8.248.24±0.48 Runtime. We additionally report the execution time of each intervention in Table 8. As expected, using TCR-DIS as a preprocessing step introduces an additional optimization cost before applying the corresponding erasure method. For lightweight methods such as LEACE and SPLINCE, the resulting runtime nevertheless remains below 9090 seconds on all datasets, while combining TCR-DIS with more computationally intensive methods such as RLACE can be substantially more expensive, reaching 1001.341001.34 seconds on DIAL-Mention. Direct TCR-ERA avoids this additional erasure stage and requires between 4.054.05 and 142.63142.63 seconds across the four datasets. In particular, it is faster than RLACE on every dataset and faster than LEOPARD on three out of four datasets, showing that direct erasure with TCR can remain computationally competitive with iterative erasure baselines. Test accuracy. In addition of the test WGA shown in Table 1, test accuracy is shown in Table 9. The test task accuracy provides a complementary view to WGA. On Waterbirds and Bias in Bios, the gains from TCR-DIS also translate into substantially higher overall accuracy, whereas on DIAL-Mention and Jigsaw the accuracy remains comparable or can slightly decrease despite consistent improvements in WGA. This difference highlights that aggregate accuracy can mask group-specific degradation, while WGA directly measures utility on the most affected group. Notably, TCR-ERA achieves higher task accuracy than every standalone erasure baseline on all four datasets and obtains the highest accuracy among all erasure methods on Waterbirds, DIAL-Mention and Jigsaw. I.1.1 Beyond-linear Concept Leakage Table 9: Test task accuracy with an adjusted balanced linear concept leakage <0.1<0.1. Method Waterbirds Bias in Bios SN Dial Mention Jigsaw Raw 88.95 96.76 64.91 71.56 INLP 85.1785.17±0.00 71.6671.66±0.00 63.4963.49±0.00 65.8265.82±0.00 LEACE 85.6785.67±0.00 76.5576.55±0.00 65.0665.06±0.0065.06 ±0.00 67.5867.58±0.0067.58 ±0.00 RLACE 85.6485.64±0.25 76.6276.62±0.33 64.7864.78±0.21 67.3067.30±0.17 SPLINCE 85.6685.66±0.00 76.5076.50±0.00 65.0665.06±0.0065.06 ±0.00 67.5167.51±0.00 LEOPARD 85.6685.66±0.00 76.5576.55±0.00 64.8564.85±0.02 67.5867.58±0.0067.58 ±0.00 KRaM 78.4478.44±0.77 68.2368.23±0.93 59.0059.00±0.40 62.2062.20±0.79 Obliviator 84.7084.70±0.46 75.0875.08±0.63 62.8762.87±0.61 64.3964.39±0.63 TCR-DIS + INLP 88.5788.57±1.32 87.1887.18±2.07 62.7262.72±0.81 59.3159.31±1.92 TCR-DIS + LEACE 88.4788.47±0.55 94.68 ±0.21 64.9964.99±0.28 64.9864.98±0.85 TCR-DIS + RLACE 88.9888.98±0.5688.98 ±0.56 94.5994.59±0.29 63.8863.88±0.61 65.1965.19±0.69 TCR-DIS + SPLINCE 88.6488.64±0.98 94.6094.60±0.2794.60 ±0.27 64.9964.99±0.31 64.8964.89±0.84 TCR-DIS + LEOPARD 88.9488.94±0.53 94.6094.60±0.31 64.9864.98±0.36 65.0165.01±0.79 TCR-DIS + KRaM 85.1485.14±1.82 74.4174.41±5.91 61.1961.19±0.85 62.4262.42±1.21 TCR-DIS + Obliviator 88.1388.13±1.50 89.0689.06±1.39 64.9064.90±0.56 64.7864.78±0.66 TCR-ERA 89.69 ±0.38 92.6992.69±0.30 66.44 ±0.31 67.66 ±0.35 Table 10: Adjusted balanced oracle-conditioned test concept leakage. Method Waterbirds Bias in Bios SN Dial Mention Jigsaw Raw 85.71 98.62 61.78 51.88 INLP 60.5860.58±0.00 77.5977.59±0.00 34.4734.47±0.00 29.9929.99±0.00 LEACE 72.3572.35±0.00 95.0995.09±0.00 38.1338.13±0.00 33.3733.37±0.00 RLACE 72.3772.37±0.26 94.9994.99±0.10 38.3038.30±0.20 33.5533.55±0.49 SPLINCE 72.1472.14±0.00 94.5694.56±0.00 38.2838.28±0.00 33.3133.31±0.00 LEOPARD 72.5972.59±0.00 94.9994.99±0.00 35.8435.84±0.26 33.3433.34±0.00 KRaM 44.58 ±2.70 58.73 ±2.74 23.6023.60±1.2723.60 ±1.27 22.8522.85±1.56 Obliviator 63.2663.26±0.96 86.4386.43±1.78 30.8330.83±1.10 24.2924.29±1.54 TCR-DIS + INLP 56.4156.41±1.84 89.2389.23±1.32 28.3128.31±1.58 5.49 ±2.74 TCR-DIS + LEACE 60.7660.76±3.15 95.7795.77±0.65 30.4430.44±1.17 10.2410.24±1.25 TCR-DIS + RLACE 59.5659.56±1.39 95.7995.79±0.67 28.4128.41±1.80 10.3410.34±1.37 TCR-DIS + SPLINCE 61.0161.01±1.81 95.9795.97±0.47 30.1830.18±1.41 9.969.96±1.239.96 ±1.23 TCR-DIS + LEOPARD 59.0359.03±1.36 95.7495.74±0.65 29.4029.40±1.24 10.4110.41±1.37 TCR-DIS + KRaM 53.0053.00±3.82 65.5265.52±5.2065.52 ±5.20 21.98 ±1.76 11.3511.35±2.34 TCR-DIS + Obliviator 59.6259.62±1.26 96.5396.53±0.37 28.8528.85±1.23 12.8712.87±1.78 TCR-ERA 51.9351.93±1.7851.93 ±1.78 91.8291.82±0.89 34.1034.10±1.65 24.8224.82±1.20 Table 11: Adjusted balanced non-linear test concept leakage. Method Waterbirds Bias in Bios SN Dial Mention Jigsaw Raw 79.83 98.89 55.99 42.77 INLP 71.0171.01±1.60 93.6493.64±0.21 52.9952.99±0.65 31.3731.37±0.95 LEACE 74.6074.60±0.92 98.0598.05±0.21 55.1455.14±0.48 35.9535.95±0.91 RLACE 74.5474.54±1.79 98.0198.01±0.27 54.9654.96±0.58 36.4436.44±1.11 SPLINCE 75.1475.14±1.42 97.4597.45±0.26 56.1856.18±0.45 36.9136.91±1.03 LEOPARD 74.0774.07±1.06 98.0298.02±0.18 53.4553.45±0.47 35.4435.44±1.24 KRaM 40.71 ±4.02 76.6476.64±2.2476.64 ±2.24 36.37 ±1.30 11.37 ±2.96 Obliviator 68.3668.36±0.96 93.3893.38±0.52 50.0050.00±1.29 21.4721.47±1.35 TCR-DIS + INLP 76.8476.84±1.18 98.3098.30±0.14 55.0755.07±0.74 32.6232.62±1.67 TCR-DIS + LEACE 78.6378.63±1.86 98.5998.59±0.12 56.4956.49±0.61 37.8337.83±1.90 TCR-DIS + RLACE 78.3578.35±1.46 98.5598.55±0.16 55.6455.64±0.73 38.2038.20±1.11 TCR-DIS + SPLINCE 74.2874.28±3.48 98.6098.60±0.17 57.2957.29±0.63 37.4537.45±1.53 TCR-DIS + LEOPARD 78.6478.64±1.46 98.6298.62±0.16 56.1556.15±0.80 38.1538.15±1.23 TCR-DIS + KRaM 63.4663.46±2.0563.46 ±2.05 76.44 ±3.92 37.1437.14±1.2237.14 ±1.22 15.7115.71±2.2015.71 ±2.20 TCR-DIS + Obliviator 75.0275.02±2.08 98.4898.48±0.24 55.0655.06±0.53 33.5933.59±1.66 TCR-ERA 65.7565.75±2.14 97.7597.75±0.22 53.0253.02±0.78 28.7328.73±1.24 The experiments of subsection 4.1 use an adjusted balanced linear leakage threshold measured through a linear probe trained with a convex loss. This follows the linear-guardedness perspective of prior concept-erasure work, for instance, LEACE guarantees that, after erasure, no linear predictor can improve over the optimal constant predictor under convex losses (Belrose et al., 2023). This criterion is therefore broader than a particular choice of logistic regression, but does not imply that the edited representation is statistically independent of the concept or guarded against arbitrary non-linear predictors. Indeed, direct linear-guardedness need not imply that a downstream model cannot infer the concept indirectly through other preserved information (Ravfogel et al., 2023). This distinction is particularly relevant when task and concept labels are correlated, since preserving task-predictive information may necessarily retain some predictability of the concept. To assess concept leakage beyond the criterion used for model selection, we therefore report two stronger diagnostics: oracle-conditioned leakage, which fits separate balanced linear concept probes within each task class, and non-linear leakage, measured with a non-linear concept probe. For the oracle-conditioned leakage, we fit two separate linear concept probes, one on samples with y=0y=0 and one on samples with y=1y=1. Each probe is an ℓ2 _2-regularized logistic regression trained with balanced concept-class weights within the corresponding task train set; its regularization strength is selected on the matching validation set by balanced accuracy. At evaluation time, each sample is routed to the probe corresponding to its ground-truth task label y, and leakage is reported as adjusted balanced accuracy, 2BalAcc−12\,BalAcc-1. This asks whether c remains linearly predictable once y is held fixed, rather than allowing predictability of y itself to explain concept leakage. For non-linear leakage, we fit an unweighted ReLU MLP concept probe on the edited representation. Inputs are standardized using training statistics, and the probe is optimized with cross-entropy and Adam. We select the architecture among one, two, or three hidden layers of width 128128, together with the learning rate and training epoch, according to validation accuracy, the learning rate is searched around 10−310^-3 on the 1,2,5×10k\1,2,5\× 10^k lattice. We train for at most 200200 epochs with batch size 256256. The selected probe is then evaluated on the test set using the same adjusted balanced leakage metric. The oracle-conditioned results (Table 10) show that low global linear leakage does not generally imply low leakage after conditioning on the task. Nevertheless, TCR-DIS often reduces this stronger conditional leakage relative to the corresponding erasure baseline, with particularly large reductions on Jigsaw and consistent improvements on Waterbirds and Dial Mention. Bias in Bios remains difficult, most methods retain high conditional leakage despite satisfying the linear leakage constraint. Non-linear leakage (Table 11) remains substantial for many methods and is often not reduced by TCR-DIS. This confirms that our main results should be interpreted as improvements in task preservation at matched linear concept leakage, rather than as guarantees of complete concept removal. KRaM achieve lower non-linear leakage on several datasets, but at a different utility-erasure trade-off, indeed it often has lower wga Table 1 and test accuracy Table 9 than the other methods. I.1.2 Utility Preservation Across Erasure Levels and Data Regimes (a) (b) (c) (d) (e) (f) Figure 4: Extended erasure-utility and data-regime analysis. Erasure-utility trade-off, showing test WGA against adjusted balanced linear leakage, on Waterbirds (a) and Bias in Bios (b); impact of label imbalance on DIAL-Mention (c) and Jigsaw (d); and of spurious correlation on DIAL-Mention (e) and Jigsaw (f). Panels (a), (c) and (f) correspond to the main-text results for direct comparison. Figure 4complements the analyses of Figure 2 with the corresponding experiments on other dataset and data-regime combinations. Panels (a), (c) and (f) reproduce the main-text results for direct comparison; we discuss the complementary results below. On Bias in Bios (Figure 4b), TCR exhibits a particularly favorable trade-off under strong erasure, reaching 86.5286.52 WGA at only 1.88%1.88\% adjusted balanced linear leakage, compared with 59.5359.53 for the best baseline under the strong-erasure constraint. Its WGA then remains nearly constant as the constraint is relaxed, reaching 84.3184.31 at 60.52%60.52\% leakage. In contrast, several baselines recover task utility primarily by retaining substantially more concept information; for instance, INLP approaches TCR only at approximately 46%46\% leakage (84.5284.52 versus 85.5085.50 WGA). The Jigsaw label-imbalance experiment (Figure 4d) shows a similar robustness advantage. TCR achieves the highest WGA at every imbalance level, with its margin over the best baseline increasing from 1.731.73 points at P(y=0)=0.50P(y=0)=0.50 to 18.0418.04 at 0.550.55 and 26.2926.29 at 0.650.65. Even at P(y=0)=0.75P(y=0)=0.75, TCR retains 40.4240.42 WGA compared with 22.0322.03 for the strongest baseline, indicating substantially better utility preservation as the task-label distribution becomes increasingly imbalanced. Finally, on DIAL-Mention (Figure 4e), TCR is essentially tied with the strongest baseline when task and concept are independent at P(y=c)=0.50P(y=c)=0.50 (68.1868.18 versus 68.1768.17 WGA), but obtains the highest WGA at every stronger spurious correlation. Its advantage ranges from 2.922.92 to 4.234.23 points for P(y=c)∈[0.6,0.9]P(y=c)∈[0.6,0.9], complementing the analogous trend observed on Jigsaw in the main text. Together, these additional experiments support the same behavior observed in the main analysis: TCR remains consistently competitive with the strongest baselines across the explored settings, while its utility-preservation advantage becomes most pronounced when strong erasure, label imbalance or spurious dependence make task and concept information harder to separate. I.1.3 Representation Similarity After Intervention (a) (b) (c) (d) Figure 5: Task utility against representational similarity after concept erasure, showing test WGA against linear CKA between the original and intervened test representations on Waterbirds (a), Bias in Bios (b), DIAL-Mention (c) and Jigsaw (d). Higher CKA indicates greater similarity. Beyond task utility and concept leakage, we evaluate how strongly each intervention modifies the representation globally. This is important because improved task preservation after erasure could otherwise result from extensively reorganizing the representation around the specific task–concept pair. We quantify this effect using linear centered kernel alignment (CKA) (Kornblith et al., 2019), a standard measure for comparing neural representations that assesses their similarity. For a test representation H∈ℝN×dH ^N× d before intervention and the corresponding representation Z∈ℝN×d′Z ^N× d after intervention, let H′H and Z′Z denote the matrices obtained by centering each feature over the same test samples. We compute linear CKA as: CKA(H,Z)=‖Z′⊤H′‖F2‖H′⊤H′‖F‖Z′⊤Z′‖F.CKA(H,Z)= \|Z H \|_F^2\|H H \|_F\,\|Z Z \|_F. Linear CKA is invariant to isotropic scaling and orthogonal transformations of either representation, and therefore does not penalize such changes of coordinates, while remaining sensitive to more general changes in representational geometry (Kornblith et al., 2019). Values closer to 11 indicate greater global similarity between the original and edited representations. Unlike a learned reconstruction or alignment score, CKA requires no fitted mapping; we therefore compute it directly between the original and final test representations. For composed TCR-DIS methods, Z denotes the representation after both disentanglement and the subsequent erasure method. We compute CKA independently for each run and report its mean across seeds as a percentage. Figure 5reports test WGA against the linear CKA between the original and intervened representations. The results first show that preserving global representation geometry and preserving task utility are distinct objectives: high CKA does not necessarily translate into high WGA. On Waterbirds, INLP, LEACE, RLACE and LEOPARD retain between 85.40%85.40\% and 86.22%86.22\% CKA while reaching only 13.7113.71–27.8827.88 WGA. Similarly, on Jigsaw, these methods retain approximately 99%99\% CKA but obtain only 52.5952.59–55.6455.64 WGA. Thus, concept erasure can leave most of the global representational structure unchanged while substantially degrading the task-relevant information required by the worst-performing groups. Conversely, TCR-DIS often induces a larger representational reorganization while preserving substantially more task utility. This is particularly pronounced on Waterbirds, where TCR-DIS variants reach 66.0166.01–74.6674.66 WGA despite lower CKA, and on Bias in Bios, where the variants based on INLP, LEACE, RLACE, SPLINCE and LEOPARD reach approximately 8181–8282 WGA. The same pattern is milder on Jigsaw: combining TCR-DIS with LEACE, RLACE or LEOPARD reduces CKA from approximately 99%99\% to 8989–90%90\%, while increasing WGA from 52.5952.59–53.2753.27 to 60.5860.58–61.0561.05. Hence, the utility gains of disentanglement do not arise from keeping the edited representation as close as possible to its original geometry; modifying this geometry can instead facilitate the separation of task-relevant information from the concept being erased. The amount of representational change also depends strongly on the form of the intervention. Across datasets, the non-linear baselines KRaM and Obliviator tend to exhibit among the lowest CKA values, whereas the linear projection-based INLP, LEACE and RLACE generally remain substantially closer to the original representation. On Waterbirds and Jigsaw, for instance, the latter retain approximately 8585–99%99\% CKA despite substantially lower WGA. At the same time, TCR does not systematically decrease CKA: TCR-DIS increases both CKA and WGA over Obliviator on Bias in Bios (19.71%19.71\% to 42.83%42.83\% CKA and 58.1858.18 to 83.5083.50 WGA) and Jigsaw (17.06%17.06\% to 42.79%42.79\% CKA and 52.5652.56 to 60.0160.01 WGA), with similar behavior for KRaM on Waterbirds. Direct TCR-ERA also retains substantial similarity to the original representation, with CKA ranging from 62.18%62.18\% on Waterbirds to 81.22%81.22\% on Jigsaw, while achieving strong WGA across the four datasets. Overall, the degree of geometric reorganization appears to depend on both the intervention and the subsequent erasure operator rather than being systematically larger for TCR. Finally, CKA measures global representational similarity rather than preservation of all information in the original representation. A lower CKA therefore indicates a change in representation geometry, but does not by itself imply that unrelated information has been discarded. We consequently interpret these results as showing that improvements from TCR can involve meaningful representational reorganization, while task utility can remain preserved despite lower global similarity. Appendix J Limitations TCR depends on the concepts used to construct it and on the probes and interaction criterion used to characterize them. Different choices may therefore induce different concept organizations, and concepts absent from the chosen set are not represented. In particular, our interference criterion is operational: it measures how erasing one concept affects the recoverability of another and should not be interpreted as establishing a causal dependence between them. The persistence reduction is intentionally lossy. It retains the multiscale connectivity induced by pairwise interactions while discarding variations of the concept mesh that do not alter this hierarchy. We do not claim that topology is the unique useful abstraction of concept organization; alternative reductions or the full concept mesh may preserve complementary information. Moreover, our current construction relies on pairwise interactions and 00-dimensional topology and therefore does not explicitly represent higher-order relationships between concepts. Although the persistence diagram contains only nC−1n_C-1 hierarchy-defining saddles, constructing the complete concept mesh requires quadratically many pairwise interaction evaluations, which may become costly for very large concept sets. Finally, our experiments focus primarily on penultimate representations and mostly linear concept probes. Nonlinear concept information may remain after linear interventions, and the estimated concept organization may also change under distribution shift. Extending TCR to larger concept collections, multiple network layers and richer interaction models provides directions for future work.