Paper deep dive

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)

Usha Bhalla, Alex Oesterling, Suraj Srinivas, Flavio P. Calmon, Himabindu Lakkaraju

Year: 2024Venue: NeurIPS 2024Area: Representation AnalysisType: EmpiricalEmbeddings: 106

Models: CLIP (OpenAI)

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%

Last extracted: 3/12/2026, 7:40:38 PM

Summary

The paper introduces Sparse Linear Concept Embeddings (SpLiCE), a task-agnostic, training-free method to decompose dense CLIP embeddings into sparse, human-interpretable semantic concepts. By leveraging the latent space structure of CLIP, SpLiCE enables model transparency, spurious correlation detection, and model editing without requiring labeled concept datasets.

Entities (4)

CLIP · model · 100%SpLiCE · method · 100%Linear Representation Hypothesis · theory · 95%CIFAR100 · dataset · 90%

Relation Signals (3)

SpLiCE → decomposes → CLIP

confidence 95% · SpLiCE is task-agnostic and can be used, without training, to explain and even replace traditional dense CLIP representations

SpLiCE → detects → Spurious Correlations

confidence 90% · We also demonstrate significant use cases of SpLiCE representations including detecting spurious correlations

CLIP → follows → Linear Representation Hypothesis

confidence 85% · Sanity checking the linearity of CLIP Embeddings... providing evidence for the third assumption, the linearity of CLIP

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:CLIP embeddings have demonstrated remarkable performance across a wide range of multimodal applications. However, these high-dimensional, dense vector representations are not easily interpretable, limiting our understanding of the rich structure of CLIP and its use in downstream applications that require transparency. In this work, we show that the semantic structure of CLIP's latent space can be leveraged to provide interpretability, allowing for the decomposition of representations into semantic concepts. We formulate this problem as one of sparse recovery and propose a novel method, Sparse Linear Concept Embeddings, for transforming CLIP representations into sparse linear combinations of human-interpretable concepts. Distinct from previous work, SpLiCE is task-agnostic and can be used, without training, to explain and even replace traditional dense CLIP representations, maintaining high downstream performance while significantly improving their interpretability. We also demonstrate significant use cases of SpLiCE representations including detecting spurious correlations and model editing.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

105,471 characters extracted from source content.

Expand or collapse full text

Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE ) Usha Bhalla∗ Harvard University a,b &Alex Oesterling∗ Harvard University b &Suraj Srinivas Harvard University b Flavio P. Calmon22footnotemark: 2 Harvard University b &Himabindu Lakkaraju22footnotemark: 2 Harvard University b,c Abstract CLIP embeddings have demonstrated remarkable performance across a wide range of multimodal applications. However, these high-dimensional, dense vector representations are not easily interpretable, limiting our understanding of the rich structure of CLIP and its use in downstream applications that require transparency. In this work, we show that the semantic structure of CLIP’s latent space can be leveraged to provide interpretability, allowing for the decomposition of representations into semantic concepts. We formulate this problem as one of sparse recovery and propose a novel method, Sparse Linear Concept Embeddings (SpLiCE ), for transforming CLIP representations into sparse linear combinations of human-interpretable concepts. Distinct from previous work, SpLiCE is task-agnostic and can be used, without training, to explain and even replace traditional dense CLIP representations, maintaining high downstream performance while significantly improving their interpretability. We also demonstrate significant use cases of SpLiCE representations including detecting spurious correlations and model editing. Code is provided at https://github.com/AI4LIFE-GROUP/SpLiCE. †footnotetext: ∗ Equal contribution, order by coin flip. † Equal contribution, alphabetic order.†footnotetext: a Kempner Institute for the Study of Natural & Artificial Intelligence†footnotetext: b School of Engineering and Applied Sciences c Harvard Business School 1 Introduction Natural images include complex semantic information, such as the objects they contain, the scenes they depict, the actions being performed, and the relationships between them. Machine learning models trained on visual data aim to encode this semantic information in their representations to perform a wide variety of downstream tasks, such as object classification, scene recognition, segmentation, or action prediction. However, it is often difficult to enforce explicit encoding of these semantics within model representations, and it is even harder to interpret these semantics post hoc to better understand what models may have learnt and how they leverage this information. Further, model representations can be brittle, encoding idiosyncratic patterns specific to individual datasets and modalities instead of general human-interpretable semantic information. Multimodal models have been proposed as a potential solution to this issue, and methods such as CLIP [1] have empirically been found to provide highly performant, semantically rich representations of image data. The richness of these representations is evident from their high performance on a variety of tasks, such as zero-shot classification and image retrieval [1], image captioning [2], and image generation [3]. However, despite their performance, it remains unclear how to quantify the semantic content contained in their dense representations. In this work, we answer the question: can we decompose CLIP embeddings into human-interpretable representations of the semantic concepts they encode? This can provide insight into the types of tasks CLIP can solve, the biases it may contain, and the manner through which downstream predictions are made. Existing literature in areas such as concept bottleneck models [4], disentangled representation learning [5], and mechanistic intepretability [6] have proposed various approaches to understanding the semantics encoded by representations. However, these methods generally require predefined sets of concepts [7], data with concept labels [8], or rely on qualitative visualizations, which can be unreliable [9]. Similar to these lines of work, we aim to recover representations that reflect the underlying semantics of the inputs. However, distinct from these works, we propose to do this in a task-agnostic manner and without concept datasets, training, or qualitative analysis of visualizations. Our method, SpLiCE , leverages the highly structured and multimodal nature of CLIP embeddings for interpretability, and decomposes CLIP representations via a semantic basis to yield a sparse, human-interpretable representation. Remarkably, these interpretable SpLiCE embeddings have favorable accuracy-interpretability tradeoffs when compared to black-box CLIP representations on metrics such as zero-shot accuracy. Our overall contributions are: Figure 1: Visualization of SpLiCE , which converts dense, uninterpretable CLIP representations (z) into sparse semantic decompositions (w) by solving for a sparse nonnegative linear combination over an overcomplete concept set (C). • In Sections 3 and 4, we formalize the sufficient conditions under sparse decomposition of CLIP is feasible, and introduce SpLiCE , a novel method that decomposes dense CLIP embeddings into sparse, human-interpretable concept embeddings. • Our extensive experiments in Section 5 reveal that SpLiCE recovers highly sparse111we recommend and use sparsity levels of ∼similar-to ∼ 10-30 in practice, interpretable representations with high performance on downstream tasks, while accurately capturing the semantics of the underlying inputs. • In Section 6, we present two case studies for applying SpLiCE : spurious correlation detection, and model editing. Using SpLiCE , we uncover a spurious correlation in the CIFAR100 dataset, where we find the "woman" concept and the "swimwear" concept to be correlated owing to the prevalence of women in swimwear in CIFAR100. 2 Related Work Linear Representation Hypothesis. In language modeling, the linear representation hypothesis suggests that many semantic concepts are approximately linear functions of model representations [10, 11, 12, 13, 14]. Recent work has also shown that multimodal models encode concepts additively, behaving like bags-of-words representations [15]. Relatedly, [16, 17] show that there exists a linear mapping between image and text embeddings in arbitrary models. Our work makes use of these distinct but related observations to convert dense CLIP representations to sparse semantic ones. Figure 2: Example images from MSCOCO shown with their captions below and their concept decompositions on the right. We display the top seven concepts for visualization purposes, but images in the figure had decompositions with 7-20 concepts. Concept Bottlenecks and Attribute Learning. Concept Bottleneck Models (CBMs) [18], and attribute-based models [19, 20, 21] learn intermediate representations of scores over concepts or image attributes for use with a final linear classification head, creating interpretable concept representations. However, these require expert-labeled concept or attribute datasets to train, which is expensive. Recent work on concept-bottlenecks for multimodal models avoids needing such labeled datasets, but still requires concept labels for specific tasks, which is obtained by querying large language models (LLMs) [22, 23, 24], making these methods task-specific and heavily reliant on the domain knowledge and subject to the biases of LLMs. On the other hand, SpLiCE uses a large-scale and overcomplete concept dictionary, avoiding dependence on training, specific domain knowledge, or a downstream task. Consequently, it can even be applied to understand unstructured, unsupervised image datasets in a label-free manner. Mechanistic Interpretability and Disentanglement. Mechanistic interpretability explains representations through model activations, by labeling circuits and neurons in networks with feature visualization [6, 25] or by measuring concept activations and directions in latent space [26, 27, 7, 28, 29, 30]. Recent work [31] combines these methods, using dictionary learning to extract visual concept activations, whose semantics can be identified via feature visualization. Work in disentangled representation learning has developed architectures that capture independent factors of variation in data [8, 32, 33, 5, 34, 35], allowing for manual probing of disentangled representations for human-interpretable concepts. In both mechanistic interpretability and disentangled representation learning, methods typically rely on labeled concept sets, manual labeling of visualizations, or computationally intensive searches over data and latent representations or neurons to identify concepts. However, associating human-interpretable semantics with arbitrary neurons or latent directions is challenging, leading to the unreliability [9, 36] exhibited by such methods. Our approach side-steps this issue by decomposing CLIP representations into a predetermined set of concepts. CLIP Interpretability. Many recent works leverage the semantic structure of CLIP and its text encoder to interpret representations. For example, [37], [38], and [39] construct concept similarity scores of image embeddings for use by downstream CBMs or probes, but these representations are not interpretable due to their lack of sparsity and the presence of negative concepts. chen2023stair create a custom vision-language architecture with a sparse latent dictionary, but it requires training from scratch and cannot be used post-hoc to explain existing models. Gandelsman et al. [40] also leverage the text encoder of CLIP to explain components of the image embedding, but are limited to ViT architectures and take a mechanistic interpretability-style approach requiring a labeled text dataset. Chattopadhyay et al. [22] build concept bottlenecks for specific classification tasks by expressing CLIP image representations as a sparse linear combination of task-specific concept vectors. However, their decomposition includes negative concepts, reducing interpretability, and uses task-specific concept dictionaries. Grootendorst [41] generate textual topics of datasets through multimodal topic modeling, which cannot provide explanations of individual representations. Distinct from these works, SpLiCE is more interpretable due to its sparsity, overcompleteness, and non-negativity, and is task-agnostic, aiming to serve as a drop-in replacement for black-box CLIP representations without requiring training. 3 When do Sparse Decompositions Exist? In this section, we aim to answer the question: under what conditions can CLIP representations be decomposed into sparse semantic representations? To do so, we must reason about both the properties of CLIP as well as the properties of the underlying data. Notation. Let img∈ℝdisuperscriptimgsuperscriptℝsubscriptx^img ^d_ixroman_img ∈ blackboard_Rditalic_i, txt∈ℝdtsuperscripttxtsuperscriptℝsubscriptx^txt ^d_txroman_txt ∈ blackboard_Rditalic_t be image and text data, respectively. Given the CLIP image encoder f:ℝdi→ℝd:→superscriptℝsubscriptsuperscriptℝf:R^d_i ^df : blackboard_Rditalic_i → blackboard_Rd and text encoder g:ℝdt→ℝd:→superscriptℝsubscriptsuperscriptℝg:R^d_t ^dg : blackboard_Rditalic_t → blackboard_Rd, we define CLIP representations in ℝdsuperscriptℝR^dblackboard_Rd as img=f⁢(img)superscriptimgsuperscriptimgz^img=f(x^img)zroman_img = f ( xroman_img ) and txt=g⁢(txt)superscripttxtsuperscripttxtz^txt=g(x^txt)zroman_txt = g ( xroman_txt ). Our method uses dictionary learning to approximate imgsuperscriptimgz^imgzroman_img with a concept decomposition ∗∈ℝ+csuperscriptsuperscriptsubscriptℝw^* _+^cw∗ ∈ blackboard_R+c over a fixed concept vocabulary ∈ℝd×csuperscriptℝC ^d× cC ∈ blackboard_Rd × c. We define the resulting reconstruction of imgsuperscriptimgz^imgzroman_img from CC and ∗superscriptw^*w∗ as ^imgsuperscript^img z^imgover start_ARG z end_ARGimg. The goal of our method is to approximate f⁢(img)≈∗superscriptimgsuperscriptf(x^img) ^*f ( ximg ) ≈ Cw∗, such that ∗superscriptw^*w∗ is non-negative and sparse, and in this section we formalize when this is possible. We begin by considering a data-generating process for coupled image and text samples. Specifically, we model the generative process parameterized by a k-dimensional latent concept vector ω∈ℝ+ksuperscriptsubscriptℝω _+^kω ∈ blackboard_R+k and a random noise vector ϵ∈ℝlitalic-ϵsuperscriptℝε ^lϵ ∈ blackboard_Rl as img=himg⁢(ω,ϵ),txt=htxt⁢(ω,ϵ),ω∼ρ,ϵ∼ϕ,formulae-sequencesuperscriptimgsuperscriptℎimgitalic-ϵformulae-sequencesuperscripttxtsuperscriptℎtxtitalic-ϵformulae-sequencesimilar-tosimilar-toitalic-ϵitalic-ϕ ^img=h^img(ω,ε),~~~% ~x^txt=h^txt(ω,ε),~~~ω% ρ,~~~ε φ,ximg = himg ( ω , ϵ ) , xtxt = htxt ( ω , ϵ ) , ω ∼ ρ , ϵ ∼ ϕ , where ρ is a prior distribution over semantic concepts, ϕitalic-ϕφϕ is a prior distribution over nonsemantic concepts (such as camera orientation and lighting for images or arbitrary choices between synonyms for text), and himg:ℝk+l→ℝdi:superscriptℎimg→superscriptℝsuperscriptℝsubscripth^img:R^k+l ^d_ihimg : blackboard_Rk + l → blackboard_Rditalic_i, and htxt:ℝk+l→ℝdt:superscriptℎtxt→superscriptℝsuperscriptℝsubscripth^txt:R^k+l ^d_thtxt : blackboard_Rk + l → blackboard_Rditalic_t represent the real-world data-generating process from latent variables (ω,ϵ)italic-ϵ(ω,ε)( ω , ϵ ) to images and text respectively. Here, each coordinate ωi∈ℝ+subscriptsubscriptℝ _i _+ωitalic_i ∈ blackboard_R+ encodes the degree of prevalence of the ithsuperscriptthi^thith concept in the underlying data. We now list a set of sufficient conditions for our data-generating process and CLIP that admit a sparse decomposition of images into concepts. Sufficient Conditions for Sparse Decomposition. 1. Images and text are sparse in concept space: for some α≪kmuch-less-thanα kα ≪ k, we have ‖ω‖0≤α,∀ω∼ρformulae-sequencesubscriptnorm0similar-tofor-all\|ω\|_0≤α,∀~ω ρ∥ ω ∥0 ≤ α , ∀ ω ∼ ρ. 2. CLIP captures semantic concepts ω and not ϵitalic-ϵεϵ: ∀ϵ,ϵ′,f∘himg⁢(ω,ϵ)=f∘himg⁢(ω,ϵ′)for-allitalic-ϵsuperscriptitalic-ϵ′ℎimgitalic-ϵsuperscriptℎimgsuperscriptitalic-ϵ′∀ε,ε ,f h^img(ω,ε)=f% h^img(ω,ε )∀ ϵ , ϵ′ , f ∘ himg ( ω , ϵ ) = f ∘ himg ( ω , ϵ′ ) and similarly for htxt.superscriptℎtxth^txt.htxt . 3. CLIP is linear in concept space: g∘htxtsuperscriptℎtxtg h^txtg ∘ htxt and f∘himgsuperscriptℎimgf h^imgf ∘ himg are linear in ω. 4. CLIP image and text encoders are aligned: for a given ω, f∘himg⁢(ω,ϵ)=g∘htxt⁢(ω,ϵ)superscriptℎimgitalic-ϵsuperscriptℎtxtitalic-ϵf h^img(ω,ε)=g h^txt(ω,ε)f ∘ himg ( ω , ϵ ) = g ∘ htxt ( ω , ϵ ). We emphasize that the goal of enumerating a set of sufficient conditions for sparse decomposition is not to claim that these exactly hold in practice, but rather to reason about when sparse decompositions–as done in this work–are appropriate. In the Appendix (Section A.1, Prop. 1) we formalize and prove this claim, but in the interest of simplicity we keep the discussion here informal. We note that many of these are natural; Assumption 1 reflects how real-world images and text are simple and rarely contain complex semantic content, and the CLIP training process optimizes for Assumption 2 and 4222In practice we find that CLIP’s image and text encoders are not fully aligned, so we apply a preprocessing step (Sec 4.1).. Of these, the most critical one is Assumption 3, which closely relates to the linear representation hypothesis [11], which we investigate below. Sanity Checking CLIP’s Linearity. Table 1: Sanity checking the linearity of CLIP Embeddings. wasubscriptw_awitalic_a wbsubscriptw_bwitalic_b cosine(z^,z z,zover start_ARG z end_ARG , z) ImageNet 0.48 ±plus-or-minus± 0.09 0.45 ±plus-or-minus± 0.09 0.76 ±plus-or-minus± 0.05 CIFAR100 0.45 ±plus-or-minus± 0.08 0.42 ±plus-or-minus± 0.08 0.75 ±plus-or-minus± 0.03 MIT States 0.48 ±plus-or-minus± 0.09 0.45 ±plus-or-minus± 0.09 0.76 ±plus-or-minus± 0.05 COCO Text 0.59 ±plus-or-minus± 0.12 0.47 ±plus-or-minus± 0.12 0.88 ±plus-or-minus± 0.04 We provide evidence for the third assumption, the linearity of CLIP, in a toy setting. We begin by asking the following question to confirm the general linearity of CLIP embeddings: “if two inputs are concatenated, does their joint embedding equal the average of their two individual embeddings?". For the image domain, we combine two images, xa,xbsubscriptsubscriptx_a,x_bxitalic_a , xitalic_b, to form their composition xa⁢bsubscriptx_abxitalic_a b by placing xasubscriptx_axitalic_a in the top left quarter and xbsubscriptx_bxitalic_b in the bottom right quarter of a blank image. For the text domain, we simply append text xbsubscriptx_bxitalic_b to text xasubscriptx_axitalic_a to form xa⁢bsubscriptx_abxitalic_a b. We then embed xa,xb,xa⁢bsubscriptsubscriptsubscriptx_a,x_b,x_abxitalic_a , xitalic_b , xitalic_a b with CLIP to get za,zb,za⁢bsubscriptsubscriptsubscriptz_a,z_b,z_abzitalic_a , zitalic_b , zitalic_a b. Solving the equation wa∗za+wb∗zb=za⁢bsubscriptsubscriptsubscriptsubscriptsubscriptw_a*z_a+w_b*z_b=z_abwitalic_a ∗ zitalic_a + witalic_b ∗ zitalic_b = zitalic_a b for scalar weights wa,wbsubscriptsubscriptw_a,w_bwitalic_a , witalic_b then allows us to assess the linearity of za,zb,za⁢bsubscriptsubscriptsubscriptz_a,z_b,z_abzitalic_a , zitalic_b , zitalic_a b. We report wa,wbsubscriptsubscriptw_a,w_bwitalic_a , witalic_b and the cosine similarity between z^a⁢b=[za,zb]⋅[wa,wb]subscript^⋅subscriptsubscriptsubscriptsubscript z_ab=[z_a,z_b]·[w_a,w_b]over start_ARG z end_ARGa b = [ zitalic_a , zitalic_b ] ⋅ [ witalic_a , witalic_b ] and za⁢bsubscriptz_abzitalic_a b in Table 1. In general, we find that the composition of two inputs results in an embedding that is approximately equal to the average of the two input components, with wa,wbsubscriptsubscriptw_a,w_bwitalic_a , witalic_b being very close to 0.5 across all datasets and for both modalities, providing preliminary evidence for the linearity of CLIP embeddings for both image and language. 4 Method In this section, we introduce SpLiCE , a method for expressing CLIP’s image representations as sparse, nonnegative, linear combinations of concept dictionary elements. We begin by framing this problem as one of sparse recovery. We then discuss our design choices, including how we choose the concept dictionary and how to address the modality gap between CLIP’s images and text representations. Finally, we formalize the optimization problem used in this work. 4.1 Sparse Nonnegative Concept Decomposition Our goal is to construct decompositions of dense CLIP representations that are human-interpretable, useful, and faithful. To do so, we formulate decomposition as a sparse recovery problem with three main desiderata. First, for the decompositions to be interpretable to humans they must be comprised of human interpretable atoms. We argue that language is a naturally interpretable interface for humans, and construct our concept vocabulary CC out of 1- and 2-word atoms, such as “coffee”, “silver”, and “birthday party”. Second, our decompositions must be simple and concise, which can be formulated as a sparsity constraint on the recovery. A large body of work in computational linguistics [42, 43, 44, 14], neuroscience [45, 46], and interpretability [47, 48, 30] have demonstrated that a human-aligned semantic model should be sparse in representation. Furthermore, [47] found that users can best understand explanations with fewer than 32 concepts while in linguistics, [49, 50, 51] find participants describe concepts and objects with up to 20 semantic properties, motivating our desiderata of sparsity. Third, our decompositions must be constructive, i.e., we must decompose representations in terms of their constituent concepts. For this reason, we require the weights of decompositions to be strictly nonnegative, to avoid having “negative” concept weights which do not always carry semantic meaning. Furthermore, prior work by Zhou et al. [30] has argued that “negations of concepts are not as interpretable as positive concepts.” More specifically, while a small set of concepts have well-defined antonyms which may be viewed as their negative counterparts (‘day’ ↔ ↔ ‘night’), negative concepts do not carry semantic meaning in general (‘tiger’ ↔ ↔ ??). Furthermore, we find that even when antonyms exist, they are not negatives of each other in CLIP latent space (see Appendix B.10). To avoid dependence on negative weights and ensure that all concepts are captured, we construct an overcomplete dictionary containing a wide range of concepts, including antonyms. We build on top of this literature and provide a semantic decomposition satisfying these properties suitable for multimodal models like CLIP. Figure 3: Performance of SpLiCE decomposition representations on zero-shot classification tasks (bottom row) and cosine similarity between CLIP embeddings and SpLiCE embeddings (top row). Our proposed semantic dictionary (yellow) closely approximates CLIP on zero-shot classification accuracy, but not on the cosine similarity. This indicates that SpLiCE captures the semantic information in CLIP, but not its non-semantic components, explaining both the high zero-shot accuracy and low cosine similarity. See §5.2 for discussion. Concept Vocabulary. Natural language is an intuitive, interpretable, and compact medium for communicating semantic information. Thus, we choose to represent the semantic content contained in CLIP embeddings as combinations of natural language semantic concepts, where we define concepts as semantic units that can be expressed concisely, by one- or two-word phrases. Given that CLIP is used in a wide variety of downstream applications and is trained without a specific task in mind, we want our concept dictionary to be task-agnostic and to span all possible concepts CLIP could have learnt. To construct this vocabulary, we consider the most frequent one- and two-word bigrams in the text captions of the LAION-400m dataset [52], the dataset that most CLIP variants are trained on. We filter the captions to remove any NSFW samples and prune our concept set such that no two concept embeddings have a cosine similarity greater than 0.9. We also remove bigrams highly similar (>0.9absent0.9>0.9> 0.9 cosine similarity) to the average of their individual words. We finally choose the top 10,0001000010,00010 , 000 most common single-word concepts and the top 5000500050005000 most common two-word concepts as our concept vocabulary. We note that this vocabulary offers distinct advantages over those used in prior works. In particular, it is task-agnostic, meaning that the efficacy of the decomposition is (in principle) independent of individual datasets. Furthermore, this dataset imposes minimal priors from outside curators, such as human experts or LLMs [22, 23, 24]. This allows us to interpret data through the lens of CLIP, to understand the information encoded, including potential biases and mistakes. Modality Alignment. In order to decompose images into nonnegative combinations of text concepts, we must ensure that our concept set spans the space of possible image embeddings. However, [53] show the existence of a modality gap in CLIP, where image and text embeddings can lie in non-identical spaces on the unit sphere. We empirically find that CLIP image and text embeddings exist on two cones, as the distribution of pairwise cosine similarities between pairs of MSCOCO images and pairs of MSCOCO text captions concentrate at positive values, whereas the distribution of pairwise cosine similarities across modalities concentrates closer to zero. (See Appendix Fig. 7). Not only does this prevent nonnegative decomposition, it also violates Assumption 4 from Section 3. To rectify this, we mean-center CLIP images with the image cone mean, estimated over MSCOCO (μsubscript _imgμbold_img), and compute decompositions over the mean-centered concept vocabulary (μsubscript _conμbold_con). Note that the embeddings need to be re-normalized after centering to ensure they lie on the unit sphere. To convert our decompositions back into dense representations (^imgsuperscript^img z^imgover start_ARG z end_ARGimg), we uncenter the normalized dense embeddings ^imgsuperscript^img z^imgover start_ARG z end_ARGimg by adding the image mean back in and normalizing once again, to ensure they lie on the same cone as the original CLIP embeddings (imgsuperscriptimgz^imgzimg). Optimization Problem. Our optimization problem is formulated as follows. Let σ⁢()=/‖2subscriptnorm2σ(x)=x/\|x\|_2σ ( x ) = x / ∥ x ∥2 be the normalization operation. Given a set of semantic concepts con=superscriptconabsentx con=~xcon =[“dog”, “tabby cat”, “cloudy”, ⋯·s⋯ ], we construct a centered vocabulary =[σ⁢(g⁢(1con)−μcon),⋯,σ⁢(g⁢(ccon)−μcon)]subscriptsuperscriptcon1subscriptcon⋯subscriptsuperscriptconsubscriptconC= [σ(g(x con_1)- _ % con),·s,σ(g(x con_c)- _ % con) ]C = [ σ ( g ( xcon1 ) - μcon ) , ⋯ , σ ( g ( xconitalic_c ) - μcon ) ], where we recall that g⁢(⋅)⋅g(·)g ( ⋅ ) is the CLIP text encoder. Now, given the dictionary CC and a centered CLIP embedding =σ⁢(img−μimg)superscriptimgsubscriptimgz=σ(z img- _ img)z = σ ( zimg - μimg ), we seek to find the sparsest solution that gives us a cosine similarity score of at least 1−ϵ1italic-ϵ1- 1 - ϵ for some small ϵitalic-ϵεϵ: min∈ℝ+c⁡‖0⁢s.t.⁢⟨,σ⁢()⟩≥1−ϵ.subscriptsuperscriptsubscriptℝsubscriptnorm0s.t.1italic-ϵ _w _+^c\|w\|_0~~% s.t.~~ ,σ(Cw) ≥ 1-ε.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ∥ w ∥0 s.t. ⟨ z , σ ( Cw ) ⟩ ≥ 1 - ϵ . (1) As is standard practice, we relax the ℓ0subscriptℓ0 _0ℓ0 constraint and reformulate this as a minimization of MSE with an ℓ1subscriptℓ1 _1ℓ1 penalty, to construct the following convex relaxation333For more discussion on the relationship between Eq. (1) and Eq. (2), see Appendix, Sec. A.2 of Eq. (1): min∈ℝ+c⁡‖−‖22+2⁢λ⁢‖1.subscriptsuperscriptsubscriptℝsuperscriptsubscriptnorm222subscriptnorm1 _w _+^c\|Cw-% z\|_2^2+2λ\|w\|_1.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ∥ Cw - z ∥22 + 2 λ ∥ w ∥1 . (2) Given the solution to the above problem ∗superscriptw^*w∗, our reconstructed embedding is: ^img=σ⁢(∗+μimg).superscript^imgsuperscriptsubscriptimg z img=σ(Cw^*+ _% img).over start_ARG z end_ARGimg = σ ( Cw∗ + μimg ) . 5 Experiments In this section, we evaluate our method to ensure that SpLiCE decompositions are interpretable, performant, and accurately reflect the semantic content of representations. 5.1 Setup Models. All experiments shown in the main paper are done with the OpenCLIP ViT-B/32 model [54] with results for an additional model in Appendix B.14. For all zero-shot classification tasks, we use the prompt template “A photo of a ”. Datasets. We use CIFAR100 [55], MIT States [56], CelebA [57], MSCOCO [58], and ImageNetVal [59] for our experiments with results for additional datasets in the Appendix (Section B.4) Decomposition. For all experiments involving concept decomposition, we use sklearn’s [60] Lasso solver with a non-negativity flag and an l1subscript1l_1l1 penalty that results in solutions with l0subscript0l_0l0 norms of 5-20 (around 0.2-0.3 for most datasets).We use a concept vocabulary chosen from a subset of LAION tokens as described in Section 3. Both image embeddings and dictionary concepts are centered and normalized as mentioned in Section 4.1, with the image mean used for centering computed over the MSCOCO train set and the concept mean computed over our chosen vocabulary. 5.2 Sparsity-Performance Tradeoffs We assess the performance of SpLiCE decompositions by evaluating the reconstruction error in terms of cosine similarity between SpLiCE representations and CLIP embeddings, the zero-shot performance of SpLiCE decompositions, and the retrieval performance of SpLiCE embeddings. We compare the performance of decompositions generated from our semantic concept vocabulary to decompositions over random vocabulary and learned dictionary vocabulary baselines. All vocabularies are of size 15,000 concepts. The random vocabulary is sampled from a 512-dimensional normalized Gaussian distribution. The learned vocabularies are generated by using the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA) [61] to learn optimal dictionaries given our sparse recovery problem (optimizing Equation (1) for both CC and w). Note that we learn separate dictionaries imgsubscriptimgC_imgCimg and textsubscripttextC_textCtext to reconstruct MSCOCO image and text embeddings respectively. In Figure 3, we plot the cosine reconstruction and zero-shot accuracy of image decompositions with the various dictionaries. We evaluate probing performance (Tables 3, 4) and text-to-image and image-to-text retrieval in the Appendix (Figure B.3). These results overall show SpLiCE efficiently navigates the interpretability-accuracy Pareto frontier and retains much of the performance of black-box CLIP representations with the semantic, human-interpretable LAION dictionary, significantly outperforming other dictionaries on semantic tasks such as zero-shot classification, probing, and retrieval. At the same time, we find that our semantic LAION dictionary does not result in accurate cosine similarity reconstructions of the original CLIP, often being on par with using random dictionaries. We believe this is because CLIP encodes both semantics of the underlying image and non-semantic "noise", which violates Assumption #2 in Section 3. Given that our SpLiCE decompositions only aim to encode semantics, they are unable to encode non-semantic aspects in the underlying representation, thus causing poor alignment in the cosine similarity sense, while simultaneously exhibiting excellent alignment on semantic tasks such as zero-shot accuracy. For ImageNet, we find that many classes are animal species that cannot easily be described by 1-2 words (e.g. ‘red-breasted merganser’, ‘American Staffordshire terrier’). Adding these class labels to our concept dictionary increases performance significantly, as shown by the dotted yellow line in Figure 3. Figure 4: Left: SpLiCE decompositions of ImageNet ‘African Elephant’, ‘Curly-coated Retriever’, ‘Monarch Butterfly’, ‘Digital Clock’ classes. Right: Distribution of “Swimwear" concept in ‘Woman’ and ‘Man’ classes of CIFAR100. 5.3 Ablation Studies We perform ablation studies to evaluate the effectiveness of the design decisions of SpLiCE , including the choice of vocabulary, the nonnegativity of the decompositions, and the modality alignment by ablating each choice and observing the effect on three metrics: zero-shot accuracy on CIFAR100, cosine similarity between the reconstructed and original embeddings of CIFAR100, and semantic relevance on MSCOCO. The first two metrics are the same as those presented in Figure 3. We compute semantic relevance by tokenizing and filtering stop-words from the MSCOCO human-generated captions and embedding each token with CLIP. Then, we take all non-zero concepts output by SpLiCE and compute the Hausdorff distance between the sets of SpLiCE concepts and caption token embeddings. This essentially measures how aligned decompositions are with human captions. We observe that replacing our dictionary with the LLM-generated concept dictionary used by [22, 24, 23] significantly worsens the decomposition in terms of zero shot accuracy and cosine reconstruction. While allowing for negative concept weights improves cosine reconstruction marginally, it decreases the semantic relevance of the decompositions, as negative concepts frequently correspond to concepts not present in images, and as such, are unlikely to be represented by human captions. Finally, we see that modality alignment is necessary across all three metrics. Overall, these ablation studies show that each aspect of SpLiCE is necessary for creating human-interpretable, semantically relevant and highly performant decompositions. Figure 5: Ablation study evaluating the efficacy of SpLiCE design choices across three metrics: Zero-shot accuracy, cosine reconstruction, and semantic relevance of recovered tags. We find that all of our design choices, namely non-negativity, modality alignment, and usage of large task-agnostic dictionary are essential to performance. See §5.3 for discussion. 5.4 Qualitative Assessment of Decompositions Concept Decompositions for Images. We visualize SpLiCE decompositions to qualitatively assess the semantic content of the images they represent. In Figure 2 we provide six sample decompositions from MSCOCO with their corresponding captions. We display the top seven concepts for each image and find that they generally well describe the semantics of the images. We also find that these qualitative examples yield interesting and unexpected insights into both CLIP and the data. In the top left image, we see that the decomposition includes the text present on the sign in the image, revealing that CLIP prioritizes text in images over objects. For the bottom left image, the decomposition correctly includes the concept “macro shot”, revealing that CLIP encodes information regarding geometric perspective. The bottom right decomposition similarly features the concept “blackandwhite bw”, indicating that CLIP encodes not only the objects present in images but also information about the lighting and color. Overall, these results suggest that SpLiCE may also be used as a zero-shot image tagging method to understand images. Concept Histograms for Datasets. Beyond concept-based explanations of individual images, we propose that SpLiCE can be used to better understand and summarize collections of images, such as entire datasets. To compute concept decompositions of sets of images, we decompose each individual image and aggregate the results, which we use to generate concept histograms of the dataset. We visualize four concept histograms for the ImageNet classes ‘African Elephant’, ‘Curly-coated Retriever’, ‘Monarch Butterfly’, and ‘Digital Clock’, in Figure 4. These decompositions provide information about the distribution of the data and how CLIP represents it. For example, digital clocks are differentiated from analog clocks through the concepts “lcd display” and “countdown”. Monarch butterflies are highly correlated with the concept “lilac flowers" in ImageNet, which we validated through manual inspection (nearly half of the monarch butterfly images in the validation set feature purple flowers). Interestingly, ‘Curly-coated retrievers’ are represented as combinations of “poodle”, “retriever dog”, and “black dog”, which perfectly describe the main characteristics of them: black retrievers with poodle-textured fur. 6 Case Studies and Applications of SpLiCE In this section, we present two example case studies using SpLiCE : (1) spurious correlation and bias detection in datasets and (2) debiasing classification models. We present additional case studies for (1) and (2), as well as (3) monitoring distribution shift in Appendix B.6, B.7, B.8 B.9. We also present results from a user study to evaluate the human interpretability of SpLiCE in Appendix B.1, where we find that users prefer explanations generated by SpLiCE over existing Concept Bottleneck Model-based methods. Discovering Spurious Correlations in CIFAR100. Existing methods to detect spurious correlations in datasets generally require subgroup and attribute labels or rely on manual human inspection of images (see [62] for an overview), making it hard to scale to large datasets. SpLiCE , on the other hand, allows for fast automatic detection of such biases, without any labels, training, or even a task. To illustrate this, we study two classes of CIFAR100: ‘man’ and ‘woman’, in Figure 4. Upon decomposing these classes, we found that “bra”,“swimwear” were two of the top ten most common concepts in the ‘woman’ class. On the other hand, the only clothing-related concepts that appear in the top 50 most activated concepts for ‘man’ are “uniform”, “tuxedo”, “apparel”. We visualize a histogram of the concept weights on swimwear- and undergarment-related concepts “swimwear”, “bra”, “trunks”, “underwear” across both the train and test sets, and find that these concepts are much more likely to be activated for women than men. Manual inspection of CIFAR100 verifies the trend highlighted by SpLiCE, where at least 70 of the 600 images in the ‘woman’ class feature women in bikinis, underclothes, or even partially undressed, revealing stereotype bias in this popular dataset. We provide a similar study of the concept “desert” with respect to the ‘camel’ and ‘kangaroo’ classes in CIFAR100 in Appendix B.6. Model Editing on CelebA Attribute Classifiers. Table 2: Evaluation of intervention on the concept ‘Glasses’ for the CelebA dataset. SpLiCE allows for surgical removal of information related to whether or not someone is wearing glasses, without impacting other features such as gender. (ZS = Zero Shot Accuracy) Gender Glasses ZS CLIP 0.98 0.91 ZS SpLiCE 0.97 0.88 ZS Intervention SpLiCE 0.96 0.69 Linear Probe 0.89 0.88 Intervention Probe 0.85 0.59 Concept-based representations unlock a key application: being able to intervene on and edit models. This edit can be performed in two equivalent ways: either on the concept representations themselves, where we can zero out a concept or on linear probes built upon the decompositions, where we can edit the weight matrix between concepts and class labels (similar to concept bottleneck models [18]). Here, we evaluate the efficacy of SpLiCE for these forms of model editing. Specifically, we consider two tasks on CelebA, classifying gender and whether the subject is wearing glasses. To test representation editing, we remove the concept of “eyewear” or “glasses” from CelebA image representations by zeroing out any weight placed on these concepts in our SpLiCE decompositions and evaluate classifier performance. We report the performance of zero-shot classification and linear probes over our SpLiCE representation in Table 2. In both cases, we find that we can surgically target and remove information pertaining to glasses and reduce classifier performance while preserving information relevant to gender classification. We perform a similar experiment on the Waterbirds dataset [63] to remove spurious background signals in B.7. 7 Discussion In this work, we show that the information contained in CLIP embeddings can be approximated by a sparse, linear combination of simple semantic concepts, allowing us to interpret representations via sparse recovery. We propose SpLiCE , a method to transform the dense, uninterpretable embeddings of CLIP into human-interpretable sparse concept decompositions. We empirically demonstrate that SpLiCE allows for an adjustable tradeoff on the interpretability-accuracy Pareto frontier, enabling users to decide the loss in performance they are willing to incur for interpretability. Furthermore, we find that the improved interpretability of SpLiCE allows for users to diagnose and fix model mistakes, ideally increasing the effectiveness and performance of the overall system using a VLM. We then provide concrete use cases for SpLiCE : spurious correlation detection and model intervention and editing, showcasing the benefits of using interpretable embeddings with known semantic content. We highlight that SpLiCE embeddings can serve as post-hoc interpretations of CLIP embeddings and can even replace them to ensure full transparency. Broader Impact. Similar to many works in the field of interpretability, our work provides greater understanding of the behavior of models, including but not limited to the broader implicit biases they perpetuate as well as mistakes made on individual samples. We believe this is particularly salient for CLIP, which is used in a variety of applications that are widely used in practice at this moment. We hope that insights gained from such interpretability allow users to make more informed decisions regarding how they interact with and use CLIP, regardless of their familiarity with machine learning or domain expertise in the task they are using CLIP for. We also highlight that SpLiCE can be used as a visualization-like tool for exploring and summarizing datasets at scale, allowing for easier auditing of spurious correlations and biases in both datasets and models. Limitations. In this work, we use a large, overcomplete dictionary of one- and two-word concepts, however future work may wish to expand this dictionary or learn a dictionary over tokens (in discrete language space), to capture concepts with more than two words. This may also reduce the size of the dictionary and improve computation time. We note that this dictionary was constructed by looking at token frequency in the LAION-5B dataset, which has its own biases and may not correctly capture all the salient concepts that CLIP encodes. Despite this, we find that SpLiCE performs well on a variety of tasks while outperforming state-of-the-art concept dictionaries (Fig. 5, Appendix Fig. 13) and thus we believe LAION is a good dataset to generate a concept vocabulary from. We also note that this vocabulary can be easily modified by practitioners to consider additional concepts as needed for specific use cases. Finally, SpLiCE also uses an ℓ1subscriptℓ1 _1ℓ1 penalty as the relaxation for ℓ0subscriptℓ0 _0ℓ0 regularization, but future work may consider alternative relaxations or even binary concept weights. Acknowledgements and Disclosure of Funding This work is supported in part by the NSF awards IIS-2008461, IIS-2040989, IIS-2238714, FAI-2040880, and research awards from Google, JP Morgan, Amazon, Adobe, Harvard Data Science Initiative, and the Digital, Data, and Design (D3) Institute at Harvard. AO is supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE-2140743, and UB is funded by the Kempner Institute Graduate Research Fellowship. The views expressed here are those of the authors and do not reflect the official policy or position of the funding agencies. References Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR, 18–24 Jul 2021. Mokady et al. [2021] Ron Mokady, Amir Hertz, and Amit H Bermano. Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:2111.09734, 2021. Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022. Koh and Liang [2017] Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In International conference on machine learning, pages 1885–1894. PMLR, 2017. Bengio [2013] Yoshua Bengio. Deep learning of representations: Looking forward. In International conference on statistical language and speech processing, pages 1–37. Springer, 2013. Olah et al. [2017] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2(11):e7, 2017. Kim et al. [2018] Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668–2677. PMLR, 2018. Hsu et al. [2023] Kyle Hsu, Will Dorrell, James CR Whittington, Jiajun Wu, and Chelsea Finn. Disentanglement via latent quantization. arXiv preprint arXiv:2305.18378, 2023. Geirhos et al. [2023] Robert Geirhos, Roland S Zimmermann, Blair Bilodeau, Wieland Brendel, and Been Kim. Don’t trust your eyes: on the (un) reliability of feature visualizations. arXiv preprint arXiv:2306.04719, 2023. Mikolov et al. [2013] Tomáš Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 746–751, 2013. Park et al. [2023] Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. arXiv preprint arXiv:2311.03658, 2023. Arora et al. [2018] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483–495, 2018. Arora et al. [2016] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A latent variable model approach to pmi-based word embeddings. Transactions of the Association for Computational Linguistics, 4:385–399, 2016. Faruqui et al. [2015] Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah Smith. Sparse overcomplete word vector representations. arXiv preprint arXiv:1506.02004, 2015. Yuksekgonul et al. [2022a] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it? In The Eleventh International Conference on Learning Representations, 2022a. Merullo et al. [2022] Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image to text space. arXiv preprint arXiv:2209.15162, 2022. Seth et al. [2023] Ashish Seth, Mayur Hemani, and Chirag Agarwal. Dear: Debiasing vision-language models with additive residuals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6820–6829, 2023. Koh et al. [2020] Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR, 2020. Lampert et al. [2009] Christoph H Lampert, Hannes Nickisch, and Stefan Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In 2009 IEEE conference on computer vision and pattern recognition, pages 951–958. IEEE, 2009. Torresani et al. [2010] Lorenzo Torresani, Martin Szummer, and Andrew Fitzgibbon. Efficient object category recognition using classemes. In Computer Vision–ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part I 11, pages 776–789. Springer, 2010. Kumar et al. [2009] Neeraj Kumar, Alexander C Berg, Peter N Belhumeur, and Shree K Nayar. Attribute and simile classifiers for face verification. In 2009 IEEE 12th international conference on computer vision, pages 365–372. IEEE, 2009. Chattopadhyay et al. [2023] Aditya Chattopadhyay, Ryan Pilgrim, and Rene Vidal. Information maximization perspective of orthogonal matching pursuit with applications to explainable ai. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. Oikarinen et al. [2023] Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. arXiv preprint arXiv:2304.06129, 2023. Panousis et al. [2023] Konstantinos Panagiotis Panousis, Dino Ienco, and Diego Marcos. Sparse linear concept discovery models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2767–2771, 2023. Olah et al. [2020] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 5(3):e00024–001, 2020. Bau et al. [2017] David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6541–6549, 2017. Fong and Vedaldi [2018] Ruth Fong and Andrea Vedaldi. Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8730–8738, 2018. McGrath et al. [2022] Thomas McGrath, Andrei Kapishnikov, Nenad Tomašev, Adam Pearce, Martin Wattenberg, Demis Hassabis, Been Kim, Ulrich Paquet, and Vladimir Kramnik. Acquisition of chess knowledge in alphazero. Proceedings of the National Academy of Sciences, 119(47):e2206625119, 2022. Lucieri et al. [2020] Adriano Lucieri, Muhammad Naseer Bajwa, Stephan Alexander Braun, Muhammad Imran Malik, Andreas Dengel, and Sheraz Ahmed. On interpretability of deep learning based skin lesion classifiers using concept activation vectors. In 2020 international joint conference on neural networks (IJCNN), pages 1–10. IEEE, 2020. Zhou et al. [2018] Bolei Zhou, Yiyou Sun, David Bau, and Antonio Torralba. Interpretable basis decomposition for visual explanation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 119–134, 2018. Fel et al. [2023] Thomas Fel, Victor Boutin, Mazda Moayeri, Rémi Cadène, Louis Bethune, Mathieu Chalvidal, Thomas Serre, et al. A holistic approach to unifying automatic concept extraction and concept importance estimation. arXiv preprint arXiv:2306.07304, 2023. Bricken et al. [2023] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, page 2, 2023. Chen et al. [2018] Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K Duvenaud. Isolating sources of disentanglement in variational autoencoders. Advances in neural information processing systems, 31, 2018. Comon [1994] Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3):287–314, 1994. Hyvärinen and Oja [2000] Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430, 2000. Makelov et al. [2023] Aleksandar Makelov, Georg Lange, and Neel Nanda. Is this the subspace you are looking for? an interpretability illusion for subspace activation patching. arXiv preprint arXiv:2311.17030, 2023. Moayeri et al. [2023] Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. In International Conference on Machine Learning, pages 25037–25060. PMLR, 2023. Yuksekgonul et al. [2022b] Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15480, 2022b. Yun et al. [2022] Tian Yun, Usha Bhalla, Ellie Pavlick, and Chen Sun. Do vision-language pretrained models learn composable primitive concepts? arXiv preprint arXiv:2203.17271, 2022. Gandelsman et al. [2023] Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip’s image representation via text-based decomposition. arXiv preprint arXiv:2310.05916, 2023. Grootendorst [2022] Maarten Grootendorst. Bertopic: Neural topic modeling with a class-based tf-idf procedure. arXiv preprint arXiv:2203.05794, 2022. Murphy et al. [2012] Brian Murphy, Partha Talukdar, and Tom Mitchell. Learning effective and interpretable semantic models using non-negative sparse embedding. In Proceedings of COLING 2012, pages 1933–1950, 2012. Fyshe et al. [2014] Alona Fyshe, Partha P Talukdar, Brian Murphy, and Tom M Mitchell. Interpretable semantic vectors from a joint model of brain-and text-based meaning. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2014, page 489. NIH Public Access, 2014. Fyshe et al. [2015] Alona Fyshe, Leila Wehbe, Partha Talukdar, Brian Murphy, and Tom Mitchell. A compositional and interpretable semantic space. In Proceedings of the 2015 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 32–41, 2015. Olshausen and Field [1997] Bruno A Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37(23):3311–3325, 1997. Olshausen and Field [1996] Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583):607–609, 1996. Ramaswamy et al. [2022] Vikram V Ramaswamy, Sunnie SY Kim, Ruth Fong, and Olga Russakovsky. Overlooked factors in concept-based explanations: Dataset choice, concept salience, and human capability. arXiv preprint arXiv:2207.09615, 2022. Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. " why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144, 2016. Vinson and Vigliocco [2008] David P Vinson and Gabriella Vigliocco. Semantic feature production norms for a large set of objects and events. Behavior Research Methods, 40(1):183–190, 2008. McRae et al. [2005] Ken McRae, George S Cree, Mark S Seidenberg, and Chris McNorgan. Semantic feature production norms for a large set of living and nonliving things. Behavior research methods, 37(4):547–559, 2005. Garrard et al. [2001] Peter Garrard, Matthew A Lambon Ralph, John R Hodges, and Karalyn Patterson. Prototypicality, distinctiveness, and intercorrelation: Analyses of the semantic attributes of living and nonliving concepts. Cognitive neuropsychology, 18(2):125–174, 2001. Schuhmann et al. [2021] Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. Liang et al. [2022] Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Advances in Neural Information Processing Systems, 35:17612–17625, 2022. Ilharco et al. [2021] Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip. July 2021. doi: 10.5281/zenodo.5143773. URL https://doi.org/10.5281/zenodo.5143773. If you use this software, please cite it as below. Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. Isola et al. [2015] Phillip Isola, Joseph J. Lim, and Edward H. Adelson. Discovering states and transformations in image collections. In CVPR, 2015. Liu et al. [2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014. Deng et al. [2009] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009. Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011. Beck and Teboulle [2009] Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1):183–202, 2009. Wu et al. [2023] Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept-aware mitigation of spurious correlation. In International Conference on Machine Learning, pages 37765–37786. PMLR, 2023. Sagawa et al. [2019] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731, 2019. Boyd et al. [2011] Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, Jonathan Eckstein, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine learning, 3(1):1–122, 2011. Fei-Fei et al. [2006] Li Fei-Fei, Robert Fergus, and Pietro Perona. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 28(4):594–611, 2006. Xiao et al. [2010] Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 3485–3492. IEEE, 2010. Coates et al. [2011] Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011. [68] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://w.pascal-network.org/challenges/VOC/voc2007/workshop/index.html. Stark et al. [2011] Michael Stark, Jonathan Krause, Bojan Pepik, David Meger, James J Little, Bernt Schiele, and Daphne Koller. Fine-grained categorization for 3d scene understanding. International Journal of Robotics Research, 30(13):1543–1552, 2011. Appendix Summary of Appendix Results • A. Further Details on the Method – A.1. When do Sparse Decompositions Exist? – A.2. Relationship between cosine similarity and MSE optimization – A.3. ADMM for batched on-device LASSO optimization – A.4. Effect of Modality Alignment – A.5. Experimental Details • B. Additional Results – B.1. User Study for Human Interpretability – B.2. Performance of SpLiCE on Probing Tasks – B.3. Performance of SpLiCE on Retrieval Tasks – B.4. Additional Zero-Shot Results – B.5. Additional ImageNet Concept Histograms – B.6. Additional Case Study: Detecting Spurious Correlations – B.7. Additional Case Study: Spurious Correlation Intervention – B.8. Additional Case Study: Distribution Shift Monitoring – B.9. Additional Case Study: Distribution Shift Monitoring – B.10. Checking the Interpretability of Negative Concepts – B.11. Understanding the Image Mean for Modality Alignment – B.12. Choice of Concept Vocabulary – B.13. Concept Type Distribution – B.14. Experiments on Alternative CLIP Architecture Appendix A Further Details on the Method A.1 When do Sparse Decompositions Exist? Proposition 1. Given Assumptions 1-5, CLIP image embeddings f can be written as a sparse linear combination of text embeddings, i.e, f⁢(img)=txt⁢;s.t.‖0≤αformulae-sequencesuperscriptimgsuperscripttxtsubscriptnorm0 f(x^img)=C^txtw;~% ~s.t.~~\|w\|_0≤ ( ximg ) = Ctxt w ; s . t . ∥ w ∥0 ≤ α where ∈ℝ+ksuperscriptsubscriptℝw _+^kw ∈ blackboard_R+k, and txt∈ℝd×ksuperscripttxtsuperscriptℝC^txt ^d× kCtxt ∈ blackboard_Rd × k, which is the text concept dictionary defined previously. Proof. Any vector ω can be written as ω=∑i=1kωi⁢isuperscriptsubscript1subscriptsubscriptω= _i=1^k _ie_iω = ∑i = 1k ωitalic_i eitalic_i, where ωi∈ℝ+subscriptsubscriptℝ _i _+ωitalic_i ∈ blackboard_R+, and i∈ℝksubscriptsuperscriptℝe_i ^keitalic_i ∈ blackboard_Rk is a one-hot vector with one at the ithsuperscriptthi^thith co-ordinate. Thus we have f⁢(img)=superscriptimgabsent f(x^img)=f ( ximg ) = f∘himg⁢(ω,ϵ)=f∘himg⁢(ω)(Assumption 2)superscriptℎimgitalic-ϵsuperscriptℎimgAssumption 2 f h^img(ω,ε)=f h^img(% ω)~~~~(Assumption~2)f ∘ himg ( ω , ϵ ) = f ∘ himg ( ω ) ( Assumption 2 ) = == f∘himg⁢(∑i=1kωi⁢i)=∑i=1kωi⁢f∘himg⁢(i)⏟iimg(Assumption 3)superscriptℎimgsuperscriptsubscript1subscriptsubscriptsuperscriptsubscript1subscriptsubscript⏟superscriptℎimgsubscriptsubscriptsuperscriptimgAssumption 3 f h^img ( _i=1^k _ie_i% )= _i=1^k _i f h^img(e_% i)_c^img_i~~~~(Assumption~3)f ∘ himg ( ∑i = 1k ωitalic_i eitalic_i ) = ∑i = 1k ωitalic_i under⏟ start_ARG f ∘ himg ( eitalic_i ) end_ARGcimg start_POSTSUBSCRIPT i end_POSTSUBSCRIPT ( Assumption 3 ) Here we define iimg=f∘himg⁢(i)superscriptsubscriptimgsuperscriptℎimgsubscriptc_i^img=f h^img(e_i)citalic_iimg = f ∘ himg ( eitalic_i ) as the ‘image’ concept basis vector; analogous to the text concept basis vector itxt=g∘htxt⁢(i)superscriptsubscripttxtsuperscriptℎtxtsubscriptc_i^txt=g h^txt(e_i)citalic_itxt = g ∘ htxt ( eitalic_i ) already defined. Thus Assumption 2 implies the existence of a sparse decomposition of f in terms of ‘image’ concept vectors iimgsubscriptsuperscriptimgc^img_icimgitalic_i. Additionally, Assumption 1 ensures that this decomposition is sparse, as ω is sparse. So far, we have f⁢(img)=img⁢ω⁢s.t.‖ω‖0≤αformulae-sequencesuperscriptimgsuperscriptimgsubscriptnorm0f(x^img)=C^imgω~~~s.t.~~\|% ω\|_0≤ ( ximg ) = Cimg ω s . t . ∥ ω ∥0 ≤ α. From Assumption 4, the image concept vectors and text concept vectors are equal to each other, i.e, iimg=f∘himg⁢(i)=g∘htxt⁢(i)=itxtsuperscriptsubscriptimgsuperscriptℎimgsubscriptsuperscriptℎtxtsubscriptsuperscriptsubscripttxtc_i^img=f h^img(e_i)=g h^% txt(e_i)=c_i^txtcitalic_iimg = f ∘ himg ( eitalic_i ) = g ∘ htxt ( eitalic_i ) = citalic_itxt. Finally, from Assumption 5, we have that the text concept vectors itxtsuperscriptsubscripttxtc_i^txtcitalic_itxt are given simply by word embeddings g of individual words. Stringing these arguments together, we have that image representations f⁢(img)superscriptimgf(x^img)f ( ximg ) can be written as a sparse linear combination of vectors obtain from CLIP word embeddings itxtsuperscriptsubscripttxtc_i^txtcitalic_itxt. We finally set =ωw= = ω, thus proving the assertion. ∎ A.2 Relationship between cosine similarity and MSE optimization. Recall our ℓ1subscriptℓ1 _1ℓ1 relaxed cosine similarity optimization problem from Eqn. (1), min∈ℝ+c⁡‖0⁢s.t.⁢⟨,‖2⟩≥1−ϵ.subscriptsuperscriptsubscriptℝsubscriptnorm0s.t.subscriptnorm21italic-ϵ _w _+^c\|w\|_0~~% s.t.~~ , Cw\|C% w\|_2 ≥ 1-ε.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ∥ w ∥0 s.t. ⟨ z , divide start_ARG Cw end_ARG start_ARG ∥ Cw ∥2 end_ARG ⟩ ≥ 1 - ϵ . (3) First we relax the ℓ0subscriptℓ0 _0ℓ0 constraint to an ℓ1subscriptℓ1 _1ℓ1 penalty. max∈ℝ+c⁡⟨,‖2⟩−λ⁢‖1.subscriptsuperscriptsubscriptℝsubscriptnorm2subscriptnorm1 _w _+^c , % Cw\|Cw\|_2 -λ\|w% \|_1.maxbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ⟨ z , divide start_ARG Cw end_ARG start_ARG ∥ Cw ∥2 end_ARG ⟩ - λ ∥ w ∥1 . (4) By observing that ‖x−y‖22=⟨x−y,x−y⟩=⟨x,x⟩+⟨y,y⟩−2⁢⟨x,y⟩superscriptsubscriptnorm222||x-y||_2^2= x-y,x-y = x,x + y,y -2% x,y | | x - y | |22 = ⟨ x - y , x - y ⟩ = ⟨ x , x ⟩ + ⟨ y , y ⟩ - 2 ⟨ x , y ⟩ and that zz, ‖2subscriptnorm2 Cw\|Cw\|_2divide start_ARG Cw end_ARG start_ARG ∥ Cw ∥2 end_ARG are unit-norm, maximizing the above inner product is equivalent to minimizing the euclidean norm, min∈ℝ+c⁢‖2−‖22+2⁢λ⁢‖1.subscriptsuperscriptsubscriptℝsuperscriptsubscriptnormsubscriptnorm2222subscriptnorm1 _w _+^c|| Cw% \|Cw\|_2-z||_2^2+2λ||w||_% 1.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT | | divide start_ARG Cw end_ARG start_ARG ∥ Cw ∥2 end_ARG - z | |22 + 2 λ | | w | |1 . (5) This is a non-convex problem, but we can relax this problem to achieve better reconstruction in terms of euclidan distance as shown in Eqn. (2), min∈ℝ+c⁡‖−‖22+2⁢λ⁢‖1.subscriptsuperscriptsubscriptℝsuperscriptsubscriptnorm222subscriptnorm1 _w _+^c\|Cw-% z\|_2^2+2λ\|w\|_1.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ∥ Cw - z ∥22 + 2 λ ∥ w ∥1 . (6) This problem will optimize euclidean distance between CwCw and zz. Consider two vectors x,yx,yx , y on the unit sphere such that ⟨x‖x‖,y‖y‖⟩>0normnorm0 x||x||, y||y|| >0⟨ divide start_ARG x end_ARG start_ARG | | x | | end_ARG , divide start_ARG y end_ARG start_ARG | | y | | end_ARG ⟩ > 0. While any vector α⁢yα yα y, α>00α>0α > 0 will have the same cosine similarity score, the optimal vector in terms of euclidean distance to x is the vector α⁢yα yα y such that α=projy⁢(x)subscriptprojα=proj_y(x)α = projy ( x ), or in other words the projection of x onto y. Thus, solving for euclidean distance to approximate x will find α⁢yα yα y which we must then normalize to find the unit-norm solution y. This explains the normalizing process described in Section 4.1. Additionally, we can view Eqn. (6) as applying shrinkage to CwCw. Reconverting from euclidean norm to inner product, Eqn. (6) becomes max∈ℝ+c⁡⟨,⟩−12⁢⟨,⟩−λ⁢‖1=⟨,⟩−12⁢‖22−λ⁢‖1.subscriptsuperscriptsubscriptℝ12subscriptnorm112subscriptsuperscriptnorm22subscriptnorm1 _w _+^c % ,z - 12 ,Cw% -λ||w||_1= ,z% - 12||Cw||^2_2-λ||w||_1.maxbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ⟨ Cw , z ⟩ - divide start_ARG 1 end_ARG start_ARG 2 end_ARG ⟨ Cw , Cw ⟩ - λ | | w | |1 = ⟨ Cw , z ⟩ - divide start_ARG 1 end_ARG start_ARG 2 end_ARG | | Cw | |22 - λ | | w | |1 . (7) In conclusion, our optimization problem maximizes the inner product while imposing a shinkage penalty and sparsity penalty. Empirically, our reconstructions CwCw are low-norm, so we normalize after solving to recover the unit-norm reconstruction. A.3 ADMM for batched on-device LASSO optimization. As each decomposition requires solving a LASSO optimization problem, we implement the Alternating Direction Method of Multipliers (ADMM) algorithm in Pytorch over batches with GPU support for efficient decomposition of large scale datasets over large numbers of concepts [64]. In practice, ADMM achieves primal and dual tolerances of 1⁢e−41e41e-41 e - 4 in fewer than 1000100010001000 iterations on a batch size of 1024.10241024.1024 . We present an empirical comparison beterrn LASSO and ADMM in 6, where we find both methods to be approximately equivalent. Next we derive the iterates for our ADMM algorithm. Recall our optimization problem, min∈ℝ+c⁡‖−‖22+2⁢λ⁢‖1.subscriptsuperscriptsubscriptℝsuperscriptsubscriptnorm222subscriptnorm1 _w _+^c\|Cw-% z\|_2^2+2λ\|w\|_1.minbold_w ∈ blackboard_R start_POSTSUBSCRIPT +c end_POSTSUBSCRIPT ∥ Cw - z ∥22 + 2 λ ∥ w ∥1 . (8) ADMM breaks down convex optimization problems into multiple sub-problems while penalizing the difference in solutions. We break Eqn. (8) into two subproblems, one solving the euclidean distance objective and one solving the ℓ1subscriptℓ1 _1ℓ1 and nonnegativity constraint. We let w denote the former solution, z the latter, and u tracks the difference between the two. Our ADMM iterates (wk,zk,uk)superscriptsuperscriptsuperscript(w^k,z^k,u^k)( witalic_k , zitalic_k , uitalic_k ) are wk+1superscript1 w^k+1witalic_k + 1 =arg⁡minw⁡(f⁢(w)+ρ2⁢‖wk−zk+uk‖22),absentsubscript2superscriptsubscriptnormsuperscriptsuperscriptsuperscript22 = _w(f(w)+ ρ2||w^k-z^k+u^k||_2^2),= arg minitalic_w ( f ( w ) + divide start_ARG ρ end_ARG start_ARG 2 end_ARG | | witalic_k - zitalic_k + uitalic_k | |22 ) , (9) zk+1superscript1 z^k+1zitalic_k + 1 =(Sλ/ρ⁢(wk+1+uk))+,absentsubscriptsubscriptsuperscript1superscript =(S_λ/ρ(w^k+1+u^k))_+,= ( Sitalic_λ / ρ ( witalic_k + 1 + uitalic_k ) )+ , (10) uk+1superscript1 u^k+1uitalic_k + 1 =uk+wk+1−zk+1,absentsuperscriptsuperscript1superscript1 =u^k+w^k+1-z^k+1,= uitalic_k + witalic_k + 1 - zitalic_k + 1 , (11) where SκsubscriptS_κSitalic_κ is a soft-thresholding function used to satisfy the LASSO constraints, Sκ⁢(a):=a−κ,a>κ0,|a|≤κa+κ,a<−κassignsubscriptcases0 S_κ(a):= casesa-κ,&a>κ\\ 0,&|a|≤κ\\ a+κ,&a<-κ casesSitalic_κ ( a ) := start_ROW start_CELL a - κ , end_CELL start_CELL a > κ end_CELL end_ROW start_ROW start_CELL 0 , end_CELL start_CELL | a | ≤ κ end_CELL end_ROW start_ROW start_CELL a + κ , end_CELL start_CELL a < - κ end_CELL end_ROW (12) As our optimization function f⁢(w)f(w)f ( w ) is quadratic, we can analytically compute wk+1superscript1w^k+1witalic_k + 1 as wk+1=(2⁢T⁢+ρ)−1⁢(ρ⁢v+2⁢w),superscript1superscript2superscript12 w^k+1=(2C^TC+ρ)^-1(ρ v+2C% w),witalic_k + 1 = ( 2 Citalic_T C + ρ )- 1 ( ρ v + 2 C w ) , (13) where v=zk−uksuperscriptsuperscriptv=z^k-u^kv = zitalic_k - uitalic_k. In our experiments we set ρ=55ρ=5ρ = 5, and stop when tolerances ϵprim=‖xk+1−zk+1‖2subscriptitalic-ϵprimsubscriptnormsuperscript1superscript12 _prim=||x^k+1-z^k+1||_2ϵroman_prim = | | xitalic_k + 1 - zitalic_k + 1 | |2, ϵdual=‖ρ⁢(zk+1−zk)‖2subscriptitalic-ϵdualsubscriptnormsuperscript1superscript2 _dual=||ρ(z^k+1-z^k)||_2ϵroman_dual = | | ρ ( zitalic_k + 1 - zitalic_k ) | |2 are less than 1⁢e−41e41e-41 e - 4. Over a batch, we iterate until every solver in the batch has reached the above tolerances. Figure 6: Comparison of ADMM (maroon) and LASSO (yellow) for solving the SpLiCE objective on zero shot accuracy (left) and cosine reconstruction (right) on CIFAR100. Both methods are approximately equal. A.4 Effect of Modality Alignment We take MSCOCO images and captions, embed them with CLIP, and compare the cosine similarity between modalities and inter-modality. Before mean-centering and renormalizing, the similarity within modalities is high, with an average of around 0.3. This indicates that the image and text embeddings do not span the entire unit-sphere but rather lie on two cones. However, the similarity across modalities has an average concentrating around zero, indicating that these two cones are non-overlapping. However, after mean-centering and normalizing, we observe that the average cosine similarity for images, text, and between images and text becomes zero and the modalities are aligned. Figure 7: Average cosine similarity across pairs of image-text, image-image, and text-text data from MSCOCO. After aligning modalities, the distribution of similarities is centered around zero. A.5 Experimental Details All experiments are able to be performed on a single A100 GPU to run fast inference with CLIP. After embedding the concept dictionary, all computation can be performed on a CPU. Code is made available at https://github.com/AI4LIFE-GROUP/SpLiCE. Appendix B Additional Results B.1 User Study for Human Interpretability We present results from a user study in 8 to assess the human interpretability of SpLiCE. We base our study off of that performed by [23] to evaluate Label-Free Concept Bottleneck Models (LF-CBMs). We benchmark our method against LF-CBMs and IP-OMP [22]. We provided users with twenty randomly chosen, correctly predicted images from ImageNet and explanations from two different methods comprising the top six most important concepts for every image. We then asked users to evaluate and compare the different concept-based explanations for (1) their relevance to the provided image inputs, (2) their relevance to model predictions, and (3) their informativeness on Likert scales from 1 to 5. We found that users significantly preferred explanations generated by SpLiCE to the two baselines for relevance to the images and informativeness, with significance determined via a one-sample two-sided t-test and a threshold of p=0.01. We also highlight that our method is able to produce similar/better concept decompositions, in terms of human interpretability, than the baselines without needing to train a classification probe or use class labels for concept mining, both of which are computationally expensive. This user study was ruled exempt by our institution’s IRB, as no risks were posed to the users. Participants were able to opt out at any time, and no questions were asked regarding the participants themselves. Figure 8: Results of a user study evaluating SpLiCE, LF-CBM, and IP-OMP in the style of the user study from LF-CBM. Overall, we find that explanations generated by SpLiCE are deemed more relevant to the image, relevant to the prediction, and more informative than prior methods. B.2 Performance of SpLiCE on Probing Tasks We evaluate the performance of the decompositions on probes trained on both regular CLIP embeddings as well as decomposed CLIP embeddings for CIFAR100 in 3 and MIT States in 4. We consider two scenarios: a probe trained on CLIP embeddings and tested on SpLiCE embeddings of various sparsities (shown in row CLIP Probe), and a probe both trained and evaluated on SpLiCE embeddings (shown in row SpLiCE Probe). We report mean over three runs, with standard deviations for each experiment being less than 0.005. We find that SpLiCE representations closely match the performance of dense CLIP embeddings, with a slight drop in performance when probes are trained directly on SpLiCE embeddings rather than trained on CLIP embeddings and evaluated on SpLiCE embeddings for CIFAR100. Table 3: Evaluation of Probing Performance on CIFAR100 l0subscript0l_0l0 = 3 l0subscript0l_0l0 = 6 l0subscript0l_0l0 = 23 l0subscript0l_0l0 = 117 CLIP SpLiCE Probe 0.95 0.95 0.95 0.95 – CLIP Probe 0.96 0.96 0.97 0.97 0.97 Table 4: Evaluation of Probing Performance on MIT States l0subscript0l_0l0 = 4 l0subscript0l_0l0 = 7 l0subscript0l_0l0 = 27 CLIP SpLiCE Probe 0.883 0.883 0.882 – CLIP Probe 0.883 0.883 0.884 0.883 B.3 SpLiCE Performance on Retrieval Tasks We test the performance of SpLiCE embeddings on text-to-image and image-to-text retrieval tasks. We evaluate retrieval over various 1024 sample subsets of MSCOCO, and assess recall performance for the top-k closest embeddings of the opposite modality for k=1,5,101510k=\1,5,10\k = 1 , 5 , 10 . We find that our semantic concept dictionaries outperform all baselines when decomposition sparsity is high, but that dictionaries learned over images perform slightly better for text to image retrieval when decompositions have greater than 30 nonzero concepts. Figure 9: Top-1 , 5, 10 performance of SpLiCE representations on image-to-text (top) and text-to-image (bottom) retrieval on MSCOCO. B.4 Additional Zero-Shot Results We present additional results comparing SpLiCE reconstructed vectors and CLIP embeddings on the Caltech101 [65], SUN397 [66], STL10 [67], and VOC2007 [68] datasets in 5. We use SpLiCE decompositions with sparsities of 20-35, and we find that they are comparable to the unaltered CLIP embeddings. Table 5: Additional zero-shot accuracy on baselines from the CLIP paper, for decompositions of sparsity 20-35. Note that at human-interpretable levels of sparsity, we see a minor drop in performance. Caltech101 SUN397 STL10 VOC 2007 CLIP Reported 0.88 0.63 0.97 0.83 CLIP Implemented 0.90 0.67 0.96 0.92 SpLiCE 0.86 0.66 0.96 0.83 We further explore the performance of SpLiCE decompositions in the limit as they approach the sparsity of the baseline CLIP embeddings (512). We find that SpLiCE completely recovers CLIP zero-shot accuracy at this limit, as shown in 6. Table 6: Zero shot performance at sparsity 512. Note that SpLiCE completely recovers baseline CLIP zero shot accuracy. CIFAR100 MITStates Imagenet CLIP Baseline 0.750 0.469 0.552 SpLiCE (512) 0.768 0.474 0.552 B.5 Additional ImageNet Concept Histograms We present concept histograms for the top seven concepts of five more ImageNet classes: ‘Face Powder’, ‘Feather Boa’, ‘Jack-O’-Lantern’, ‘Kimono’, ‘Dalmation’, similar to Figure 10. These decompositions give insights both into the distribution of each class as well as some biases of CLIP. For example, for the class ‘Face Powder’, the concept “benefit" is the fifth most common concept, and it is indeed a common cosmetic brand name in the images. For the ‘Dalmation’ class, we see that the decompositions consists of concepts relating to dogs and black and white spots, which together make up the high-level concept of a dalmation. Finally, for the class ‘Kimono’, the concept “doll" is the seventh most common, although all of the images in the ‘Kimono’ class were of real humans, not of dolls. This highlights an implicit bias in CLIP’s representations or in the descriptions of people wearing kimonos in CLIP’s training set. Figure 10: Example concept histograms of various ImageNet classes. The top seven concepts for each class are visualized along with their relative weighting, with the average ℓ0subscriptℓ0 _0ℓ0 norm of individual sample decompositions also being 7. B.6 Additional Case Study: Detecting Spurious Correlations We present an additional case study for detecting spurious correlations in CIFAR100. In particular, we look at the prevalence of the spurious concept “desert” in the classes ‘camel’ and ‘kangaroo’ in Figure 11. We observe that camels are more frequently pictured in the desert, creating a spurious signal that may be leveraged by downstream classifiers. This figure provides an additional example of how we can understand biases and trends in data with SpLiCE decompositions. Figure 11: Distribution of “Desert" concept in ‘Camel’ and ‘Kangaroo’ classes of CIFAR100. B.7 Additional Case Study: Spurious Correlation Intervention We further test the ability of SpLiCE to enable intervention on intermediate representations and linear classifiers by attempting to remove information pertaining to spurious signals. In particular, we consider the Waterbirds dataset [63], which spuriously correlates landbirds with land backgrounds, resulting in trained classifiers performing poorly on waterbirds on land. We thus remove information about whether or not birds are on land backgrounds by ablating concept weights on “bamboo”, “forest”, “hiking”, and “rainforest” as well as any bigrams containing the word “forest,” as shown in Table 7. This significantly improves worst-case subgroup performance for waterbirds on land from 0.48 to 0.60. For both this experiment and the intervention on CelebA described in the main paper, we train linear probes using the LogisticRegressionClassifier module in scikit-learn using an ℓ1subscriptℓ1 _1ℓ1 penalty. Table 7: Evaluation of intervention on spurious correlations for Waterbirds dataset. Removing information about land backgrounds improves worst-case subgroup performance. Landbirds on land Waterbirds on land Linear Probe 0.98 0.48 Intervention Probe 0.97 0.60 B.8 Additional Case Study: Distribution Shift Monitoring We present a final case study using SpLiCE to monitor distribution shift. This can help identify differences between training and inference distributions or evaluate how a continually sampled dataset changes over time. In this experiment we consider the Stanford Cars dataset [69], which contains photos of cars from 1991 to 2012, including their make and year labels. By decomposing photos of cars from each year, we can view how the distribution changed yearly. We visualize the weights of the concepts “convertible" and “yellow" from our decompositions, as well as the actual percentage of cars from each year that were convertibles or yellow in Figure 12. Note the right-hand y-axis, corresponding to the weight of the given concept cisubscriptc_icitalic_i over the sum of the weights of all concepts ∑icisubscriptsubscript _ic_i∑i citalic_i, does not have a meaningful unit of measure or scale. We find that the trends in the groundtruth concept prevalence generally closely match that of the predicted/decomposed concepts, allowing us to visualize which years convertibles or yellow cars were popular or out-of-distribution with respect to other years. Most notably, we see that SpLiCE picks up on the out-of-distribution rise in popularity of brightly colored sports cars in the early 2000s. Figure 12: Visualization of the presence of convertibles (pink lines) and yellow cars (yellow lines) in Stanford Cars over time. SpLiCE concept weights (dotted) closely track the groundtruth concept prevalence (solid) for both concepts. B.9 Additional Case Study: Distribution Shift Monitoring To further verify that SpLiCE allows for identification and tracking of distribution shift, we study the Waterbirds dataset, which is known to have differently balanced train, valodation, and test splits. To identify distribution shifts, we can simply look at the norm of the difference between the class decompositions of the two classes for each splot, as shown in 8. We find that the validation and test splits are much more similar than the training and validation splits or the training and test splits, which can be verified by the construction process of the Waterbirds dataset. Table 8: Study of the differences in distributions between train, validation, and test splits of Waterbirds. The validation and test splits are much more similar to each other than they are to the train split. Train, Val Train, Test Val, Test Class Landbird 0.0182 0.0182 0.005 Class Waterbird 0.0229 .0188 0.009 We also find that the most weighted concept in the ‘landbird’ class of the train split is “bamboo” but the corresponding weight for “bamboo” in the ‘waterbird’ class is much lower. The “bamboo” concept weight for both classes and all splits is shown below, where we see that the validation and test splits are very similar and mostly evenly balanced, whereas the train split is highly unbalanced. Table 9: Study of the prevalence of the concept “bamboo” in the different classes and splits of Waterbirds. Train Val Test Class Landbird 0.0196 0.010 0.010 Class Waterbird 0.0007 0.008 0.008 B.10 Checking the Interpretability of Negative Concepts We take a set of 71 concept-antonym pairs from the MIT States dataset and embed the terms in CLIP. With and without concept centering, we observe that these concept-antonym pairs have an average cosine similarity well above -1, indicating that CLIP does not place antonyms in opposite directions, as shown in 10. Next, we take our concept dictionary and prepend “not" to all of the words and compare the average cosine similarity between concept and not-concept pairs. Similarly, we observe that with and without centering, concept and not-concept pairs are highly similar. Note that the average similarity for true pairs of images and text in MSCOCO is less than the similarity between concepts and not-concepts with and without centering. Table 10: Evaluation of the similarity of antonyms and negative concepts in CLIP. Pairwise Cosine Similarity (without concept centering) Pairwise Cosine Similarity (with concept centering) Concept and antonym 0.7176 ±plus-or-minus± 0.1109 0.1366 ±plus-or-minus± 0.2197 Concept and “not” concept 0.8661 ±plus-or-minus± 0.0498 0.6130 ±plus-or-minus± 0.0498 B.11 Understanding the Image Mean for Modality Alignment In order to empirically check that the mean centering of images does not result in a loss of information, we decompose the img mean, μi⁢m⁢gsubscript _imgμitalic_i m g, that we used for all experiments. If we decompose it with uncentered concepts, the following concepts are highlighted: “closeup", “flickr", “posed". The decomposition with centered concepts results in the following concepts: “flickr", “posed", “pics", “angle view", “last post". These concepts all seem to be generally related to images, with minimal other semantic information, suggesting that centering does not remove any discriminative semantic content of embeddings, but simply removes information about the modality. B.12 Choice of Concept Vocabulary We perform a simple ablation study to assess the sensitivity of our method to choices in concept vocabulary. We collect a second vocabulary in the same exact manner as the LAION vocabulary from the MSCOCO caption dataset. We consider both the top 10k and top 5k most common words for both, and repeat the zero-shot accuracy and reconstruction cosine similarity experiments from Section 5.2 on CIFAR100. We see that the MSCOCO10k and LAION10k vocabularies perform almost exactly the same for both metrics. The smaller vocabularies perform the same for cosine reconstruction but underperform the 10k vocabularies for zero-shot classification tasks. Figure 13: Change in SpLiCE performance when considering another semantic concept dictionary derived from MSCOCO as well as a smaller concept vocabulary. B.13 Concept Type Distribution In order to better understand any biases produced by the decomposition process or that CLIP itself has, we visualize the types of concepts most commonly activated across multiple datasets, labelling them by part of speech in Figure 14. We see that nouns are by far the most common concepts across datasets, indicating that both CLIP and the decompositions are highly object centric. Note that the low weight on verbs and adjective is due to far fewer concepts of those types being activated (low l0subscript0l_0l0 norm) as well as the weight upon those concepts being significantly smaller (low l1subscript1l_1l1 norm). We hypothesize that the information in many adjective and verbs can actually be encoded into the noun itself, resulting in this phenomenon. For example, the concept “lemon" is a more succinct form of “yellow" and “fruit". Figure 14: SpLiCE decompositions are mostly comprised of nouns across multiple datasets. B.14 Experiments on Alternative CLIP Architecture We present cosine reconstruction and zero-shot accuracy experiments with an alternative CLIP architecture from OpenAI with a ResNet50 backbone for the vision encoder. Note that these experiments were done with a 10000 size vocabulary of only one-word concepts. We find that results are similar to those presented in 3, save for OpenAI’s ResNet50 CLIP performing much worse than OpenCLIP’s ViT B/32 backbone in general. Figure 15: Performance of SpLiCE decomposition representations on zero-shot classification tasks (bottom row) and cosine similarity between CLIP embeddings and SpLiCE embeddings (top row) for OpenAI’s ResNet50 CLIP model.

Instant research discovery