Paper deep dive
ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers
Effrosyni Sokli, Isaac Roberts, Alexander Schulz, Barbara Hammer, Gabriella Pasi
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/9/2026, 5:56:49 AM
Summary
The paper introduces ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers. It utilizes discriminative embedding space visualizations via DeepView and automatically extracted Concept Activation Vectors (CAVs) to analyze how expert routing influences embedding space structure and retrieval effectiveness. Experiments on five IR benchmarks demonstrate that MoE routing improves the geometric neighborhood of relevant documents and reveals distinct linguistic specializations among experts.
Entities (8)
Relation Signals (8)
DeepView → istypeof → Discriminative Dimensionality Reduction
confidence 95% · DeepView [40], a Discriminative Dimensionality Reduction (DiDi) framework
ExpertLens → uses → DeepView
confidence 95% · ExpertLens leverages discriminative embedding space visualizations derived from DeepView
SB-MoE → hasgatingmechanism → unsupervised
confidence 93% · SB-MoE trains its router in an unsupervised manner
DenseC3 → hasgatingmechanism → supervised
confidence 93% · DenseC3 instead trains its router with Cognitive Complexity labels ... providing a weakly supervised signal
ExpertLens → uses → Concept Activation Vectors
confidence 92% · ExpertLens leverages ... automatically extracted Concept Activation Vectors
ExpertLens → evaluates → SB-MoE
confidence 90% · We evaluate ExpertLens on ... two MoE-enhanced dense retrievers ... SB-MoE
ExpertLens → evaluates → DenseC3
confidence 90% · We evaluate ExpertLens on ... two MoE-enhanced dense retrievers ... DenseC3
UMAP → istypeof → Dimensionality Reduction
confidence 90% · Data visualization techniques such as ... UMAP [32] comprise popular tools
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art performance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Existing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative embedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Experiments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves embedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Analysis of expert subspaces further reveals general-purpose dominant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is publicly available.
Tags
Links
- Source: https://arxiv.org/abs/2609.06155v1
- Canonical: https://arxiv.org/abs/2609.06155v1
Trouble viewing inline? Open PDF directly →
Full Text
73,551 characters extracted from source content.
Expand or collapse full text
ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers Effrosyni Sokli effrosyni.sokli@unimib.it Department of Informatics, Systems and Communication, DISCo University of Milano-Bicocca Milan, Italy Isaac Roberts iroberts@techfak.uni-bielefeld.de CITEC - Center for Cognitive Interaction Technology Bielefeld University Bielefeld, Germany Alexander Schulz aschulz@techfak.uni-bielefeld.de CITEC - Center for Cognitive Interaction Technology Bielefeld University Bielefeld, Germany Barbara Hammer bhammer@techfak.uni-bielefeld.de CITEC - Center for Cognitive Interaction Technology Bielefeld University Bielefeld, Germany Gabriella Pasi gabriella.pasi@unimib.it Department of Informatics, Systems and Communication, DISCo University of Milano-Bicocca Milan, Italy Abstract Neural models, including dense retrievers, have been widely adopted in Information Retrieval (IR), often delivering state-of-the-art per- formance. Despite their effectiveness, these models operate as black boxes, limiting the interpretability of their ranking decisions. Exist- ing post-hoc explainability methods for neural rankers primarily focus on feature-level attributions, which can be insufficient to capture the complexity of learned embedding spaces. In this work, we propose ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retrievers that shifts focus from local scalar feature importance to representation-level global interpretability. ExpertLens leverages discriminative em- bedding space visualizations jointly with automatically extracted Concept Activation Vectors to reveal how expert routing drives embedding space formulation and retrieval effectiveness. Exper- iments across five IR benchmarks and two MoE-enhanced dense retrievers show that expert routing consistently improves em- bedding space structure, positioning queries and their relevant documents into better-defined geometric neighborhoods. Anal- ysis of expert subspaces further reveals general-purpose dom- inant experts, along with minority experts exhibiting distinct linguistic specialization, with subspaces arranged according to multi-semantic concept similarity. Our code is available at: https: //anonymous.4open.science/r/ExpertLens. CCS Concepts • Information systems→Retrieval models and ranking;• Computing methodologies→Learning paradigms;• Human- centered computing→ Information visualization. Keywords Post-Hoc Explainability, Dense Retrieval, Mixture-of-Experts 1 Introduction The recent and increasing use of neural networks in Information Re- trieval (IR) has reshaped how retrieval systems estimate relevance, moving from lexical matching toward semantic representation learning. Various IR approaches (e.g., learning-to-rank algorithms) rely on selected features to learn scoring functions offering partial interpretability to the model’s ranking decisions. Neural sparse retrievers assign learned lexical weights to query and document tokens, enabling efficient inverted-index retrieval while providing partial interpretability through token-level weight inspection [14]. However, their reliance on explicit lexical signals can limit their ability to capture deep semantic relationships. In contrast, dense retrievers encode queries and documents into high-dimensional em- beddings, capturing semantic relationships beyond explicit lexical matching [33]. While dense retrievers have demonstrated strong performance across many IR benchmarks, they behave as black boxes, making their ranking decisions difficult to interpret. Under- standing how their internal representations structure the embed- ding space and drive retrieval decisions remains an open challenge, and a necessary step toward interpretable neural IR systems. More recently, modern neural rankers and dense retrievers lever- age Mixture-of-Experts (MoE)-based architectures [19,22] to tailor general-purpose Language Models (LMs) to IR tasks. Specifically, MoE dynamically route inputs through specialized experts, allow- ing models to scale efficiently while retaining task-specific exper- tise [27,43]. In IR, MoE-enhanced retrievers have demonstrated promising gains in retrieval effectiveness, task generalization, and domain adaptation [7,17,30], establishing MoE as a prominent paradigm in neural IR. At the same time, the high-dimensional representations inherent to these models challenge the ability of traditional feature attribution methods to provide adequate expla- nations. Existing post-hoc explainability approaches for neural rankers largely operate at the feature level and have been adapted from classification to search tasks, producing attributions that es- timate the contribution of individual input features to ranking scores [18,29,37]. Understanding how ranking behavior and tex- tual representations evolve under expert specialization requires moving beyond scalar feature importance towards interpretable structure within embedding spaces. In this paper, we introduce ExpertLens, a global, representation- level post-hoc explainability framework [35] tailored to MoE- enhanced dense retrievers. Instead of assigning attribution scores arXiv:2609.06155v1 [cs.IR] 5 Sep 2026 Sokli et al. to individual input features, ExpertLens examines how expert rout- ing affects the geometry of the embedding space and organizes document representations structurally. We leverage discriminative embedding space visualizations derived from DeepView [40], a Discriminative Dimensionality Reduction (DiDi) framework that projects high-dimensional representations into interpretable, dis- criminative views. We evaluate ExpertLens on five IR benchmarks and two MoE-enhanced dense retrievers with contrasting gating mechanisms, isolating how expert routing shapes the induced em- bedding spaces. Our main contributions are: (1)We leverage DeepView to analyze the embedding spaces produced by MoE-enhanced dense retrievers, providing interpretable insights into expert routing and embedding space structure across domains and IR tasks. (2)We conduct qualitative and quantitative analyses of expert- induced subspaces, characterizing expert specialization in terms of linguistic properties and semantic themes. (3)We use automatically extracted Concept Activation Vec- tors (CAVs) [25] to associate semantic attributes with the underlying structure of the embedding space. ExpertLens provides explainable insights into both the embedding space structure and the specialization of experts. Our approach com- plements existing feature-level attribution methods to advance the post-hoc global explainability of MoE-enhanced dense retrievers. 2 Related Work and Background Explainability in IR (ExIR) has traditionally centered on feature at- tribution methods, with foundational techniques such as SHAP [29] and LIME [37] adapted to ranking tasks to quantify the contribu- tion of individual query-document features to relevance predictions. LIRME [49] extended this paradigm with local, model-agnostic in- terpretability for neural rankers, and RankingSHAP [18] formalized listwise Shapley value estimation that also accounts for interde- pendencies among ranked documents. The growing adoption of neural models in IR, especially MoE architectures [7,17,45], has drastically shifted the field. MoE models [22] introduce a gating mechanism that dynamically routes inputs through specialized experts, inducing a structured partition of the embedding space whose boundaries reflect routing decisions rather than input fea- tures. Therefore, explaining how expert routing shapes retrieval decisions requires moving beyond scalar feature importance to- wards interpretation of the embedding space geometry. Research in neural ExIR has taken steps in this direction. Fernando et al. [13]ap- plied gradient-based attribution to interpret relevance predictions in neural rankers, Völske et al. [50]provided an axiomatic perspec- tive on neural ranking behavior, while Wallat et al. [51]investigated probing techniques for understanding learned representations in BERT-based rankers, presenting a representation-level analysis of neural IR models. However, the geometry of the embedding spaces induced by these models remains underexplored. Interpreting such a structure at the concept level also calls for methods that can associate embedding directions with human- interpretable semantic attributes. CAVs [10,25] paired with sparse dictionary learning [3,15] provide faithful concept-level NLP expla- nations, making them ideal for MoE settings where experts induce subspace directions. Because this unsupervised approach requires post-hoc concept labeling [3,20], manually scaling to thousands of concepts is labor-intensive. Consequently, automated LLM-based la- beling, which generates concise labels from top-activating samples, has emerged as a promising, efficient alternative [12,20]. Despite these works, their application to IR remains underexplored, with one notable exception employing supervised CAVs to improve per- sonalized semantics in recommender systems [16]. The two MoE-enhanced IR models most relevant to this work are SB-MoE [44] and DenseC3 [45]. Both models enhance a standard dense bi-encoder with a single MoE layer placed atop the encoder backbone, where six experts receive the query or document embed- ding produced by the backbone, and their outputs are fused into a single representation via a gating-weighted sum. SB-MoE trains its router in an unsupervised manner, dynamically optimizing expert assignments based on the downstream IR tasks. The router is a learned gating function that takes the query or document em- bedding as input and produces a weight vector over the experts, indicating the relevance of each expert’s contribution to the given input. Formally, let푥be the input embedding,푓 푖 (푥)the output of the푖-th expert, and푔 푖 (푥)the weight assigned to the푖-th expert by the gating function; the final refined representation푦is the weighted sum of all experts’ outputs: 푦= 푛 ∑︁ 푖=1 푓 푖 (푥)·푔 푖 (푥)(1) During training, noisy Top-1 gating [42] ensures that all experts are explored, while during inference, the weighted sum of all ex- perts’ outputs is used to produce the final representation. DenseC3 instead trains its router with Cognitive Complexity labels derived from Bloom’s Taxonomy [1], a six-level classification of texts based on their understandability and intended usage, providing a weakly supervised signal that directly informs expert assignment. Rather than learning routing decisions from the retrieval signal alone, DenseC3’s gating mechanism is implemented as a multi-label clas- sifier pre-trained on external annotated datasets, producing com- plexity weights that route each input to the expert corresponding to each Cognitive Complexity level. Data visualization techniques such as t-SNE [31] or UMAP [32] comprise popular tools for the visualization and inspection of em- bedding space structure. Because they are unsupervised, however, they can miss relevant structure in the projections. In contrast, DeepView [40] utilizes a discriminative metric together with UMAP in order to emphasize user-relevant information. See Figure 4a and 4b for an example in the IR setting, where regular UMAP does not reveal any structure, but DeepView (DiDi UMAP) does. For- mally, the discriminative distances used in DeepView emphasize changes in a discrete variable like a class label and are defined as shortest path integrals, which locally measure the divergence be- tween probability distributions over the discrete variable on nearby positions. This is equivalent to the Fisher information metric [40] and is approximated by 푑(푥,푦) := 푛 ∑︁ 푖=1 (1− 휆)푑 퐽푆 (푓(푝 푖−1 ), 푓(푝 푖 ))+ 휆푑 푆 (푝 푖−1 ,푝 푖 )(2) ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers Phase I MoE Induced Embedding Space Structure & Analysis Phase I Subspace Disentanglement & Expert Specialization Phase I Concept Extraction & Expert Attribution Input Query & Document Embeddings MoE- Enhanced Dense Retriever ExpertLens Post-Hoc Explainability Figure 1: The ExpertLens framework. Phase I: Visualization of the MoE induced embedding space through discriminative DeepView projections for structure analysis. Phase I: Examination of the partitioning of the embedding space into subspaces and association with expert specialization. Phase I: CAVs are automatically extracted and mapped to individual experts. for two points푥,푦in the original input space,푓the classifier outputting a probability distribution over classes,푑 퐽푆 the Jensen- Shannon metric,푑 푆 an unsupervised regularization metric,푝 푖 equidistant points on the straight line from푥to푦, and휆 ∈ [0,1] balancing the discriminative and unsupervised components. This metric, together with UMAP, is referred to as discriminative dimen- sionality reduction (DiDi). In this work, we propose ExpertLens, a post-hoc explainabil- ity framework for MoE-enhanced dense retrievers that leverages DeepView to project the embedding space through the lens of each MoE model’s own routing behavior. By deriving푓from the internal gating mechanism, the discriminative boundaries encoded in the DeepView projections correspond directly to expert routing deci- sions, yielding visualizations that reflect the subspace geometry induced by MoE in a way that generic dimensionality reduction methods cannot. 3 The Proposed Framework This section motivates the adoption of DeepView as the visualiza- tion framework of ExpertLens, and the selection of SB-MoE and DenseC3 as a controlled contrastive pair of MoE-enhanced dense re- trievers for investigating MoE-induced embedding space structure within the scope of post-hoc ExIR (Section 3.1). We then describe the three phases of the ExpertLens framework (Section 3.2). 3.1 Motivation Dense retrievers project both queries and documents into a shared high-dimensional embedding space, where retrieval decisions are determined by geometric proximity. When enhanced with MoE mechanisms, these models induce distinct expert-specific subspaces, effectively imposing a structured geometry that reflects the divi- sion of representational capacity across experts. These additional subspaces, absent in standard dense retrievers, have important implications for retrieval, as the expert to which a document is routed influences the region of the embedding space it occupies and, consequently, the set of queries for which it can be retrieved. Revealing the subspace geometry induced by MoE requires a vi- sualization framework that encodes routing boundaries into its projection. As shown in Figure 4, conventional dimensionality re- duction techniques can obscure the distinct expert subspaces, as they do not leverage the model’s routing decisions. DeepView ad- dresses this directly, and within the ExpertLens setting, the classifier 푓(Eq. 2) is derived from each model’s internal gating mechanism, meaning the discriminative boundaries encoded in the projection correspond directly to expert routing decisions. Investigating how the gating mechanism shapes embedding space structure requires a controlled setting in which architectural variation is minimized, and the routing strategy is the primary variable. We therefore se- lect SB-MoE [44] and DenseC3 [45] as a controlled contrastive pair. Both models share the same single-layer MoE architecture with six experts and a gate-weighted summed output, differing solely in their gating mechanism (unsupervised in SB-MoE and super- vised in DenseC3). This architectural alignment ensures that any observed differences in embedding space structure or expert spe- cialization are attributable to the gating mechanism rather than to architectural variation. 3.2 ExpertLens Figure 1 illustrates the ExpertLens framework, which consists of three main phases: (I) MoE Induced Embedding Space Structure and Analysis, where discriminative projections of the embedding space are computed, and its geometric structure is analyzed; (I) Sub- space Disentanglement and Expert Specialization, where the expert- induced subspaces emerging from the projected embedding space are characterized and associated with expert specialization; and (I) Concept Extraction and Expert Attribution, where CAVs are auto- matically extracted and mapped to individual experts, linking their specialization to interpretable semantic concepts. 3.2.1 Phase I. In MoE-enhanced dense retrievers, a learned gat- ing mechanism assigns each input representation to one or more Sokli et al. (a) Fine-tuned (without MoE)(b) Unsupervised MoE(c) Supervised MoE Figure 2: UMAP (a) and DeepView (b & c) visualizations of the embedding space for the same query (yellow circle) and its relevant document (red circle) across three model variants: (a) Fine-tuned (without MoE), (b) Unsupervised MoE (SB-MoE), and (c) Supervised MoE (DenseC3). Each plot shows the top-1000 ranked documents retrieved for this query by each model, with colors indicating expert assignment in MoE variants. specialized experts, effectively partitioning the embedding space according to the model’s routing behavior. DeepView projects data points using the model’s decision boundaries, combining feature- wise similarity with classifier-induced distance to emphasize direc- tions in the embedding space where model predictions change [38]. The classifier distilled for DeepView 1 (푓in Eq. 2) is derived from each MoE model’s internal gating mechanism, ensuring that the dis- criminative projection reflects expert assignment boundaries rather than externally imposed class labels. From the derived DeepView visualizations, we observe that MoE-enhanced variants progres- sively reposition relevant documents into closer relative proximity to their corresponding queries compared to standard dense retriev- ers, suggesting that expert routing induces a more structured query neighborhood within the embedding space (Figure 2). To quantify how expert routing affects the organization of the embedding space, we analyze the neighborhood structure around each query from two complementary perspectives. First, we measure the proportion of relevant documents appearing among the푘nearest neighbors of each query 푞 by dot product, equivalent to Precision@푘 : P@푘(푞)= |퐷 + ∩ 푁 푘 (푞)| |푁 푘 (푞)| (3) where푁 푘 (푞)denotes the set of푘nearest neighbors (i.e., docu- ments) of푞and퐷 + its set of relevant documents. This reflects the relative positioning of relevant documents within the immediate query neighborhood. Second, we measure the absolute proximity of each relevant document푑 + to its corresponding query푞via cosine distance: 푐표푠 푑푖푠푡 (푞,푑 + )= 1− 푞·푑 + ∥푞∥·∥푑 + ∥ (4) 1 Technical details of the classifier and its distillation procedure are reported in Sec- tion 4.2. Figure 3: DeepView plot of Query ID: “181476" from MS- MARCO by the Unsupervised MoE variant (top), and Query ID: “5a7199725542994082a3e88f" from HotpotQA by the Su- pervised MoE variant (bottom). Together, these two measures allow us to assess whether MoE in- duce a better-structured embedding space by positioning relevant ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers documents into better-defined geometric neighborhoods with re- spect to their queries, comparing the unsupervised and supervised MoE implementations with standard fine-tuned retrievers. 3.2.2 Phase I. Based on the DeepView projections obtained in Phase I, we conduct a linguistic and statistical analysis of each expert-associated embedding subspace to characterize the textual properties each expert specializes in. For the linguistic profiling to yield meaningful expert-specific outcomes, the top-푘retrieved docu- ments of a given query must form clearly separated, expert-aligned clusters in the embedding space. Queries whose retrieved docu- ments are spread uniformly across experts or occupy overlapping regions provide no discriminative basis for subspace characteri- zation. Rather than manually selecting representative queries or analyzing all queries indiscriminately, we therefore introduce an automatic query scoring procedure that identifies queries whose top-푘 retrieved documents exhibit well-structured, expert-aligned clustering in the embedding space (Figure 3). For each query푞, we obtain the two-dimensional DeepView projections of its top-1000 retrieved documents from Phase I, together with the expert ID each document was routed to. We then apply DBSCAN [9] to the 2D projected space using Euclidean distance, discovering natural density-based clusters in the discriminative projection. DBSCAN makes no assumption about cluster shape or count, grouping docu- ments that lie within휀Euclidean distance of at leastmin_samples neighbors, and labeling documents that cannot be attached to any dense region as noise. We then measure how well the discovered clusters align with the expert assignments using the Adjusted Rand Index (ARI) [21], computed between the DBSCAN cluster labels and the expert IDs, restricted to non-noise documents: ARI= RI− E[RI] max(RI)− E[RI] ∈ [−1, 1](5) whereRIis the Rand Index between the two partitions, andE[RI] is its expected value under random assignments. An ARI of+1 in- dicates that DBSCAN clusters perfectly mirror expert assignments, 0 indicates no better than random agreement, and−1 indicates systematic anti-correlation. To account for the reliability of the clustering itself, we weight the ARI by the core ratio휌, defined as the fraction of non-noise documents: 휌= |푑 : label(푑)≠−1| |퐷 푞 | (6) where퐷 푞 is the set of top-1000 retrieved documents for query푞. A high휌indicates that most retrieved documents form compact, high-density clusters, making the ARI estimate more trustworthy. The composite clustering quality score for query 푞 is then: score(푞)= 휌· max(0, ARI)(7) This score is high only when the retrieved documents form com- pact clusters within the embedding space, and the expert routing correctly mirrors those clusters. Queries whose top-푘documents are spread uniformly in embedding space, or whose expert labels disagree with the natural density structure, score near 0. Queries for which DBSCAN finds fewer than two clusters are assigned a score of−1 and excluded from the analysis. We compute this score across all queries in each dataset and retain the top-100 queries per dataset per model for the subsequent linguistic profiling, pro- viding a substantial and automatically grounded empirical basis for the qualitative expert specialization analysis. Finally, for each selected query, we profile the subspace associated with each ac- tive expert using five corpus-level linguistic metrics that are light- weight, domain-agnostic, and comparable across corpora of varying sizes [2,28,36,41]. The Automated Readability Index (푎푟푖) estimates text complexity from average word and sentence length. Shannon’s Entropy (퐻) measures the average information content of tokens, normalized by vocabulary size, capturing the uniformity of the token distribution. The Relative Vocabulary Size (푅 Voc ) quantifies lexical diversity as the ratio of vocabulary size to the number of non-function tokens. 2 Corpus Predictability (퐶푃) captures sequen- tial token dependencies via a first-order Markov model, where higher values indicate more predictable token sequences. Finally, Average Sentence Length (퐿 푆 ) measures the mean number of tokens per sentence. Expert specialization is assessed per subspace and contrasted against the top-1000 retrieval pool and the full corpus baseline. 3.2.3 Phase I. While Phase I characterizes expert subspaces through linguistic properties, Phase I identifies the semantic con- cepts that structure the embedding space and evaluates how con- sistently these concepts align with expert routing decisions. To this end, we extract CAVs via Sparse Autoencoder (SAE) decomposi- tion [10], and subsequently label each concept using an LLM [12]. We decompose the푛-document embedding matrix퐴 ∈ R 푛×푑 into a sparse weight matrix푈 ∈ R 푛×푐 and a concept dictionary푉 ∈ R 푑×푐 , where 푐 ≫ 푑 forms an overcomplete basis: (푈,푉)= arg min 푈,푉 ∥퐴−푈푉 ⊤ ∥ 2 퐹 (8) Each column of푉defines a learned concept direction in the em- bedding space, and the non-zero entries of each row of푈iden- tify which concepts are active for a given document. The over- complete basis encourages sparsity, ensuring that each document is described by a small, interpretable subset of concepts. To as- sign human-interpretable labels to learned concepts, we embed the queries to obtain퐴 푞 ∈ R 푛 푞 ×푑 and project them onto the con- cept dictionary to obtain query activations푈 푞 ∈ R 푛 푞 ×푐 . For each concept푐 푖 , we rank queries by their activation score and pass the top-20 to an LLM requesting a common semantic theme. Queries are preferred over documents for labeling as they are shorter and topically focused, yielding cleaner concept descriptions. Long doc- uments often span multiple topics, risking the LLM latching onto incidental rather than defining semantic properties of a concept. We acknowledge that this simplification may cause the LLM to capture associations that are cleaner but potentially narrower than what the full document text would reveal; we therefore treat concept labels as indicative semantic summaries rather than exhaustive characterizations. The expert-concept association is determined by aggregating the non-zero concept activations per expert across the full document corpus, subsequently visualized as a Sankey diagram to reveal how semantic concepts propagate across experts (Figure 5). To evaluate whether the embedding space organizes itself around 2 The function words푁 푓 defined in Bank et al. [2]: the, a, an, he, him, she, her, they, us, we, them, it, his, to, on, above, below, before, from, in, for, after, of, with, at, and, or, but, nor, yet, so, either, neither, both, whether. Sokli et al. semantically coherent structure, we define the푄 푐표푛푐푒푝푡 (푛)error to measure how consistently the top-푛active concepts of a document are shared with its nearest neighbors. A low error indicates that geometrically proximate documents activate similar concepts, con- firming that embedding subspaces are structured along semantically coherent directions rather than organized arbitrarily. Conversely, a high error suggests that neighboring documents do not share dominant concepts, implying that the subspace geometry is not driven by semantic content. For a document푑, let푆 (푛) 푑 denote its top-푛concept activations and푁 퐾 (푑)its퐾nearest neighbors by cosine similarity: 푄 푐표푛푐푒푝푡 (푛)= 1 |푄|· 퐾 ∑︁ 푑∈푄 ∑︁ 푗∈푁 퐾 (푑) © « 1− |푆 (푛) 푑 ∩ 푆 (푛) 푗 | 푛 ª ® ¬ (9) where푄is a set of sampled documents. We evaluate푄 푐표푛푐푒푝푡 across supervised and unsupervised MoE variants to assess how routing affects the semantic coherence of the induced embedding space. 4 Experimental Setup In this section, we report the empirical evaluation conducted to address the following research questions (RQs): RQ1. How can MoE enhancements in dense retrievers affect the structure and immediate query neighborhood within the embedding space (Phase I)? RQ2. What textual characteristics of their associated subspaces define expert specialization (Phase I)? RQ3. Can CAVs reveal interpretable semantic concepts associated with the specialization of individual experts (Phase I)? 4.1 Datasets, Models & Baselines We conduct experiments on five publicly available benchmarks (Table 1) spanning two IR tasks: (i) passage retrieval and (i) open- domain question answering (Q&A) formulated as a search task [4]. TREC DL 19 & 20 share the MSMARCO corpus and provide dense relevance judgments, offering a rich signal for the embedding space structure analysis in Phase I. MSMARCO, NQ, and HotpotQA (all part of BEIR [47]) are used across all three phases of ExpertLens, providing corpus scale and topical diversity required for the per- expert subspace analysis in Phases I and I. We evaluate Ex- pertLens on two MoE-enhanced dense retrievers, SB-MoE [44] and DenseC3 [45], instantiated on four encoder backbones of varying scale and pre-training strategy (Table 2). TinyBERT and BERT- Small are obtained via knowledge distillation [39]. BERT-Base uses standard masked language modeling pre-training, and ColBERT is further fine-tuned with a late-interaction retrieval objective, making it the only backbone with IR-specific pre-training. For each back- bone, we compare three model variants: (i) Fine-Tuned (without MoE), (i) Unsupervised (SB-MoE), and (i) Supervised (DenseC3). The Fine-Tuned baseline is a standard dense bi-encoder trained without any MoE layer, using the same training data, number of epochs, learning rate, and batch size as the MoE variants, following the configurations reported in the original works [44,45], ensuring that observed differences in embedding space structure are attrib- utable to MoE routing rather than training differences. Phase I is conducted across all four backbones. Phases I and I, which require per-query analysis and SAE training, are conducted on TinyBERT to maintain comparability while managing computational costs. Table 1: Dataset statistics. The average number of relevance judgments per query is indicated in parentheses. DatasetCorpus SizeTrain QueriesTest Queries MSMARCO v1 passage corpus (MSMARCO [34])8.8M passages532k (1.1 avg rel.)7k TREC Deep Learning Track 2019 (TREC DL 19 [6])8.8M passages503k (215 avg rel.)43 TREC Deep Learning Track 2020 (TREC DL 20 [5]) 8.8M passages503k (211 avg rel.)54 Natural Questions (NQ [26])2.6M passages132k (1.2 avg rel.)3.5k HotpotQA [52]5.2M documents85k (2 avg rel.)7.4k Table 2: Backbone bi-encoder models statistics. Parameter count, embedding size, and pre-training strategy. ModelParametersEmbedding SizePre-Training Strategy TinyBERT [23]14.5M312Knowledge Distillation BERT-Small [48]29M512Knowledge Distillation BERT-Base [8]110M768Self-Supervised Learning ColBERT [24] 110M768IR Fine-Tuned (a) Regular UMAP(b) DiDi UMAP 휆 푄 data ↑ 푄 ¬푑푎푡푎 ↑ 푄 knn ↓ 0.095.6%94.7%0.0929 0.196.3%98.0%0.0979 0.294.6%95.7%0.0989 0.395.9%96.7%0.0939 0.496.4%98.0%0.0899 0.596.6%95.3%0.0959 0.697.4%95.3%0.0959 0.796.7%96.0%0.1009 0.894.7%89.0%0.1259 0.987.7%67.8%0.2028 1.080.1%48.8%0.4545 (c) 휆 values span Figure 4: Discriminative UMAP (center) reveals clear clus- ters of documents processed by the same experts for a given query compared to regular UMAP (left). Evaluation metrics across휆values for query5a78c56455429974737f7876(right). Unsupervised MoE on HotpotQA. 4.2 Phase I Setup The DeepView visualizations are produced by distilling a classifier from the gating mechanism of each model. The classifier consists of two down-projection layers, the first reducing the input from the backbone embedding size to half, and the second reducing it further to the number of experts (six). It is trained with Kullback–Leibler divergence loss against the expert logits for 30 epochs on the train- ing set of each dataset, with a 5% validation split, retaining the checkpoint with the lowest validation loss. To select the optimal휆 balancing discriminative and unsupervised components in Deep- View (Eq. 1), we perform a grid search over휆 ∈ [0,1]with a step of 0.1. For each value, we evaluate the resulting projection using three metrics [40]:푄 푑푎푡푎 and푄 ¬푑푎푡푎 , which assess how well the scatter plot aligns with the classification function for the plotted and background points respectively, and푄 푘푛 , which measures the extent to which the five nearest neighbors of a given point share the same expert. Figure 4c reports the full grid search results for a representative query. High values (휆 ≥0.8) increasingly suppress the discriminative signal, with휆=1.0 reducing to regular UMAP, which fails to reflect expert assignment boundaries as evidenced by 푄 푘푛 deteriorating sharply to 0.455 (Figure 4a). At휆=0.6,푄 푑푎푡푎 maximizes (97.4%) while푄 ¬푑푎푡푎 (95.3%) and푄 푘푛 (0.096) remain stable, and is therefore applied uniformly across all models, back- bones, and datasets (Figure 4b). The choice of Precision@푘cutoffs is ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers Table 3: Assessment of the neighborhood structure within three different embedding spaces, as these are formulated with and without the employment of two contrastive MoE variants across all datasets. Scores measure the precision@k per model and dataset at different cutoffs (%), as well as the absolute proximity of each relevant document푑 + to its corresponding query푞via cosine distance. Best results for each setting are in bold text. TREC19TREC20MSMARCONQHotpotQA RetrieverVariant P@20 P@100 cos 푑푖푠푡 ↓P@20 P@100 cos 푑푖푠푡 ↓P@1 P@2 cos 푑푖푠푡 ↓P@1 P@2 cos 푑푖푠푡 ↓P@1 P@2 cos 푑푖푠푡 ↓ TinyBERT fine-tuned.333.171.341.276.107.334.120.095 .243.129.105 .239.256.175 .349 unsupervised.341 .175.368.276.107.360.118.094.262.137.113.268.263 .178.382 supervised .330.171.417.278 .109.406.123 .095.298.148 .118.309.260.177.406 BERT-Small fine-tuned .344.184.193.268.114.191.115.097 .157.085.075 .202.308.217.263 unsupervised.349 .185.220.268.113.218.115 .097.180.088.078.225.316.222 .259 supervised.3221.181.277.274.112.275.109.094.227.097 .082.261.341 .240.286 BERT-Base fine-tuned.400.203.266.319.130.267.134 .112 .206.172.142 .210.467.324 .253 unsupervised .385.201.267.330.130.268.135.108.207.173.142.213.450.315.258 supervised.387.205.297.328.132.299.129.109.229.194 .158.228.491 .337.288 ColBERT fine-tuned.433.226.262.354.139.256.143.120 .192.200.165 .198.440.304 .255 unsupervised.429.226.267.357 .142.262.147 .122.195.205.167.204.453.314.275 supervised .452 .226.284.349.139.284.141.118.207.213 .177.214.466 .320.297 grounded in the relevance judgment density of each dataset. TREC DL 19 & 20 provide an average of over 200 relevance judgments per query; we therefore evaluate at푘 ∈ 20,100, corresponding to approximately 10% and 50% of their average relevance count, respec- tively, to capture meaningful portions of the relevant document pool within the query neighborhood. MSMARCO, NQ, and HotpotQA provide sparse judgments, with an average of only 1-2 relevant documents per query; hence, we select푘 ∈ 1,2accordingly. We report Precision@푘to capture improvements in the relative query neighborhood structure by measuring whether relevant documents are positioned closer to the query compared to the remaining re- trieved documents. Cosine distance assesses the absolute geometric proximity between queries and their relevant documents, and re- veals whether the overall geometric spread of the embedding space contracts or expands under expert routing. Together, these two metrics allow us to characterize how MoE reshape the embedding space. We opt for this formulation because our goal is to evaluate the quality of the induced embedding spaces, rather than mere re- trieval performance of the examined MoE models, which is already reported in the original works [44, 45]. 4.3 Phase I Setup For each query, we apply DBSCAN [9] to the two-dimensional Deep- View projections of the top-1000 retrieved documents obtained in Phase I, using Euclidean distance as the metric. The maximum Eu- clidean distance between two documents for one to be considered in the neighborhood of the other is defined by휀, and the minimum number of documents required to form a dense region (core point) is defined bymin_samples. We set휀=0.25 andmin_samples=7, se- lected via a small grid search to ensure that at least 100 queries with well-structured, expert-aligned partitions are retained per dataset per MoE variant for the subsequent linguistic profiling. The com- posite clustering quality score (Eq. 7) is computed for every query in each setting. Queries for which DBSCAN finds fewer than two clusters are assigned a score of−1.0, while all remaining queries are scored by휌·max(0, ARI)and ranked accordingly. The ARI between DBSCAN cluster labels and expert IDs is computed exclusively on non-noise documents (i.e., documents not labeled−1 by DBSCAN), so that they do not distort the cluster-to-expert alignment signal. The top-100 queries scored per dataset per MoE variant are then retained for linguistic profiling. For each retained query, the textual characteristics are computed separately for the set of documents assigned to each active expert, treating it as an independent cor- pus, and mean values are reported across the 100 queries. Expert specialization is assessed per subspace and contrasted against the top-1000 retrieval pool and the full corpus baseline. We consider as active those experts handling at least 10 documents of the top-1000 retrieved for a given query on average across the analyzed queries. 4.4 Phase I Setup We train a Top-푘SAE [15] with a concept dictionary of twice the embedding size (624 atoms for TinyBERT’s 312-dimensional embed- dings) and a 90% sparsity penalty, on document embeddings from both MoE variants with a learning rate of 1푒−3 and batch size of 64 for 50 epochs. We compared its푅 2 performance with an Archetypal Top-푘SAE with 10000 archetypes and the same hyperparameters; ultimately, we selected a vanilla Top-푘due to its better performance [11]. Concepts are labeled using Gemma-3-27b [46] by providing the top-20 most activating queries per concept with a zero-shot prompt requesting a common semantic theme (prompt details and discovered example concepts in Appendix A). We evaluate푄 푐표푛푐푒푝푡 at푛corresponding to 1%–10% of the dictionary size (6–62 concepts), spanning from the single most dominant concept to the full top-푘 activation set, with퐾=10 nearest neighbors. For 40 runs, we sample 20000 points and compute the푄 푐표푛푐푒푝푡 for each point and report the average. The expert-concept association is determined by the non-zero concept activations per expert aggregated across the full document corpus. 5 Results & Discussion This section presents the results of the empirical evaluations con- ducted to address our Research Questions. 5.1 RQ1: Embedding Space Structure Figure 2 presents DeepView visualizations of the embedding space derived from the three examined models for the same query and its relevant document, along with the top-1000 retrieved documents Sokli et al. Table 4: Textual characteristics analysis for Unsupervised (SB-MoE), reporting the mean of each metric across the top-100 DBSCAN-selected queries per dataset. Considering active experts only, i.e., those handling at least 10 documents for a given query on average per dataset. Bold: highest value per expert per dataset; underline: lowest value per expert per dataset. MSMARCONQHotpotQA Metric Corpus top-1000E0E3E4 Corpus top-1000E0E2E4E5 Corpus top-1000E0E1E2E3E5 Docs8.8M1000477.2338.35480.642.6M1000 732.1027.3815.94217.395.2M1000 418.9985.5913.24367.33114.24 푎푟푖 ↓8.7628.500 7.742 8.746 9.234 14.32913.521 13.653 12.618 11.48512.975 10.27010.115 10.00410.270 10.947 10.143 10.367 퐻0.5420.765 0.792 0.875 0.796 0.5620.751 0.7610.889 0.902 0.813 0.5250.736 0.7710.852 0.907 0.776 0.838 푅 Voc 0.0030.182 0.262 0.549 0.264 0.0060.180 0.2080.663 0.731 0.373 0.0130.205 0.2950.525 0.732 0.299 0.476 퐶푃0.6000.674 0.7030.798 0.707 0.5830.649 0.6590.810 0.843 0.714 0.6560.695 0.723 0.783 0.829 0.7210.767 퐿 푆 15.63514.873 14.39915.892 15.260 25.14924.519 24.777 22.567 19.73223.487 16.77017.202 17.03117.208 17.996 17.310 17.354 Table 5: Textual characteristics analysis for Supervised (DenseC3), reporting the mean of each metric across the top-100 DBSCAN-selected queries per dataset. Considering active experts only, i.e., those handling at least 10 documents for a given query on average per dataset. Bold: highest value per expert per dataset; underline: lowest value per expert per dataset. MSMARCONQHotpotQA Metric Corpus Top-1kE1E2E3E4E5 Corpus Top-1kE1E2E3E4E5 Corpus Top-1kE2E3E5 Docs8.8M100096.07146.22 484.9451.96212.272.6M100070.2593.58 528.8734.09268.465.2M100079.10451.40 451.86 푎푟푖 ↓8.762 8.839 8.337 9.471 8.505 9.347 9.398 14.329 13.051 12.904 12.358 13.573 14.819 12.583 10.270 9.993 11.023 9.252 10.508 퐻0.542 0.756 0.839 0.819 0.7780.860 0.814 0.562 0.749 0.840 0.832 0.7720.860 0.789 0.525 0.734 0.833 0.7620.767 푅 Voc 0.003 0.163 0.383 0.338 0.2230.470 0.325 0.006 0.179 0.483 0.456 0.2480.553 0.294 0.013 0.197 0.468 0.282 0.269 퐶푃0.600 0.678 0.754 0.731 0.7100.776 0.721 0.583 0.648 0.744 0.738 0.6780.766 0.683 0.656 0.697 0.760 0.739 0.708 퐿 푆 15.635 15.352 14.43916.787 14.506 16.733 16.541 25.149 24.569 24.890 22.97325.398 27.495 23.927 16.770 16.951 18.978 15.64317.893 by each model. A visual inspection reveals a progressive improve- ment in embedding space structure across model variants. The relevant document (red circle) is geometrically distant from the query (yellow circle) in the Fine-Tuned embedding space, whereas MoE-enhanced variants reposition it closer to the query, suggesting that expert routing induces a more structured query neighborhood in which semantically related content is drawn into tighter rela- tive proximity. Table 3 reports mean Precision@푘evaluated for all three models with four different backbone bi-encoders across five IR benchmarks at various cutoffs (see Section 4.2 for more details). The results corroborate the qualitative observations from the Deep- View visualizations across all backbones, i.e., the MoE-enhanced variants consistently achieve higher P@푘than Fine-Tuned. This finding indicates that expert routing reduces the relative distance between queries and their relevant documents compared to the remaining top-1000 retrieved pool in the embedding space. The Su- pervised variant generally yields stronger results, with a marked 14.78% gain over Fine-Tuned on NQ with TinyBERT at푘=1. Gains are most pronounced at tighter cutoffs (P@1), indicating that expert routing draws relevant documents into the immediate geo- metric neighborhood of the query rather than merely improving the broader embedding space structure, for a higher proportion of queries. These findings are further examined through the mean cosine distance (cos 푑푖푠푡 ) between queries and their relevant doc- uments. Interestingly, MoE-enhanced variants consistently yield higher cosine distances than Fine-Tuned, suggesting that expert routing increases the overall geometric spread of the embedding space rather than compressing it, despite the reported improve- ments in retrieval effectiveness. Notably, the increase in cosine distance is most pronounced for the Supervised variant, which simultaneously achieves the strongest P@푘gains, confirming that improved relative neighborhood structure and increased absolute spread co-occur as a direct consequence of expert-induced sub- space partitioning. Taken together, these results indicate that MoE routing consistently improves the relative positioning of relevant documents within the immediate query neighborhood as well as the overall embedding space structure for retrieval, while inducing a broader geometric spread across domains, IR tasks, and bi-encoders of varying scale and pre-training paradigms. 5.2 RQ2: Expert Specialization Tables 4 & 5 report the linguistic profiles of representative expert subspaces for both MoE variants, illustrating the expert specializa- tion patterns. Across all three datasets, the Unsupervised variant exhibits consistent and interpretable expert specialization in terms of textual characteristics. For every analyzed query, there is a dom- inant expert that absorbs the majority of the top-1000 retrieved documents and scores lowest on lexical diversity (푅 푉표푐 ), Shannon’s entropy (퐻), and corpus predictability (퐶푃), remaining close to the top-1000 and full corpus baselines. In contrast, minority experts consistently achieve the highest values on these three metrics, cap- turing lexically rich subspaces. This inverse relationship between routing load and lexical richness holds without exception for all analyzed queries across datasets and IR tasks, suggesting that the unsupervised gating mechanism induces high-volume experts that produce general-purpose subspaces for structurally average docu- ments, while low-volume experts capture text with more complex textual characteristics. The expert specialization boundary is there- fore formed by the intrinsic linguistic properties of the documents rather than by their semantic content. The Supervised variant mir- rors the specialization tendencies observed in the Unsupervised ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers Scientific & Technical Terminology Scientific & Technical TerminologyScientific & Technical Terminology Scientific & Technical Terminology Scientific & Technical Terminology Geographical Locations Geographical LocationsGeographical Locations Geographical Locations Geographical Locations Geographic & Astronomical Features Geographic & Astronomical FeaturesGeographic & Astronomical Features Geographic & Astronomical Features Geographic & Astronomical Features Canals & Waterways Canals & WaterwaysCanals & Waterways Canals & Waterways Canals & Waterways Cities and Regions - Geographic Focus Cities and Regions - Geographic FocusCities and Regions - Geographic Focus Cities and Regions - Geographic Focus Cities and Regions - Geographic Focus Biographical Details & Origins Biographical Details & OriginsBiographical Details & Origins Biographical Details & Origins Biographical Details & Origins Nationality & Geographic Origin Nationality & Geographic OriginNationality & Geographic Origin Nationality & Geographic Origin Nationality & Geographic Origin Biographical Comparison Biographical ComparisonBiographical Comparison Biographical Comparison Biographical Comparison Indian Culture & History Indian Culture & HistoryIndian Culture & History Indian Culture & History Indian Culture & History US Geography & History US Geography & HistoryUS Geography & History US Geography & History US Geography & History MOE0 MOE0MOE0 MOE0 MOE0 MOE1 MOE1MOE1 MOE1 MOE1 MOE2 MOE2MOE2 MOE2 MOE2 MOE3 MOE3MOE3 MOE3 MOE3 MOE4 MOE4MOE4 MOE4 MOE4 MOE5 MOE5MOE5 MOE5 MOE5 (a) Supervised MoE for HotpotQA Scientific & Medical Terminology Scientific & Medical TerminologyScientific & Medical Terminology Scientific & Medical Terminology Scientific & Medical Terminology Italian Geography & Culture Italian Geography & CultureItalian Geography & Culture Italian Geography & Culture Italian Geography & Culture Geographic Location of Parks and Landmarks Geographic Location of Parks and LandmarksGeographic Location of Parks and Landmarks Geographic Location of Parks and Landmarks Geographic Location of Parks and Landmarks Biographical Comparisons Biographical ComparisonsBiographical Comparisons Biographical Comparisons Biographical Comparisons Indian Cinema & Culture Indian Cinema & CultureIndian Cinema & Culture Indian Cinema & Culture Indian Cinema & Culture Geography and Locations Geography and LocationsGeography and Locations Geography and Locations Geography and Locations School & District Geography School & District GeographySchool & District Geography School & District Geography School & District Geography UK Politics & Geography UK Politics & GeographyUK Politics & Geography UK Politics & Geography UK Politics & Geography Nationality/Origin/Cultural Identity Nationality/Origin/Cultural IdentityNationality/Origin/Cultural Identity Nationality/Origin/Cultural Identity Nationality/Origin/Cultural Identity African Geography, Politics, and Ethnography African Geography, Politics, and EthnographyAfrican Geography, Politics, and Ethnography African Geography, Politics, and Ethnography African Geography, Politics, and Ethnography MOE0 MOE0MOE0 MOE0 MOE0 MOE1 MOE1MOE1 MOE1 MOE1 MOE2 MOE2MOE2 MOE2 MOE2 MOE3 MOE3MOE3 MOE3 MOE3 MOE4 MOE4MOE4 MOE4 MOE4 MOE5 MOE5MOE5 MOE5 MOE5 (b) Unsupervised MoE for HotpotQA Figure 5: Sankey diagrams displaying how concepts propa- gate through the experts. case, with one key difference. The additional weak supervision sig- nal produces a more balanced load distribution among experts, con- sistent with the routing distributions reported in the original work (Figure 6), and indicating that the supervised gating mechanism distributes documents across a finer partition of the embedding space. Despite this broader routing breadth, the inverse relationship between routing load and lexical richness holds without exception, and the dominant expert maintains metrics close to the top-1000 baseline while minority experts consistently achieve the highest 퐻,푅 푉표푐 , and퐶푃scores. The absolute metric profiles differ across datasets (HotpotQA:퐿 푆 ≈15–20, NQ:퐿 푆 ≈25–30, MSMARCO: 퐿 푆 ≈13–16), yet the relative specialization pattern holds across both MoE variants and all three corpora. These findings indicate that expert specialization is a property of the gating mechanism, with the intrinsic linguistic complexity of documents shaping how expert routing partitions the embedding space. 5.3 RQ3: Concept Extraction and Expert Attribution Figure 5 presents Sankey diagrams illustrating how the top-10 se- mantic concepts propagate across experts for both MoE variants. In the Supervised embedding space, semantic groups formed by individual concepts cannot be localized to a single expert, indicating that experts do not specialize in isolated semantic topics. As shown in Figure 5a, the dominant expert accounts for approximately two- thirds of the positive activations across nearly all concepts, aligning Figure 6: Expert assignment distributions on all datasets as reported in the original paper of the supervised MoE variant (DenseC3) [45]. 1%2%3%4%5%6%7%8%9%10% Percentage of Top Concepts 0.175 0.200 0.225 0.250 0.275 0.300 0.325 0.350 0.375 Q concept Error • MSMarco • HotpotQA • NQ SupervisedUnsupervised Figure 7: We plot the푄 concept error, including 1%-10% of the top concepts for each dataset (denoted by color) and embed- ding type (denoted by line type). with the expert routing behavior observed in RQ2 and with the gat- ing mechanism prediction distributions (Figure 6) reported by Sokli et al. [45]. A similar specialization pattern occurs in the Unsuper- vised case, albeit with a different expert assignment distribution. Figure 5b shows two primary experts sharing most of the concept activations, while activation levels are significantly lower for the remaining experts. This observation repeated itself across both MoE variants across all datasets. The broad distribution of concept activations across experts confirms that expert routing is not driven by semantic topic isolation. We hypothesize that experts do not spe- cialize in single concepts because of the intrinsic complexity of the documents, since a single document is typically characterized by the presence of multiple semantic themes. Consequently, the gating mechanism likely relies on a broader set of characteristics for expert assignment, consistent with the linguistic specialization findings of RQ2. To test this hypothesis, we analyzed the푄 푐표푛푐푒푝푡 error, as Sokli et al. depicted in Figure 7. For the supervised variant, the error is lower when evaluating a smaller number of concepts. Conversely, in the unsupervised case, a smaller number of concepts induces a higher error. The differences in error between the two variants hover around 0.11 for NQ, 0.08 for MSMARCO, and 0.05 for HotpotQA. In the supervised case, when only 1% (or 6) of the concepts are included in the set, neighboring documents share over two-thirds of them on average. This behavior remains relatively consistent regardless of the percentage of concepts included. Interestingly, the 푄 푐표푛푐푒푝푡 error for the unsupervised variant drops as the number of included concepts increases. This suggests that a broader set of concepts better explains neighborhood arrangements in the un- supervised embedding space. Notably, at the 10% threshold, the unsupervised variant demonstrates a 0.04 to 0.05 improvement over its supervised counterpart. Ultimately, this analysis reveals that document neighborhoods are shaped by underlying data com- plexities that a single, monosemantic concept cannot adequately capture. This reinforces our conclusion that individual experts do not handle isolated concepts, as the relationships between docu- ments are more complex than a single concept can explain. 6 Conclusions In this work, we introduce ExpertLens, a post-hoc explainability framework for Mixture-of-Experts (MoE)-enhanced dense retriev- ers that operates at the representation level, complementing ex- isting feature-level attribution Explainability methods in Informa- tion Retrieval (ExIR). ExpertLens leverages discriminative embed- ding space visualizations via DeepView (a discriminative variant of UMAP), for expert subspace characterization, and concept-level attribution to reveal how expert routing shapes the geometry of the embedding space. Our empirical evaluation across five IR bench- marks and two MoE-enhanced dense retrievers with contrasting gating mechanisms shows that MoE routing improves the relative neighborhood structure around queries in the embedding space, while expert-induced subspace partitioning increases the overall geometric spread of the embedding space. Expert specialization ex- hibits consistent linguistic profiles across all datasets, with general- purpose dominant experts absorbing structurally average docu- ments, and minority experts forming lexically rich and concentrated subspaces. Concept activations are broadly distributed across ex- perts, with dominant experts accounting for most activations in all settings. Document neighborhoods are shaped by a broad set of underlying semantic concepts, suggesting that expert assign- ment is driven by the linguistic properties of documents rather than by semantic isolation. Together, these findings provide an in- terpretable view of how MoE routing shapes the embedding space of dense retrievers, linking embedding space geometry and expert specialization across IR tasks. A Prompt for Concept Labeling In Figure 8, we provide the prompt given to the Gemma-3-27b model to label the discovered concepts. Prompt: Gemma-3-27b Concept Labeling You are an expert in Semantic Analysis. I am analyzing a Sparse Autoencoder trained on search queries. Below are several groups of queries. Each group activates a specific latent feature. Your task is to analyze each group and identify the shared intent, topic, or linguistic feature. Analyze the following groups of search queries and assign a semantic label. INSTRUCTIONS: • Return a valid JSON object. • Keys must be the "Group ID" strings. • Do NOT add any markdown formatting, explanation, or chatter. • Start immediately with and end with . REQUIRED JSON FORMAT: "182": "label": "Crime Films" , "196": "label": "Literary and Cultural References", "description": "Queries about authors, books, comics, dolls, and magazines." , "250": "label": "Linguistics", "description": "Queries about languages, words, and linguistic concepts." , Figure 8: The zero-shot prompt provided to Gemma-3-27b to automatically generate semantic labels and descriptions for discovered SAE concepts based on top-activating queries. The shown concepts are from the HotpotQA dataset. Acknowledgments This work has received funding from the European Union’s Hori- zon Europe research and innovation programme under the Marie Skłodowska-Curie grant agreement No 101073307. References [1]Lorin W. Anderson and David R. Krathwohl. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Longman, New York. [2]Mathias Bank, Robert Remus, and Martin Schierle. 2012. Textual Characteristics for Language Engineering. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, May 23- 25, 2012, Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Ugur Dogan, Bente Maegaard, Joseph Mariani, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA), 515–519. http://w.lrec- conf.org/proceedings/lrec2012/summaries/182.html [3]Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, et al. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread 2, 5 (2023), 6. [4]Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to Answer Open-Domain Questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1870–1879. doi:10.18653/ ExpertLens: Visualizing Embedding Spaces for Post-Hoc Explainability in MoE Enhanced Retrievers V1/P17-1171 [5]Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. CoRR abs/2102.07662 (2021). arXiv:2102.07662 https://arxiv.org/abs/2102.07662 [6]Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. CoRR abs/2003.07820 (2020). arXiv:2003.07820 https://arxiv.org/abs/2003.07820 [7]Damai Dai, Wenbin Jiang, Jiyuan Zhang, Yajuan Lyu, Zhifang Sui, and Baobao Chang. 2023. Mixture-of-Experts for Biomedical Question Answering. In Natural Language Processing and Chinese Computing - 12th National CCF Conference, NLPCC 2023, Foshan, China, October 12-15, 2023, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 14302), Fei Liu, Nan Duan, Qingting Xu, and Yu Hong (Eds.). Springer, 604–615. doi:10.1007/978-3-031-44693-1_47 [8]Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computa- tional Linguistics, 4171–4186. doi:10.18653/V1/N19-1423 [9]Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al.1996. A density- based algorithm for discovering clusters in large spatial databases with noise. In kdd, Vol. 96. 226–231. [10]Thomas FEL, Victor Boutin, Louis Béthune, Remi Cadene, Mazda Moayeri, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. 2023. A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Glober- son, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 54805–54818. [11] Thomas Fel, Ekdeep Singh Lubana, Jacob S. Prince, Matthew Kowal, Victor Boutin, et al.2025. Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models. In ICML. [12]Nils Feldhus and Laura Kopf. 2025. Interpreting Language Models Through Concept Descriptions: A Survey. In Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, Yonatan Belinkov, Aaron Mueller, Najoung Kim, Hosein Mohebbi, Hanjie Chen, Dana Arad, and Gabriele Sarti (Eds.). Association for Computational Linguistics, Suzhou, China, 149–162. doi:10.18653/v1/2025.blackboxnlp-1.8 [13] Zeon Trevor Fernando, Jaspreet Singh, and Avishek Anand. 2019. A study on the Interpretability of Neural Retrieval Models using DeepSHAP. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, Benjamin Piwowarski, Max Chevalier, Éric Gaussier, Yoelle Maarek, Jian-Yun Nie, and Falk Scholer (Eds.). ACM, 1005–1008. doi:10.1145/3331184.3331312 [14]Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). ACM, 2288– 2292. doi:10.1145/3404835.3463098 [15]Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025. Scaling and evaluating sparse autoencoders. In The Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id=tcsZt9ZNKD [16]Christina Göpfert, Alex Haig, Chih-Wei Hsu, Yinlam Chow, Ivan Vendrov, Tyler Lu, Deepak Ramachandran, Hubert Pham, Mohammad Ghavamzadeh, and Craig Boutilier. 2024. Discovering Personalized Semantics for Soft Attributes in Rec- ommender Systems Using Concept Activation Vectors. ACM Trans. Recomm. Syst. 2, 4, Article 30 (July 2024), 37 pages. doi:10.1145/3658675 [17]Jiafeng Guo, Yinqiong Cai, Keping Bi, Yixing Fan, Wei Chen, Ruqing Zhang, and Xueqi Cheng. 2025. CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval. ACM Trans. Inf. Syst. 43, 2 (2025), 35:1–35:25. doi:10.1145/3678880 [18]Maria Heuss, Maarten de Rijke, and Avishek Anand. 2025. RankingSHAP - Faith- ful Listwise Feature Attribution Explanations for Ranking Models. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, SIGIR 2025, Padua, Italy, July 13-18, 2025, Nicola Ferro, Maria Maistro, Gabriella Pasi, Omar Alonso, Andrew Trotman, and Suzan Verberne (Eds.). ACM, 381–391. doi:10.1145/3726302.3729971 [19]Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th In- ternational Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA (Proceedings of Machine Learning Research, Vol. 97), Ka- malika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 2790–2799. http: //proceedings.mlr.press/v97/houlsby19a.html [20]Robert Huben, Hoagy Cunningham, Logan Riggs Smith, Aidan Ewart, and Lee Sharkey. 2024. Sparse Autoencoders Find Highly Interpretable Features in Lan- guage Models. In The Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=F76bwRSLeK [21]Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification 2, 1 (1985), 193–218. [22] Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. 1991. Adaptive Mixtures of Local Experts. Neural Comput. 3, 1 (1991), 79–87. doi:10.1162/NECO.1991.3.1.79 [23]Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. TinyBERT: Distilling BERT for Natural Language Un- derstanding. In Findings of the Association for Computational Linguistics: EMNLP 2020, Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 4163–4174. doi:10.18653/v1/2020.findings-emnlp.372 [24]Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, Jimmy X. Huang, Yi Chang, Xueqi Cheng, Jaap Kamps, Vanessa Murdock, Ji-Rong Wen, and Yiqun Liu (Eds.). ACM, 39–48. doi:10.1145/3397271.3401075 [25]Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al.2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In ICML. PMLR, 2668–2677. [26]Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural Questions: A Benchmark for Question Answering Research. Transactions of the Association for Computational Linguistics 7 (08 2019), 453–466. doi:10.1162/tacl_a_00276 [27] Bo Li, Yifei Shen, Jingkang Yang, Yezhen Wang, Jiawei Ren, Tong Che, Jun Zhang, and Ziwei Liu. 2023. Sparse Mixture-of-Experts are Domain Generalizable Learners. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview. net/forum?id=RecZ9nB9Q4 [28] Yuliang Liu, Zhiwei Jiang, Yafeng Yin, Cong Wang, Sheng Chen, Zhaoling Chen, and Qing Gu. 2023. Unsupervised Readability Assessment via Learning from Weak Readability Signals. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2023, Taipei, Taiwan, July 23-27, 2023, Hsin-Hsi Chen, Wei-Jou (Edward) Duh, Hen- Hsen Huang, Makoto P. Kato, Josiane Mothe, and Barbara Poblete (Eds.). ACM, 1324–1334. doi:10.1145/3539618.3591695 [29]Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 4765–4774.https://proceedings.neurips.c/paper/2017/hash/ 8a20a8621978632d76c43dfd28b67767-Abstract.html [30] Guangyuan Ma, Xing Wu, Peng Wang, and Songlin Hu. 2023. CoT-MoTE: Ex- ploring ConTextual Masked Auto-Encoder Pre-training with Mixture-of-Textual- Experts for Passage Retrieval. CoRR abs/2304.10195 (2023). arXiv:2304.10195 doi:10.48550/ARXIV.2304.10195 [31]Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605. [32]Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform man- ifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018). [33]Bhaskar Mitra and Nick Craswell. 2018. An Introduction to Neural Information Retrieval. Found. Trends Inf. Retr. 13, 1 (2018), 1–126. doi:10.1561/1500000061 [34]Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proceedings of the Workshop on Cogni- tive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, December 9, 2016 (CEUR Workshop Proceedings, Vol. 1773), Tarek Richard Besold, Antoine Bordes, Artur S. d’Avila Garcez, and Greg Wayne (Eds.). CEUR-WS.org. https://ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf [35]Juri Opitz, Lucas Moeller, Andrianos Michail, Sebastian Padó, and Simon Clematide. 2025. Interpretable text embeddings and text similarity explana- tion: A survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 22314–22330. [36]Filippo Pallucchini, Xulang Zhang, Rui Mao, and Erik Cambria. 2025. Self- explanatory and Retrieval-augmented LLMs for Financial Sentiment Analysis. In Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing, SAC 2025, Catania International Airport, Catania, Italy, 31 March 2025 - 4 April 2025, Jiman Hong, Sebastiano Battiato, Christian Esposito, Juw Won Park, and Adam Przybylek (Eds.). ACM, 131–137. doi:10.1145/3672608.3707894 Sokli et al. [37]Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, Balaji Krishnapuram, Mohak Shah, Alexander J. Smola, Charu C. Aggarwal, Dou Shen, and Rajeev Rastogi (Eds.). ACM, 1135–1144. doi:10.1145/2939672.2939778 [38] Isaac Roberts, Alexander Schulz, Sarah Schröder, Luca Hermes, and Barbara Ham- mer. 2026. Targeted Visualization of the Backbone of Encoder LLMs. In Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Mattia Cerrato, Danguol ̇ e Kalinauskait ̇ e, Mantas Lukoševičius, Mykola Pechenizkiy, and Kristina Šutien ̇ e (Eds.). Springer Nature Switzerland, Cham, 82–97. [39]Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for Thin Deep Nets. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6550 [40]Alexander Schulz, Fabian Hinder, and Barbara Hammer. 2020. DeepView: Visual- izing Classification Boundaries of Deep Neural Networks as Scatter Plots Using Discriminative Dimensionality Reduction. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, Christian Bessiere (Ed.). ijcai.org, 2305–2311. doi:10.24963/IJCAI.2020/319 [41]Richard J Senter and Edgar A Smith. 1967. Automated readability index. Technical Report. [42]Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Con- ference Track Proceedings. OpenReview.net. https://openreview.net/forum?id= B1ckMDqlg [43]Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Y. Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, and Denny Zhou. 2024. Mixture-of-Experts Meets Instruction Tuning: A Winning Combination for Large Language Models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https://openreview.net/forum?id= 6mLjDwYte5 [44]Effrosyni Sokli, Pranav Kasela, Georgios Peikos, and Gabriella Pasi. 2025. Mixture of Experts Approaches in Dense Retrieval Tasks. CoRR abs/2510.15683 (2025). arXiv:2510.15683 doi:10.48550/ARXIV.2510.15683 [45] Effrosyni Sokli, Georgios Peikos, Pranav Kasela, and Gabriella Pasi. 2025. Lever- aging Cognitive Complexity of Texts for Contextualization in Dense Retrieval. In Proceedings of the 2025 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2025, Suzhou, China, November 4-9, 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, 27083–27096. doi:10.18653/V1/2025. EMNLP-MAIN.1377 [46] Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieil- lard, Ramona Merhej, Sarah Perrin, et al.2025. Gemma 3 Technical Report. arXiv:2503.19786 [cs.CL] https://arxiv.org/abs/2503.19786 [47]Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models. CoRR abs/2104.08663 (2021). arXiv:2104.08663 https://arxiv.org/abs/2104.08663 [48] Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Well- read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962 (2019). [49]Manisha Verma and Debasis Ganguly. 2019. LIRME: Locally Interpretable Ranking Model Explanation. In Proceedings of the 42nd International ACM SI- GIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, Benjamin Piwowarski, Max Chevalier, Éric Gaussier, Yoelle Maarek, Jian-Yun Nie, and Falk Scholer (Eds.). ACM, 1281–1284. doi:10.1145/3331184.3331377 [50]Michael Völske, Alexander Bondarenko, Maik Fröbe, Benno Stein, Jaspreet Singh, Matthias Hagen, and Avishek Anand. 2021. Towards Axiomatic Explanations for Neural Ranking Models. In ICTIR ’21: The 2021 ACM SIGIR International Conference on the Theory of Information Retrieval, Virtual Event, Canada, July 11, 2021, Faegheh Hasibi, Yi Fang, and Akiko Aizawa (Eds.). ACM, 13–22. doi:10. 1145/3471158.3472256 [51]Jonas Wallat, Fabian Beringer, Abhijit Anand, and Avishek Anand. 2023. Probing BERT for Ranking Abilities. In Advances in Information Retrieval - 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2-6, 2023, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 13981), Jaap Kamps, Lorraine Goeuriot, Fabio Crestani, Maria Maistro, Hideo Joho, Brian Davis, Cathal Gurrin, Udo Kruschwitz, and Annalina Caputo (Eds.). Springer, 255–273. doi:10.1007/978-3-031-28238-6_17 [52]Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 2369–2380. doi:10.18653/v1/D18- 1259