Paper deep dive

CeQe: Grounding Lexical Retrieval in Semantic Evidence

Adam Kahirov, Umesh Deshpande, Swaminathan Sundararaman

Year: 2026Venue: arXiv preprintArea: cs.IRType: PreprintEmbeddings: 32

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 8/4/2026, 4:28:08 AM

Summary

The paper introduces Cross-Encoder Query Expansion (CE-QE), a method to bridge the semantic vocabulary gap in lexical retrieval (BM25). CE-QE uses per-token relevance attributions from a cross-encoder applied to top semantic search results to select decisive terms, which are then appended to the BM25 query. This approach avoids the query drift of classical pseudo-relevance feedback and the hallucination risks of generative expansion. The authors also propose Semantically Enriched Score Fusion (SESF), which fuses the enriched lexical results with semantic scores. Evaluations on seven BEIR datasets show significant improvements in lexical recall and nDCG@10 compared to standard BM25, RRF fusion, SPLADEv2, and ColBERTv2.

Entities (16)

BM25 · algorithm · 99%CE-QE · method · 98%BEIR · dataset · 97%Cross-encoder · model · 95%MSMARCO · dataset · 95%NDCG@10 · metric · 95%NQ · dataset · 95%Recall@100 · metric · 95%SESF · method · 95%TREC-COVID · dataset · 95%ColBERTv2 · model · 90%RUMIR · pipeline · 90%SPLADEv2 · model · 90%HyDE · method · 85%Query2doc · method · 85%RM3 · method · 85%

Relation Signals (10)

CE-QE → evaluatedon → BEIR

confidence 95% · On seven BEIR datasets, CE-QE improves lexical recall substantially

CE-QE → improves → BM25

confidence 95% · CE-QE improves lexical recall substantially... leaving the underlying BM25 index completely unmodified

CE-QE → uses → Cross-encoder

confidence 95% · CE-QE... reads the per-token relevance attributions of a cross-encoder applied to top semantic search results

CE-QE → avoids → Hallucination

confidence 90% · cannot introduce vocabulary the corpus does not contain

CE-QE → avoids → query drift

confidence 90% · avoiding self-reinforcing query drift

HyDE → istypeof → generative query expansion

confidence 90% · recent generative query expansion (HyDE, Query2doc)

RM3 → istypeof → pseudo-relevance feedback

confidence 90% · classical pseudo-relevance feedback (RM3)

CE-QE → outperforms → SPLADEv2

confidence 90% · beats SPLADEv2... by 5.3%... on nDCG@10

CE-QE → outperforms →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Lexical retrieval (BM25) captures exact keyword matches and weights terms by corpus-wide significance, but it is blind to the semantic vocabulary gap: when a relevant document phrases an answer differently from the query, BM25 never retrieves it, and no amount of downstream reranking or fusion can recover a document that was never in the candidate set. We present Cross-Encoder Query Expansion (CE-QE), which reads the per-token relevance attributions of a cross-encoder applied to top semantic search results, selects the terms the cross-encoder treats as decisive, and appends them to the BM25 query. Unlike classical pseudo-relevance feedback, which reuses BM25's own (possibly wrong) top results, CE-QE seeds expansion from the semantic retriever's results, avoiding self-reinforcing query drift. Unlike recent generative query expansion (HyDE, Query2doc), which prompts a large language model to hallucinate text from its parametric knowledge, every CE-QE expansion term is copied verbatim from a retrieved passage, so it cannot introduce vocabulary the corpus does not contain, and its only added cost is attribution extraction on a cross-encoder a hybrid pipeline already runs for reranking. On seven BEIR datasets, CE-QE improves lexical recall substantially where query and answer vocabulary diverge (e.g., NQ Recall@100 from 0.32 to 0.47), and its score-fusion variant (SESF) beats cross-encoder score fusion by 2.5% on Recall@100 and beats SPLADEv2 and ColBERTv2 by 5.3% and 4.6% on nDCG@10, while leaving the underlying BM25 index completely unmodified.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

30,705 characters extracted from source content.

Expand or collapse full text

CeQe: Grounding Lexical Retrieval in Semantic Evidence Adam Kahirov, Umesh Deshpande, Swaminathan Sundararaman IBM Research Abstract. Lexical retrieval (BM25) captures exact keyword matches and weights terms by corpus-wide significance, but it is blind to the semantic vocabulary gap: when a relevant document phrases an answer differently from the query, BM25 never retrieves it, and no amount of downstream reranking or fusion can recover a document that was never in the candidate set. We present Cross-Encoder Query Expansion (CE-QE), which reads the per-token relevance attributions of a cross-encoder applied to top semantic search results, selects the terms the cross-encoder treats as decisive, and appends them to the BM25 query. Unlike classical pseudo-relevance feedback, which reuses BM25’s own (possibly wrong) top results, CE-QE seeds expansion from the semantic retriever’s results, avoiding self-reinforcing query drift. Unlike recent generative query expansion (HyDE, Query2doc), which prompts a large language model to hallucinate text from its parametric knowledge, every CE-QE expansion term is copied verbatim from a retrieved passage, so it cannot introduce vocabulary the corpus does not contain, and its only added cost is attribution extraction on a cross-encoder a hybrid pipeline already runs for reranking. On seven BEIR datasets, CE-QE improves lexical recall substantially where query and answer vocabulary diverge (e.g., NQ Recall@100 from 0.32 to 0.47), and its score-fusion variant (SESF) beats cross-encoder score fusion by 2.5% on Recall@100 and beats SPLADEv2 and ColBERTv2 by 5.3% and 4.6% on nDCG@10, while leaving the underlying BM25 index completely unmodified. Keywords: query expansion, cross-encoder reranking, hybrid search, BM25, dense retrieval, BEIR, retrieval-augmented generation. 1 Introduction Hybrid retrieval that fuses lexical (BM25) and dense seman- tic search is now the default for retrieval-augmented workloads, because each retriever recovers what the other misses: lexical matching captures exact terms and corpus-wide term signifi- cance, while dense retrieval captures synonyms and paraphrase [2]. The value of the lexical stage is concrete—it captures ex- act matches (a dense model can rank “Apple” the company and “apple” the fruit as near-identical) and weights rare, discrim- inative terms by their corpus-wide significance—but it has a structural weakness that fusion alone does not fix. The problem. BM25 retrieves only documents that share surface terms with the query. When a relevant document phrases the answer differently (“side effects” vs. “adverse reactions”, “ibuprofen” vs. “NSAID”), lexical search misses it entirely, and simply fusing its result list with a semantic list does not repair the lexical ranking itself: fusion and reranking can only reorder or combine documents that were retrieved in the first place. If a relevant document shares no surface terms with the query, BM25 never puts it in the candidate set, and no amount of downstream re-scoring can recover it—the recall ceiling is fixed at the first stage. This paper contributes a query-side fix for this problem, together with an account of why it works and how it compares to the two nearest alternatives. •Cross-Encoder Query Expansion (CE-QE) (Section 3): a query-expansion method that reads the per-token relevance attributions of a cross-encoder applied to top semantic re- sults, selects the terms the cross-encoder treats as decisive, and appends them to the BM25 query. This injects seman- tic signal into the lexical stage directly, without modifying the index. Its fusion variant, Semantically Enriched Score Fusion (SESF), combines the enriched lexical results with semantic scores and delivers the best overall quality. •A comparison to the two nearest alternatives (Sec- tion 2.1): classical pseudo-relevance feedback (RM3), which is self-referential in a way that reinforces exactly the failure CE-QE targets, and recent generative query ex- pansion (HyDE, Query2doc), which expands a query with text an LLM hallucinates from its parametric knowledge rather than from the corpus itself, at the cost of a separate generation call per query. CE-QE expands the query only with terms already present in passages the corpus’s own semantic retriever returned, using a cross-encoder a hybrid pipeline already runs for reranking, which grounds every expansion term in the corpus and adds no separate model call. We evaluate CE-QE and SESF on BEIR through RUMIR, a reproducible pipeline built for this work (Section 4), against standard flat BM25 as the lexical stage; a companion paper addresses lexical retrieval’s separate scaling problem at billion- chunk scale, and Section 5 discusses how the two compose. 2 Background and Related Work Lexical retrieval. BM25 scores a documentDfor queryQ as a sum over query terms of an IDF weight times a saturated, 1 arXiv:2608.00452v1 [cs.IR] 1 Aug 2026 length-normalized term-frequency factor [1]. Term saturation (parameterk 1 ) stops a single repeated term from dominating; length normalization (parameterb) discounts long documents. Dense retrieval and ANN. Bi-encoders encode text once and search by similarity [6,8], using approximate nearest- neighbor indexes such as HNSW [4] or IVF with product quantization [5] to remain tractable at scale. Fusion. Two ranked lists are merged by combining ranks or scores. Reciprocal Rank Fusion (RRF) sums1/(k + rank i ) across lists, needs no tuning, and ignores raw scores [3]; it is used by Elasticsearch and LanceDB. Weighted fusion com- bines normalized scores asα L(c)+(1−α) S(c)(OpenSearch, Elasticsearch) but is sensitive to score normalization because BM25 and cosine live on different scales. Reranking. Cross-encoders jointly encode the query and each candidate to produce a precise relevance score; they raise quality but cost far more than first-stage retrieval, motivating two-stage pipelines that rerank only a small candidate set [7]. Query expansion. Classical pseudo-relevance feedback (e.g., relevance models / RM3 [11]) expands a query with terms that co-occur in top-ranked documents, using term statis- tics. CE-QE differs in the selection signal: it picks expansion terms from a cross-encoder’s per-token relevance attributions rather than from co-occurrence counts, so the added terms are those a supervised relevance model treats as decisive in the query–passage interaction. Learned sparse models such as SPLADEv2 [9] and late-interaction models such as Col- BERTv2 [10] attack the same vocabulary gap by changing the index; CE-QE leaves the BM25 index unchanged and expands the query instead. A more recent line of work expands queries generatively: HyDE prompts an LLM to hallucinate a hypo- thetical answer document and embeds it for dense retrieval [13], and Query2doc few-shot-prompts an LLM to write a pseudo-document that is appended to the query for both sparse and dense retrieval [12]. Section 2.1 contrasts CE-QE with this generative family directly. Evaluation. BEIR is the standard heterogeneous, zero-shot IR benchmark [2]. We report Recall@100 (coverage) and nDCG@10 (ranking quality); the two diverge—two rankings with identical Recall@5 can have very different nDCG@5 (e.g., 1.0 vs. 0.62) depending on where the relevant items land—so we track both throughout. Retrieval quality of this kind underlies retrieval-augmented generation broadly [14], which is the deployment setting motivating this work. 2.1 Positioning relative to recent work CE-QE vs. generative (LLM) query expansion. HyDE [13] and Query2doc [12] both expand a query using text an LLM generates from its own parametric knowledge: a hy- pothetical answer document or pseudo-document that may contain no terms actually present in the target corpus, and that costs a full LLM decoding pass per query (typically on the order of a paragraph of generated tokens). CE-QE expands the query with terms extracted from passages the corpus’s own semantic retriever actually returned, using attribution weights from a cross-encoder that a hybrid pipeline already runs for reranking (Section 3.3). This has two concrete consequences. First, grounding: every CE-QE expansion term is copied verba- tim from a retrieved passage, so it cannot introduce vocabulary the corpus does not contain, whereas a generated hypothetical document can drift into fluent but corpus-absent phrasing—a documented failure mode of generative expansion on unfamil- iar or ambiguous queries. Second, cost: CE-QE adds attribution extraction on a component already in the serving path (Table 2 shows the full SESF pipeline, including this step, at∼557 ms), whereas generative expansion adds a separate LLM call whose latency and expense scale with generated length and are in- curred independently of any reranking the pipeline already performs. The two approaches are not mutually exclusive— an LLM-generated expansion could be fed through the same attribution-based filtering CE-QE uses to select terms from it before appending them to the BM25 query—but CE-QE’s grounding and marginal-cost advantages come specifically from sourcing expansion terms from retrieval rather than gen- eration. CE-QE vs. classical pseudo-relevance feedback. RM3 and relatives [11] expand a query with terms drawn from the top documents the same lexical retriever already returned—self- referential in a way that hurts exactly the queries CE-QE tar- gets, since if BM25’s top results are already wrong because of a vocabulary gap, feeding terms back from those same wrong results reinforces the error rather than correcting it (query drift). CE-QE instead seeds expansion from the semantic retriever’s top passages, which remain a trustworthy seed set precisely in the cases where BM25’s own results are not (Section 3). 3 Cross-Encoder Query Expansion This section motivates each design choice before describing the mechanism, because the choices are not arbitrary—each responds to a specific way naive alternatives fail. 3.1 Why fix the query, not just the ranking The standard way to add semantic awareness to a lexical result is to rerank it: retrieve with BM25, then re-score the candidates with a cross-encoder or fuse them with a seman- tic list. Reranking and fusion, however, can only reorder or combine documents that were retrieved in the first place. If a relevant document shares no surface terms with the query, BM25 never puts it in the candidate set, and no amount of downstream re-scoring can recover it—the recall ceiling is fixed at the first stage. This is precisely the vocabulary-gap failure mode motivating this paper (Section 1): lexical search retrieves the wrong set, not just ranks the right set poorly. Fix- ing it therefore requires intervening before retrieval, at the query itself, so that the missing document has a chance to be retrieved at all. CE-QE is a query-side fix for exactly this reason: it changes what BM25 searches for, not how BM25’s results are consumed afterward. 2 3.2Why semantic passages, not BM25’s own top results, seed expansion Classical pseudo-relevance feedback (RM3 and relatives [11]) expands the query with terms drawn from the top docu- ments that the same lexical retriever already returned. This is self-referential in a way that hurts exactly the queries CE-QE targets: if BM25’s top results are wrong because of a vocabu- lary gap, feeding terms back from those same wrong results reinforces the error rather than correcting it (query drift). CE- QE instead seeds expansion from the semantic retriever’s top passages. Dense retrieval finds topically related passages by meaning, independent of surface term overlap, so its top re- sults remain a trustworthy seed set precisely in the cases where BM25’s own results are not. This is the central design insight: use the retriever that does not have the failure mode to repair the retriever that does. 3.3 Why cross-encoder attribution, not raw term statistics Given a set of semantically relevant passages, the next ques- tion is which of their words to add to the query. A pseudo- relevance-feedback-style answer would score candidate terms by frequency or TF-IDF within those passages. This is a weak signal here: a passage relevant to “side effects of ibuprofen” also contains many generic, high-frequency words (include, common, effects itself) that are not what makes the passage relevant, and frequency statistics cannot tell the two apart. A cross-encoder, by contrast, is trained end-to-end to predict query–passage relevance, and its per-token attributions reveal which tokens the model actually used to make that judgment. For the query “side effects of ibuprofen” and the passage “Com- mon side effects of ibuprofen include nausea and dizziness”, the cross-encoder assigns high attribution to nausea (0.94) and dizziness—content terms absent from the query—and low attribution to the generic scaffolding around them. This is a supervised, discriminative-term selector obtained for free from a component the pipeline needs anyway for reranking, rather than a separate model that must be trained or tuned. A second reason to prefer attribution over whole-passage injection is query compactness. Appending an entire top pas- sage to the query would dilute BM25’s IDF weighting with a burst of common terms and inflate query length past the point where term-frequency saturation (k 1 ) still discriminates—long, diffuse queries under-reward the terms that matter. Extracting only a handful of high-attribution tokens keeps the expanded query short and each added term individually discriminative, which is what BM25’s scoring function is designed to exploit. 3.4 The CE-QE algorithm CE-QE (Figure 1) runs semantic search for the original query, retrieves the top passages, applies the cross-encoder, and extracts the highest-attribution content tokens across those passages (e.g., nausea, NSAID, pain). It appends the extracted tokens to the original query and runs BM25 on the expanded query. In our experiments up to 10 tokens are extracted from up to 10 documents; both limits are deliberate rather than incidental. Bounding the number of source passages caps the “side effects of ibuprofen” Semantic search→ top passages Cross-encoder token attribution extract terms: [nausea] [NSAID] [pain] expanded query: “. . . ibuprofen nausea NSAID pain” BM25 on expanded query enriched lexical top-k→ fuse / rerank Figure 1. CE-QE. Cross-encoder attributions over top semantic pas- sages select discriminative terms that are appended to the BM25 query, transferring semantic signal into lexical retrieval without changing the index. risk of topic drift—attribution scores taper quickly past the first few passages, and terms drawn from lower-ranked, less relevant passages are increasingly likely to be generic or off- topic—while bounding the token budget keeps the expanded query short for the reason given in Section 3.3. Terms are appended to the original query rather than replacing it, so the original terms keep contributing their own IDF weight and BM25 can still reward an exact match; expansion only adds recall opportunities, it does not remove the precision the original query already had. The BM25 index itself is untouched; only the query changes. This is a deliberate separation of concerns: selecting expansion terms is the one part of the pipeline that needs a neural model, and it runs once per query over a small, bounded set of candi- date passages (at most 10) rather than over the corpus, so its cost does not grow with corpus size regardless of which BM25 index answers the expanded query. In this paper we evaluate CE-QE against standard, flat BM25 (Section 4.1); we have not run CE-QE against a hierarchical, billion-scale BM25 index end-to-end. The separation of concerns is what makes that combination architecturally straightforward—the expanded query is ordinary BM25 input, so it needs no special handling by any particular lexical index’s internal structure—but we present it as a compositional argument in Section 5 rather than as a measured result. CE-QE targets a specific failure mode. Consider the query “Brown State Fishing Lake is in a country that has a population of how many inhabitants?” whose answer document is about “Brown County, Kansas . . . the county population was 9,984.” Lexical search fixates on population/country; semantic search drifts to lakes/countries. CE-QE extracts county, kansas, cen- sus from the top passages and adds them to the BM25 query, which then matches the answer document that neither stage found alone. 3 3.5Why the benefit concentrates on high vocabulary- divergence datasets CE-QE’s mechanism predicts where it should help: datasets where queries and their answer passages are phrased in dif- ferent registers should benefit most, and datasets where they already share vocabulary should benefit least. This matches the evaluation (Section 4). Natural Questions and TREC-COVID pair informal or lay-phrased queries against encyclopedic or technical-scientific answer text, respectively—a wide regis- ter gap that expansion closes. FEVER, by contrast, is a fact- verification dataset whose claims are written to closely para- phrase the Wikipedia sentences they check against, so query and answer vocabulary already overlap and expansion has little left to add (and, as Section 4 shows, can slightly hurt by adding terms the original query did not need). This dataset-dependent pattern is evidence that CE-QE is doing what it is designed to do—closing a vocabulary gap—rather than improving retrieval through some unrelated, dataset-independent effect. 3.6 Semantically Enriched Score Fusion (SESF) SESF uses CE-QE as the lexical stage of a score-fusion hybrid: the enriched BM25 results are combined with the se- mantic results, and the merged set is reranked. The motivation for fusing rather than using CE-QE’s lexical list alone is that fusion and expansion attack the same problem from different angles and their gains are not redundant: expansion raises the ceiling on what BM25 can retrieve, while fusion still con- tributes independent evidence from the semantic embedding whenever the two retrievers disagree. Because the lexical list going into fusion is now itself semantically aware, the two lists overlap more on genuinely relevant documents than plain BM25+vector fusion would, giving the fusion step—and the reranker downstream—a stronger and more concentrated candidate pool to work with. A reranker can only reorder the candidates it is given, so improving the pool improves the ceiling on what reranking can achieve, which is why SESF’s gains over CE score fusion (Section 4) show up primarily as higher Recall@100 rather than as a change in how well already-retrieved items are ranked. 4 Evaluation 4.1 Setup Experiments run through RUMIR, a pipeline built for this work that indexes BEIR datasets, executes lexical, semantic, fusion, reranking, and expansion variants, and evaluates them consistently. Table 1 lists the seven BEIR datasets; each config- uration is evaluated over 1,000 queries at top-k=100. Embed- dings usegranite-embedding-30m-english; the seman- tic index is IVF_PQ (LanceDB) or HNSW (OpenSearch). Rerankers arebge-reranker-v2-m3(568M params) and ms-marco-MiniLM-L12-v2(33.4M params, the OpenSearch default). The lexical stage in every result below—BM25, RRF, CE score fusion, CE-QE, and SESF alike—is standard, flat BM25 over each dataset’s own (at most 8.8M-document) cor- pus. Table 1. BEIR datasets used in the evaluation. DatasetDocumentsDomain MSMARCO8.8MWeb / QA HotpotQA5.2MWikipedia NQ2.7MWikipedia FEVER5.4MWikipedia (fact check) Climate-FEVER5.4MClimate claims TREC-COVID171KBiomedical FiQa57KFinance MSMARCOHotpotQANQFEVERClimate FEVER TREC COVID FiQa 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Recall@100 BM25SemanticRRF (Hybrid) Figure 2. Recall@100 for BM25, semantic (IVF_PQ), and their RRF hybrid across seven BEIR datasets. Hybrid never loses and often wins substantially, e.g. MSMARCO 0.66→0.80 and NQ 0.75→0.92. 4.2 Hybrid fusion vs. single retrievers Fusing BM25 with semantic search via RRF improves Re- call@100 on every dataset (Figure 2): on average∼4.3% over semantic-only and∼10% over BM25-only, with the largest gains where the two retrievers disagree most (MSMARCO, FiQa). Comparing the two fusion strategies, RRF and weighted fusion reach near-identical Recall@100, but weighted fusion ranks better—higher nDCG@10 on five of seven datasets (Fig- ure 3)—because it preserves score magnitude that RRF dis- cards. 4.3 Two-stage reranking: gain and cost Adding a cross-encoder reranker (CE score fusion) over the fused candidates raises quality, and the gain is larger in nDCG@10 than in Recall@100—the reranker mostly reorders items already retrieved rather than surfacing new ones. On TREC-COVID, for instance, nDCG@10 rises from 0.71 (RRF) to 0.79 while Recall@100 rises from 0.70 to 0.83. The cost is the cross-encoder pass, which scales with the number of passages reranked (Figure 4): at 128 passages, MiniLM costs 43.5 ms per query but BGE costs 409.6 ms on an A100—a ∼9×gap that makes reranker choice the dominant latency lever once the candidate set is fixed. That per-passage cost compounds quickly if reranking is ap- plied naively over a full, unbounded candidate set rather than a small one. We measure this directly, single-threaded, with the MiniLM-12L reranker (118M parameters) over the full hybrid candidate list for each of the seven BEIR datasets, com- 4 MSMARCOHotpotQANQFEVERClimate FEVER TREC COVID FiQa 0.2 0.4 0.6 0.8 nDCG@10 RRFWeighted Fusion Figure 3. RRF vs. weighted fusion, nDCG@10. Recall is nearly identical (not shown); weighted fusion ranks better by retaining score magnitude. 1248163264128256 Passages per query (log) 10 1 10 2 10 3 ms/query (log) 410 44 BGE (568M)MiniLM (33.4M) Figure 4. Cross-encoder reranking latency vs. passages per query (A100, log–log). BGE is an order of magnitude slower than MiniLM at every candidate-set size. paring hybrid retrieval alone against hybrid retrieval followed by full-candidate-set reranking (Figure 5). Hybrid retrieval alone answers in 29–58 ms per query across datasets; adding full reranking raises this to 25–86 s per query—a 485–2051× slowdown, worst on TREC-COVID (86.17 s,2051×) and best on HotpotQA (26.21 s,485×), tracking each dataset’s average candidate-set size more than any other factor. This is the result that motivates confining reranking to a small, bounded candi- date set rather than the full retrieved list: SESF and CE score fusion both rerank only the fused top-k(Table 2), which is why their end-to-end latency lands in the hundreds of milliseconds rather than tens of seconds. 4.4 CE-QE: enriching the lexical stage CE-QE improves BM25 directly (Figure 6). The largest gains appear where lexical and semantic retrieval otherwise di- verge: NQ Recall@100 rises from 0.32 to 0.47, TREC-COVID from 0.56 to 0.67, and Climate-FEVER from 0.19 to 0.26. FEVER dips slightly (0.71→0.70) where surface terms al- ready match, so added tokens contribute little. Because CE-QE only rewrites the query, these gains require no change to the underlying BM25 index—here, standard flat BM25—and the same argument applies unchanged if a different, larger-scale BM25 index answers the expanded query instead (Section 5). FiQAHotpot QA FeverMS MARCO Climate FEVER NQTREC COVID 10 2 10 1 10 0 10 1 10 2 Latency per query, s (log) Hybrid (ST) Hybrid + Reranker (ST) Figure 5. Per-query latency, hybrid retrieval alone vs. hybrid retrieval with full-candidate-set cross-encoder reranking (single-threaded, log scale, 7 BEIR datasets). Unbounded reranking costs 485–2051× more than retrieval alone, motivating the bounded candidate sets used elsewhere in this paper. MSMARCOHotpotQANQFEVERClimate FEVER TREC COVID FiQa 0.0 0.2 0.4 0.6 0.8 Recall@100 BM25CE-QE Enriched BM25 Figure 6. CE-QE enriched BM25 vs. plain BM25, Recall@100. Ex- pansion terms drawn from cross-encoder attributions lift recall most where surface vocabulary and answer vocabulary differ (NQ, TREC- COVID, Climate-FEVER). 4.5 SESF: fusion quality and comparison to SOTA Using CE-QE inside score fusion (SESF) beats cross- encoder score fusion by 2.5% on Recall@100 at comparable nDCG@10 (Figure 7a), with the clearest wins on HotpotQA, TREC-COVID, and Climate-FEVER. Against strong learned models, SESF beats ColBERTv2 by 4.6% and SPLADEv2 by 5.3% on nDCG@10 (Figure 7b), while leaving the lexical in- dex a standard BM25 index rather than a specialized sparse or late-interaction structure. Overall, query expansion with cross- encoders improves 6.6% over RRF, the common production default. 4.6 End-to-end latency Table 2 places the query-time cost of each method (top- k=100, MSMARCO). RRF is cheapest because it fuses ranks only; CE score fusion adds the reranker pass; SESF adds se- mantic retrieval and cross-encoder token extraction on top, trad- ing∼400 ms of extra latency for its recall and SOTA-beating ranking. The extra cost is spent on CPU-friendly BM25 and a bounded reranker call, not on scaling the index. 5 MSMARCOHotpotQANQFEVERClimate FEVER TREC COVID FiQa 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Recall@100 (a) SESF vs. CE Score Fusion CE Score FusionSESF (ours) HotpotQANQFEVERClimate FEVER TREC COVID FiQa 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 nDCG@10 (b) SESF vs. SOTA sparse / late-interaction SPLADEv2ColBERTv2SESF (ours) Figure 7. SESF quality. (a) SESF vs. cross-encoder score fusion, Recall@100:+2.5%on average. (b) SESF vs. SPLADEv2 and ColBERTv2, nDCG@10: +5.3% and +4.6% respectively, using an unmodified BM25 index. Table 2. End-to-end query latency (top-k=100, MSMARCO) and the quality each method buys. MethodLatencyQuality note RRF∼153.8 msfusion baseline CE score fusion ∼400.3 ms +reranker; better nDCG@10 SESF (ours) ∼556.9 msbest recall; beats SOTA nDCG@10 Reranker cost @128 passages: MiniLM 43.5 ms, BGE 409.6 ms (A100). 5 Discussion and Limitations Composing with a scalable lexical index. CE-QE is evalu- ated here against standard flat BM25 over BEIR-scale corpora (at most 8.8M documents); we have not run it against a billion- chunk lexical index end-to-end. What makes that composition architecturally straightforward is exactly the separation of con- cerns in Section 3: CE-QE’s output is an ordinary, expanded text query, not a modified index, a custom scoring function, or a change to term statistics, so any BM25 index—including a sharded or hierarchical one—answers it without special han- dling. We would expect CE-QE’s recall gain to hold at scale, since it is a property of the query, not of how the index is internally partitioned, but we present this as an architectural expectation, not a measured result, and confirming it end-to- end is the natural next step. Cost is reserved for queries that need it. CE-QE adds a semantic-search plus cross-encoder pass at query time (Ta- ble 2), so it trades latency for quality and is best applied selec- tively rather than unconditionally—on datasets whose surface terms already match the answer vocabulary (FEVER), the ex- pansion adds cost for little gain (Section 3.5). A deployment could use a cheap signal (e.g., lexical-semantic agreement on the unexpanded query) to decide when expansion is worth its cost, though we have not evaluated such a gate here. Attribution quality bounds expansion quality. CE-QE’s expansion terms are only as good as the cross-encoder’s attri- butions; a reranker with poorly calibrated or noisy token-level attributions would select less discriminative terms, and we have not studied sensitivity to reranker choice beyond the two models used in Section 4. 6 Conclusion Lexical retrieval’s blindness to the semantic vocabulary gap cannot be fixed by reranking or fusion alone, because both operate only on documents already retrieved. CE-QE fixes it at the source by importing a cross-encoder’s token-level relevance judgments into the BM25 query, grounding every ex- pansion term in a passage the corpus’s own semantic retriever actually returned rather than in text a language model halluci- nates, and adding no separate generation call. Lexical recall improves substantially where query and answer vocabulary di- verge (e.g., NQ Recall@100 from 0.32 to 0.47), and its fusion form, SESF, beats cross-encoder score fusion by 2.5% on Re- call@100 and beats SPLADEv2 and ColBERTv2 by 5.3% and 4.6% on nDCG@10, all while leaving the underlying BM25 index completely unmodified—a property that should let it compose with any lexical index, including one built for scale, without architectural change, pending direct measurement of that composition. References [1]S. Robertson and H. Zaragoza. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in IR, 3(4):333–389, 2009. [2]N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych. BEIR: A Heterogeneous Benchmark for Zero- shot Evaluation of Information Retrieval Models. In NeurIPS Datasets and Benchmarks, 2021. [3]G. V. Cormack, C. L. A. Clarke, and S. Büttcher. Recipro- cal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. In SIGIR, 758–759, 2009. [4] Y. A. Malkov and D. A. Yashunin. Efficient and Robust Approx- imate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI, 42(4):824–836, 2018. 6 [5]H. Jégou, M. Douze, and C. Schmid. Product Quantization for Nearest Neighbor Search. IEEE TPAMI, 33(1):117–128, 2011. [6]V. Karpukhin, B. O ̆ guz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih. Dense Passage Retrieval for Open- Domain Question Answering. In EMNLP, 6769–6781, 2020. [7]R. Nogueira and K. Cho. Passage Re-ranking with BERT. arXiv:1901.04085, 2019. [8]N. Reimers and I. Gurevych. Sentence-BERT: Sentence Em- beddings using Siamese BERT-Networks. In EMNLP-IJCNLP, 3982–3992, 2019. [9] T. Formal, C. Lassance, B. Piwowarski, and S. Clinchant. SPLADE v2: Sparse Lexical and Expansion Model for Infor- mation Retrieval. arXiv:2109.10086, 2021. [10]K. Santhanam, O. Khattab, J. Saad-Falcon, C. Potts, and M. Zaharia. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In NAACL, 3715–3734, 2022. [11]V. Lavrenko and W. B. Croft. Relevance-Based Language Mod- els. In SIGIR, 120–127, 2001. [12]L. Wang, N. Yang, and F. Wei. Query2doc: Query Expansion with Large Language Models. In EMNLP, 9414–9423, 2023. [13]L. Gao, X. Ma, J. Lin, and J. Callan. Precise Zero-Shot Dense Retrieval without Relevance Labels. In ACL, 1762–1777, 2023. [14]Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang. Retrieval-Augmented Generation for Large Lan- guage Models: A Survey. arXiv:2312.10997, 2023. 7

Instant research discovery