Paper deep dive
GEM: A Generative Embedding Model Bridging Reasoning and Retrieval
Zhili Shen, Craig Macdonald
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/17/2026, 3:57:44 AM
Summary
The paper introduces GEM (Generative Embedding Model), a unified model that augments retrieval by explicitly reasoning about user intent and relevance criteria before generating an embedding. GEM combines causal language modeling for reasoning and contrastive learning for embedding, trained on a tailored dataset where documents are generated based on validated reasoning. It outperforms non-reasoning variants and matches larger baselines on reasoning-intensive and instruction-following retrieval tasks.
Entities (15)
Relation Signals (13)
Zhili Shen → affiliatedwith → University of Glasgow
confidence 95% · Zhili Shen University of Glasgow
Craig Macdonald → affiliatedwith → University of Glasgow
confidence 95% · Craig Macdonald University of Glasgow
GEM → createdby → Craig Macdonald
confidence 95% · GEM: A Generative Embedding Model Bridging Reasoning and Retrieval ... Craig Macdonald
GEM → createdby → Zhili Shen
confidence 95% · GEM: A Generative Embedding Model Bridging Reasoning and Retrieval Zhili Shen
GEM → evaluatedon → BRIGHT
confidence 95% · Evaluated on reasoning-intensive and instruction-following retrieval tasks... report nDCG@10 on BRIGHT
GEM → outperforms → Qwen3-4B-Instruct
confidence 95% · GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning variant
GEM → evaluatedon → InstructIR
confidence 90% · and InstructIR with their official metrics
GEM → evaluatedon → FollowIR
confidence 90% · we evaluate instruction-following retrieval using FollowIR
GEM → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern LLMs excel at reasoning and instruction following, enabling users to express complex and diverse information needs. However, conventional retrievers largely rely on surface-level matching between queries and documents, resulting in a growing gap between how users express their needs and how retrievers interpret them. In this paper, we present GEM, a generative embedding model that augments retrieval through its own knowledge by explicitly reasoning about user intent and relevance criteria. GEM unifies generation and embedding within a single model: it first reasons over the query, then appends an embedding token to encode the enriched context for retrieval. Evaluated on reasoning-intensive and instruction-following retrieval tasks, GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning variant and matching baselines using substantially larger models. Furthermore, GEM's generative nature allows test-time compute scaling via prompting to further enhance retrieval performance. Our code is available at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.13200v2
- Canonical: https://arxiv.org/abs/2608.13200v2
Trouble viewing inline? Open PDF directly →
Full Text
85,786 characters extracted from source content.
Expand or collapse full text
GEM: A Generative Embedding Model Bridging Reasoning and Retrieval Zhili Shen University of Glasgow z.shen.2@research.gla.ac.uk Craig Macdonald University of Glasgow craig.macdonald@glasgow.ac.uk Abstract Modern LLMs excel at reasoning and instruc- tion following, enabling users to express com- plex and diverse information needs. However, conventional retrievers largely rely on surface- level matching between queries and documents, resulting in a growing gap between how users express their needs and how retrievers inter- pret them. In this paper, we present GEM, a generative embedding model that augments re- trieval through its own knowledge by explicitly reasoning about user intent and relevance cri- teria. GEM unifies generation and embedding within a single model: it first reasons over the query, then appends an embedding token to encode the enriched context for retrieval. Eval- uated on reasoning-intensive and instruction- following retrieval tasks, GEM demonstrates the effectiveness of its reasoning-augmented retrieval, outperforming its non-reasoning vari- ant and matching baselines using substantially larger models. Furthermore, GEM’s genera- tive nature allows test-time compute scaling via prompting to further enhance retrieval per- formance. Our code is available at:https: //anonymous.4open.science/r/GEM. 1 Introduction The rapid evolution of large language models (LLMs) is transforming traditional information- access processes. Driven by advanced reason- ing (Wei et al., 2022b; Kojima et al., 2022) and instruction-following (Wei et al., 2022a; Ouyang et al., 2022) capabilities, these models demonstrate remarkable abilities in understanding diverse in- puts and generalising to challenging tasks. Conse- quently, user behaviour is evolving. For instance, users are increasingly accustomed to interacting with modern AI systems through natural-language instructions (Knoth et al., 2024), allowing them to express complex information needs. In contrast, retrievers have not fully kept pace with advances in LLMs. Conventional retrievers GEM User What is this syntax called? Help me understand it with simple examples. @func2 def func1(arg1, arg2): pass Search Intent: The user is seeking to understand the name and purpose of a specific programming syntax they've encountered, with a request for simple, illustrative examples to aid comprehension. ... ✅ Relevant documents will: - Name the syntax as a "decorator" - Explain it as a way to modify or extend function behavior - Provide a simple, concrete example - Clearly link the syntax to Python’s function definition structure ❌ Irrelevant documents will: - Misidentify the syntax (e.g., call it a "module" or "annotation") - Skip examples or use overly abstract language - Focus on unrelated topics (e.g., syntax in other languages) Generation Why does the syntax func1(func2)(parameters); work and what is it called? From the book Eloquent Javascript Third Edition ... Retrieve Python Decorators Decorators let you add extra behavior to a function without changing the function's code. # Example def log(func): ... Doc 2: Irrelevant despite high lexical and semantic similarity Doc 1: Relevant Query Figure 1: An example use case of GEM. primarily rely on lexical and/or semantic similar- ities between queries and documents (Robertson et al., 1994; Wang et al., 2024a; Ma et al., 2023; Izacard et al., 2022). However, when relevant infor- mation is not directly aligned with a query’s surface form, effective retrieval requires deeper query un- derstanding—for example, to infer underlying user intent (Su et al., 2025), or to satisfy constraints specified in instructions (Weller et al., 2025a). This mismatch creates a growing gap between how users express their information needs and how retrieval systems interpret them (Zhou et al., 2025). Recent research has increasingly sought to ad- dress these challenges.Su et al. (2025) pro- arXiv:2608.13200v2 [cs.CL] 14 Aug 2026 posed the reasoning-intensive retrieval task, where queries are challenging and require reasoning be- yond direct, surface-level matching. Follow-up work (Shao et al., 2025; Das et al., 2025; Sun et al., 2026) has shown that retrievers can be augmented with reasoning generated by upstream LLMs. How- ever, these methods rely on pipelines with sep- arate models for reasoning and retrieval, rather than leveraging the inherent generative capabili- ties of their retrieval models. More importantly, it remains unclear whether the retrievers genuinely understand reasoning, or simply benefit from in- creased lexical and semantic overlap. In parallel, another line of research has explored instruction- following retrieval (Weller et al., 2025a; Oh et al., 2024). Unlike earlier studies (Asai et al., 2023; Su et al., 2023) that prepend queries with a task- specific instruction template, recent work enables a dense retriever to represent detailed, query-specific instructions that specify user preferences and con- straints (Weller et al., 2025b). Yet, user-provided instructions are often underspecified and ambigu- ous in practice (Zhou et al., 2026). Whether rea- soning can enhance a retriever’s ability to interpret and follow retrieval instructions is underexplored. In this paper, we present GEM, a Generative Embedding Model.We argue that the chal- lenges outlined above stem from the need to en- hance query understanding in retrievers. GEM addresses this by integrating reasoning and em- bedding within a single model. Figure 1 illustrates GEM’s generate-then-encode paradigm: given a query, GEM reasons about the user’s intent and relevance criteria, then encodes the context for re- trieval by appending an embedding token. Exist- ing LLM-based embedding models are typically trained as bi-encoders (Zhang et al., 2025; Wang et al., 2024b; Ma et al., 2023); the generative ca- pabilities of their backbones are often degraded due to catastrophic forgetting (Li et al., 2024; Luo et al., 2025) and/or architectural choices such as bidirectional attention masking (BehnamGhader et al., 2024; Zeng et al., 2025). In contrast, GEM is jointly trained with contrastive and causal language modelling objectives, preserving generation while enabling effective embedding. To align embed- ding with generation, we introduce a tailored data synthesis strategy that constructs document pairs conditioned on validated reasoning: positives align with the reasoning, while hard negatives share simi- lar topics but contain subtle contradictions, thereby discouraging simple surface-level matching. Our core contributions are: (1) we propose GEM, which unifies representation learning and causal language modelling; (2) we introduce a tailored data generation strategy to align GEM’s embedding with its reasoning; and (3) our experiments demon- strate that GEM’s generative capabilities effectively augment retrieval, with further gains achievable by prompting to exploit test-time compute. To the best of our knowledge, GEM is the first embedding model to leverage its own generative capabilities to produce reasoning-aligned embeddings. 2 Related Work Reasoning-Intensive Retrieval Complex real- world queries can exhibit low lexical and semantic similarity to relevant documents (Su et al., 2025). In such cases, effective retrieval requires reason- ing beyond surface-level matching. Prior work, such as ReasonIR (Shao et al., 2025), trained LLM- based dense retrievers with hard queries. The best performance is often achieved when retrievers are augmented with reasoning generated by upstream LLMs (Su et al., 2025; Shao et al., 2025; Lei et al., 2025; Das et al., 2025). However, the alignment between reasoning and retrieval behaviour is un- clear: the improvements may stem from additional surface-level overlap rather than genuine under- standing of the reasoning (see RQ4 in Section 4.2). GEM is a unified model that leverages its inherent generative capabilities to augment retrieval, with its embedding trained to align with reasoning. Further- more, GEM’s generative nature allows test-time compute scaling through prompting. Instruction-Following Retrieval Training em- bedding models with instructions aims to improve generalisation across diverse information needs. Weller et al. (2025a) proposed to evaluate retrievers on queries with flexible, query-specific instructions that can extend beyond topical relevance. For ex- ample, a user may prefer documents that are easier to understand and avoid technical jargon. Within this line of research, Promptriever (Weller et al., 2025b) is a bi-encoder that allows users to spec- ify relevance criteria through prompting (e.g. "A document is relevant if ..."). However, relying on user-provided instructions to precisely define the problems is often unrealistic (Zhou et al., 2026), for example, due to a lack of domain expertise. Unlike prior work, we propose a generative embedding model that explicitly reasons about user intent and relevance criteria to augment retrieval. Query Expansion Using LLMs Query expan- sion is a well-established technique for improv- ing retrieval performance by mitigating problems such as vocabulary mismatch (Wang et al., 2023b). Prior work such as HyDE (Gao et al., 2023) and Query2Doc (Wang et al., 2023a) leverages the parametric knowledge of LLMs to generate hy- pothetical documents (answers) for query expan- sion. While recent studies instead use LLM reason- ing (Sun et al., 2026; Lei et al., 2025; Das et al., 2025; Yan et al., 2026), they largely follow the conventional query expansion perspective. Our ex- periments on instruction-following retrieval reveal a fundamental limitation: retrievers may not gen- uinely understand reasoning over user instructions, such as what not to retrieve. 3 Generative Embedding Model Given a queryqand a corpusd i N i=1 containingN documents, the retrieval task aims to find the top- kmost relevant documents forq, wherek ≪ N. Dense retrieval represents text as embeddings and ranks documents based on the similarity between the query and document embeddings. Unlike conventional dense retrievers, GEM in- corporates generation into its query embedding. Specifically, given a queryq, we construct a prompt p = I ◦ q, whereIis an instruction that prompts GEM to reason about user intent and relevance criteria. We refer to this instruction as our meta- instruction to distinguish it from user-provided in- structions in queries. GEM then generates a re- sponser. For retrieval, GEM learns the similar- ity between a documentdand the concatenated prompt-response pair, p◦ r. However, incorporating reasoning into embed- ding raises challenges in both data and training. Re- garding data, two key challenges are: (1) the gener- ated reasoning may misinterpret the query, and (2) the need to align the notion of relevance with rea- soning. From the training perspective, the model must retain its generative capability while learning effective embeddings. We address these challenges through a tailored data generation pipeline (Sec- tion 3.1) and joint training with generation and embedding objectives (Section 3.2). 3.1 Data Generation Response GenerationRetrieval datasets such as MS MARCO (Nguyen et al., 2016) typically pro- vide a collection of text pairs(q,d + )whered + denotes a relevant (positive) document for query q. As illustrated in Figure 2, given a queryq, we construct the promptpand sample a set of candi- date responsesR q = r (1) ,r (2) ,...,r (K) from an LLM parameterized byθ, i.e.r (k) ∼ P θ (· | p). Each responser (k) may reflect a plausible interpre- tation ofq, but could vary in quality and alignment with the original relevance annotations due to hal- lucinations. To mitigate this, we employ a simple filtering step using an LLM-based relevance classi- fier. For each candidate responser (k) and the orig- inal positive documentd + , we prompt the LLM to assess whetherd + remains relevant underr (k) . Concretely, the classifier outputs a binary decision f(r (k) ,d + ) ∈ 0, 1, wheref(r (k) ,d + ) = 1indi- cates thatd + is still relevant givenr (k) . We retain only the responses that preserve the relevance of the original positive document: ̃ R q =r (k) ∈R q | f(r (k) ,d + ) = 1.(1) If ̃ R q is empty, the query is discarded from the training set. Otherwise, we randomly select a re- sponser ∼ ̃ R q to associate withq. This filter- ing step serves as a coarse-grained alignment to avoid obvious contradictions between a generated response and the original positive document. Document Generation Original relevance an- notations between queries and documents may not fully transfer when conditioning on reasoning. Even after filtering, the relevance of documents may shift subtly due to additional criteria intro- duced by a responser. To achieve fine-grained alignment, we follow Wang et al. (2024b) to gener- ate positive and negative documents using an LLM. In our case, the document generation is conditioned on reasoning. For each responser, we prompt the model to generate a positive document based on r. In parallel, hard negative documents may share similar topics but fail to satisfy the user intent or some relevance criteria specified inr. The result- ing training set isT = (p,r,d + ,d − ), where, for simplicity, we reused + andd − to denote the generated positive and hard negative, respectively. Detailed prompts are provided in Appendix C. 3.2 Training GEM is trained with both causal language mod- elling and contrastive losses. For each queryq i , we construct a promptp i with the same meta- instruction used for response generation (see Query # Task Identify the search intent. Then provide a concise analysis of the key criteria that make a document relevant. # Query query Query Prompt (with Meta-Instruction) Response 1 <|embed|> # Task Represent the text for retrieval. # Text doc Response K ... Positive Hard Negative Sampling & Filtering aligned misaligned Document Generation Query Prompt Response Document Prompt <|embed|> Generation Loss Embedding Loss 1. Data Generation 2. Training Figure 2: Data generation pipeline (top) and training workflow (bottom) of GEM. Prompt in Figure 2). Given a batch ofNprompt- response pairs(p i ,r i ) N i=1 , we compute the causal language modelling loss over the responses: L gen =− 1 P N i=1 |r i | N X i=1 |r i | X t=1 logP θ (r i,t | p i ,r i,<t ) (2) where r i,t is the t-th token of response r i . Embeddings of prompt-response pairs are ob- tained through last-token pooling, i.e. using the hidden state of the last token. Unlike previous stud- ies that use the embedding of the end-of-sequence (EOS) token (Ma et al., 2023; Wang et al., 2024b), we reserve the EOS token for generation. Instead, we append a dedicated token,<|embed|>, to the end of each response to produce an embedding. Since generation terminates upon encountering an EOS token, our embedding token is never predicted during decoding and is therefore excluded from the causal language modelling loss. At inference time, the embedding token is appended after gener- ation completes, and its representation is computed efficiently by reusing the KV cache from gener- ation. For document encoding, we prepend each document with a simple instruction (see Document Prompt in Figure 2) and apply the same embedding token. As generation is not performed on docu- ments,L gen is not applicable. LetE θ (p i ◦ r i )be the embedding of the con- catenated promptp i and responser i , andE θ (d)be the embedding of documentd(with document-side prompt). Following prior work (Muennighoff et al., 2025; Gao et al., 2021; Ma et al., 2023), we adopt the InfoNCE loss (Chen et al., 2020) as follows: L emb =− 1 N N X i=1 log exp s θ (p i ◦ r i ,d + i )/τ P d∈D i exp s θ (p i ◦ r i ,d)/τ (3) whereD i =d + i ∪ D − i , andD − i consists of both hard negatives and in-batch negatives.τis a tem- perature hyperparameter. The similarity function s θ is defined as: s θ (p i ◦ r i ,d) = cos E θ (p i ◦ r i ),E θ (d) (4) wherecos(·,·)denotes cosine similarity. Our final training objective is a weighted sum of generation and embedding losses: L GEM = λ gen L gen + λ emb L emb (5) Overall, our Generative Embedding Model uni- fies reasoning and embedding within a single model. Our tailored data generation pipeline aug- ments existing retrieval datasets for training GEM and aligns the relevance notion with reasoning. The joint training objective prevents GEM from forget- ting token prediction while learning embeddings. 4 Experiments 4.1 Experimental Setup Evaluation Datasets Following Shao et al. (2025), we report nDCG@10 on BRIGHT (Su et al., 2025). BRIGHT comprises 12 subsets with reasoning-intensive queries spanning diverse do- mains: Biology (Bio.), Earth Science (Earth.), Economics (Econ.), Psychology (Psy.), Robotics (Rob.), Stack Overflow (Stack.), Sustainable Living (Sus.), Leet Code (Leet.), Pony, math Olympiad problems (AoPS), scientific theorem question answering using either questions for retrieval (TheoQ.) or theorems for retrieval (TheoT.). Following Weller et al. (2025b), we evalu- ate instruction-following retrieval using FollowIR (Weller et al., 2025a) and InstructIR (Oh et al., 2024) with their official metrics. For FollowIR, we report nDCG@5 on News21, MAP@1000 on Core17 and Robust04, and p-MRR on all splits. As retrievers can ignore instructions while achieving high nDCG or MAP scores on these TREC datasets (Weller et al., 2025a), p-MRR∈ [−100, 100]is proposed to measure the sensi- tivity to instruction changes, where higher val- ues indicate better instruction following (Weller et al., 2025b). For InstructIR, which is built from MS MARCO (Nguyen et al., 2016), we report nDCG@10 and Robustness@10. The latter is the minimum nDCG@10 for the same query when paired with different prompts, which define distinct user contexts (e.g. job) (Oh et al., 2024). Implementation Details We train GEM from Qwen3-4B-Instruct-2507(Yang et al., 2025a). Experiments with different LLM backbones are presented in Appendix B.3. All our models are trained for 500 steps with an effective batch size of 512. The learning rate is set to1× 10 −5 with 50 warmup steps. Following prior work (Muennighoff et al., 2025; Shao et al., 2025), we use a training group size of 2, i.e. one positive and one hard nega- tive per query. In-batch negatives are then gathered across all GPUs. For training loss, we empirically setλ gen = 0.1andλ emb = 1.0; a parameter study is provided in Appendix B.2. The temperatureτ for contrastive loss is set to0.02. Training takes approximately 14 hours on 2 NVIDIA H100 GPUs. For data generation, we augment the training data from Promptriever (Weller et al., 2025b) and the collection of hard queries from ReasonIR (Shao et al., 2025) to train GEM. For each query, we sampleK = 8responses from GEM’s backbone using a temperature of1.0. For simplicity, fil- tering is performed by the same backbone using greedy decoding. For document generation, we useLlama-3.1-8B-Instructwith greedy decod- ing to generate one positive and one hard negative per response. Additionally, we randomly sample 60,000 instances from the original Promptriever data without applying our data generation pipeline. For these samples, standard contrastive learning is applied, and queries are encoded using the document-side prompt (see Figure 2). Our final training set for GEM consists of 370K samples: 320K from Promptriever (comprising 260K with reasoning and 60K original), and 50K reasoning samples based on the hard queries. The size of our 260K reasoning subset is chosen to match our fixed training budget (500 steps with batch size 512), en- suring fair ablation experiments (Table 4) where models never see repeated samples during training. For evaluation, GEM’s reasoning is generated using greedy decoding in all experiments. Ap- pendix A provides further implementation details. BaselinesFor BRIGHT, we adopt the baseline re- sults reported in ReasonIR. These include: (1) non- LLM retrievers: BM25 (Robertson et al., 1994) and Contriever (Izacard et al., 2022); (2) LLM- based retrievers: GritLM-7B (Muennighoff et al., 2025) and ReasonIR-8B (Shao et al., 2025); (3) LLM-based rerankers: Rank1-7B and Rank1- 32B (Weller et al., 2025c) which perform reason- ing to rerank the top-100documents retrieved by BM25 with query expansion using GPT-4. For instruction-following retrieval, we compile results reported in Promptriever and FollowIR: (1) non-LLM retrievers: BM25 (Robertson et al., 1994), BGE-large (Xiao et al., 2024), and Instructor-XL (Su et al., 2023); (2) LLM-based retrievers: RepLLaMA (Ma et al., 2023), E5- Mistral (Wang et al., 2024b), GritLM-7B (Muen- nighoff et al., 2025), and Promptriever (Weller et al., 2025b);(3) LLM-based rerankers: FollowIR-7B (Weller et al., 2025a), Mistral-7B- Instruct (Jiang et al., 2023), and GritLM-7B (generation mode) (Muennighoff et al., 2025). Additionally, we evaluate ReasonIR-8B and Promptriever (7B) across all benchmarks. To en- able a direct comparison using the same backbone, we trainQwen3-4B-Instruct-2507on the origi- nal Promptriever data, 1 keeping the training config- uration consistent with GEM. This variant is solely an embedding model trained using contrastive loss. We refer to this model as Qwen3-4B-Instruct. 4.2 Results and Analysis In this section, we present the results and address our main research questions. Additional results are provided in Appendix B. 1 We are unable to train on the original ReasonIR data because part of its training set is not released. Model StackExchangeCodingTheorem-basedAvg. Bio.Earth.Econ.Psy.Rob.Stack.Sus.Leet.PonyAoPSTheoQ.TheoT. Single Model BM2519.227.114.912.513.516.515.224.47.96.013.06.914.8 Contriever9.213.610.512.19.59.68.924.514.77.210.43.211.1 GritLM-7B25.032.819.019.917.311.618.029.822.08.825.121.120.9 Qwen3-4B-Instruct18.935.918.824.918.920.617.338.53.814.427.117.821.4 Promptriever25.238.521.025.019.118.818.832.41.79.921.18.820.0 ReasonIR-8B26.231.423.330.018.023.920.535.010.514.731.927.224.4 GEM34.740.826.533.724.629.326.035.52.816.637.641.829.1 Pipeline with GPT-4 Reasoner BM2553.653.624.338.618.822.725.919.317.73.920.218.926.5 Contriever37.540.522.627.115.222.619.622.513.88.124.116.222.5 GritLM-7B33.233.023.330.615.217.521.733.211.76.826.928.023.4 Qwen3-4B-Instruct31.141.526.437.719.324.922.136.82.610.133.031.926.4 Promptriever40.750.230.836.719.431.326.130.17.912.126.323.027.9 Rank1-7B48.836.720.835.022.018.736.212.731.26.323.737.827.5 Rank1-32B49.735.822.037.522.521.735.018.832.510.822.943.729.4 ReasonIR-8B43.642.932.738.820.925.827.531.519.67.433.135.729.9 GEM39.944.228.436.425.231.228.134.75.215.336.235.330.0 Table 1: nDCG@10 for reasoning-intensive retrieval on BRIGHT. Best results are shown in bold. Qwen3-4B- Instruct is the embedding-only variant of GEM with the same 4B-parameter backbone. RQ1: How effective is GEM at reasoning- intensive retrieval? Table 1 presents results on BRIGHT. Our embedding-only variant, Qwen3- 4B-Instruct, serves as a strong baseline, performing comparably to larger models, including GritLM- 7B and Promptriever. Overall, GEM achieves an average nDCG@10 of 29.1, outperforming single- model baselines by leveraging its own knowledge. Specifically, GEM excels at theorem-based tasks, substantially improving their average nDCG@10 over Qwen3-4B-Instruct (19.8→ 32.0) using the same backbone. However, GEM underperforms on the Pony programming language task. Similar results are observed for Promptriever and Qwen3- 4B-Instruct, despite Promptriever using a different backbone. We provide a detailed discussion in Ap- pendix B.6 to explain this out-of-domain problem. Following previous studies (Su et al., 2025; Shao et al., 2025), we also report results where retrievers are augmented with reasoning generated by GPT- 4. 2 In this setting, the average response length is approximately37%longer than that generated by GEM. Additionally, to investigate whether GEM can effectively represent outputs from a stronger model, we evaluate GEM as a bi-encoder, directly encoding the input instead of generating its own reasoning. With the same reasoning, GEM achieves an average nDCG@10 of 30.0, which is competi- tive with ReasonIR-8B (29.9), despite GEM being 2 ThedataisprovidedbyBRIGHT:https: //huggingface.co/datasets/xlangai/BRIGHT 6425651210242048 Target response length (words) 27.5 28.0 28.5 29.0 29.5 30.0 nDCG@10 nDCG (modified prompt) nDCG (default prompt) Encoding Time w/o cache Encoding Time w/ cache 1072354569341503 Actual generated length (words) 20 30 40 50 60 70 80 Time (ms) Figure 3: Average nDCG@10 (left) on BRIGHT and per-query encoding time (right) in milliseconds for GEM with test-time compute scaling. a 4B-parameter model trained with substantially less compute. RQ2: Can GEM’s test-time compute be scaled via prompting, and what is the impact on re- trieval performance? To investigate whether GEM’s retrieval can be influenced by scaling test- time compute, we modify our prompt at test time as follows: (1) following prior studies (Su et al., 2025; Shao et al., 2025), we add an instruction for answer generation, and (2) we explicitly in- struct the model to generate approximatelyn ∈ 64, 256, 512, 1024, 2048 words. The resulting prompt is provided in Appendix C. Figure 3 presents the nDCG@10 scores (left) across different target response lengthsn(bottom). GEM is aware of our length requirements, adjust- Model FollowIRInstructIR Robust04News21Core17AverageMS MARCO MAPp-MRRnDCGp-MRRMAPp-MRRScorep-MRRnDCGRobust. Rerankers GritLM-7B9.7+6.110.2+3.49.8+8.69.9+6.0-- Mistral-7B-Instruct23.2+12.627.2+4.819.7+13.023.4+10.163.135.3 FollowIR-7B 24.8 +13.729.6+6.320.0 +16.524.8 +12.281.371.5 Retrievers BM2512.1-3.119.3-2.18.1-1.113.2-2.176.026.9 BGE-large 17.5-7.822.3+0.615.0+0.118.3-2.4-- Instructor XL19.7-8.126.1-0.916.8+0.720.9-2.848.621.5 LLMs≥7B parameters GritLM-7B28.6-1.724.4-1.020.8+2.624.6-0.0-- RepLLaMA24.0-8.924.5-1.820.6+1.323.0-3.185.750.2 E5-Mistral23.1-9.627.8-0.918.3+0.123.1-3.586.355.4 ReasonIR-8B25.3-2.823.1+0.618.7+0.922.4-0.487.153.1 Promptriever28.3 +11.728.5+6.421.6 +15.426.1 +11.292.163.1 LLMs∼4B parameters Qwen3-4B-Instruct25.7+7.325.2+3.221.0+9.924.0+6.882.046.2 GEM26.7 +11.626.2+9.323.5 +14.225.5 +11.787.554.8 Table 2: Results for instruction-following retrieval on the FollowIR and InstructIR datasets. Higher is better for all metrics. MAP@1000, nDCG@5, and Robustness@10 are reported on a 0–100 scale, while p-MRR ranges from -100 to 100. "-" indicates results not reported in previous studies. Best results are shown in bold. ing its actual response length (top) accordingly. Overall, the average nDCG@10 on BRIGHT im- proves with increased test-time compute, peaking at30.1when prompted withn = 1024. However, similar to findings in prior work (Shao et al., 2025), performance gains saturate for long generations, likely because GEM generates redundant content for simple queries to meet our length requirements, while failing to produce additional useful signals for challenging problems beyond its capacity. Additionally, Figure 3 benchmarks the aver- age per-query encoding time (right) asnvaries. As a baseline, we use GEM to re-encode the same sequence (i.e. prompt and response) without reusing the KV cache from generation, analogous to pipelines with independent models. Under this setting, encoding time increases sharply for longer sequences. In contrast, when reusing the KV cache in GEM’s generate-then-encode process, encoding time remains stable across the evaluated lengths. Appendix B.4 provides further latency analysis. RQ3: Can GEM’s reasoning augment instruction-following retrieval? Table2 compares GEM with baselines on FollowIR and In- structIR. GEM performs on par with Promptriever and outperforms other LLM-based retrievers on FollowIR. Specifically, GEM achieves a p-MRR of+11.7, which indicates strong instruction- following retrieval performance (Weller et al., 2025b), matching the performance of larger models including Promptriever and FollowIR-7B. To answer our RQ3, GEM shows consistent improve- ments over Qwen3-4B-Instruct (same backbone), with notable gains in p-MRR (+6.8 → +11.7) and Robustness@10 (46.2→ 54.8). As FollowIR provides detailed relevance instruc- tions (e.g. "A relevant document will contain . . . "), we present qualitative case studies in Appendix D to investigate scenarios where such detailed criteria are absent in practice. RQ4: Can LLM-based query expansion achieve strong instruction-following retrieval like GEM? We compare GEM with representa- tive LLM-based query expansion methods, includ- ing HyDE (Gao et al., 2023) and Query2Doc (Wang et al., 2023a).For a fair comparison, we useQwen3-4B-Instruct-2507as the hypotheti- cal document (answer) generator for HyDE and Query2Doc. Implementation details are provided in Appendix A.3. Furthermore, we evaluate a vari- ant that concatenates the original query with the reasoning generated by GEM for query expansion. Results are presented in Table 3. While HyDE and Query2Doc often improve nDCG and MAP, their impact on instruction-following metric (p- MRR) is inconsistent. Notably, both methods de- grade p-MRR for Promptriever. While GEM’s rea- soning contributes to its strong p-MRR, directly applying it to other models may lead to poor results. In answer to our RQ4, LLM-based query expan- Setting Robust04News21Core17Average MAPp-MRRnDCGp-MRRMAPp-MRRScorep-MRR BGE-large17.5-7.822.3+0.615.0+0.118.3-2.4 + HyDE21.0-4.123.1+0.320.1+1.421.4↑-0.8↑ + Query2Doc19.5-3.923.3-0.119.0+3.020.6↑-0.3↑ + GEM Resp.15.8-6.219.9-1.914.8-1.016.9↓-3.0↓ RepLLaMA24.0-8.924.5-1.820.6+1.323.0-3.1 + HyDE27.8-4.524.0-0.622.9+4.524.9↑-0.2↑ + Query2Doc27.2-2.422.9+0.221.5+2.623.8↑+0.1↑ + GEM Resp.22.7-8.723.4-1.318.9+3.021.6↓-2.3↑ ReasonIR-8B25.3-2.823.1+0.618.7+0.922.4-0.4 + HyDE26.0-1.022.5+1.721.8+4.623.4↑+1.7↑ + Query2Doc23.1-4.422.8+0.420.6+1.022.2↓-1.0↓ + GEM Resp.19.6-8.522.3-0.818.9+0.420.2↓-2.9↓ Promptriever28.3+11.728.5+6.421.6+15.426.1+11.2 + HyDE31.7+4.728.5+4.624.3+13.328.2↑+7.5↓ + Query2Doc30.3+6.326.0+3.123.3+13.426.6↑+7.6↓ + GEM Resp.28.6+10.030.1+3.722.1+15.126.9↑+9.6↓ GEM26.7+11.626.2+9.323.5+14.225.5+11.7 Table 3: Comparisons between GEM and dense retrieval using LLM-based query expansion on the FollowIR dataset. "GEM Resp." means query expansion using GEM’s response.↑indicates improvements over the corresponding baselines;↓ denotes declines. SettingFollowIRBRIGHT p-MRRnDCG@10 GEM+11.729.1 Model Ablation GEM reasoner + GEM encoder (w/oL gen )+12.530.0** GEM w/o generation+9.5*21.0*** Data Ablation GEM w/o HQ+8.5**28.4* GEM w/o Orig. Samples +8.7*27.5** GEM w/o HQ and Orig. Samples+9.2*28.2* GEM w/o Doc. Gen.+11.725.8*** Embedding-Only Variants Qwen3-4B-Instruct+6.8**21.4*** Qwen3-4B-Instruct + HQ+5.9***22.0*** Qwen3-4B-Instruct w/ GEM data+7.3**24.5*** Table 4: Ablation study. We report the averaged metrics for FollowIR and BRIGHT. */**/*** denotes a signif- icant difference from GEM atp < 0.05/0.01/0.001, respectively, based on the two-tailed pairwise t-test. sion is insufficient to achieve strong instruction- following retrieval performance like GEM. RQ5: What is the trade-off of unifying gen- eration and embedding in GEM? Intuitively, causal language modelling may constrain the model’s capacity to learn embeddings. To test this, we train an embedding-only version of GEM by disabling generation loss on identical training data, referred to as GEM encoder (w/oL gen ). Due to catastrophic forgetting, this bi-encoder generates repetitive or mixed-language content (see exam- ple in Appendix D). To evaluate embedding per- formance under the same encoded sequences, we reuse the reasoning generated by GEM for this embedding-only variant. This setup is analogous to a two-stage pipeline of separate models, and the results are reported in Table 4. Indeed, our unified model (GEM) shows a slight degradation in p-MRR on FollowIR (+12.5 → +11.7), and nDCG@10 on BRIGHT (30.0 → 29.1). Inter- estingly, GEM remains strong on generative tasks with notable improvements on reasoning. Results on generation tasks are provided in Appendix B.1. RQ6: What are the effects of different compo- nents in our training data?Table 4 presents our ablation study for GEM and Qwen3-4B-Instruct un- der different training data settings. First, incorporat- ing hard queries (HQ) from ReasonIR into our data generation substantially improves p-MRR on Fol- lowIR (+8.5→ +11.7), whereas training Qwen3- 4B-Instruct with such hard queries (i.e. Qwen3-4B- Instruct + HQ) does not yield similar gains. We postulate that reasoning over these hard queries pro- duces more complex relevance criteria than short factual queries, providing stronger contrastive sig- nals for learning instruction-following retrieval. Additionally, adding non-reasoning, original samples (Orig. Samples) from Promptriever im- proves both FollowIR p-MRR and BRIGHT nDCG@10.We conjecture that these non- reasoning samples regularise GEM, mitigating overfitting to its reasoning. Interestingly, disabling document generation (i.e. GEM w/o Doc. Gen.) substantially reduces nDCG@10 on BRIGHT (29.1 → 25.8), while FollowIR p-MRR remains stable. This is likely because FollowIR queries contain detailed relevance instructions, which ease alignment given GEM’s reasoning. Finally, GEM achieves significant improvements over Qwen3-4B- Instruct trained on the same queries and documents. 5 Conclusion We propose GEM, a generative embedding model that unifies causal language modelling and repre- sentation learning. We present a tailored training data generation method to align GEM’s embed- ding with reasoning, and a joint training objective to maintain its generative capabilities while learn- ing embeddings. GEM demonstrates strong per- formance on reasoning-intensive and instruction- following retrieval tasks, despite being a 4B- parameter model trained with substantially less compute than larger baselines. GEM’s generative nature allows test-time compute scaling through prompting like regular instruction-tuned LLMs. Limitations Due to limited computing resources, we are un- able to replicate our experiments with larger back- bones (e.g. 7B parameters) and larger-scale training data. Our model generalisation experiments in Ap- pendix B.3 are restricted to LLM backbones with up to 4B parameters. Although our data generation pipeline filters out low-quality responses, hallucinations during document generation and inference are inevitable. Whether hallucinations in generated documents and/or GEM’s reasoning can bias its retrieval, and how to quantify the impact, are not investigated. Similarly, inaccuracies in our LLM-based filtering may propagate noise. Similar to existing studies that use LLM reason- ing to improve retrieval (Lei et al., 2025; Sun et al., 2026; Su et al., 2025; Shao et al., 2025), GEM is also limited by the cost of generation. While GEM saves query-side encoding time by reusing the KV cache and allows test-time compute scaling through prompting (see Figure 3), autoregressive decoding remains expensive. Further latency discussion is provided in Appendix B.4. While we note that GEM remains strong on gen- erative tasks (see Appendix B.1), extending it to broader applications that involve retrieval—such as retrieval-augmented generation and conversational search—is valuable for future research. We hope GEM will motivate further exploration of genera- tive embedding models to fill the gaps. References Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Ha- jishirzi, and Wen-tau Yih. 2023. Task-aware Re- trieval with Instructions. In Findings of the Asso- ciation for Computational Linguistics: ACL 2023, pages 3650–3675, Toronto, Canada. Association for Computational Linguistics. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2Vec: Large Language Mod- els Are Secretly Powerful Text Encoders. In First Conference on Language Modeling. Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sidney Black, Jordan Clive, Anthony DiPofi, Julen Etxaniz, Ben- jamin Fattori, Jessica Zosa Forde, Charles Foster, Jef- frey Hsu, Mimansa Jaiswal, Wilson Y. Lee, Haonan Li, and 11 others. 2024. Lessons from the Trenches on Reproducible Evaluation of Language Models. Preprint, arXiv:2405.14782. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. Evaluating Large Language Models Trained on Code. Preprint, arXiv:2107.03374. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Pro- ceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have Solved Question An- swering? Try ARC, the AI2 Reasoning Challenge. Preprint, arXiv:1803.05457. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Prob- lems. Preprint, arXiv:2110.14168. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. Preprint, arXiv:2102.07662. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. Preprint, arXiv:2003.07820. Tri Dao. 2024. FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning. In The Twelfth International Conference on Learning Repre- sentations. Debrup Das, Sam O’Nuallain, and Razieh Rahimi. 2025. RaDeR: Reasoning-aware Dense Retrieval Models. In Proceedings of the 2025 Conference on Empiri- cal Methods in Natural Language Processing, pages 19970–19997, Suzhou, China. Association for Com- putational Linguistics. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise Zero-Shot Dense Retrieval without Relevance Labels. In Proceedings of the 61st Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 1762–1777, Toronto, Canada. Association for Computational Lin- guistics. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 6894–6910, Online and Punta Cana, Do- minican Republic. Association for Computational Linguistics. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schel- ten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mi- tra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The Llama 3 Herd of Models. Preprint, arXiv:2407.21783. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Un- derstanding. In International Conference on Learn- ing Representations. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebas- tian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Informa- tion Retrieval with Contrastive Learning. Transac- tions on Machine Learning Research. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Men- sch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guil- laume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. Preprint, arXiv:2310.06825. Diederik P. Kingma and Jimmy Ba. 2017.Adam: A Method for Stochastic Optimization. Preprint, arXiv:1412.6980. Nils Knoth, Antonia Tolzin, Andreas Janson, and Jan Marco Leimeister. 2024. AI literacy and its impli- cations for prompt engineering strategies. Computers and Education: Artificial Intelligence, 6:100225. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Pro- ceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gon- zalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, page 611–626, New York, NY, USA. As- sociation for Computing Machinery. Yibin Lei, Tao Shen, and Andrew Yates. 2025. ThinkQE: Query Expansion via an Evolving Think- ing Process. In Findings of the Association for Com- putational Linguistics: EMNLP 2025, pages 17772– 17781, Suzhou, China. Association for Computa- tional Linguistics. Hongyu Li, Liang Ding, Meng Fang, and Dacheng Tao. 2024. Revisiting Catastrophic Forgetting in Large Language Model Tuning. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2024, pages 4297–4308, Miami, Florida, USA. Association for Computational Linguistics. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An Empirical Study of Catas- trophic Forgetting in Large Language Models During Continual Fine-Tuning. IEEE Transactions on Audio, Speech and Language Processing, 33:3776–3786. Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. 2023. Fine-Tuning LLaMA for Multi- Stage Text Retrieval. Preprint, arXiv:2310.08319. Niklas Muennighoff, Hongjin SU, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2025. Generative Representational Instruction Tuning. In The Thirteenth International Conference on Learning Representations. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MA- chine Reading COmprehension Dataset.CoRR, abs/1611.09268. Hanseok Oh, Hyunji Lee, Seonghyeon Ye, Haebin Shin, Hansol Jang, Changwook Jun, and Minjoon Seo. 2024. INSTRUCTIR: A Benchmark for Instruction Following of Information Retrieval Models. Preprint, arXiv:2402.14334. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Proceedings of the 36th Interna- tional Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, An- ton Tsitsulin, and 178 others. 2024. Gemma 2: Im- proving Open Language Models at a Practical Size. Preprint, arXiv:2408.00118. Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In TREC, volume 500- 225 of NIST Special Publication, pages 109–126. Na- tional Institute of Standards and Technology (NIST). Rulin Shao, Rui Qiao, Varsha Kishore, Niklas Muen- nighoff, Xi Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Wen tau Yih, Pang Wei Koh, and Luke Zettlemoyer. 2025. Rea- sonIR: Training Retrievers for Reasoning Tasks. In Second Conference on Language Modeling. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. One Embedder, Any Task: Instruction-Finetuned Text Em- beddings. In Findings of the Association for Compu- tational Linguistics: ACL 2023, pages 1102–1121, Toronto, Canada. Association for Computational Lin- guistics. Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Liu Haisu, Quan Shi, Zachary S Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O Arik, Danqi Chen, and Tao Yu. 2025. BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval. In The Thirteenth International Conference on Learning Representations. Duolin Sun, Meixiu Long, Dan Yang, Junjie Wang, Yecheng Luo, Yue Shen, Jian Wang, Hualei Zhou, Chunxiao Guo, Peng Wei, Jiahai Wang, and Jinjie Gu. 2026. DIVER: A Multi-Stage Approach for Reasoning-intensive Information Retrieval. Preprint, arXiv:2508.07995. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024a. Text Embeddings by Weakly- Supervised Contrastive Pre-training.Preprint, arXiv:2212.03533. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024b. Improving Text Embeddings with Large Language Models. In Proceedings of the 62nd Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 11897–11916, Bangkok, Thai- land. Association for Computational Linguistics. Liang Wang, Nan Yang, and Furu Wei. 2023a. Query2doc: Query Expansion with Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 9414–9423, Singapore. Association for Com- putational Linguistics. Xiao Wang, Sean MacAvaney, Craig Macdonald, and Iadh Ounis. 2023b. Generative Query Refor- mulation for Effective Adhoc Search.Preprint, arXiv:2308.00415. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a. Finetuned Language Models are Zero-Shot Learners. In International Conference on Learning Representations. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-thought prompt- ing elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, and Luca Soldaini. 2025a. FollowIR: Evalu- ating and Teaching Information Retrieval Models to Follow Instructions. In Proceedings of the 2025 Con- ference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11926–11942, Albuquerque, New Mexico. As- sociation for Computational Linguistics. Orion Weller, Benjamin Van Durme, Dawn Lawrie, Ash- win Paranjape, Yuhao Zhang, and Jack Hessel. 2025b. Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models. In The Thirteenth International Conference on Learning Representa- tions. Orion Weller, Kathryn Ricci, Eugene Yang, Andrew Yates, Dawn Lawrie, and Benjamin Van Durme. 2025c. Rank1: Test-Time Compute for Reranking in Information Retrieval. In Second Conference on Language Modeling. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muen- nighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources For General Chinese Embeddings. Preprint, arXiv:2309.07597. Ruiran Yan, Wen Xiong, Ze Liu, Chaozhuo Li, Hao Liao, Defu Lian, and Zheng Liu. 2026. Let retrievers think before action: Thought-augmented embedding for dense retrieval. In Findings of the Association for Computational Linguistics: ACL 2026, pages 32032–32052, San Diego, California, United States. Association for Computational Linguistics. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Day- iheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025a. Qwen3 Technical Report. Preprint, arXiv:2505.09388. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jian- hong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 oth- ers. 2025b. Qwen2.5 Technical Report. Preprint, arXiv:2412.15115. Hansi Zeng, Julian Killingback, and Hamed Zamani. 2025.Scaling Sparse and Dense Retrieval in Decoder-Only LLMs. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’25, page 2679–2684, New York, NY, USA. Associa- tion for Computing Machinery. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Mod- els. Preprint, arXiv:2506.05176. Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Des- maison, Can Balioglu, Pritam Damania, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Ajit Mathews, and Shen Li. 2023. PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel. Preprint, arXiv:2304.11277. Enyu Zhou, Zhiheng Xi, Long Ma, Zhihao Zhang, Shi- han Dou, Zhikai Lei, Guoteng Wang, Rui Zheng, Hang Yan, Tao Gui, Qi Zhang, and Xuanjing Huang. 2026. Steering LLMs via Scalable Interactive Over- sight. Preprint, arXiv:2602.04210. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Sid- dhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. Instruction-Following Evaluation for Large Language Models. Preprint, arXiv:2311.07911. Jianqun Zhou, Yuanlei Zheng, Wei Chen, Qianqian Zheng, Shang Zeyuan, Wei Zhang, Rui Meng, and Xi- aoyu Shen. 2025. Beyond Content Relevance: Evalu- ating Instruction Following in Retrieval Models. In The Thirteenth International Conference on Learning Representations. A Implementation Details A.1 Training All our models, including those in the ablation study, are trained for 500 steps. We use a per-device batch size of 8 and 32 gradient accumulation steps, which correspond to an effective batch size of 512 on 2 GPUs. Following (Muennighoff et al., 2025; Su et al., 2023), we use a training group size of 2, i.e. every query is paired with one positive and one hard negative. In-batch negatives are then gathered across all GPUs (Muennighoff et al., 2025). We use a max length of 1024 tokens for both documents and prompt-response pairs. We use the Adam optimiser (Kingma and Ba, 2017). The learning rate is set to1× 10 −5 with 50 warmup steps. For training loss, we setλ gen to0.1, andλ emb to1.0; a parameter study is pro- vided in Appendix B.2. We empirically used a smaller weight (i.e.λ gen = 0.1) for the generation loss because responses are sampled from the same backbone with the same prompt, and our goal is to preserve the original output distribution while primarily optimising GEM for embedding. The temperatureτfor contrastive loss is set to0.02. Although Eq. 5 allows decoupled batches for gen- eration and embedding—potentially with differ- ent batch sizes—we compute both using a shared batch of prompt-response pairs. We interpret this as a contrastive objective regularised by causal lan- guage modelling, which helps prevent the model from forgetting token prediction while learning to encode the same sequences. This design choice also reduces training cost, as the hidden states for computingL gen can be reused forL emb . Investi- gating the effects of a decoupled training strategy is left for future work. To optimise GPU memory usage, we use Py- Torch Fully Sharded Data Parallel (FSDP) (Zhao et al., 2023) with CPU offloading and gradient checkpointing. Mixed precision training is en- abled withbfloat16. Implementation is supported through Hugging Face Accelerate. 3 All our models are trained on a single node with 2 NVIDIA H100 GPUs. For our proposed GEM withQwen3-4B-Instruct-2507backbone, train- ing completes in approximately 14 hours. A.2 Data Generation Weaugmentthetrainingdatafrom Promptriever (Weller et al., 2025b) and a collection of hard queries from ReasonIR (Shao et al., 2025) to train GEM. The Promptriever data contains 491K samples from the Tevatron’s version 4 of MS MARCO (Nguyen et al., 2016) and around 500K instruction-rich samples curated by Weller et al. (2025b). These instruction-rich samples were constructed based on MS MARCO queries by adding detailed relevance instructions, and documents were generated using GPT-4. We refer readers to (Weller et al., 2025b) for details. Note that the non-instructed queries from MS MARCO are typically short and straightforward. The collection from ReasonIR consists of ap- proximately 100K hard queries involving long, challenging problems. For each query, we sampleK = 8responses from GEM’s backbone and filter them by validat- ing against the original positives. The prompt for sampling is provided in Figure 6. For filtering, we use the prompt in Figure 8, where the same LLM backbone is provided with its generated rea- soning and the original positive document. The sampling temperature is set to1.0. As described in Section 3.1, we discard queries with no valid responses. For each of the remaining queries, we randomly select one valid response to associate 3 https://github.com/huggingface/accelerate 4 https://huggingface.co/datasets/Tevatron/ msmarco-passage-aug with them. Fordocumentgeneration,weemploy Llama-3.1-8B-Instructwithgreedyde- coding to generate one positive and one hard negative per query, given the sampled response. Particularly, the instruction-rich queries from Promptriever contain clearly defined relevance criteria and the corresponding documents (posi- tives and hard negatives) are already generated by GPT-4 based on the criteria. Therefore, we exclude this subset from our document generation and rely on our filtering to ensure the alignment between the responses and GPT-4 generated documents.Figure 9 provides a prompt for document generation.For generating positive documents, we setrelevanceto "is relevant". For hard negatives, we randomly assign one of two constraints—"is irrelevant because it fails to satisfy the search intent" or "is irrelevant because it fails to meet some requirements"—each with50% probability. Following Wang et al. (2024b), we generate documents with varying lengths. Instead of specifying an exact word count, we instruct the model with "short", "medium", "long (less than 1000 words)", with probabilities of50%, 45%, and5%, respectively. To further improve the diversity of generated documents and mitigate potential hallucinations, we use a modified prompt in Figure 10 and include one example positive or hard negative document from the original sample. This variant is applied for50%of the time. Finally, we reuse the filtering prompt from Figure 8 but to filter out generated documents that are inconsistent with our relevance instruction. An example of our document generation is provided in Table 17. We run both response and document genera- tion on a single NVIDIA H100 GPU using vLLM (Kwon et al., 2023). Additionally, we randomly sample 60,000 in- stances from the original Promptriever data. For these non-reasoning samples, standard contrastive learning is applied, and queries are encoded sim- ply using the document-side prompt in Figure 7. Our final training set for GEM consists of 370,000 samples: 320,000 from Promptriever (comprising 260,000 with reasoning and 60,000 original), and 50,000 reasoning samples with hard queries from ReasonIR. The size of our 260K reasoning subset is chosen to match our fixed training budget (500 steps with batch size 512), ensuring fair ablation experiments where models never see repeated sam- ples during training. For training our embedding-only variant, Qwen3-4B-Instruct, we sampled 320,000 instances from the original Promptriever data to ensure a con- sistent ratio with GEM. For our ablation study in Ta- ble 4, we additionally added 50,000 samples (with hard queries) from ReasonIR for training Qwen3- 4B-Instruct, similar to GEM’s data setting. A.3 Evaluation Setup For reproducibility, we use greedy decod- ing in our evaluation. Unless otherwise specified, we set the maximum generation length to 1024 to- kens. The batch size of generate-then-encode is set to 16. For documents and queries when generation is disabled, the encoding batch size is set to 32. For all datasets, both queries and documents are truncated to a maximum length of 1024 tokens. All evaluation experiments are conducted on a single GPU. Unless otherwise specified, we use NVIDIA A6000 for BRIGHT and NVIDIA L40S for Fol- lowIR and InstructIR. To save GPU memory usage, all models are loaded in bfloat16. Particularly, for test-time compute scaling exper- iments (see Figure 3), we run models on NVIDIA L40S using a batch size of 1. We randomly sampled 512 queries from BRIGHT to benchmark per-query encoding time under different target length settings. The maximum generation length is set to 8192 to ensure sufficient generation budget. For simplic- ity, we report the actual generated length based on whitespace-separated words rather than tokens. As described in Section 4.1, for our Table 1 and 2, we compile baseline results from Rea- sonIR (Shao et al., 2025), Promptriever (Weller et al., 2025b), and FollowIR (Weller et al., 2025a). Evaluation Tools For evaluation, we use the of- ficial implementations released by the authors of BRIGHT (Su et al., 2025), FollowIR (Weller et al., 2025a), and InstructIR (Oh et al., 2024), available on GitHub. 5 Following their original evaluation protocols, we report results using the standard met- rics provided in these codebases. LLM-based Query Expansion We reproduced HyDE (Gao et al., 2023) and Query2Doc (Wang et al., 2023a) for our experiments in Table 3. For HyDE, we follow the implementation from its of- ficial GitHub. 6 For Query2Doc, the official code 5 BRIGHT: https://github.com/xlang-ai/BRIGHT; FollowIR: https://github.com/orionw/FollowIR; InstructIR: https://github.com/kaistAI/InstructIR 6 https://github.com/texttron/hyde Model IFEvalMMLUARC-ChallengeGSM8KHumanEval Acc.Acc.Acc.EMPass@1 GEM (Qwen3-4B-Instruct)84.365.148.082.189.6 Qwen3-4B-Instruct86.461.843.376.991.5 Llama3.2-3B-Instruct 75.957.744.176.251.8 Llama3.1-8B-Instruct80.763.152.784.467.1 Gemma2-2B-Instruct62.529.143.554.142.1 Qwen2.5-3B-Instruct65.964.643.061.374.4 Table 5: Results on generation tasks. is not available, and thus, we implement it based on the paper. Instead of randomly sampling few- shot examples for each query during inference, we use the few-shot prompt provided in their pa- per (Wang et al., 2023a) to ensure reproducibility. For a fair comparison, we employ GEM’s back- boneQwen3-4B-Instruct-2507as the hypotheti- cal document (answer) generator for both HyDE and Query2Doc, and each method generates one response for query expansion through greedy de- coding. B Additional Experiments B.1 Results on Generation Tasks To show that GEM maintains its generative capabil- ities across different tasks, we use Language Model Evaluation Harness 7 (lm-eval) (Biderman et al., 2024) to conduct evaluation on the following tasks: •IFEval (Zhou et al., 2023) evaluates a model’s instruction-following capability. We report the average accuracy scores across its 4 dif- ferent settings, including prompt-level strict accuracy, instruction-level strict accuracy, prompt-level loose accuracy, and instruction- level loose accuracy. •MMLU (Hendrycks et al., 2021) contains knowledge-intensive questions from diverse domains. We report the average accuracy. • ARC-Challenge is the "Challenge" subset from ARC (Clark et al., 2018), consisting of scientific reasoning questions. We report the average accuracy. •GSM8K (Cobbe et al., 2021) focuses on math- ematical reasoning. We report the exact match 7 https://github.com/EleutherAI/ lm-evaluation-harness accuracy in the chain-of-thought setting im- plemented in lm-eval. •HumanEval (Chen et al., 2021) evaluates Python code generation. We report Pass@1. For all these tasks, we use their default configu- rations fromlm-evalexcept that sampling is dis- abled to ensure reproducibility. Table 5 presents results from GEM, Qwen3-4B- Instruct (i.e.Qwen3-4B-Instruct-2507) (Yang et al., 2025a), Llama3.2-3B-Instruct (Grattafiori et al., 2024), Llama3.1-8B-Instruct (Grattafiori et al., 2024), Gemma2-2B-Instruct 8 (Riviere et al., 2024), and Qwen2.5-3B-Instruct (Yang et al., 2025b). Overall, while GEM is enabled for embed- ding, it maintains strong generative performance that is competitive with its backbone, Qwen3-4B- Instruct. In particular, GEM shows substantial gains on the evaluated reasoning benchmarks, im- proving accuracy on ARC-Challenge from43.3to 48.0 and EM score on GSM8K from 76.9 to 82.1. 0.010.030.10.31.0 gen 10 20 30 40 50 Score BRIGHT nDCG@10 FollowIR p-MRR InstructIR Robustness@10 Figure 4: Average nDCG@10 on BRIGHT, p-MRR on FollowIR, and Robustness@10 on InstructIR with λ gen ∈0.01, 0.03, 0.1, 0.3, 1.0. 8 https://huggingface.co/google/gemma-2-2b-it Setting StackExchangeCodingTheorem-basedAvg. Bio.Earth.Econ.Psy.Rob.Stack.Sus.Leet.PonyAoPSTheoQ.TheoT. λ gen = 0.0131.736.627.031.723.429.225.238.32.917.243.640.028.9 λ gen = 0.0332.538.927.732.023.929.126.437.22.416.642.740.729.2 λ gen = 0.1 (default)34.740.826.533.724.629.326.035.52.816.637.641.829.1 λ gen = 0.333.439.827.132.725.130.129.536.81.916.937.440.929.3 λ gen = 1.030.235.625.930.323.426.827.735.21.315.636.638.827.3 Table 6: nDCG@10 on BRIGHT for GEM with different λ gen . Best results are shown in bold. Setting FollowIRInstructIR Robust04News21Core17AverageMS MARCO MAPp-MRRnDCGp-MRRMAPp-MRRScorep-MRRnDCG@10Robust.@10 λ gen = 0.0128.0 +13.326.5+8.725.4 +15.126.6 +12.483.949.2 λ gen = 0.0326.4+10.526.0+9.923.7+13.725.4+11.488.356.2 λ gen = 0.1 (default)26.7+11.626.2+9.323.5+14.225.5+11.787.554.8 λ gen = 0.327.0+8.726.9+8.324.9+14.426.3+10.586.852.2 λ gen = 1.0 26.3+9.127.4+7.523.6+14.925.7+10.587.153.3 Table 7: Results on FollowIR and InstructIR for GEM with different λ gen . Best results are shown in bold. B.2 Results with Different Generation Loss Weights To investigate the effects ofλ gen for loss weight- ing (see Eq. 5), we trained GEM withλ gen ∈ 0.01, 0.03, 0.1, 0.3, 1.0 , whereλ gen = 0.1is the default setting used in previous experiments. Table 6 and Table 7 provide the full results on BRIGHT and instruction-following retrieval benchmarks, respectively. Figure 4 shows the averaged metrics across different values ofλ gen . Overall, we do not observe a clear trend asλ gen varies, although the results appear more stable for λ gen ∈ 0.03, 0.1, 0.3. These findings suggest that GEM’s retrieval performance is relatively in- sensitive to the exact weighting of the generation loss, provided thatλ gen lies within a reasonable range. B.3 Results with Different Backbones We evaluate GEM across different backbones under two settings: (1) the default setting de- scribed in Section 3.1, where reasoning is gener- ated by the corresponding backbone being trained, and (2) a distillation setting in which student models are trained using reasoning generated by Qwen3-4B-Instruct-2507. In the first setting, document generation is also re-run to align with reasoning produced by a different backbone. Par- ticularly, we increaseλ gen to 0.2 for the distillation setting, as the responses are not sampled from the same backbone. The backbones we used include Qwen3-4B-Instruct (Yang et al., 2025a), Llama3.2- 3B-Instruct (Grattafiori et al., 2024), Gemma2-2B- Instruct (Riviere et al., 2024), and Qwen2.5-3B- Instruct (Yang et al., 2025b). Table 8 provides results on FollowIR and Instruc- tIR. When trained using reasoning generated by the corresponding backbone, GEM (Llama3.2-3B- Instruct) achieves stronger performance on Instruc- tIR. We also observe that models can be effectively trained in the distillation setting, producing results competitive to GEM (Qwen3-4B-Instruct). Table 9 presents results on BRIGHT. Interest- ingly, for GEM (Llama3.2-3B-Instruct), the dis- tillation setting yields stronger performance than using self-generated reasoning. This suggests that the responses sampled from this backbone may be of comparatively lower quality for BRIGHT. In addition, GEM (Gemma2-2B-Instruct) performs substantially worse on BRIGHT, likely due to its limited reasoning capacity. 01020304050 Latency (s) 22 24 26 28 30 nDCG@10 64 256 512 1024 Default BRIGHT Reasoning Prompt Embedding-Only Latency vs nDCG@10 GEM with instructed length 641024 GEM with default prompt BRIGHT Reasoning Prompt Qwen3-4B-Instruct (embedding-only) Figure 5: Latency and nDCG@10 on BRIGHT. Model FollowIRInstructIR Robust04News21Core17AverageMS MARCO MAPp-MRRnDCGp-MRRMAPp-MRRScorep-MRRnDCG@10Robust.@10 Training with responses from the corresponding backbone GEM (Qwen3-4B-Instruct)26.7+11.626.2+9.323.5+14.225.5+11.787.554.8 GEM (Llama3.2-3B-Instruct) 28.9+10.924.5+6.523.1+11.425.5+9.689.859.0 Distillation using Qwen3 data GEM (Llama3.2-3B-Instruct)29.3+8.324.8+6.423.7+14.125.9+9.686.451.2 GEM (Gemma2-2B-Instruct)22.4+13.625.4+11.721.5+11.223.1+12.284.448.3 GEM (Qwen2.5-3B-Instruct)28.2+11.621.8+10.723.9+9.324.6+10.588.155.5 Table 8: Results on FollowIR and InstructIR with different backbones. Model StackExchangeCodingTheorem-basedAvg. Bio.Earth.Econ.Psy.Rob.Stack.Sus.Leet.PonyAoPSTheoQ.TheoT. Training with responses from the corresponding backbone GEM (Qwen3-4B-Instruct)34.740.826.533.724.629.326.035.52.816.637.641.829.1 GEM (Llama3.2-3B-Instruct)27.133.922.327.411.919.521.634.03.112.928.330.522.7 Distillation using Qwen3 data GEM (Llama3.2-3B-Instruct)33.241.726.729.420.530.427.132.610.09.933.834.627.5 GEM (Gemma2-2B-Instruct)22.431.217.522.915.623.319.728.10.496.426.912.318.9 GEM (Qwen2.5-3B-Instruct)28.735.920.729.117.425.223.235.72.611.935.627.524.5 Table 9: nDCG@10 on BRIGHT with different backbones. B.4 Inference Latency To supplement Figure 3, we evaluate the combined latency of generation and embedding. Following prior work (Su et al., 2025; Shao et al., 2025), we consider a pipeline comprising an LLM-based reasoner and an LLM-based retriever as our base- line. For reasoning, we adopt the same prompt from BRIGHT, which is available on their offi- cial GitHub. 9 This prompt is also adopted in Rea- sonIR (Shao et al., 2025). For a fair comparison with GEM, this baseline uses GEM’s backbone, Qwen3-4B-Instruct-2507, as its reasoner. Down- stream encoding is performed by our embedding- only variant (i.e. Qwen3-4B-Instruct). All models are benchmarked on the same node with a single NVIDIA L40S GPU. We measure per-query la- tency by randomly sampling a subset of 128 queries from BRIGHT. We use greedy decoding and a batch size of 1. The maximum length for gen- eration and the truncation length for embedding are increased to 8192. Figure 5 compares the total inference la- tency and nDCG@10 of different settings on BRIGHT. While reasoning substantially improves nDCG@10 on BRIGHT, autoregressive generation incurs considerable latency overhead compared to the millisecond-scale embedding-only setup (i.e. Qwen3-4B-Instruct with original queries). Al- 9 https://github.com/xlang-ai/BRIGHT/blob/main/ reason.py though GEM allows test-time compute scaling through prompting, the cost of generation remains a limitation, similar to prior studies that rely on LLM- based query expansion to enhance retrieval (Shao et al., 2025; Sun et al., 2026; Lei et al., 2025; Gao et al., 2023; Wang et al., 2023a). Compared with the baseline prompting approach used in BRIGHT and ReasonIR, GEM achieves the same nDCG@10 of 27.3 with a much smaller generation budget (n = 64), reducing inference latency to 2.85 seconds. This represents approximately a 20× speedup, demonstrating that GEM’s test-time com- pute scaling can preserve retrieval quality while substantially reducing generation costs. In practice, GEM’s autoregressive generation can be accelerated using well-established optimisa- tion techniques, such as vLLM (Kwon et al., 2023) and FlashAttention (Dao, 2024). For simplicity and to provide a consistent evaluation setting, we do not incorporate these optimisations. All latency measurements in this paper are evaluated using the default HuggingFace implementation. B.5 Evaluation on Additional IR Datasets Table 10 reports nDCG@10 on TREC-DL- 2019 (Craswell et al., 2020) and TREC-DL- 2020 (Craswell et al., 2021) with the MS MARCO corpus (Nguyen et al., 2016) which contains8.8M passages. Compared with the embedding-only vari- ants, GEM improves retrieval performance on these non-reasoning-intensive benchmarks by leveraging its own knowledge. The results are comparable to those of 7B models reported by Weller et al. (2025b). ModelDL-2019DL-2020 7B Models RepLLaMA74.571.8 Promptriever73.272.3 4B Models DIVER-4B 10 67.863.6 Qwen3-4B-Instruct69.069.1 GEM w/o generation67.465.8 GEM70.473.6 Table 10: nDCG@10 on TREC-DL-2019 and TREC- DL-2020. B.6 Discussion about Performance on Pony We noted that GEM and Promptriever underper- form on the Pony programming language subset of BRIGHT (see Table 1). This subset has a cor- pus of approximately 8K documents. We refer readers to the official HuggingFace repository 11 for examples. We postulate the problem is because our training data is largely adopted from Promptriever (MS MARCO). Notably, Promptriever itself col- lapses on Pony (1.7nDCG@10 in Table 1). Fur- thermore, all GEM variants, regardless of back- bone (see Table 9), perform poorly on Pony, similar to Promptriever. These observations indicate that Pony is out-of-domain with respect to our training data. Additionally, we found that proprietary embed- dings (OpenAI, Voyage, and Google) reported in the ReasonIR paper (Shao et al., 2025) collapse on Pony as well, with nDCG@10 ranging from1.5to 3.6. B.7 Differences from Prior Work Here we reiterate the differences between GEM and our main external baselines, as follows: • GritLM: GritLM (Muennighoff et al., 2025) operates in separate modes to perform either embedding or generation. While GritLM sup- ports general question answering and embed- ding, it is not a model that leverages the gener- ative (reasoning) capabilities to enhance query representations. Additionally, its embedding 11 https://huggingface.co/datasets/xlangai/ BRIGHT/viewer/documents/pony mode requires bidirectional attention masking, which can not be unified with causal language modelling in the incremental manner of GEM. GritLM is therefore orthogonal to the research questions in our paper. Results for GritLM-7B are presented in Table 1 and 2. •ReasonIR: ReasonIR-8B (Shao et al., 2025) is a bi-encoder trained for reasoning-intensive retrieval. However, it is not a reasoning model itself. Furthermore, GEM’s embedding is trained to align with its own reasoning. •Promptriever:Similarly, Promptriever (Weller et al., 2025b) is also a bi-encoder that is not generative. For instruction-following re- trieval, Promptriever requires detailed instruc- tions to specify document relevance, while user instructions are often underspecified in practice. In contrast, GEM is promptable as it remains an instruction-tuned LLM, and is inherently capable of reasoning over diverse user inputs. C Prompts We provide our prompts as follows: • Generate-Then-Encode. Figure 6 is our de- fault prompt for generate-then-encode and was used for GEM’s data generation and train- ing. • Embedding-Only. Figure 7 is the prompt for encoding documents and also queries when generation is disabled. •Filtering. Figure 8 provides our prompt for filtering. •Document Generation. Figure 9 and 10 pro- vide our document generation prompts with- out and with an example, respectively. Each prompt is selected with a 50% probability. •Test-Time Compute Scaling. Figure 11 pro- vides our prompt for test-time scaling (see Fig- ure 3). Following BRIGHT (Su et al., 2025), we instruct GEM to generate an answer. Ad- ditionally, we explicitly specify the target gen- eration length. Generate-then-encode prompt # Task Identify the search intent. Then provide a concise analysis of the key criteria that make a document relevant. # Query text Figure 6: Prompt for generate-then-encode. Embedding-only prompt # Task Represent the text for retrieval. # Text text Figure 7: Prompt for encoding documents and queries when generation is disabled. Filtering prompt # 1. User query analysis reason # 2. Example document doc # 3. Based on the analysis, is the example document relevant? Just answer Yes or No. Figure 8: Prompt for filtering out low-quality data. D Examples In this section, we provide the following examples: •Case Studies. Table 11–Table 15 present our qualitative case studies, where queries lack detailed relevance criteria. We examine the similarity scores produced by GEM and Promptriever. •Response Generation. Table 16 illustrates that embedding models suffer from catas- trophic forgetting when trained without a causal language modelling objective. In this case, embedding-only models fail to follow our meta-instruction; they generate repetitive content without emitting an EOS token to terminate the generation, although they are trained to use the dedicated embedding token like GEM. •Document Generation. Table 17 provides an example of our document generation, where the positive document aligns with the reason- ing (e.g. location and authoritative source), whereas the hard negative document fails to meet the search intent despite its topical simi- larity. Document generation prompt # Given the following user query analysis reason # Your task You are a data annotator. Based on the above analysis, draft a document that relevance. **Important**: - Preferred length: length - Directly reply with the content **only** - Do NOT disclose/explain why it is relevant or irrelevant # Your document Figure 9: Prompt for document generation without an example. Document generation prompt with example # Given the following user query analysis reason # Your task You are a data annotator. Based on the above analysis, draft a document that relevance. **Important**: - Preferred length: length - Directly reply with the content **only** - Do NOT disclose/explain why it is relevant or irrelevant - You can write in a similar style to the example below; however, make sure your document relevance # Example document example # Your document Figure 10: Prompt for document generation with an example. Prompt for test-time compute scaling # Task Identify the search intent. Then think step by step to reason about the criteria that make a document relevant. Finally, draft an answer. Your response should be approximately length words. # Query text Figure 11: Prompt for test-time compute scaling. QueryWhat is this syntax called? Help me understand it with simple examples. @func2 def func1(arg1, arg2): pass Positive DocumentDecorators let you add extra behavior to a function without chang- ing the function’s code. # Example def log(func): ... Negative DocumentWhy does the syntax func1(func2)(parameters); work and what is it called? From the book Eloquent Javascript Third Edition ... Scores by GEMPositive Document: 0.723 Negative Document: 0.570 Scores by PromptrieverPositive Document: 0.578 Negative Document: 0.594 Table 11: Example of document similarity scores produced by GEM and Promptriever. Scores are highlighted in green when the positive document is ranked higher and in red otherwise. Query"There is always a day when the flower blooms again." Find me some similar poems. Positive DocumentNo winter stays forever. Somewhere beneath the frost, a quiet seed is already dreaming of spring. Negative DocumentEvery bloom has its limit. The perfume fades to nothing. Tomorrow brings only empty stems. The wind carries away the remaining memory. Scores by GEMPositive Document: 0.633 Negative Document: 0.563 Scores by PromptrieverPositive Document: 0.459 Negative Document: 0.471 Table 12: Example of document similarity scores produced by GEM and Promptriever. Scores are highlighted in green when the positive document is ranked higher and in red otherwise. QueryExplain IBS in simple language. Positive DocumentIrritable Bowel Syndrome (IBS) is a common problem with how your gut works. It can cause stomach pain, bloating, gas, diarrhea, or constipation. IBS does not damage your digestive system, but it makes your gut extra sensitive. Think of it like a miscommuni- cation between your brain and your belly. Certain foods, stress, and lack of sleep can trigger these uncomfortable symptoms. You can often manage IBS by changing your diet, reducing stress, and drinking plenty of water. Negative DocumentIrritable Bowel Syndrome (IBS) is a chronic gastrointestinal disor- der characterized by visceral hypersensitivity and altered intestinal motility. The pathophysiology involves dysregulation of the brain- gut axis, microscopic inflammation, and alterations in the gut microbiota. Quantitative diagnostic criteria, such as the Rome IV parameters, require recurrent abdominal pain associated with defecation or changes in stool frequency and form. Management necessitates targeted pharmacotherapy, including antispasmodics, prokinetics, or secretagogues, to address specific pathophysiologi- cal mechanisms. Scores by GEMPositive Document: 0.832 Negative Document: 0.676 Scores by PromptrieverPositive Document: 0.430 Negative Document: 0.430 Table 13: Example of document similarity scores produced by GEM and Promptriever. Scores are highlighted in green when the positive document is ranked higher and in red otherwise. QueryWhich marine organisms do not rely on sunlight for energy? Positive DocumentHydrothermal vent communities use chemosynthesis rather than photosynthesis. Bacteria oxidize toxic chemicals like hydrogen sulphide, meaning they do not require any solar energy to survive. Negative DocumentPhytoplankton and marine algae rely on sunlight for energy to perform photosynthesis. These microscopic organisms form the base of the marine food web by capturing solar radiation. Scores by GEMPositive Document: 0.703 Negative Document: 0.516 Scores by PromptrieverPositive Document: 0.338 Negative Document: 0.391 Table 14: Example of document similarity scores produced by GEM and Promptriever. Scores are highlighted in green when the positive document is ranked higher and in red otherwise. QueryIf I have wind speeds at 80m and 120m above the surface and I am interested in approximating the speed at 100m, is it valid to just average the 80 and 120 speeds? In other words, should I expect wind speed to be a linear function of height or would a nonlinear function be more appropriate? Positive DocumentWind speed increases non-linearly with height due to surface fric- tion and atmospheric stability. Simple linear averaging between 80m and 120m introduces systematic errors because the wind profile follows a logarithmic or power law. To approximate wind speed at 100m accurately, you must use non-linear models like the Logarithmic Wind Profile or the Hellmann power law. These equations account for surface roughness lengths and stability pa- rameters, which cause the velocity curve to shear sharply near the ground and taper off at higher altitudes. Negative DocumentTo find the wind speed at 100m using data from 80m and 120m, you can apply a standard linear interpolation formula. Because 100m sits exactly halfway between 80m and 120m, calculating the simple arithmetic mean of the two known velocities provides a highly accurate and valid approximation. Wind velocity increases at a constant, linear rate within this specific boundary layer in- terval. This straightforward mathematical average eliminates the need for complex, non-linear atmospheric equations or specialized aerodynamic profiling software. Scores by GEMPositive Document: 0.848 Negative Document: 0.680 Scores by PromptrieverPositive Document: 0.707 Negative Document: 0.727 Table 15: Example of document similarity scores produced by GEM and Promptriever. Scores are highlighted in green when the positive document is ranked higher and in red otherwise. Prompt# Task Identify the search intent. Then provide a concise analysis of the key criteria that make a document relevant. Your response should be around 100 words. # Query “There is always a day when the flower blooms again.” Find me some similar poems. Response from GEM**Search Intent:** The user seeks poetic inspiration or similar verses that convey the theme of renewal, resilience, or cyclical beauty—specifically echoing the idea of life returning, like a flower blooming again. **Relevance Criteria:** A document is relevant if it features poems with themes of rebirth, persistence, or natural cycles; uses metaphorical language (e.g., flowers, seasons); and expresses hope or continuity. Poems with emotional depth, rhythmic structure, and symbolic imagery align best with the query’s intent. The tone should be reflective, hopeful, or serene to match the original sentiment. Response from GEM w/oL gen Poem: "The Flower That Blooms Again" by Mary Oliver Key criteria: cyclical nature, renewal, blooming again, daily recurrence, hope, resilience Analysis: This poem captures the recurring cycle of flowers blooming daily, emphasizing hope and resilience through natural repetition. It reflects on the enduring beauty and renewal of life, aligning with the theme of daily renewal and the flower’s persistent return. The poem’s cyclical imagery and hopefulness match the original quote’s essence of continuous blooming and daily hope. cyclic renewal, daily hope, resilience, natural cycles, blooming again. poetic hope, cyclical renewal, daily bloom, resilience, natural cycles. poetic hope, cyclical renewal, daily bloom, resilience, natural cycles. poetic hope, cyclical renewal, daily bloom, resilience, natural cycles . . . Response from Qwen3-4B-Instruct (ourembedding- only variant) Poems similar to "flower blooms again" include "The Sun Also Rises" and "Daybreak" — both capture renewal and cyclical life. Key criteria: themes of rebirth, daily cycles, and hope. These poems share the poetic structure of nature imagery and recurring daily patterns, mirroring the flower’s return. They emphasize resilience and continuity, aligning with the original poem’s intent. Similarity is based on emotional tone, imagery, and cyclical themes. Bloom again poems focus on renewal, not just flowers, but life’s persistence. Poems: "Daybreak," "The Sun Also Rises," "Spring Returns" — all reflect daily renewal and hope. Criteria: renewal, cyclical nature, hope, daily life. Similarity: emotional and thematic resonance with rebirth. Flower bloom poems: not just flowers, but life’s return. Poems: "Daybreak," "The Sun Also Rises," "Spring Returns" — all reflect daily renewal and hope. Criteria: renewal, cyclical nature, hope, daily life. Similarity: . . . Table 16: Example responses from GEM and embedding models trained without generation loss. Repetitive content is truncated and denoted by ". . . ". We prompt with a response-length requirement to save space. GEM’s response contains 123 tokens, while GEM w/oL gen and Qwen3-4B-Instruct continue generating repetitive content until reaching the maximum generation length in our setting. Querywhat is the largest lake in oklahoma Reasoning**Search Intent:** The user is seeking factual, location-based information about the largest natural lake in the state of Oklahoma. This is a straightforward informational query with a clear geographic and physical attribute focus. **Relevance Criteria Analysis:** A document is relevant if it: 1. **Accurately identifies the largest lake in Oklahoma** (e.g., Lake Eufaula or Lake Oklahoma, with precise clarification). 2. Provides **verified geographical and size data** (e.g., surface area in square miles or acres). 3. Specifies the **location** of the lake within Oklahoma. 4. Is **authoritative** (e.g., from government sources, reputable environmental organizations, or geographic databases). 5. Avoids confusion with man-made reservoirs or non-lake bodies of water. *Note:* Lake Thunderbird or Lake Eufaula are often cited, but Lake Eufaula (partially in Oklahoma) is commonly recognized as the largest natural lake in the state by surface area. Accuracy and clarity in distinguishing natural vs. reservoir lakes are critical for relevance. PositiveLake Eufaula is the largest natural lake in Oklahoma, covering an area of approximately 102,000 acres. Located in the southeastern part of the state, it spans across the Oklahoma-Georgia border, with the majority of its surface area situated within Oklahoma. According to the U.S. Army Corps of Engineers, Lake Eufaula has a conservation storage capacity of about 2.3 million acre-feet, making it a vital water resource for the region. Hard NegativeDiscover Lake Pontchartrain. Lake Pontchartrain is a brackish coastal lagoon in Louisiana, covering approximately 630 square miles. It is connected to the Gulf of Mexico and is a popular spot for boating and fishing. The lake is surrounded by the cities of New Orleans and Mandeville. Table 17: An example of our generated documents.