Paper deep dive
Aligning Dense Retrievers with LLM Utility via DistillationAligning Dense Retrievers with LLM Utility via Distillation
Rajinder Sandhu, Di Mu, Cheng Chang, Md Shahriar Tasjid, Himanshu Rai, Maksims Volkovs, Ga Wu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 4/27/2026, 4:08:43 AM
Summary
The paper introduces Utility-Aligned Embeddings (UAE), a framework that improves dense retrieval for Retrieval-Augmented Generation (RAG) by aligning bi-encoders with the generative utility of Large Language Models (LLMs). Instead of relying on semantic similarity, UAE uses a two-stage distillation process: first, distilling noisy perplexity-based utility signals into a stable, parameterized reward model (using DeBERTa), and second, training a dense bi-encoder to imitate the reward model's distribution via a Utility-Modulated InfoNCE objective. Experimental results on QASPER and NewsQA benchmarks show that UAE significantly outperforms standard semantic baselines like BGE-Base in retrieval metrics (Recall, MAP) and downstream generation quality (Token F1), while being over 180x faster than LLM-based re-ranking methods, making it suitable for real-time applications.
Entities (8)
Relation Signals (4)
Utility-Aligned Embeddings โ evaluatedon โ QASPER
confidence 100% ยท On the QASPER benchmark, UAE improves retrieval Recall@1 by 30.59%
DeBERTa โ isusedas โ Reward Model
confidence 100% ยท We initialize the utility reward model with microsoft/deberta-v3-base
Utility-Aligned Embeddings โ uses โ Utility-Modulated InfoNCE
confidence 100% ยท training a bi-encoder to imitate a utility distribution derived from perplexity reduction using a Utility-Modulated InfoNCE objective.
Llama-3-8B-Instruct โ providesutilityfor โ Utility-Aligned Embeddings
confidence 90% ยท we utilize Llama-3-8B-Instruct [7] as the fixed generator... to measure downstream utility
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Dense vector retrieval is the practical backbone of Retrieval- Augmented Generation (RAG), but similarity search can suffer from precision limitations. Conversely, utility-based approaches leveraging LLM re-ranking often achieve superior performance but are computationally prohibitive and prone to noise inherent in perplexity estimation. We propose Utility-Aligned Embeddings (UAE), a framework designed to merge these advantages into a practical, high-performance retrieval method. We formulate retrieval as a distribution matching problem, training a bi-encoder to imitate a utility distribution derived from perplexity reduction using a Utility-Modulated InfoNCE objective. This approach injects graded utility signals directly into the embedding space without requiring test-time LLM inference. On the QASPER benchmark, UAE improves retrieval Recall@1 by 30.59%, MAP by 30.16% and Token F1 by 17.3% over the strong semantic baseline BGE-Base. Crucially, UAE is over 180x faster than the efficient LLM re-ranking methods preserving competitive performance, demonstrating that aligning retrieval with generative utility yields reliable contexts at scale.
Tags
Links
- Source: https://arxiv.org/abs/2604.22722v1
- Canonical: https://arxiv.org/abs/2604.22722v1
Trouble viewing inline? Open PDF directly โ
Full Text
31,008 characters extracted from source content.
Expand or collapse full text
Aligning Dense Retrievers with LLM Utility via Distillation Rajinder Sandhu rajinder@layer6.ai Layer 6 AI Toronto, ON, Canada Di Mu di@layer6.ai Layer 6 AI Toronto, ON, Canada Cheng Chang jason@layer6.ai Layer 6 AI Toronto, ON, Canada Md Shahriar Tasjid tasjid@dal.ca Dalhousie University Halifax, NS, Canada Himanshu Rai himanshu@layer6.ai Layer 6 AI Toronto, ON, Canada Maksims Volkovs maks@layer6.ai Layer 6 AI Toronto, ON, Canada Ga Wu ga.wu@dal.ca Dalhousie University Halifax, NS, Canada Abstract Dense vector retrieval is the practical backbone of Retrieval- Aug- mented Generation (RAG), but similarity search can suffer from precision limitations. Conversely, utility-based approaches lever- aging LLM re-ranking often achieve superior performance but are computationally prohibitive and prone to noise inherent in perplex- ity estimation. We propose Utility-Aligned Embeddings (UAE), a framework designed to merge these advantages into a practical, high-performance retrieval method. We formulate retrieval as a distribution matching problem, training a bi-encoder to imitate a utility distribution derived from perplexity reduction using a Utility-Modulated InfoNCE objective. This approach injects graded utility signals directly into the embedding space without requir- ing test-time LLM inference. On the QASPER benchmark, UAE improves retrieval Recall@1 by 30.59%, MAP by 30.16% and Token F1 by 17.3% over the strong semantic baseline BGE-Base. Crucially, UAE is over 180รfaster than the efficient LLM re-ranking methods preserving competitive performance, demonstrating that aligning retrieval with generative utility yields reliable contexts at scale. Keywords Retrieval-Augmented Generation, Dense Retrieval, Generative Util- ity, Representation Learning, Distribution Matching 1 Introduction Dense vector similarity search remains the bedrock of real-world Retrieval-Augmented Generation (RAG) systems. By mapping queries and candidates into a shared representation space, these systems leverage efficient Approximate Nearest Neighbor (ANN) search to handle large-scale datasets with minimal latency [2,5,11]. However, this paradigm is increasingly criticized for its reliance on semantic similarity as a proxy for generative utility. Growing evidence sug- gests that passages with high semantic similarity (topical overlap) often fail to provide answer-critical information and can even intro- duce semantic distractors that mislead the generator, especially in long-context settings where incorrect but similar passages increase decoding uncertainty [2, 13, 27, 33]. To bridge this gap, current state-of-the-art approaches shift to- ward utility-based retrieval [3,12,28,34], where relevance is defined by how effectively a document helps a Large Language Model (LLM) produce a correct response [11]. In practice, this is often measured via perplexity reduction: a document is considered useful if its pres- ence as context makes the ground-truth answer more predictable 51010010003000 Latency (ms) 38 40 42 44 46 48 50 52 Gen-F1 (%) UAE (Ours) UAE+SePer UAE+LLM Reranker UAE+BGE-Reranker SePer RankGPT BGE-M3 Rerank SPLADE V3 ColBERTv2 NV-Embed V2 E5-Mistral BM25 BGE-Base Ours (Single) Ours (Multi) Base (Dense) Base (Rerank) Frontier Figure 1: Efficiency vs. Performance. UAE (Read Star) occu- pies the optimal sweet spot: it approaches the performance of strong baselines while beingโผ100x faster. to the model [12]. While conceptually sound, these utility-based approaches face a significant practicality wall. Relying on LLMs for query generation or post-hoc re-ranking is computationally prohibitive for large-scale deployment [12,15,22]. Furthermore, utility signals derived from perplexity are notoriously noisy and stochastic, sensitive to token-level variations and decoding dynam- ics that make them difficult to use as stable training targets [9,28]. This necessitates complex, multi-stage architectures that improve performance at the cost of extreme inference latency and high computational overhead [19, 25]. In this work, we propose Utility-Aligned Embeddings (UAE), a framework designed to merge the efficiency of dense retrieval with the superior performance of utility-based methods. Our core insight is that generative utility should be distilled directly into the bi-encoderโs (dual-encoder dense retriever) embedding space, bypassing expensive test-time LLM inference while capturing the generatorโs actual preferences. We formulate this alignment as a distribution matching problem; rather than treating noisy utility scores as direct regression targets (which can lead to unstable learn- ing and overfitting [6,24]) we train the bi-encoder to imitate a utility-induced target distribution via a Utility-Modulated InfoNCE objective (Figure 2). Our empirical results on the QASPER bench- mark [4] demonstrate that UAE significantly outperforms standard semantic retrievers [31], improving reterival Recall@1 by 30.59% and Token F1 by 17.3%. Crucially, UAE occupies a unique Pareto- optimal position: it provides substantial performance gains while arXiv:2604.22722v1 [cs.IR] 24 Apr 2026 , ,Rajinder Sandhu, Di Mu, Cheng Chang, Md Shahriar Tasjid, Himanshu Rai, Maksims Volkovs, and Ga Wu being 180รfaster than the efficient LLM-based re-ranking methods [26]. Figure 1 highlights the advantage of the proposed method. By maintaining standard ANN compatibility, UAE offers a practical, scalable solution for building high-fidelity RAG systems that are both informative and computationally efficient. 2 Utility Aligned Embeddings Our objective is to develop a dense retrieval system where the em- bedding space structure is supervised by generative utility rather than simple semantic overlap. However, the high variance and heavy-tailed nature of raw utility scores (Figure 3 top) make them unsuitable as direct supervision targets for a bi-encoder. To bridge this gap, we decompose the alignment process into a two-stage distillation framework (Figure 2). First, we stabilize the noisy, com- putationally expensive utility signals by distilling them into a pa- rameterized reward model. Second, we align the dense retriever with this model by training it to imitate the resulting utility-induced ranking distribution through supervised distribution matching. 2.1 Parameterized Utility Approximation The utility of a context documentํfor correctly answering a ques- tionํis denoted asํ(ํ,ํ,ํ), whereํrepresents the ground-truth answer. In practice, utility is typically estimated via perplexity: ํ(ํ,ํ,ํ)= exp 1 |ํ| |ํ| โ๏ธ ํก=1 logํ ํ (ํฅ ํก | ํฅ <ํก ,ํ,ํ) ! ,(1) whereํ ํ represents the token-level generation probability of a lan- guage model. Given that natural language allows for multiple valid expressions of the same answerํ โ A ํ , we extend this definition to the expected utilityํ(ํ,ํ,A ํ )=E ํโA ํ [ํ(ํ,ํ,ํ)] . Theoreti- cally, this utility function is deterministic relative to the finite set of ground-truth answersA ํ . In practice, however, the estimation is highly stochastic and fragile; The instability arises from three pri- mary factors: (1) the intractable variety of linguistic expressions for ground-truth answers; (2) non-deterministic model outputs caused by matrix parallelism and floating-point associativity errors; and (3) a high sensitivity to answer length|ํ|. Directly using raw generative utility through regression (us- ing standard objectives like Mean Squared Error or Huber losses) is unreliable due to the aforementioned stochasticity and heavy- tailed distribution. Instead, we formulate reward learning as a pair- wise ranking problem, focusing exclusively on preserving the gen- eratorโs ordinal preferences. We construct training quadruplets (ํ,A ํ ,ํ ํ ,ํ ํ )and optimize a parameterized reward modelํ ํ using a margin-based ranking loss: L Reward = max(0,ํฟโ(ํ ํ (ํ,ํ ํ )โ ํ ํ (ํ,ํ ํ ))),(2) where we enforceํ ํ (ํ,ํ ํ )> ํ ํ (ํ,ํ ํ )+ํฟwheneverํ(ํ,ํ ํ ,A ํ )> ํ(ํ,ํ ํ ,A ํ ). By prioritizing the relative ordering of candidates over the approximation of exact utility values, this formulation provides a more robust signal for learning utility-driven representations. Here,ํ ํ is a Transformer-based encoding model (BERT). As illus- trated in Figure 3 (bottom), this learned reward model effectively discriminates between truly useful contexts and the "hard semantic negatives" that often deceive standard top-1 similarity search. 2.2 Reward-Guided Embeddings Training The reward modelํ ํ effectively captures utility but, as a cross- encoder, is computationally expensive for large-scale inference. We therefore useํ ํ as an offline teacher to distill utility preferences into a dense bi-encoder, preserving ANN search efficiency. Rather than using reinforcement learning (which is destabilized by massive discrete action spaces and high reward variance), we adopt an imitation learning formulation. We treat the normalized reward distribution as a target expert policyํ โ and train the retrieverํ ํ to minimize their divergence via supervised gradients, bypassing the exploration challenges of traditional RL. Distribution Matching Objective. To align the retriever with generative utility, we define a target (teacher) distributionํ โ and a retriever (student) distributionํ ํ over a set of candidate contexts B as follows: ํ โ (ํ ํ | ํ)= exp(ํยท ํ ํ (ํ,ํ ํ )) ร ํ ํ โB exp(ํยท ํ ํ (ํ,ํ ํ )) ,(3) ํ ํ (ํ ํ | ํ)= exp(โจํ ํ (ํ), ํ ํ (ํ ํ )โฉ/ํ) ร ํ ํ โB exp(โจํ ํ (ํ), ํ ํ (ํ ํ )โฉ/ํ) ,(4) whereํis a temperature hyperparameter controlling the sharpness of the target utility distribution andํis the retrieverโs scaling temperature. The retriever is optimized by minimizing the Kullback- Leibler (KL) divergence between the two distributions: L UAE (ํ)=E ํ ํท KL (ํ โ โฅ ํ ํ ) โโ โ๏ธ ํ โ๏ธ ํ ํ โB ํ โ (ํ ํ | ํ) logํ ํ (ํ ํ | ํ). (5) This objective reshapes the embedding space to reflect the gener- atorโs preferences: high-utility documents are pulled toward the query, while low-utility contexts are pushed away. Critically, unlike standard InfoNCE which treats all negatives as equally irrelevant, this formulation preserves the graded, ordinal structure of the util- ity space, allowing the retriever to distinguish between varying degrees of relevance. Utility-Aware Hard Negative Mining. Since computing the full distribution over the entire corpus is computationally intractable, we adopt the Noise Contrastive Estimation (NCE) paradigm to approximate the global distribution using a combination of gold contexts and informative negative samples. However, random neg- atives often provide trivial gradients, while standard semantic hard negatives may accidentally include documents that the generator finds useful. To address this, we implement a Utility-Gated Mining, where a documentํ โ is selected as a negative only if it satisfies two conditions: (1) it is semantically similar to the query (ranked in the top-ํby a base similarity functionํ sem ), and (2) it is assigned substantially lower utility than the gold evidenceํ + by the reward model, satisfyingํ ํ (ํ,ํ + )โ ํ ํ (ํ,ํ โ )> ํฟ. The marginํฟgoverns the trade-off between coverage and signal purity; a conservative margin ensures that the retriever focuses on resolving semantic distractors, documents that are proximally located in embedding space but offer negligible generative utility. This gated approach, combined with the robustness of the KL-divergence objective, pre- vents the propagation of noise from the reward model to the final embeddings. Aligning Dense Retrievers with LLM Utility via Distillation, , Raw Scores LLM Frozen Reward Model (Cross-Encoder) Dot-product Similarity ANN/FAISS/HNSW Ranked Results qda Gradients Flow Candidate batch Trained bi-encoder Corpus (Once) Queries Stage C: Final Inference Scalar Reward Online Encoding Semantic Similarity (base) Hard Negative Miner Utility Modulated Loss Stage A: Utility Distillation Offline / Not Used in Inference pair-wise margin loss Target Distribution Imitation Learning Stage B: Utility Aligned Embeddings Student Distribution Similarity Distribution Trainable Shared Encoder Figure 2: Overview of Utility-Aligned Embeddings (UAE). Utility is distilled offline into a reward model (Stage A), which defines a target utility distribution used to align a dense retriever via distribution matching (Stage B). At inference time, the trained bi-encoder supports standard ANN retrieval without any reward model or LLM inference (Stage C). CandidateHard Negative 10 4 10 3 10 2 0 10 2 10 3 Utility (SymLog Scale) Med: 3.98 Med: 0.44 (a) Raw Utility: High Variability 0200040006000 0 500 1000 1500 2000 Frequency Pairwise Accuracy: 96.5% (b) Raw Utility: Pairwise Delta CandidateHard Negative 1 2 3 4 5 6 7 Reward Score Med: 4.09 Med: 1.98 (c) Reward Score: Low Variability 1012345 0 20 40 60 80 Frequency Pairwise Accuracy: 98.3% (d) Reward Score: Pairwise Delta Figure 3: Raw generative utility scores (top) are heavy-tailed and highly variable, making direct regression unstable and motivating reward-based smoothing. Reward model learned (bottom) mitigates the problem. Scalability. To ensure training efficiency, we adopt an in-batch encoding protocol similar to DPR [14]. Rather than re-encoding the entire corpus, we dynamically encode only the sampled candidate setB=ํ + ,ํ โ 1 , . . .,ํ โ ํ for each query batch. This enables end-to- end optimization of the embedding space with gradients flowing through a shared encoder, while keeping training costs independent of the total corpus size. Crucially, the expensive reward model is utilized only during this offline phase; at inference time, the corpus is indexed for standard ANN search, fully decoupling utility alignment from runtime latency. 3 Experiments Datasets & Evaluation Protocol. We evaluate UAE on two dis- tinct RAG benchmarks: QASPER [4] (long-doc scientific QA) and NewsQA [29] (short-doc news extraction). We employ a hard- negative setting where the candidate pool (ํ=50) for each query is constructed via dense retrieval (BGE-Base [31]) and reward model utility. This populates the pool with the semantic distractors de- scribed in Section 2.2, rigorously testing the modelโs ability to prioritize true generative utility over high-similarity non-answers. Lexical Semantic Cross-Enc Ours (RM) 0.0 0.2 0.4 0.6 0.8 1.0 NDCG@5 Lexical Semantic Cross-Enc Ours (RM) 0.00 0.05 0.10 0.15 Recall@5 Lexical Semantic Cross-Enc Ours (RM) 0.0 0.2 0.4 0.6 0.8 Pairwise Acc@5 Figure 4: Alignment of various retrieval models with the LLMโs utility distribution. Reward Model (RM), while not a practical retriever, significantly outperforms lexical, seman- tic, and cross-encoder baselines across all ranking metrics, providing a higher-fidelity supervision signal for distillation. Generation Protocol. To measure downstream utility, we utilize Llama-3-8B-Instruct [7] as the fixed generator with greedy decod- ing (temperature=0) for reproducibility. Dataset-specific system prompts align the generatorโs output with the ground-truth for- mat: extractive phrases for NewsQA and evidence-based summaries for QASPER. Performance is quantified using Token F1 [21] and ROUGE-L [18] to assess both informational accuracy and structural fluency. Model Configuration. We initialize the utility reward model with microsoft/deberta-v3-base [8] and the retriever with BAAI/bge- base-en-v1.5 [31]. We also employ Low-Rank Adaptation (LoRA) [10] for parameter-efficient fine-tuning across both components. Reward Model Validation. We verify the distillation fidelity of the reward model (RM) in approximating the Llama-3 generatorโs utility (Eq. 1) on the NewsQA validation set. To assess cross-architecture transfer, we benchmark the DeBERTa-based RM against general- purpose retrievers (BM25, BGE, and BGE-Reranker). As shown in Figure 4, while standard models are frequently misled by semantic distractors (NDCG@1โค0.72), RM achieves 0.86 NDCG@1 and 0.70 pairwise accuracy. This confirms RM as a high-fidelity proxy that successfully distills the heavy LLMโs preferences into a compact supervisor. We set a utility threshold of 0.1 (calibrated to the top 10% of the score distribution) for relevance. While LLM utility may diverge from human labels, a known trait in LLM-as-a-judge frameworks, we prioritize generator-specific alignment to ensure the retriever surfaces contexts that the fixed LLM can effectively utilize, which is the core objective of utility-aligned RAG. , ,Rajinder Sandhu, Di Mu, Cheng Chang, Md Shahriar Tasjid, Himanshu Rai, Maksims Volkovs, and Ga Wu Table 1: Comparison of UAE against baselines on QASPER and NewsQA. We report Recall@1 (R@1), Recall@3(R@3), Expected Utility @1 (ExpUtil@1), Mean Average Precision (MAP), and Generation Metrics (Gen-F1, ROUGE-L). Efficiency is measured as latency (Lat.) in milliseconds. N/A denotes not available. QASPER (Long-doc)NewsQA (Short-doc) MethodR@1 R@3 ExpUtil@1 MAP Gen-F1 ROUGE-L Lat. R@1 R@3 ExpUtil@1 MAP Gen-F1 ROUGE-L Lat. Classic Retrieval BM2526.2649.700.679227.8120.520.75 49.6169.795.185952.1341.842.26 SPLADE V3 [16]38.4661.760.983632.1325.225.25 50.5873.675.358853.3444.745.24 InfoNCE Tuned6.2118.230.09328.615.96.09 22.4551.294.181932.6025.525.99 BGE-Base [31]36.8761.280.820138.0223.924.18 34.1557.374.737843.8438.639.08 ColBERTV2 [23]40.7564.870.979033.9826.726.846 55.1176.535.502458.3147.347.835 LLM Dense Retriever NV-Embed V2 [17]47.0471.750.974938.6727.827.752 47.3572.065.332551.2843.443.954 E5-Mistral [34]40.4665.240.868833.9326.026.139 45.1569.995.248947.6441.942.538 Multi-Stage (LLM) Rerank BGE-V2-M3 reranker [1] 43.0566.791.088242.9327.427.552 60.6777.625.733965.4946.547.034 RankGPT [26]48.8971.521.232846.1329.229.31663 49.6871.544.968046.7843.744.21610 GainRAG [12]46.3069.231.305745.2928.027.92293 41.2064.175.032950.9640.741.42104 UtilityQwen [34]40.9564.251.09637.6624.024.13200 42.7765.355.01347.9038.939.33194 SePer [3]59.8472.712.344550.1232.632.11696 60.6178.656.113166.1747.648.01711 UAE (Ours)48.1570.561.326949.4927.027.09 54.9077.305.818462.3145.646.19 Rerank Ablation UAE + BGE-reranker44.4570.711.145845.9428.528.543 65.6586.225.872772.0449.850.333 UAE + SePer61.9174.112.419453.3934.033.31704 66.4388.106.239473.1550.551.01709 UAE + LLM Reranker50.4875.701.263150.6529.829.71617 60.8787.325.741667.9249.850.31599 Figure 5: Zero-shot transfer performance. The model was trainedonly on QASPER and evaluated on unseen datasets. UAE improves or retains performance across all domains. Main Results. Table 1 presents a comprehensive evaluation of UAE against three distinct categories of baselines: classic lexical re- trievers (e.g., BM25, SPLADE), state-of-the-art dense retrievers (e.g., BGE, NV-Embed), and computation-heavy multi-stage rerankers (e.g., RankGPT, SePer). We analyze performance across three key dimensions: Retrieval Performance. UAE significantly outperforms all single- stage retrievers across both datasets. On the long-document QASPER benchmark, UAE achieves a MAP of 49.49, surpassing the strongest dense baseline (BGE-Base, 38.02) byโ11 points, the late-interaction model ColBERTv2 (33.98) byโ15 points and even outperforming the massive LLM-based embedder NV-Embed V2 (38.67). Notably, on NewsQA, UAEโs retrieval quality (Recall@1 54.90) exceeds that of the computationally expensive RankGPT (49.68), demonstrating that aligning embeddings with generative utility can yield reranker- level precision in a single retrieval step. This advantage extends to ExpUtil@1 (average utility of the top-1 context). On NewsQA, UAE (5.818) surpasses both BGE-Base (4.738) and even the computation- heavy RankGPT (4.968), confirming that UAE prioritizes contexts maximally conducive to generation rather than mere semantic rele- vance. Generation Quality. These gains in retrieval translate directly to downstream generation fidelity. On NewsQA, UAE achieves a Gen- F1 of 45.6, surpassing standard dense retrieval (BGE-Base: 38.6) and outperforming the multi-stage RankGPT pipeline (43.7). Similarly, on QASPER, UAE improves Gen-F1 scores over classic baselines (27.0 vs. 23.9 for BGE), confirming that the retriever selects contexts that are not merely semantically relevant, but factually sufficient for the LLM to generate correct answers. Efficiency & Compatibility. A critical barrier for deploying RAG in real-time production environments is latency; HCI research es- tablishes that system response times must remain under 100 ms to be perceived as instantaneous by users [20]. While multi-stage rerankers like RankGPT and SePer offer strong performance, they incur prohibitive latencies (>1600 ms), rendering them unsuitable for interactive applications. In contrast, UAE maintains a latency of โ9 ms, matching the speed of standard bi-encoders. Furthermore, UAE is not mutually exclusive with reranking; as shown in the ablation study, combining UAE with rerankers (e.g., UAE + SePer) yields new state-of-the-art results (MAP 53.39 on QASPER), proving that UAE serves as a superior "first-stage" retriever that enhances the entire pipeline when latency budgets permit. Zero-Shot Generalization and Robustness. A common failure mode of domain-specific fine-tuning is catastrophic forgetting, where the model loses its ability to generalize to new tasks. To evaluate this, our QASPER-trained model is tested against four out-of-domain datasets: NewsQA , SQuAD [21], HotpotQA [32], and SciFact [30]. Figure 5 summarizes the results. 4 Conclusion We present Utility-Aligned Embeddings (UAE), a framework that bridges the gap between the efficiency of dense retrieval and the Aligning Dense Retrievers with LLM Utility via Distillation, , high performance of utility-based models through distribution matching. Our results demonstrate that UAE significantly reduces semantic distractors and improves generation quality while oper- ating 180รfaster than LLM-based re-ranking methods. By main- taining standard ANN compatibility and serving as a high-quality foundation for multi-stage pipelines, UAE provides a practical and scalable solution for utility-driven RAG systems. References [1]Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2402.03216 [2] Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The Power of Noise: Redefining Retrieval for RAG Systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. [3]Lu Dai, Yijie Xu, Jinhui Ye, Hao Liu, and Hui Xiong. 2025. SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction. In The Thirteenth International Conference on Learning Representations. [4] Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gard- ner. 2021. A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers. In Proceedings of the 2021 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies. 4599โ4610. [5]Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. RAGAs: Automated Evaluation of Retrieval Augmented Generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 150โ158. [6]Aritra Ghosh, Himanshu Kumar, and P.S. Sastry. 2017. Robust Loss Functions un- der Label Noise for Deep Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. [7] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [8]Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In International Confer- ence on Learning Representations. [9]Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations (ICLR). [10]Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. [11] Oz Huly, Idan Pogrebinsky, David Carmel, Oren Kurland, and Yoelle Maarek. 2024. Old IR Methods Meet RAG. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2559โ2563. [12]Yi Jiang, Sendong Zhao, Jianbo Li, Haochun Wang, and Bing Qin. 2025. GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesis. In Proceedings of the 63rd Annual Meeting of the Association for Com- putational Linguistics. Association for Computational Linguistics, 10746โ10757. [13]Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. 2025. Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG. In The Thir- teenth International Conference on Learning Representations. [14]Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 6769โ6781. [15]Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Bridging the Preference Gap between Retrievers and LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 10438โ10451. [16]Carlos Lassance, Hervรฉ Dรฉjean, Thibault Formal, and Stรฉphane Clinchant. 2024. SPLADE-v3: New baselines for SPLADE. arXiv:2403.06789 [17] Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. In The Thirteenth International Conference on Learning Representations. [18]Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. 74โ81. [19]Hao Liu, Zhengren Wang, Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. 2025. HopRAG: Multi-Hop Reasoning for Logic-Aware Retrieval- Augmented Generation. In Findings of the Association for Computational Linguis- tics: ACL 2025. Association for Computational Linguistics, 1897โ1913. [20]Robert B. Miller. 1968. Response time in man-computer conversational transac- tions. In Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I. Association for Computing Machinery, 267โ277. [21]Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceed- ings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2383โ2392. [22]Alireza Salemi and Hamed Zamani. 2024. Evaluating Retrieval Quality in Retrieval-Augmented Generation. In Proceedings of the 47th International ACM SI- GIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2395โ2400. [23]Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 3715โ3734. [24]Chenze Shao, Yang Feng, Jinchao Zhang, Fandong Meng, and Jie Zhou. 2021. Sequence-level training for non-autoregressive neural machine translation. Com- putational Linguistics 47, 4 (2021), 891โ925. [25]Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, and Ninghao Liu. 2024. Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2056โ2066. [26] Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investi- gating Large Language Models as Re-Ranking Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 14918โ14937. [27]Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2025. Is Relevance Propagated from Retriever to Generator in RAG?. In Advances in Information Retrieval. Springer Nature Switzerland, 32โ48. [28] Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2026. Predicting Re- trieval Utility and Answer Quality in Retrieval-Augmented Generation. arXiv preprint arXiv:2601.14546 (2026). [29]Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A Machine Comprehension Dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP. 191โ200. [30] David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or Fiction: Verifying Scientific Claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 7534โ 7550. [31] Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources For General Chinese Embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 641โ649. [32] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP). [33]Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. Making Retrieval- Augmented Language Models Robust to Irrelevant Context. In The Twelfth Inter- national Conference on Learning Representations. [34]Hengran Zhang, Keping Bi, Jiafeng Guo, Jiaming Zhang, Shuaiqiang Wang, Dawei Yin, and Xueqi Cheng. 2025. Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation. In Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. Association for Computing Machinery, 22โ30.