Paper deep dive
Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
Zhan-Lun Chang, Dong-Jun Han, Seyyedali Hosseinalipour, Mung Chiang, Christopher G. Brinton
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 9/14/2026, 4:44:26 AM
Summary
The paper proposes a two-stage generator-in-the-loop alignment framework for multimodal Retrieval-Augmented Generation (RAG) to bridge the gap between semantic similarity and answer utility. Stage 1 uses a frozen Vision-Language Model (VLM) to generate a hypothetical textual rationale (HyDE) from an image-query pair to improve dense text retrieval. Stage 2 fine-tunes a LoRA-adapted cross-encoder reranker using answer-supervised preference pairs mined from the frozen VLM, where documents are labeled positive or negative based on whether they help the VLM produce the correct answer. This method outperforms baselines on VQA-X and A-OKVQA datasets.
Entities (13)
Relation Signals (10)
Generator-in-the-Loop Alignment → evaluatedon → VQA-X
confidence 98% · Experiments on VQA-X and A-OKVQA... show that our proposed framework consistently outperforms
Generator-in-the-Loop Alignment → evaluatedon → A-OKVQA
confidence 98% · Experiments on VQA-X and A-OKVQA... show that our proposed framework consistently outperforms
Generator-in-the-Loop Alignment → addresses → Semantic-Utility Gap
confidence 97% · Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment
Cross-encoder → finetunedby → Generator-in-the-Loop Alignment
confidence 95% · a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned using answer-supervised preference pairs
Generator-in-the-Loop Alignment → uses → HyDE
confidence 95% · a VLM generates a hypothetical text passage from the image-query pair... bridging the image-to-text modality gap
Generator-in-the-Loop Alignment → uses → LoRA
confidence 95% · a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned
Generator-in-the-Loop Alignment → usesmodel → Qwen3.5-2B
confidence 95% · Experiments on VQA-X and A-OKVQA with Qwen3.5-2B... show that our proposed framework consistently outperforms
Generator-in-the-Loop Alignment → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers and rerankers optimize for semantic similarity rather than answer utility, creating a preference gap: documents that appear relevant may not help the generator produce a correct answer. Motivated by this, we propose a two-stage generator-in-the-loop alignment framework that closes this gap without human document-level relevance annotations. Our framework consists of two stages: in Stage 1, a VLM generates a hypothetical text passage from the image-query pair, which is used as the retrieval query for dense text search, bridging the image-to-text modality gap. In Stage 2, a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned using answer-supervised preference pairs mined from the frozen VLM: given the dataset answer label, a candidate document is labeled positive if the VLM produces the correct answer when given that document as context, and negative otherwise. This generator-guided signal is compatible with multiple alignment loss functions, including contrastive (triplet) loss, pairwise direct preference optimization (DPO), and supervised fine-tuning (SFT), and supports periodic re-mining to refresh preference pairs as the reranker improves. Experiments on VQA-X and A-OKVQA with Qwen3.5-2B and Qwen3-VL-4B-Instruct show that our proposed framework consistently outperforms rank-order, random, and REPLUG-style likelihood baselines under various alignment losses and pool size settings, suggesting that answer-level generator feedback is an effective supervision signal for preference alignment.
Tags
Links
- Source: https://arxiv.org/abs/2609.08188v1
- Canonical: https://arxiv.org/abs/2609.08188v1
Trouble viewing inline? Open PDF directly →
Full Text
90,702 characters extracted from source content.
Expand or collapse full text
Bridging the Semantic-Utility Gap in Multimodal RAG via Generator-in-the-Loop Alignment PubID: pubid: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Zhan-Lun Chang Dong-Jun Han Seyyedali Hosseinalipour Mung Chiang Christopher G. Brinton †thanks: Z.-L. Chang, M. Chiang, and C. G. Brinton are with the Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA.†thanks: D.-J. Han is with the Department of Computer Science and Engineering, Yonsei University, Seoul 03722, Republic of Korea.†thanks: S. Hosseinalipour is with the Department of Electrical Engineering, University at Buffalo–SUNY, Buffalo, NY 14260, USA. Abstract Vision-language models (VLMs) augmented with retrieval-augmented generation (RAG) benefit from access to external evidence. However, standard retrievers and rerankers optimize for semantic similarity rather than answer utility, creating a preference gap: documents that appear relevant may not help the generator produce a correct answer. Motivated by this, we propose a two-stage generator-in-the-loop alignment framework that closes this gap without human document-level relevance annotations. Our framework consists of two stages: in Stage 1, a VLM generates a hypothetical text passage from the image-query pair, which is used as the retrieval query for dense text search, bridging the image-to-text modality gap. In Stage 2, a cross-encoder reranker adapted with low-rank adaptation (LoRA) is fine-tuned using answer-supervised preference pairs mined from the frozen VLM: given the dataset answer label, a candidate document is labeled positive if the VLM produces the correct answer when given that document as context, and negative otherwise. This generator-guided signal is compatible with multiple alignment loss functions, including contrastive (triplet) loss, pairwise direct preference optimization (DPO), and supervised fine-tuning (SFT), and supports periodic re-mining to refresh preference pairs as the reranker improves. Experiments on VQA-X and A-OKVQA with Qwen3.5-2B and Qwen3-VL-4B-Instruct show that our proposed framework consistently outperforms rank-order, random, and REPLUG-style likelihood baselines under various alignment losses and pool size settings, suggesting that answer-level generator feedback is an effective supervision signal for preference alignment. Index Terms: Cross-modal retrieval, Information retrieval, Retrieval-augmented generation, Visual question answering. †impactstatement: RAG-enabled multimodal systems are becoming a key component of knowledge-intensive visual reasoning applications, where retrieving misleading evidence can cause models to produce confident yet incorrect answers. This work addresses a fundamental limitation of existing multimodal RAG pipelines by shifting retrieval optimization from semantic similarity toward downstream answer utility. Rather than relying on costly human document-level relevance annotations, the proposed generator-in-the-loop alignment framework leverages a frozen VLM to automatically identify evidence that improves answer correctness and uses this signal to train a lightweight LoRA-adapted reranker. The framework is compatible with multiple alignment objectives and consistently improves performance across different datasets and model architectures, demonstrating that answer-level supervision is a more effective retrieval objective than semantic similarity alone. By improving the reliability and faithfulness of multimodal RAG systems, this approach has the potential to benefit high-stakes applications such as medical decision support, scientific knowledge retrieval, education, document understanding, and other VLM-empowered systems that require trustworthy multimodal reasoning while reducing the dependence on expensive manual supervision. I Introduction Large language models (LLMs) and multimodal vision-language models (VLMs) have fundamentally transformed artificial intelligence, demonstrating unprecedented capabilities in natural language understanding, visual reasoning, and content generation [1, 2]. However, despite their scale, these models remain plagued by intrinsic limitations: they are prone to hallucinations, restricted by static parametric knowledge cutoffs, and lack transparency in their reasoning processes [3]. To mitigate these issues, retrieval-augmented generation (RAG) has emerged as the de facto standard by replacing reliance on static parametric knowledge alone with the retrieval of external evidence at inference time [4, 5]. By grounding generation in retrieved evidence, RAG systems significantly enhance factual accuracy and interpretability, particularly in knowledge-intensive domains where hallucination remains a persistent reliability concern [6]. The design of RAG systems has evolved from simple retrieve-then-generate pipelines to architectures that incorporate query rewriting, iterative retrieval, retrieval verification, and multi-step reasoning [7, 8, 9]. While these techniques improve candidate retrieval and evidence refinement, retrievers and rerankers continue to optimize semantic similarity rather than the generator’s downstream objective. Consequently, a document that appears highly relevant to the query may still fail to provide the evidence required for the generator to produce the correct answer. More critically, semantically relevant documents can contain incorrect or misleading information that steers the generator toward an erroneous answer. Such documents may therefore increase, rather than reduce, the likelihood of hallucination [10]. A reranker must thus distinguish documents that help the generator produce the correct answer from those that are semantically relevant but unhelpful or misleading. Training a reranker to make this distinction typically requires document-level relevance annotations; however, manually determining whether each candidate document improves the generator’s answer correctness requires costly, task-specific annotation at scale. This motivates the central question of this work: Q: How can we train a reranker to identify evidence that improves the generator’s answer correctness using only ground-truth answer labels with no human document-level relevance annotations? Answering this question becomes more challenging in multimodal RAG (M-RAG), where retrieval must account for information contained in both visual and textual inputs. Specifically, visual embeddings optimized for general image-text alignment may fail to preserve fine-grained visual information, such as object relationships or structural details in scientific diagrams, that is necessary to retrieve evidence for downstream reasoning. This mismatch between the multimodal query and the retrieved textual evidence compounds the misalignment between retrieval relevance and answer correctness. Prior approaches, such as ATLAS [11] and REPLUG [12], attempt to align retrieval relevance with the generator’s downstream objective through likelihood- or perplexity-based training signals. However, these signals measure how a retrieved document changes the probability of a target output rather than whether the generator actually produces the correct answer. Therefore, M-RAG requires addressing two coupled problems: (i) translating multimodal inputs into effective retrieval queries and (i) selecting retrieved evidence based on its contribution to downstream answer correctness. To jointly address these problems, we propose a two-stage generator-in-the-loop alignment framework. In Stage 1, a frozen VLM converts the image-question pair into a hypothetical textual rationale using hypothetical document embeddings (HyDE). This rationale is concatenated with the original question and encoded by a Sentence Transformer [13] to retrieve a broad pool of candidate documents using Facebook AI similarity search (FAISS). Stage 2 addresses what we term the Preference Gap: conventional rerankers prioritize documents based on semantic relevance, whereas the generator requires documents that help it produce the correct answer. A semantically relevant document may therefore lack the evidence needed to answer the question [14] or may even act as a distractor that degrades performance below the closed-book baseline [15, 16]. To directly align reranking with answer correctness, we provide each candidate document to the frozen VLM as context and compare the generated answer with the ground-truth answer. A document is labeled positive if the model produces the correct answer and negative otherwise, yielding preference pairs without human document-level relevance annotations. We use these pairs to fine-tune a cross-encoder reranker through low-rank adaptation (LoRA) and evaluate the same generator-derived supervision under contrastive learning, direct preference optimization (DPO), and supervised fine-tuning (SFT). By comparing the same supervision signal across three training objectives, we evaluate whether reranking based on the generator’s answer correctness consistently improves evidence selection independent of the specific training objective. The main contributions of this paper can be summarized as follows: 1. We propose a two-stage multimodal RAG framework that separates multimodal query translation from generator-aligned reranking. In Stage 1, a frozen VLM generates a hypothetical textual rationale from the image-question pair using HyDE. The rationale is concatenated with the question and encoded by a Sentence Transformer to retrieve a broad pool of candidate documents using FAISS. 2. We introduce an answer-supervised preference mining procedure that trains the Stage 2 reranker without human document-level relevance annotations. Specifically, for each candidate document, the frozen VLM generates an answer using the document as context; documents that lead to correct and incorrect answers are used to construct positive-negative preference pairs for reranker training. 3. We use the generator-derived preference pairs to fine-tune a LoRA-adapted cross-encoder reranker under three training objectives: contrastive triplet loss, DPO, and SFT. Experiments on VQA-X and A-OKVQA with Qwen3.5-2B and Qwen3-VL-4B-Instruct show that generator-guided preference mining consistently outperforms rank-order, random, and REPLUG-style likelihood preference mining across the considered training objectives and various candidate pool sizes. 4. We develop an iterative preference-alignment mechanism with periodic preference re-mining that dynamically reconstructs generator-derived preference pairs as the reranker evolves, thereby keeping the training signal aligned with the reranker’s current ranking behavior rather than relying on a fixed set of initially mined pairs. Experiments across various generators and datasets demonstrate consistent improvements over fixed-pair training, with gains of up to 2.58 percentage points. I Related Work We organize the related literature into the following categories: (i) evolution of RAG; (i) multimodal retrieval and the semantic gap; (i) retriever-generator alignment. I-A Evolution of RAG RAG has emerged as a standard method for mitigating the limitations of LLMs/VLMs, specifically their susceptibility to hallucinations and outdated parametric knowledge [4, 17]. Following recent surveys [7, 18], we summarize the progression of RAG through three phases: Naive RAG methods, Advanced RAG methods, and Modular RAG methods. The initial phase, Naive RAG methods, followed a linear “retrieve-then-generate” protocol. These systems utilized sparse (BM25) or dense passage retrieval (DPR) retrievers to fetch context, which was then directly concatenated with the user query [19]. While effective for simple factual retrieval, Naive RAG methods struggle with low precision and noise induction, where irrelevant retrieved context degrades the generator’s performance [20]. To mitigate these shortcomings, approaches such as Chain-of-Note [21] were introduced, teaching models to explicitly generate notes assessing the relevance of retrieved documents before answering, thereby improving robustness against noisy inputs. To further improve performance, Advanced RAG methods introduced sophisticated pre-retrieval and post-retrieval optimizations. Specifically, pre-retrieval techniques, such as query rewriting [9], multi-aspect query retrieval [22], and HyDE [23], aim to align the user’s intent with the document space. HyDE in particular prompts a language model to generate a hypothetical answer-oriented document from the query and uses that generated text, rather than the raw query, as the retrieval input; because this text resembles the corpus documents more closely, its embedding can have higher similarity to answer-bearing passages and improve recall without retrieval-specific training. Also, post-retrieval strategies focused on refining the context window. Notably, reranking models [24] and cross-encoder filtering [25] were developed to identify the most relevant retrieved passages, thereby reducing irrelevant context before generation. Despite these improvements, recent studies on the “lost in the middle” phenomenon [15] suggest that simply stuffing context windows is insufficient, as models struggle to access information buried in the middle of long retrieval sequences. Most recently, the field has shifted toward modular and active RAG [26]. In this paradigm, the pipeline is decomposed into independent, swappable modules orchestrated by programming frameworks such as DSPy [8], which optimizes the flow of information between modules. Beyond static retrieval, recent frameworks further employ active strategies. For instance, FLARE [27] drafts upcoming generated text and triggers retrieval when low-confidence tokens suggest that additional evidence is needed, while CRAG (corrective RAG) [28] introduces a lightweight retrieval evaluator that triggers a web search fallback when the retrieved documents are deemed incorrect. These systems improve when and how retrieval is invoked, but evidence selection often remains based on semantic similarity, meaning how closely a retrieved document matches the query in topic, wording, or embedding space. However, a semantically similar document may still be unhelpful or even misleading for answering the question. Therefore, evidence should instead be selected according to its downstream answer utility, namely whether providing that document as context actually helps the generator produce the correct answer. I-B Multimodal Retrieval and the Semantic Gap The success of RAG in the textual domain has extended to large multimodal models, giving rise to M-RAG [29, 30], systems that retrieve external textual or visual evidence to support reasoning over image–text queries. Unlike text-only RAG, M-RAG must align heterogeneous query inputs, such as images and natural-language questions, with evidence stored in an external knowledge base. Standard retrieval architectures commonly rely on dual-encoder backbones such as CLIP [31] to map visual and textual content into a shared semantic space. Meanwhile, foundational models such as Flamingo [32] demonstrated the effectiveness of conditioning generation on interleaved image–text inputs. However, neither multimodal conditioning nor coarse image–text alignment alone guarantees that the retrieved evidence contains the fine-grained information needed for downstream reasoning. This leads to a critical semantic gap: visual embeddings optimized for general image–caption matching may overlook subtle objects, relationships, and structural details that are essential for retrieving answer-supporting evidence [26]. For example, Qwen-VL [33] attempts to preserve finer visual information through resolution-aware visual encoders, but still relies on conventional retrieval mechanisms. Recent efforts have therefore sought to narrow this gap by integrating stronger generative or fine-grained visual representations into retrieval. RA-CM3 [34] was among the first to jointly retrieve and generate mixed-modal content within a unified architecture. More recently, ColPali [35] avoids the lossy conversion of document pages into text by using a VLM to produce multi-vector embeddings for page patches, enabling late-interaction retrieval that better preserves visual layout and textual semantics. Our work takes a complementary approach to these methods by decoupling multimodal query alignment from evidence-utility estimation. Rather than requiring the retriever itself to learn a shared representation that preserves all task-relevant visual and textual information, we use a frozen VLM to translate the image–question pair into a hypothetical textual rationale. This converts the multimodal query into the same modality as the document corpus, enabling effective text-based dense retrieval without training a dedicated multimodal retriever. However, resolving the modality gap alone does not ensure that the retrieved documents will help the generator answer correctly. We therefore introduce a second stage in which a separately trained cross-encoder reranker is aligned with the frozen generator’s downstream behavior, prioritizing documents according to their answer utility rather than semantic similarity. I-C Retriever-Generator Alignment The most significant challenge in RAG is the alignment gap between retrieval or reranking objectives and the generator’s reasoning objective [7]. In particular, retrievers are typically optimized for query-document semantic similarity, an objective decoupled from whether the retrieved context actually helps the generator produce the correct answer. This mismatch frequently leads to failure in complex reasoning tasks [36], despite evidence that LLMs can effectively utilize in-context knowledge when the right documents are retrieved [37]. Several methods have sought to reduce the alignment gap by incorporating generator feedback into retrieval training. For instance, Atlas [11] jointly trains the retriever and generator using generation perplexity, while REPLUG [12] treats the language model as a black box and trains the retriever using each document’s effect on the probability of the ground-truth continuation. These methods make retrieval more generator-aware, but their likelihood-based signals remain indirect proxies for downstream correctness and were developed primarily for text-only generation settings. BGM [38] addresses a related preference gap, in which the retriever’s highest-ranked documents do not necessarily produce the best generation outcomes. Rather than updating the frozen retriever and generator, BGM inserts a trainable sequence-to-sequence bridge between them. The bridge is trained using supervised learning on synthesized silver passages and reinforcement learning to rerank and reorganize the retrieved context. Although this approach moves closer to task-level optimization, it introduces an additional large trainable component into the RAG pipeline. Our framework instead aligns reranking directly with the deployed generator’s answer behavior: for each candidate document, the frozen VLM is prompted to produce the constrained dataset answer output, either yes/no or one of the provided options, and the document is labeled useful when that output matches the dataset-provided ground-truth. The resulting signal directly measures whether a document helps the generator answer correctly, rather than whether it increases the likelihood of the ground-truth answer when preceding answer tokens are supplied. These generator-specific document-level preferences are then distilled into a lightweight cross-encoder reranker. Our method therefore complements conventional semantic rerankers [24, 25], likelihood- and reward-based retrieval alignment methods [11, 12, 38], and multimodal retrieval approaches that primarily address cross-modal representation alignment [34, faysse2024colpali]. Our proposed supervision mechanism is related to LLM-as-judge and synthetic-data methods. LLM-as-judge approaches use language models to assign general quality scores or preferences to open-ended outputs [39], while self-generated instruction methods synthesize training examples to reduce dependence on manual annotation [40]. In our setting, however, the VLM neither produces a general evaluation nor generates new instructions. Instead, it reveals whether a particular retrieved document causes the same frozen generator used at deployment to produce the correct answer. The resulting labels differ from general-purpose LLM judgments or synthetic training data: they provide generator-specific, document-level utility signals tailored explicitly to reranker alignment. Fig. 1: A schematic of our proposed two-stage multimodal RAG system. Stage 1 (Retrieval): given a multimodal input (v,q)(v,q), the frozen VLM G generates a short HyDE rationale h; the composite query qtext=q⊕hq_text=q h is encoded by a text dense encoder and used to retrieve the top-k corpus documents from a FAISS index. Stage 2 (Reranking): a LoRA-adapted cross-encoder scores each of the k candidate documents and is fine-tuned on preference pairs (d+,d−)(d^+,d^-) mined by querying the frozen VLM for answer correctness. Preference pairs are periodically re-mined using the live LoRA model, and the reranker can be trained under contrastive (triplet), SFT, or DPO losses. Inference: the trained reranker selects the top-1 document, which is passed together with the image and question to the frozen VLM to produce the final answer. I Proposed Architecture In this section, we present the architecture of our two-stage M-RAG system and the alignment framework used to close the gap between candidate ranking and answer utility. The key idea is to separate broad multimodal recall from generator-specific utility alignment: Stage 1 uses the frozen deployment generator as an image-to-text bridge for dense retrieval, while Stage 2 uses the same frozen generator to mine answer-level preferences for reranker training. Table I summarizes the mathematical notations used throughout. TABLE I: Mathematical Notations Symbol Meaning v Input image q Natural-language question string agolda_gold Ground-truth answer G Frozen VLM generator h=(v,q)h=G(v,q) HyDE hypothetical rationale text qtext=q⊕hq_text=q h Concatenated query string ϕ(⋅)φ(·) Qwen3-Embedding-8B text encoder q=ϕ(qtext)e_q=φ(q_text) ℓ2 _2-normalized query embedding ℐI FAISS IndexFlatIP index qC_q Stage 1 top-K candidate pool K Stage 1 retrieval depth (default 100) K′K Stage 2 preference-mining pool cap, with K′≤K ≤ K EθE_θ Cross-encoder (θ=θbase∪θLoRAθ= _base∪ _LoRA) ErefE_ref Frozen reference reranker (no LoRA) sθ(qtext,d)s_θ(q_text,d) Scalar relevance logit for pair (qtext,d)(q_text,d) D Mined preference dataset d A text document (e.g., textual justification or rationale) d+d^+ Positive (preferred) document d−d^- Negative (rejected) document +,−S_+,\,S_- Correct / incorrect candidate sets BmB_m VLM mini-batch size during mining N Re-mining frequency (epochs between re-mines) η AdamW learning rate β DPO inverse-temperature m Triplet margin Ψ Persistent VLM prediction cache I-A Problem Formulation We formulate reranking as an answer-utility alignment problem rather than a standard relevance-ranking problem. Formally, we presume that the corpus D consists of text documents d∈d (e.g., textual justifications for VQA-X or rationales for A-OKVQA) that serve as retrieval targets: each document provides external textual evidence that the generator G can use as context to help answer a question. Each query consists of an image v and a natural-language question q; the dataset also provides a ground-truth answer agolda_gold that is available only during preference mining. The textual query representation used by the retriever and reranker is denoted qtextq_text and is formed by concatenating the question q with the hypothetical rationale h generated by G. Retrieving with qtextq_text yields the candidate pool qC_q consisting of K documents. Given a frozen generator G, a query (v,q)(v,q), and a candidate document d∈qd _q, we define the answer utility of d as the binary indicator u(d)=[(v,q,d)=agold],u(d)=1 [G(v,\,q,\,d)=a_gold ], (1) which equals 1 if and only if G produces the correct answer when given d as context. We further partition the candidate pool qC_q accordingly into a set of useful documents +=d∈q:u(d)=1S_+=\d _q:u(d)=1\ and a set of harmful documents −=d∈q:u(d)=0S_-=\d _q:u(d)=0\. Over this pool, a pre-trained cross-encoder EθE_θ scores each pair (qtext,d)(q_text,d) with a relevance logit sθ(qtext,d)s_θ(q_text,d) based on semantic similarity, selecting d∗=argmaxd∈qsθ(qtext,d)d^*= _d _q\,s_θ(q_text,d). However, semantic similarity does not imply answer utility: a document semantically close to the query may fail to support correct reasoning, while a document that appears less similar may provide exactly the missing evidence. This gives rise to the preference gap, which can be interpreted as follows: Eθ is misaligned when d∗∉+.E_θ is misaligned when d^* _+. (2) Research Problem. Given a candidate pool qC_q and a frozen generator G, we aim to learn reranker parameters θ such that sθ(qtext,d+)>sθ(qtext,d−)∀d+∈+,d−∈−,s_θ(q_text,d^+)>s_θ(q_text,d^-) ∀\,d^+ _+,\;d^- _-, (3) without any human-annotated document relevance labels. In this paradigm, the supervision signal is the answer utility u(d)u(d), which is computed by querying the frozen G and comparing its output to the dataset-provided answer label. I-B Architecture Overview Our proposed framework is built on a two-stage pipeline, illustrated in Fig. 1. This stage-wise decomposition is intentional: Stage 1 is optimized for recall across the text corpus, while Stage 2 learns which recalled evidence actually improves the frozen generator’s answer behavior. More specifically, Stage 1 converts the multimodal input (v,q)(v,q) into a short hypothetical text rationale h via HyDE, which is concatenated with the question to form the composite text query qtext=q⊕hq_text=q h. This query is encoded by a text encoder (e.g., Qwen3-Embedding-8B) ϕφ and used to search a FAISS index ℐI, producing a candidate pool qC_q of up to K documents. Subsequently, Stage 2 takes qC_q and applies a LoRA-adapted cross-encoder EθE_θ to rerank the candidates by assigning a scalar relevance logit sθ(qtext,d)s_θ(q_text,d) to each text pair (qtext,d)(q_text,d). The reranker is fine-tuned on preference pairs mined by querying the frozen G for answer correctness; pairs are periodically re-mined using the live model to keep the training signal fresh. At inference, the top-scored document d∗=argmaxd∈qsθ(qtext,d)d^*= _d _qs_θ(q_text,d) is passed together with (v,q)(v,q) to the frozen G for final answer generation. One of our key design choices is to separate multimodal understanding from reranker training. Specifically, the frozen generator G is the only component that processes the image v: in Stage 1, it translates the image–question pair into a textual rationale that enables text-based retrieval, while in Stage 2, it evaluates whether each retrieved document helps produce the correct answer. In contrast, the cross-encoder EθE_θ operates exclusively on text, allowing it to leverage strong pre-trained language representations without requiring multimodal adaptation. This separation keeps the architecture lightweight: G remains frozen and provides both modality conversion and supervision, while only the text-based reranker is trained. I-C Datasets and Document Corpora Fig. 2 illustrates what a real-world multimodal query looks like in the two datasets considered in this work: each query consists of an image v and a natural-language question q, while the knowledge base is populated with dataset-specific textual documents (justifications for VQA-X, rationales for A-OKVQA) that serve as the retrieval targets for Stage 1. The following paragraphs briefly describe how each dataset defines the multimodal query, constructs the external knowledge base, and uses the retrieved evidence for answer generation. • A-OKVQA (Augmented Outside Knowledge VQA): This dataset focuses on visual question answering tasks that require commonsense or world knowledge beyond what is directly visible in the image. As shown in Fig. 2, the pipeline accepts the query image (e.g., a cat watching TV) and the associated question text (e.g., Why is the cat looking at the screen?). The external database is populated with the dataset’s Rationales, which provide the necessary background information (e.g., The cat is intrigued by the image on the screen). The generator uses these retrieved rationales to select the correct option from multiple-choice answers or provide a direct response. • VQA-X (Visual Question Answering with eXplanations): This dataset extends standard VQA by providing textual justifications for answers. As shown in Fig. 2, the query is formed by the image (e.g., a giraffe) and the question (e.g., Does the giraffe have a goofy look on his face?). The Textual Justification serves as the retrievable document. By retrieving these ground-truth reasons (e.g., …his jaw is all out of whack…), the pipeline aligns the generator’s output with the expected reasoning process. To prevent test-set evidence leakage, the retrieval corpus is constructed exclusively from the training split and fixed before Stage 2 training. For A-OKVQA, the FAISS index contains the unique, non-empty rationales from the training set, whereas for VQA-X, it contains the unique, non-empty explanations associated with yes/no training examples. Stage 1 retrieves candidates from this fixed corpus and stores them in split-specific candidate files. Stage 2 then combines these files with the corresponding dataset split to recover the images, answer choices, and ground-truth labels required for preference mining and evaluation. In this way, the reranker is trained on candidates drawn only from the training-derived corpus, while its supervision is obtained from answer correctness rather than human document-level relevance annotations. Fig. 2: Dataset Integration into the M-RAG Pipeline. For each dataset (A-OKVQA and VQA-X), we map the original inputs to a unified query format comprising the image and question text. The external knowledge base is populated with the dataset-specific ground-truth explanations (rationales and justifications). The retrieval target is defined as the textual evidence required to bridge the reasoning gap between the visual query and the final answer. I-D (Stage 1) HyDE-Based Modality Bridge and Dense Recall A fundamental challenge in multimodal RAG is the modality gap between the multimodal query and the text-only retrieval space: the text encoder ϕφ accepts only text, while the query is inherently multimodal, comprising an image v and a natural-language question q. Using q alone therefore discards potentially essential visual information from v, while training a dedicated cross-modal retriever would require substantial amounts of aligned image–text data and additional model adaptation. To bridge this gap, we use HyDE [23] as a multimodal-to-text conversion mechanism rather than merely as a generic query-expansion technique. Specifically, rather than searching with the raw question q or training a separate multimodal retriever, the frozen generator G is first prompted with the full multimodal input (v,q)(v,q) to produce a hypothetical rationale h=(v,q),h=G(v,q), (4) where h is a short, image-grounded passage that captures the visual information relevant to the question and resembles the type of evidence contained in the retrieval corpus. The rationale is then concatenated with the original question to form the composite textual query qtext=q⊕hq_text=q h. Since qtextq_text is entirely textual, it can be processed directly by ϕφ without any multimodal adaptation. HyDE thus serves as an image-to-text modality bridge, projecting the task-relevant information in v into the textual domain and enabling text-only dense retrieval for an inherently multimodal query. The resulting query embedding is computed as q=ϕ(qtext)∈ℝne_q=φ(q_text) ^n, where ϕφ prepends an instruction prefix and applies ℓ2 _2 normalization. The corpus documents are encoded offline using the same encoder, and their embeddings are stored in a FAISS IndexFlatIP index ℐI that supports exact inner-product search. Because each rationale or justification is already a short, self-contained unit, no document chunking is applied; instead, each document is indexed in its entirety. At query time, the top-K candidates are retrieved as q=top-K(q,ℐ).C_q=top-K\! (e_q,\,I ). (5) To avoid repeated VLM inference, the hypothetical rationale h is cached per question identifier with atomic disk writes. The resulting candidate set is represented as q=(d1,r1),…,(dK,rK)C_q=\(d_1,r_1),…,(d_K,r_K)\, where did_i denotes the i-th retrieved document and rir_i its retrieval rank. These candidate sets are then stored and passed to Stage 2 for generator-aligned reranking. Fig. 3: Iterative preference alignment with periodic re-mining. Training begins with the base reranker weights wbasew_base. Preference pairs are first mined under wbasew_base, producing dataset baseD_base. The reranker is then LoRA fine-tuned on baseD_base, advancing the weights from wbasew_base to w1w_1 and then to w2w_2. After a fixed number of epochs, the preference pairs are re-mined under the current model w2w_2, yielding a refreshed dataset 2D_2 that reflects the model’s updated ranking. This cycle can keep the training signal aligned with the reranker’s current behavior, although the benefit of additional refreshes may saturate once the model has stabilized. Fig. 4: Generator-guided preference pair mining (proposed). Given a multimodal query (v,q)(v,q), Stage 1 retrieves 100 candidate documents. The trainable reranker with current weights w re-scores those candidates, and Stage 2 retains the top-K′=10K =10 subset for mining. The frozen VLM then evaluates each retained document in descending score order: if the generated answer is correct, the document is taken as d+d^+ and subsequent correct answers are skipped; if the answer is incorrect, the document is taken as d−d^-. Evaluation exits immediately once both d+d^+ and d−d^- are found, avoiding unnecessary VLM calls. The resulting training triple is (query text == question ⊕ HyDE rationale, d+d^+, d−d^-). I-E (Stage 2) Generator-Guided Cross-Encoder Alignment Stage 2 is responsible for reranking the candidate pool qC_q retrieved by Stage 1. Its key objective is to replace indirect supervision signals, such as semantic relevance or ground-truth-token likelihood, with direct feedback on whether the generator’s constrained answer output matches the ground-truth. Specifically, the reranker is trained to prefer documents that enable the frozen generator G to produce the dataset-provided ground-truth answer agolda_gold. To learn this preference while keeping training lightweight, we adapt a GTE-ModernBERT-base cross-encoder [41] with LoRA (r=16r=16, α=32α=32, dropout =0.1=0.1), where only the low-rank adapter weights θLoRA _LoRA are updated during training while the pre-trained encoder θbase _base is kept frozen. For each query–document pair, the cross-encoder jointly processes the composite textual query qtextq_text and the candidate document d, truncated to a maximum of 512 tokens, and produces a scalar score sθ(qtext,d)=Eθ([qtext;d])∈ℝ,s_θ(q_text,d)=E_θ\! ([q_text;\,d] ) , (6) This score represents the reranker’s learned estimate of how useful the document is for supporting the generator’s answer. To control VLM inference cost during preference mining, Stage 2 evaluates only the top-K′K prefix of the Stage 1 pool, where K′≤K ≤ K. Training supervision is derived automatically through preference mining: for each training query, the current reranker EθE_θ first scores the Stage 1 candidates and retains only the top-K′K prefix for mining. A preference pair (d+,d−)(d^+,d^-) is then constructed, where d+d^+ denotes a preferred document and d−d^- denotes a rejected one. All four mining strategies follow this same pairwise formulation but differ in how they assign the positive and negative labels. The proposed generator-guided strategy feeds each retained candidate document to the frozen VLM G together with the image v and question q, and checks whether G produces the correct answer under the dataset answer label. The first retained candidate (in reranker-score order) that leads to a correct answer becomes d+d^+, and the first retained candidate that leads to an incorrect answer becomes d−d^-. If the retained pool does not contain both outcomes, the query is skipped and contributes no training pair for that epoch. The controlled baselines construct the same pairwise labels from different signals. The REPLUG-style likelihood baseline [12] does not rely on constrained answer generation; instead, it scores each retained candidate according to the likelihood of agolda_gold when preceding ground-truth answer tokens are supplied and selects the highest- and lowest-scoring documents as d+d^+ and d−d^-. The first baseline requires no VLM: it uses only the current reranker EθE_θ, assigning its highest-ranked candidate to d+d^+ and its lowest-ranked candidate to d−d^-. The random baseline ignores both reranker scores and VLM feedback: d+d^+ and d−d^- are drawn uniformly at random from the retained pool. The mined pairs are used to fine-tune EθE_θ under one of three alignment loss functions: triplet margin loss, SFT, or pairwise DPO [42]. To keep preference data fresh as the model improves, pairs are periodically re-mined against the current EθE_θ checkpoint rather than a fixed initial ranking. Section IV provides pseudocode and walk-throughs for the training loop and mining strategies. IV Reranker Alignment and Preference Mining Section I introduced the two-stage M-RAG architecture, in which Stage 1 translates the multimodal query into text and retrieves a candidate document pool, while Stage 2 reranks these candidates according to their utility to the frozen generator. Building on this architecture, this section describes how the reranker is trained and evaluated. Specifically, we present the iterative LoRA-based alignment procedure with periodic re-mining, define the three alignment loss functions, and describe both the proposed generator-guided preference-mining strategy and the First, Random, and REPLUG-style likelihood baselines. Figs. 3 and 4 illustrate the two central components of our proposed alignment procedure. Specifically, Fig. 3 depicts the iterative mine–train–re-mine loop, whereas Fig. 4 shows how generator feedback is used to construct a positive–negative preference pair for an individual query. In essence, our proposed procedure is enabled by three key mechanisms: generator-guided preference mining uses downstream answer correctness as the supervision signal; batching, early exit, and persistent caching reduce the cost of repeated VLM evaluation; and periodic re-mining refreshes the preference pairs as the reranker evolves. Fig. 3 shows the iterative alignment loop. At the start, the reranker EθE_θ is used to score the Stage 1 candidates, and a preference dataset 1D_1 is mined from those scores. The reranker then undergoes LoRA fine-tuning using 1D_1 for N consecutive epochs. After N epochs, the now-improved reranker is used to re-score the same candidates and produce a refreshed dataset 2D_2. Fine-tuning continues from 2D_2 for another N epochs, and this mine-then-train cycle repeats until the total number of training epochs E is reached. The key intuition is that as the reranker changes, re-mining can refresh the preference pairs so that training continues to reflect the model’s current ranking behavior. Further, Fig. 4 shows how a single preference pair (d+,d−)(d^+,d^-) is constructed for one query. The reranker scores all retrieved candidate documents and evaluates them in descending score order using the frozen VLM G. For each candidate, G generates an answer conditioned on the image, the question, and that candidate as context. The first candidate for which G produces the correct answer is taken as d+d^+, and the first candidate for which G produces a wrong answer is taken as d−d^-. Once both documents have been identified, the evaluation terminates through an early-exit mechanism, thereby avoiding unnecessary VLM calls for the remaining candidates. If the entire candidate pool is examined without finding both a positive and a negative document (e.g., when all candidates produce the same correctness outcome), the query is skipped and contributes no preference pair for that mining round. Subsection IV-A presents the iterative loop. Section IV-B defines the three alignment loss functions. Subsection IV-C details the preference mining subroutine for all four strategies. IV-A Iterative Reranker Alignment Algorithm 1 Iterative LoRA Fine-Tuning with Periodic Re-mining 1: Queries Q, base cross-encoder EθbaseE_ _base, frozen VLM G 2: Pool cap K′K , re-mine frequency N, epochs E, learning rate η, method 3: randomly initialize θLoRA _LoRA; freeze θbase _base 4: ←PreferenceMine(,Eθbase,,K′)D (Q,\,E_ _base,\,G,\,K ) 5: Eθ←Eθbase+θLoRAE_θ← E_ _base+ _LoRA 6: for e=1e=1 to E do 7: if N>0N>0 and (e−1)modN=0(e-1) N=0 and e>1e>1 then 8: ←PreferenceMine(,Eθ,,K′)D (Q,\,E_θ,\,G,\,K ) 9: for each (qtext,d+,d−)∈(q_text,d^+,d^-) do 10: s+←sθ(qtext,d+)s^+← s_θ(q_text,\,d^+) 11: s−←sθ(qtext,d−)s^-← s_θ(q_text,\,d^-) 12: ℒ←ComputeLoss(s+,s−,method)L (s^+,\,s^-,\,method) 13: update θLoRA _LoRA via AdamWAdamW on ∇θLoRAℒ _ _LoRAL with learning rate η 14: return final model EθE_θ Algorithm 1 describes the reranker-alignment procedure in two phases: initialization and iterative training. In the initialization phase (Lines 3–5), the LoRA parameters θLoRA _LoRA are randomly initialized and the preference dataset D is mined using the unmodified base cross-encoder EθbaseE_ _base. In other words, the randomly initialized adapters are not used to rank candidates during the initial mining step, preventing them from introducing arbitrary ordering effects into the first set of preference pairs. The LoRA adapters are then incorporated into the reranker, and the base parameters θbase _base are frozen. Only θLoRA _LoRA is updated during training. In the iterative training phase (Lines 6–13), the outer loop over epochs e triggers a re-mining step at the start of epoch e whenever (e−1)modN=0(e-1) N=0 and e>1e>1. Re-mining replaces stale preference pairs (scored under an older model snapshot) with fresh pairs scored by the current EθE_θ, so that d+d^+ and d−d^- reflect the model’s present ranking. This can provide a useful refresh when the reranker has changed substantially, but additional refreshes may yield diminishing returns after the model saturates. Within each epoch, the training set D is iterated over sample by sample (in simulations, in mini-batches of 16). Each training sample is a triple (qtext,d+,d−)(q_text,\,d^+,\,d^-) for which the cross-encoder produces two scalar logits s+=sθ(qtext,d+)s^+=s_θ(q_text,d^+) and s−=sθ(qtext,d−)s^-=s_θ(q_text,d^-), which are passed to ComputeLossComputeLoss to compute the alignment loss (the specific loss depends on the chosen method: Contrastive Triplet, SFT, or Pairwise DPO, as defined in Section IV-B). The resulting gradient is then back-propagated through θLoRA _LoRA, and the parameters are updated using AdamW. After the final epoch, the aligned reranker EθE_θ is saved. IV-B Alignment Loss Functions Once a preference pair (d+,d−)(d^+,d^-) has been mined, the reranker is trained to assign a higher score to the document that helps the frozen generator answer correctly than to the document that does not. For each pair, the cross-encoder produces two scalar logits, s+=sθ(qtext,d+)s^+=s_θ(q_text,d^+) and s−=sθ(qtext,d−)s^-=s_θ(q_text,d^-). • SFT: The first objective treats answer utility as a supervised regression target. The positive document is assigned the target score 11, while the negative document is assigned the target score 00: ℒSFT=(sθ(qtext,d+)−1)2+(sθ(qtext,d−)−0)2.L_SFT= (s_θ(q_text,d^+)-1 )^2+ (s_θ(q_text,d^-)-0 )^2. (7) This objective directly pushes useful documents toward a high absolute score and unhelpful documents toward a low absolute score. However, because reranking ultimately depends on relative ordering rather than calibrated score values, these fixed targets impose a stronger constraint than is strictly necessary. • Contrastive Triplet Loss: The second objective focuses directly on relative preference. Rather than assigning absolute score targets, the triplet loss trains the reranker by enforcing a relative ordering constraint: the relevance score of the positive document s+s^+ must exceed that of the negative document s−s^- by at least a margin m: ℒtriplet=max(0,m−(s+−s−)),m=0.2.L_triplet= \! (0,\;m-(s^+-s^-) ), m=0.2. (8) The max(0,⋅) (0,·) hinge structure implies that the loss is zero whenever s+−s−≥ms^+-s^-≥ m, i.e., the reranker already ranks d+d^+ sufficiently above d−d^-. A gradient is produced only when this margin is violated, pushing s+s^+ up and s−s^- down until the gap is restored. This makes triplet loss a natural objective for reranking because it optimizes the positive-over-negative ordering directly without requiring the logits to take any particular absolute values. Further, the margin m=0.2m=0.2 discourages near-ties by encouraging a small score buffer between preferred and rejected documents. • Pairwise DPO: The third objective adapts DPO [42] to reranking. Following the score definition in (6), the trainable logit is mapped to a bounded DPO-style log-score, ℓθ(qtext,d)=logσ(sθ(qtext,d))=logσ(Eθ([qtext;d])) _θ(q_text,d)= σ(s_θ(q_text,d))= σ(E_θ([q_text;d])), where σ(⋅)σ(·) denotes the logistic sigmoid. For the frozen reference reranker ErefE_ref, the analogous logit is sref(qtext,d)=Eref([qtext;d])s_ref(q_text,d)=E_ref([q_text;d]), yielding ℓref(qtext,d)=logσ(sref(qtext,d)) _ref(q_text,d)= σ(s_ref(q_text,d)). The preference margins are then defined as Δθ _θ =ℓθ(qtext,d+)−ℓθ(qtext,d−), = _θ(q_text,d^+)- _θ(q_text,d^-), (9) Δref _ref =ℓref(qtext,d+)−ℓref(qtext,d−), = _ref(q_text,d^+)- _ref(q_text,d^-), The DPO loss is then given by ℒDPO=−logσ(β[(Δθ)−(Δref)]).L_DPO=- σ\! (β [( _θ)-( _ref) ] ). (10) This objective encourages the LoRA-adapted reranker to prefer d+d^+ over d−d^- more strongly than the original base reranker does. At the same time, the reference margin anchors the update to the pre-trained model, reducing unnecessary deviation from its initial ranking behavior. The parameter β controls the strength of this reference-relative preference update. Together, the above objectives provide three complementary views of alignment: SFT learns absolute utility targets, triplet loss enforces relative ordering, and DPO improves the preferred-over-rejected margin relative to the frozen base reranker. Using the same mined preference pairs across all three objectives allows the experiments to isolate the effect of the supervision signal from the effect of the loss function. IV-C Preference-Mining Strategies The previous subsection defined how a mined pair (d+,d−)(d^+,d^-) is used to optimize the reranker under SFT, triplet, or DPO losses. We now describe how these positive-negative pairs are constructed. This distinction is important because all training objectives operate on the same pairwise format, while the supervision strategy determines what qualifies a document as preferred or rejected. For mining, qC_q is the Stage 1 top-K pool retrieved in (5), and qK′C_q^K denotes its top-K′K subset after sorting by the current reranker score, with K′≤K ≤ K. Algorithm 2 presents the proposed generator-guided strategy. Algorithm 3 covers the three baselines: first, random, and REPLUG-style likelihood (REPLUG-L). Algorithm 2 Generator-Guided Preference Mining (Proposed) 1: Queries Q, reranker EθE_θ, frozen VLM G, cap K′K , batch size BmB_m, cache Ψ 2: ←∅D← 3: for each (q,v,agold)∈(q,v,a_gold) do 4: qK′←top-K′(q,sθ(qtext,⋅))C_q^K -K (C_q;s_θ(q_text,·)) ⊳ descending score 5: d+←∅d^+← ; d−←∅d^-← 6: for i=0,Bm, 2Bm,…i=0,\,B_m,\,2B_m,… do 7: batch←qK′[i:i+Bm]batch _q^K [i:i+B_m] 8: for each d∈batchd do 9: key(d)←(doc-id(d),pos(d))key(d)←(doc -id(d),\,pos(d)) 10: if key(d)∉Ψkey(d)∉ then 11: Ψ[key(d)]←(v,q,d) [key(d)] (v,q,d) 12: if Ψ[key(d)]=agold [key(d)]=a_gold and d+=∅d^+= then 13: d+←d^+← d 14: if Ψ[key(d)]≠agold [key(d)]≠ a_gold and d−=∅d^-= then 15: d−←d^-← d 16: if d+≠∅d^+≠ and d−≠∅d^-≠ then 17: break ⊳ Early exit: both examples identified 18: if d+≠∅d^+≠ and d−≠∅d^-≠ then 19: ←∪(qtext,d+,d−)D ∪\(q_text,\,d^+,\,d^-)\ 20: return D 1) Generator-Guided Preference Mining: Algorithm 2 specifies the proposed generator-guided mining strategy. Its purpose is not to estimate generic document relevance, but to ask whether each candidate changes the deployed generator’s answer behavior. For each query triple (q,v,agold)(q,v,a_gold), the retained candidate set qK′C_q^K is first sorted by sθ(qtext,⋅)s_θ(q_text,·) in descending order (Line 4), so that the most confidently ranked documents are evaluated first. The algorithm then iterates over mini-batches of BmB_m candidates in that order (Lines 6–17). For each candidate d, a cache key is formed from its document identifier and position in the retrieved pool (Line 9); if the key is absent from the persistent cache Ψ , the frozen VLM G is queried with (v,q,d)(v,q,d) and the predicted answer is stored (Lines 10–11). If the key is already cached, the stored prediction is reused instead of prompting the VLM again. If the prediction matches agolda_gold and no positive has been assigned yet, d is taken as d+d^+ (Lines 12–13). If it does not match and no negative has been assigned yet, d is taken as d−d^- (Lines 14–15). Once both d+d^+ and d−d^- are identified, an early exit terminates the remaining mini-batches immediately (Lines 16–17), avoiding superfluous VLM calls. A pair is added to D only when both d+d^+ and d−d^- are found (Lines 18–19); queries for which the pool contains no correct or no incorrect prediction are silently skipped. 2) Controlled Baselines: Algorithm 3 defines the three controlled baselines. All baselines use the same Stage 1 candidate pools, reranker architecture, LoRA training procedure, and alignment losses as the proposed method. The only difference is how d+d^+ and d−d^- are defined. By comparing the proposed method with these baselines, we can assess whether answer-utility supervision is the key source of the gains. The random baseline (Lines 6–7) draws d+d^+ and d−d^- uniformly at random from qK′C_q^K without replacement. It uses neither reranker scores nor VLM feedback. This baseline therefore measures how the alignment losses behave when the preference labels contain no meaningful supervision and provides a noise-sensitivity reference. The first baseline (Lines 8–10) relies only on the current reranker ordering. It selects the highest-scored candidate as d+d^+ and the bottom-ranked candidate as d−d^-, without consulting the VLM. This strategy tests whether simply reinforcing the reranker’s existing semantic ranking is sufficient to improve performance. Because it never prompts the VLM, the selected positive may be semantically relevant without helping the generator answer correctly. The REPLUG-style likelihood baseline (REPLUG-L) (Lines 11–17) uses the VLM differently from our generator strategy. Rather than asking the VLM to produce the constrained answer output, it scores every candidate with a REPLUG-style LM-likelihood signal: the log-likelihood of agolda_gold conditioned on (v,q,d)(v,q,d), where agolda_gold is provided token-by-token as input rather than being generated: ℓ(d)=1|agold|∑t=1|agold|logp(agold,t∣v,q,d,agold,<t). (d)= 1|a_gold| _t=1^|a_gold| p_G\! (a_gold,t v,\,q,\,d,\,a_gold,<t ). (11) Here, p_G denotes the frozen VLM’s next-token probability, |agold||a_gold| is the number of target answer tokens, agold,ta_gold,t is the t-th token, and agold,<ta_gold,<t denotes preceding target tokens. At each step t, the VLM G is conditioned on (v,q,d,agold,<t)(v,q,d,a_gold,<t), and the log probability assigned to agold,ta_gold,t is averaged across the target answer tokens to produce ℓ(d) (d). This is a REPLUG-style ranking score rather than the full REPLUG LSR objective, because we use it only to choose d+d^+ and d−d^- instead of training a retriever with a document-level KL loss. A higher ℓ(d) (d) means that G assigns greater probability to the target answer when conditioned on document d and the preceding ground-truth answer tokens. The highest-scoring document is selected as d+d^+ and the lowest-scoring document as d−d^- (Lines 16–17), so all K′K candidates must be evaluated before selection and early exit is not possible. In contrast, our generator-guided strategy asks the VLM to produce the constrained answer output and checks whether it matches the ground-truth. Thus, a document may have a high ℓ(d) (d) even if the VLM would fail to produce the correct constrained answer without those ground-truth answer tokens. Algorithm 3 Baseline Preference Mining 1: Queries Q, reranker EθE_θ, cap K′K , method m 2: If m=replug-lm=replug -l: frozen VLM G, batch size BmB_m, cache Ψ 3: ←∅D← 4: for each (q,v,agold)∈(q,v,a_gold) do 5: qK′←top-K′(q,sθ(qtext,⋅))C_q^K -K (C_q;s_θ(q_text,·)) ⊳ descending score 6: if m=randomm=random then 7: d+,d−∼Uniform(qK′)d^+,d^- (C_q^K ) without replacement 8: if m=firstm=first then 9: d+←argmaxd∈qK′sθ(qtext,d)d^+← _d _q^K s_θ(q_text,d) 10: d−←argmind∈qK′sθ(qtext,d)d^-← _d _q^K s_θ(q_text,d) 11: if m=replug-lm=replug -l then 12: for each d∈qK′d _q^K in mini-batches of BmB_m do 13: key(d)←(doc-id(d),pos(d))key(d)←(doc -id(d),\,pos(d)) 14: if key(d)∉Ψkey(d)∉ then 15: Ψ[key(d)]←ℓ(d) [key(d)]← (d) 16: d+←argmaxd∈qK′Ψ[key(d)]d^+← _d _q^K [key(d)] 17: d−←argmind∈qK′Ψ[key(d)]d^-← _d _q^K [key(d)] 18: if d+≠∅d^+≠ and d−≠∅d^-≠ then 19: ←∪(qtext,d+,d−)D ∪\(q_text,\,d^+,\,d^-)\ 20: return D V Experiments We first describe the implementation details, followed by the results and analysis. V-A Implementation Details TABLE I: Default Hyperparameter Configuration for VQA-X and A-OKVQA Category Hyperparameter Value Architecture VLM Generator Qwen3-VL-4B-Instruct, Qwen3.5-2B Sentence Transformer Qwen3-Embedding-8B Reranker Backbone GTE-ModernBERT reranker Trainable Module LoRA-adapted reranker LoRA Adapter Rank (r) 16 Alpha (α) 32 Dropout 0.1 Optimization Optimizer AdamW Learning Rate 5×10−55× 10^-5 Training Epochs 2 Batch Size 16 Objective DPO Beta (β) 0.1 Mining Stage 2 Pool Size K′=10K =10 of 100 candidates Mining Batch Size 16 Re-mining Frequency 0 (disabled) REPLUG-L Pool Size K′=10K =10 of 100 candidates We apply one default hyperparameter configuration across both datasets, summarized in Table I; experiments that deviate from these defaults state the changed value explicitly. Stage 1 retrieves 100 candidates per query using the Qwen3-Embedding-8B dense encoder with a FAISS index. For Stage 2, the cross-encoder backbone is Alibaba-NLP/gte-reranker-modernbert-base. Preference mining uses the top 10 retrieved candidates by default (K′=10K =10), with re-mining disabled unless otherwise stated. We evaluate two frozen VLM generators, Qwen3-VL-4B-Instruct and Qwen3.5-2B, to assess whether the proposed training-signal strategy generalizes across generators with different capacities. The reranker is fine-tuned with LoRA using rank r=16r=16, scaling factor α=32α=32, and dropout 0.10.1. Optimization uses AdamW with a learning rate of 5×10−55× 10^-5, batch size of 16, and 2 epochs unless otherwise stated. For Pairwise DPO, the inverse-temperature parameter is set to β=0.1β=0.1. We do not apply periodic re-mining in the main comparison (remine_every=0); re-mining frequency is studied separately in Table VII. The REPLUG-style likelihood baseline scores all 10 candidates by ground-truth-token log-likelihood. Stage 1 and Stage 2 are split-aware: the retrieval index is built from training-split documents only, while Stage 2 loads the selected dataset split to recover images and labels for the candidate question identifiers. A-OKVQA is evaluated as a multiple-choice task: the prompt lists choices as A), B), etc. Although the Qwen model cards and generation configs provide sampling-oriented default or recommended values, our evaluation uses deterministic greedy decoding by setting do_sample=False with max_new_tokens=10; temperature and top-p sampling are therefore not used. The output is parsed first as a leading valid answer letter, with a fallback regex over standalone letters. Accuracy is exact match against correct_choice_idx. VQA-X is evaluated on yes/no questions: the prompt instructs the VLM to answer strictly with “yes” or “no”, generation uses the same deterministic decoding settings, outputs are lowercased, simple markers such as *, _, and periods are removed before yes/no matching, and accuracy is the standard VQA soft score min(#matching annotators/3,1) (\#matching annotators/3,1). During binary preference mining for VQA-X, a prediction is treated as correct when this VQA score reaches 1. V-B Results and Analysis TABLE I: Impact of Training-Signal Construction Across Alignment Loss Functions (2 Epochs, Generator: Qwen3-VL-4B-Instruct, Candidate Pool: Top 10). Results reported as mean ± std across 3 random seeds. Alignment Loss Training Signal VQA-X (%) A-OKVQA (%) Base No training 94.75 85.04 Contrastive (Triplet) First 94.72± 0.22 85.07± 0.11 Random 92.96± 0.14 82.77± 0.72 REPLUG-L 95.85± 0.10 82.03± 0.28 Generator 96.27± 0.03 87.00± 0.13 Pairwise DPO First 94.82± 0.04 85.15± 0.05 Random 93.84± 0.82 83.63± 1.16 REPLUG-L 95.79± 0.09 82.40± 0.27 Generator 96.11± 0.04 87.16± 0.03 SFT First 94.75± 0.05 85.16± 0.02 Random 93.35± 0.37 80.95± 0.72 REPLUG-L 95.69± 0.04 82.65± 0.14 Generator 95.73± 0.01 85.75± 0.08 TABLE IV: Impact of Training-Signal Construction Across Alignment Loss Functions (2 Epochs, Generator: Qwen3.5-2B, Candidate Pool: Top 10). Results reported as mean ± std across 3 random seeds. Alignment Loss Training Signal VQA-X (%) A-OKVQA (%) Base No training 90.28 79.83 Contrastive (Triplet) First 90.24± 0.13 79.76± 0.11 Random 89.13± 2.01 77.02± 1.37 REPLUG-L 92.44± 0.30 76.98± 0.07 Generator 94.82± 0.11 84.67± 0.21 Pairwise DPO First 90.20± 0.18 79.99± 0.08 Random 89.49± 0.78 79.50± 0.61 REPLUG-L 92.76± 0.18 76.97± 0.11 Generator 93.32± 0.04 82.50± 0.06 SFT First 90.12± 0.16 79.94± 0.08 Random 89.27± 0.71 76.72± 0.63 REPLUG-L 92.13± 0.14 76.95± 0.12 Generator 92.54± 0.10 80.75± 0.16 We evaluate our proposed framework across three dimensions: (i) the superiority of the generator-guided training signal over alternative mining strategies under all three alignment losses (Tables I and IV), (i) the robustness of the generator strategy to the size of the Stage 1 candidate pool (Tables V and VI), and (i) the impact of periodic preference re-mining on accuracy (Table VII). We note that direct comparison with existing retriever–generator alignment systems is difficult because prior methods often target text-only tasks, use different retrieval corpora and generators, or require joint retriever–generator training or an additional trainable bridge module. We therefore adopt a controlled comparison in which the Stage 1 candidate pool, reranker architecture, frozen generator, training configuration, and alignment loss are held fixed, while only the strategy used to construct (d+,d−)(d^+,d^-) is varied. Under this design, the First baseline tests whether reinforcing the reranker’s existing score order is sufficient; the Random baseline measures the behavior of pairwise training under uninformative preference labels; and the REPLUG-style likelihood baseline represents likelihood-based generator-aware supervision adapted from prior work [12]. Our proposed method differs by assigning preferences according to whether each candidate enables the frozen VLM to produce the correct constrained answer output. All trained results are reported as mean ± standard deviation over three random seeds. The seeds vary the LoRA initialization, training-data order, and random pair sampling where applicable, while the frozen generators and deterministic decoding procedure remain unchanged. We therefore assess a strategy based not only on its mean accuracy, but also on whether its gains are consistent across datasets, generators, alignment losses, and random seeds. The following subsections analyze the three dimensions in sequence. V-B1 Generator-Guided Mining Outperforms All Baselines Tables I and IV compare all training-signal strategies under three alignment losses for Qwen3-VL-4B-Instruct and Qwen3.5-2B, respectively, on VQA-X and A-OKVQA. The base cross-encoder selects the first document as the context and achieves 94.75% / 85.04% (Qwen3-VL-4B-Instruct) and 90.28% / 79.83% (Qwen3.5-2B), establishing the no-training reference that an effective strategy must surpass. Our proposed generator strategy achieves the highest accuracy in the evaluated cells of both tables, regardless of alignment loss, generator, or dataset. With Qwen3-VL-4B-Instruct, it reaches 96.27% / 87.00% under Contrastive (Triplet) and 96.11% / 87.16% under Pairwise DPO, with SFT trailing at 95.73% / 85.75%. With the smaller Qwen3.5-2B, Contrastive training achieves the best result (94.82% / 84.67%), followed by Pairwise DPO (93.32% / 82.50%) and SFT (92.54% / 80.75%). The consistent advantage across both generators suggests that the improvement stems from the quality of the training signal rather than any capacity-specific effect. The controlled baselines further clarify which supervision signal matters. The First strategy represents rank-order self-training: it reinforces the reranker’s own semantic ordering without answer-level feedback, and it often fails to improve over the untrained base (e.g., 94.72% / 85.07% vs. the base 94.75% / 85.04% for Qwen3-VL-4B-Instruct under Contrastive). The Random strategy tests whether gains arise merely from LoRA fine-tuning with pairwise objectives; its frequent degradation below the base (e.g., 92.96% on VQA-X for Qwen3-VL-4B-Instruct under Contrastive, and 89.13% for Qwen3.5-2B) shows that arbitrary pair construction can harm the reranker. The REPLUG-style likelihood baseline (REPLUG-L) is the closest prior-work-connected comparison, adapting likelihood-based generator-aware retrieval supervision. It improves VQA-X over the base across all configurations (e.g., 95.85%, 95.79%, and 95.69% for Qwen3-VL-4B-Instruct under the three losses), but it hurts A-OKVQA accuracy relative to the base in these experiments (e.g., 82.03%–82.65% vs. the base 85.04% for Qwen3-VL-4B-Instruct). This asymmetry suggests that ground-truth-token likelihood is an imperfect proxy for constrained-answer correctness: a high likelihood of agolda_gold when preceding answer tokens are supplied does not reliably indicate that G would produce the correct constrained answer. Across the three alignment losses, Contrastive (Triplet) emerges as a strong default: it is consistently competitive and becomes the clear best-performing objective with Qwen3.5-2B, which motivates using it in the pool-sensitivity and re-mining studies below. At the same time, Pairwise DPO reaches comparable accuracy with Qwen3-VL-4B-Instruct under the same generator-guided mining signal. Thus, the main finding is not that triplet loss universally dominates, but that answer-utility preference mining is the decisive factor, with triplet loss serving as a reliable and simple instantiation. V-B2 Robustness to Stage 1 Candidate Pool Size Tables V and VI investigate how sensitive each training signal is to K′K , the number of Stage 1 candidates used during preference mining, under Contrastive (Triplet) training at 2 epochs for Qwen3-VL-4B-Instruct and Qwen3.5-2B, respectively. The results show that the generator strategy is stable across pool sizes, though the degree of stability differs by generator. For Qwen3-VL-4B-Instruct, VQA-X accuracy increases modestly from 95.88% at Top 2 to 96.27% at Top 10 (a range of 0.39 p), and A-OKVQA grows from 86.22% to 87.00% (0.78 p). For Qwen3.5-2B, the variation is more pronounced: VQA-X rises from 92.76% at Top 2 to 94.82% at Top 10 (2.06 p), and A-OKVQA from 81.45% to 84.67% (3.22 p), suggesting that Qwen3.5-2B, despite being smaller and from a different model generation, benefits more from a richer candidate pool when constructing preference pairs. Notably, our proposed generator strategy with the smallest pool of Top 2 already outperforms the evaluated baselines at their largest pool of Top 10 across both generators and datasets. This aligns with recent budget-aware RAG findings that the utility of retrieved chunks, not merely the number of chunks used, is central to effective retrieval augmentation [43]; in our setting, the quality of the training signal matters more than the quantity of candidates evaluated. Further inspection shows that the Random strategy has the highest variance across pool sizes (e.g., 87.73%–89.13% on VQA-X for Qwen3.5-2B), because the quality of the sampled pair depends heavily on which documents happen to fall inside the pool. The First strategy is essentially insensitive to pool size, remaining near base-level performance regardless of K′K , because it always selects the top-1 and bottom-1 of the ranked list and the pool boundary only marginally affects the bottom candidate. REPLUG-L scores all K′K candidates in the pool, so it naturally uses more information as the pool grows; even so, at Top 10 it remains below our proposed generator strategy by 0.42 p on VQA-X and 4.97 p on A-OKVQA for Qwen3-VL-4B-Instruct (95.85% vs. 96.27% and 82.03% vs. 87.00%). For Qwen3.5-2B the gap is even larger: 2.38 p on VQA-X (92.44% vs. 94.82%) and 7.69 p on A-OKVQA (76.98% vs. 84.67%). This suggests that REPLUG-L’s shortfall is not only a matter of how many candidates it evaluates, but also that its ground-truth-token likelihood signal can be a weaker proxy for downstream answer correctness than the direct binary judgment used by the generator strategy. TABLE V: Sensitivity to Stage 1 Candidate Pool in Contrastive (Triplet) Training (2 Epochs, Generator: Qwen3-VL-4B-Instruct). Results reported as mean ± std across 3 random seeds. Training Signal Candidate Pool VQA-X (%) A-OKVQA (%) Generator (Ours) Top 2 95.88± 0.03 86.22± 0.09 Top 4 96.06± 0.03 86.68± 0.06 Top 6 96.16± 0.03 86.88± 0.05 Top 8 96.18± 0.08 86.93± 0.06 Top 10 96.27± 0.03 87.00± 0.13 First Top 2 94.73± 0.04 85.05± 0.01 Top 4 94.76± 0.09 85.05± 0.03 Top 6 94.68± 0.07 85.03± 0.02 Top 8 94.61± 0.18 85.06± 0.05 Top 10 94.72± 0.22 85.07± 0.11 Random Top 2 92.62± 1.05 82.24± 0.60 Top 4 93.50± 0.85 82.50± 0.48 Top 6 93.07± 1.31 84.15± 0.25 Top 8 93.18± 0.42 82.61± 0.92 Top 10 92.96± 0.14 82.77± 0.72 REPLUG-L Top 2 95.52± 0.03 81.45± 0.20 Top 4 95.75± 0.10 81.58± 0.38 Top 6 95.86± 0.10 81.89± 0.11 Top 8 95.83± 0.10 82.04± 0.09 Top 10 95.85± 0.10 82.03± 0.28 TABLE VI: Sensitivity to Stage 1 Candidate Pool in Contrastive (Triplet) Training (2 Epochs, Generator: Qwen3.5-2B). Results reported as mean ± std across 3 random seeds. Training Signal Candidate Pool VQA-X (%) A-OKVQA (%) Generator (Ours) Top 2 92.76± 0.02 81.45± 0.02 Top 4 93.01± 0.05 81.87± 0.16 Top 6 93.11± 0.06 82.02± 0.01 Top 8 93.29± 0.04 81.97± 0.02 Top 10 94.82± 0.11 84.67± 0.21 First Top 2 90.28± 0.05 79.85± 0.03 Top 4 90.25± 0.12 79.76± 0.06 Top 6 90.24± 0.05 79.85± 0.01 Top 8 90.33± 0.24 79.74± 0.11 Top 10 90.24± 0.13 79.76± 0.11 Random Top 2 87.73± 0.85 78.18± 0.69 Top 4 88.94± 1.64 76.64± 1.12 Top 6 88.34± 1.00 77.76± 0.90 Top 8 88.80± 1.77 78.30± 1.12 Top 10 89.13± 2.01 77.02± 1.37 REPLUG-L Top 2 91.58± 0.27 76.89± 0.22 Top 4 91.92± 0.23 76.91± 0.25 Top 6 92.17± 0.12 77.13± 0.06 Top 8 92.33± 0.20 77.10± 0.07 Top 10 92.44± 0.30 76.98± 0.07 TABLE VII: Impact of Re-mining Frequency on Downstream Accuracy (Candidate Pool: Top 10). Results reported as mean ± std across 3 random seeds. Generator # of Re-mines VQA-X (%) A-OKVQA (%) Qwen3-VL-4B -Instruct 0 96.27± 0.03 87.00± 0.13 2 97.45± 0.09 89.01± 0.23 4 97.44± 0.09 89.58± 0.04 Qwen3.5-2B 0 94.43± 0.09 82.24± 0.14 2 94.82± 0.11 84.47± 0.30 4 95.16± 0.13 84.67± 0.21 V-B3 Impact of Periodic Re-mining Table VII examines the effect of re-mining frequency on downstream accuracy over 4 training epochs using the generator strategy and Contrastive (Triplet) loss. Three configurations are compared: no re-mining (the preference dataset is fixed throughout training), re-mining twice (once at the start before training and once at mid-training), and re-mining four times (once per epoch). Periodic re-mining generally enhances the performance compared with using a fixed preference dataset, but the benefit of increasing the re-mining frequency is model- and dataset-dependent. For Qwen3-VL-4B-Instruct on VQA-X, re-mining twice and four times are effectively tied within the reported seed variation (97.45± 0.09% vs. 97.44± 0.09%), indicating saturation rather than a meaningful gain from additional refreshes. In contrast, re-mining four times yields the best A-OKVQA accuracy for Qwen3-VL-4B-Instruct (89.58% vs. 87.00% without re-mining, +2.58 p). For Qwen3.5-2B, re-mining four times produces the best results on both datasets (95.16% on VQA-X, +0.73 p; 84.67% on A-OKVQA, +2.43 p over no re-mining). These results suggest that refreshing stale preference pairs can improve training, while the marginal gains from more frequent refreshes can saturate once the reranker stabilizes. This is also evident from the fact that the incremental gain from four rather than two re-mines is small and dataset-dependent. For Qwen3-VL-4B-Instruct, VQA-X is effectively tied (97.44% vs. 97.45%), while four re-mines add 0.57 p on A-OKVQA (89.58% vs. 89.01%). For Qwen3.5-2B, four re-mines give modest gains over two on both datasets (+0.34 p VQA-X, +0.20 p A-OKVQA). Re-mining also adds cost because each refresh must re-rank the mining queries, rebuild pairs from the current top-K′K subset, and evaluate any newly reached uncached query-document pairs. Thus, re-mining twice offers a favorable accuracy-cost balance, while additional refreshes should be treated as optional. VI Limitations This work presents several opportunities for further development. First, although the framework eliminates the need for human document-level relevance annotations, preference mining still relies on answer-labeled training examples to determine whether a candidate document helps the frozen generator produce the correct answer. Future work could reduce this dependence by using self-supervised consistency signals, confidence-based pseudo-labels, or other forms of weak supervision. Second, preference mining introduces additional computational cost because the VLM must evaluate multiple candidate documents for each query, and periodic re-mining repeats this process as the reranker evolves. This overhead can be further reduced through more selective candidate evaluation, adaptive re-mining schedules, and lightweight surrogate models that approximate the generator’s utility judgments. Third, the learned reranker is intentionally generator-specific, as it is optimized to select evidence that improves the frozen VLM’s behavior during training rather than to learn a universal notion of document relevance. Future work could improve transferability by incorporating preferences from multiple generators or by developing generator-agnostic utility representations that require only limited adaptation to a new VLM. Finally, the current pairwise training procedure uses only queries whose candidate pools contain at least one helpful and one unhelpful document. While this requirement produces clean positive–negative pairs, it excludes queries for which all candidates yield the same outcome and uses only a limited portion of the available utility information. This limitation could be addressed through graded, listwise, or multi-document utility supervision that uses more of the information available in the candidate pool. VII Conclusion This paper introduced a generator-in-the-loop alignment framework for multimodal RAG. Our proposed two-stage framework separates modality bridging from answer-utility alignment: Stage 1 uses a frozen VLM to generate a HyDE-style text query for dense recall, while Stage 2 trains a LoRA-adapted cross-encoder reranker from answer-supervised preference pairs mined with the same frozen generator. We demonstrated that across VQA-X and A-OKVQA datasets, generator-guided preferences improve over rank-order, random, and REPLUG-style likelihood baselines under multiple alignment losses, suggesting that downstream answer behavior is a useful signal for reranker alignment. The results also showed that periodic re-mining can improve accuracy by refreshing preference pairs as the reranker changes, although the marginal value of more frequent refreshes depends on the model and dataset. Overall, the findings support a generator-centered view of RAG reranking: the most useful evidence is not the passage semantically closest to the query, but the passage that helps the deployed generator answer correctly. VIII Acknowledgments Generative AI tools, including ChatGPT (OpenAI) [44] and Claude (Anthropic) [45], were used to assist with the preparation and refinement of manuscript text, including sentence-level language improvement. All technical content was developed by the authors, and the full manuscript was carefully reviewed and verified by the authors to ensure accuracy and correctness. References [1] L. Yuan, D. Han, S. Wang, and C. Brinton (2025) Local-cloud inference offloading for llms in multi-modal, multi-task, multi-dialogue settings. In Proceedings of the Twenty-sixth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, p. 201–210. Cited by: §I. [2] W. Fang, D. Han, L. Yuan, E. Chen, and C. Brinton (2026) Bridging on-device and cloud llms for collaborative reasoning: a unified methodology for local routing and post-training. In Forty-third International Conference on Machine Learning, Cited by: §I. [3] L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. (2025) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM transactions on information systems 43 (2), p. 1–55. Cited by: §I. [4] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems 33, p. 9459–9474. Cited by: §I, §I-A. [5] S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J. Lespiau, B. Damoc, A. Clark, et al. (2022) Improving language models by retrieving from trillions of tokens. In International conference on machine learning, p. 2206–2240. Cited by: §I. [6] T. R. McIntosh, T. Liu, T. Susnjak, P. Watters, A. Ng, and M. N. Halgamuge (2023) A culturally sensitive test to evaluate nuanced GPT hallucination. IEEE Transactions on Artificial Intelligence 5 (6), p. 2739–2751. Cited by: §I. [7] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, H. Wang, and H. Wang (2023) Retrieval-augmented generation for large language models: a survey. arXiv preprint arXiv:2312.10997 2 (1). Cited by: §I, §I-A, §I-C. [8] O. Khattab, A. Singhvi, P. Maheshwari, Z. Zhang, K. Santhanam, S. Vardhamanan, S. Haq, A. Sharma, T. T. Joshi, H. Moazam, et al. (2023) DSPy: compiling declarative language model calls into self-improving pipelines. arXiv preprint arXiv:2310.03714. Cited by: §I, §I-A. [9] X. Ma, Y. Gong, P. He, H. Zhao, and N. Duan (2023) Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 5303–5315. Cited by: §I, §I-A. [10] F. Cuconasu, G. Trappolini, F. Siciliano, S. Filice, C. Campagnano, Y. Maarek, N. Tonellotto, and F. Silvestri (2024) The power of noise: redefining retrieval for RAG systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 719–729. Cited by: §I. [11] G. Izacard, P. Lewis, M. Lomeli, L. Hosseini, F. Petroni, T. Schick, J. Dwivedi-Yu, A. Joulin, S. Riedel, and E. Grave (2023) Atlas: few-shot learning with retrieval augmented language models. Journal of Machine Learning Research 24 (251), p. 1–43. Cited by: §I, §I-C, §I-C. [12] W. Shi, S. Min, M. Yasunaga, M. Seo, R. James, M. Lewis, L. Zettlemoyer, and W. Yih (2024) REPLUG: retrieval-augmented black-box language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 8371–8384. Cited by: §I, §I-C, §I-C, §I-E, §V-B. [13] Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, et al. (2025) Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §I. [14] L. Wang, N. Yang, and F. Wei (2024) Learning to retrieve in-context examples for large language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1752–1767. Cited by: §I. [15] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024) Lost in the middle: how language models use long contexts. Transactions of the association for computational linguistics 12, p. 157–173. Cited by: §I, §I-A. [16] O. Yoran, T. Wolfson, O. Ram, and J. Berant (2024) Making retrieval-augmented language models robust to irrelevant context. In International Conference on Learning Representations, Vol. 2024, p. 29862–29883. Cited by: §I. [17] K. Guu, K. Lee, Z. Tung, P. Pasupat, and M. Chang (2020) Retrieval augmented language model pre-training. In International conference on machine learning, p. 3929–3938. Cited by: §I-A. [18] S. Gupta, R. Ranjan, and S. N. Singh (2024) A comprehensive survey of retrieval-augmented generation (RAG): evolution, current landscape and future directions. arXiv preprint arXiv:2410.12837. Cited by: §I-A. [19] V. Karpukhin, B. Oguz, S. Min, P. S. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih (2020) Dense passage retrieval for open-domain question answering.. In EMNLP (1), p. 6769–6781. Cited by: §I-A. [20] K. Shuster, S. Poff, M. Chen, D. Kiela, and J. Weston (2021) Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, p. 3784–3803. Cited by: §I-A. [21] W. Yu, H. Zhang, X. Pan, P. Cao, K. Ma, J. Li, H. Wang, and D. Yu (2024) Chain-of-note: enhancing robustness in retrieval-augmented language models. In Proceedings of the 2024 conference on empirical methods in natural language processing, p. 14672–14685. Cited by: §I-A. [22] Z. Xu, G. Pang, X. Chen, M. Ding, B. Vucetic, Y. Li, and Z. Li (2026) MAQ-retrieval: multi-aspect queries retrieval for large language models. IEEE Transactions on Artificial Intelligence. Cited by: §I-A. [23] L. Gao, X. Ma, J. Lin, and J. Callan (2023) Precise zero-shot dense retrieval without relevance labels. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1762–1777. Cited by: §I-A, §I-D. [24] R. Nogueira and K. Cho (2019) Passage re-ranking with bert. arXiv preprint arXiv:1901.04085. Cited by: §I-A, §I-C. [25] W. Sun, L. Yan, X. Ma, S. Wang, P. Ren, Z. Chen, D. Yin, and Z. Ren (2023) Is ChatGPT good at search? investigating large language models as re-ranking agents. In Proceedings of the 2023 conference on empirical methods in natural language processing, p. 14918–14937. Cited by: §I-A, §I-C. [26] A. J. Oche, A. G. Folashade, T. Ghosal, and A. Biswas (2025) A systematic review of key retrieval-augmented generation (RAG) systems: progress, gaps, and future directions. arXiv preprint arXiv:2507.18910. Cited by: §I-A, §I-B. [27] Z. Jiang, F. F. Xu, L. Gao, Z. Sun, Q. Liu, J. Dwivedi-Yu, Y. Yang, J. Callan, and G. Neubig (2023) Active retrieval augmented generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 7969–7992. Cited by: §I-A. [28] S. Yan, J. Gu, Y. Zhu, and Z. Ling (2024) Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884. Cited by: §I-A. [29] D. Zhu, X. Shen, X. Li, M. Elhoseiny, et al. (2024) Minigpt-4: enhancing vision-language understanding with advanced large language models. In International Conference on Learning Representations, Vol. 2024, p. 18378–18394. Cited by: §I-B. [30] H. Liu, C. Li, Q. Wu, and Y. J. Lee (2023) Visual instruction tuning. Advances in Neural Information Processing Systems 36, p. 34892–34916. Cited by: §I-B. [31] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. (2021) Learning transferable visual models from natural language supervision. In International conference on machine learning, p. 8748–8763. Cited by: §I-B. [32] J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. (2022) Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems 35, p. 23716–23736. Cited by: §I-B. [33] J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. (2023) Qwen technical report. arXiv preprint arXiv:2309.16609. Cited by: §I-B. [34] M. Yasunaga, A. Aghajanyan, W. Shi, R. James, J. Leskovec, P. Liang, M. Lewis, L. Zettlemoyer, and W. Yih (2022) Retrieval-augmented multimodal language modeling. arXiv preprint arXiv:2211.12561. Cited by: §I-B, §I-C. [35] M. Faysse, H. Sibille, T. Wu, B. Omrani, G. Viaud, C. Hudelot, and P. Colombo (2025) Colpali: efficient document retrieval with vision language models. In International Conference on Learning Representations, Vol. 2025, p. 61424–61449. Cited by: §I-B. [36] B. He, N. Chen, X. He, L. Yan, Z. Wei, J. Luo, and Z. Ling (2024) Retrieving, rethinking and revising: the chain-of-verification can improve retrieval augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 10371–10393. Cited by: §I-C. [37] O. Ram, Y. Levine, I. Dalmedigos, D. Muhlgay, A. Shashua, K. Leyton-Brown, and Y. Shoham (2023) In-context retrieval-augmented language models. Transactions of the Association for Computational Linguistics 11, p. 1316–1331. Cited by: §I-C. [38] Z. Ke, W. Kong, C. Li, M. Zhang, Q. Mei, and M. Bendersky (2024) Bridging the preference gap between retrievers and LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 10438–10451. Cited by: §I-C, §I-C. [39] Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu (2023) G-Eval: NLG evaluation using GPT-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 2511–2522. Cited by: §I-C. [40] Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi (2023) Self-Instruct: aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 13484–13508. Cited by: §I-C. [41] B. Warner, A. Chaffin, B. Clavié, O. Weller, O. Hallström, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsen, et al. (2025) Smarter, better, faster, longer: a modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference. In Proceedings of the 63rd annual meeting of the association for computational linguistics (volume 1: Long papers), p. 2526–2547. Cited by: §I-E. [42] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn (2023) Direct preference optimization: your language model is secretly a reward model. Advances in Neural Information Processing Systems 36, p. 53728–53741. Cited by: §I-E, 3rd item. [43] S. Al-Maliki, A. Gharaibeh, M. Rahouti, M. R. Amin, M. Abdallah, J. Qadir, and A. Al-Fuqaha (2026) Budget-constrained online retrieval-augmented generation: the chunk-as-a-service model. IEEE Transactions on Artificial Intelligence. Cited by: §V-B2. [44] OpenAI ChatGPT. Note: [Online]. Available: https://chatgpt.com/ Cited by: §VIII. [45] Anthropic Claude. Note: [Online]. Available: https://claude.ai/ Cited by: §VIII.