Paper deep dive
Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental Refinement
Shenao Chen, Yidan Xu, Xiangmin Han, Rundong Xue, Duanpo Wu, Yuhan Gao, Chenggang Yan, Yue Gao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/23/2026, 2:28:58 AM
Summary
The paper introduces Hyper-M2RAG, a framework for Multimodal Retrieval-Augmented Generation that utilizes High-order Hypergraph Representation Learning to capture complex, N-ary relationships between text, images, and tables, overcoming the limitations of binary graph models. It employs an Anchor-driven Incremental Refinement mechanism to efficiently bridge cross-page knowledge gaps by reconstructing local hyper-topologies around boundary-crossing anchor nodes, thereby reducing computational redundancy and contextual noise compared to full-page reconstruction methods.
Entities (8)
Relation Signals (6)
Hyper-M2RAG → uses → Multimodal Hypergraph
confidence 95% · Hyper-M2RAG constructs a multimodal hypergraph in which vertices represent textual, visual, and tabular entities, while hyperedges connect multiple heterogeneous elements
Hyper-M2RAG → employs → Anchor-driven Incremental Refinement
confidence 94% · To further support long-document reasoning, Hyper-M2RAG introduces an anchor-driven surgical refinement mechanism.
Anchor-driven Incremental Refinement → targets → Cross-page Anchors
confidence 92% · we introduce a refinement stage centered on Cross-page Anchors Va. These anchors are identified as entities with high provenance frequency
Multimodal Hypergraph → models → N-ary Relationships
confidence 90% · utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling.
Hyper-M2RAG → outperforms → State-of-the-Art Methods
confidence 88% · Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence.
MinerU → usedby → Hyper-M2RAG
confidence 85% · we utilize MinerU as a high-fidelity engine to parse raw PDFs into a structured sequence
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities, such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data. Furthermore, existing refinement strategies often rely on exhaustive, full-page reconstruction to align cross-modal information, leading to prohibitive computational redundancy and the introduction of contextual noise in long-form document processing. In this paper, we propose Hyper-M2RAG, a novel framework that redefines multimodal document retrieval through High-order Hypergraph Representation Learning. We first formalize the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling. To mitigate semantic fragmentation caused by physical pagination, we introduce an Anchor-driven Incremental Refinement mechanism. Rather than performing a global sweep, our approach identifies boundary-crossing anchor nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts. This targeted refinement effectively bridges cross-page knowledge gaps with minimal computational footprints. Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence. Our code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.16628v1
- Canonical: https://arxiv.org/abs/2608.16628v1
Trouble viewing inline? Open PDF directly →
Full Text
51,598 characters extracted from source content.
Expand or collapse full text
Hypergraph-based Multimodal Retrieval-Augmented Generation with Incremental RefinementConference: Proceedings of the 34th ACM International Conference on Multimedia; November 10–14, 2026; Rio de Janeiro, Brazil.Proceedings of the 34th ACM International Conference on Multimedia (M ’26), November 10–14, 2026, Rio de Janeiro, BrazilISBN: 979-8-4007-2213-4/2026/11DOI: 10.1145/3767308.3835434CCS: Information systems Document representationCCS: Computing methodologies Knowledge representation and reasoningCCS: Information systems Multimedia information systemsCCS: Computing methodologies Natural language generation Shenao Chen Note: Both authors contributed equally to this research. email: 241080005@hdu.edu.cn Affiliation: Hangzhou Dianzi University , Hangzhou , China , Yidan Xu email: yidanxu2024@163.com Affiliation: Hangzhou Dianzi University , Hangzhou , China , Xiangmin Han email: hanxiangmin@bjut.edu.cn Affiliation: Beijing University of Technology , Beijing , China , Rundong Xue email: xuerundong2002@gmail.com Affiliation: Xi’an Jiaotong University , Xi’an , China , Duanpo Wu email: wuduanpo@hdu.edu.cn Affiliation: Hangzhou Dianzi University , Hangzhou , China , Yuhan Gao email: yuhangao@hdu.edu.cn Affiliation: Hangzhou Dianzi University , Hangzhou , China , Chenggang Yan email: cgyan@hdu.edu.cn Affiliation: Hangzhou Dianzi University , Hangzhou , China and Yue Gao email: gaoyue@tsinghua.edu.cn Affiliation: Tsinghua University , Beijing , China 2026; © c Abstract. Modern Multimodal Retrieval-Augmented Generation (M-RAG) systems are fundamentally limited by the binary connectivity paradigm of traditional simple graphs, which fails to capture the intricate, high-order correlations among heterogeneous entities—such as the N-ary relationships between a visual chart, its scattered textual descriptions, and underlying numerical data. Furthermore, existing refinement strategies often rely on exhaustive, full-page reconstruction to align cross-modal information, leading to prohibitive computational redundancy and the introduction of contextual noise in long-form document processing. In this paper, we propose Hyper-M2RAG, a novel framework that redefines multimodal document retrieval through High-order Hypergraph Representation Learning. We first formalize the document structure as a Multimodal Hypergraph, utilizing hyperedges as unified semantic containers to encapsulate multi-way associations across text, images, and tables, thereby transcending point-to-point modeling. To mitigate semantic fragmentation caused by physical pagination, we introduce an Anchor-driven Incremental Refinement mechanism. Rather than performing a global sweep, our approach surgically identifies boundary-crossing anchors—nodes and reconstructs their local hyper-topology using one-hop neighborhood contexts. This targeted refinement effectively bridges cross-page knowledge gaps with minimal computational footprints. Extensive evaluations on multimodal benchmarking datasets demonstrate that Hyper-M2RAG significantly outperforms state-of-the-art methods in both retrieval precision and generation coherence. Our code is accessible at https://github.com/ShenAoChen2001/MMHRAG. Keywords: Hypergraph, Multimodal RAG, Document Understanding, Cross-page Association, Incremental Refinement †c-license: by 1. Introduction Document Question Answering (Doc-QA) (19; 22) aims to generate accurate answers grounded in document content. With the rapid progress of Large Vision-Language Models (LVLMs), Doc-QA has evolved from text-only understanding to multimodal reasoning over long and structurally complex documents. In real-world scenarios such as academic papers, technical reports, and business documents, critical evidence is often distributed across text, figures, tables, formulas, and page layouts, and may span many pages. This setting requires not only long-context modeling, but also faithful preservation of cross-modal structure and document-level semantic dependencies. Retrieval-augmented generation (RAG) has become an effective paradigm for document understanding by retrieving relevant evidence before answer generation. Existing RAG frameworks, however, are still largely rooted in text-centric assumptions (23; 9). Even recent multimodal variants often remain limited in how they represent document structure. A common strategy is to linearize visual content into captions or OCR-derived text, and then perform retrieval over textual surrogates (26; 4). Although practical, this design inevitably weakens the original spatial organization of the document and discards part of the visual semantics embedded in figures, tables, and layouts. As a result, evidence that depends on visual context, layout proximity, or joint interpretation across modalities can be overlooked during retrieval. Beyond this general limitation, a more fundamental issue lies in the relational assumption inherited by many graph-based or knowledge-enhanced RAG systems. In text-centric settings, knowledge is commonly modeled through binary relations, typically represented as triples of the form (head, relation, tail). This abstraction is often adequate when semantic dependencies can be decomposed into pairwise links between entities or text spans. However, multimodal documents exhibit a different structural nature. Key evidence is frequently formed not by a single pairwise connection, but by the joint interaction among multiple heterogeneous elements, such as a figure region, its caption, a nearby paragraph, a table cell, and their shared layout context. In such cases, the meaning emerges from a group-wise association rather than from isolated binary edges. This mismatch makes conventional graph modeling inherently limited for multimodal RAG. When high-order multimodal dependencies are decomposed into a set of pairwise edges, the original evidence unit becomes fragmented. The system may preserve local connections, yet lose the higher-level semantic coherence that binds multiple elements into a single interpretable structure. For example, a textual claim may only be fully supported when a chart, its legend, and the surrounding explanatory paragraph are considered together; reducing this evidence to independent pairwise links weakens structural fidelity and makes downstream reasoning less reliable. Therefore, the challenge in multimodal RAG is not merely to add more modalities into existing retrieval pipelines, but to redesign the underlying relational structure from graph-based pairwise modeling to hypergraph-based high-order organization. Recent advances in document parsing pipelines (13), such as MinerU2.5 (17), DeepSeekOCR (21), and PaddleOCR (2), make this shift increasingly practical. These systems can extract high-fidelity multimodal elements from complex PDFs, including text blocks, figure regions, table structures, and layout coordinates, thereby providing a stronger foundation for structured multimodal indexing. This progress opens the door to a retrieval framework that operates directly on rich document elements and their structural relations, instead of relying on lossy textual abstractions alone. Motivated by this gap, we ask the following question: how can we extend RAG from graph-based pairwise retrieval to hypergraph-based high-order multimodal reasoning, while keeping refinement efficient for long documents? To answer this question, we identify two core challenges. Challenge 1: Pairwise graph modeling is insufficient for high-order multimodal evidence. Existing graph-based retrieval structures are designed for binary relations and therefore struggle to preserve group-wise dependencies among text, figures, tables, and layout context. This limitation leads to fragmented evidence representation and weakens the retriever’s ability to discover visually grounded or structurally coupled information. Challenge 2: Efficient refinement in long multimodal documents remains difficult. In long documents, relevant evidence and entity relations often span many pages. Existing refinement strategies frequently revisit entire pages or large contexts to recover missing cross-page connections (10), which introduces substantial computational redundancy, especially when multimodal tokens dominate the cost. We propose Hyper-M2RAG, a hypergraph-based framework for multimodal retrieval-augmented generation. The central idea is to move the structural foundation of RAG from graphs to hypergraphs. Hyper-M2RAG constructs a multimodal hypergraph in which vertices represent textual, visual, and tabular entities, while hyperedges connect multiple heterogeneous elements that jointly express a semantic unit. This design allows the retriever to preserve and exploit high-order multimodal dependencies in a unified topological space, rather than approximating them through decomposed pairwise links. As a result, retrieval becomes natively aware of textual semantics, visual evidence, and layout structure. To further support long-document reasoning, Hyper-M2RAG introduces an anchor-driven surgical refinement mechanism. Our key observation is that entities recurring across pages often signal incomplete or ambiguous structural associations. Instead of reprocessing entire pages, the system identifies these boundary-crossing entities as anchors and selectively reconstructs their local one-hop neighborhoods. By refining only the hyperedges around these anchors and merging the updates back into the global hypergraph, our method concentrates computation on the structurally uncertain regions that matter most. This makes refinement substantially more efficient while preserving document-level reasoning capacity. Our contributions are summarized as follows: • From graph to hypergraph for multimodal RAG. We show that the pairwise relational assumption underlying conventional graph-based RAG is insufficient for multimodal documents, and introduce a hypergraph formulation that naturally models high-order dependencies among text, figures, tables, and layout elements. • Native multimodal hypergraph indexing. We leverage visual and layout information during document parsing to extract structured entities and relationships. These extracted textual representations are then organized into a unified hypergraph, supporting retrieval over semantically enriched structural evidence. • Anchor-driven surgical refinement. We propose a localized refinement strategy that updates only the structurally ambiguous regions around cross-page anchors, avoiding exhaustive page-level reprocessing and improving efficiency in long-document settings. • Effective and efficient multimodal reasoning. Through extensive experiments, we show that Hyper-M2RAG improves multimodal retrieval and downstream answer quality while achieving a better efficiency profile on long and complex documents. The remainder of this paper details the proposed method. Section 4 formalizes Hyper-M2RAG and presents its multimodal hypergraph construction and anchor-driven refinement mechanism. Figure 1. Overview of the Hyper-M2RAG framework. The system progresses through three core stages: (1) Multi-modal Data Ingestion via PMUs; (2) Two-stage Hypergraph Construction featuring anchor-driven structural refinement; (3) Cognitive-inspired Hybrid Retrieval. The comparative case (bottom right) highlights our advantage in cross-page factual grounding over vanilla LLMs. 2. Related Work Traditional RAG typically relies on flattened chunk retrieval, which struggles to capture long-range dependencies. To bridge this gap, graph-enhanced approaches like GraphRAG (3) and LightRAG (8) extract entity-centric relations to improve multi-hop reasoning. However, these binary graph models are limited to pairwise associations. To model higher-order group semantics, Hypergraph-based RAG (HyperRAG) (5) introduces hyperedges to connect multiple vertices simultaneously. Recent variants like CogRAG(11) further explore cognitive-inspired hypergraph structures to enhance adaptive reasoning. While these methods excel in text-based high-order modeling, they remain largely "vision-blind" when faced with complex multi-modal layouts. Recent studies have begun extending RAG to multi-modal domains, encompassing cross-modal alignment, specialized document parsing, and multi-source retrieval strategies (27). Systems such as RAG-Anything (7), MegaRAG (10), and MHier-RAG (6) attempt to retrieve interleaved visual and textual components from complex layouts, while complementary approaches address query reformulation (14), heterogeneous source aggregation (16), multi-turn reasoning (12), and evidence fusion (25). However, existing methods suffer from retrieval-modality decoupling: they either treat images as secondary "post-retrieval supplements" or rely on lossy text-only indices (e.g., captions) that collapse the original structural layout. This leads to critical information decay and visual blind spots, especially when evidence spans across multiple pages. Unlike these approaches, our work pursues a native multi-modal indexing paradigm that preserves high-order structural and visual integrity within a unified hypergraph. 3. Task Description and Preliminaries The objective of multi-modal long-context Document-based Question Answering (Doc-QA) is to generate a comprehensive answer ansY_ans based on a query Q and a document D containing heterogeneous modalities. We formalize the generation process as: (1) Yans=LLMgen(yi∣y<i,Q,S),Y_ans=LLM_gen(y_i y_<i,Q,S), where S represents the Structured Evidence Package retrieved from the document-indexed corpus. In our Hyper-M2RAG framework, the document D is organized into a multi-modal hypergraph G=(V,E)G=(V,E), where V denotes a set of normalized semantic entities vertices and E=Elow∪Ehigh∪ErefineE=\E_low∪ E_high∪ E_refine\ represents the set of hyperedges capturing multi-order relationships. Specifically, ElowE_low and EhighE_high capture intra-page pairwise and group associations, respectively, while ErefineE_refine encapsulates latent cross-page dependencies reconstructed through anchor-driven refinement. The retrieval component is defined as a mapping function: (2) S=HyperRetriever(G,Q),S=HyperRetriever(G,Q), where HyperRetriever(⋅)HyperRetriever(·) executes a dual-path alignment across the symbolic hypergraph and the vector space. This strategy ensures that S integrates both vertex-anchored entity evidence CentC_ent and hyperedge-directed relational context CrelC_rel, facilitating complex reasoning over the high-order multi-modal substrate. 4. Methodology The core workflow of Hyper-M2RAG is illustrated in Fig. 1, highlighting the transition from fragmented multi-modal pages to a unified, high-order hypergraph index. 4.1. Multi-modal Data Ingestion To better preserve heterogeneous document structures, we redefine the primary unit of data ingestion for structured knowledge extraction. Unlike conventional RAG systems that treat documents as linear text streams, we adopt the Page-level Multi-modal Unit (PMU) as the fundamental atomic unit. Specifically, we utilize MinerU as a high-fidelity engine to parse raw PDFs into a structured sequence =p1,p2,…,pnD=\p_1,p_2,…,p_n\. The i-th unit pip_i is defined as: (3) pi=Tiocr,Iipage,ℱi,local,p_i=\T_i^ocr,I_i^page,F_i,local\, where TiocrT_i^ocr is the aggregated textual content, IipageI_i^page is the global page image, and ℱi,local=Ii,jfigj=1niF_i,local=\I_i,j^fig\_j=1^n_i represents localized image patches (e.g., figures, tables). This representation assigns distinct semantic roles to visual inputs: IipageI_i^page provides macro-visual context for layout awareness, while ℱi,localF_i,local offers fine-grained evidence from detected regions. To ensure robust ingestion, vision-dominant pages with sparse text are preserved via semantic placeholders (e.g., “Please see the Figures”). Furthermore, this unified schema treats visual fields as optional (∅ ), allowing the system to handle both text-only and multi-modal data through a consistent interface. 4.2. Multi-modal Hypergraph Index Construction The construction of the HyperIndex follows a hierarchical paradigm: Initial Page-level Extraction followed by Anchor-driven Structural Refinement. This process transforms fragmented page-level units into a cohesive, high-fidelity semantic network. 4.2.1. Initial Page-level Indexing and Aggregation For each PMU pi∈p_i , we first execute a multi-modal extraction operator LLMextLLM_ext to capture the localized entity set ViV_i and their corresponding high-order associations EiE_i. Unlike traditional text-based extraction, this process simultaneously digests the textual substrate and hierarchical visual evidence. The extraction is formalized as follows: (4) Vi=LLMext(Pent(pi))Ei,low=LLMext(Plow(pi,Vi))Ei,high=LLMext(Phigh(pi,Vi))for pi∈, \ aligned V_i&=LLM_ext(P_ent(p_i))\\ E_i,low&=LLM_ext(P_low(p_i,V_i))\\ E_i,high&=LLM_ext(P_high(p_i,V_i)) aligned . p_i , where PentP_ent, PlowP_low, and PhighP_high are specialized prompts (detailed in Appendix) for identifying vertices, pairwise relations, and high-order correlations, respectively. Each extracted element is tagged with a provenance attribute src=isrc=\i\ to anchor semantic entities to their original page index i. To synthesize a document-level structure, a Canonical Merging operator ℳM is applied to aggregate page-level sets. Vertices are normalized via case-insensitive alignment, while hyperedges are collapsed based on their unique, sorted constituent vertex sets: (5) init=ℳ(⋃i=1nVi,Ei,low,Ei,high),G_init=M ( _i=1^n\V_i,E_i,low,E_i,high\ ), where initG_init denotes the initial global hypergraph. In this stage, cross-page alignment is achieved through entity unification, where each global node or edge maintains an aggregated provenance index srcglobal=⋃srcisrc_global= _i. 4.2.2. Anchor-driven Structural Refinement While initG_init aggregates local evidence, it remains constrained by the page-level receptive field, often failing to capture long-range semantic dependencies that transcend individual page boundaries. To bridge these disparate fragments, we introduce a refinement stage centered on Cross-page Anchors VancV_anc. These anchors are identified as entities with high provenance frequency, representing consistent semantic pivots across the document: (6) Vanc=v∈V∣|src(v)|≥τ,V_anc=\v∈ V |src(v)|≥τ\, where τ is the frequency threshold. Instead of re-processing entire page sequences with high computational redundancy as in MegaRAG, our approach performs targeted refinement within the topological neighborhoods of VancV_anc. For each anchor v∈Vancv∈ V_anc, the system extracts its star-expansion subgraph Gsub(v)=(Vsub,Esub)G_sub(v)=(V_sub,E_sub) to encapsulate the multi-source contexts linked to this pivot through shared hyperedges: (7) Esub(v)=e∈(Elow∪Ehigh)∣v∈eVsub(v)=u∈V∣∃e∈Esub(v),u∈e casesE_sub(v)=\e∈(E_low∪ E_high) v∈ e\\\ V_sub(v)=\u∈ V ∃ e∈ E_sub(v),u∈ e\ cases where EsubE_sub represents the set of all hyperedges (both pairwise and high-order) incident to v, and VsubV_sub denotes the union of all vertices contained within these incident hyperedges. This localized sub-hypergraph GsubG_sub acts as a structural nexus, providing a condensed reasoning substrate for the refinement operator ℛR to bridge disparate page-level fragments and synthesize latent cross-page associations: (8) Erefine=LLMref(ℛ(Gsub(v),Pref)),E_refine=LLM_ref(R(G_sub(v),P_ref)), where PrefP_ref is a specialized prompt directing the model to synthesize novel associations or consolidate existing relations by reconciling multi-source evidence. Finally, the refined relations are integrated into the global multi-modal hypergraph finalG_final: (9) final=V,Elow,Ehigh,Erefine.G_final=\V,E_low,E_high,E_refine\. By shifting the refinement focus from raw page sequences to topological anchor neighborhoods, the system effectively eliminates the redundant processing of stable local information. This ensures a more precise discovery of high-order relational structures while maintaining significant computational efficiency. 4.3. Cognitive-inspired Hybrid Retrieval Guided by the hierarchical structure of the HyperIndex, we design a cognitive-inspired hybrid retrieval strategy to bridge user queries with high-fidelity evidence. As illustrated in our framework, the process initiates with a Query Decomposition stage, followed by a Dual-channel Retrieval mechanism that traverses both the symbolic hypergraph and the dense vector space. 4.3.1. Multi-modal Query Decomposition For a given user query q, the system first employs an MLLM-based decomposition operator to extract two levels of semantic anchors: (10) Klow,Khigh=LLMdec(Pdec(q)),\K_low,K_high\=LLM_dec(P_dec(q)), where KlowK_low represents fine-grained entity keywords (e.g., specific objects or technical terms), and KhighK_high denotes high-level thematic keywords (e.g., abstract concepts or cross-page relations). PdecP_dec is the decomposition prompt detailed in the Appendix. 4.3.2. Dual-channel Evidence Retrieval The retrieval process simultaneously executes two complementary paths to collect structured evidence from the HyperIndex: Path I: Vertex-anchored Entity Retrieval This path targets fine-grained evidence by anchoring the query to specific semantic vertices within the hypergraph. For each keyword k∈Klowk∈ K_low, the system identifies the most relevant vertices VqV_q via semantic similarity search in the entity vector space: (11) Vq=v∈V∣sim(ϕ(k),ϕ(v))>γ,V_q=\v∈ V (φ(k),φ(v))>γ\, where ϕ(⋅)φ(·) denotes the embedding function and γ is the similarity threshold. Using VqV_q as entry points, the entity-centric context CentC_ent is expanded via their incident hyperedges in the HypergraphDB: (12) Cent=e∈(Elow∪Ehigh∪Erefine)∣∃v∈Vq,v∈e.C_ent=\e∈(E_low∪ E_high∪ E_refine) ∃ v∈ V_q,v∈ e\. This operation captures the topological neighbors of the query entities as structural evidence, preserving the immediate relational context and leveraging refined cross-page associations. Path I: Hyperedge-directed Relational Retrieval To capture overarching themes and cross-page narratives, this path interfaces directly with the relational vector index using KhighK_high. For each thematic keyword k∈Khighk∈ K_high, the system retrieves a candidate set of high-order hyperedges EqE_q whose semantic embeddings align with the query intent: (13) Eq=e∈(Elow∪Ehigh∪Erefine)∣sim(ϕ(k),ϕ(e))>γ,E_q=\e∈(E_low∪ E_high∪ E_refine) (φ(k),φ(e))>γ\, where EqE_q specifically targets the refined associations that transcend individual page boundaries. The relation-centric context CrelC_rel is then synthesized by back-tracing the constituent vertices and their multi-source provenance: (14) Crel=src(e),nodes(e)∣e∈Eq,C_rel=\src(e),nodes(e) e∈ E_q\, where nodes(e)nodes(e) extracts the semantic entities within the hyperedge and src(e)src(e) retrieves the original multi-page text chunks. This path ensures the model reconciles broad conceptual queries with the high-order structural evidence consolidated during the refinement stage. 4.3.3. Evidence Fusion and Generation Finally, the system integrates the multi-source evidence from both retrieval channels into a unified, structured context S=Fuse(Cent,Crel)S=Fuse(C_ent,C_rel). Unlike conventional RAG systems that return fragmented raw text, S is a Structured Evidence Package comprising: (i) normalized entities, (i) high-order hyperedges, and (i) their synchronized multi-page source chunks. The final response R is synthesized as: (15) R=LLMgen(Pgen(q,S)),R=LLM_gen(P_gen(q,S)), where PgenP_gen is a generation prompt that instructs the model to reason over the provided topological structures and cross-page evidence. The efficacy of this dual-channel paradigm is fundamentally rooted in the structural depth of the HyperIndexHyperIndex. Specifically, the cross-page hyperedges ErefineE_refine generated during the anchor-driven refinement stage enable Path I to retrieve integrated evidence that transcends individual page boundaries. By reconciling these high-order associations with local entity-centric details, the framework provides a holistic and factually grounded understanding of complex, long-form documents. 5. Experiments In this section, we provide a comprehensive evaluation of Hyper-M2RAG. We first detail our experimental configurations and benchmark selections, followed by a quantitative analysis comparing our approach against state-of-the-art RAG baselines. 5.1. Datasets To evaluate Hyper-M2RAG’s proficiency in managing multi-modal document structures encompassing both holistic understanding and granular retrieval, we conduct experiments across two task granularities: Global QA and Local QA. 5.1.1. Global QA To evaluate the system’s ability to synthesize information across entire document collections (book-level), we employ both textual and multi-modal benchmarks: • Textual Corpus: We utilize the Mixed-Domain (≈ 0.62M tokens) from the UltraDomain (18) dataset and NeurologyCorp (≈ 1.97M tokens) from MedRAG (24). These corpora specifically test the model’s capacity for maintaining long-range semantic consistency within dense, specialized knowledge domains. • multi-modal Benchmark: To address the scarcity of standardized multi-modal global QA datasets, we utilize a specialized evaluation set characterized by dense, visually-rich content. This benchmark incorporates a World History textbook (788 pages) and a Sustainable Development Report (SDR) slide deck (491 pages), both of which feature a high concentration of interleaved graphical elements, including intricate charts, tables, and thematic illustrations that are deeply integrated with the primary text. 5.1.2. Global Question Generation Protocol To address the lack of human-labeled queries, we use an automated protocol with two modality-specific pipelines: • Textual Generation: Using the document outline as a scaffold, we prompt an LLM to simulate 5 distinct professional users. For each user, we define 5 strategic tasks, each generating 5 complex questions that require a holistic understanding of the document. This results in 125 global questions per dataset. • multi-modal Generation: For vision-heavy documents, we establish a generation pipeline. We sample 25 representative pages per dataset (organized into 5 batches of 5 pages). Each batch involves a specific user persona and 5 tasks. Crucially, we enforce a multi-modal-dependency constraint: questions must be unanswerable by text alone, requiring reasoning over visual data such as map coordinates, chart trends, or diagram structures. This ensures 125 high-quality multi-modal questions per dataset. 5.1.3. Local QA To evaluate retrieval precision at the granular level (page- or slide-level), we utilize the RealMMBench (20) benchmark. RealMMBench is specifically designed to assess multi-modal RAG systems under stressed conditions, including visual-rich layouts, table-heavy content, and sophisticated query rephrasing. While RealMMBench spans multiple domains, we perform a strategic evaluation on the TechReport (1,674 pages) and TechSlides (1,963 pages) sub-datasets. These technical domains are characterized by dense high-order structural dependencies, making them ideal for validating our high-order hypergraph construction and anchor-driven refinement mechanisms. This selection ensures a rigorous stress test of document intelligence while maintaining computational feasibility during the evaluation process. 5.2. Baselines and Evaluation Metrics We compare Hyper-M2RAG against several state-of-the-art baselines: GraphRAG (3), a widely adopted knowledge graph framework; HyperRAG (5), which utilizes text-only hypergraphs; and MegaRAG (10), a multi-modal graph-based RAG system. To ensure a fair assessment, we evaluate all methods across both textual and multi-modal benchmarks to test their core retrieval and reasoning capabilities. 5.2.1. Global QA Metrics Given the absence of ground-truth answers for book-level queries, prior works often rely on LLM-based evaluation. However, such evaluations are frequently prone to position bias, where the evaluator model favors responses presented earlier in the prompt. To mitigate this and ensure reliability, we implement a Dual-Response Swap & Average strategy: • Dual-Prompt Evaluation: For every pair of responses (A,B)(A,B), we generate two distinct evaluation prompts: a forward prompt (A followed by B) and a reversed prompt (B followed by A). • Result Normalization: To maintain a consistent reference frame, results from the reversed evaluation are flipped (e.g., if B was preferred in the reversed prompt, it is mapped to a preference for the second position in the forward frame). • Consistency Arbitration: We compare the forward and normalized reversed results. If the evaluator remains consistent across both prompts, the result is recorded. If the two evaluations conflict, the pair is marked as a Tie to filter out stochastic noise and ensure robust win-rate calculation. Responses are assessed across four qualitative dimensions following (8): (1) Comprehensiveness: Coverage of all query facets; (2) Diversity: Richness of perspectives; (3) Empowerment: Support for user understanding; and (4) Overall: An aggregate measure of the preceding criteria. Table 1. Win-rate analysis comparing Hyper-M2RAG against GraphRAG, HyperRAG, and MegaRAG across four distinct domains. Results are presented as percentages (%). Bold values indicate the winning method (excluding ties). Metrics Mix Neurology World History Sustainable Report GraphRAG Ours Tie GraphRAG Ours Tie GraphRAG Ours Tie GraphRAG Ours Tie Comprehensiveness 0.0 46.4 53.6 0.8 37.6 61.6 0.8 47.2 52.0 0.0 62.4 37.6 Diversity 11.2 60.8 28.0 9.6 56.0 34.4 4.8 69.6 25.6 3.2 91.2 5.6 Empowerment 4.0 56.0 40.0 3.2 61.6 35.2 5.6 66.4 28.0 1.6 77.6 20.8 Overall 2.4 51.2 46.4 2.4 52.0 45.6 4.8 61.6 33.6 0.8 77.6 21.6 HyperRAG Ours Tie HyperRAG Ours Tie HyperRAG Ours Tie HyperRAG Ours Tie Comprehensiveness 0.0 8.8 91.2 6.4 17.6 76.0 0.0 44.0 56.0 1.6 29.6 68.8 Diversity 23.2 35.2 41.6 3.2 18.4 78.4 15.2 54.4 30.4 17.6 43.2 39.2 Empowerment 15.2 28.8 56.0 14.4 28.8 56.8 16.8 46.4 36.8 20.8 26.4 52.8 Overall 13.6 27.2 59.2 11.2 22.4 66.4 14.4 44.8 40.8 11.2 25.6 63.2 MegaRAG Ours Tie MegaRAG Ours Tie MegaRAG Ours Tie MegaRAG Ours Tie Comprehensiveness 4.8 53.6 41.6 0.0 20.0 80.0 0.0 44.0 56.0 0.0 64.0 36.0 Diversity 8.8 44.8 46.4 6.4 52.8 40.8 2.4 75.2 22.4 1.6 90.4 8.0 Empowerment 3.2 63.2 33.6 0.8 44.8 54.4 3.2 62.4 34.4 1.6 90.4 8.0 Overall 5.6 58.4 36.0 0.0 36.8 63.2 1.6 59.2 39.2 1.6 88.8 9.6 5.2.2. Local QA Metrics For granular (page- or slide-level) QA, performance is measured by semantic alignment with ground-truth reference answers. We utilize a powerful LLM as an automated judge to determine semantic consistency between the generated and reference answers, reporting Accuracy as the primary metric. 5.3. Implementation Details To ensure consistency across all evaluated RAG frameworks, we standardize the backbone models and processing pipeline. For response generation, we utilize Qwen3-VL-8B (1), while DeepSeek-v3 (15) is employed for synthetic question generation and evaluation due to its superior reasoning robustness. All methods share a unified embedding space facilitated by gme-Qwen2-VL-2B-Instruct (28), which supports single-, cross-, and fused-modality retrieval tasks. Textual corpora are partitioned into 1,200-token chunks with a 100-token overlap. For multi-modal documents, we leverage the MinerU2.5 toolkit (17) to extract text, figures, and tables. MinerU’s ability to preserve complex layouts and symbols in machine-readable formats is particularly effective for the technical documents used in our study. 5.4. Global QA Performance The primary results in Table 1 compare Hyper-M2RAG against three state-of-the-art baselines across four domains: two purely textual corpora (Mix and Neurology) and two multimodal datasets (World History and Sustainable Report). While our method maintains a consistent lead in textual tasks, it achieves a decisive, overwhelming victory in multimodal scenarios. Our analysis yields the following key insights: Superiority across Diverse Domains Hyper-M2RAG consistently outperforms GraphRAG, HyperRAG, and MegaRAG across all qualitative dimensions. Notably, in the Sustainable Report domain characterized by high-density multimodal content and complex layouts, our method achieves a dominant Overall win rate of 88.8% against MegaRAG and 77.6% against GraphRAG. These results demonstrate that our high-order hypergraph structure captures multi-scale semantic dependencies in technical documents more effectively than traditional binary graphs. Gains in Diversity and Empowerment Hyper-M2RAG demonstrates a substantial lead in Diversity and Empowerment metrics. For instance, in the World History domain, our method achieves a 75.2% win rate in Diversity against MegaRAG. This performance gain is driven by our hyper-relational modeling mechanism, which transcends the limitations of binary-edge retrieval. Unlike traditional graphs that only link pairwise entities, our high-order hyperedges group multi-modal entities into unified semantic clusters. This allows the retriever to capture non-local, multi-hop connections across disparate document sections in a single traversal. Robustness against Hypergraph Baselines Compared to the text-only HyperRAG, Hyper-M2RAG maintains a consistent margin of improvement, particularly in Overall win rates ranging from 22.4% to 44.8%. While HyperRAG introduces basic hyper-structures, our anchor-driven refinement and multimodal integration ensure that the constructed hypergraphs are semantically precise rather than merely dense. The high tie rates in Comprehensiveness (e.g., 91.2% in Mix) indicate that while most hypergraph methods achieve basic content coverage, Hyper-M2RAG excels in the qualitative depth and organizational clarity of the generated responses. Multimodal Advantage In multimodal datasets like World History and Sustainable Report, the performance gap between Hyper-M2RAG and baselines widens significantly. This advantage stems from our layout-aware hyperedges, which directly link visual elements to their corresponding textual descriptions within a unified topological space. For instance, the 90.4% Empowerment score in the Sustainable domain demonstrates the model’s superior ability to interpret and synthesize information from complex tables and figures. Table 2. Local QA performance (Accuracy %) on RealMMBench. Method TechReport TechSlides GraphRAG 30.0 57.0 HyperRAG 56.0 67.0 MegaRAG 68.0 69.0 Hyper-M2RAG (Ours) 70.0 78.0 5.5. Local QA Performance Table 2 reports the accuracy of different methods on the TechReport and TechSlides sub-datasets of RealMMBench. Hyper-M2RAG achieves superior performance, reaching 70.0% and 78.0% accuracy, respectively. Compared with the state-of-the-art baseline MegaRAG, our method achieves a notable 9 percentage point improvement on TechSlides. This specific gain highlights the efficacy of our framework in visual-heavy contexts. Technical slides often contain fragmented information across different visual blocks; while MegaRAG relies on dense graph connectivity, our high-order hypergraph structure more effectively clusters spatially disparate but semantically related elements (e.g., a figure and its corresponding bullet points), leading to more precise local retrieval. 5.6. Ablation Study To investigate the contribution of different components in Hyper-M2RAG, we progressively introduce multi-modal extraction and anchor-driven refinement based on the HyperRAG baseline, as shown in Table 3. Our analysis yields two primary insights: Effect of Multi-modal Hypergraph Construction Adding multi-modal extraction (+ M Extraction) consistently improves performance over the text-only HyperRAG baseline on both TechReport and TechSlides datasets. This demonstrates that incorporating visual elements and layout information helps preserve cross-modal semantic dependencies, enabling more complete evidence retrieval from complex documents. Effect of Anchor-driven Refinement Introducing anchor-driven refinement further improves the performance over the multi-modal hypergraph variant. This verifies that selectively reconstructing local neighborhoods around cross-page anchors effectively captures long-range dependencies while avoiding unnecessary processing of irrelevant regions. Overall, the ablation results confirm that both multi-modal representation and anchor-driven refinement are essential components of Hyper-M2RAG, with their combination enabling more effective multimodal document understanding. Table 3. Ablation study of Hyper-M2RAG components. Variant TechReport TechSlides HyperRAG (Baseline) 56.0 67.0 + M Extraction 64.0 74.0 + Anchor Refinement 60.0 69.0 Hyper-M2RAG (Full) 70.0 78.0 Table 4. Comparison of structural hyperedge counts across datasets. Variant TechReport (1,674 pages) TechSlides (1,963 pages) Low High Total Low High Total HyperRAG (Baseline) 12,957 2,579 15,536 8,543 1,716 10,259 HyperRAG + M 10,016 2,078 12,094 9,309 2,004 11,313 Hyper-M2RAG (Ours) 14,221 8,257 22,478 12,522 6,571 19,093 5.7. Structural Complexity Analysis To investigate the structural characteristics of Hyper-M2RAG, Table 4 compares the hypergraph statistics of different methods, with particular focus on high-order relations (degree >2>2). High-Order Relational Modeling Hyper-M2RAG constructs substantially more high-order hyperedges than existing approaches. On TechReport, Hyper-M2RAG builds 8,257 high-order hyperedges, compared with 2,579 from HyperRAG, representing a 220% increase. Similarly, on TechSlides, the number of high-order hyperedges increases from 1,716 to 6,571. Since high-order hyperedges explicitly represent dependencies among multiple entities, this increase indicates that Hyper-M2RAG provides a richer structural representation beyond conventional pairwise relations. Effect of Multi-modal Hypergraph Construction Interestingly, simply incorporating multi-modal information does not necessarily lead to more high-order structures. The HyperRAG+M variant produces fewer high-order hyperedges than HyperRAG on TechReport (2,078 vs. 2,579). This suggests that modality augmentation alone is insufficient to establish effective high-order relations. In contrast, Hyper-M2RAG jointly models textual, visual, and layout elements through hypergraph construction and anchor-driven refinement, resulting in substantially richer high-order relational structures. Overall Structural Connectivity Beyond high-order relations, Hyper-M2RAG also achieves the largest number of total hyperedges across both datasets, with 22,478 edges on TechReport and 19,093 edges on TechSlides. This demonstrates that our framework constructs a more densely connected document representation, providing a stronger structural foundation for retrieving evidence distributed across long and complex documents. Table 5. Multi-Judge Audit (%). H-RAG = HyperRAG. Sustainable Report World History Evaluator H-RAG Ours Tie H-RAG Ours Tie Kimi-K2.6 25.6 55.2 19.2 24.0 44.0 32.0 GLM-5.1 25.6 51.2 23.2 24.8 38.4 36.8 DeepSeek-V4 Pro 15.2 41.6 43.2 20.0 36.0 44.0 5.8. Cross-Model Judge Audit Table 5 presents a multi-judge evaluation on the Sustainable Report and World History corpora, conducted with three recently released, architecturally diverse LLMs: Kimi-K2.6, GLM-5.1, and DeepSeek-V4 Pro. We select HyperRAG as the primary baseline for this audit because it is the most structurally aligned counterpart to Hyper-M2RAG: both methods organize knowledge using hypergraphs, while only Hyper-M2RAG introduces multimodal anchors and star-expansion refinement. This design isolates the contribution of our multimodal mechanism from general hypergraph-induced gains. Across both domains and all three evaluators, Hyper-M2RAG consistently outperforms HyperRAG, with win rates ranging from 36.0% to 55.2%. Crucially, the preference for Hyper-M2RAG is not judge-specific: the same ranking trend holds for all model families, indicating that the observed improvements are not a single-judge artifact but a robust, cross-model phenomenon. Table 6. Efficiency on World History (per 100 pages). Metric MegaRAG HyperRAG + Page Ours Build & Refine Time (min) 16 18 12 Token Consumption (10310^3) 2,742.2 3,376.1 765.9 5.9. Reduction in Token Cost and Latency Table 6 provides a head-to-head efficiency comparison on the World History corpus (per 100 pages). Hyper-M2RAG reduces build and refinement time to 12 min and lowers token consumption by 72% relative to MegaRAG and 77% relative to HyperRAG+Page. This gain stems from a fundamental shift in refinement granularity. Rather than performing document- or page-level rescans during incremental updates, Hyper-M2RAG organizes multimodal evidence into a hypergraph, where anchors serve as structural pivots linking text, tables, figures, and layout elements. Refinement is then confined to anchor-centered star neighborhoods: only hyperedges incident to updated anchors are recomputed. Consequently, the update process is transformed from a global sweep into a topology-guided local operation. This design eliminates redundant computation across unrelated content while preserving rich cross-modal linkages. Because the number of anchors requiring updates grows more slowly than document length, the refinement cost scales sub-linearly, allowing Hyper-M2RAG to maintain high efficiency on long, multimodal documents without altering the input scope or representation. 6. Conclusion In this paper, we presented Hyper-M2RAG, a novel multi-modal retrieval-augmented generation framework designed for complex, high-density technical documents. By transitioning from traditional binary graphs to a high-order hypergraph structure, our approach effectively captures the multi-scale semantic dependencies that are often fragmented across disparate document sections and modalities. Our core contribution lies in the integration of layout-aware hyperedge construction and an anchor-driven refinement mechanism. This synergy ensures that visual elements, such as tables and figures, are not merely treated as isolated inputs but are topologically aligned with their textual context within a unified semantic space. Furthermore, our dual-path retrieval strategy—comprising Vertex-anchored Entity Retrieval and Hyperedge-directed Relational Retrieval—enables the model to reconcile fine-grained evidence with overarching thematic narratives. Experimental results across four diverse domains demonstrate that Hyper-M2RAG consistently outperforms state-of-the-art baselines, particularly in multi-modal scenarios where it achieves significant gains in Diversity and Empowerment. 7. Acknowledgments This work was supported by Brain Science and Brain-like Intelligence Technology—National Science and Technology Major Project (2025ZD0217300), the National Key Research and Development Program of China under Grant (2023YFB4502803), the National Natural Science Foundation of China (No. U25A20532) and the Beijing Natural Science Foundation under Grant (No. L242167). References Bai et al. (2025) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §5.3. Cui et al. (2025) C. Cui, T. Sun, M. Lin, T. Gao, Y. Zhang, J. Liu, X. Wang, Z. Zhang, C. Zhou, H. Liu, et al. Paddleocr 3.0 technical report. arXiv preprint arXiv:2507.05595. Cited by: §1. Edge et al. (2024) D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson From local to global: a graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130. Cited by: §2, §5.2. Faysse et al. (2025) M. Faysse, H. Sibille, T. Wu, B. Omrani, G. Viaud, C. Hudelot, and P. Colombo Colpali: efficient document retrieval with vision language models. In International Conference on Learning Representations, Vol. 2025, p. 61424–61449. Cited by: §1. Feng et al. (2026) Y. Feng, H. Hu, X. Hou, S. Liu, S. Ying, S. Du, H. Hu, and Y. Gao Hyper-rag: combating llm hallucinations using hypergraph-driven retrieval-augmented generation. Nature Communications 17, p. 5778. External Links: Document, Link Cited by: §2, §5.2. Gong et al. (2025) Z. Gong, C. Mai, and Y. Huang MHier-rag: multi-modal rag for visual-rich document question-answering via hierarchical and multi-granularity reasoning. arXiv preprint arXiv:2508.00579. Cited by: §2. Guo et al. (2025a) Z. Guo, X. Ren, L. Xu, J. Zhang, and C. Huang Rag-anything: all-in-one rag framework. arXiv preprint arXiv:2510.12323. Cited by: §2. Guo et al. (2025b) Z. Guo, L. Xia, Y. Yu, T. Ao, and C. Huang LightRAG: simple and fast retrieval-augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, p. 10746–10761. External Links: Link, Document, ISBN 979-8-89176-335-7 Cited by: §2, §5.2.1. Hindi et al. (2025) M. Hindi, L. Mohammed, O. Maaz, and A. Alwarafy Enhancing the precision and interpretability of retrieval-augmented generation (rag) in legal technology: a survey. IEEE Access. Cited by: §1. Hsiao et al. (2026) C. Hsiao, Y. Wang, T. Lin, Y. Yeh, and C. Chen MegaRAG: multimodal knowledge graph-based retrieval augmented generation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 48031–48059. Cited by: §1, §2, §5.2. Hu et al. (2026) H. Hu, Y. Feng, R. Li, R. Xue, X. Hou, Z. Tian, Y. Gao, and S. Du Cog-rag: cognitive-inspired dual-hypergraph with theme alignment retrieval-augmented generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 31032–31040. Cited by: §2. Joshi et al. (2024) P. Joshi, A. Gupta, P. Kumar, and M. Sisodia Robust multi model rag pipeline for documents containing text, table & images. In 2024 3rd International Conference on Applied Artificial Intelligence and Computing (ICAAIC), p. 993–999. Cited by: §2. Ke et al. (2025) W. Ke, Y. Zheng, Y. Li, H. Xu, D. Nie, P. Wang, and Y. He Large language models in document intelligence: a comprehensive survey, recent advances, challenges, and future trends. ACM Transactions on Information Systems 44 (1), p. 1–64. Cited by: §1. Li et al. (2024) Z. Li, J. Wang, Z. Jiang, H. Mao, Z. Chen, J. Du, Y. Zhang, F. Zhang, D. Zhang, and Y. Liu Dmqr-rag: diverse multi-query rewriting for rag. arXiv preprint arXiv:2411.13154. Cited by: §2. Liu et al. (2024) A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. Cited by: §5.3. Liu et al. (2025) P. Liu, X. Liu, R. Yao, J. Liu, S. Meng, D. Wang, and J. Ma Hm-rag: hierarchical multi-agent multimodal retrieval augmented generation. In Proceedings of the 33rd ACM international conference on multimedia, p. 2781–2790. Cited by: §2. Niu et al. (2026) J. Niu, Z. Liu, Z. Gu, B. Wang, L. Ouyang, Z. Zhao, T. Chu, T. He, F. Wu, Q. Zhang, et al. Mineru2.5: a decoupled vision-language model for efficient high-resolution document parsing. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), San Diego, California, USA, p. 13–42. External Links: Link, Document, ISBN 979-8-89176-394-4 Cited by: §1, §5.3. Qian et al. (2025) H. Qian, Z. Liu, P. Zhang, K. Mao, D. Lian, Z. Dou, and T. Huang MemoRAG: boosting long context processing with global memory-enhanced retrieval augmentation. In Proceedings of the ACM Web Conference 2025 (TheWebConf 2025), Sydney, Australia. External Links: Link Cited by: 1st item. Wang et al. (2024) M. Wang, L. Chen, F. Cheng, S. Liao, X. Zhang, B. Wu, H. Yu, N. Xu, L. Zhang, R. Luo, et al. Leave no document behind: benchmarking long-context llms with extended multi-doc qa. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 5627–5646. Cited by: §1. Wasserman et al. (2025) N. Wasserman, R. Pony, O. Naparstek, A. R. Goldfarb, E. Schwartz, U. Barzelay, and L. Karlinsky Real-m-rag: a real-world multi-modal retrieval benchmark. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 31660–31683. Cited by: §5.1.3. Wei et al. (2026) H. Wei, Y. Sun, and Y. Li DeepSeek-ocr 2: visual causal flow. arXiv preprint arXiv:2601.20552. Cited by: §1. Wu et al. (2025) J. Wu, Y. Xia, T. Yu, X. Chen, S. S. Harsha, A. V. Maharaj, R. Zhang, V. Bursztyn, S. Kim, R. A. Rossi, et al. Doc-react: multi-page heterogeneous document question-answering. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p. 67–78. Cited by: §1. Wu et al. (2026) S. Wu, Y. Xiong, Y. Cui, H. Wu, C. Chen, Y. Yuan, L. Huang, X. Liu, T. Kuo, N. Guan, et al. Retrieval-augmented generation for natural language processing: a survey. Artificial Intelligence Review. External Links: ISSN 1573-7462, Document, Link Cited by: §1. Xiong et al. (2024) G. Xiong, Q. Jin, Z. Lu, and A. Zhang Benchmarking retrieval-augmented generation for medicine. In Findings of the Association for Computational Linguistics: ACL 2024, p. 6233–6251. Cited by: 1st item. Yang et al. (2024) D. Yang, J. Rao, K. Chen, X. Guo, Y. Zhang, J. Yang, and Y. Zhang Im-rag: multi-round retrieval-augmented generation through learning inner monologues. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 730–740. Cited by: §2. Yu et al. (2025) S. Yu, C. Tang, B. Xu, J. Cui, J. Ran, Y. Yan, Z. Liu, S. Wang, X. Han, Z. Liu, et al. Visrag: vision-based retrieval-augmented generation on multi-modality documents. In International Conference on Learning Representations, Vol. 2025, p. 21074–21098. Cited by: §1. Zhang et al. (2025) R. Zhang, C. Liu, Y. Su, R. Li, X. Huang, X. Li, and P. S. Yu A comprehensive survey on multimodal rag: all combinations of modalities as input and output. Authorea Preprints. Cited by: §2. Zhang et al. (2024) X. Zhang, Y. Zhang, W. Xie, M. Li, Z. Dai, D. Long, P. Xie, M. Zhang, W. Li, and M. Zhang GME: improving universal multimodal retrieval by multimodal llms. arXiv preprint arXiv:2412.16855. Cited by: §5.3.