Paper deep dive

HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document

Xin He, Yili Wang, Wenqi Fan, Qing Li, Qinggang Zhang, Yi Chang, Xin Wang

Year: 2026Venue: arXiv preprintArea: cs.IRType: PreprintEmbeddings: 140

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 87%

Last extracted: 8/1/2026, 10:35:52 AM

Summary

The paper introduces HVM-GraphRAG, a framework for complex document question answering that addresses limitations in existing Multimodal GraphRAG methods, specifically unreliable cross-modal evidence indexing and expensive graph traversal. HVM-GraphRAG utilizes a holistic view to guide graph construction, creating a compact concept-level graph with reliable indices to supporting multimodal chunks. This approach reduces noisy updates and allows for efficient retrieval by searching the compact graph and directly accessing evidence, outperforming baselines in accuracy and efficiency.

Entities (12)

HVM-GraphRAG · method · 98%Graph-Guided Holistic Retrieval Module · component · 92%Holistic-View-Guided Graph Construction Module · component · 92%Multimodal GraphRAG · concept · 90%Concept Graph · data-structure · 88%Cross-Modal Holistic View · mechanism · 85%Document Tree · data-structure · 85%BM25 · baseline-method · 80%GraphRAG · baseline-method · 80%HippoRAG · baseline-method · 80%QASPER · dataset · 80%RAPTOR · baseline-method · 80%

Relation Signals (10)

HVM-GraphRAG → uses → Holistic-View-Guided Graph Construction Module

confidence 95% · HVM-GraphRAG consists of two main modules: ❶ The Holistic-View-Guided Graph Construction Module (HGCM)

HVM-GraphRAG → uses → Graph-Guided Holistic Retrieval Module

confidence 95% · ❷ The Graph-Guided Holistic Retrieval Module (GHRM) retrieves query-relevant concept nodes

HVM-GraphRAG → addresses → Unreliable Cross-Modal Evidence Indexing

confidence 90% · To address these issues, we propose HVM-GraphRAG... HVM-GraphRAG uses a holistic view to guide graph construction, thereby reducing noisy and conflicting graph updates

HVM-GraphRAG → addresses → Expensive Graph Traversal

confidence 90% · existing methods often suffer from unreliable cross-modal evidence indexing and expensive graph traversal. To address these issues, we propose HVM-GraphRAG

Holistic-View-Guided Graph Construction Module → constructs → Concept Graph

confidence 90% · HVM-GraphRAG constructs a concept-level indexing graph through three designs

Graph-Guided Holistic Retrieval Module → searches → Concept Graph

confidence 90% · During online retrieval... HVM-GraphRAG searches over a compact concept-level graph

HVM-GraphRAG → outperforms → BM25

confidence 85% · HVM-GraphRAG achieves the best answer performance... while substantially improving online retrieval efficiency over representative graph-based baselines.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Question answering (QA) over complex documents requires models to retrieve and integrate evidence distributed across distant document regions and modalities. Multimodal GraphRAG provides a promising direction by organizing document evidence with graph structures. However, existing methods often suffer from unreliable cross-modal evidence indexing and expensive graph traversal. To address these issues, we propose HVM-GraphRAG, a holistic-view multimodal GraphRAG framework on complex document. HVM-GraphRAG uses a holistic view to guide graph construction, thereby reducing noisy and conflicting graph updates and building reliable indices between concept-level graph nodes and supporting multimodal chunks. During retrieval, HVM-GraphRAG searches over a compact concept-level graph and directly accesses supporting evidence through the constructed index, avoiding costly traversal over dense entity-level graphs. After obtaining the retrieved evidence, HVM-GraphRAG further reorganizes chunks into modality-specific groups, enabling the answering model to better integrate heterogeneous evidence. Experiments on three datasets show that HVM-GraphRAG achieves the best answer performance in most evaluated settings while substantially improving online retrieval efficiency over representative graph-based baselines.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

138,802 characters extracted from source content.

Expand or collapse full text

HVM-GraphRAG: Holistic-View Multimodal Graph Retrieval-Augmented Generation on Complex Document Xin He Jilin UniversityChangchunChina hex23@mails.jlu.edu.cn , Yili Wang Jilin UniversityChangchunChina wangyili@jlu.edu.cn , Wenqi Fan The Hong Kong Polytechnic UniversityHong Kong SARChina wenqifan03@gmail.com , Qing Li The Hong Kong Polytechnic UniversityHong Kong SARChina csqli@comp.polyu.edu.hk , Qinggang Zhang Jilin UniversityChangchunChina qinggangzhang@jlu.edu.cn , Yi Chang Jilin UniversityChangchunChina yichang@jlu.edu.cn and Xin Wang Jilin UniversityChangchunChina xinwang@jlu.edu.cn (2018) Abstract. Question answering (QA) over complex documents requires models to retrieve and integrate evidence distributed across distant document regions and modalities. Multimodal GraphRAG provides a promising direction by organizing document evidence with graph structures. However, existing methods often suffer from unreliable cross-modal evidence indexing and expensive graph traversal. To address these issues, we propose HVM-GraphRAG, a holistic-view multimodal GraphRAG framework on complex document. HVM-GraphRAG uses a holistic view to guide graph construction, thereby reducing noisy and conflicting graph updates and building reliable indices between concept-level graph nodes and supporting multimodal chunks. During retrieval, HVM-GraphRAG searches over a compact concept-level graph and directly accesses supporting evidence through the constructed index, avoiding costly traversal over dense entity-level graphs. After obtaining the retrieved evidence, HVM-GraphRAG further reorganizes chunks into modality-specific groups, enabling the answering model to better integrate heterogeneous evidence. Experiments on three datasets show that HVM-GraphRAG achieves the best answer performance in most evaluated settings while substantially improving online retrieval efficiency over representative graph-based baselines. †copyright: none†copyright: acmlicensed†journalyear: 2018†doi: X.X†conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY†isbn: 978-1-4503-X-X/2018/06 1. Introduction Complex document question answering (QA) (Deng et al., 2025; Yu et al., 2025a) is a fundamental task for accessing and synthesizing knowledge in real-world information systems. Its applications span scientific literature analysis (Zheng et al., 2025; Tang et al., 2026), financial report analysis (Li et al., 2025; Choi et al., 2025), legal document understanding (Zhang et al., 2025c; Sadowski and Chudziak, 2025; Barron et al., 2025), and technical manual assistance (Zhang et al., 2025a; Bai et al., 2025). Unlike short-passage QA (Wu et al., 2026; Chen et al., 2026), complex document QA requires integrating evidence distributed across distant document sections and heterogeneous modalities, including text, tables and images. Multimodal Retrieval-Augmented Generation (Multimodal RAG) provides a general paradigm for this task (Riedler and Langer, 2024; Xia et al., 2025; Chen et al., 2022): it retrieves question-relevant evidence from multimodal document content and then generates answers grounded in the retrieved evidence. However, most existing Multimodal RAG methods rely on flat chunk retrieval (Fig.1(a)), which treats multimodal chunks as isolated evidence units (Robertson and Zaragoza, 2009; Quinn et al., 2025). Such isolated evidence modeling often fails to capture document-level structures, cross-modal correspondences, and semantic relations among evidence units (Wu et al., 2026; Yang et al., 2025b). Figure 1. Flat Multimodal RAG and Multimodal GraphRAG. (a) Flat Multimodal RAG loses document hierarchy and cross-modal dependencies by retrieving isolated chunks. (b) Multimodal GraphRAG introduces structural knowledge, but may suffer from unresolved conflicts and costly graph traversal. Recent Multimodal RAG studies (Bu et al., 2025; Hsiao et al., 2026; Yuan et al., 2025; Wang et al., 2025c) have therefore explored graph structures for organizing multimodal evidence with explicit nodes and edges. In this paradigm, the graph provides a structured organization layer for modeling relations among multimodal information, whether the information is represented as extracted knowledge or retained as source evidence. Existing Multimodal GraphRAG methods can be broadly grouped into two categories according to whether the graph primarily represents extracted knowledge or indexes source evidence (Peng et al., 2025; Zhang et al., 2025e). The first category is Knowledge-based Multimodal GraphRAG (Wan and Yu, 2025; Dai et al., 2026; Yu et al., 2026b), which constructs or leverages multimodal knowledge graphs by extracting entities, attributes, and relations from multimodal content. The second category is Index-based Multimodal GraphRAG (Wang et al., 2025c; Wu et al., 2025b; Liu et al., 2025b), in which graphs serve as indexing structures over original multimodal evidence units. Their topology guides evidence localization, expansion, and aggregation. Overall, graph structures shift Multimodal RAG from independent chunk matching to relation-aware evidence modeling, which is crucial when answers require multi-hop evidence aggregation across pages, regions, and modalities. Despite these structural advantages, existing graph-based RAG methods (Sarthi et al., 2024; Edge et al., ; Gutiérrez et al., 2024) do not consistently outperform the flat retrieval method in complex document QA (Fig. 2). Their practical benefits are often limited by two challenges introduced during graph construction and retrieval. The first challenge is Unreliable Cross-Modal Evidence Indexing (Wu et al., 2026). Existing methods often extract knowledge from multimodal chunks independently and insert it into the graph without a holistic view of the document-wide graph state (Fig. 1(b)). Consequently, locally extracted concepts, entities, and relations are not systematically checked against existing graph elements or evidence from other modalities. This may introduce noisy, redundant, or conflicting graph structures and unreliable graph-to-chunk indices, ultimately leading to suboptimal question-answering performance. The second challenge is Overly Expensive Graph Traversal (Hu et al., 2026; Li et al., 2026). As complex documents contain many pages, regions, and entities, the constructed graph can become large and dense. Retrieving evidence from such a graph often requires costly node matching, relation traversal, neighborhood expansion, and evidence aggregation (Fig.1(b)). This substantially increases inference latency and reduces retrieval efficiency, especially when repeated graph expansion is required for each question. Figure 2. Comparison of answer accuracy and total online query time on Qasper. To address these challenges, we propose HVM-GraphRAG, a Holistic-View Multimodal GraphRAG framework that uses holistic context to construct a cleaner concept graph, improving evidence-indexing reliability and retrieval efficiency for question answering over complex documents. Specifically, HVM-GraphRAG consists of two main modules: ❶ The Holistic-View-Guided Graph Construction Module (HGCM) incorporates document-level structural and semantic context into graph construction, building a holistically coherent concept graph with reliable evidence indexing. ❷ The Graph-Guided Holistic Retrieval Module (GHRM) retrieves query-relevant concept nodes from the compact concept graph and expands them to supporting multimodal evidence through the evidence index, reducing traversal cost for answer generation. Together, these modules construct a cleaner concept graph and perform retrieval over a more compact concept space, thereby improving answer reliability and inference efficiency. The contributions of this work are summarized as follows: • We identify key limitations of existing Multimodal GraphRAG methods on complex document: Unreliable Cross-Modal Evidence Indexing, and Overly Expensive Graph Traversal. • We propose HVM-GraphRAG, a holistic-view Multimodal GraphRAG framework that improves answer accuracy and retrieval efficiency through Holistic-View Graph Construction and Graph-Guided Holistic Retrieval. • We conduct comprehensive experiments on three complex document question-answering datasets, demonstrating that HVM-GraphRAG improves both answer accuracy and inference efficiency over existing baselines. 2. Preliminary Study This section defines the core components of our knowledge representation, formalizes complex document QA within a two-stage GraphRAG framework, and empirically analyzes the limitations of existing Multimodal GraphRAG methods. 2.1. Definitions To facilitate the presentation of our method, we briefly formalize the core components of our knowledge representation. Concept (c) and Entity (e): A concept c denotes a high-level category, while an entity e denotes a concrete document instance. Each entity is assigned to exactly one concept by a concept assignment function ϕ(e)φ(e). Schema (s) and Fact (f): A schema s=(ch,r,ct)s=(c_h,r,c_t) represents a concept-level relation between two concepts. A fact f=(eh,r,et)f=(e_h,r,e_t) instantiates a schema at the entity level. Chunk (b): A chunk b=(x,m)b=(x,m) denotes a basic document unit, where x represents its content and m∈Ωm∈ specifies its modality. The modality set is defined as Ω=text,table,image =\text,table,image\. Detailed definitions and examples are provided in Appendix A.1. 2.2. Problem Statement Complex document QA (Zhao et al., 2024) often requires integrating evidence distributed across chunks, pages, and modalities. To support such evidence aggregation, Multimodal GraphRAG organizes document information into graph structures and uses them to guide evidence retrieval. We formulate the process as two stages: offline graph construction and online graph-guided retrieval and answering. Offline Graph Construction. Given a complex document D, the offline stage organizes its multimodal chunks into a graph-based representation for evidence indexing. Formally, the construction process produces a graph =(,ℰ)G=(V,E) and an evidence index ℐ∗I^*: (,ℐ∗)=GraphConstructor(D),(G,I^*)=GraphConstructor(D), where V denotes graph nodes, ℰE denotes relations among them, and ℐ∗I^* maps graph nodes to their supporting multimodal chunks. The evidence index connects the graph structure with the original document evidence. Online Graph-Guided Retrieval and Answering. Given a user query q, the online stage retrieves evidence through the constructed graph instead of directly searching over all chunks. The retriever first localizes query-relevant graph elements and then expands them to supporting chunks through ℐ∗I^*: ℬq=Retriever(q,,ℐ∗).B_q=Retriever(q,G,I^*). The retrieved chunks ℬqB_q are then used to generate the final answer: A=LLM(q,ℬq).A=LLM(q,B_q). Detailed task formulation is provided in Appendix A.2. Figure 3. The offline graph construction and online retrieval of HVM-GraphRAG. (1) A multimodal document tree provides structural context for each chunk. (2) Holistic-view construction resolves cross-modal conflicts and builds a reliable concept-level index. (3) Graph-guided retrieval searches the compact graph and organizes evidence by modality for answer generation. 2.3. Performance and Efficiency Analysis To better understand graph-based RAG over complex multimodal documents, we conduct a preliminary study on answer performance and retrieval efficiency. As shown in Fig. 2, we compare representative graph-based RAG methods against the flat retrieval baseline BM25 (Robertson and Walker, 1994) under the same evaluation setting. The comparison covers the text-based RAPTOR (Sarthi et al., 2024) and GraphRAG (Edge et al., ). It also includes GraphRanker as a multimodal variant of HippoRAG (Gutiérrez et al., 2024). The results show that graph structures do not always improve answer performance and can incur additional retrieval overhead. Performance Bottleneck. As shown in the left part of Fig. 2, graph-based RAG methods do not consistently outperform the flat BM25 baseline. A possible cause is the absence of a Holistic View during graph construction. Knowledge extracted from local chunks may be inserted without being checked against existing graph elements or evidence from other modalities (Fig. 1(b)). This can produce unreliable graph-to-chunk indices that introduce incomplete, redundant, or conflicting evidence into the retrieved context. Efficiency Bottleneck. As shown in the right part of Fig. 2, graph-based retrieval often requires more time than flat retrieval. This overhead arises from repeated node matching and relation traversal over dense graphs. Neighborhood expansion and evidence aggregation further increase the cost (Fig. 1(b)). These results show that effective Multimodal GraphRAG requires reliable evidence indexing and efficient retrieval over compact graph structures. This motivates our holistic-view-guided graph construction and graph-guided holistic retrieval. 3. Our Method To address the limitations of existing Multimodal GraphRAG methods (Liu et al., 2025a; Luo et al., 2026) in evidence indexing reliability and retrieval efficiency, we propose HVM-GraphRAG, a holistic-view multimodal GraphR-AG framework on complex document. The core idea is to build reliable evidence indices under holistic-view guidance and enable accurate and efficient retrieval over a compact concept graph. As illustrated in Fig.3, HVM-GraphRAG consists of two main modules: the Holistic-View-Guided Graph Construction Module (HGCM) and the Graph-Guided Holistic Retrieval Module (GHRM). 3.1. Holistic-View-Guided Graph Construction Existing Multimodal GraphRAG methods often construct graphs from local chunks without holistic document-level and cross-modal context. This can produce incomplete, redundant, or conflicting indices, resulting in Unreliable Cross-Modal Evidence Indexing (Wu et al., 2026). To address this problem, HVM-GraphRAG constructs a concept-level indexing graph through three designs: (1) Document Tree provides chunks with document-level structural context; (2) Cross-Modal Holistic View ℳM reduces noise, redundancy, and conflicts during graph construction; and (3) Reliable Evidence Indexing via Entity-Level Graph Bridging connects concept graph nodes directly to supporting multimodal chunks. Together, these designs produce a compact and reliable knowledge indexing structure. 3.1.1. Document Tree Construction A key reason for unreliable cross-modal evidence indexing is that chunks are often processed as isolated units during graph construction. Without the document-hierarchy context of a chunk, entities, values, and visual elements may be interpreted outside their intended section-level scope. To alleviate this issue, HVM-GraphRAG organizes the input document into a multimodal document tree (Wang et al., 2025c; Shin et al., 2026, 2025) that preserves both document hierarchy and modality information. Given a document D, a layout parser first extracts a sequence of multimodal chunks: (1) ℬ=LayoutParser⁡(D)=bii=1M,bi=(xi,mi), =LayoutParser(D)=\b_i\_i=1^M, b_i=(x_i,m_i), where LayoutParser⁡(⋅)LayoutParser(·) denotes the document layout parser, M is the number of parsed chunks, and xix_i and mim_i denote the content and modality of bib_i, respectively. These chunks are then organized into a document tree using the detected hierarchy and reading order: (2) =(,ℰ)=TreeBuilder(ℬ,D), =(N,E_T)=TreeBuilder(B,D), where TreeBuilder(⋅)TreeBuilder(·) is an LLM-assisted tree constructor, N contains structural and chunk nodes, and ℰE_T contains parent–child edges. For each chunk bib_i, Path(root,bi)Path_T(root,b_i) returns the ordered sequence of structural nodes from the document root to bib_i, which is used as document-level structural context: (3) hi=Path(root,bi). h_i=Path_T(root,b_i). This structural context specifies the document scope of each chunk, helping subsequent graph construction extract knowledge and build evidence indices under the correct section-level semantics. 3.1.2. Cross-Modal Holistic View During indexing graph construction, knowledge extracted from multimodal chunks may be redundant or semantically conflicting with the existing graph (Hong et al., 2024; Lee et al., 2025). To improve the consistency of cross-modal knowledge extraction, HVM-GraphRAG maintains a Cross-Modal Holistic View ℳM during graph construction. Here, ℳi−1M^i-1 denotes the holistic view before processing chunk bib_i, which contains the accepted facts from processed chunks. Based on this view, HVM-GraphRAG performs conflict detection and resolution, thereby reducing noisy graph updates and improving the reliability of evidence indexing. After constructing the document tree, HVM-GraphRAG uses the structural context of each chunk to support multimodal knowledge extraction. For each chunk bi=(xi,mi)b_i=(x_i,m_i), its root-to-leaf path hih_i is concatenated with the chunk content xix_i and processed by a modality-specific extractor from a set of multimodal extractors: (4) (i,i,i,ℱi)=Extractormi⁡(hi,xi,ρextmi), (C_i,U_i,S_i,F_i)=Extractor_m_i (h_i,x_i, _ext^m_i ), where iC_i, iU_i, iS_i, and ℱiF_i denote the sets of concepts, entities, schemas, and facts extracted from bib_i, respectively. Both the extraction prompt ρextmi _ext^m_i and the extractor Extractormi⁡(⋅)Extractor_m_i(·) are selected based on the modality mim_i to accommodate different content. The structural context hih_i helps the extractor interpret each chunk within its document-level scope, reducing extraction ambiguity. The extracted facts are combined with those in the existing Cross-Modal Holistic View ℳi−1M^i-1 to identify potential conflict groups: (5) ℱi+=ℱi∪ℳi−1, ^+_i=F_i ^i-1, where ℱiF_i denotes the facts extracted from the current chunk bib_i, ℳi−1M^i-1 contains the accepted facts accumulated from all chunks processed prior to bib_i, and ℱi+F^+_i represents their combined fact set for conflict candidate construction. For each fact f=(eh,r,et)f=(e_h,r,e_t), we define three matching keys: khr(f)=(eh,r)k_hr(f)=(e_h,r), kht(f)=(eh,et)k_ht(f)=(e_h,e_t), and kh(f)=ehk_h(f)=e_h. The group of a newly extracted fact f under key k is: (6) gk(f)=f′∈ℱi+∣k(f′)=k(f). g_k(f)=\f _i^+ k(f )=k(f)\. The set of potential conflict groups is defined as: (7) ℱconfi=gk(f)∣f∈ℱi,k∈khr,kht,kh,|gk(f)|>1. _conf^i= \g_k(f) f _i,\ k∈\k_hr,k_ht,k_h\,\ |g_k(f)|>1 \. Each retained group contains at least one fact from the current chunk and captures a potential intra-chunk conflict or a conflict with previously accepted facts. However, structural overlap does not necessarily indicate a factual contradiction (Jiayang et al., 2024; Huang et al., 2025). Therefore, an LLM-based detector examines each candidate group and retains only confirmed conflicts: (8) ℱ^confi=Detector⁡(g∣g∈ℱconfi,ρdet), F_conf^i=Detector(\g g _conf^i\, _det), where ρdet _det denotes the conflict-detection prompt and ℱ^confi F_conf^i contains the confirmed conflict groups. To support evidence-based resolution, we collect the entity-level knowledge nodes involved in each confirmed conflict group: (9) g=eh,et∣(eh,r,et)∈g. _g=\e_h,e_t (e_h,r,e_t)∈ g\. Their supporting chunks are then accessed through the entity-level evidence index: (10) ℬg=⋃e∈gℐfac(e),g∈ℱ^confi, _g= _e _gI_fac(e), g∈ F_conf^i, where e denotes an entity-level node involved in the conflict group. The function ℐfac(⋅)I_fac(·) maps each entity-level knowledge node to its supporting chunks. For nodes newly extracted from bib_i, the current chunk bib_i is recorded as their supporting chunk. The confirmed conflicts and their supporting chunks are subsequently passed to an LLM-based resolver: (11) ℳi,ℐfaci=Resolver⁡(ℳi−1,ℐfaci−1,ℱi,(g,ℬg)∣g∈ℱ^confi,bi,ρres), ^i,I_fac^i=Resolver(M^i-1,I_fac^i-1,F_i,\(g,B_g) g∈ F_conf^i\,b_i, _res), where ℳiM^i and ℐfaciI_fac^i denote the updated holistic view and entity-level evidence index after processing bib_i, respectively. The resolver updates the evidence index accordingly: links supported only by discarded facts are removed, while those associated with retained, revised, or newly accepted facts are preserved or updated. Based on the supporting evidence, the resolver merges redundant facts and revises inconsistent ones. It also retains complementary facts and removes unsupported ones. Facts outside the confirmed conflict groups in ℱiF_i are directly added to ℳiM^i. The updated holistic view then guides the processing of subsequent chunks. 3.1.3. Reliable Evidence Indexing via Entity-Level Graph Bridging To establish direct links from concept nodes to supporting chunks, HVM-GraphRAG uses the resolved entity-level facts in ℳM as an intermediate bridge. Let entV_ent denote the set of entity nodes induced by these facts. Each entity e∈ente _ent is assigned to a unique concept ϕ(e)φ(e). A concept-level graph con=(con,ℰcon)G_con=(V_con,E_con) is constructed as: (12) con _con =ϕ(e)∣e∈ent, =\φ(e) e _ent\, (13) ℰcon _con =(ϕ(eh),r,ϕ(et))∣(eh,r,et)∈ℳ. =\(φ(e_h),r,φ(e_t)) (e_h,r,e_t) \. For each concept node c∈conc _con, its entity-level preimage is: (14) ϕ−1(c)=e∈ent∣ϕ(e)=c. φ^-1(c)=\e _ent φ(e)=c\. The concept-level evidence index is then constructed as: (15) ℐcon(c)=⋃e∈ϕ−1(c)ℐfac(e),c∈con, _con(c)= _e∈φ^-1(c)I_fac(e), c _con, where ℐfac(e)I_fac(e) maps each entity node e to its supporting multimodal chunks. Therefore, ℐcon(c)I_con(c) aggregates the evidence associated with all entity nodes assigned to concept c, enabling direct concept-to-evidence access. This index enables direct evidence access over the compact concept graph without traversing the entity-level graph. 3.2. Graph-Guided Holistic Retrieval During online retrieval, existing Multimodal GraphRAG methods often search dense entity-level graphs through node matching, relation traversal, and neighborhood expansion. As complex documents contain numerous entities, relations, and cross-modal evidence links, this process leads to Overly Expensive Graph Traversal (Hu et al., 2026; Li et al., 2026). To reduce this cost, HVM-GraphRAG retrieves over the compact concept-level indexing graph con=(con,ℰcon)G_con=(V_con,E_con) constructed offline. The online process comprises three stages: (1) Concept Anchor Retrieval selects query-relevant concepts as retrieval anchors; (2) Similarity-Based Graph Propagation expands them through query-relevant relations; and (3) Modality-Aware Evidence Organization maps the selected elements to supporting chunks and organizes them by modality for answer generation. 3.2.1. Concept Anchor Retrieval Rather than directly searching over all entity-level facts or multimodal chunks, HVM-GraphRAG first localizes the query within the compact concept-level graph. Concept nodes capture high-level semantics shared by heterogeneous document content, making them suitable retrieval anchors for queries whose supporting evidence may span multiple entities and modalities. Given a query q, an LLM-based encoder Enc⁡(⋅)Enc(·) maps the query and each concept node c∈conc _con into a shared semantic space (Reimers and Gurevych, 2019; Karpukhin et al., 2020), and their relevance is computed as: (16) s(q,c)=sim⁡(Enc⁡(q),Enc⁡(c)), s(q,c)=sim(Enc(q),Enc(c)), where sim⁡(⋅,⋅)sim(·,·) denotes cosine similarity. The top-kck_c concept nodes with high relevance scores are selected as the initial anchors: (17) q=c∣(c,s(q,c))∈TopKkc⁡(con;s),s(q,c)≥θ, _q=\c (c,s(q,c)) _k_c(V_con;s),\ s(q,c)≥θ\, where qA_q denotes the concept-anchor set, TopKkc⁡(con;s)TopK_k_c(V_con;s) selects the top-kck_c concepts ranked by s(q,c)s(q,c), and θ filters low-confidence concepts. These anchors identify the most relevant semantic regions of the graph and constrain subsequent propagation to query-related structures, reducing unnecessary traversal. 3.2.2. Similarity-Based Graph Propagation Although these anchors capture concepts directly related to the query, they may still miss related concepts needed for complete evidence retrieval. To expand the retrieval scope, HVM-GraphRAG measures the semantic similarity s(q,r)s(q,r) between the query q and each relation r in the concept-level graph. The top-krk_r relations with the highest similarity scores form the candidate relation set ℛqR_q. To keep propagation constrained by reliable anchors, only candidate relations connected to at least one concept in qA_q are retained: (18) ℛ^q=r∈ℛq∣∃(ch,r,ct)∈ℰcon,ch,ct∩q≠∅, R_q=\r _q ∃(c_h,r,c_t) _con,\c_h,c_t\ _q≠ \, where ℰconE_con denotes the set of concept-level edges in conG_con, and each edge is a triplet (ch,r,ct)(c_h,r,c_t). The concepts reached through the retained relations are then merged with the initial anchors: (19) ^q=q∪ch,ct∣(ch,r,ct)∈ℰcon,r∈ℛ^q,ch,ct∩q≠∅. A_q=A_q∪\c_h,c_t (c_h,r,c_t) _con,r∈ R_q,\c_h,c_t\ _q≠ \. This constrained propagation expands the retrieved concept set through query-relevant relations while avoiding propagation toward unrelated graph regions. 3.2.3. Modality-Aware Evidence Organization Retrieved evidence from complex documents often spans text, tables, and images. Interleaving these heterogeneous chunks may obscure modality-specific semantics and hinder effective evidence integration (Wang et al., 2026b). To address this issue, HVM-GraphRAG first constructs a candidate evidence set relevant to the query and then organizes the selected chunks into modality-specific groups before answer generation. Table 1. Statistics of the datasets. Dataset MMLongBench M3DocVQA Qasper Questions 669 633 640 Documents 85 500 192 Avg. Pages 42.16 8.52 10.95 Avg. Images 25.92 3.51 3.43 Cross-Modal Questions (%) 22.73 0.00 8.31 Tokens 2,816,155 3,553,774 2,265,349 Metrics EM, F1 EM, F1 Acc, F1 Table 2. Performance comparison on three complex document QA datasets (MMLongBench, M3DocVQA and Qasper). Best and second-best results are shown in bold and underlined, respectively. Baseline Type Method MMLongBench M3DocVQA Qasper Exact Match F1-score Exact Match F1-score Accuracy F1-score Conventional RAG !10BM25 !2518.3 !2520.2 3 !254.6 !2537.8 !2538.1 !2542.5 !10Vanilla RAG !2516.5↓1.8 !2518.0↓2.2 !2536.5↑1.9 !2540.2↑2.4 !2540.6↑2.5 !2544.4↑1.9 !10Layout + Vanilla !2518.1↓0.2 !2519.8↓0.4 !2536.9↑2.3 !2540.2↑2.4 !2540.7↑2.6 !2544.6↑2.1 GraphRAG !25RAPTOR !2521.3↑3.0 !2521.8↑1.6 !2534.3↓0.3 !2537.3↓0.5 !2539.4↑1.3 !2544.1↑1.6 !25GraphRAG-Local !257.7↓10.6 !258.5↓11.7 !2523.7↓10.9 !2525.6↓12.2 !2535.9↓2.2 !2539.2↓3.3 !25GraphRAG-Global !255.3↓13.0 !255.6↓14.6 !2520.2↓14.4 !2522.0↓15.8 !2524.0↓14.1 !2524.1↓18.4 Layout segmented RAG !25M-Vanilla !256.8↓11.5 !258.4↓11.8 !2525.1↓9.5 !2527.7↓10.1 !2527.9↓10.2 !2529.3↓13.2 !25Tree-Traverse !2512.7↓5.6 !2514.4↓5.8 !2533.3↓1.3 !2536.2↓1.6 !2527.3↓10.8 !2532.1↓10.4 !25DocETL !2527.5↑9.2 !2528.6↑8.4 !2540.9↑6.3 !2543.3↑5.5 !2542.3↑4.2 !2550.4↑7.9 Multimodal GraphRAG !25GraphRanker !2521.2↑2.9 !2522.7↑2.5 !2543.0↑8.4 !2547.8↑10.0 !2532.9↓5.2 !2537.6↓4.9 !25BookRAG !2543.8↑25.5 !2544.9↑24.7 !2561.0↑26.4 !2566.2↑28.4 !2555.2↑17.1 !2561.1↑18.6 Our proposed !25HVM-GraphRAG !2554.1↑35.8 !2562.8↑42.6 !2563.0↑28.4 !2566.0↑28.2 !2566.1↑28.0 !2571.5↑29.0 After graph propagation, ^q A_q and ℛ^q R_q define the query-relevant concept subgraph. HVM-GraphRAG directly accesses its supporting chunks through ℐcon(⋅)I_con(·), avoiding entity-level graph traversal: (20) ℬqgraph=⋃c∈^qℐcon(c), _q^graph= _c∈ A_qI_con(c), where ℬqgraphB_q^graph denotes the graph-guided candidate chunk set, c denotes a concept-level graph node from ^q A_q, and ℐcon(⋅)I_con(·) maps each graph node to its supporting chunks. Each graph-retrieved chunk b∈ℬqgraphb _q^graph is converted into a modality-specific textual representation Repmb⁡(b)Rep_m_b(b). This function preserves the original representations of text and table chunks. For image chunks, it uses a VLM to generate textual summaries. The semantic similarity between the query q and chunk b is then computed as: (21) s(q,b)=sim⁡(Enc⁡(q),Enc⁡(Repmb⁡(b))). s(q,b)=sim(Enc(q),Enc(Rep_m_b(b))). The top-K chunks are retained as the graph-guided evidence set ℬqKB_q^K. Graph-guided retrieval localizes semantically connected evidence. Direct query–chunk matching supplements the evidence set with relevant chunks that may be overlooked during semantic graph construction. The supplementary evidence set is obtained by retrieving the top-kbk_b chunks from ℬB: (22) ℬqrag=TopKkb⁡((b,s(q,b))∣b∈ℬ). _q^rag=TopK_k_b (\(b,s(q,b)) b \ ). The graph-guided evidence and conventional RAG evidence are then merged: (23) ℬqfinal=ℬqK∪ℬqrag, _q^final=B_q^K _q^rag, where ℬqfinalB_q^final denotes the final candidate chunk set before modality-aware organization. Although ℬqfinalB_q^final contains relevant evidence, modality-interleaved chunks may make heterogeneous evidence harder for the answering model to compare and integrate. To present the retrieved information more clearly, HVM-GraphRAG partitions ℬqfinalB_q^final by modality: (24) ℬqm=b∈ℬqfinal∣mb=m,m∈Ω, _q^m=\b _q^final m_b=m\, m∈ , where Ω=text,table,image =\text,table,image\ denotes the modality set. Chunks in each modality group are ranked by s(q,b)s(q,b). The top-kmk_m chunks are retained to construct the final evidence context: (25) ℬqmaeo=⋃m∈ΩTopKkm⁡(ℬqm), _q^maeo= _m∈ TopK_k_m(B_q^m), where ℬqmaeoB_q^maeo denotes the modality-aware evidence set. The selected chunks are arranged in a predefined modality order and provided to the answering model: (26) A=Answer⁡(q,ℬqmaeo,ρans), A=Answer(q,B_q^maeo, _ans), where Answer⁡(⋅)Answer(·) and ρans _ans denote the answer-generation pipeline and prompt, respectively. Within this pipeline, a VLM converts each image chunk into a textual summary. The summaries and textual representations of non-image chunks are ordered by modality and passed to the LLM for answer generation. This modality-specific organization enables the answering model to distinguish and integrate heterogeneous evidence more effectively. 4. Experiments In this section, we conduct extensive experiments to answer the following research questions: • RQ1: How does HVM-GraphRAG perform against state-of-the-art methods on complex document QA? • RQ2: How does HVM-GraphRAG compare with existing RAG methods in online latency and token consumption? • RQ3: What extent does each component contribute to the overall performance of HVM-GraphRAG? • RQ4: How do different hyperparameter settings affect the performance of HVM-GraphRAG? Additional experimental results, including case studies and further analyses, are provided in the Appendix B. 4.1. Experimental Setting 4.1.1. Datasets We evaluate HVM-GraphRAG on three complex document QA datasets: MMLongBench (Ma et al., 2024), M3DocVQA (Cho et al., 2024), and Qasper (Dasigi et al., 2021). MMLongBench contains long documents from diverse domains, including guidebooks, financial reports, and industry documents. M3DocVQA comprises multimodal HTML-style documents derived from Wikipedia and requires QA across heterogeneous document elements. Qasper focuses on scientific papers, with supporting evidence often distributed throughout the document. We use only the processed benchmark versions of these datasets provided by prior work (Wang et al., 2025c), without introducing any additional data filtering, question augmentation, or split modification. The statistics of the three datasets are summarized in Table 1, with detailed descriptions provided in the Appendix B.1. 4.1.2. Baselines We compare HVM-GraphRAG with representative methods from four RAG categories. Conventional RAG includes BM25 (Robertson and Walker, 1994), Vanilla RAG, and Layout+Vanilla. Graph-Based RAG includes RAPTOR (Sarthi et al., 2024) and two variants of GraphRAG (Edge et al., ): GraphRAG-Local and GraphRAG-Global. Layout segmented RAG includes M-Vanilla, Tree-Traverse (Zhang et al., 2025d), and DocETL (Shankar et al., 2024). Multimodal GraphRAG includes GraphRanker (Gutiérrez et al., 2024) and BookRAG (Wang et al., 2025c). Details of these methods are provided in the Appendix B.2. 4.1.3. Metrics We evaluate all methods using a unified answer-extraction and normalization pipeline and report Exact Match (EM), token-level F1 score, and string-based accuracy (Acc). EM measures the proportion of predictions that exactly match the reference answers. F1 evaluates token-level overlap between predicted and reference answers. Acc measures the proportion of correctly answered questions. Higher values indicate better performance for all three metrics. Detailed definitions and calculation procedures for these metrics are provided in the Appendix B.3. 4.1.4. Implementation Details For a fair comparison, all methods use the same embedding model and answer-generation model. We employ Qwen3-Embedding-0.6B (Zhang et al., 2025f) to encode queries and retrieval units. The document layout is parsed using Mineru (Wang et al., 2024). During graph construction, Qwen3-8B-AWQ (Yang et al., 2025a) extracts knowledge from textual chunks while Qwen2.5-VL (Wu et al., 2025a) processes visual chunks. The same Qwen3-8B-AWQ is used for conflict resolution and final answer generation. For online retrieval, we select the K=10K=10 for top-K retrieval in all methods. And we set the temperature to 0 for all LLM and VLM calls to ensure reproducibility. The experiments are conducted on an Intel(R) Xeon(R) Gold 5120 CPU and four NVIDIA L40 GPUs with 48 GB of memory each. Further implementation details and hyperparameter settings are provided in the Appendix B.4. 4.2. Overall Performance (RQ1) To answer RQ1, we compare the answer generation performance of HVM-GraphRAG with all baseline methods on MMLongBench, M3DocVQA, and Qasper. The results are reported in Table 2. Based on these results, we make the following observations: Graph-based RAG shows inconsistent performance gains. RAPTOR achieves 39.4% Accuracy on Qasper and outperforms BM25 by 1.3 percentage points. In contrast, GraphRAG-Local and GraphRAG-Global achieve 35.9% and 24.0% Accuracy. They underperform BM25 by 2.2 percentage points and 14.1 percentage points, respectively. These results suggest that graph structures alone do not guarantee better retrieval. A possible explanation is that unreliable graph-to-evidence associations introduce conflicting information into the retrieved context. Layout segmented RAG is sensitive to model design. DocETL consistently performs best within this category. It achieves Exact Match of 27.5% on MMLongBench dataset. In comparison, Tree-Traverse obtains 12.7% while M-Vanilla reaches only 6.8%. This gap may arise from their different uses of document structure. Unlike simple multimodal retrieval or tree navigation, DocETL performs structured document transformations before retrieval. This suggests that layout information requires an effective processing mechanism to improve QA performance. Multimodal GraphRAG achieves inconsistent gains. Graph-Ranker achieves 37.6% F1 on Qasper and trails BM25 by 4.9%. In contrast, BookRAG achieves F1-scores of 61.1% on the Qasper dataset. This gap shows that multimodal graph construction alone does not guarantee strong QA performance. BookRAG benefits from hierarchical document organization and adaptive retrieval. However, neither method explicitly verifies the reliability of graph-to-evidence indices. Noisy or conflicting evidence associations may therefore limit their retrieval quality. HVM-GraphRAG achieves the best result on five of the six reported metrics. These gains are consistent with the intended benefit of holistic-view-guided graph construction. The improvements are particularly large on MMLongBench and Qasper with F1-score gains of 17.9 and 10.4 percentage points over BookRAG. Cross-modal questions account for 22.73% and 8.31% of these datasets. By contrast, M3DocVQA contains no such questions and shows only limited improvement. This pattern suggests that the gains of HVM-GraphRAG may be more pronounced when answering requires cross-modal evidence integration. 4.3. Efficiency Comparison (RQ2) To answer RQ2, we compare the online query efficiency of HVM-GraphRAG with baseline methods by reporting total query time and token consumption across all evaluation questions on MMLongBench and Qasper. All measurements are collected after document preprocessing and indexing and therefore exclude offline construction costs. The results are presented in Fig. 4, with complete results and further analyses provided in Appendix B.5. Based on these results, we make the following observations: Graph-based and layout-segmented RAG methods generally incur higher retrieval costs. Graph-based methods require matching and traversal over complex graph structures. Layout-segmented methods also process more structural units with richer content. Consequently, most of these methods consume more query time and tokens than conventional RAG on both datasets. BookRAG reduces some graph-related overhead but still relies on hierarchical traversal and LLM-driven adaptive retrieval. Its efficiency therefore remains less competitive than conventional RAG. Figure 4. Comparison of query efficiency. Figure 5. Ablation results on MMLongBench/M3DocVQA (EM) and Qasper (Acc). HVM-GraphRAG achieves efficiency comparable to conventional RAG. Our proposed HVM-GraphRAG maintains low total query time and token consumption on both datasets. This efficiency comes from retrieving over a compact concept-level graph and directly mapping selected graph nodes to supporting chunks. Moreover, the retrieval process requires no additional LLM-based query analysis. These designs avoid entity-level graph traversal and substantially reduce online retrieval costs. 4.4. Ablation Study (RQ3) To answer RQ3, we conduct ablation studies on all three datasets to evaluate the contribution of each component. We compare the full model with three variants that remove the Concept-Guided Graph (w/o CG), Conflict Resolution (w/o CR), and Modality-Aware Evidence Organization (w/o ME), respectively. As shown in Fig.5, the full model consistently achieves the best performance of 54.1%, 63.0%, and 66.1%. Removing any component leads to performance degradation across all datasets. These results demonstrate that the three components jointly improve evidence retrieval and final QA performance. w/o Concept-Guided Graph. Removing the Concept-Guided Graph leads to the largest performance decline with scores dropping to 49.0%, 57.1%, and 59.1%. Without compact concept-level retrieval anchors, the model cannot effectively localize relevant graph regions and may retrieve less relevant evidence. w/o Conflict Resolution. Removing Conflict Resolution causes consistent degradation across all datasets and yields scores of 50.7%, 59.9%, and 64.4%. Without global conflict detection and resolution, redundant or contradictory facts remain in the graph and weaken the reliability of evidence indexing. w/o Modality-Aware Evidence Organization. Excluding Mod-ality-Aware Evidence Organization produces scores of 50.8%, 60.3%, and 63.4%. Without modality-specific organization, heterogeneous chunks remain interleaved in the retrieved context and increase the difficulty for the answering model to interpret and integrate multimodal evidence. The edge degree distribution Figure 6. Effects of the similarity threshold θ and the number of supplementary chunks kbk_b on MMLongBench and M3DocVQA. Darker colors indicate higher performance. The edge degree distribution Figure 7. Effects of the number of retrieved concepts kck_c on MMLongBench and Qasper. 4.5. Hyperparameter Analysis (RQ4) To answer RQ4, we examine two groups of retrieval hyperparameters. First, we jointly analyze the similarity threshold θ for filtering concept anchors and the number kbk_b of supplementary chunks retrieved by conventional RAG method. Second, we vary the number kck_c of top-ranked concept nodes selected from the compact concept graph. All remaining parameters are fixed during each experiment. The results are presented in Fig.6 and Fig.7. Effects of θ and kbk_b. As shown in Fig. 6, the influence of θ varies across datasets. On MMLongBench, lower thresholds generally achieve stronger performance, especially around θ=0.25θ=0.25 and kb=2k_b=2. This suggests that retaining more concept anchors helps cover distributed evidence in long documents. In contrast, M3DocVQA performs better under higher thresholds, such as θ=0.5θ=0.5. Since M3DocVQA contains more localized visual question answering cases, stricter concept-anchor filtering can reduce unnecessary graph expansion and keep retrieval focused on highly matched evidence. The effect of kbk_b is also dataset-dependent. MMLongBench benefits from a small number of supplementary chunks, while M3DocVQA remains relatively stable when kbk_b increases. Overall, these results show that properly setting θ and kbk_b helps HVM-GraphRAG balance graph-guided retrieval and local evidence supplementation, thereby improving the final QA performance. Effects of kck_c. As shown in Fig. 7, a moderate number of retrieved concepts leads to better performance. On MMLongBench, the best result appears around concept Top-k=10k=10, while introducing more concepts gradually degrades performance. On Qasper, the model remains relatively stable from k=10k=10 to k=40k=40, but performance drops when k further increases. These results suggest that retrieving too few concepts may miss useful graph retrieval directions, whereas retrieving too many concepts can introduce noisy or weakly relevant structures. Overall, an appropriate concept Top-k improves QA performance by balancing concept coverage and retrieval focus. 5. Related Work 5.1. RAG on Complex Document Retrieval-Augmented Generation (RAG) retrieves external evidence to support answer generation (Lewis et al., 2020; Karpukhin et al., 2020; Izacard and Grave, 2021). Conventional methods such as BM25 (Robertson and Walker, 1994) and VanillaRAG retrieve textual chunks through lexical or dense semantic matching. To better handle complex documents, recent methods further incorporate document layout and multimodal content. For example, layout-aware retrieval constructs more coherent document units, M-Vanilla retrieves textual and visual chunks with multimodal embeddings, Tree-Traverse (Zhang et al., 2025d) navigates document hierarchies with an LLM, and DocETL (Shankar et al., 2024) performs structured document transformations before retrieval. Although these methods improve evidence segmentation, they still provide limited modeling of semantic relations among distributed evidence units. This limitation becomes critical when reasoning requires evidence aggregation across pages, regions, and modalities. 5.2. GraphRAG and Multimodal GraphRAG GraphRAG (Wang et al., 2026a; Niu et al., 2026; Song et al., 2026) introduces structured relations into retrieval by organizing document knowledge or evidence as graphs. RAPTOR (Sarthi et al., 2024) builds hierarchical summaries over textual chunks, while GraphRAG (Edge et al., ) constructs entity-centric graphs and supports local or global search. For multimodal documents, GraphRanker (Gutiérrez et al., 2024) applies graph-based ranking to multimodal document graphs, and BookRAG (Wang et al., 2025c) combines hierarchical document organization with adaptive retrieval. These methods show the benefit of relation-aware evidence modeling. However, existing graph-based methods often rely on locally constructed graph elements or dense entity-level traversal. As a result, they may suffer from unreliable cross-modal evidence indexing and high retrieval cost. In contrast, HVM-GraphRAG introduces a holistic view during graph construction and performs retrieval over a compact concept-level indexing graph. 6. Conclusion This paper proposes Holistic-View Multimodal Graph Retrieval-Augmented Generation (HVM-GraphRAG) on complex document. HVM-GraphRAG incorporates a holistic-view into graph construction to address unreliable cross-modal evidence indexing, and uses a compact concept-level graph to guide retrieval for improved efficiency. By building reliable links between concept-level graph elements and supporting multimodal chunks, HVM-GraphRAG retrieves grounded evidence while avoiding costly traversal over dense entity-level graphs. Experiments on three complex document QA datasets show that HVM-GraphRAG improves both answer performance and online retrieval efficiency over representative RAG baselines. These results highlight the importance of reliable evidence indexing and compact graph-guided retrieval for effective complex document QA. References S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. (2025) Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §1. R. C. Barron, M. E. Eren, O. M. Serafimova, C. Matuszek, and B. S. Alexandrov (2025) Bridging legal knowledge and ai: retrieval-augmented generation with vector stores, knowledge graphs, and hierarchical non-negative matrix factorization. arXiv preprint arXiv:2502.20364. Cited by: §1. C. Bu, G. Chang, Z. Chen, C. Dang, Z. Wu, Y. He, and X. Wu (2025) Query-driven multimodal graphrag: dynamic local knowledge graph construction for online reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, p. 21360–21380. Cited by: §1. W. Chen, H. Hu, X. Chen, P. Verga, and W. Cohen (2022) Murag: multimodal retrieval-augmented generator for open question answering over images and text. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 5558–5570. Cited by: §1. Z. Chen, Q. Zhang, Z. Xiang, Z. Wei, L. Gao, X. Huang, Z. Zhang, and J. Su (2026) LegalGraphRAG: multi-agent graph retrieval-augmented generation for reliable legal reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 37455–37484. Cited by: §1. J. Cho, D. Mahata, O. Irsoy, Y. He, and M. Bansal (2024) M3docrag: multi-modal retrieval is what you need for multi-page multi-document understanding. arXiv preprint arXiv:2411.04952. Cited by: §B.1, §4.1.1. C. Choi, J. Kwon, A. Lopez-Lira, C. Kim, M. Kim, J. Hwang, J. Ha, H. Choi, S. Yun, Y. Kim, et al. (2025) Finagentbench: a benchmark dataset for agentic retrieval in financial question answering. In Proceedings of the 6th ACM International Conference on AI in Finance, p. 632–637. Cited by: §1. S. Dai, Q. Huang, X. You, and J. Yu (2026) MG2-rag: multi-granularity graph for multimodal retrieval-augmented generation. arXiv preprint arXiv:2604.04969. Cited by: §1. P. Dasigi, K. Lo, I. Beltagy, A. Cohan, N. A. Smith, and M. Gardner (2021) A dataset of information-seeking questions and answers anchored in research papers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, p. 4599–4610. Cited by: §B.1, §4.1.1. C. Deng, J. Yuan, P. Bu, P. Wang, Z. Li, J. Xu, X. Li, Y. Gao, J. Song, B. Zheng, et al. (2025) Longdocurl: a comprehensive multimodal long document benchmark integrating understanding, reasoning, and locating. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1135–1159. Cited by: §1. [11] D. Edge, H. Trinh, J. B. Newman Cheng, A. Chao, A. Mody, S. Truitt, D. Metropolitansky, R. O. Ness, and J. Larson From local to global: a graph rag approach to query-focused summarization, 2025. URL https://arxiv. org/abs/2404.16130. Cited by: §B.2, §C.2, §1, §2.3, §4.1.2, §5.2. X. Fang, W. Fang, and C. Wang (2026) Cogniverse: revolutionizing multi-modal retrieval-augmented generation with cognitive reflection and geometric reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 7923–7935. Cited by: §C.1. B. J. Gutiérrez, Y. Shu, Y. Gu, M. Yasunaga, and Y. Su (2024) Hipporag: neurobiologically inspired long-term memory for large language models. Advances in neural information processing systems 37, p. 59532–59569. Cited by: §B.2, §C.2, §1, §2.3, §4.1.2, §5.2. G. Hong, J. Kim, J. Kang, S. Myaeng, and J. J. Whang (2024) Why so gullible? enhancing the robustness of retrieval-augmented models against counterfactual noise. In Findings of the Association for Computational Linguistics: NAACL 2024, p. 2474–2495. Cited by: §3.1.2. C. Hsiao, Y. Wang, T. Lin, Y. Yeh, and C. Chen (2026) MegaRAG: multimodal knowledge graph-based retrieval augmented generation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 48031–48059. Cited by: §1. Y. Hu, J. Zhu, L. Tang, and C. Huang (2026) ReMindRAG: low-cost llm-guided knowledge graph traversal for efficient rag. Advances in Neural Information Processing Systems 38, p. 53757–53798. Cited by: §1, §3.2. H. Huang, C. Chen, Z. Sheng, Y. Li, and W. Zhang (2025) Can llms be good graph judge for knowledge graph construction?. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 10940–10959. Cited by: §3.1.2. G. Izacard and E. Grave (2021) Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th conference of the european chapter of the association for computational linguistics: main volume, p. 874–880. Cited by: §5.1. C. Jiayang, C. Chan, Q. Zhuang, L. Qiu, T. Zhang, T. Liu, Y. Song, Y. Zhang, P. Liu, and Z. Zhang (2024) ECON: on the detection and resolution of evidence conflicts. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 7816–7844. Cited by: §3.1.2. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih (2020) Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), p. 6769–6781. Cited by: §3.2.1, §5.1. J. Lee, K. Lee, and T. Kim (2025) MAGIC: a multi-hop and graph-based benchmark for inter-context conflicts in retrieval-augmented generation. arXiv preprint arXiv:2507.21544. Cited by: §3.1.2. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, p. 9459–9474. Cited by: §5.1. H. Li, H. Gao, C. Wu, and M. A. Vasarhelyi (2025) Extracting financial data from unstructured sources: leveraging large language models. Journal of Information Systems 39 (1), p. 135–156. Cited by: §1. L. Li, Y. Zou, J. Wu, Y. Wen, J. Li, H. Qian, and I. Tsang (2026) SCOUT-rag: scalable and cost-efficient unifying traversal for agentic graph-rag over distributed domains. arXiv preprint arXiv:2602.08400. Cited by: §1, §3.2. P. Liu, X. Liu, R. Yao, J. Liu, S. Meng, D. Wang, and J. Ma (2025a) Hm-rag: hierarchical multi-agent multimodal retrieval augmented generation. In Proceedings of the 33rd ACM international conference on multimedia, p. 2781–2790. Cited by: §3. Z. Liu, Y. Liu, J. Yuan, M. Yan, L. Yue, H. Xiong, and Y. Yang (2025b) Graph-guided concept selection for efficient retrieval-augmented generation. arXiv preprint arXiv:2510.24120. Cited by: §1. H. Luo, G. Chen, Y. Zheng, X. Wu, Y. Guo, Q. Lin, Y. Feng, Z. Kuang, M. Song, Y. Zhu, et al. (2026) Hypergraphrag: retrieval-augmented generation via hypergraph-structured knowledge representation. Advances in Neural Information Processing Systems 38, p. 152206–152234. Cited by: §3. Y. Ma, Y. Zang, L. Chen, M. Chen, Y. Jiao, X. Li, X. Lu, Z. Liu, Y. Ma, X. Dong, et al. (2024) Mmlongbench-doc: benchmarking long-context document understanding with visualizations. Advances in Neural Information Processing Systems 37, p. 95963–96010. Cited by: §B.1, §4.1.1. A. Nagori, R. A. Casonatto, A. Gautam, A. M. S. Cheruvu, and R. Kamaleswaran (2025) Open-source agentic hybrid rag framework for scientific literature review. arXiv preprint arXiv:2508.05660. Cited by: §B.1. M. Niu, L. Shan, Z. Yu, J. Zhu, and T. Xiao (2026) EfficientGraph-rag: structured retrieval-state management for cross-task retrieval-augmented generation. arXiv preprint arXiv:2605.25379. Cited by: §5.2. B. Peng, Y. Zhu, Y. Liu, X. Bo, H. Shi, C. Hong, Y. Zhang, and S. Tang (2025) Graph retrieval-augmented generation: a survey. ACM Transactions on Information Systems 44 (2), p. 1–52. Cited by: §1. H. Qian, Z. Liu, P. Zhang, K. Mao, D. Lian, Z. Dou, and T. Huang (2025) Memorag: boosting long context processing with global memory-enhanced retrieval augmentation. In Proceedings of the ACM on Web Conference 2025, p. 2366–2377. Cited by: §C.1. D. Quinn, M. Nouri, N. Patel, J. Salihu, A. Salemi, S. Lee, H. Zamani, and M. Alian (2025) Accelerating retrieval-augmented generation. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, p. 15–32. Cited by: §1. N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), p. 3982–3992. Cited by: §3.2.1. M. Riedler and S. Langer (2024) Beyond text: optimizing rag with multimodal inputs for industrial applications. arXiv preprint arXiv:2410.21943. Cited by: §1. S. E. Robertson and S. Walker (1994) Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR’94: Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval, organised by Dublin City University, p. 232–241. Cited by: §B.2, §C.1, §2.3, §4.1.2, §5.1. S. Robertson and H. Zaragoza (2009) The probabilistic relevance framework: bm25 and beyond. Vol. 4, Now Publishers Inc. Cited by: §1. A. Sadowski and J. A. Chudziak (2025) On verifiable legal reasoning: a multi-agent framework with formalized knowledge representations. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, p. 2535–2545. Cited by: §1. B. Sarmah, D. Mehta, B. Hall, R. Rao, S. Patel, and S. Pasquali (2024) Hybridrag: integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction. In Proceedings of the 5th ACM International Conference on AI in Finance, p. 608–616. Cited by: §B.1. P. Sarthi, S. Abdullah, A. Tuli, S. Khanna, A. Goldie, and C. Manning (2024) Raptor: recursive abstractive processing for tree-organized retrieval. In International Conference on Learning Representations, Vol. 2024, p. 32628–32649. Cited by: §B.2, §C.2, §1, §2.3, §4.1.2, §5.2. H. Scaffidi, M. Hodkiewicz, C. Woods, and N. Roocke (2025) GraphRAG on technical documents-impact of knowledge graph schema. Transactions on Graph Data and Knowledge 3 (2), p. 3–1. Cited by: §B.1. S. Shankar, T. Chambers, T. Shah, A. G. Parameswaran, and E. Wu (2024) Docetl: agentic query rewriting and evaluation for complex document processing. arXiv preprint arXiv:2410.12189. Cited by: §B.2, §C.1, §C.1, §4.1.2, §5.1. J. Shin, C. Park, J. Park, J. Seo, and H. Lim (2025) MultiDocFusion: hierarchical and multimodal chunking pipeline for enhanced rag on long industrial documents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 20996–21015. Cited by: §3.1.1. J. Shin, G. Shim, J. Park, J. Seo, and H. Lim (2026) HiKEY: hierarchical multimodal retrieval for open-domain document question answering. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 17967–17987. Cited by: §3.1.1. J. K. Song, D. B. Youk, H. Kim, and S. Hwang (2026) Multimodal knowledge graph–guided rag-llm for clinical decision support in pediatric leukemia. Cancer Research and Treatment. Cited by: §5.2. M. Suri, P. Mathur, F. Dernoncourt, K. Goswami, R. A. Rossi, and D. Manocha (2025) Visdom: multi-document qa with visually rich elements using multimodal retrieval-augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 6088–6109. Cited by: §C.1. J. Tang, L. Xia, Z. Li, and C. Huang (2026) Ai-researcher: autonomous scientific innovation. Advances in Neural Information Processing Systems 38, p. 9481–9520. Cited by: §1. X. Wan and H. Yu (2025) Mmgraphrag: bridging vision and language with interpretable multimodal knowledge graphs. arXiv preprint arXiv:2507.20804. Cited by: §1. B. Wang, C. Xu, X. Zhao, L. Ouyang, F. Wu, Z. Zhao, R. Xu, K. Liu, Y. Qu, F. Shang, et al. (2024) Mineru: an open-source solution for precise document content extraction. arXiv preprint arXiv:2409.18839. Cited by: §B.4, §4.1.4. H. Wang, A. Prasad, E. Stengel-Eskin, and M. Bansal (2025a) Retrieval-augmented generation with conflicting evidence. arXiv preprint arXiv:2504.13079. Cited by: §C.1. J. Wang, H. Huang, X. Ge, J. Su, W. Liu, and S. Lian (2026a) OMD-graphrag: enhancing graphrag with ontology-guided extraction, multi-dimensional clustering and dual-channel fusion. arXiv preprint arXiv:2603.25152. Cited by: §5.2. Q. Wang, R. Ding, Z. Chen, W. Wu, S. Wang, P. Xie, and F. Zhao (2025b) Vidorag: visual document retrieval-augmented generation via dynamic iterative reasoning agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 9124–9145. Cited by: §C.1. S. Wang, Y. Zhou, and Y. Fang (2025c) BookRAG: a hierarchical structure-aware index-based approach for retrieval-augmented generation on complex documents. arXiv preprint arXiv:2512.03413. Cited by: §B.1, §B.2, §B.3, §C.2, §1, §3.1.1, §4.1.1, §4.1.2, §5.2. X. Wang, Z. Wang, C. Huang, Q. Z. Sheng, and L. Yao (2026b) MEG-rag: quantifying multi-modal evidence grounding for evidence selection in rag. arXiv preprint arXiv:2604.24564. Cited by: §3.2.3. C. Wu, J. Li, J. Zhou, J. Lin, K. Gao, K. Yan, S. Yin, S. Bai, X. Xu, Y. Chen, et al. (2025a) Qwen-image technical report. arXiv preprint arXiv:2508.02324. Cited by: §B.4, §4.1.4. C. Wu, Z. Xiang, Y. Tang, Z. Chen, Q. Zhang, and J. Su (2026) MemGraphRAG: memory-based multi-agent system for graph retrieval-augmented generation. arXiv preprint arXiv:2606.00610. Cited by: §1, §1, §3.1. X. Wu, Y. Tan, N. Hou, R. Zhang, and H. Cheng (2025b) Molorag: bootstrapping document understanding via multi-modal logic-aware retrieval. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 14035–14056. Cited by: §1. P. Xia, K. Zhu, H. Li, T. Wang, W. Shi, S. Wang, L. Zhang, J. Y. Zou, and H. Yao (2025) Mmed-rag: versatile multimodal rag system for medical vision language models. In International Conference on Learning Representations, Vol. 2025, p. 66188–66217. Cited by: §1. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025a) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §B.4, §4.1.4. R. Yang, B. Yang, A. Feng, S. Ouyang, M. Blum, T. She, Y. Jiang, F. Lecue, J. Lu, and I. Li (2025b) Graphusion: a rag framework for knowledge graph construction with a global perspective (no. arxiv: 2410.17600). arxiv. Cited by: §1. C. Yu, K. Zhao, Y. Li, H. Chang, M. Feng, X. Jiang, Y. Sun, J. Li, Y. Zhang, Q. Sun, et al. (2026a) Graphrag-r1: graph retrieval-augmented generation with process-constrained reinforcement learning. In Proceedings of the ACM Web Conference 2026, p. 1398–1409. Cited by: §C.2. J. Yu, Y. Liu, J. Gu, P. Torr, and D. Zhou (2026b) Can knowledge-graph-based retrieval augmented generation really retrieve what you need?. Advances in Neural Information Processing Systems 38, p. 95653–95682. Cited by: §1. W. Yu, Z. Yang, Y. Liu, and X. Bai (2025a) Docthinker: explainable multimodal large language models with rule-based reinforcement learning for document understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 837–847. Cited by: §1. X. Yu, P. Jian, and C. Chen (2025b) Tablerag: a retrieval augmented generation framework for heterogeneous document reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 14074–14093. Cited by: §C.1. X. Yuan, L. Ning, W. Fan, and Q. Li (2025) MKG-rag: multimodal knowledge graph-enhanced rag for visual question answering. arXiv preprint arXiv:2508.05318. Cited by: §1. H. Zhang, S. Zhao, Z. Zhou, W. Zhang, and Y. Meng (2025a) Domain-specific rag with semantic normalization and contrastive feedback for document question answering. In 2025 7th International Conference on Internet of Things, Automation and Artificial Intelligence (IoTAAI), p. 750–753. Cited by: §1. H. Zhang, T. Feng, and J. You (2025b) Graph of records: boosting retrieval augmented generation for long-context summarization with graphs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 23780–23799. Cited by: §C.2. K. Zhang, W. Yu, Z. Sun, and J. Xu (2025c) Syler: a framework for explicit syllogistic legal reasoning in large language models. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, p. 4117–4127. Cited by: §1. M. Zhang, Y. Tang, and P. Team (2025d) PageIndex: next-generation vectorless, reasoning-based rag. pageindex blog (september 2025). Cited by: §B.2, §C.1, §C.1, §4.1.2, §5.1. Q. Zhang, S. Chen, Y. Bei, Z. Yuan, H. Zhou, Z. Hong, H. Chen, Y. Xiao, C. Zhou, J. Dong, et al. (2025e) A survey of graph retrieval-augmented generation for customized large language models. arXiv preprint arXiv:2501.13958. Cited by: §1. Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, et al. (2025f) Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §B.4, §4.1.4. Y. Zhao, Y. Long, H. Liu, R. Kamoi, L. Nan, L. Chen, Y. Liu, X. Tang, R. Zhang, and A. Cohan (2024) DocMath-eval: evaluating math reasoning capabilities of llms in understanding long and specialized documents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16103–16120. Cited by: §2.2. J. Zheng, Y. Chen, Z. Zhou, C. Peng, H. Deng, and S. Yin (2025) Information-constrained retrieval for scientific literature via large language model agents. In 2025 6th International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), p. 366–370. Cited by: §1. F. Zhu, W. Lei, F. Feng, C. Wang, H. Zhang, and T. Chua (2022) Towards complex document understanding by discrete reasoning. In Proceedings of the 30th ACM International Conference on Multimedia, p. 4857–4866. Cited by: §B.1. Appendix A Preliminary Study A.1. Definitions To facilitate the presentation of our method, we first formalize the core components of our knowledge representation: Concept (c) and Entity (e): A concept c denotes a high-level category that abstracts related document entities (e.g., Dataset). An entity e denotes a concrete instance mentioned in the document (e.g., Qasper). Each entity e is assigned to exactly one concept through a concept assignment function ϕ(e)φ(e). Schema (s) and Fact (f): A schema s=(ch,r,ct)s=(c_h,r,c_t) describes a concept-level relational pattern, where chc_h and ctc_t denote the head and tail concepts, and r denotes the relation between them (e.g., (Method,evaluated_on,Dataset)(Method,evaluated\_on,Dataset)). A fact f=(eh,r,et)f=(e_h,r,e_t) is an entity-level instantiation of a schema, where ehe_h and ete_t denote the head and tail entities connected by the relation r (e.g., (HVM-GraphRAG, evaluated on, Qasper)). A fact is valid under schema s when ehe_h and ete_t are assigned to the corresponding concepts chc_h and ctc_t. The modality set is defined as Ω=text,table,image =\text,table,image\. Chunk (b): A chunk b denotes a basic content unit extracted from a complex document. Unlike text-only QA settings, a chunk in complex document QA may contain content from different modalities, such as text, tables or images. Each chunk is represented as b=(x,m)b=(x,m), where x denotes the chunk content and m denotes its modality type (e.g., m=Tablem=Table). A.2. Problem Statement We study question answering over complex documents. Formally, a document D consists of a sequence of N pages, denoted as D=Pii=1ND=\P_i\_i=1^N, where PiP_i is the i-th page. These pages contain a set of M multimodal chunks ℬ=bjj=1MB=\b_j\_j=1^M, where each chunk bjb_j represents a document unit, such as a text segment, table, or figure. A QA system S maps the document and a query q to an answer A: A=(D,q).A=S(D,q). Different from QA over short passages, complex document QA often requires evidence distributed across multiple chunks, pages, and modalities. This motivates a two-stage GraphRAG process that organizes distributed document evidence offline and retrieves query-relevant evidence online. Offline Graph Structure Construction. In the offline stage, the document is transformed into a graph structure for efficient evidence organization. Formally, the construction process produces a graph =(,ℰ)G=(V,E) and an evidence index ℐ∗I^*: (,ℐ∗)=GraphConstructor(D),(G,I^*)=GraphConstructor(D), where V contains nodes, ℰE encodes their relations, and ℐ∗I^* maps graph nodes to their supporting multimodal chunks in ℬB. The evidence index serves as the bridge between the graph structure and the original document evidence. Online Graph-Guided Retrieval and Answering. In the online stage, the system uses the constructed graph and evidence index to answer a user query q. Instead of retrieving directly from all document chunks, the retriever first localizes query-relevant graph nodes and then expands them to supporting chunks through ℐ∗I^*: ℬq=Retriever(q,,ℐ∗).B_q=Retriever(q,G,I^*). The final answer is generated based on the query and the retrieved evidence: A=LLM(q,ℬq).A=LLM(q,B_q). This graph-guided process aims to improve answer accuracy by retrieving semantically connected evidence. Table 3. Statistics of the datasets. Dataset MMLongBench M3DocVQA Qasper Questions 669 633 640 Documents 85 500 192 Avg. Pages 42.16 8.52 10.95 Avg. Images 25.92 3.51 3.43 Cross-Modal Questions (%) 22.73 0.00 8.31 Tokens 2,816,155 3,553,774 2,265,349 Metrics EM, F1 EM, F1 Acc, F1 Algorithm 1 Holistic-View-Guided Graph Construction 1:Document D; multimodal chunks ℬ=bi=(xi,mi)i=1MB=\b_i=(x_i,m_i)\_i=1^M; extraction prompt ρextmi _ext^m_i; detection prompt ρdet _det; resolution prompt ρres _res 2:Concept-level indexing graph con=(con,ℰcon)G_con=(V_con,E_con); evidence index ℐcon(⋅)I_con(·) 3:Stage I: Document Tree Construction 4:=(,ℰ)←TreeBuilder⁡(ℬ,D)T=(N,E_T) (B,D) 5:for all bi=(xi,mi)∈ℬb_i=(x_i,m_i) do 6: hi←Path⁡(root,bi)h_i _T(root,b_i) ⊳ Obtain document-level structural context 7:end for 8:Stage I: Holistic-View-Guided Multimodal Knowledge Extraction 9:Initialize holistic view ℳ0←∅M^0← and evidence index ℐfac(⋅)I_fac(·) 10:for i=1i=1 to M do 11: (i,i,i,ℱi)←Extractormi⁡(hi,xi,ρextmi)(C_i,U_i,S_i,F_i) _m_i(h_i,x_i, _ext^m_i) ⊳ Extract concepts, entities, schemas, and facts 12:Stage I: Conflict Detection and Resolution 13: ℱi+←ℱi∪ℳi−1F_i^+ _i ^i-1 14: Construct ℱconfiF_conf^i from ℱi+F_i^+ using matching rules ⊳ Same head-relation, same head-tail, or same head 15: ℱ^confi←Detector⁡(g∣g∈ℱconfi,ρdet) F_conf^i (\g g _conf^i\, _det) 16: for all g∈ℱ^config∈ F_conf^i do 17: g←eh,et∣(eh,r,et)∈gZ_g←\e_h,e_t (e_h,r,e_t)∈ g\ 18: ℬg←⋃e∈gℐfac(e)B_g← _e _gI_fac(e) ⊳ Collect supporting chunks 19: end for 20: ℳi,ℐfaci←Resolver⁡(ℳi−1,ℐfaci−1,ℱi,(g,ℬg)∣g∈ℱ^confi,bi,ρres)M^i,I_fac^i (M^i-1,I_fac^i-1,F_i,\(g,B_g) g∈ F_conf^i\,b_i, _res) ⊳ Keep, merge, revise, or remove facts 21:end for 22:Stage IV: Entity-Level Bridging to Concept-Level Indexing 23:Construct entity-level fact graph from the resolved holistic view ℳM 24:Build concept nodes conV_con according to concept assignments 25:Build concept-level edges ℰconE_con according to schemas and resolved facts 26:for all z∈conz _con do 27: ℐcon(z)←⋃e∈ϕ−1(z)ℐfac(e)I_con(z)← _e∈φ^-1(z)I_fac(e) ⊳ Create direct concept-to-chunk indices 28:end for 29:con←(con,ℰcon)G_con←(V_con,E_con) 30:return con,ℐcon(⋅)G_con,I_con(·) Appendix B Experimental Detail B.1. Dataset Details We evaluate HVM-GraphRAG on three complex document QA (Zhu et al., 2022; Nagori et al., 2025; Sarmah et al., 2024; Scaffidi et al., 2025) datasets: MMLongBench (Ma et al., 2024), M3DocVQA (Cho et al., 2024), and Qasp-er (Dasigi et al., 2021). These datasets cover different document domains and QA scenarios, including long-form reports, multimodal web-style documents, and scientific papers. Following prior work (Wang et al., 2025c), we use the exact processed versions of MMLongBench, M3DocVQA, and Qasper released by the authors, including the same document collection, supplementary questions, and evaluation splits. We introduce no additional data filtering or question augmentation. The statistics of the three datasets are summarized in Table 3. Algorithm 2 Graph-Guided Holistic Retrieval 1:Query q; concept-level graph con=(con,ℰcon)G_con=(V_con,E_con); evidence index ℐcon(⋅)I_con(·); chunk set ℬB; modality set Ω=text,table,image =\text,table,image\; hyperparameters kc,θ,kr,K,kb,kmm∈Ωk_c,θ,k_r,K,k_b,\k_m\_m∈ ; answer prompt ρans _ans 2:Answer A 3:Stage I: Concept Anchor Retrieval 4:for all c∈conc _con do 5: s(q,c)←sim⁡(Enc⁡(q),Enc⁡(c))s(q,c) (Enc(q),Enc(c)) 6:end for 7:q←c∣(c,s(q,c))∈TopKkc⁡(con;s),s(q,c)≥θA_q←\c (c,s(q,c)) _k_c(V_con;s),\ s(q,c)≥θ\ ⊳ Select high-confidence concept anchors 8:Stage I: Similarity-Based Graph Propagation 9:for all r appearing in ℰconE_con do 10: s(q,r)←sim⁡(Enc⁡(q),Enc⁡(r))s(q,r) (Enc(q),Enc(r)) 11:end for 12:ℛq←TopKkr⁡(r;s)R_q _k_r(\r\;s) 13:ℛ^q←r∈ℛq∣∃(ch,r,ct)∈ℰcon,ch,ct∩q≠∅ R_q←\r _q ∃(c_h,r,c_t) _con,\c_h,c_t\ _q≠ \ ⊳ Keep relations connected to anchors 14:^q←q∪ch,ct∣(ch,r,ct)∈ℰcon,r∈ℛ^q,ch,ct∩q≠∅ A_q _q∪\c_h,c_t (c_h,r,c_t) _con,r∈ R_q,\c_h,c_t\ _q≠ \ ⊳ Expand to related concepts 15:Stage I: Graph-Guided Evidence Access 16:ℬqgraph←⋃z∈^qℐcon(z)B_q^graph← _z∈ A_qI_con(z) ⊳ Directly access supporting chunks 17:for all b∈ℬqgraphb _q^graph do 18: s(q,b)←sim⁡(Enc⁡(q),Enc⁡(Repmb⁡(b)))s(q,b) (Enc(q),Enc(Rep_m_b(b))) 19:end for 20:ℬqK←TopKK⁡(ℬqgraph;s)B_q^K _K(B_q^graph;s) 21:Stage IV: Conventional RAG Supplementation 22:for all b∈ℬb do 23: s(q,b)←sim⁡(Enc⁡(q),Enc⁡(Repmb⁡(b)))s(q,b) (Enc(q),Enc(Rep_m_b(b))) 24:end for 25:ℬqrag←TopKkb⁡(ℬ;s)B_q^rag _k_b(B;s) 26:ℬqfinal←ℬqK∪ℬqragB_q^final _q^K _q^rag ⊳ Merge graph-guided and local evidence 27:Stage V: Modality-Aware Evidence Organization 28:for all m∈Ωm∈ do 29: ℬqm←b∈ℬqfinal∣mb=mB_q^m←\b _q^final m_b=m\ 30: ℬqm←TopKkm⁡(ℬqm;s)B_q^m _k_m(B_q^m;s) 31:end for 32:ℬqmaeo←⋃m∈ΩℬqmB_q^maeo← _m∈ B_q^m 33:A←Answer⁡(q,ℬqmaeo,ρans)A (q,B_q^maeo, _ans) 34:return A • MMLongBench: This is a long-document benchmark designed to evaluate multimodal understanding and QA over lengthy documents. It contains documents from diverse domains, including guidebooks, financial reports, and industry documents. The documents are typically long and contain rich multimodal content, making the dataset suitable for evaluating long-range evidence localization and aggregation. • M3DocVQA: This is a multimodal document question answering dataset constructed from Wikipedia-style HTML documents. It contains heterogeneous document elements such as text, tables, images, captions, and layout structures. The dataset evaluates whether a model can retrieve and QA over multimodal evidence distributed across structured document pages. • Qasper: This is a question answering dataset built from scientific papers. Its questions are information-seeking and often require evidence from different sections of a paper, such as the method, experiment, and result sections. We use Qasper to evaluate QA over scientific papers where supporting evidence may be distributed across long textual contexts and multiple document sections. B.2. Baselines We compare HVM-GraphRAG with representative methods from four RAG categories. Conventional RAG first extracts textual content and divides it into retrieval chunks. This category includes BM25 (Robertson and Walker, 1994) and VanillaRAG. BM25 performs sparse retrieval through lexical matching. VanillaRAG retrieves fixed-size chunks using dense semantic representations. Layout+Vanilla further employs document layout analysis for semantic chunking. Graph-Based RAG extracts textual content and exploits graph structures during retrieval. RAPTOR (Sarthi et al., 2024) recursively clusters and summarizes text into a hierarchical structure. GraphRAG (Edge et al., ) is evaluated with two search strategies. GraphRAG-Local searches local entity neighborhoods. GraphRAG-Global retrieves information from graph-community summaries. Layout-Segmented RAG partitions documents into structural units through layout analysis. M-Vanilla retrieves visual and textual units using multimodal embeddings. Tree-Traverse is inspired by PageIndex (Zhang et al., 2025d) and uses an LLM to navigate the document tree. DocETL (Shankar et al., 2024) provides a declarative framework for complex document processing. Multimodal Graph-Structured RAG includes GraphRanker and BookRAG (Wang et al., 2025c). Graph-Ranker extends HippoRAG (Gutiérrez et al., 2024) and applies Personalized PageRank to rank relevant graph nodes. BookRAG organizes multimodal document content into a hierarchical tree and adaptively executes retrieval operators for different queries. • BM25: BM25 is a sparse retrieval method based on lexical matching. It ranks document chunks according to term-frequency and inverse-document-frequency statistics, and retrieves the top-ranked chunks as evidence for answer generation. • Vanilla RAG: Vanilla RAG first divides the document into fixed-size textual chunks and encodes them with a dense retriever. Given a query, it retrieves the most semantically similar chunks and feeds them to the generation model. • Layout+Vanilla: Layout+Vanilla extends Vanilla RAG by incorporating document layout information during chunk construction. Instead of using only fixed-size text segmentation, it forms more coherent retrieval units based on layout-aware document parsing. • RAPTOR: RAPTOR recursively clusters textual chunks and summarizes them into a hierarchical tree structure. During retrieval, it searches over this hierarchy to obtain evidence at different levels of abstraction. • GraphRAG-Local: GraphRAG-Local constructs an entity-centric graph from documents and performs local search around query-relevant entities. It retrieves evidence from the neighborhoods of matched entities for answer generation. • GraphRAG-Global: GraphRAG-Global uses graph-community summaries to answer queries that require broader document-level information. It retrieves relevant community-level summaries rather than directly searching local entity neighborhoods. • M-Vanilla: M-Vanilla extends vanilla dense retrieval to multimodal document chunks. It encodes textual and visual units with multimodal representations and retrieves chunks according to query–chunk similarity. • Tree-Traverse: Tree-Traverse organizes document content into a hierarchical document tree and uses an LLM to navigate the tree. The retrieved nodes are then used as evidence for final answer generation. • DocETL: DocETL is a declarative document processing framework for complex document analysis. It transforms document content into structured intermediate representations before retrieval and answer generation. • GraphRanker: GraphRanker is adapted from HippoRAG, which performs graph-based retrieval over entity-centric knowledge structures. To handle complex multimodal documents, GraphRanker extends this idea by constructing a multimodal document graph and applying Personalized PageRank to rank query-relevant graph nodes for evidence retrieval. • BookRAG: BookRAG organizes multimodal document content into a hierarchical structure and adaptively selects retrieval operators for different queries. It combines document-structure modeling with multimodal evidence retrieval for complex document question answering. B.3. Evaluation Metrics In this section, we describe the evaluation metrics used in our experiments. Following prior work (Wang et al., 2025c), we evaluate the generated answers with Accuracy, Exact Match (EM), and token-level F1-score. Since RAG systems usually generate free-form natural language responses, we first extract and normalize the predicted answer before computing the final scores. B.3.1. Answer Extraction and Normalization The outputs of RAG-based systems are often natural language sentences rather than short answer spans. For example, a model may generate a response such as “The answer is C” or “According to the table, the value is 50.19”. Directly comparing such responses with concise ground-truth answers may underestimate model performance. Therefore, we follow the standard evaluation setting and apply an answer extraction step before metric calculation. Let yirawy_i^raw denote the raw response generated by the model for the i-th query, and let yigoldy_i^gold denote the corresponding ground-truth answer. We use an LLM-based extractor to obtain the concise predicted answer: (27) y^i=LLMextract⁡(yiraw,ρext), y_i=LLM_extract(y_i^raw, _ext), where ρext _ext denotes the extraction instruction, and y^i y_i is the extracted answer used for metric computation. After extraction, we apply a normalization function ℒ(⋅)NL(·) to both predictions and ground-truth answers. The normalization process includes common text processing operations such as lowercasing, removing punctuation, and eliminating redundant whitespace. B.3.2. Accuracy Accuracy is used as an inclusion-based soft matching metric. A prediction is considered correct if the normalized ground-truth answer is contained in the normalized model response. This setting is suitable for free-form LLM outputs, where the generated response may include additional explanatory text around the answer. Formally, Accuracy is computed as: (28) Acc=1N∑i=1N(ℒ(yigold)⊆ℒ(yiraw)), = 1N _i=1^NI (NL(y_i^gold) (y_i^raw) ), where N is the number of questions, (⋅)I(·) is the indicator function, and ⊆ denotes the substring inclusion relation. For Qasper, we report inclusion-based Accuracy as part of our unified evaluation protocol. B.3.3. Exact Match Exact Match (EM) is a stricter metric than Accuracy. It measures whether the normalized extracted answer is exactly the same as the normalized ground-truth answer. Given the extracted answer y^i y_i and the gold answer yigoldy_i^gold, EM is computed as: (29) EM=1N∑i=1N(ℒ(y^i)=ℒ(yigold)). = 1N _i=1^NI (NL( y_i)=NL(y_i^gold) ). This metric requires character-level equivalence after normalization and is mainly used for datasets with short and unambiguous answers. B.3.4. F1-score F1-score measures token-level overlap between the extracted answer and the ground-truth answer. We first tokenize the normalized extracted answer and the normalized ground truth into token sets Ty^iT_ y_i and TgoldiT_gold_i. Precision and recall are then computed as: (30) Pi=|Ty^i∩Tgoldi||Ty^i|,Ri=|Ty^i∩Tgoldi||Tgoldi|. P_i= |T_ y_i∩ T_gold_i||T_ y_i|, R_i= |T_ y_i∩ T_gold_i||T_gold_i|. The F1-score for the i-th sample is: (31) F1i=2PiRiPi+Ri. 1_i= 2P_iR_iP_i+R_i. If both precision and recall are zero, we set F1i=0F1_i=0. The final F1-score is averaged over all questions: (32) F1=1N∑i=1NF1i. 1= 1N _i=1^NF1_i. In our experiments, EM and F1 are reported for MMLongBench and M3DocVQA, while Accuracy and F1 are reported for Qasper following their evaluation protocols. B.4. Implementation Details For a fair comparison, all methods are evaluated under the same retrieval and generation setting whenever applicable. We use Qwen3-Embedding-0.6B (Zhang et al., 2025f) as the embedding model to encode user queries, document chunks, concept nodes, and relations. All retrieved evidence is finally provided to the same answer-generation model, Qwen3-8B-AWQ (Yang et al., 2025a), to ensure that performance differences mainly come from retrieval and evidence organization rather than generation capability. The same Qwen3-8B-AWQ is also used for answer extraction during evaluation. Document Processing. We use Mineru (Wang et al., 2024) to parse the input documents and extract layout-aware multimodal chunks. Each chunk is associated with its content, modality type, page position, and document structural information. The extracted chunks are further organized into a document tree according to title hierarchy, section structure, and reading order. This document tree provides the structural context used during graph construction. Table 4. Hyperparameter settings of HVM-GraphRAG on different datasets. Dataset kck_c θ krk_r K kbk_b ktextk_text ktablek_table kimagek_image MMLongBench 10 0.25 10 10 2 5 3 2 M3DocVQA 10 0.50 15 10 4 5 3 2 Qasper 15 0.25 10 10 5 5 3 2 Graph Construction. During offline graph construction, Qwen3-8B-AWQ is used to extract concepts, entities, schemas, and facts from textual chunks. For visual chunks, such as figures and images, we use Qwen2.5-VL (Wu et al., 2025a) as the visual-language extractor. The extracted knowledge is incrementally inserted into the holistic view. For each newly processed chunk, the system constructs potential conflict groups by comparing newly extracted facts with previously accepted facts. Conflict detection and resolution are performed by Qwen3-8B-AWQ. The temperature is set to 0 for all LLM and VLM calls to ensure deterministic outputs. Online Retrieval. For concept anchor retrieval, we retrieve the top-kck_c concept nodes from the concept-level graph and filter low-confidence anchors using the similarity threshold θ. We set kc=10k_c=10 for MMLongBench, kc=15k_c=15 for Qasper, and kc=10k_c=10 for M3DocVQA. The threshold θ is set to 0.250.25 for MMLongBench and Qasper, and 0.50.5 for M3DocVQA. For similarity-based graph propagation, we retrieve the top-krk_r relations according to their semantic similarity to the query and retain only relations connected to the selected concept anchors. We set kr=10k_r=10 for MMLongBench and Qasper, and kr=15k_r=15 for M3DocVQA. The graph-guided retrieval stage maps selected concept nodes to their supporting chunks through the evidence indexing function. The top-K graph-guided chunks are retained with K=10K=10 for all datasets. For conventional RAG supplementation, we retrieve the top-kbk_b chunks from the original chunk set, where kb=2k_b=2 for MMLongBench, kb=5k_b=5 for Qasper, and kb=4k_b=4 for M3DocVQA. The graph-guided chunks and supplementary chunks are merged and organized by modality. For all datasets, we retain 55 text chunks, 33 table chunks, and 22 image chunks in the final modality-aware evidence context. The detailed hyperparameter settings for each dataset are summarized in Table 4. Baseline Settings. For all baseline methods, we follow their original retrieval procedures when available. To ensure fairness, all methods use the same embedding model and answer-generation model as HVM-GraphRAG. For methods requiring top-k retrieval, we set k=10k=10 unless otherwise specified. For graph-based baselines, including RAPTOR, GraphRAG-Local, GraphRAG-Global, and GraphRanker, we use the same document inputs and generation model as HVM-GraphRAG. For multimodal and layout-aware baselines, including M-Vanilla, Tree-Traverse, DocETL, and BookRAG, we use the same parsed document chunks whenever possible. Generation and Reproducibility. We do not impose a maximum output length for answer generation. The temperature is set to 0 for all generation, extraction, and conflict-resolution calls. All experiments are conducted on a machine equipped with an Intel(R) Xeon(R) Gold 5120 CPU and four NVIDIA L40 GPUs with 48 GB memory each. All reported results are obtained under the same dataset splits and evaluation protocols described in the main paper. Table 5. Performance comparison with different graph constructors and retrievers. GraphConstructor Retriever MMLongBench M3DocVQA Qasper EM F1-score EM F1-score Accuracy F1-score GraphRAG-Local GraphRAG-Local 7.7 8.5 23.7 25.6 35.9 39.2 HVM-GraphRAG GraphRAG-Local 9.2 10.3 25.5 27.2 37.8 40.9 GraphRAG-Global GraphRAG-Global 5.3 5.6 20.2 22.0 24.0 24.1 HVM-GraphRAG GraphRAG-Global 6.0 6.7 20.8 23.1 25.2 25.6 GraphRanker GraphRanker 21.2 22.7 43.0 47.8 32.9 37.6 HVM-GraphRAG GraphRanker 22.1 24.0 43.4 47.9 34.3 39.4 BookRAG BookRAG 43.8 44.9 61.0 66.2 55.2 61.1 HVM-GraphRAG BookRAG 44.5 46.3 61.6 66.6 56.2 62.0 HVM-GraphRAG HVM-GraphRAG 54.1 62.8 63.0 66.0 66.1 71.5 B.5. Efficiency Comparison Fig. 8 reports the total inference time and token consumption of all methods on MMLongBench, Qasper, and M3DocVQA. The results provide a more complete view of the efficiency differences among conventional RAG, graph-based RAG, layout-segmented RAG, multimodal GraphRAG, and HVM-GraphRAG. Graph-based methods introduce additional online retrieval overhead. Compared with conventional RAG methods such as BM25, Vanilla RAG, and Layout+Vanilla, graph-based methods usually require more inference time. This trend is especially clear for GraphRAG-Global and GraphRanker across the three datasets. The main reason is that these methods need to perform graph-related operations during retrieval, including entity matching, graph traversal, neighborhood expansion, community-level search, or graph ranking. Although such operations can improve evidence connectivity, they also increase the computational cost of online retrieval. RAPTOR is relatively more efficient than other graph-based methods, but it still requires hierarchical clustering or summary-based retrieval, which makes it less lightweight than simple chunk retrieval. Figure 8. Comparison of query efficiency. Layout-segmented methods can also be costly. Layout-segm-ented methods process document structures more explicitly, but this often leads to more retrieval units or richer intermediate representations. As shown in Fig. 8, DocETL incurs particularly high token consumption on all three datasets. This is because structured document transformation and declarative processing introduce additional intermediate text and reasoning steps. Tree-Traverse also requires more inference time than conventional RAG in most settings, since it relies on LLM-based navigation over document hierarchies. These results indicate that layout information is useful for complex document QA, but processing such structures without compact retrieval control may introduce substantial efficiency overhead. BookRAG reduces part of the graph cost but remains expensive. BookRAG achieves stronger efficiency than the most expensive graph-based or layout-segmented methods in some cases, but its cost is still higher than conventional RAG. This is because BookRAG still relies on hierarchical document traversal and adaptive retrieval operators during online inference. Such adaptive retrieval improves flexibility, but it also requires extra LLM calls and increases token consumption. Therefore, BookRAG is more efficient than some dense graph traversal methods, but it is still not as lightweight as conventional retrieval. HVM-GraphRAG maintains low time and token cost across datasets. Across MMLongBench, Qasper, and M3DocVQA, HVM-GraphRAG shows query time and token consumption comparable to conventional RAG methods. This efficiency comes from two design choices. First, HVM-GraphRAG performs retrieval over a compact concept-level graph rather than a dense entity-level graph, which reduces the cost of graph matching and traversal. Second, once concept nodes are selected, their supporting chunks are directly accessed through the evidence index, avoiding repeated expansion over entity-level nodes. Moreover, HVM-GraphRAG does not require additional LLM-based query routing or adaptive query analysis during retrieval. These designs allow HVM-GraphRAG to preserve the benefit of graph-guided evidence modeling while keeping online retrieval cost close to flat retrieval methods. Overall, the efficiency results support the motivation of HVM-GraphRAG. Existing graph-based and layout-aware methods often improve evidence organization at the cost of higher inference time or token consumption. In contrast, HVM-GraphRAG uses a compact concept-level indexing graph to achieve efficient graph-guided retrieval, leading to a better balance between QA performance and computational cost. B.6. Generalization of the Graph Constructor To further examine the generality of our graph construction strategy, we replace the original graph constructors of several graph-based baselines with the graph construction component of HVM-GraphRAG. It is worth noting that HVM-GraphRAG ultimately builds a concept-level indexing graph, while these baseline methods mainly operate on entity-level graphs. For a fair and compatible comparison, we only use the entity-level graph construction part of HVM-GraphRAG to replace the original graph constructors of the baselines, while keeping their original retrievers unchanged. The results are reported in Table 5. Our graph constructor consistently improves different graph-based retrievers. Replacing the original graph constructor with our construction module improves all baseline retrievers across the three datasets. For GraphRAG-Local, the performance increases from 7.7%/8.5% to 9.2%/10.3% on MMLongBench, from 23.7%/25.6% to 25.5%/27.2% on M3DocVQA, and from 35.9%/39.2% to 37.8%/40.9% on Qasper. Similar improvements are observed for GraphRAG-Global, GraphRanker, and BookRAG. These results suggest that the holistic-view-guided construction process can produce more reliable entity-level graph structures, even when used with retrievers that are not specifically designed for our framework. The edge degree distribution Figure 9. Effects of the similarity threshold θ and the number of supplementary chunks kbk_b on MMLongBench, M3DocVQA and Qasper. Darker colors indicate higher performance. The edge degree distribution Figure 10. Effects of the number of retrieved concepts kck_c on MMLongBench, M3DocVQA and Qasper. The gains are more visible for retrievers that directly depend on local graph structures. GraphRAG-Local benefits more clearly from our graph constructor than GraphRAG-Global. This is reasonable because GraphRAG-Local retrieves evidence by expanding neighborhoods around matched entities, making it sensitive to the quality of local entity links. In contrast, GraphRAG-Global relies on community-level summaries, where local graph improvements may be partially smoothed by aggregation. For example, on Qasper, GraphRAG-Local improves by 1.9% in Accuracy and 1.7% in F1-score, while GraphRAG-Global improves by 1.2% in Accuracy and 1.5% in F1-score. The improvement on stronger multimodal baselines is smaller but still consistent. For GraphRanker and BookRAG, replacing the graph constructor also brings stable gains. GraphRanker improves from 21.2%/22.7% to 22.1%/24.0% on MMLongBench and from 32.9%/37.6% to 34.3%/39.4% on Qasper. BookRAG further improves from 43.8%/44.9% to 44.5%/46.3% on MMLongBench, from 61.0%/66.2% to 61.6%/66.6% on M3DocVQA, and from 55.2%/61.1% to 56.2%/62.0% on Qasper. These results indicate that our graph construction module can provide cleaner graph evidence even for methods that already consider document structure or graph-based ranking. The full HVM-GraphRAG still achieves the best overall performance. Although replacing the graph constructor improves existing baselines, the complete HVM-GraphRAG remains substantially stronger. For example, on MMLongBench, the best constructor-replaced baseline achieves 44.5 EM and 46.3% F1-score, while the full HVM-GraphRAG reaches 54.1% EM and 62.8% F1-score. On Qasper, the strongest constructor-replaced baseline obtains 56.2% Accuracy and 62.0% F1-score, whereas HVM-GraphRAG achieves 66.1% Accuracy and 71.5% F1-score. This gap shows that better graph construction alone is helpful but not sufficient. The full advantage of HVM-GraphRAG comes from combining holistic-view-guided graph construction, concept-level indexing, graph-guided retrieval, and modality-aware evidence organization. Overall, this experiment demonstrates that the proposed graph construction strategy has good generality across different graph-based retrieval frameworks. At the same time, the superior performance of the full model confirms that reliable graph construction and compact concept-level retrieval are complementary components for complex document QA. B.7. Detailed Hyperparameter Analysis We provide a more detailed analysis of the retrieval hyperparameters in HVM-GraphRAG. Fig. 9 shows the joint effect of the similarity threshold θ and the number of supplementary chunks kbk_b. Fig. 10 shows the effect of the number of retrieved concept nodes kck_c. These hyperparameters control different aspects of online retrieval: θ controls the strictness of concept anchors filtering, kbk_b controls the amount of complementary evidence from conventional RAG, and kck_c controls the coverage of concept-level retrieval. Figure 11. Case Study of Conflict Resolution. Effects of θ. The optimal threshold varies across datasets, reflecting their different evidence distributions. On MMLongBench, lower thresholds generally lead to better performance. The strongest results appear around θ=0.25θ=0.25, especially when kb=2k_b=2. This indicates that long documents benefit from retaining more concept anchors, because useful evidence may be distributed across multiple sections and graph regions. A strict threshold may discard weakly matched but still useful concept anchors before graph-guided retrieval begins. On Qasper, a similar trend can be observed: lower thresholds such as θ=0θ=0 and θ=0.25θ=0.25 often achieve stronger performance, especially when more supplementary chunks are included. This suggests that scientific documents also benefit from broader concept anchors coverage, since supporting evidence may be distributed across different sections of a paper. In contrast, M3DocVQA performs better under higher thresholds, especially around θ=0.5θ=0.5 to θ=1.0θ=1.0. This dataset contains more localized visual question answering cases, where strict concept anchors filtering helps avoid unnecessary graph expansion and keeps retrieval focused on highly matched evidence. Effects of kbk_b. The number of supplementary chunks also shows dataset-dependent behavior. On MMLongBench, the best performance is achieved with a small number of supplementary chunks, such as kb=2k_b=2. Increasing kbk_b does not consistently improve performance and may introduce locally similar but less useful chunks in long documents. On Qasper, larger kbk_b values are generally more helpful, with strong performance appearing when kb=5k_b=5 under low thresholds. This suggests that direct query–chunk matching can provide useful complementary evidence for scientific papers. On M3DocVQA, performance remains relatively stable as kbk_b increases, especially under higher thresholds. This indicates that the task is less sensitive to supplementary textual retrieval, probably because many questions are grounded in localized visual or page-level evidence. Overall, kbk_b should be viewed as a knob for balancing local evidence supplementation and retrieval noise. Effects of kck_c. Fig. 10 shows that a moderate number of retrieved concept nodes usually leads to better performance. On MMLongBench, the best result appears around kc=10k_c=10, and performance generally decreases when more concepts are retrieved. This suggests that excessive concept anchors may introduce weakly relevant graph regions in long documents. On M3DocVQA, the performance is relatively stable across different kck_c values, with slightly stronger results around kc=5k_c=5, kc=10k_c=10, kc=40k_c=40, and kc=45k_c=45. This stability indicates that the dataset is less sensitive to concept-anchor coverage, likely because many questions require localized evidence rather than broad concept-level expansion. On Qasper, performance remains stable from kc=5k_c=5 to kc=40k_c=40, but drops when kck_c increases to 4545 or 5050. This suggests that retrieving too many concepts may introduce noisy anchors and weaken retrieval focus. Overall trend. The hyperparameter results show that HVM-GraphRAG benefits from dataset-specific retrieval settings. For long and evidence-distributed documents such as MMLongBench and Qasper, broader concept anchors coverage and a controlled amount of supplementary chunks help improve evidence recall. For M3DocVQA, stricter thresholding is more effective because the required evidence is often more localized. Across datasets, an overly large concept Top-k is not always beneficial, since excessive concept anchors can introduce irrelevant graph regions. These observations support the design of HVM-GraphRAG: graph-guided retrieval should balance evidence coverage, retrieval focus, and local evidence supplementation. B.8. Case Study We provide two qualitative examples to further illustrate how HVM-GraphRAG improves complex document QA. The first case focuses on conflict detection and resolution during graph construction. The second case shows how modality-aware evidence retrieval and organization help the model integrate image, table, and text evidence. B.8.1. Case 1: Conflict Resolution in Graph Construction Fig. 11 illustrates a case where traditional GraphRAG produces an incorrect answer due to unresolved system-assignment conflicts. The input document contains multiple component systems under the broader Rhine-Ruhr Stadtbahn network. One chunk lists several systems, including Düsseldorf Stadtbahn and Essen Stadtbahn. Another chunk places Düsseldorf Stadtbahn, Essen-related public transport, and the queried route/city pair within a similar route-oriented context. When these chunks are processed independently, traditional GraphRAG keeps all extracted facts in the same coarse neighborhood. As a result, the graph incorrectly connects Essen Stadtbahn to the queried Düsseldorf, Neuss route/city entry. Figure 12. Case Study of Multimodal Evidence Integration. During retrieval, the query asks whether Essen Stadtbahn is the system for Düsseldorf, Neuss in the component-system table of Rhine-Ruhr Stadtbahn. Because the graph contains noisy connections among Rhine-Ruhr Stadtbahn, Essen Stadtbahn, Düsseldorf Stadtbahn, and Düsseldorf, Neuss, the retriever activates evidence from both the correct and incorrect system assignments. The model then follows the broad umbrella-system path and incorrectly answers “Yes”, treating Essen Stadtbahn as the queried system. This shows that graph connectivity alone is insufficient when locally extracted facts introduce ambiguous or conflicting evidence associations. In contrast, HVM-GraphRAG uses the Cross-Modal Holistic View during graph construction to compare newly extracted facts with the existing global fact state. For this example, the conflict detector identifies that multiple candidate systems may be associated with the same route or city context. The resolver then checks the supporting local evidence and distinguishes the Düsseldorf Stadtbahn assignment from the broader Rhine-Ruhr umbrella-system membership. Consequently, the incorrect Essen Stadtbahn connection is filtered or weakened, while the verified path from Düsseldorf Stadtbahn to Düsseldorf, Neuss is preserved. During retrieval, the query is anchored to the verified system-route mapping, allowing the model to answer “No” correctly. This case demonstrates that conflict-aware graph construction can reduce noisy evidence indexing and prevent misleading graph activation during retrieval. B.8.2. Case 2: Multimodal Evidence Integration Fig. 12 presents a case where the answer requires combining visual, tabular, and textual evidence. The query asks which player in the scoring leaders of the 2000–01 AHL season plays for the team with a pouncing wild cat on the jersey. Solving this question requires three pieces of evidence: the table identifies the relevant scoring leader and team, the image provides the visual clue of the pouncing wild cat jersey, and the text helps connect the player information with the document context. Traditional GraphRAG mainly relies on textualized information such as image captions, table captions, and related text passages. Although these captions mention Mikael Samuelsson, regular season and playoffs, and related team assignments, they do not expose the actual visual content of the image or the detailed table entries. As a result, the retriever cannot reliably align the visual clue in the jersey with the correct player entry in the table. The evidence remains fragmented across captions and text snippets, leading to an uncertain or incomplete answer that is not fully supported by the original multimodal content. HVM-GraphRAG retrieves and organizes evidence from multiple modalities. The image evidence preserves the visual clue of the jersey, the table evidence provides the scoring-leader row, and the text evidence supplies complementary background information. Through graph-guided retrieval, the relevant image, table, and text chunks are jointly selected. The modality-aware evidence organization then groups the retrieved chunks by modality, making it easier for the answering model to compare the visual clue with the table entry and textual context. This forms a complete reasoning path from the scoring table to the Kentucky Thoroughblades, then to the pouncing wild cat jersey, and finally to Mikael Samuelsson. The model therefore produces the correct answer, “Mikael Samuelsson”. This case shows that HVM-GraphRAG can better preserve and integrate heterogeneous evidence, especially when the answer depends on information that cannot be recovered from captions or text alone. Overall, these case studies support the two main design motivations of HVM-GraphRAG. The conflict-resolution case shows that the holistic view improves the reliability of graph construction and evidence indexing. The multimodal QA case shows that graph-guided retrieval and modality-aware evidence organization help the answering model integrate evidence across modalities. Together, they explain why HVM-GraphRAG achieves more reliable and accurate QA over complex multimodal documents. Appendix C Related Work C.1. RAG on Complex Document Retrieval-Augmented Generation (RAG) (Fang et al., 2026; Wang et al., 2025a; Qian et al., 2025) has become a widely used paradigm for answering questions over external documents (Robertson and Walker, 1994; Zhang et al., 2025d; Shankar et al., 2024). Early RAG methods usually retrieve evidence from text chunks through sparse or dense matching. BM25 (Robertson and Walker, 1994) retrieves relevant chunks based on lexical overlap, while VanillaRAG uses dense semantic representations to match queries with fixed-size document chunks. These methods are simple and efficient, but they often ignore the structural organization of complex documents. To better preserve document semantics, Layout+Vanilla introduces layout-aware chunking before retrieval, which helps form more coherent retrieval units. Recent studies (Zhang et al., 2025d; Shankar et al., 2024) further explore layout-segmented and multimodal document retrieval. M-Vanilla extends dense retrieval to both textual and visual document units using multimodal embeddings. Tree-Traverse follows the document hierarchy and uses an LLM to navigate structured document trees (Zhang et al., 2025d). DocETL (Shankar et al., 2024) provides a declarative framework for processing complex document content before retrieval. Additionally, recent systems further support iterative visual-document retrieval, multimodal evidence fusion, and heterogeneous document reasoning (Wang et al., 2025b; Suri et al., 2025; Yu et al., 2025b). These methods show that document layout and multimodal information are useful for complex document QA. However, they mainly improve how evidence units are segmented or accessed, and still provide limited modeling of semantic relations among distributed evidence units. As a result, they may struggle when answering requires multi-hop aggregation across pages, regions, and modalities. Text Entity Extraction -Goal- Given a text document that is potentially relevant to this activity and a list of entity types, identify all entities of those types from the text. -Steps- 1. Identify all entities. For each identified entity, extract the following information: • entity_name: Name of the entity, capitalized. • entity_type: One of the following types: [entity_types]. • description: A brief, one-sentence description summarizing the entity’s role or attributes based only on the provided text. If the text offers limited information, keep the description simple, e.g., ‘A person mentioned in the text.’ Do not add any external knowledge. 2. Return output in English as a single valid JSON object with this exact schema:"entities": [ "entity_name": "Entity Name", "entity_type": "ENTITY_TYPE", "description": "A short description based only on the text."] 3. Important output rules: • Output only valid JSON. • Do not output markdown code fences. • Do not output any explanation before or after the JSON. • Do not output tuple-style records. • Do not output relationships. • If no entities are found, return: "entities": [] -Example- Entity_types: [PERSON, ORGANIZATION] Text: Yubo Ma 1 , Yixin Cao 2 , YongChing Hong 1 , Aixin Sun 1 Output:"entities": [ "entity_name": "Yubo Ma", "entity_type": "PERSON", "description": "Yubo Ma is a person listed as an author of the document." , ...] -Real Data- Entity_types: entity_types Text: input_text Output: Figure 13. Prompt for extracting typed entities from textual chunks. Table Row Entity Extraction -Goal- Act as a precise, row-by-row information extraction engine. Your task is to analyze a batch of semi-structured row data strings. You must correlate each row’s data with the provided column_headers, using the main description for overall context, to extract all relevant entities found strictly within the row data. -Instructions- 1. Use Context, Don’t Extract From It: First, carefully study the description and column_headers. Use this information strictly as context to understand the row data’s meaning and to assign correct entity types. Do not extract entities that appear only in the description or column_headers. 2. Correlate and Extract from Each Row: Iterate through each row_string in the rows_batch list. For each string: • It represents a single table row, with cells likely separated by a delimiter, e.g., ‘|’. • You must mentally map the data cells in the string to their corresponding header in the column_headers list based on their order. • Extract entities from the row’s categorical data, e.g., names of models, methods, or groups like ‘CODEX’ or ‘SLM + LLM’. You generally should NOT extract entities from purely numerical data, e.g., ‘53.8(0.5)’. • Minimum Extraction Mandate: Each row string describes a primary subject. You must aim to extract at least one main entity from each row, representing the subject of that row, e.g., the specific model or method being evaluated. 3. Consolidate and Format: Collect all unique entities found across all rows into a single list. Your final response must be a single, valid JSON object with a single root key named entities. 4. Strict Extraction Boundaries: This is the final and most important rule. • All extracted entities MUST originate from the data within the rows_batch strings. • Therefore, you MUST NOT extract entities that appear only in the description or column_headers. These are for context only. • You also MUST NOT create an entity for the table itself, e.g., ‘Table 1’. -Output JSON Schema-"entities": [ "entity_name": "<String>", "entity_type": "<String>", "description": "<String>"] Field Descriptions • For an Entity object: • entity_name String: The primary name of the identified entity, capitalized. • entity_type String: The category of the entity. It MUST be one of the following types: entity_types. • description String: A brief, comprehensive description of the entity’s attributes and role in the text. -Example- Input:"description": "Table 1: The inference seconds over 500 sentences (run on single V100 GPU).", "column_headers": [ "Dataset (Task)", "Roberta", "T5", "LLaMA", "CODEX"], "rows_batch": [ "FewNERD (NER)|2.8|39.4|1135.4|179.4", "TACREV (RE)|1.4|45.6|1144.9|151.6", "ACE05 (ED)|6.6|62.5|733.4|171.7"] Output:"entities": [ "entity_name": "FewNERD (NER)", "entity_type": "DATASET_OR_CORPUS", "description": "A Named Entity Recognition dataset used as a benchmark to measure model inference speeds." ,...] -Task Execution- Analyze the following input data using the allowed entity types provided. Your response must be the complete JSON object and nothing else. -Allowed Entity Types- entity_types -Input to Process- input_json -JSON Output- Figure 14. Prompt for extracting typed entities from table rows using table descriptions and column headers as context. C.2. GraphRAG and Multimodal GraphRAG Graph-based RAG methods (Zhang et al., 2025b; Yu et al., 2026a) introduce graph structures to organize document knowledge and support relation-aware retrieval. RAPTOR (Sarthi et al., 2024) recursively clusters and summarizes textual chunks into a hierarchical structure, enabling retrieval at different levels of abstraction. GraphRAG (Edge et al., ) constructs entity-centric graphs from documents and supports both local and global search strategies. GraphRAG-Local retrieves information from neighborhoods around query-relevant entities, while GraphRAG-Global answers queries using graph-community summaries. Compared with flat chunk retrieval, these methods can better capture semantic connections among evidence units. However, their retrieval process often depends on entity matching, graph traversal, or community-level summarization, which can introduce additional cost and may be less effective for multimodal document evidence. Multimodal GraphRAG (Gutiérrez et al., 2024; Wang et al., 2025c) further extends graph-based retrieval to complex documents containing text, tables, figures, and other modalities. GraphRanker (Gutiérrez et al., 2024) adapts graph-based ranking to multimodal document graphs and applies Personalized PageRank to identify relevant graph nodes. BookRAG (Wang et al., 2025c) organizes multimodal document content with hierarchical structures and adaptively selects retrieval operations for different queries. These methods demonstrate the potential of structured retrieval for complex document QA. Nevertheless, existing multimodal graph-based methods usually focus on building or traversing graph structures, while paying less attention to the reliability of links between graph elements and their supporting multimodal chunks. In contrast, our work emphasizes holistic-view-guided graph construction and reliable concept-level evidence indexing, so that graph-guided retrieval can be both accurate and efficient. Appendix D Prompt This section presents the prompt templates used in HVM-GraphRAG. These prompts support two major stages of our framework: multimodal knowledge extraction and holistic-view-guided conflict handling. For knowledge extraction, we design modality-specific prompts for textual chunks, table rows, and image chunks, so that each type of document content can be processed under suitable extraction constraints. For conflict handling, we design prompts for detecting and resolving inconsistent facts during graph construction. All prompts require structured JSON outputs, which enables automatic parsing and reduces uncontrolled free-form generations. D.1. Multimodal Knowledge Extraction Text Entity Extraction Prompt. Fig. 13 shows the prompt used to extract typed entities from textual chunks. Given a text chunk and a predefined list of entity types, the prompt asks the LLM to identify all entities that belong to the allowed types and to generate a short description for each extracted entity. The description is required to be based only on the provided text, preventing the model from introducing external knowledge. The prompt also enforces a strict JSON schema with the key entities, which contains the entity name, entity type, and entity description. If no valid entity is found, the model must return an empty entity list. This prompt is mainly used for text-like chunks, including paragraphs and textual descriptions extracted from complex documents. Table Row Entity Extraction Prompt. Fig. 14 presents the prompt for extracting entities from table rows. Different from ordinary text extraction, table understanding requires the model to interpret each row together with its column headers and table description. Therefore, the prompt provides the table description and column headers as contextual information, while explicitly restricting entity extraction to the row content. This design prevents the model from incorrectly extracting entities that only appear in headers or captions. The prompt further asks the model to map row cells to their corresponding columns and to extract categorical entities such as methods, datasets, systems, or organizations, while avoiding pure numerical values. The output follows the same JSON format as the text entity prompt, making table entities compatible with the later graph construction process. Image Entity Extraction Prompt. Fig. 15 shows the prompt used for extracting entities from visual chunks. For image-based content, the prompt is given to a VLM and requires the model to jointly analyze the image and its accompanying textual description. The prompt first asks the model to include the image itself as an entity, which preserves a direct index from the visual chunk to the graph. It then instructs the model to identify visual objects, labels, annotations, titles, and other meaningful elements appearing in the image. When the accompanying description contains useful context, the model may also use it to clarify the role of visual entities. This prompt allows HVM-GraphRAG to preserve visual information that cannot be fully recovered from captions alone. Relation Extraction Prompt. Fig. 16 gives the prompt for extracting relations among the entities identified from a chunk. Given the chunk content and the extracted entities, the prompt asks the LLM to identify high-confidence directional relations. Each relation is represented as a structured triple with a head entity, relation name, tail entity, and their corresponding entity types. The prompt requires relation names to be normalized using lowercase words with underscores, and it instructs the model to avoid weak, speculative, or unsupported relations. This step converts extracted entities into entity-level facts, which are later used to construct the entity-level fact graph and establish direct indices from concept nodes to their supporting chunks. D.2. Conflict Detection and Resolution Conflict Detection Prompt. Fig. 17 presents the prompt for detecting conflicts among candidate fact groups. During graph construction, newly extracted facts may overlap or conflict with facts already accepted in the Cross-Modal Holistic View. The prompt asks the LLM to examine each candidate group and determine whether the triples truly conflict. It considers three conflict types: mutual conflict, temporal conflict, and granularity conflict. Mutual conflict captures mutually exclusive facts, temporal conflict captures time-dependent facts with incompatible or missing time scopes, and granularity conflict captures facts expressed at different levels of specificity. The prompt requires the model to return whether a conflict exists, the conflicting triple pairs, the conflict type, and a short explanation. This design avoids treating all similar facts as conflicts and helps distinguish actual contradictions from compatible facts. Conflict Resolution Prompt. Fig. 18 shows the prompt used to resolve detected conflicts. Given conflicting triples and their supporting chunks, the prompt asks the LLM to decide whether each triple should be kept, discarded, or modified. For mutual conflicts, the resolver selects the fact better supported by the evidence. For temporal conflicts, it may modify the relation with time information when such evidence is available. For granularity conflicts, it may preserve compatible facts by adding scope information to the relation. The output contains resolved triples, unresolved conflicts, and a brief summary of the resolution process. By using supporting chunks during resolution, this prompt helps HVM-GraphRAG reduce noisy or conflicting graph updates and maintain a more reliable evidence index. Image Entity Extraction -Goal- Act as an expert AI system for visual and semantic analysis. Your primary task is to comprehensively analyze a given image and an accompanying textual description. You must identify all possible relevant entities, formatting the output as a single, valid JSON object that adheres to the provided Pydantic schema. -Steps- 1. Analyze Full Context: Carefully examine both the provided image and the textual description. The description often provides crucial context, names, or other details that are not visually obvious. 2. Identify Entities: • Crucial First Step: The Image Itself. You MUST identify the entire image as the very first entity. Its entity_type must be IMAGE and it must be the first object in the final entities list. This is a non-negotiable rule. If the description provides a title or figure number, e.g., ‘Figure 2’, use that as the entity_name; otherwise, use the first few words of the description as the entity name. If the description is empty, use ‘The Image’ as the entity name. • From Visual Objects: Identify distinct physical objects, people, animals, and general locations shown in the image. • From Text Within the Image High Priority: Pay attention to any text inside the image, such as labels, titles, annotations, or data points in diagrams and flowcharts. This text is a critical source of entities. You MUST treat every distinct label, title, or significant term as a candidate for an entity. • From the Description: Extract any additional relevant entities mentioned in the text that might not be visible or clearly identifiable in the image. • Principle of Comprehensiveness: When in doubt, it is better to extract a potential entity than to omit it. Be thorough and aim for maximum detail. • Create Entity Objects: For each unique entity found, create a JSON object following the ExtractEntity structure. 3. Construct the Final Output: Combine all identified entities into a single JSON object with the root key ‘entities’, ensuring it strictly follows the specified output format. -Output Format- 1. General Instruction Your response MUST be a single, valid JSON object with the root key ‘entities’. 2. JSON Structure "entities": [ "entity_name": "<String>", "entity_type": "<String>", "description": "<String>" ] 3. Field Descriptions • entity_name String: The name of the entity. • entity_type String: The category of the entity. It MUST be one of the following types: entity_types. • description String: A brief, comprehensive summary of the entity’s role and attributes. -Example- Conceptual Input: • Image: A simple diagram showing three boxes. ‘User PC’ has an arrow pointing to ‘Web Server’, which has an arrow pointing to ‘Database’. • Description: ‘Figure 1: A diagram of a basic three-tier web application architecture.’ Correct Output: "entities": [ "entity_name": "Figure 1", "entity_type": "IMAGE", "description": "A diagram illustrating a basic three-tier web application architecture, showing the data flow between components." , "entity_name": "User PC", "entity_type": "SYSTEM_COMPONENT", "description": "The client-tier component in the architecture diagram, representing the end-user’s machine." , ... ] -Task- Now, analyze the provided image and its description. Generate the JSON object according to all the instructions above. -Allowed Entity Types- entity_types -Image description- image_description -JSON Output- Figure 15. Prompt for extracting typed entities from images and accompanying textual descriptions. Relation Extraction You are an expert in knowledge graph construction and relation extraction. Your task is to extract relations between given entities from the provided text. -Instructions- 1. Analyze the text carefully and identify meaningful relations between the provided entities. 2. Only extract relations that are explicitly stated or strongly implied in the text. 3. Use lowercase and underscores for relation names, e.g., ‘works_for’, ‘located_in’. 4. Ensure relations are directional: subject_entity, relation, object_entity. 5. Do not invent relations that are not supported by the text. 6. Return only high-confidence relations. Skip weak, generic, repetitive, or speculative relations. 7. Do not repeat the same relation triple. If the same head, relation, tail appears multiple times, return it only once. 8. Prefer specific factual relations such as affiliation, citation, authorship, part_of, type_of, evaluates, uses, improves, inspired_by, or related_to only when the text strongly supports them. 9. Do not create placeholder entities, inferred aliases, or broad summary relations that are not directly grounded in the text. 10. Return at most 20 distinct relations, ordered from most explicit/high-confidence to least. -Output Format- Return exactly one valid JSON object with this structure: "relations": [ "head": "the subject entity (exactly as provided)", "relation": "the relation type", "tail": "the object entity (exactly as provided)", "head_type": "the entity type/label of the head entity (exactly as provided)", "tail_type": "the entity type/label of the tail entity (exactly as provided)" ] The ‘relations’ value must always be a JSON array. If multiple valid relations exist, include all of them in the array. If no valid relations can be extracted, return:"relations": [] Before returning, deduplicate the array so that each relation triple appears only once. -Example input- Text: Apple Inc. was founded by Steve Jobs in 1976. Entities to consider: • Apple Inc. (ORG) • Steve Jobs (PERSON) • 1976 (DATE) -Example output- "relations": [ "head": "Apple Inc.", "relation": "founded_in", "tail": "1976", "head_type": "ORG", "tail_type": "DATE" , "head": "Steve Jobs", "relation": "founded", "tail": "Apple Inc.", "head_type": "PERSON", "tail_type": "ORG" ] Do not return markdown, explanations, or any keys other than ‘relations’. Figure 16. Prompt for extracting high-confidence relations between given entities from textual chunks. Conflict Detection System: You are an expert fact checker. Given one candidate conflict group of triples, determine whether any triples in the group truly conflict with each other. Your task: Analyze the triples within the group together, compare them pairwise where useful, and detect real conflicts. Classify conflicts into three types: • mutual conflict: mutual exclusivity / one-to-one relations • temporal conflict: time-dependent facts; conflicts arise when time scopes overlap or are missing • granularity conflict: different levels of specificity; may be compatible via containment/hypernymy -Definitions and rules- 1) Mutual conflict, type = ‘mutual’ A mutual conflict happens when: • Same subject and predicate, but different objects, AND the predicate is one-to-one / mutually exclusive. Example: (X, birthplace, Shanghai) vs (X, birthplace, Beijing) • Or cyclic/contradictory relational structure that cannot both be true under common-sense constraints. Example: (A, father, B) vs (B, father, A) 2) Temporal conflict, type = ‘temporal’ A temporal conflict happens when: • The predicate describes a role/state that can change over time and is typically unique at a given moment, e.g., president/CEO/champion/current location. • If both triples claim different objects for the same subject-predicate: – If explicit time scopes exist and overlap → hard temporal conflict. – If time scopes exist and do NOT overlap → not a conflict. – If time scopes are missing but the predicate is time-variant and moment-unique → suspected temporal conflict. Ask for time ranges; do NOT assert a hard conflict without time info. 3) Granularity conflict, type = ‘granularity’ A granularity conflict happens when: • Triples differ due to specificity/abstraction level. Example: (X, birthplace, Shanghai) vs (X, birthplace, China) • If one object is a parent/superset/contains the other, i.e., hypernym/meronym/administrative containment, then it is usually compatible → classify as ‘granularity’. • If objects are incompatible, cannot contain each other, and cannot both be true → logical conflict. -Important instructions- • Only judge conflicts among the triples inside the provided group. • Use the group type and possible conflict reason as hints, not as proof of conflict. • Do not mark a conflict unless it is clearly supported by the triples. • If there is no real conflict in the group, return has_conflict=false. Output MUST be a valid JSON object following the required schema. User: Analyze the following candidate conflict group.Group ID: $group_id Group Type: $group_type Group Scope: $group_scope Possible Conflict Reason: $group_reason Triples in Group: $group_triples Output a JSON object with the following structure: "has_conflict": true/false, "conflicts": [ "triple1_id": "id1", "triple1": ["head", "relation", "tail"], "triple2_id": "id2", "triple2": ["head", "relation", "tail"], "conflict_type": "mutual|temporal|granularity", "conflict_reason": "brief explanation of why these triples conflict" ], "conflicting_triple_ids": ["id1", "id2", ...] Analyze the whole group. If conflicts exist, list all conflicting triple pairs. If has_conflict is false, return empty arrays for conflicts and conflicting_triple_ids. JSON payload: Figure 17. Prompt for detecting conflicts among candidate groups of extracted triples. Conflict Resolution System: You are an expert knowledge graph curator. Given a set of conflicting triples and their source passages, your task is to resolve the conflicts and produce corrected triples. -Conflict Resolution Strategies- 1. Mutual Conflict, type = ‘mutual’: • These are contradictory claims about the same entity, e.g., same subject-predicate but different objects. • Resolution: Analyze the source passages to determine which triple is more accurate. • Keep only the CORRECT triple, discard the incorrect one(s). • If both seem equally valid based on context, prefer the one with more specific/credible source. 2. Temporal Conflict, type = ‘temporal’: • These are time-dependent facts where time scopes overlap or are missing. • Resolution: Add time information to the relation to distinguish the facts. • Modify the predicate to include time context, e.g., ‘was president of [2000-2005]’ vs ‘was president of [2005-2010]’. • If time info is not in sources, note it as ‘temporal_conflict_unresolved’. 3. Granularity Conflict, type = ‘granularity’: • These are facts at different levels of specificity, e.g., ‘born in Shanghai’ vs ‘born in China’. • Resolution: Add granularity description to the relation to clarify the scope. • Modify the predicate to include granularity context, e.g., ‘was born in [city: Shanghai]’ vs ‘was born in [country: China]’. • Both can be kept if they are compatible, i.e., containment relationship. Output MUST be a valid JSON object following the required schema. User: Resolve the following conflicting triples using their source passages.Conflicting Triples and Their Sources: $conflicting_triples_with_sources Output a JSON object with the following structure: "resolved_triples": [ "original_triple": ["head", "relation", "tail"], "triple_id": "fact_id", "conflict_type": "mutual|temporal|granularity", "resolution": "kept|discarded|modified", "resolved_triple": ["head", "modified_relation", "tail"] or null if discarded, "reason": "explanation of why this resolution was chosen" ], "unresolved_conflicts": [ "triple_ids": ["id1", "id2"], "reason": "reason why conflict could not be resolved" ], "summary": "brief summary of how conflicts were resolved" For each conflicting triple: • If resolution is ‘kept’: Keep the triple as is, because it is correct. • If resolution is ‘discarded’: The triple is incorrect, set resolved_triple to null. • If resolution is ‘modified’: Provide the modified triple with time/granularity information in the relation. JSON payload: Figure 18. Prompt for resolving conflicting triples using their source passages.

Instant research discovery