Paper deep dive
Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking
Shuzhi Gong, Richard O. Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, Zhuohan Xie
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/20/2026, 6:26:51 AM
Summary
The paper introduces WKGFC, a multi-sourced, multi-agent fact-checking framework that utilizes an open knowledge graph and web retrieval within a Markov Decision Process (MDP) to verify claims. It addresses limitations of previous RAG methods by using LLM agents to adaptively retrieve structured subgraphs and unstructured web content, optimizing retrieval policies via prompt optimization to improve veracity prediction accuracy.
Entities (10)
Relation Signals (9)
WKGFC → uses → LLM
confidence 95% · LLM-enabled retrieval is designed to assess the claims and retrieve the most relevant knowledge subgraphs... implemented as an automatic Markov Decision Process (MDP): A reasoning LLM agent decides what actions to take
WKGFC → formulatesas → MDP
confidence 92% · The above process is implemented as an automatic Markov Decision Process (MDP): A reasoning LLM agent decides what actions to take according to the current evidence and the claims.
WKGFC → evaluatedon → FEVER
confidence 90% · We evaluate our proposed Web-enhanced Knowledge Graph Fact-Checking (WKGFC) framework across diverse benchmarks, including Wikipedia-based (FEVER [39]...
WKGFC → evaluatedon → HoVer
confidence 90% · We evaluate our proposed Web-enhanced Knowledge Graph Fact-Checking (WKGFC) framework across diverse benchmarks, including Wikipedia-based (FEVER [39], HOVER [19])...
GraphRAG → istypeof → RAG
confidence 90% · More recently, graph-based RAG methods such as GraphRAG [9] combine LLM reasoning with KGs...
WKGFC → retrievesfrom → DBpedia
confidence 88% · our agent first performs KG retrieval by issuing SPARQL queries over open-resource KGs (e.g., Wikidata or DBpedia)
WKGFC → retrievesfrom → Wikidata
confidence 88% · our agent first performs KG retrieval by issuing SPARQL queries over open-resource KGs (e.g., Wikidata or DBpedia)
RAG → addresses → LLM
confidence 85% · A step forward is the introduction of Retrieval-Augmented Generation (RAG) [22], which enriches LLM reasoning with dynamically retrieved external evidence.
WKGFC → optimizesvia → TextGrad
confidence 85% · These structured reflections populate an experience buffer that guides prompt-level optimization using the TextGrad [51] framework.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Misinformation spreading over the Internet poses a significant threat to both societies and individuals, necessitating robust and scalable fact-checking that relies on retrieving accurate and trustworthy evidence. Previous methods rely on semantic and social-contextual patterns learned from training data, which limits their generalization to new data distributions. Recently, Retrieval Augmented Generation (RAG) based methods have been proposed to utilize the reasoning capability of LLMs with retrieved grounding evidence documents. However, these methods largely rely on textual similarity for evidence retrieval and struggle to retrieve evidence that captures multi-hop semantic relations within rich document contents. These limitations lead to overlooking subtle factual correlations between the evidence and the claims to be fact-checked during evidence retrieval, thus causing inaccurate veracity predictions. To address these issues, we propose WKGFC, which exploits authorized open knowledge graph as a core resource of evidence. LLM-enabled retrieval is designed to assess the claims and retrieve the most relevant knowledge subgraphs, forming structured evidence for fact verification. To augment the knowledge graph evidence, we retrieve web contents for completion. The above process is implemented as an automatic Markov Decision Process (MDP): A reasoning LLM agent decides what actions to take according to the current evidence and the claims. To adapt the MDP for fact-checking, we use prompt optimization to fine-tune the agentic LLM.
Tags
Links
- Source: https://arxiv.org/abs/2603.00267v1
- Canonical: https://arxiv.org/abs/2603.00267v1
Trouble viewing inline? Open PDF directly →
Full Text
67,064 characters extracted from source content.
Expand or collapse full text
Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking Shuzhi Gong The University of Melbourne Melbourne, VIC, Australia shuzhig@student.unimelb.edu.au Richard Sinnott The University of Melbourne Melbourne, VIC, Australia rsinnott@unimelb.edu.au Jianzhong Qi The University of Melbourne Melbourne, VIC, Australia jianzhong.qi@unimelb.edu.au Cecile Paris Data61, CSIRO Sydney, NSW, Australia Cecile.Paris@data61.csiro.au Preslav Nakov MBZUAI Abu Dhabi, UAE preslav.nakov@mbzuai.ac.ae Zhuohan Xie MBZUAI Abu Dhabi, UAE zhuohan.xie@mbzuai.ac.ae Abstract Misinformation spreading over the Internet poses a significant threat to both societies and individuals, necessitating robust and scalable fact-checking that relies on retrieving accurate and trust- worthy evidence. Previous methods rely on semantic and social- contextual patterns learned from training data, which limits their generalization to new data distributions. Recently, Retrieval Aug- mented Generation (RAG) based methods have been proposed to utilize the reasoning capability of LLMs with retrieved ground- ing evidence documents. However, these methods largely rely on textual similarity for evidence retrieval and struggle to retrieve evidence that captures multi-hop semantic relations within rich document contents. These limitations lead to overlooking subtle factual correlations between the evidence and the claims to be fact- checked during evidence retrieval, thus causing inaccurate veracity predictions. To address these issues, we propose a Web-enhanced Knowledge Graph retrieval Fact-Checking agentic framework (WKGFC), which exploits authorized open knowledge graph as a core resource of evidence. LLM-enabled retrieval is designed to assess the claims and retrieve the most relevant knowledge subgraphs, forming struc- tured evidence for fact verification. To augment the knowledge graph evidence, we retrieve web contents for completion. The above process is implemented as an automatic Markov Decision Process (MDP): A reasoning LLM agent decides what actions to take accord- ing to the current evidence and the claims. To adapt the MDP for fact-checking, we use prompt optimization to fine-tune the agentic LLM. Our extensive experiments over datasets in three categories (Wikipedia, websites, and article summaries) show thatWKGFCout- perform several advanced state-of-the-art fact-checking methods in balanced accuracy score by over 5% absolute. These results high- light the effectiveness of knowledge-centric evidence retrieval for fact-checking under open-world settings. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. SIGIR, MELBOURNE, AUSTRALIA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/X.X ACM Reference Format: Shuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie. 2026. Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-Checking. In Proceedings of SIGIR. ACM, Melbourne, VIC, Australia, 11 pages. https://doi.org/X.X 1 Introduction The rapid proliferation of misinformation poses a profound threat to public trust, democratic processes, and public health communica- tion [54]. Efforts to combat misinformation have spanned decades. Traditional studies explored content semantics [16, 11], propaga- tion patterns [38, 3], and user activity contexts [6, 12] in a pattern- recognition manner. Such approaches depend heavily on training data distributions and often fail to generalize to unseen domains, as misinformation topics and linguistic styles evolve over time [13]. From an information retrieval perspective, this challenge funda- mentally concerns how to retrieve reliable and sufficient evidence from large, evolving information sources to support verification. Recent advances in Large Language Models (LLMs) have trans- formed automated reasoning and generation capabilities across NLP tasks. Fact-checking has emerged as a key frontier where LLMs can expose misinformation and mitigate its societal harms [4]. Yet, LLMs remain limited by their fixed training windows and lack of integration with domain-specific or time-sensitive knowledge, lead- ing to hallucinated factual errors. A step forward is the introduction of Retrieval-Augmented Generation (RAG) [22], which enriches LLM reasoning with dynamically retrieved external evidence. The ef- fectiveness of RAG-based fact-checking critically depends on the quality of evidence retrieval, making missing supportive evidence and retrieval failures primary sources of verification errors. The RAG framework enables a retrieve–then–verify paradigm: given a claim to be verified, external knowledge is retrieved and presented to the LLM, which then reasons over both the claim and the retrieved evidence to make a prediction on veracity. Figure 1 illustrates this evolution of fact-checking paradigms, highlighting the differences between text-based, graph-based, and our proposed agentic retrieval framework. A lot of work retrieves unstructured textual evidence based on semantic similarity. Systems such as HerO [49] and InFact [32] estab- lished strong RAG baselines by sourcing evidence from large textual knowledge bases. Similar pipelines extend to open-web retrieval [41, 45], i.e., to harness web-scale corpora for evidence gathering. De- spite their success, these methods face semantic ambiguity and arXiv:2603.00267v1 [cs.AI] 27 Feb 2026 SIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIAShuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie ClaimSub-claim Verdict &Justification Evidence Retrieval (1) Unstructured Text-based Paradigm Claim Preprocessing - NER, Entity Linking - KG Query Generation - Pseudo KG Generation Subgraph Retrieval Verdict &Justification (2) Structured KG-based Paradigm Claim Knowledge Graph Retrieval AgentReasoning - Insufficiency ? - Ambiguity? - Web Query Generation - Action (Query/Verdict) Retrieved KGEnhanced KG Web Retrieval & KG Completion Observe Verdict &Justification (2) Our Paradigm - WKGFC Figure 1: Illustration of different fact-checking paradigms; ours is at the bottom. retrieval uncertainty: textual similarity does not guarantee factual relevance between claims and retrieved evidence. Another line of research emphasizes structured reasoning based on knowledge graphs (KGs). Early, graph-based methods [14, 53] leverage Graph Neural Networks (GNNs) for claim verification. They are constrained by limited representational power and depen- dence on manually curated Knowledge Graphs (KG). More recently, graph-based RAG methods such as GraphRAG [9] combine LLM rea- soning with KGs, improving factual consistency assessment in com- plex domains such as medicine [15] and summary verification [5]. Besides, KG question answering frameworks (e.g., ChatKBQA [24] and SymAgent [23]) adopt elaborate retrieval mechanisms to ex- tract subgraphs for reasoning, which have been recently adapted to claim verification tasks [31]. While structured methods exhibit stronger factual grounding, they often assume that evidence must exist within the KG, a constraint that does not always hold in open-world settings. Moreover, most of these methods operate on datasets with gold or domain-specific evidence (e.g., biomedical or legal corpora [5, 33], which limits their generalization capability to broader real-world fact-checking scenarios. These limitations indicate that fact-checking poses a distinct retrieval problem that requires structured reasoning over explicit relations and adaptive retrieval strategies beyond one-shot similarity-based search. From a retrieval perspective, RAG-based fact-checking remains constrained by three major issues: (i) Ambiguity in unstructured evidence, as text-based retrieval often fails to ensure accurate claim– evidence alignment, (i) Lack of adaptive retrieval intelligence, as existing systems lack agentic reasoning to determine what to re- trieve and when, and (i) Insufficient integration of structured and unstructured knowledge, as few frameworks effectively combine KGs with web-scale information to support comprehensive reasoning. To address these issues, we propose a multi-sourced, agentic fact-checking framework that unifies structured KGs and open-web evidence under a single reasoning paradigm. Motivated by the cog- nitive workflow of human fact-checkers [1, 43], our system dynam- ically determines how, what, or when to retrieve and reason across heterogeneous sources, operating as a reasoning agent as formu- lated within a Markov Decision Process (MDP). Given a claim, our agent first performs KG retrieval by issuing SPARQL queries over open-resource KGs (e.g., Wikidata or DBpedia) using an expand- and-prune search guided by LLM reasoning. The retrieved subgraph serves as the agent’s initial observation. If sufficient evidence is found, the LLM directly performs entity-level reasoning to decide the veracity of an input claim. Otherwise, the agent adaptively se- lects among several actions, including additional KG expansion or targeted web retrieval, to gather complementary information. This formulation explicitly treats evidence acquisition as a sequential information retrieval process under partial observations. Unlike closed-world KG question answering, fact-checking op- erates under an open-world retrieval assumption, where relevant evidence may not exist in structured knowledge bases. To address incomplete or noisy KG coverage, we introduce a web retrieval module that complements KG evidence. Retrieved passages from the open web are filtered through a coarse-to-fine grained pipeline, combining web information retrieval and LLM-based factual consis- tency evaluation. The surviving passages are then converted into knowledge triplets aligned with the KG schema. The resulting web- enhanced KG fuses factual precision from structured KGs with the contextual breadth of the web, providing a comprehensive evidence space for reasoning. Building upon the MDP formulation, we further enable policy improvement without model fine-tuning through self-reflection and prompt optimization. After each reasoning trajectory, the agent evaluates its own performance, identifying error types such as miss- ing relations, premature verdicts, or redundant retrievals. These structured reflections populate an experience buffer that guides prompt-level optimization using the TextGrad [51] framework. The prompt, treated as a trainable policy parameter, is iteratively re- fined to maximize a composite reward that considers correctness, evidence coverage, citation quality, and efficiency. It serves as a retrieval decision policy, guiding when to expand, stop, or conclude. This reflection-driven optimization equips the agent with adaptive decision-making capabilities while keeping the base LLM frozen. We evaluate our proposed Web-enhanced Knowledge Graph Fact- Checking (WKGFC) framework across diverse benchmarks, including Wikipedia-based (FEVER [39], HOVER [19]), web-sourced (LIAR- New [30], AveriTeC [35]), and gold-evidence (e.g. SummEval [10]- series) datasets.WKGFCreports consistent improvements over large- scale LLMs and representative baselines, including RAG-based, graph-based, and web-agent methods. These demonstrate the ef- fectiveness ofWKGFCand its agentic reasoning and self-reflective optimization. To the best of our knowledge, our framework provides the first unified agentic solution that: •Formulates fact-checking as a multi-source informa- tion retrieval MDP process, enabling the agent to reason and to act across heterogeneous evidence environments. Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-CheckingSIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIA •Enables adaptive retrieval over both structured and un- structured knowledge, by dynamically integrating knowl- edge graphs and open-web information. •Achieves self-improvement through reflection-driven prompt adaptation, by allowing the agent to refine its decision policy without modifying LLM parameters. By bridging KG reasoning, web retrieval, and LLM decision poli- cies,WKGFCadvances towards robust, open-world, and self-adaptive fact-checking. 2 Related Work 2.1 Fact-Checking Methods Fact-checking aims to determine the veracity of a claim. Traditional methods relied on linguistic, statistical, and social-contextual fea- tures [54, 48], modeling misinformation using textual semantics [16, 11], propagation patterns [38, 3], or user credibility [6, 12]. From an information retrieval perspective, the core limitation of these approaches lies not in verification itself, but in their inability to re- liably retrieve evidence that is both factually relevant and sufficient for decision making. Although these systems achieved strong in-domain balanced accuracy, they are inherently pattern-recognition models and often fail to generalize to unseen topics, domains, or temporal shifts. This limitation arises from their dependence on dataset-specific language cues and domain distributions. The advent of LLMs [29] has transformed fact-checking into a reasoning-driven process [52]. RAG-based frameworks [36, 42, 44] integrate external knowledge retrieval with generative reason- ing, enabling models to access up-to-date information. Iterative RAGs [32, 41, 45, 49] further enhance factual grounding through multi-step retrieval, but remain limited by semantic similarity re- trieval, which often fails to capture deeper logical relations between claims and evidence at retrieval time. Systems such as HerO [49] and InFact [32] exemplify this line of work, where the retrieved doc- uments are semantically close to the claim but may not correspond factually, leading to false veracity predictions. Most of these methods rely on unstructured text, lacking mech- anisms for multi-hop relational reasoning or handling evidence insufficiency. In contrast, our proposed framework explicitly pri- oritizes structured retrieval over KGs as the first-stage evidence acquisition process, before invoking open-web retrieval when nec- essary. 2.2 LLM Applications over Knowledge Graphs KGs provide a structured and interpretable foundation for factual reasoning. Each fact is encoded as a triplet linking entities through semantic relations, allowing for transparent and compositional inference. Early KG-based approaches focused on specialized neural architectures such as GNNs and relational encoders for entity-level classification and reasoning [17, 25]. These studies demonstrated that structured relational signals can enhance factual precision, especially in domains where relational dependencies are explicit, such as science, medicine, and law. Recent advances in LLM–KG integration combine symbolic struc- ture with neural reasoning. Systems such as StructGPT [17] and RoG [26] allow LLMs to reason over structured inputs through schema-aware prompts or graph-conditioned decoding. Efforts in knowledge-based question answering (KBQA) [23, 24, 37] high- light how subgraph retrieval and query decomposition improve the grounding of language models. Meanwhile, systmes such as FactKG [21], ClaimPKG [31], and GraphCheck [5] explore verifi- cation tasks using retrieved or constructed subgraphs, achieving improved factual alignment compared to unstructured baselines. Though LLM-guided graph reasoning frameworks advance this direction by performing efficient sub-KG discovery, most of these methods assume evidence completeness within the KG or rely on gold-standard data. For example, KBQA tasks assume that the ques- tion answers must exist in the KG. FactKG [21] and ClaimPKG [31] assume that the evidence must exist in the KG. Such assumptions rarely hold in real-world scenarios because it reduces fact-checking to a closed-world retrieval problem, which differs fundamentally from real-world verification where evidence availability is uncer- tain. New claims may refer to evolving or obscure knowledge. Our work bridges this gap by treating KG retrieval as the first stage in a dynamic retrieval decision process, augmenting structured evidence with open-web information only when factual coverage is insufficient. 2.3 Knowledge Graph Retrieval KG retrieval serves as the foundation for structured reasoning and ourWKGFCframework. It plays a central role in domain-specific fact- checking, where evidence must be acquired under incomplete and evolving knowledge assumptions. Classical KGQA pipelines follow a generate-then-retrieve paradigm, where a model formulates an explicit query (e.g., SPARQL) to retrieve subgraphs containing rele- vant entities and relations [2, 24, 50]. While precise, these methods assume that the KG already contains the correct facts, making them less robust in open-domain or incomplete-resource settings. To improve coverage, modern frameworks adopt an expand-and-prune paradigm [27, 37]. Instead of issuing a single rigid query, the model iteratively expands from seed entities and prunes irrelevant nodes using semantic relevance estimated by an LLM. This strategy en- sures multi-hop exploration while maintaining manageable graph size. Beyond traditional retrieval, graph-aware retrieval–generation hybrids combine the efficiency of KG traversal with the flexibility of textual RAG. GraphRAG [8] introduces graph-context aggrega- tion for summarization and fact verification, demonstrating im- proved factual grounding in domains such as biomedical text [15]. GraphCheck [5] extends this principle by automatically construct- ing KGs from long documents, encoding them through GNNs, and reasoning jointly with LLMs. This research shows that graph struc- ture provides an effective scaffold for factual reasoning, especially in multi-hop or long-text contexts. Recent studies also emphasized the role of intelligent agents and self-improving policies in retrieval workflows. Agentic reasoning frameworks [23, 34, 47, 45] enable models to plan, retrieve, and reflect iteratively, and to refine their retrieval actions based on feed- back. Such mechanisms improve efficiency and factual alignment by treating retrieval as a policy optimization problem rather than a static lookup. Our proposed framework inherits these ideas, but specializes them for fact-checking. We formulate KG retrieval, web SIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIAShuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie retrieval, and reasoning as sequential actions within an MDP, al- lowing an LLM-based agent to decide adaptively when to query, expand, or conclude with a verdict. Overall, existing approaches highlight that fact-checking bal- anced accuracy is ultimately bounded by evidence retrieval quality, motivating the need for domain-specific IR frameworks that sup- port adaptive, multi-source evidence acquisition. In summary, while both text-based RAG and KG-based verifica- tion contribute valuable perspectives, they face challenges when used in isolation. Textual systems struggle with semantic ambiguity, whereas structured systems lack coverage. OurWKGFCframework unifies them through agentic control, ensuring both factual ground- ing and adaptive coverage in open-world environments. 3 Preliminaries Knowledge Graph: A knowledge Graph (KG)Grepresents facts as triplets of subject entity, relation, and object entity, denoted as 푡= (푒,푟,푒 ′ ), where entities푒,푒 ′ ∈ Eare connected by a relation 푟 ∈ R;푟can also be referred as푟(푒,푒 ′ ). In open-resource KGs, the same relation of a subject entity may correspond to multiple object entities. For example, the instance_of relation in Wikidata describes which topic a subject entity corresponds/belongs to. Claim Verification: Given a claim푐, a verification modelFdeter- mines its veracity푣as Supported or Refuted based on an external knowledge baseK, while also providing a justification푗to ex- plain the predicted label. We consider the scenario whereKis a hybrid of a structured KGGand unstructured web information K 푤푒푏 , enabling reasoning over graph knowledge to infer푣and푗. Formally, the verification process is defined as:(푣, 푗)= F(푐,K), whereK=G⊕K 푤푒푏 . Markov Decision Process: We transform the fact-checking rea- soning task on external information into an LLM-based agent task, where the KG and web serve as the environment providing execu- tion feedback rather than merely acting as a knowledge base. The reasoning process can thus be viewed as a multi-step interaction with partial observations from the retrieved evidence. This inter- active process can be formalized as a Partially Observable Markov Decision Process (POMDP):(C,S,A,O,T)with claim spaceC, state spaceS, action spaceA, observation spaceO, and state tran- sition functionT:S×A →S. Note that in our language agent scenario,C,A, andOare subspaces of the natural language space, and the transition functionT is determined by the environment. Given a claim to be verified푐 ∈ Cand the evidence knowledge baseK, the LLM agent generates the action푎 0 ∼ 휋 휃 (·|푐,K) ∈ A based on its policy휋 휃 . This action leads to a state transition, and the agent receives execution feedback as observation표 0 ∈ O. The agent then continues to explore the environment until it is confident to make a final verdict, or the stop condition is met. The historical trajectoryH 푛 at step푛, consisting of a sequence of actions and observations, can be represented as follows: H 푛 =(푞,K,푎 0 ,표 표 , . . .,푎 푛−1 ,표 푛−1 ) ∼ 휋 휃 (H 푛 |푐,K)(1) 휋 휃 (H 푛 |푐,K)= 푛 Ö 푗=1 휋 휃 (푎 푗 |푐,K,푎 0 ,표 0 , . . .,푎 푗−1 ,표 푗−1 )(2) The final reward푟(푐,H 푛 ) ∈ [0,1]is computed, with 1 indicating a correct prediction. 4 Methodology Our proposedWKGFCframework integrates open-resource KGs and web-based information as external evidence sources, combined with the reasoning ability of LLMs, to achieve automated fact-checking. Given a claim to be verified, the LLM collaborates with pre-defined SPARQL queries to retrieve the most relevant knowledge subgraphs from the KG using an expand-and-prune strategy (Section 4.2). When the KG evidence is insufficient for verification, a web retrieval agent is triggered to collect complementary, up-to-date information from the open web. The retrieved web evidence is then aligned with the KG schema and merged with the KG subgraph to construct an augmented, multi-source knowledge representation. Finally, the LLM reasons over the aggregated evidence to output a veracity verdict and its corresponding justification. The overall architecture is illustrated in Figure 2. 4.1 Agentic Module We model the evidence acquisition and verification process as an agent operating within a POMDP environment. The POMDP for- mulation explicitly models evidence acquisition as a sequential information retrieval process, where state transitions corresponds to retrieval decisions and observations correspond to retrieved evidence, as shown in Figure 2. The first action푎 0 is defined as an initial knowledge graph re- trieval based on the claiminitKGRetrieval(claim), and the first observation표 0 is corresponding initial knowledge subgraph. If this observation is assessed as sufficient, the agent proceeds to make a verdict; otherwise, it selects actions to expand the current KG expandKG() or complete it by web search webSearch(). The action space is defined as function tools to capture distinct retrieval and decision operations. This design allows the agent to seamlessly integrate structured KG data and unstructured web documents, addressing the inherent limitations of LLMs in handling mixed data modalities. We define the following actions: • initKGRetrieval(claim): Extract entities (i.e., topic enti- ties) from the claim using Named Entity Recognition and entity linking techniques [28]. These entities are mapped to KG nodes to retrieve the initial sub-KG, which forms the agent’s first observation. • expandKG(claim, currentKG, topicEntities): Expand the current KG by one additional hop when the evidence is insufficient to make a confident verdict. The expanded KG is then pruned and refined based on relevance to the claim to be verified. • webSearch(query, currentKG) : When the KG evidence remains incomplete, the agent reasons and formulates a fo- cused web query guided by the missing information. The retrieved web passages are processed and aligned with the KG schema, bridging structured (triplets) and unstructured (description/annotation) evidence sources. • verdict(claim, G, K 푤푒푏 ): Perform the final reasoning step to decide the claim’s veracity based on all gathered evidence from the KG (G) and the web (K 푤푒푏 ). The function Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-CheckingSIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIA Claim: In the 1980s, the oldest justice on the United States Supreme Court was Justice William O.Douglas. justice the United States Supreme Court William O.Douglas Entity Mapping KG entities Multi-Hop Beam Search Expansion Extracted Sub KG webSearch() Raw Texts Evidence Relevant sentences BM25 Knowledge Triplets Fit to KG Entity Extraction Partially Observable Markov Decision Process Init KG Expanded KG Web Enriched KG Verdict Action: expandKG() Action: web- Search() Action: Enforced to Verdict Exceed Tollerance LLM Reasoning LLM Reasoning Action: Confident to Verdict Reward Agentic Module initKGRetrieval() Online Retrieval Expanded KG expandKG() More KG Retrieval Prompt Optimization Multiple Steps Multiple Steps Mapped Seed KG Entities Retrieved KG and Web Entities (subject, predicate, object) ActionObservation Multi-Hop Beam Search Expansion Web Enriched KG Newly Added Entities Init KG Multiple Steps Multiple Steps Multiple Steps Figure 2: Our agentic framework WKGFC. outputs both the predicted verdict and the corresponding evidence justification. This modular design enables the agent to iteratively retrieve, assess, and conclude within a unified decision-making framework, bal- ancing efficiency and performance across heterogeneous evidence sources. 4.2 Knowledge Graph Retrieval Module The objective of KG retrieval in our framework is to acquire a com- pact, claim-relevant evidence subgraph that is sufficient to support verification. We follows an expand-and-prune paradigm [37], in contrast to the generate-then-retrieve strategies commonly used in KGQA systems such as ChatKBQA [24] and SymAgent [23]. The latter approaches rely on generating explicit KG queries (e.g., SPARQL) that assume the answer already exists in the KG, which is an unrealistic assumption for open-world fact-checking, where evidence must often be discovered dynamically. Our retrieval pipeline integrates Wikidata APIs, SPARQL queries, and LLM-based pruning. As shown in Figure 2, the KG retrieval begins with the initKGRetrieval(), a Spacy 1 -implemented entity extraction performed on the input claim. Then it is followed by 1 https://spacy.io/ entity mapping to Wikidata nodes done by Wikidata API. The system then executes SPARQL queries to iteratively expand the subgraph while pruning irrelevant branches via beam search guided by the LLM. Here are some implementation details: KG Entity Mapping. The entities are extracted from the claim us- ing SpaCy 2 , then mapped to Wikidata entries through thewbsearch entities API 3 . These mapped entities serve as seed nodes for the subsequent graph expansion. Beam Search Expansion. Given the seed entities, the system exe- cutes two SPARQL query templates to retrieve both incoming and outgoing relations (Appendix??). Because a single relation can connect to multiple entities, the retrieved candidates are ranked using LLM reasoning on semantic relevance to the claim. In each it- eration (hop), the top-푘most relevant relations and their associated entities are retained, while others are pruned. The process repeats for푁hops, resulting in a compact, semantically relevant subgraph serving as the structured evidence base. In addition, when the current retrieved knowledge graph is as- sessed as not sufficient by the Agentic Module, more KG retrieval 2 https://pypi.org/project/spacy/ 3 https://w.mediawiki.org/wiki/Wikibase/API/ SIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIAShuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie will be triggered by expandKG(), based on more Beam Search Ex- pansion. 4.3 Web Retrieval Module The web retrieval module complements the KG retrieval by pro- viding additional evidence when the KG coverage is incomplete. However, web information is often noisy and may contain misin- formation itself. To mitigate this, web retrieval is only triggered when KG coverage is insufficient and the web evidence is treated as hypothesis-expanding, not authoritative. We introduce a two-stage selection and integration pipeline to fuse the web evidence to the retrieved KG. Coarse-to-Fine Evidence Filtering. Following standard RAG practices, we first perform coarse-grained retrieval using the Ser- per 4 API to collect top-ranked (BM25 5 ) web documents. Then, a finer-grained LLM filter re-evaluates the candidate passages for factual relevance and claim consistency. Each retained passage is associated with a consistency confidence used downstream. Web-to-KG Integration. To unify the structured and the unstruc- tured evidence, selected web sentences are converted into knowl- edge triplets and annotations to be aligned with the KG schema. If a triplet already exists in the KG, it is skipped; if it enriches an exist- ing entity (e.g., by providing explanatory details), it is appended as a textual annotation. This controlled fusion constructs an augmented knowledge graph that captures both factual precision from struc- tured KGs and contextual breadth from the web. KG-oriented facts are treated as higher-precision anchors and web-derived evidence is treated as expansions. By maintaining the agentic reasoning loop across KG and web retrievals, our framework enables dynamic, multi-source evidence synthesis, thus allowing the LLM agent to reason under partial observations and iteratively approach a reliable verdict. 4.4Policy Improvement via Self-Reflection and Prompt Optimization Within the POMDP formulation, the agent’s policy휋 휃 determines which retrieval or decision action to take given the current ob- servation, which consists of the accumulated evidence state. In our framework, this policy is instantiated by the reasoning behav- ior of the agentic module. While the intrinsic parameters of the LLM remain frozen during inference, the policy can be refined at the behavioral level by modifying the prompt that governs how the LLM selects actions. This design enables policy improvement without gradient-based model updates, allowing the agent to adapt its retrieval behavior through experience while remaining fully parameter-free. We adopt a two-stage policy improvement scheme inspired by self-reflective LLM prompt optimization [20, 51], consisting of struc- tured self-reflection followed by prompt-level optimization. Im- portantly, this process targets retrieval coordination and stopping decisions. Self-Reflection. For each claim verification episode, the agent pro- duces a complete decision trajectory퐻 푛 , represented as a sequence of retrieval and reasoning steps (e.g.,InitialKG → Reasoning 4 https://serper.dev 5 https://en.wikipedia.org/wiki/Okapi_BM25 → EnhancedKG → Reasoning → Web-EnhancedKG → Verdict). After generating a final prediction, the agent evaluates the out- come and produces a structured self-critique that attributes success or failure to specific decision patterns. These critiques focus on retrieval-related error types, including insufficient evidence cover- age, premature termination of retrieval, redundant expansion steps, or mismanagement of contradictory evidence. Each time step in the trajectory is recorded in an experience bufferD= (푠 푡 ,푎 푡 ,표 푡 ,푟 푡 , critique 푡 ), where푠 푡 denotes the belief state,푎 푡 the selected action,표 푡 the resulting observation,푟 푡 a scalar reward signal, andcritique 푡 the corresponding textual feedback. This buffer captures how retrieval decisions influence verification outcomes across diverse evidence environments, forming the basis for subsequent policy refinement. Prompt Optimization. Rather than updating model weights, we treat the agent’s prompts as trainable policy descriptors휃and opti- mize it directly over the experience buffer using the TextGrad frame- work [51]. TextGrad interprets textual critiques as gradient-like constructive signals, enabling iterative refinement of the prompt to better align action selection with effective retrieval behavior. In this context, prompt optimization serves as an outer-loop policy improvement mechanism that shapes how the agent prioritizes retrieval depth, source selection, and stopping conditions. Each trajectory is associated with a composite reward that re- flects (i) prediction correctness, (i) evidence sufficiency, measured by whether retrieved information adequately supports or refutes the claim, and (i) retrieval efficiency, penalizing unnecessary or re- dundant retrieval actions. Contradictory evidence between KG and web sources is not treated as noise to be eliminated; instead, failures arising from improper handling of such contradictions are explicitly surfaced in self-critiques and incorporated into the optimization signal. Prompt optimization is conducted offline using an experience buffer constructed from 150 non-test samples drawn from the FEVER [39] and HOVER [19] datasets. Optimization is performed for a fixed budget of 20 epochs over 100 training samples’ trajecto- ries, with validation conducted on a held-out set of 50 trajectories. The final prompts휃 ★ are selected based on validation performance, ensuring stability and mitigating overfitting. Throughout this pro- cess, the base LLM parameters remain unchanged. The resulting policy휋 휃 ★ exhibits improved retrieval coordina- tion, more reliable stopping decisions, and better handling of evi- dence conflicts, leading to higher verification accuracy and reduced retrieval cost. Empirical comparisons before and after prompt op- timization are reported in Table 2, demonstrating that policy re- finement at the prompt level yields consistent performance gains without sacrificing interpretability or scalability. 5 Experiment and Evaluation 5.1 Datasets and Statistics We evaluate our framework on diverse fact-checking benchmarks covering multiple evidence sources. The datasets are selected to represent three distinct categories: (i) Wikipedia-based evidence (FEVER, HOVER), which are extracted from the Wikipedia, (i) Web-sourced evidence (LIAR-New, AveriTeC), which are drawn from web resources, and (i) Provided textual evidence (SummEval, Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-CheckingSIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIA Table 1: Balanced Accuracy comparison across general and medical fact-checking datasets. The best and the second-best results are shown in bold andunderlined, respectively, ‘–’ indicates not applicable due to experiment settings, and grey-highlighted columns denote models that use self-retrieved rather than gold evidence. MethodWikipedia SourcedWeb SourcedWith Gold Evidence Overall Avg. (%) FEVERHOVER LIAR-NewAveriTecSummEvalAFact-CNNPubHealthAvg.(%) Large-scale LLMs Gemini-2.5-flash54.862.259.161.069.964.670.868.463.2 GPT-451.462.558.660.269.760.773.267.962.3 GPT-4o55.363.860.161.576.366.867.070.064.4 Claude 3.5-Sonnet57.164.461.363.077.368.873.873.366.5 DeepSeek-V3 67B53.561.758.860.568.363.272.968.162.7 Small-scale LLMs Llama3 8B [7]48.255.150.354.751.751.370.757.954.6 Qwen2.5 7B [46]50.157.652.956.358.545.359.154.354.3 Llama3.3 70B [7]57.461.260.562.157.653.576.962.761.3 Qwen2.5 72B [46]58.162.059.761.453.449.976.7 60.060.2 Specialized Methods HerO [49]67.558.170.271.465.862.470.366.266.5 GraphRAG (GPT-4o) [8]----68.260.474.267.6- GraphCheck [5]----66.361.970.966.4- FIRE [45]90.667.072.872.858.256.964.159.768.9 WKGFC(Ours)91.972.881.373.269.858.472.867.074.3 AggreFact-CNN, PubHealth), which are summarized from gold evi- dential paragraphs. This design ensures comprehensive evaluation across structured, unstructured, and provided-evidence settings. FEVER. [40] contains human-written claims labeled as SUP- PORTED, REFUTED, or NOT ENOUGH INFO (NEI), each paired with Wikipedia evidence, serving as a standard single-hop fact verifica- tion benchmark. HOVER. [18] extends to multi-hop (i.e., 2 to 4) reasoning across up to four Wikipedia articles, merging REFUTED and NEI into NOT-SUPPORTED for more challenging evidence aggregation. LIAR-New. [30] updates PolitiFact claims beyond 2021 to reduce pretraining leakage, including multilingual claims and rebalanced truth distributions for open-world settings. AveriTeC. [35] offers large-scale, web-sourced claims paired with retrieved evidence documents, reflecting real-world verification scenarios where information is dispersed online. SummEval-Series. [5] covers factual consistency in long-form text (e.g., summarization, biomedical claims), testing models’ ability to verify complex summaries against their source passages. Baseline Setup. To ensure fair comparison across all datasets, we standardize the labels into two classes: Factual Supported and Fac- tual Refuted. The samples lacking verifiable evidence are removed, and partially correct or ambiguous cases are merged into Factual Re- futed. This unified benchmark allows consistent evaluation across datasets and evidence types. Detailed statistics and information are shown in Table ??. 5.2 Baselines We compare our proposed frameworkWKGFCagainst three families of systems listed in Table 1. All methods are evaluated under a Table 2: The statistic of Datasets. (Neg Rate represents the ratio of evidential refuted samples.) DatasetsSize Have Evidence Neg Rate (%) FEVER1,332No50.0 HOVER1,332No50.0 LIAR-New200No76.0 AveriTeC465No73.7 SummEval1,600Yes18.4 AggreFact-CNN558Yes10.2 PubHealth1,232No51.3 unified protocol with standardized binary labels, ensuring a fair comparison across heterogeneous datasets and evidence types. General-purpose LLMs (large-scale). We include closed propri- etary and open-weight models as strong general LLM baselines: GPT-4, GPT-4o, Claude 3.5-Sonnet, Gemini-2.5-flash, and DeepSeek- V3 67B. These models are prompted as generic veracity judges under the same evaluation setup. We report balanced accuracy across all evidence categories to benchmark the reasoning ability of large LLMs without external retrieval. General-purpose LLMs (small-scale). To contextualize perfor- mance versus model scale, we also evaluate open-weight models such as Llama3 8B, Qwen2.5 7B, Llama3.3 70B, and Qwen2.5 72B using the same fact-checking prompt configuration. These mod- els demonstrate the lower bound of zero-shot factual reasoning without retrieval augmentation. Specialized fact-checking and agentic methods. HerO [49] rep- resents a strong RAG-based pipeline with coarse-to-fine document ranking over large evidence databases. In our setup, it consumes SIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIAShuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie dynamically retrieved web passages as its evidence source for real- time verification. FIRE [45] is an iterative retrieval-and-verification framework that alternates retrieval and reasoning to refine factual grounding, serving as another text-based comparison. GraphRAG (GPT-4o) [8] integrates structured graph context aggregation with LLM reasoning and is evaluated on gold-evidence datasets where structured inputs are available. GraphCheck [5] constructs knowl- edge graphs from provided evidence and performs GNN-augmented reasoning. Evaluation fairness. Across all methods that requires retrieval (e.g., HerO, FIRE) or gold evidence (e.g., GraphRAG, GraphCheck), we align the input evidence scope or limit evaluation to appropri- ate dataset categories. This ensures that performance differences stem from reasoning strategy rather than evidence accessibility. For example, for large-scale and small-scale LLMs, they are asked to pre- dict the veracity of claims directly on Wikipedia and web sourced datasets, and are able to refer to the gold evidence on SummEval- series dataset. GraphRAG and GraphCheck are only evaluated on SummEval-series because they require gold evidence to perform graph-based retrieval. For open-sourced retrieval enabled methods (e.g., FIRE andWKGFC), we do not use the SummEval’s provided evi- dence but let them to retrieve evidence by themselves, as indicated in Table 1. 5.3 Experimental Settings Fact-checking research is often fragmented, with methods relying on different types of evidence sources. To establish a fair compar- ison, we design a unified experimental protocol across datasets that differ significantly in content. Specifically, FEVER, HOVER, and LIAR-New provide only claims with veracity labels; AveriTeC offers a large corpus of candidate evidence; and the SummEval- series datasets supply concise, curated evidence paragraphs for each claim. This heterogeneity introduces challenges when applying to ex- isting baselines. For instance, GraphCheck [5] is tailored to the concise evidence provided in SummEval and is thus limited when confronted with open-domain claims lacking such gold-standard evidence. To preserve these methods’ original implementation, we limit the evaluation of GraphCheck and GraphRAG to the given textual evidence category. For methods requiring GPU computing (e.g., HerO, Llama3 8B), we use 4 NVIDIA A100 80GB in our ex- periments. For methods requiring LLM APIs (e.g., FIRE), we use the APIs claimed in their original papers. For ourWKGFC, we exper- iment on top of the OpenAI GPT-4o API. Note that, for FIRE and WKGFC, we do not use the gold evidence, but the provided retrieved evidence, as highlighted in Table 1. 5.4 Performance Analysis Table 1 compares our proposed Web-enhanced Knowledge Graph Fact-Checking (WKGFC) framework to both large-scale LLMs and specialized fact-checking baselines across three evidence categories: Wikipedia-sourced, Web-sourced, and Gold-evidence benchmarks. We can see thatWKGFCachieved the highest overall balanced ac- curacy (74.3%), outperforming the best-performing baseline (FIRE, 68.9%) by +5.4 points and exceeding all large-scale LLMs such as Claude 3.5 Sonnet (66.5%) and GPT-4o (64.4%) despite operating without any task-specific fine-tuning. Wikipedia-Sourced Datasets. On FEVER and HOVER,WKGFCyields 91.9% and 72.8% balanced accuracy, surpassing text-based RAG systems (e.g., HerO 58.1%) and FIRE (90.6% on FEVER, but 67.0% on HOVER). This indicates that integrating knowledge-graph re- trieval prior to web reasoning yields stronger factual grounding and better multi-hop inference. The gain on HOVER further high- lightsWKGFC’s ability to perform compositional reasoning across Wikipedia articles, an ability that similarity-based retrievers often do not have. Web-Sourced Datasets. For LIAR-New and AveriTeC,WKGFCachieves 81.3% and 73.2%, outperforming HerO (70.2 / 71.4) and FIRE (72.8 / 72.8). These results demonstrate that agentic retrieval policies improve veracity prediction in open-domain contexts where rele- vant information is distributed across heterogeneous web pages. Unlike baselines that rely on fixed retrieval rounds,WKGFC’s adap- tive reasoning dynamically decides whether to expand the KG or to trigger web retrieval, leading to higher evidence coverage with fewer redundant calls. Gold-Evidence Datasets. Under the gold-evidence group (Sum- mEval, AFact-CNN, PubHealth),WKGFCmaintains strong balanced accuracy (67.0% avg.), which is on par with specialized graph-based models (GraphRAG 67.6, GraphCheck 66.4). Although those base- lines rely on gold annotations, which are unavailable in open set- tings,WKGFCachieves comparable or better results using only re- trieved evidence, validating the reliability of its knowledge-grounded reasoning pipeline. This robustness across both short- and long- form factuality tasks underscores the generality of our proposed framework. Overall Trends.WKGFCexhibits a consistent upward trend across all evidence categories, showing both high factual precision (Wikipedia tasks) and adaptability (web tasks). Its ability to unify structured and unstructured knowledge under a single agentic workflow re- sults in a balanced performance profile, rather than excelling only in one domain as most baselines do. The strong results against 70 B–100 B-parameter LLMs also emphasize the effectiveness of retrieval-driven reasoning and self-reflective policy optimization over sheer model scale. Together, these results substantiate WKGFC as a robust, data-efficient, and interpretable fact-checking system suitable for real-world multi-source scenarios. 5.5 Ablation Experiment Variant 1. KG retrieval only: In the first variant, we only utilize the evidence from Wikidata to make a verdict. Variant 2. Web retrieval only: Only web source is retrieved. Variant 3. KG retrieval + agentic module: This variant enables agentic reasoning, thus allowing more flexible KG retrieval, the agentic module is also optimized through prompt tuning. Variant 4: KG retrieval + Web retrieval w/o agentic module: This variant enables web retrieval, but limits the agentic actions. The web queries are based on the questions but not generated from existing KG evidence. Variant 5:WKGFCw/o agent optimization: This variant enables all KG, web, and agentic modules, but does not optimize the prompts. Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-CheckingSIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIA Table 3: Ablation experiments for theWKGFC(balanced accu- racy in %). KG Web Agent Optim Wiki-sourcedWeb-sourced FEVERHOVERLIAR-NewAveriTec Variants✓---72.356.361.863.4 -✓--79.666.268.972.4 ✓--85.466.669.069.4 ✓-✓76.564.265.768.5 ✓-89.167.969.871.0 WKGFC✓91.969.870.873.2 Table 3 examines the contributions of each module (knowledge- graph retrieval, web retrieval, and agentic reasoning) on four repre- sentative datasets. When using only KG retrieval, the performance drops notably (e.g., FEVER 72.3, AveriTeC 63.4), reflecting limited evidence coverage. Adding agentic actions on top of KG retrieval improves the balanced accuracy to 76.5–68.5, confirming the bene- fit of dynamic decision-making in when to retrieve and when to reason. Finally, enabling both web retrieval and agentic control (fullWKGFC) yields the best results (e.g., FEVER 91.9, AveriTeC 73.2), suggesting complementarity between structured KG evidence and unstructured web information. These ablations verify that the per- formance gains arise not merely from larger evidence pools, but from the policy-guided integration of heterogeneous sources. 5.6 KG Retrieval Parameter Impact In the KG retrieval module, several parameters, specifically the beam size푘and the number of hops푁, control the exploration scope of the beam search. It is therefore worth investigating whether a broader retrieval scope (i.e., larger beam sizes and deeper hops) consistently leads to better performance. To study this, we analyze the impact of these two parameters on fact-checking balanced accuracy on FEVER and HOVER. To ensure a clean com- parison and to avoid interference from web information, we conduct experiments under Variant 1 from the ablation study (Section 5.5). We vary the beam size푘from 3 to 5 and the number of hopes푁 from 2 to 5. The experimental results on FEVER and HOVER are presented in Fig. 3. 234 Beam Size k 5 4 3 2 Hop Number N 0.610.720.73 0.580.690.72 0.550.670.72 0.520.670.71 Parameter Impact on FEVER 0.55 0.60 0.65 0.70 234 Beam Size k 5 4 3 2 Hop Number N 0.580.600.62 0.550.570.60 0.540.550.55 0.480.530.52 Parameter Impact on HOVER 0.48 0.50 0.52 0.54 0.56 0.58 0.60 0.62 Figure 3: Parameter study on FEVER and HOVER (balanced accuracy in %). Overall, the results indicate that a broader search scope generally improves predictive balanced accuracy. On the single-hop FEVER dataset, performance saturates quickly when푘=4, even with a Table 4: The statistic of Error Rate (%) of WKGFC. Error FEVER HOVER LIAR-New AveriTeC AFact-CNN IK29.674.683.375.098.3 EM4.674.160.072.598.0 OC64.819.816.715.01.3 small number of hops. In contrast, the multi-hop HOVER dataset benefits from deeper traversal (at least four hops) to achieve strong balanced accuracy. However, these improvements come at a computational cost. A broader search requires more SPARQL queries and LLM invocations. Under the Variant 1 configuration, after the entities are extracted and mapped to the KG, the retrieval process can require up to푘×푁 SPARQL queries (at each hop, k entities to expand) to construct the subgraph and up to(푁 +푘× 푁 +1)LLM calls (푁calls for pruning at each hop,푘×푁calls for entity expansion pruning, 1 call for final verdict) to reach a verdict. To balance performance and efficiency, we set 푘= 4 and 푁= 4 in our main experiments. 5.7 Error Analysis To gain an insight into theWKGFC’s performance, we performed a thorough error analysis on the data samples from FEVER, HOVER, LIAR-New, AveriTec and AggreFact-CNN thatWKGFCfailed (incor- rect predictions). We define three types of observations for this analysis: (i) Insufficient KG: An insufficient KG occurs when the agentic module requests a web retrieval even after the retrieved KG is enhanced. This indicates that the information from the KG is not sufficient to support a prediction. (i) Exceeding the Maximum Steps: Exceeding the maximum steps occurs when the agentic module performs all allowed retrieval steps, but is still not confident to make a final verdict. In this case, we use a special prompt to enforce the agent to give a verdict based on the existing evidence. (i) Over-confidence: In contrast to Exceeding the Maximum Steps, over- confidence occurs when the agent does not perform any additional retrieval actions and predicts wrongly based on the initial KG only. Note that these three observations do not cover all failure cases, as some failures arise from execution errors, and there is one sce- nario in which the agent requests additional KG hops without requesting a web search (i.e., it is neither overconfident nor limited by insufficient KG information). The results reveal that insufficient KG coverage dominates the errors in complex and less Wiki-related datasets, which motivates the need for better KG construction and retrieval. In contrast, exceeding-step errors are more frequent in multi-hop reasoning tasks such as HOVER (74.1%), where the agent exhausts retrieval attempts without gaining confidence, which in- dicates challenges in balancing retrieval depth and decision thresh- olds. Finally, over-confidence errors are more pronounced in single- hop datasets such as FEVER (64.8%), reflecting cases where the agent prematurely concludes without sufficient evidence. Overall, the error distribution highlights that improving KG completeness and adaptive stopping strategies is crucial for enhancingWKGFC’s robustness and decision reliability. SIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIAShuzhi Gong, Richard Sinnott, Jianzhong Qi, Cecile Paris, Preslav Nakov, and Zhuohan Xie 6 Conclusion and Future Work We presentedWKGFC, a unified agentic framework that formulates fact-checking as a domain-specific information retrieval problem under open-world and partially observable conditions. By model- ing evidence acquisition and verification as a Partially Observable Markov Decision Process (POMDP),WKGFCenables an LLM-based agent to adaptively control retrieval and stopping decisions across structured knowledge graphs and unstructured web sources. The framework integrates expand-and-prune KG retrieval, coarse-to- fine web retrieval, and self-reflective prompt-based policy improve- ment, allowing retrieval behaviors to be refined without modifying model parameters. Extensive experiments on Wikipedia-based, web- sourced, and gold-evidence benchmarks demonstrate thatWKGFC consistently outperforms large-scale LLMs and representative base- lines such as HerO, FIRE, and GraphCheck, particularly in open- world scenarios where evidence is incomplete or unavailable a priori. In future work, we plan to extend this retrieval-centric frame- work toward collaborative multi-agent verification, enabling co- ordinated evidence acquisition, credibility assessment, and causal reasoning for large-scale misinformation mitigation. References [1]Michelle A Amazeen. 2015. Revisiting the epistemology of fact-checking. Criti- cal review, 27, 1, 1–22. [2]Farah Atif, Ola El Khatib, and Djellel Difallah. 2023. Beamqa: multi-hop knowl- edge graph question answering with sequence-to-sequence prediction and beam search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 781–790. [3] Tian Bian, Xi Xiao, Tingyang Xu, Peilin Zhao, Wenbing Huang, Yu Rong, and Junzhou Huang. 2020. Rumor detection on social media with bi-directional graph convolutional networks. In Proceedings of the AAAI conference on artificial intelligence number 01. Vol. 34, 549–556. [4]Canyu Chen and Kai Shu. 2024. Combating misinformation in the age of llms: opportunities and challenges. AI Magazine, 45, 3, 354–368. [5]Yingjian Chen et al. 2025. Graphcheck: breaking long-term text barriers with ex- tracted knowledge graph-powered fact-checking. arXiv preprint arXiv:2502.16514. [6]Jian Cui, Kwanwoo Kim, Seung Ho Na, and Seungwon Shin. 2022. Meta-path- based fake news detection leveraging multi-level social context information. In Proceedings of the 31st ACM international conference on information & knowledge management, 325–334. [7] Abhimanyu Dubey et al. 2024. The llama 3 herd of models. arXiv e-prints, arXiv–2407. [8] Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph RAG approach to query-focused summarization. CoRR, abs/2404.16130. arXiv: 2404.16130. doi:10.48550/ARXIV.2404.16130. [9]Darren Edge et al. 2024. From local to global: a graph rag approach to query- focused summarization. arXiv preprint arXiv:2404.16130. [10]Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021. Summeval: re-evaluating summa- rization evaluation. Transactions of the Association for Computational Linguistics, 9, 391–409. [11]Yi Fung, Christopher Thomas, Revanth Gangi Reddy, Sandeep Polisetty, Heng Ji, Shih-Fu Chang, Kathleen McKeown, Mohit Bansal, and Avirup Sil. 2021. Infosurgeon: cross-media fine-grained information consistency checking for fake news detection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 1683–1698. [12]Li Gao, Lingyun Song, Jie Liu, Bolin Chen, and Xuequn Shang. 2022. Topology imbalance and relation inauthenticity aware hierarchical graph attention net- works for fake news detection. In Proceedings of the 29th international conference on computational linguistics, 4687–4696. [13] Shuzhi Gong, Richard Sinnott, Jianzhong Qi, and Cecile Paris. 2025. Unseen fake news detection through casual debiasing. In Companion Proceedings of the ACM on Web Conference 2025 (W ’25). Association for Computing Machinery, Sydney NSW, Australia, 981–985. isbn: 9798400713316. doi:10.1145 /3701716.3715517. [14] Shuzhi Gong, Richard O Sinnott, Jianzhong Qi, and Cecile Paris. 2023. Fake news detection through temporally evolving user interactions. In Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 137–148. [15] Ching Nam Hang, Pei-Duo Yu, and Chee Wei Tan. 2024. Trumorgpt: query op- timization and semantic reasoning over networks for automated fact-checking. In 2024 58th Annual Conference on Information Sciences and Systems (CISS), 1–6. doi:10.1109/CISS59072.2024.10480162. [16]Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, and Ming Zhou. 2021. Compare to the knowledge: graph neural fake news detection with external knowledge. In Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers), 754–763. [17]Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Xin Zhao, and Ji-Rong Wen. 2023. Structgpt: A general framework for large language model to reason over structured data. In EMNLP. [18]Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. 2020. HoVer: a dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020. Trevor Cohn, Yulan He, and Yang Liu, (Eds.) Association for Computational Linguistics, Online, (Nov. 2020), 3441–3460. doi:10.18653/v1/20 20.f indings-emnlp.309. [19] Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Kumar Singh, and Mohit Bansal. 2020. Hover: A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguis- tics: EMNLP 2020, Online Event, 16-20 November 2020 (Findings of ACL). Trevor Cohn, Yulan He, and Yang Liu, (Eds.) Vol. EMNLP 2020. Association for Compu- tational Linguistics, 3441–3460. doi:10.18653/V1/2020.FINDINGS-EMNLP.309. [20]Omar Khattab et al. 2024. Dspy: compiling declarative language model calls into state-of-the-art pipelines. In The Twelfth International Conference on Learning Representations. [21]Jiho Kim, Sungjin Park, Yeonsu Kwon, Yohan Jo, James Thorne, and Edward Choi. 2023. Factkg: fact verification via reasoning on knowledge graphs. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023. Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki, (Eds.) Association for Computational Linguistics, 16190–16206. doi:10 . 18653 / V1 / 2023 . ACL - LONG.895. [22]Patrick Lewis et al. 2020. Retrieval-augmented generation for knowledge- intensive nlp tasks. Advances in neural information processing systems, 33, 9459–9474. [23] Ben Liu, Jihai Zhang, Fangquan Lin, Cheng Yang, Min Peng, and Wotao Yin. 2025. Symagent: a neural-symbolic self-learning agent framework for complex reasoning over knowledge graphs. In Proceedings of the ACM on Web Conference 2025, 98–108. [24]Haoran Luo et al. 2024. Chatkbqa: a generate-then-retrieve framework for knowledge base question answering with fine-tuned large language models. In Findings of the Association for Computational Linguistics ACL 2024, 2039–2056. [25]Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024. Reasoning on graphs: faithful and interpretable large language model reasoning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https://openreview.net/f orum?id =ZGNWW7xZ6Q. [26]Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, and Shirui Pan. 2024. Reasoning on graphs: faithful and interpretable large language model reasoning. In ICLR. [27]Shengjie Ma, Chengjin Xu, Xuhui Jiang, Muzhi Li, Huaren Qu, Cehao Yang, Jiaxin Mao, and Jian Guo. [n. d.] Think-on-graph 2.0: deep and faithful large language model reasoning with knowledge-guided retrieval augmented gener- ation. In The Thirteenth International Conference on Learning Representations. [28]Cedric Möller, Jens Lehmann, and Ricardo Usbeck. 2022. Survey on english entity linking on wikidata: datasets and approaches. Semantic Web, 13, 6, 925– 966. [29]OpenAI. 2023. GPT-4 technical report. CoRR. [30]Kellin Pelrine, Anne Imouza, Camille Thibault, Meilina Reksoprodjo, Caleb Gupta, Joel Christoph, Jean-François Godbout, and Reihaneh Rabbany. 2023. Towards reliable misinformation mitigation: generalization, uncertainty, and GPT-4. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Houda Bouamor, Juan Pino, and Kalika Bali, (Eds.) As- sociation for Computational Linguistics, Singapore, (Dec. 2023), 6399–6429. doi:10.18653/v1/2023.emnlp-main.395. [31] Hoang Pham, Thanh-Do Nguyen, and Khac-Hoai Nam Bui. 2025. Claimpkg: enhancing claim verification via pseudo-subgraph generation with lightweight specialized llm. arXiv preprint arXiv:2505.22552. [32] Mark Rothermel, Tobias Braun, Marcus Rohrbach, and Anna Rohrbach. 2024. Infact: a strong baseline for automated fact-checking. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER), 108–112. [33]Hannah Sansford, Nicholas Richardson, Hermina Petric Maretic, and Juba Nait Saada. 2024. Grapheval: a knowledge-graph based llm hallucination evaluation framework. arXiv preprint arXiv:2407.10793. Multi-Sourced, Multi-Agent Evidence Retrieval for Fact-CheckingSIGIR, July 20-24, 2026, MELBOURNE, AUSTRALIA [34]Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 68539–68551. [35]Michael Sejr Schlichtkrull, Zhijiang Guo, and Andreas Vlachos. 2023. Averitec: a dataset for real-world claim verification with evidence from the web. In Thirty-thh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. https://openreview.net/f orum?id=f KzSz0oyaI. [36] Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. Enhancing retrieval-augmented large language models with iter- ative retrieval-generation synergy. (2023). https://arxiv.org/abs/2305.15294 arXiv: 2305.15294 [cs.CL]. [37] Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel Ni, Heung-Yeung Shum, and Jian Guo. [n. d.] Think-on-graph: deep and responsible reasoning of large language model on knowledge graph. In The Twelfth International Conference on Learning Representations. [38]Mengzhu Sun, Xi Zhang, Jiaqi Zheng, and Guixiang Ma. 2022. Ddgcn: dual dynamic graph convolutional networks for rumor detection on social media. In Proceedings of the AAAI conference on artificial intelligence number 4. Vol. 36, 4611–4619. [39]James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and verification. In Pro- ceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers). Marilyn A. Walker, Heng Ji, and Amanda Stent, (Eds.) Association for Computational Linguistics, 809–819. doi:10.18653/V1/N18-1074. [40] James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mit- tal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Associ- ation for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Marilyn Walker, Heng Ji, and Amanda Stent, (Eds.) Association for Computational Linguistics, New Orleans, Louisiana, (June 2018), 809–819. doi:10.18653/v1/N18-1074. [41] Jacob-Junqi Tian et al. 2024. Web retrieval agents for evidence-based misinfor- mation detection. In First Conference on Language Modeling. https://openrevie w.net/f orum?id=pKMxO0wBYZ. [42]Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge- intensive multi-step questions. (2023). https://arxiv.org/abs/2212.10509 arXiv: 2212.10509 [cs.CL]. [43]Greta Warren, Irina Shklovski, and Isabelle Augenstein. 2025. Show me the work: fact-checkers’ requirements for explainable automated fact-checking. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, 1–21. [44] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in Neural Information Processing Systems, 35, 24824–24837. [45]Zhuohan Xie, Rui Xing, Yuxia Wang, Jiahui Geng, Hasan Iqbal, Dhruv Sahnan, Iryna Gurevych, and Preslav Nakov. 2025. Fire: fact-checking with iterative retrieval and verification. In Findings of the Association for Computational Linguistics: NAACL 2025, 2901–2914. [46]An Yang et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. [47] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. [48]Long Ying, HUI Yu, Jinguang Wang, Yongze Ji, and Shengsheng Qian. 2021. Fake news detection via multi-modal topic memory network. IEEE Access, 9, 132818–132829. [49]Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, and Kunwoo Park. 2024. HerO at AVeriTeC: the herd of open large language models for verifying real-world claims. In Proceedings of the Seventh Fact Extraction and VERification Workshop (FEVER). Michael Schlichtkrull et al., (Eds.) Association for Computational Linguistics, Miami, Florida, USA, (Nov. 2024), 130–136. doi:10.18653/v1/2024.f e ver-1.15. [50]Donghan Yu et al. [n. d.] Decaf: joint decoding of answers and logical forms for question answering over knowledge bases. In The Eleventh International Conference on Learning Representations. [51] Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. 2025. Optimizing generative ai by backpropagating language model feedback. Nature, 639, 8055, 609–616. [52] Xiaofan Zheng, Zinan Zeng, Heng Wang, Yuyang Bai, Yuhan Liu, and Min- nan Luo. 2025. From predictions to analyses: rationale-augmented fake news detection with large vision-language models. In Proceedings of the ACM on Web Conference 2025, 5364–5375. [53]Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI Open, 1, 57–81. doi:10.101 6/J.AIOPEN.2021.01.001. [54]Xinyi Zhou and Reza Zafarani. 2020. A survey of fake news: fundamental theo- ries, detection methods, and opportunities. ACM Computing Surveys (CSUR), 53, 5, 1–40.