Paper deep dive
ReflectFact: Self-Reflective Agents for Improving Comprehension and Reasoning in Multi-Hop Fact Verification
Runze Zhao, Zixin Tang, Xiaoshuai Hao, Leyuan Chang, Xiaopeng Fu, Boyu Qiao, Dongyang Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/14/2026, 5:26:22 AM
Summary
The paper introduces ReflectFact, a self-reflective agent framework for multi-hop fact verification that addresses limitations in existing multi-agent methods, specifically objective conflicts and knowledge conflicts (evidence drift). ReflectFact employs three key tasks: Explicit Reasoning Path Planning (resolving implicit entities, semantic decomposition, and logical integration), Evidence-Drift Verification (ensuring answers are grounded in evidence rather than parametric priors), and Reasoning Reflection Verification (self-evaluating reasoning steps for consistency). The framework achieves state-of-the-art performance on the HOVER and EX-FEVER datasets.
Entities (8)
Relation Signals (8)
ReflectFact → evaluatedon → HoVer
confidence 95% · Extensive experiments on HOVER and EX-FEVER demonstrate that ReflectFact effectively remedies...
ReflectFact → evaluatedon → EX-FEVER
confidence 95% · Extensive experiments on HOVER and EX-FEVER demonstrate that ReflectFact effectively remedies...
ReflectFact → uses → Reasoning Reflection Verification
confidence 95% · Reasoning Reflection Verification re-examines each reasoning step and regenerates it once an inconsistency is detected...
ReflectFact → uses → Explicit Reasoning Path Planning
confidence 95% · ReflectFact introduces three key tasks. Explicit Reasoning Path Planning builds an evidence-grounded reasoning path...
ReflectFact → uses → Evidence-Drift Verification
confidence 95% · Evidence-Drift Verification makes the agent re-answer by quoting the supporting evidence...
Evidence-Drift Verification → addresses → knowledge conflicts
confidence 90% · Conflicts may arise when the parametric knowledge in an agent is inconsistent with evidential knowledge and is referred to as evidence drift.
Reasoning Reflection Verification → addresses → objective conflicts
confidence 90% · agents may perform individual subtasks without sufficient awareness of the global verification objective, causing their reasoning to deviate...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-hop fact verification, which verifies claims by reasoning over multiple pieces of evidence, is critical for combating misinformation on social media yet remains highly challenging. Recent methods primarily rely on multi-agent collaboration to decompose fact verification into specialized subtasks. However, these methods face two critical limitations: (1) agents may perform individual subtasks without sufficient awareness of the global verification objective, causing their reasoning to deviate from the intended direction; and (2) conflicts between parametric knowledge and the provided evidence may undermine evidence-grounded reasoning and lead to incorrect verdicts. To address these challenges, we propose ReflectFact, a novel self-reflective agent framework for multi-hop fact verification. ReflectFact introduces three key tasks. Explicit Reasoning Path Planning builds an evidence-grounded reasoning path by resolving implicit entities, decomposing the claim into sub-questions, and integrating the verified facts into a verdict. Evidence-Drift Verification makes the agent re-answer by quoting the supporting evidence when a grounded answer merely echoes its parametric prior, thereby calibrating evidence deviation to ensure grounded comprehension. Reasoning Reflection Verification re-examines each reasoning step and regenerates it once an inconsistency is detected, correcting reasoning flaws such as location bias and replacement bias through a global task perspective. Subsequently, the agent aggregates validated reasoning chains to yield reliable verdicts. Extensive experiments on HOVER and EX-FEVER demonstrate that ReflectFact effectively remedies the comprehension and reasoning defects of existing methods, achieving state-of-the-art performance and respectively outperforming the strongest baseline by 3.32\% and 2.78\% on the two datasets.
Tags
Links
- Source: https://arxiv.org/abs/2608.12877v1
- Canonical: https://arxiv.org/abs/2608.12877v1
Trouble viewing inline? Open PDF directly →
Full Text
45,811 characters extracted from source content.
Expand or collapse full text
ReflectFact: Self-Reflective Agents for Improving Comprehension and Reasoning in Multi-Hop Fact Verification Runze Zhao 2 , Zixin Tang 1∗ , Xiaoshuai Hao 3 , Leyuan Chang 1 , Xiaopeng Fu 1 , Boyu Qiao 2 , Dongyang Zhang 1 1 Zhongguancun Laboratory 2 Institute of Information Engineering, Chinese Academy of Sciences 3 Xiaomi EV Abstract Multi-hop fact verification, which verifies claims by reasoning over multiple pieces of evidence, is critical for combating mis- information on social media yet remains highly challenging. Recent methods primarily rely on multi-agent collaboration to decompose fact verification into specialized subtasks. How- ever, these methods face two critical limitations: (1) agents may perform individual subtasks without sufficient awareness of the global verification objective, causing their reasoning to deviate from the intended direction; and (2) conflicts between parametric knowledge and the provided evidence may un- dermine evidence-grounded reasoning and lead to incorrect verdicts. To address these challenges, we propose Reflect- Fact, a novel self-reflective agent framework for multi-hop fact verification. ReflectFact introduces three key tasks. Ex- plicit Reasoning Path Planning builds an evidence-grounded reasoning path by resolving implicit entities, decomposing the claim into sub-questions, and integrating the verified facts into a verdict. Evidence-Drift Verification makes the agent re- answer by quoting the supporting evidence when a grounded answer merely echoes its parametric prior, thereby calibrat- ing evidence deviation to ensure grounded comprehension. Reasoning Reflection Verification re-examines each reason- ing step and regenerates it once an inconsistency is detected, correcting reasoning flaws such as location bias and replace- ment bias through a global task perspective. Subsequently, the agent aggregates validated reasoning chains to yield reliable verdicts. Extensive experiments on HOVER and EX-FEVER demonstrate that ReflectFact effectively remedies the compre- hension and reasoning defects of existing methods, achieving state-of-the-art performance and respectively outperforming the strongest baseline by 3.32% and 2.78% on the two datasets. Introduction The pervasive dissemination of misinformation on social me- dia has driven an urgent need for automated fact verification (Park et al. 2021; Si, Zhu, and Zhou 2023; Liu, Li, and Zuo 2023; Wang et al. 2024). Multi-hop fact verification, which assesses claim veracity through multi-step reasoning over multiple pieces of evidence (Ma et al. 2023; Zhu et al. 2023; Pan et al. 2023; Liu et al. 2025), more closely mirrors real-world misinformation scenarios and has thus attracted substantial research attention. As a result, developing robust ∗ Corresponding author. Which film ？ Claim: Miles Teller is an actor who starred in a 2017 American black comedy-crime film directed by Todd Phillips. War Dogs Miles Teller is an actor who starred in War Dogs directed by Todd Phillips. Predicted：SUPPORTED War Dogs is a 2016 American black comedy crime film. (a) Objective conflicts. Claim: The actor who played Spock in The City on the Edge of Forever was born in 1931. Zachary Quinto was born in 1931. Leonard Nimoy Who played？ Zachary Quinto Predicted：REFUTED Leonard Nimoy was born in 1931. (b) Knowledge conflicts. Figure 1: Examples illustrating objective conflicts and knowl- edge conflicts in existing fact verification methods, highlight- ing their limitations in multi-hop fact verification. multi-hop verification systems significantly impacts the re- liability and trustworthiness of information ecosystems in practical applications. Recently, the rapid development of large language models (LLMs) has significantly advanced fact verification capabili- ties (Pan et al. 2023; Zhang and Gao 2023; Yue et al. 2023; Si et al. 2024), enabling more sophisticated multi-hop reasoning approaches. Researchers have explored three main categories of solutions: knowledge graphs, fine-tuned natural language inference (NLI) models, and agent-based approaches. Early methodologies employ knowledge graphs or fine-tuned NLI models to verify multi-hop facts (Ren and Leskovec 2020; He, Gao, and Chen 2023; Lei et al. 2025), effectively captur- ing logical dependencies through structured representations. Recent approaches leverage intelligent agents that mimic hu- man fact-verification workflows (Yang et al. 2026; Zhang and Gao 2023; Xu et al. 2025; Wang et al. 2024), decom- posing verification into sequential subtasks with progressive execution. Despite their promise, agent-based methods still face two critical limitations, as illustrated in Figure 1. (1)Ob- jective conflicts. Agents performing individual subtasks may lack sufficient awareness of the global verification objective, causing their reasoning to deviate from the intended direc- tion. As shown in Figure 1(a), the entity resolution agent directly replaces the descriptive mention “a 2017 American arXiv:2608.12877v1 [cs.AI] 13 Aug 2026 black comedy crime film” with the actual film title “War Dogs”. Although this substitution satisfies the local entity resolution objective, it overlooks the inconsistency between the stated release year and the film’s actual release year, thereby obscuring critical evidence for fact verification. (2) Knowledge conflicts. Conflicts may arise when the paramet- ric knowledge in an agent is inconsistent with evidential knowledge and is referred to as evidence drift. As illustrated in Figure 1(b), it can lead to unsupported modifications to an otherwise valid claim. Therefore, we propose ReflectFact, a self-reflective agent framework with post-verification that align local reasoning with the global objective and ensure evidence-grounded verification. In this paper, we propose ReflectFact, a novel self- reflective agent framework to address limitations of existing agent-based methods in multi-hop fact verification through self-reflective post-hoc verification. ReflectFact introduces three tasks coordinated across verification stages. Explicit Reasoning Path Planning builds an evidence-grounded rea- soning path through three automated pipelines: Implicit Entity Resolution identifies implicitly referenced entities, Semantic Decomposition breaks claims into atomic sub- claims, and Integrative Logical Reasoning constructs co- herent logical chains for verdicts. Unlike standard work- flows that execute subtasks sequentially without verifica- tion, ReflectFact ensures comprehensive verification cover- age. Evidence-Drift Verification ensures agents prioritize grounded evidence over parametric knowledge, making the agent re-answer by quoting the supporting evidence to pre- vent knowledge conflicts. Reasoning Reflection Verification provides agents with global information to re-examine af- ter each step. Leveraging the observation that LLMs exhibit stronger verification than generation capabilities, it facilitates self-evaluation of reasoning coherence with the overarching objective, thereby mitigating biases arising from sub-task op- timization. The synergy between structured reasoning-path construction, evidence comprehension, and reflective veri- fication enables robust adaptation to diverse multi-hop sce- narios where error accumulation and knowledge bias are prevalent. Extensive experiments on HOVER (Jiang et al. 2020) and EX-FEVER (Ma et al. 2023) demonstrate that Re- flectFact consistently outperforms the strongest of ten com- petitive baselines by 3.32% and 2.78% in overall Macro-F1, achieving SOTA with robust scalability and interpretability. Our main contributions are summarized as follows: • We identify two key limitations faced by agent-based methods—namely, a lack of a global reasoning per- spective and an over-reliance on parametric knowl- edge—which render them prone to both objective con- flicts and knowledge conflicts. • We propose ReflectFact, a novel self-reflective agent framework for multi-hop fact verification, introducing self-reflective post-hoc verification at each reasoning step to address the existing limitations in subtask-level rea- soning quality, ensuring accurate verification in complex multi-hop scenarios. • ReflectFact achieves significant performance gains on HOVER and EX-FEVER, outperforming the strongest of ten competitive baselines and demonstrating superior adaptability in handling complex fact verification. Related Work Fact Verification. Fact verification predicts claim veracity given retrieved evidence (Park et al. 2021; Botnevik, Sakari- assen, and Setty 2020; Si, Zhu, and Zhou 2023; Liu, Li, and Zuo 2023; Cekinel, Karagoz, and Çöltekin 2025). Early approaches encode claims with textual evidence for classifi- cation, such as entity linking with enhanced language mod- els (Hanselowski et al. 2018) and BERT-based explainable verification (Kotonya and Toni 2020), or reason over graphs built from claims and evidence, such as semantic-level graph reasoning (Zhong et al. 2020) and kernel graph attention net- works (Liu et al. 2020). However, these single-hop methods struggle when verdicts depend on jointly combining facts from multiple sources. To address multi-hop scenarios, re- cent methods improve reasoning through counterfactual data augmentation (Zhu et al. 2023) and LLM-extracted struc- tured knowledge injection (Cao et al. 2025). However, these approaches primarily focus on evidence retrieval and knowl- edge augmentation while overlooking reasoning quality at each intermediate step, leading to error accumulation dur- ing multi-hop inference. To improve multi-hop reasoning, Zhu et al. (Zhu et al. 2023) generate additional data through a counterfactual procedure to enhance generalization, and (Cao et al. 2025) inject LLM-extracted structured knowl- edge to strengthen multi-hop verification. However, these methods mainly rely on predefined representations or exter- nal knowledge structures, which limits their ability to flexibly reason over complex claims involving implicit relations and multiple evidence sources. These methods mainly improve evidence representation, training data, or relation modeling, rather than explicitly verifying the intermediate outputs pro- duced during multi-hop fact verification. LLMs for Fact Verification. The strong understanding and reasoning abilities of LLMs have opened new opportu- nities for fact verification (Zhang and Gao 2023; Pan et al. 2023; Si et al. 2024). One line of work elicits step-by- step reasoning through prompting, such as Chain-of-Thought (Wei et al. 2022) and Tree-of-Thought (Yao et al. 2024). A second line decomposes a claim into sub-problems: HiSS (Zhang and Gao 2023) partitions a claim and verifies each part, ProgramFC (Pan et al. 2023) prompts LLMs to gen- erate reasoning programs from a shared function library, and Factcheck-GPT (Wang et al. 2024) builds a fine-grained checking pipeline. A third line augments LLMs with external structure or collaboration, e.g., multiple role-playing agents (Liu et al. 2025) and knowledge graphs extracted from the input (Chen et al. 2025). However, these approaches largely rely on the correctness of generated reasoning trajectories and lack mechanisms to verify whether intermediate reason- ing steps remain consistent with the evidence and the overall verification objective. Agent-based Fact Verification. Agent-based systems fur- ther organize verification as coordinated workflows. BiDeV integrates multiple role-played LLMs and performs bilateral defusing to resolve claim vagueness and filter redundant ev- idence (Liu et al. 2025). AgentFact coordinates specialized q 1: How many members are in Goodbye Mr. Mackenzie? Check q 2 :How many members are in Chvrches.? I. Reasoning Reflection Verification Evidence-Grounded Answer Parametric-Only Answer k=1 to K Locate Search Replace Generate a question → Answer from Evidence → Replace description → Claim c: Goodbye Mr. Mackenzie is a band with more members than the band that recorded "Make Them Gold" Implicit Entity Resolution Semantic Decomposition Integrative Logical Reasoning a 1 :5 a 2 :3 Verified Fact 1:Goodbye Mr. Mackenzie has 5 members. Verified Fact 2:Chvrches has 3 members. Evidence E： The band that recorded "Make Them Gold" is Chvrches. Logical Composition: 5>3. Re-answer by quoting the span of evidence Yes I. Explicit Reasoning Path Planning Regenerate the output given the inconsistency. Verdict： True Question Is ？ Claim Is inconsistency？ Yes I. Evidence-Drift Verification Figure 2: Overview of ReflectFact. ReflectFact is divided into three tasks. Explicit Reasoning Path Planning builds an evidence-grounded reasoning path by resolving implicit entities, decomposing the claim into sub-questions, and integrating the verified facts into a verdict. Evidence-Drift Verification makes the agent re-answer by quoting the supporting evidence when a grounded answer merely echoes its parametric prior. Reasoning Reflection Verification re-examines each reasoning step and regenerates it once an inconsistency is detected. agents for strategy planning, evidence retrieval, visual anal- ysis, reasoning, and explanation generation in multimodal fact checking (Xu et al. 2025).These methods decompose verification into subtasks and execute them through agent workflows. These methods decompose fact verification into subtasks and execute them through agent workflows. How- ever, existing agent-based methods primarily focus on task decomposition and workflow orchestration, while paying limited attention to the reliability of reasoning within in- dividual subtasks, leaving the overall process vulnerable to objective and knowledge conflicts. In contrast, ReflectFact applies post-verification after each subtask, using reflective reasoning to evaluate intermediate outputs against the global fact-verification objective and the available evidence. Methodology Task Definition Multi-hop fact verification can be formulated as follows. Given a claim c and a set of relevant evidence E = e 1 , e 2 , . . . , e n drawn from a large textual corpus such as Wikipedia, the goal is to determine whether the claim is supported or refuted through multi-step reasoning over the evidence. We denote the final verification label by y ∈ T rue, F alse. Since our task focuses on evidence comprehension and reasoning rather than evidence retrieval, we use dataset-provided gold evidence, where each e i ∈ E contributes to supporting or refuting the claim. Explicit Reasoning Path Planning We design the execution pipeline of the agent framework to construct an explicit, evidence-grounded reasoning path through step-by-step multi-hop claim verification. Further- more, we introduce two self-reflective verification tasks, Evidence-Drift Verification and Reasoning Reflection Ver- ification, to suppress errors that arise during the execution of sub-tasks. We first describe how Explicit Reasoning Path Planning builds the reasoning path, followed by a detailed introduction of the two verification tasks employed through- out the execution process. As illustrated in Figure 2, Explicit Reasoning Path Planning organizes the verification process into three stages: Implicit Entity Resolution, Semantic De- composition, and Integrative Logical Reasoning. Implicit Entity Resolution The reasoning path begins by identifying implicit entities within the claim. We formalize this process as follows. Let T = t 1 , t 2 , t 3 , . . . , t m denote the components of a claim associated with external evidence, where t i represents a word in the claim and T typically rep- resents a description of an entity. For example, in the claim "The Rookie of The Year in the 1997 CART season drives it in the NASCAR Sprint Cup Series," T = "The", "Rookie", "of", "The", "Year", "in", "the", "1997", "CART", "season". Implicit Entity Resolution replaces the entity description T with the concrete entity T ′ that it denotes. To implement this stage, we introduce three components (L,S,R), corre- sponding to the Locate, Search, and Replace operations. Locate (L): This component identifies the span T corre- sponding to an implicit entity description within a claim c. To support the subsequent reasoning steps, the LLM generates a targeted question Q l = L(c) about the entity description. For instance, in Figure 2, given the description "the band that recorded ’Make Them Gold’", the LLM generates the ques- tion: "Which band recorded ’Make Them Gold’?" Specifi- cally, we use the following prompt format: "<claim> De- sign a question to discover the implicit entity. If no implicit entity is found, print: ’No implicit entity’". Search (S): Given the generated query Q l , this compo- nent accesses external evidence to derive an answer A l that explicitly contains the identified entity T ′ . This operation is denoted as A l =S(Q l , Evidence). Specifically, we generate prompts in the following format: "<evidence> Based on the above facts, answer the question. <question>". Sub- sequently, LLMs will return the answer A l to the question Q l , and A l includes the identified implicit entity T ′ . Replace (R): This final operation leverages LLMs to re- place the original description T in claim c with the entity T ′ found in A l . The constructed prompt format is as follows: "<Answer> on the above information, replace the implicit entity and its description in the following expressions with specific names: <claim>". We also adopt dynamic sample selection to provide demonstrations of the replacement pro- cess. Ultimately, we obtain a claim c ′ = R(c, A l ) that has undergone the replacement process. Semantic Decomposition After Implicit Entity Resolu- tion, Semantic Decomposition verifies the refined claim by examining each of its semantic components individually. This explicit decomposition prevents the LLM from over- looking fine-grained details that become increasingly impor- tant in claims requiring more reasoning hops. Specifically, Semantic Decomposition generates sub-questions Q s for the components of claim c ′ and obtains the corresponding an- swers from external knowledge, where Q s = q i n i=1 rep- resents a question. For the claim "Greater Swiss Mountain Dog and Harrier are both dog breeds," separate questions can be posed for its complete semantic components, such as "Is the Greater Swiss Mountain Dog a dog breed?" and "Is the Harrier a dog breed?" In Figure 2, we likewise generate two questions for a claim. We subsequently verify the answer to each sub-question against the evidence repository. As in the first stage, we provide the evidence to the LLM as external knowledge. For each direct question q i , the LLM produces the corresponding answer a i , i ∈ [1, n]. All the answers are represented as the set A s =a 1 , a 2 , ...a n . The specific con- struction of prompts is as follows: "<evidence> Base on the above facts, <question>". Integrative Logical Reasoning Finally, Integrative Logi- cal Reasoning combines the outputs of the preceding stages into a coherent reasoning chain. We manually construct chain-of-thought templates and integrate the outputs from Implicit Entity Resolution and Semantic Decomposition into these templates. The LLM can then review the preceding ver- ification process and derive the final answer solely from the provided content. Specifically, the chain-of-thought (CoT) template is based on c ′ and A s and guides the LLM through the final reasoning process. The prompt for constructing the CoT is: "Determine whether the following statement is true: <c ′ > True or false? Think step by step: <A s > So, the statement is:". In the end, LLMs provide the answer y ∈ T rue, F alse. At this stage, the LLM focuses on in- tegrating the verified intermediate results without requiring additional external knowledge. Although explicit reasoning path planning improves the interpretability of the verification process, errors may still arise during individual sub-tasks and propagate to subse- quent stages. In particular, the execution pipeline contains two functionally different types of sub-tasks. Evidence- comprehension tasks require the agent to derive an answer from retrieved evidence, whereas instruction-driven reason- ing tasks require it to manipulate, transform, or integrate in- formation according to predefined instructions. We therefore introduce two corresponding post-verification mechanisms: Evidence-Drift Verification for evidence-comprehension tasks and Reasoning Reflection Verification for instruction- driven reasoning tasks. Evidence-Drift Verification Evidence-comprehension tasks require the agent to derive an answer directly from the provided evidence. They include, for example, the Search operation in Implicit Entity Resolution and the answer-generation steps in Semantic Decomposition. To detect evidence drift, we introduce an evidence-free counterpart of the same query. Specifically, given the ques- tion q, we additionally prompt the LLM to answer q without access to the evidence e, relying solely on its parametric memory, denoted as a p = S(q,∅), where∅ indicates that no evidence is supplied. We then compare the evidence- grounded answer a with the parametric answer a p . If the two answers converge, i.e., a ≡ a p , we cannot rule out the pos- sibility that a was in fact produced from parametric memory rather than genuinely grounded in e, in which case we flag the sub-task as a candidate instance of evidence drift. Once flagged, we explicitly require the agent to re-derive the an- swer while quoting the specific span of evidence e ∗ ⊆ e that supports its conclusion, formulated as ˆa = S(q, e, e ∗ ). By forcing the agent to cite e ∗ verbatim, this post-verification step re-anchors the answer to the retrieved evidence rather than the LLMs internal prior, thereby suppressing evidence drift and improving the faithfulness of comprehension. Reasoning Reflection Verification Reasoning tasks correspond to the sub-steps in which the agent follows our predefined instructions to resolve the cur- rent claim, such as the Locate operationL in Implicit Entity Resolution and the reasoning step in Integrative Logical Rea- soning. Rather than taking the reasoning output at face value, we design a self-reflective post-hoc validation that recasts the produced result as an object to be checked, exploiting the observation that verification capabilities of LLM typically surpass the generative abilities. Formally, let x denote the input to a reasoning sub-task (e.g., the claim c ′ or the intermediate answers A s ) and o =F(x) denote the corresponding output produced by the agent, whereF ∈L,R,· denotes the reasoning operation being examined. We treat the pair (x, o) as an executable instruction-output pair and construct a verification prompt V(x, o) that asks the LLM to check whether o correctly and consistently follows from x. Crucially, we prepend the verifi- cation prompt with an explicit task framing: “This is part of a fact-checking task, and any error or inconsistency found must be reported.” This framing shifts the LLM from a generator to a verifier, i.e., ˆo = o,ifV(x, o) = consistent, F(x|V(x, o)), otherwise, (1) whereV(x, o) ∈ consistent, inconsistent denotes the ver- ification verdict, and F(x | V(x, o)) regenerates the output given the flagged inconsistency when a flaw is detected. By decoupling verification from generation and explicitly so- liciting error reports under the fact-checking framing, this leverages the superior discriminative ability of LLM to sub- stantially improve the correctness of the agent reasoning. Experiments Datasets. To better align with the requirements of real-world fact verification, our experiments are conducted on two multi- hop fact verification datasets, HOVER (Jiang et al. 2020) and EX-FEVER (Ma et al. 2023). HOVER is a multi-hop dataset derived from English Wikipedia articles. We utilize the vali- dation set of HOVER for evaluation, which consists of 4,000 claims. The claims in HOVER require evidence from up to four Wikipedia articles to determine whether the claims are true. EX-FEVER involves 2-hop and 3-hop reasoning, claims are created by summarizing and modifying information from hyperlinked Wikipedia documents. We evaluate the model using the test set of EX-FEVER. To maintain consistency with HOVER, we remove data with the NEI label. After that, the number of claims is 4,071. Baselines. We use ten baselines for comparison with Re- flectFact, which can be categorized into three categories. (I). Vanilla LLM, we employ both open-source and closed- source LLMs as verifiers. For the open-source model, we use Flan-T5 (Chung et al. 2022), a unified Text-to-Text Trans- former. We create a prompt by combining the claim and evi- dence and obtain the classification result through the Text-to- Text approach. For the closed-source and open-source mod- els with stronger backbones, we use GPT-4o-mini (OpenAI et al. 2024) and Qwen3 (Yang et al. 2025), which are queried with the same prompt to directly produce a verdict for the claim given its evidence. (I). Inference augmented model, ScandiNLI (Alexandrainst 2022) fine-tunes nb-bert-large for Natural Language Inference (NLI). DeBERTaV3-NLI (Lau- rer et al. 2024) fine-tunes the DeBERTaV3 on annotations from FEVER and four NLI datasets. ProgramFC (Pan et al. 2023) uses a shared library of specialized functions to rea- son with the help of codex and Flan T5. (I). Agent-based Fact Verification, based on the GPT-4o-mini engine. Rather than producing a verdict in a single pass, these agents au- tonomously plan a sequence of processing steps. Compared with directly prompting an LLM for the final label, this agen- tic paradigm offers a more transparent and flexible reasoning process for handling complex multi-hop claims. The HiSS (Zhang and Gao 2023) uses LLMs to partition the claim and determine the truth of each subpart. Factcheck-GPT (Wang et al. 2024) is a fine-grained system for fact-checking. Be- sides, we use BiDeV (Liu et al. 2025), which integrates multiple role-playing LLM agents that mutually defuse and cross-validate each other to fact-check claims. For fairness, we replace the backbone of every agent-based baseline with GPT-4o-mini and impose a constraint that the model cannot access the web. Metrics. Following prior multi-hop fact-verification stud- ies (Wang et al. 2024; Liu et al. 2025), we adopt Macro-F1 as the primary evaluation metric to jointly assess performance on supported and refuted claims. Specifically, we report hop- wise Macro-F1 scores to examine how verification accuracy varies with increasing reasoning depth, together with the overall Macro-F1 on each dataset. Implementation Details. For the fine-tune models, we use cross-entropy loss and the AdamW optimizer with a learning rate of 1e-5. We employ gpt-4o-mini as the backbone for the agent-based fact verification. To stabilize the output of model, we use a greedy decoding strategy, setting the temperature to 0. We also use the FLAN-T5-XL 3B as the backbone of the T5 module. Comparison with State-of-the-art Methods As shown in Table 1, we demonstrate the performance of ReflectFact and baselines. Furthermore, we can observe the following findings: Stronger large language models and better reasoning paradigms help improve fact-checking. Although Scandi- NLI and DeBERTaV3-NLI are specifically fine-tuned on NLI annotations, they still lag behind Vanilla LLMs such as GPT- 4o-mini and Qwen3 by a large margin, e.g., DeBERTaV3- NLI trails GPT-4o-mini by 9.91% and 8.35% on HOVER and EX-FEVER, respectively. We attribute this to the strong rea- soning capability of fine-tuned NLI models, despite the fact that their underlying base models are not sufficiently pow- erful. Notably, ProgramFC also benefits from this property, as it relies on an LLM to generate reasoning programs. Nev- ertheless, even the strongest Vanilla LLM still falls short of ReflectFact by 4.40% and 9.31% on both datasets, indicating that LLMs still require explicit guidance to reason reliably over multiple hops. Compared with directly using LLMs, agent-based fact verification methods exhibit a higher performance ceiling when handling complex claims. On simpler 2-hop claims, the strongest Vanilla LLM, GPT-4o-mini, already performs competitively, but this advantage fades as the number of hops grows. On the most challenging 4-hop claims, the best agent- based baselines, BiDeV and StepByStepFV, reach 69.95% and 69.87%, surpassing GPT-4o-mini (68.89%), while HiSS and Factcheck-GPT, which merely partition the claim with- out an explicit verification workflow, still fall behind. This indicates that decomposing a claim and progressively re- solving it through an agent workflow is more effective than a single LLM call for long-chain reasoning. Furthermore, Re- flectFact demonstrates even more substantial improvements, outperforming the strongest agent-based baseline and Vanilla LLM by 3.79% and 4.85% on 4-hop claims, respectively. The self-reflective post-verification unlocks the full potential of the agent, enabling ReflectFact to achieve the best overall results. HiSS (Zhang and Gao 2023) and Factcheck-GPT (Wang et al. 2024) only focus on partition- ing the claim without further examining whether the resulting sub-answers and reasoning steps are actually reliable, while Table 1: Main results (%). Macro-F1 scores of ReflectFact and baselines on two datasets. Results in bold are the best performance. Model HOVEREX-FEVER 2-hop 3-hop 4-hop Total 2-hop 3-hop Total I. Vanilla LLM FLAN-T5 (Chung et al. 2022)64.23 56.39 54.19 56.41 72.42 61.72 67.64 Qwen3 (Yang et al. 2025)76.09 74.48 59.74 71.48 79.61 73.04 73.93 GPT-4o-mini (OpenAI et al. 2024)76.72 73.10 68.89 74.11 80.10 72.11 74.45 I. Inference augmented model Scandi-NLI (Alexandrainst 2022)72.98 64.85 57.66 65.35 65.66 59.48 62.75 DeBERTaV3-NLI (Laurer et al. 2024) 73.93 63.88 51.69 64.20 69.25 61.83 66.10 ProgramFC (Pan et al. 2023)76.32 68.85 64.96 69.31 83.65 78.24 80.98 I. Agent-based Fact Verification HiSS (Zhang and Gao 2023)75.45 65.10 66.02 68.21 82.28 74.91 78.61 Factcheck-GPT (Wang et al. 2024)75.33 67.84 67.31 71.34 82.81 75.10 78.96 StepByStepFV (Vladika et al. 2025)77.86 73.42 69.87 74.53 82.34 77.11 79.68 BiDeV (Liu et al. 2025)80.08 74.57 69.95 75.19 83.48 76.38 79.66 ReflectFact83.3376.9173.7478.5186.6880.5583.76 Table 2: Ablation results (%). Macro-F1 scores for Reflect- Fact and the ablation setting. HOVEREX-FEVER 2-hop 3-hop 4-hop 2-hop 3-hop w RRV80.68 73.77 69.98 84.04 76.68 w EDV82.69 76.02 72.57 84.21 77.84 ReflectFact 83.33 76.91 73.74 86.68 80.55 BiDeV (Liu et al. 2025) relies solely on multi-agent cross- validation without revisiting its own evidence usage. Conse- quently, even the strongest agent baseline, BiDeV, still trails ReflectFact by 3.32% and 4.10% in overall performance on HOVER and EX-FEVER, respectively. In contrast, by explic- itly re-examining evidence grounding and reasoning validity after each execution step, ReflectFact corrects flawed inter- mediate outputs before they propagate to the final verdict, allowing it to consistently surpass all agent-based baselines and achieve the best overall results on both datasets. Ablation Study In this section, we conduct an ablation analysis of the Re- flectFact, investigating the impact of two verification tasks: Evidence-Drift Verification, Reasoning Reflection Verifica- tion. The results are shown in Table 2. Effect of Evidence-Drift Verification (EDV). Removing EDV causes a moderate but consistent performance drop that widens as the number of hops increases. Since EDV de- tects and corrects evidence drift, i.e., cases where the agent answer coincides with its evidence-free prediction, its re- moval leaves such drifted answers uncorrected, and these early comprehension errors are more likely to propagate into later reasoning steps on longer claims. Effect of Reasoning Reflection Verification (RRV). Re- moving RRV leads to a substantially larger degradation than removing EDV. Without RRV, the outputs of reasoning sub- steps, such as locating implicit entities or constructing the final logical chain, are accepted at face value rather than be- ing re-examined from a perspective of verifier. This confirms that recasting the agent own reasoning output as an object to be verified is the primary source of ReflectFact robust- ness, particularly on claims that demand longer chains of reasoning. Model Generalizability ReflectFact is designed as a general, model-agnostic frame- work for enhancing the multi-hop fact-verification ability of different LLM backbones. To assess its generalizability, we instantiate it on two representative backbones of different scales and access levels: the proprietary GPT-4o-mini (Ope- nAI et al. 2024) and the open-source Qwen3-8B (Yang et al. 2025). For each backbone, we compare ReflectFact against its standard prompting baseline under an identical evaluation protocol. As shown in Figure 3(a) and Figure 3(b), Reflect- Fact delivers consistent Macro-F1 improvements across both backbones and all settings on the HOVER and EX-FEVER benchmarks. The gains hold for both the proprietary and the open-source model, indicating that the benefits of Re- flectFact are not tied to a particular model scale or family. More notably, its advantage becomes more pronounced as the reasoning depth increases: while the baselines degrade sharply on instances requiring more hops, ReflectFact sub- stantially alleviates this decline and attains its largest im- provements precisely in the most challenging multi-hop set- tings. These results confirm that ReflectFact robustly and generally strengthens the multi-hop fact-verification reason- ing of diverse LLMs. HoVer 2-hop HoVer 3-hop HoVer 4-hop EX-FEVER 2-hop EX-FEVER 3-hop 0 10 20 30 40 50 60 70 80 90 100 Macro-F1 GPT-4o-mini GPT-4o-mini-ReflectFact (a) ReflectFact and baselines based on GPT-4o-mini HoVer 2-hop HoVer 3-hop HoVer 4-hop EX-FEVER 2-hop EX-FEVER 3-hop 0 10 20 30 40 50 60 70 80 90 100 Macro-F1 Qwen3 Qwen3-ReflectFact (b) ReflectFact and baselines based on Qwen3-8B Figure 3: Macro-F1 scores of different LLMs based ReflectFact and baselines on two datasets. Table 3: Results of different models generating explanations (%). Results in bold are the best performance. MDR BERT-based GPT ReflectFact Rouge-1 54.8846.8852.2856.71 Rouge-2 41.3432.8033.7439.87 Rouge-L 49.4235.5248.1352.32 Interpretability Analysis EX-FEVER dataset provides golden explanations for the la- bel, which is a textual explanation that describes the mini- mally sufficient information in each hop to verify a claim. To verify the interpretability of the ReflectFact, we con- ducted experiments using the golden explanations data from the EX-FEVER dataset. We consider that the constructed CoT, as an intermediate result of ReflectFact, is accessible and can provide explanations for the judgments. We use the ROUGE metrics to measure the matching degree between the constructed CoT and the golden explanation. Ultimately, we found that Constructed CoT has an explanatory role. The experimental results are shown in Table 3. We compared Re- flectFact with three models: MDR (Xiong et al. 2021), BERT (Devlin et al. 2018), and GPT (Brown et al. 2020), as used in the study by (Ma et al. 2023). Although our model is not ex- plicitly designed for generating explanations, it outperforms three models on average by 5.36% and 7.96% in Rouge-1 and Rouge-L metrics. Error Type Analysis To better understand the behavior of our ReflectFact and fa- cilitate future research, we randomly sampled 40 examples that are generated by ReflectFact. As shown in Figure 4(a), we categorize the causes of errors into three types: 1) logical mistake, 2) factual hallucination, and 3) thoughts omission. We find that the errors corresponding to the three stages are distributed as 10%, 82.5%, and 7.5%, respectively. Conse- quently, we further analyzed the types of errors occurring in the Semantic Decomposition. As shown in Figure 4(b), 72.7% of the errors are concentrated in hallucinations gen- erated by LLM itself. Meanwhile, although decomposing Figure 4: Error analysis of ReflectFact on 40 samples. sub-information enhances the logical reasoning of LLMs, it also ignores longer dependencies in the text, resulting in some degree of performance loss. Conclusion In this paper, we identify two critical limitations of existing agent-based fact-checking methods, namely their susceptibil- ity to objective conflicts and knowledge conflicts. To address these issues, we propose ReflectFact, a novel self-reflective agent framework for multi-hop fact verification. ReflectFact employs Evidence-Drift Verification and Reasoning Reflec- tion Verification to post-verify comprehension and reasoning sub-tasks, respectively, thereby correcting flawed intermedi- ate outputs before they propagate to the final verdict. Ex- tensive experiments on two datasets show that ReflectFact consistently achieves state-of-the-art performance, and fur- ther analyses confirm that it generalizes well across different backbones while offering strong interpretability. References Alexandrainst. 2022. ScandiNLI: Natural language infer- ence for the Scandinavian languages. https://github.com/ alexandrainst/ScandiNLI. Accessed: 2024-03-16. Botnevik, B.; Sakariassen, E.; and Setty, V. 2020. Brenda: Browser extension for fake news detection. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, 2117–2120. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Ad- vances in neural information processing systems, 33: 1877– 1901. Cao, H.; Wei, L.; Zhou, W.; and Hu, S. 2025. Enhanc- ing multi-hop fact verification with structured knowledge- augmented large language models. In Thirty-Ninth AAAI Conference on Artificial Intelligence, Thirty-Seventh Con- ference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI. AAAI Press. ISBN 978-1-57735-897-8. Cekinel, R. F.; Karagoz, P.; and Çöltekin, Ç. 2025. Mul- timodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strate- gies. In Proceedings of the 31st International Conference on Computational Linguistics, 4622–4633. Abu Dhabi, UAE: Association for Computational Linguistics. Chen, Y.; Liu, H.; Liu, Y.; Xie, J.; Yang, R.; Yuan, H.; Fu, Y.; Zhou, P. Y.; Chen, Q.; Caverlee, J.; and Li, I. 2025. GraphCheck: Breaking Long-Term Text Barriers with Ex- tracted Knowledge Graph-Powered Fact-Checking. In Pro- ceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, 14976–14995. Association for Computational Linguistics. Chung, H. W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, Y.; Wang, X.; Dehghani, M.; Brahma, S.; et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805. Hanselowski, A.; Zhang, H.; Li, Z.; Sorokin, D.; Schiller, B.; Schulz, C.; and Gurevych, I. 2018. UKP-Athene: Multi- Sentence Textual Entailment for Claim Verification. In Pro- ceedings of the First Workshop on Fact Extraction and VERi- fication (FEVER), 103–108. Brussels, Belgium: Association for Computational Linguistics. He, P.; Gao, J.; and Chen, W. 2023. DeBERTaV3: Im- proving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing. In The Eleventh International Conference on Learning Representations. Jiang, Y.; Bordia, S.; Zhong, Z.; Dognin, C.; Singh, M.; and Bansal, M. 2020. HoVer: A dataset for many-hop fact extrac- tion and claim verification. arXiv preprint arXiv:2011.03088. Kotonya, N.; and Toni, F. 2020. Explainable automated fact-checking for public health claims. arXiv preprint arXiv:2010.09926. Laurer, M.; Van Atteveldt, W.; Casas, A.; and Welbers, K. 2024. Less annotating, more classifying: Addressing the data scarcity issue of supervised machine learning with deep transfer learning and BERT-NLI. Political Analysis, 32(1): 84–100. Lei, D.; Li, Y.; Li, S.; Hu, M.; Xu, R.; Archer, K.; Wang, M.; Ching, E.; and Deng, A. 2025. FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data. In Proceedings of the 2025 Conference of the Nations of the Americas Chap- ter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Pa- pers, 5002–5020. Association for Computational Linguistics. Liu, F.; Li, Y.; and Zuo, M. 2023. KESHEM: Knowledge Enabled Short Health Misinformation Detection Framework. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 372–388. Springer. Liu, Y.; Sun, H.; Guo, W.; Xiao, X.; Mao, C.; Yu, Z.; and Yan, R. 2025. BiDeV: bilateral defusing verification for complex claim fact-checking. In Thirty-Ninth AAAI Conference on Ar- tificial Intelligence, Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI, AAAI’25/IAAI’25/EAAI’25. AAAI Press. ISBN 978-1- 57735-897-8. Liu, Z.; Xiong, C.; Sun, M.; and Liu, Z. 2020. Fine-grained Fact Verification with Kernel Graph Attention Network. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 7342–7351. Online: Associ- ation for Computational Linguistics. Ma, H.; Xu, W.; Wei, Y.; Chen, L.; Wang, L.; Liu, Q.; and Wu, S. 2023. EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification. arXiv preprint arXiv:2310.09754. OpenAI; :; Hurst, A.; Lerer, A.; Goucher, A. P.; et al. 2024. GPT-4o System Card. Pan, L.; Wu, X.; Lu, X.; Luu, A. T.; Wang, W. Y.; Kan, M.-Y.; and Nakov, P. 2023. Fact-checking complex claims with program-guided reasoning. arXiv preprint arXiv:2305.12744. Park, J.; Min, S.; Kang, J.; Zettlemoyer, L.; and Hajishirzi, H. 2021. FAVIQ: FAct Verification from Information-seeking Questions. arXiv preprint arXiv:2107.02153. Ren, H.; and Leskovec, J. 2020. Beta Embeddings for Multi- Hop Logical Reasoning in Knowledge Graphs. In Neural Information Processing Systems. Si, J.; Zhao, Y.; Zhu, Y.; Zhu, H.; Lu, W.; and Zhou, D. 2024. CHECKWHY: Causal Fact Verification via Argument Structure. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 15636–15659. Bangkok, Thailand: Association for Computational Linguistics. Si, J.; Zhu, Y.; and Zhou, D. 2023. Exploring faithful ratio- nale for multi-hop fact verification via salience-aware graph learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, 13573–13581. Vladika, J.; Hacajova, I.; Matthes, F.; and Others. 2025. Step- by-step fact verification system for medical claims with ex- plainable reasoning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), 805–816. Wang, Y.; Gangi Reddy, R.; Mujahid, Z. M.; Arora, A.; Rubashevskii, A.; Geng, J.; Mohammed Afzal, O.; Pan, L.; Borenstein, N.; Pillai, A.; Augenstein, I.; Gurevych, I.; and Nakov, P. 2024. Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers. In Findings of the Association for Computational Linguistics: EMNLP 2024, 14199–14230. Miami, Florida, USA: Association for Com- putational Linguistics. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompt- ing elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824–24837. Xiong, W.; Li, X.; Iyer, S.; Du, J.; Lewis, P.; Wang, W. Y.; Mehdad, Y.; Yih, S.; Riedel, S.; Kiela, D.; and Oguz, B. 2021. Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval. In International Conference on Learning Representations. Xu, D.; Fan, S.; Cheng, X.; and Kankanhalli, M. 2025. Mul- timodal Fact-Checking: An Agent-based Approach. arXiv preprint arXiv:2512.22933. Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Yang, R.; He, S.; Zhu, S.; and Yu, C. 2026. From Verdict to Process: Agentic Reinforcement Learning for Multi-Stage Fact Verification. arXiv preprint arXiv:2606.13262. Yao, S.; Yu, D.; Zhao, J.; Shafran, I.; Griffiths, T.; Cao, Y.; and Narasimhan, K. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. Yue, Z.; Zeng, H.; Zhang, Y.; Shang, L.; and Wang, D. 2023. Metaadapt: Domain adaptive few-shot misinformation detec- tion via meta learning. arXiv preprint arXiv:2305.12692. Zhang, X.; and Gao, W. 2023. Towards llm-based fact ver- ification on news claims with a hierarchical step-by-step prompting method. arXiv preprint arXiv:2310.00305. Zhong, W.; Xu, J.; Tang, D.; Xu, Z.; Duan, N.; Zhou, M.; Wang, J.; and Yin, J. 2020. Reasoning Over Semantic-Level Graph for Fact Checking. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 6170–6180. Online: Association for Computational Linguis- tics. Zhu, Y.; Si, J.; Zhao, Y.; Zhu, H.; Zhou, D.; and He, Y. 2023. EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Veri- fication. arXiv preprint arXiv:2310.14508.