Paper deep dive
When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation
Lauren Pothuru
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/24/2026, 4:53:40 AM
Summary
This paper introduces AgenticRAG-FP, an interventional benchmark for causal failure attribution in agentic Retrieval-Augmented Generation (RAG). The benchmark injects faults at specific hops in multi-hop trajectories and evaluates whether post-hoc diagnostic methods can identify the root cause after the failure has propagated. Using Claude Haiku 4.5 on MuSiQue, the study finds that coverage-based diagnosis is highly effective at hop 1 (0.91 accuracy) but fails completely at hops 2 and 3 (0.00 accuracy). Counterfactual probes, particularly frozen-hop repair, show promise in content-corruption scenarios where structural signals are lost.
Entities (10)
Relation Signals (9)
Doctor-RAG → hasaccuracy → 0.91
confidence 95% · coverage-based diagnosis is 0.91 at hop 1
Doctor-RAG → hasaccuracy → 0.00
confidence 95% · and 0.00 at hops 2 and 3
AgenticRAG-FP → uses → MuSiQue
confidence 95% · In the completed strict dense Claude Haiku 4.5 sweep on 80 three-hop MuSiQue questions...
AgenticRAG-FP → uses → Claude Haiku 4.5
confidence 95% · In the completed strict dense Claude Haiku 4.5 sweep...
AgenticRAG-FP → evaluates → Suf-Regen
confidence 90% · suffix-regeneration repair (Suf-Regen).
AgenticRAG-FP → evaluates → LLM Judge
confidence 90% · The main comparisons use a coverage-based localizer (Doctor-RAG), a trace-reading LLM judge (LLM-Judge)...
AgenticRAG-FP → evaluates → Doctor-RAG
confidence 90% · The main comparisons use a coverage-based localizer (Doctor-RAG)...
AgenticRAG-FP → evaluates → Propagation-Aware
confidence 90% · frozen-hop counterfactual repair (Propagation-Aware)...
Propagation-Aware → hasaccuracy →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Agentic retrieval-augmented generation (RAG) interleaves retrieval, reasoning, and answer generation across multiple hops. A retrieval error at hop 1 can surface only as a wrong answer at hop 3, while later retrieval can also repair the trajectory. This paper introduces AgenticRAG-FP, an interventional benchmark for causal failure attribution in agentic RAG. The benchmark injects a certified fault at a specified hop, re-executes the downstream trajectory, and evaluates diagnosers against the known intervention. Its central question is whether a post-hoc trace still identifies the injected hop after the suffix changes. In the completed strict dense Claude Haiku 4.5 sweep on 80 three-hop MuSiQue questions, coverage-based diagnosis is 0.91 at hop 1 and 0.00 at hops 2 and 3 (n=43,36,21 failed trajectories). A smaller content-corruption study changes an answer-bearing or bridge fact in topically intact evidence. At depth 2, where 18 failed cases remain after filtering, coverage-based diagnosis is 0.00 and a frozen-hop counterfactual probe is 0.67 in an exploratory pooled comparison. Depth-3 content estimates are descriptive only because they contain three failed cases. These results make propagation depth an explicit evaluation axis for diagnosing agentic RAG failures while distinguishing broad evidence of post-hoc signal loss from small-sample method comparisons.
Tags
Links
- Source: https://arxiv.org/abs/2608.20627v1
- Canonical: https://arxiv.org/abs/2608.20627v1
Trouble viewing inline? Open PDF directly →
Full Text
37,501 characters extracted from source content.
Expand or collapse full text
When Failures Propagate: Causal Failure Attribution in Agentic Retrieval-Augmented Generation Lauren Pothuru Anote laurenpothuru100@gmail.com Abstract Agenticretrieval-augmentedgeneration (RAG) interleaves retrieval, reasoning, and answer generation across multiple hops. A retrieval error at hop 1 can surface only as a wrong answer at hop 3, while later retrieval can also repair the trajectory.This paper introduces AgenticRAG-FP, an interventional benchmark for causal failure attribution in agentic RAG. The benchmark injects a certi- fied fault at a specified hop, re-executes the downstream trajectory, and evaluates diag- nosers against the known intervention.Its central question is whether a post-hoc trace still identifies the injected hop after the suf- fix changes.In the completed strict dense Claude Haiku 4.5 sweep on 80 three-hop MuSiQue questions, coverage-based diag- nosis is 0.91 at hop 1 and 0.00 at hops 2 and 3 (n=43, 36, 21 failed trajectories).A smaller content-corruption study changes an answer-bearing or bridge fact in topically intact evidence. At depth 2, where 18 failed cases remain after filtering, coverage-based diagnosis is 0.00 and a frozen-hop counter- factual probe is 0.67 in an exploratory pooled comparison. Depth-3 content estimates are descriptive only because they contain three failed cases. These results make propagation depth an explicit evaluation axis for diagnos- ing agentic RAG failures while distinguishing broad evidence of post-hoc signal loss from small-sample method comparisons. 1 Introduction Retrieval-augmented generation (RAG) condi- tions generation on retrieved evidence rather than only model parameters (Lewis et al., 2020). Agen- tic RAG extends this process across several deci- sions: a language model selects a sub-query, ob- serves evidence, and decides whether to search again or answer, as in ReAct-style reasoning and acting (Yao et al., 2023). This enables multi-hop question answering, but it also separates a visible end-to-end error from its cause. Consider a three-hop trajectory. If the evidence at hop 1 is irrelevant, the agent can form a drifted sub-query at hop 2, retrieve evidence on the wrong premise, and produce an ungrounded final answer. An evaluator that inspects only the final trace may identify a late low-coverage hop or label the fail- ure as answer generation, while missing the ear- liest causal fault. A different agent may retrieve compensating evidence at a later hop and answer correctly. Standard answer accuracy, retrieval re- call, and final-answer correctness conflate an in- jected fault, its propagation, and its recovery. Existing diagnostic methods can inspect traces or attempt local repair, but post-hoc traces do not provide a certified root-cause label. This leaves two evaluation questions unresolved: can a di- agnoser recover a known intervention hop after the failure has propagated, and when do counter- factual repair probes supply information missing from the final trace? Answering either question requires an intervention that allows the agent to react and retains the intervention label. AgenticRAG-FP provides this setting. It cor- rupts a trace prefix at hop h, runs the remaining trajectory with the same agent, and scores diag- noses against the certified intervention. The paper centers on how propagation changes the informa- tion available for exact-hop attribution. Its three contributions are: 1. An interventional benchmark that supports structural retrieval faults and certified con- tent corruption. The latter changes a known fact span in otherwise relevant evidence and records the span pair, enabling deterministic absorbed, resisted, and derailed outcome la- bels without an LLM judge. 2. A depth-conditioned analysis of post-hoc sig- nal. In the completed strict dense MuSiQue 1 arXiv:2608.20627v1 [cs.CL] 20 Aug 2026 sweep, coverage-based attribution falls from 0.91 at hop 1 to 0.00 at hops 2 and 3. This pattern concerns information available af- ter suffix re-execution rather than end-to-end task accuracy. 3. A bounded comparison of counterfactual probes on content faults. Frozen local repair and suffix regeneration ask different ques- tions about downstream dependence. The re- ported depth-2 content results illustrate this distinction, while the low-count depth-3 re- sults are retained only as descriptive evi- dence. 2 Related Work RAG and active retrieval. RAG combines a parametric generator with a non-parametric re- trieval module (Lewis et al., 2020). Active and reflective variants decide when to retrieve during generation (Jiang et al., 2023; Asai et al., 2023). These approaches aim to improve answer quality by making retrieval responsive to uncertainty or the evolving generation. That same responsive- ness creates the attribution problem studied here: a change to an early retrieval can alter the questions the agent asks later, rather than simply changing one fixed evidence set. Agentic RAG and diagnosis. ReAct-style agents interleave reasoning traces with external actions (Yao et al., 2023). Their trajectories can be inspected, localized, and repaired, but inspec- tion alone cannot establish whether a suspicious hop caused the final answer. Doctor-RAG uses coverage-gated localization and prefix reuse for diagnosis and repair (Jiao et al., 2026); Doctor- RAG instantiates its coverage-localization idea as a post-hoc baseline. The present evaluation differs in its target: it asks whether a diagnoser recovers a certified injected cause after an agent has gen- erated a new suffix. This requires observing both the intervention and the resulting trajectory. Multi-hopbenchmarks. HotpotQA(Yang et al., 2018) and MuSiQue (Trivedi et al., 2021) provide multi-hop questions with supporting facts. FRAMES (Krishna et al., 2024) evaluates factu- ality, retrieval, and reasoning, and CRAG (Yang et al., 2024) includes dynamic, long-tail, and false-premise questions. These datasets provide question and corpus substrates, but their standard answer labels do not identify the hop that caused a failed trajectory. The benchmark in this pa- per supplies that missing causal label through a controlled intervention. Causal evaluation. The intervention perspec- tive follows causal evaluation (Pearl, 2009): the fault and its hop are set before the resulting trajec- tory is observed. The object of intervention is not a model parameter or an internal activation. It is the trajectory-level context available to a retriev- ing agent. This framing makes recovery informa- tive rather than inconvenient, because a correct an- swer after intervention is evidence that the agent did not propagate the fault to its output. 3 Task and Trace Model Each example is a question-answer pair (q,y ∗ ) to- gether with a retrieval corpus C. An agent pro- duces a trace τ = (q,(q h ,D h ) H h=1 ,A,y,c), where q h is the sub-query issued at hop h, D h ⊆C is the retrieved evidence, A is the final answer, y is the reference answer, and c is total token cost. The trace records the information available at each decision and is sufficient to resume execution from an observed prefix. Failure stages. Each hop has a stage s∈retrieval, tool, answer, none. Retrieval failures include empty or irrelevant ev- idence, query drift, false-premise evidence, stale evidence, and a wrong fact in otherwise relevant documents. Tool failures include missing actions and premature termination. Answer failures in- clude empty answers, incorrect answers, and hal- lucinations that are unsupported by the retrieved evidence. The none stage denotes a trace with- out an identified failure at that hop. A diagnosis record contains a predicted stage ˆs, predicted hop ˆ h, propagation flag, severity estimate, and root- cause description. Identifiability. A diagnoser d is identifiable at depth h when it reliably recovers ˆ h = h for faults injected at hop h. This is a stricter property than detecting that the final answer is wrong or naming the broad failure stage. When an injected context changes the later trajectory, the decisive question is whether the final trace still contains information 2 InterventionEffect at the selected hop Empty retrievalReplace retrieved evidence with∅. Irrelevant docu- ments Replace evidence with off-topic documents. Query driftSubstitute a drifted sub-query, then retrieve. False premiseAdd a confidently wrong factual claim to evidence. Stale evidenceAdd outdated temporal evidence. Content corrup- tion Change a certified fact span in oth- erwise relevant evidence. Early termina- tion Force an answer from the preceding evidence. Table 1: Interventions used by AgenticRAG-FP. In ev- ery live intervention, the downstream suffix is gener- ated after the selected-hop corruption. that distinguishes the injected hop from its down- stream consequences. The benchmark treats the selected failure type, stage, and hop as certified labels. Certification matters because a late low-coverage hop can be a cause, a downstream effect, or a benign con- sequence of an earlier intervention. Without a known intervention, these possibilities are obser- vationally similar. The live comparison fixes the cause before execution and then evaluates the di- agnosis only after the agent has had an opportunity to propagate or repair that cause. 4 Interventional Benchmark Resumable live intervention. Given an exe- cuted hop prefix, the agent resumes at the next hop and continues to generate sub-queries, retrieve ev- idence, and decide when to answer. A live inter- vention replaces evidence or a sub-query at hop h, then resumes the agent at hop h + 1. The down- stream suffix is therefore an agent response to the corrupted context, not a post-hoc trace edit. This distinction separates the causal effect of the inter- vention from the behavior of a static edited trace, which cannot reveal whether an agent would prop- agate or recover from a fault. For every live intervention, the evaluation re- tains the intended failure family and selected hop as the target label. The intervention can be applied to retrieval evidence, a sub-query, or the decision to terminate. Structural faults test whether a visi- ble retrieval anomaly remains localizable after the suffix changes. Content faults test the harder case in which the document still looks relevant and only its factual content is wrong. The same trace model therefore supports both readily observable and se- mantically hidden failures. Intervention semantics. Empty and irrelevant retrieval remove answer support or replace it with off-topic evidence. Query drift changes the search intent before retrieval. False-premise and stale- evidence interventions preserve retrieved material but introduce a specific misleading claim. Early termination changes the available trajectory by re- quiring an answer before a later retrieval could occur.These families span missing evidence, misleading evidence, and altered agent decisions. They also differ in how visible they are to a post- hoc inspection, which makes them useful tests of whether a diagnoser is recognizing a local symp- tom or identifying an intervention after propaga- tion. All interventions preserve the selected-hop tar- get while allowing the suffix to change. An an- swer error following an intervention is thus not as- sumed to be caused by an edited final answer. It is an observed outcome of the agent’s reaction to the modified prefix. This construction also per- mits a recovered trajectory to remain part of the evaluation: recovery is evidence about the agent’s resilience, while failed trajectories are the subset on which a root cause can be localized. Certified content corruption. Structural inter- ventions replace whole documents or queries, so they can be visible in the trace. Content corruption instead changes one fact in retrieved documents while preserving topical relevance. This models an error such as a stale fact, a corrupted document, or an upstream extraction error without making the selected retrieval obviously off-topic. The changed span is selected by priority. An an- swer fact is selected when the gold answer occurs in the retrieved evidence and can be replaced by a certified wrong value. Otherwise, a bridge entity is selected when it occurs both in the hop evidence and in a later sub-query, but not in the original question. This identifies information that the agent carried forward. A salient entity or number sup- plies a fallback when neither condition holds. Nu- meric spans receive deterministic perturbations; entity spans receive in-domain distractors with no token overlap with the original span. Corruption is seeded per trace and hop, so the selected replacement is reproducible.Samples with no certifiable span are skipped and counted rather than receiving an unverified change. The recorded original and corrupted spans certify the intervention and make deterministic answer-level 3 evaluation possible. They also make it possible to distinguish copying a wrong fact from producing another unrelated error. Diagnosers. The main comparisons use a coverage-based localizer (Doctor-RAG), a trace- reading LLM judge (LLM-Judge), frozen-hop counterfactual repair (Propagation-Aware), and suffix-regeneration repair (Suf-Regen).The frozen-hop probe repairs one candidate hop, holds all other hops fixed, and tests whether the answer becomes correct. The suffix-regeneration probe repairs the candidate hop and then regenerates the remaining trajectory. Thus the two active probes differ only in their treatment of downstream evi- dence: one preserves the observed suffix and the other permits the agent to construct a new suffix in the repaired context. The judge can inspect semantic trace content but incurs additional token cost; coverage gating requires no re-execution but uses answer support as its signal. A rule-based di- agnostic control and fuller implementation-neutral descriptions appear in Appendix A. 5 Metrics Attribution identifiability. For injected traces that remain incorrect after suffix re-execution, exact-hop accuracy for diagnoser d is Acc d (h) = 1 |F h | X τ i ∈F h 1[ ˆ h d,i = h i ], where F h is the set of failed injected traces at depth h. Recovered trajectories are excluded be- cause there is no final failure to attribute. Accu- racy is accompanied by bootstrap 95% confidence intervals using B=1,000 resamples. Cells with fewer than 10 failed traces, including many CRAG cells, are reported as descriptive estimates rather than used for comparative conclusions. In particu- lar, the main content-fault comparison excludes its depth-3 method ranking because only three failed cases remain after filtering. The denominator is deliberately the set of traces that remain failed after live re-execution. Includ- ing recovered trajectories as incorrect diagnoses would conflate root-cause localization with the separate question of whether a fault reached the answer. Conversely, reporting only answer ac- curacy would discard the information carried by a certified intervention. This conditional metric makes the two outcomes visible: recovery mea- sures resilience, and attribution measures localiza- tion among failures that actually propagated. Localization beyond exact hop. Exact-hop ac- curacy is intentionally demanding when faults are causally entangled across hops. Stage accuracy asks whether ˆs = s. Hop-tolerance accuracy ac- cepts | ˆ h − h| ≤ 1. Ancestor-hit rate counts a prediction when ˆ h ≤ h and ˆs = s, giving par- tial credit when a predicted earlier hop is a causal ancestor of the injected fault. Mean absolute hop error summarizes the distance between ˆ h and h. These metrics separate broad failure recognition from localization of the intervention. They are secondary diagnostics and are not used for the cen- tral empirical claim. Counterfactual recovery. Recovery is the frac- tion of live interventions whose corrupted trajec- tory still produces a correct answer: Recovery(h) = Pr[correct(A,y)| do(f,h)]. It is a separate robustness outcome. High recovery means that an intervention did not propagate into an end-to-end failure, not that a diagnoser local- ized it successfully. Deterministic generation outcomes. For con- tent faults, the certified span pair supports a judge- free answer-level classification: absorbed A contains changed-span tokens, resistedif correct(A,y), derailedotherwise. Correctness takes precedence when an answer contains both the gold and corrupted values. Query contamination records whether the cor- rupted span appears in a later sub-query, provid- ing an observable channel for propagation. The outcome labels answer a different question from exact-hop attribution. They characterize the effect of a known content change on the generated an- swer even when the resulting trajectory is correct or has too little evidence for a localization compar- ison. Because the labels are computed from the certified span pair, a value can be checked con- sistently across backbones without introducing a second model’s interpretation of whether the error was copied. Cost per correct diagnosis. For token-spending diagnosers, total diagnosis tokens are divided by 4 the number of correct localizations. This distin- guishes a method’s localization quality from de- ployability: an accurate method can still be un- suitable for online diagnosis if each correct local- ization requires a disproportionate amount of re- execution or judging. 6 Experimental Setup Datasetsandcorpora. HotpotQAand MuSiQueareanchormulti-hopquestion- answering datasets with annotated supporting facts.FRAMES supplies variable-depth RAG questions with Wikipedia passage corpora, and CRAG contributes dynamic, long-tail, and false- premise questions. Headline FRAMES conditions use fetched and cached Wikipedia passage text. Conditions that use only link titles as a stand-in corpus are excluded from headline comparisons and flagged separately because that substitute changes retrieval fidelity. Backbones and retrieval. The strict structural result reported in the main text uses Claude Haiku 4.5 with dense retrieval on three-hop MuSiQue examples and a four-probe budget. The reposi- tory also contains BM25, GPT-4o-mini, and local- model conditions, but incomplete or low-failure- count cells are not used for the main comparison. Static controls use BM25 and token-overlap re- trieval; their results are retained as appendix con- trols, not as the central causal experiment. Structural interventions and depth eligibility. The structural experiment injects empty retrieval, irrelevant retrieval, false-premise evidence, and stale evidence at hops 1, 2, and 3. For each sam- ple and requested depth, all four intervention types apply to the same base trace, then the downstream suffix is re-executed. A depth-specific case is eli- gible only when the base trace answered correctly and the actual injected hop equals the requested depth. The primary structural summary uses the available records that support these checks. The matrix is not yet a complete factorial sweep across every backbone, retriever, and depth, so the results are reported as evidence about the observed post- hoc signal rather than as a universal method rank- ing. Content-corruption conditions. The content- corruption study uses GPT-4o-mini and Claude Haiku 4.5 on HotpotQA and MuSiQue with BM25, n=40 base examples per condition, and DepthDoctor-RAGLLM-JudgePropagation-Awaren Hop 10.91 [0.81, 0.98]0.26 [0.12, 0.40]0.51 [0.37, 0.67]43 Hop 20.00 [0.00, 0.00]0.25 [0.11, 0.42]0.25 [0.11, 0.39]36 Hop 30.00 [0.00, 0.00]0.43 [0.24, 0.67]0.48 [0.29, 0.67]21 Table 2: Exact-hop attribution for the strict dense Claude Haiku 4.5 MuSiQue sweep. Entries are accu- racy [bootstrap 95% interval]; n is the per-depth count of failed injected traces. depths 1–3. All five diagnosers are evaluated. Traces from one model family are judged by the other family, reducing self-diagnosis while retain- ing the same trace-level task. The resulting answer labels use the certified span pair and therefore do not depend on the LLM judge. Recovery leaves 44 failed cases at hop 1, 18 at hop 2, and 3 at hop 3 in the pooled analysis. The main text therefore treats the depth-2 comparison as exploratory and reports depth-3 method estimates only in the appendix. Evaluation units. For structural conditions, a requested depth and intervention family define an evaluation unit over eligible base traces. For con- tent conditions, the span selection strategy addi- tionally determines whether a fault changes an an- swer fact, bridge entity, or salient fallback. All di- agnosers receive the same post-intervention trace for a unit. Active probes then create their own counterfactual repairs from that trace, so their ad- ditional evidence is part of the diagnoser rather than a change to the benchmark label. 7 Results 7.1 Structural Attribution Across Depths The completed strict structural sweep shows a loss of post-hoc coverage signal after suffix re- execution.Table 2 reports all three requested depths for dense Claude Haiku 4.5 on MuSiQue. The per-cell n values are failed injected trajecto- ries eligible for exact-hop scoring; each estimate includes its nonparametric bootstrap 95% interval. At hop 1, coverage identifies the deliberately visible structural fault. At later hops, coverage is 0.00 in both cells, while the judge and frozen- hop probe retain partial, overlapping-interval sig- nal. The result does not support a general ordering between active and post-hoc methods. It shows in- stead that a regenerated suffix can erase the cover- age signature of the injected hop while preserving some information that a more expensive probe can exploit. 5 DiagnoserHop 1 (n=44)Hop 2 (n=18) Doctor-RAG (coverage)1.00 [1.00, 1.00]0.00 [0.00, 0.00] LLM-Judge (cross-family)0.59 [0.43, 0.73]0.89 [0.72, 1.00] † Propagation-Aware0.89 [0.80, 0.98]0.67 [0.44, 0.89] Suf-Regen0.91 [0.82, 0.98]0.11 [0.00, 0.28] Table 3: Exact-hop attribution on content faults, pooled over GPT-4o-mini and Claude Haiku 4.5 on HotpotQA and MuSiQue with BM25. Column headings give the failed-case n. The hop-2 estimates are exploratory: their intervals quantify uncertainty within this pooled sample, not a definitive head-to-head comparison. The degenerate 0.00 interval has all bootstrap resamples equal to zero. † The judge’s deep result includes a po- sitional prior. Depth-3 values (n=3) are reported de- scriptively in Appendix B. 7.2 Counterfactual Probe Performance The available structural results do not establish a broad exact-hop advantage for frozen-hop repair over post-hoc diagnosis. The content-fault study instead provides a targeted examination of how two counterfactual scopes behave when the evi- dence remains topically relevant. Table 3 pools the four content conditions. At hop 2, coverage has no correct attributions in the observed sample, while frozen-hop repair reaches 0.67. These esti- mates are exploratory because the pooled depth-2 denominator is 18 failed cases. The two counterfactual probes differ in how they treat downstream evidence.At depth 2, Suf-Regen is 0.11 because regenerating a suffix can retrieve clean evidence and remove the con- tent fault under test. Frozen-hop repair prevents that change. In a small bridge-entity slice, Suf- Regen is 1.00 versus 0.73 for Propagation-Aware (n=11), which is consistent with the need to re- generate later hops that depend on a corrupted bridge.This slice is descriptive rather than a method-ranking result. The judge’s 0.89 at hop 2 should also be qualified: cross-family judg- ing excludes self-recognition, but the predicted- hop distribution has a strong mid-trace preference. This makes the estimate unsuitable as a standalone measure of semantic localization. 7.3 Answer Outcomes After Content Corruption Attribution evaluates only injected trajectories that remain failed. The certified span pair also makes it possible to measure what happened to every gen- erated answer, including trajectories that recover. Table 4 reports absorbed, resisted, and derailed outcomes over all content-fault interventions. At DepthAbsorbedResistedDerailedn Hop 10.150.580.26106 Hop 20.090.740.1868 Hop 30.000.850.1520 Table 4: Deterministic answer outcomes against cer- tified corruption spans, pooled over the four content- corruption conditions. Per-backbone hop-1 rates are reported in Appendix C. hop 1, 0.15 of cases are absorbed verbatim, 0.58 are resisted, and 0.26 are derailed. The later-depth rows contain all injected cases at their respective depths, not only the failed cases used in the attri- bution table. Absorption is strongly answer-shaped. Of 22 absorbed cases, 20 are answer-fact corruptions; salient-entity corruptions are absorbed in 0 of 40 cases. Thus a corrupted value that resembles a di- rect answer is often repeated, whereas a corrupted chain link more often prevents a coherent answer. Query contamination is only 3 of 129 cases. The changed values therefore usually affect the rea- soning that reaches an answer rather than being copied into a later sub-query. Monitoring query text alone would consequently miss most observed content-fault propagation. 8 Discussion Post-hoc signal after suffix re-execution. Cov- erage gating attributes a failure to the earliest hop with low answer support.This can be effec- tive when an intervention remains locally visible: empty evidence at hop 1 produces low coverage at hop 1 and the agent does not recover. The as- sumption fails once the agent re-executes a suf- fix. A later hop can mask the original fault by re- trieving compensating evidence, or it can propa- gate the fault by issuing a query already biased by the corrupted context. Both outcomes replace the local signature at the injection point with evidence about the later trajectory. The observed collapse is evidence of an iden- tifiability limitation for the coverage signal avail- able in these interventions. A final trace may show that an answer is unsupported, yet contain no reli- able feature that distinguishes an error injected at one hop from consequences that emerge later. Im- proving a coverage threshold cannot restore infor- mation that the regenerated suffix has overwritten. The result does not establish an impossibility the- orem for all agentic RAG systems; it identifies a failure mode that any post-hoc localizer must ad- 6 dress. This interpretation does not require every later hop to be wrong. A later retrieval can be topically relevant and still be a downstream consequence of a faulty earlier premise. It can also partially re- pair an earlier failure without restoring an identi- fiable trace signature. The relevant distinction is between the information an agent needs to answer and the information a post-hoc diagnoser needs to identify the intervention hop. The former can be recovered by compensating evidence even when the latter has been lost. Counterfactual repair under downstream de- pendence. Frozen-hop repair tests a useful causal question: does repairing candidate hop h change the answer when all later observed hops are held fixed? It can mislocalize when those later hops were generated from the original fault. Re- pairing the true early cause may not change the fi- nal answer because a later hop still carries the cor- rupted dependency. Conversely, repairing a later answer-bearing hop can restore correctness and receive credit despite being a downstream conse- quence. This is why broad stage identification can be more favorable than exact injected-hop attribu- tion. Suffix regeneration addresses the first problem by rebuilding downstream hops after repairing a candidate. It is therefore suited to bridge-entity corruption, where later queries and retrievals de- pend on the corrupted bridge. The same operation can obscure deep answer-fact corruption: regen- erating a suffix retrieves clean evidence and re- moves the very fault under test. In that setting, frozen-hop repair preserves the corruption at the other hops and is the more diagnostic probe. The observed contrast between 0.11 for suffix regen- eration and 0.67 for frozen-hop repair at depth 2, together with the bridge-entity slice, is consistent with this difference in counterfactual scope. The depth-2 and bridge-entity samples are small, so this is a mechanism-oriented interpretation rather than a definitive comparative result. The two probes should therefore not be read as interchangeable versions of the same repair. Frozen-hop repair asks whether the observed downstream trajectory would support a correct an- swer after changing one local input. Suffix regen- eration asks whether a repaired prefix can lead the agent to a different downstream trajectory. Each question is useful, but each can award credit to a different position in a causal chain.Exact- hop evaluation exposes this difference directly, whereas an answer-only evaluation would collapse both repairs into a single correctness outcome. Recovery as a separate outcome. Live inter- ventions reveal a second property that static trace edits cannot measure: the agent can sometimes heal a corrupted trajectory through a later retrieval. This is not a failure of the intervention. It identifies a case in which the selected fault did not reach the final answer, so no root cause should be credited for an end-to-end failure. The distinction is partic- ularly important for content corruption, where the executed trajectory contains a changed span but later retrieval can access clean corpus evidence. A persistent corpus-level corruption would test a dif- ferent mechanism and remains future work rather than a feature of the present evaluation. Implications for diagnosis. The appropriate re- pair probe depends on the path by which informa- tion flows through the trajectory. A single uni- versal localizer must choose between preserving downstream evidence and regenerating it, and ei- ther choice can hide a different fault family. Fu- ture path-aware or cascaded diagnosis can select a probe based on suspected fault type, then report uncertainty when multiple hops remain causally entangled. Such a design targets the dependence structure that the present exact-hop evaluation ex- poses. 9 Limitations The strict dense Claude Haiku 4.5 MuSiQue sweep has adequate failed-case counts at all three requested depths (43, 36, and 21). A matching strict dense HotpotQA sweep has only 9 and 2 failed trajectories at its valid depths because the agent recovers nearly all injected faults; it is there- fore descriptive and excluded from the main com- parison. The paper does not claim a complete backbone-by-dataset factorial evaluation, and it excludes CRAG from headline results because CRAG is normalized here as a single-turn bench- mark. The content intervention corrupts a trajectory copy while the retrieval corpus remains clean, al- lowing re-retrieval to heal some faults. This leaves 18 failed cases at depth 2 and 3 at depth 3 in the pooled content study; the depth-2 comparison is exploratory and no depth-3 method comparison is 7 made. A final content study should pre-specify a larger failed-case target at each depth and include persistent corpus corruption to distinguish recov- ery from a clean index from recovery caused by later reasoning. Exact-hop accuracy is intention- ally strict, and stage-level scoring can credit a di- agnoser that finds the failure type but not its in- jected hop. LLM judges are sensitive to prompting and token cost, and their apparent depth robustness can include a positional prior. Corpus construction also affects intervention and recovery behavior. 10 Conclusion AgenticRAG-FP evaluates causal failure attribu- tion by injecting a known fault and allowing the agent to generate its downstream response. In the completed strict dense Claude-MuSiQue sweep, coverage-based post-hoc attribution is 0.91 at hop 1 and 0.00 at hops 2 and 3, with 43, 36, and 21 failed trajectories respectively. The smaller content-fault study produces the same coverage pattern at depth 2, where frozen-hop repair reaches 0.67 in an exploratory pooled comparison. Its de- terministic labels further show that 15% of hop- 1 corruptions are absorbed verbatim.Broader benchmark and backbone coverage is needed be- fore making a universal claim about agentic RAG diagnosis. Future work can evaluate persistent corpus corruption and path-aware diagnosis that chooses a probe by fault type. References Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2023. Self-RAG: Learn- ing to retrieve, generate, and critique through self- reflection. arXiv preprint arXiv:2310.11511. Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023.Ac- tive retrieval augmented generation. arXiv preprint arXiv:2305.06983. Shuguang Jiao, Chengkai Huang, Shuhan Qi, Xuan Wang, Yifan Li, and Lina Yao. 2026. Doctor-RAG: Failure-aware repair for agentic retrieval-augmented generation. arXiv preprint arXiv:2604.00865. Satyapriya Krishna, Kalpesh Krishna, Anhad Mo- hananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. 2024.Fact, fetch, and reason:A unified evaluation of retrieval-augmented generation. arXiv preprint arXiv:2409.12941. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Kuttler, Mike Lewis, Wen-tau Yih, Tim Rock- taschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge- intensive nlp tasks. In Advances in Neural Infor- mation Processing Systems. Judea Pearl. 2009. Causality: Models, Reasoning, and Inference, 2 edition. Cambridge University Press. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2021. MuSiQue: Mul- tihop questions via single-hop question composition. arXiv preprint arXiv:2108.00573. Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla,XiangsenChen,SajalChoudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, Lingkun Kong, Brian Moran, Jiaqi Wang, Yifan Ethan Xu, An Yan, Chenyu Yang, Eting Yuan, Hanwen Zha, Nan Tang, and 8 others. 2024. CRAG: Comprehensive RAG benchmark. arXiv preprint arXiv:2406.04744. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Ben- gio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018.HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pro- cessing. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in lan- guage models.In International Conference on Learning Representations. A Diagnoser details Rule detects empty retrieval, missing actions, empty answers, grounding overlap, and incorrect answers from the final trace. Doctor-RAG selects the earliest hop with insufficient gold-answer cov- erage and otherwise attributes the error to answer generation. LLM-Judge reads the hop-level trace and predicts a failing stage and hop. Propagation- Aware repairs a candidate hop, freezes the remain- ing hops, and tests the answer. Suf-Regen repairs a candidate hop and regenerates the suffix. All ac- tive probes test candidates in causal order and re- turn the earliest repair that restores a correct an- swer. B Descriptive depth-3 content-fault estimates Only three content-corruption trajectories at depth 3 remain failed after filtering. The point estimates below are included for completeness, but no com- parison or mechanism claim is based on them. 8 DiagnoserHop 3 (n=3) Doctor-RAG (coverage)0.00 LLM-Judge (cross-family)0.33 Propagation-Aware0.00 Suf-Regen0.00 Table 5: Descriptive exact-hop estimates for the pooled depth-3 content-corruption cases. The denominator is too small for an inferential comparison. C Additional content-fault outcome details The hop-1 absorbed rates are 0.18 / 0.19 / 0.11 / 0.11 across the four conditions. Content-fault recovery ranges from 0.54–0.85, compared with 0.00–0.71 for structural faults. The content curves in Figure 1 show 0.75 for Propagation-Aware at hop 2 on both HotpotQA backbones, while cover- age gating and suffix regeneration are zero; hop 3 is omitted when no injected trace remains failed. At depth ≥ 2, the discordant-pair comparison of Propagation-Aware with the best post-hoc en- velope contains 1 versus 6 pairs. This count is too small for a McNemar test, so the direct content- fault comparison concerns Propagation-Aware versus coverage gating rather than Propagation- Aware versus all post-hoc methods. 9 123 Injection depth (hop) 0.0 0.2 0.4 0.6 0.8 1.0 Root-cause attribution accuracy Attribution identifiability openai:gpt-4o-mini · hotpotqa (n_failed=[13, 4, 0]; recovery=[0.54, 0.64, 1.0]) PALLM-judge by depth: [0.154, 0.0, 0.0] Rule-based Doctor-RAG (coverage) LLM-as-judge Propagation-aware (ours) Suffix-regeneration (ours) Best post-hoc 123 Injection depth (hop) 0.0 0.2 0.4 0.6 0.8 1.0 Root-cause attribution accuracy Attribution identifiability claude:claude-haiku-4-5 · hotpotqa (n_failed=[13, 4, 0]; recovery=[0.59, 0.78, 1.0]) PALLM-judge by depth: [0.538, -0.25, 0.0] Rule-based Doctor-RAG (coverage) LLM-as-judge Propagation-aware (ours) Suffix-regeneration (ours) Best post-hoc Figure 1: Content-corruption curves on HotpotQA with BM25. Left: GPT-4o-mini agent judged by Claude Haiku 4.5. Right: Claude Haiku 4.5 agent judged by GPT-4o-mini. 10