Paper deep dive

INTRYGUE: Induction-Aware Entropy Gating for Reliable RAG Uncertainty Estimation

Alexandra Bazarova, Andrei Volodichev, Daria Kotova, Alexey Zaytsev

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 55

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/26/2026, 2:31:22 AM

Summary

INTRYGUE is a mechanistically grounded uncertainty quantification method for Retrieval-Augmented Generation (RAG) that addresses the 'mechanistic paradox' where induction heads, while promoting grounded responses, simultaneously trigger entropy neurons that inflate predictive entropy. By gating predictive entropy with induction head activation patterns (SinkRate), INTRYGUE improves hallucination detection performance across multiple LLMs and benchmarks.

Entities (5)

Entropy Neurons · mechanism · 100%Induction heads · mechanism · 100%INTRYGUE · method · 100%RAG · task · 100%SinkRate · metric · 95%

Relation Signals (3)

INTRYGUE → gates → Predictive Entropy

confidence 95% · INTRYGUE (Induction-Aware Entropy Gating for Uncertainty Estimation), a mechanistically grounded method that gates predictive entropy based on the activation patterns of induction heads.

Induction heads → activates → Entropy Neurons

confidence 90% · induction heads activate entropy neurons, thereby not deliberately decreasing LLM confidence

Entropy Neurons → inflates → Predictive Entropy

confidence 90% · These neurons inflate the LayerNorm denominator, artificially increasing the predictive entropy.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:While retrieval-augmented generation (RAG) significantly improves the factual reliability of LLMs, it does not eliminate hallucinations, so robust uncertainty quantification (UQ) remains essential. In this paper, we reveal that standard entropy-based UQ methods often fail in RAG settings due to a mechanistic paradox. An internal "tug-of-war" inherent to context utilization appears: while induction heads promote grounded responses by copying the correct answer, they collaterally trigger the previously established "entropy neurons". This interaction inflates predictive entropy, causing the model to signal false uncertainty on accurate outputs. To address this, we propose INTRYGUE (Induction-Aware Entropy Gating for Uncertainty Estimation), a mechanistically grounded method that gates predictive entropy based on the activation patterns of induction heads. Evaluated across four RAG benchmarks and six open-source LLMs (4B to 13B parameters), INTRYGUE consistently matches or outperforms a wide range of UQ baselines. Our findings demonstrate that hallucination detection in RAG benefits from combining predictive uncertainty with interpretable, internal signals of context utilization.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

55,198 characters extracted from source content.

Expand or collapse full text

INTRYGUE: Induction-Aware Entropy Gating for Reliable RAG Uncertainty Estimation Alexandra Bazarova1, Andrei Volodichev1, Daria Kotova1, Alexey Zaytsev1, 1Applied AI Institute, Correspondence: bazarovaai.239@gmail.com Abstract While retrieval-augmented generation (RAG) significantly improves the factual reliability of LLMs, it does not eliminate hallucinations, so robust uncertainty quantification (UQ) remains essential. In this paper, we reveal that standard entropy-based UQ methods often fail in RAG settings due to a mechanistic paradox. An internal “tug-of-war” inherent to context utilization appears: while induction heads promote grounded responses by copying the correct answer, they collaterally trigger the previously established “entropy neurons.” This interaction inflates predictive entropy, causing the model to signal false uncertainty on accurate outputs. To address this, we propose INTRYGUE (Induction-Aware Entropy Gating for Uncertainty Estimation), a mechanistically grounded method that gates predictive entropy based on the activation patterns of induction heads. Evaluated across four RAG benchmarks and six open-source LLMs (4B to 13B parameters), INTRYGUE consistently matches or outperforms a wide range of UQ baselines. Our findings demonstrate that hallucination detection in RAG benefits from combining predictive uncertainty with interpretable, internal signals of context utilization111Our code is available at the following link https://anonymous.4open.science/r/tda4hallucinations-1B39/README.md. INTRYGUE: Induction-Aware Entropy Gating for Reliable RAG Uncertainty Estimation Alexandra Bazarova1, Andrei Volodichev1, Daria Kotova1, Alexey Zaytsev1, 1Applied AI Institute, Correspondence: bazarovaai.239@gmail.com 1 Introduction Large language models (LLMs) are widely deployed Xie et al. (2024); Amorese et al. (2023), but their static memory is often insufficient. This is typically addressed with retrieval-augmented generation (RAG), which improves reliability by grounding output in external documents Fan et al. (2024); Lewis et al. (2020). Nonetheless, RAG does not fully eliminate LLM hallucinations Zhang et al. (2025). As ungrounded errors limit the use of LLMs in high-stakes environments, reliable methods for detecting these failures remain essential. Although many hallucination detection methods have been proposed Alansari and Luqman (2025), they frequently face practical limitations, such as relying on external databases Li et al. (2024) or introducing substantial computational overhead via complex prompting and multi-step verification pipelines Es et al. (2024); Friel and Sanyal (2023). To avoid these bottlenecks, we approach hallucination detection through uncertainty quantification (UQ) lens, directly exploiting the LLM’s internal confidence scores to gauge the reliability of its outputs. Figure 1: The mechanistic tug-of-war in RAG uncertainty quantification. ➀ During generation, Induction Heads exhibit intense attention activations, locking onto specific patterns in the retrieved context to promote relevant tokens into the output. This mechanism sharpens the logit distribution toward the correct answer, increasing entropy-based model confidence. However, ➁ it simultaneously activates Entropy Neurons. These neurons inflate the LayerNorm denominator, ➂ artificially increasing the predictive entropy. This dual activation results in a False Alarm, where correct, context-grounded outputs are incorrectly flagged as highly uncertain. While numerous UQ methods have been developed for closed-book settings Kuhn et al. (2023); Qiu and Miikkulainen (2024); Vazhentsev et al. (2025), there is only a very limited amount of work on UQ for the RAG scenario. Recent studies note that standard UQ techniques are often poorly suited for RAG, as their design does not incorporate specific RAG-setup properties, e.g. prompt-response semantic alignment Soudani et al. (2025). To address this gap, the existing approaches inject the document–response relationship into the uncertainty score; however, because these solutions rely on external models or multiple model generations, their applicability remains limited Soudani et al. (2025); Perez-Beltrachini and Lapata (2025). Moreover, the internal model mechanisms causing poor performance of UQ in RAG remain underexplored. In this work, we explore the mechanistic limitation that entropy-based uncertainty estimation Fomicheva et al. (2020) — a widely used baseline for hallucination detection Nikitin et al. (2024); Fadeeva et al. (2025); Qiao et al. (2026) — faces in RAG scenarios. We demonstrate that correct RAG-based responses are associated with the activation of induction heads, which are attention mechanisms that copy and propagate retrieved information Elhage et al. (2021). While these heads promote context-aligned responses and increase model confidence, they simultaneously trigger an opposing mechanism: entropy neurons Stolfo et al. (2024). These specialized units modulate predictive entropy and conversely act to reduce the LLM’s confidence. As a result, this internal tug-of-war makes entropy-based scores less reliable estimators of model uncertainty (Figure 1). To address this issue, we introduce INTRYGUE (Induction-Aware Entropy Gating), a mechanistically grounded method that adjusts entropy scores based on the activation strength of induction heads. INTRYGUE accounts for induction-driven entropy inflation and improves entropy’s effectiveness as a hallucination signal in the RAG settings. Our contributions: • We demonstrate that the correctness of LLM responses in RAG settings is strongly associated with the activation of the model’s induction heads, which significantly increase both the probability of the correct answer and overall model confidence; • We validate a connection between induction heads and entropy neurons, previously identified only in controlled induction settings involving exact token repetition and short naturally occurring n-gram repetitions. For real RAG-based decoding, induction heads activate entropy neurons, thereby not deliberately decreasing LLM confidence; • Consequently, the process of generating accurate RAG responses is shown to inherently involve two interconnected but competing mechanisms, which pull LLM confidence scores in opposite directions, resulting in unreliable output entropy-based uncertainty estimation; • To address this conflict, we propose INTRYGUE, a novel method that gates entropy using induction head attention sinks, effectively correcting induction-driven entropy inflation in RAG settings; • Our INTRYGUE is demonstrated to consistently outperform or perform on par with a wide range of uncertainty estimation baselines across four RAG benchmarks and six models, ranging from 4B to 13B parameters in size. 2 Background 2.1 Induction Heads Induction heads, which fundamentally drive in-context learning in Transformers, were formalized in Olsson et al. (2022). They implement a strict sequence-copying heuristic: given a token pattern (tA,tB,…,tA)(t_A,t_B,…,t_A), the head attends strongly to tBt_B to predict it as the subsequent token. To empirically identify them, one evaluates the model on a randomly generated, repeated sequence S=(s1,…,sL,s1,…,sL)S=(s_1,…,s_L,s_1,…,s_L) and computes an induction score. This score is defined as the average attention mass allocated to the correct historical offsets of S: InductionScore⁡(A(l,h))=1L∑j=1LαL+j,1+j(l,h),InductionScore(A^(l,h))= 1L _j=1^L _L+j,1+j^(l,h), (1) where A(l,h)A^(l,h) is the attention matrix of head h at layer l, and αt,k(l,h) _t,k^(l,h) denotes the attention weight from the current position t to the past position k. 2.2 Entropy Neurons In this paper, the term neurons refers to individual entries of the pre-activation hidden states of the MLP sublayer in the final transformer block: MLP()=outσ(in+in)+out,MLP(x)=W_outσ(W_inx+ β_in)+ β_out, (2) where ∈ℝdmodelx ^d_model is a normalized residual stream hidden state, out⊤,in∈ℝdmlp×dmodelW_out ,W_in ^d_mlp× d_model are learned weight matrices, in,out∈ℝdmlp β_in,\, β_out ^d_mlp are learned bias vectors, and σ is an activation function. The output of the final transformer block is then passed through layer normalization: LN()=−m()Var()+ϵ⊙+,LN(x)= x-m(x) Var(x)+ε γ+ β, (3) where m()m(x) and Var()Var(x) denote the mean and variance of the entries of x, and ,∈ℝdmodel γ, β ^d_model are learned biases. The resulting vector is then projected onto the vocabulary logits by the unembedding matrix U∈ℝ||×dmodelW_U ^|V|× d_model, with V being a dictionary size, followed by a softmax to obtain a probability distribution over tokens. Entropy neurons Gurnee et al. (2024) are characterized by a large norm of their corresponding column outw_out of outW_out and a near-uniform shift to the logits, quantified by the LogitVarLogitVar score: LogitVar⁡(out)=Var⁡(Uout‖U‖dim=1‖out‖).LogitVar(w_out)=Var ( W_Uw_out\|W_U\|_ =1\|w_out\| ). (4) The large norm, in turn, affects the LayerNorm scaling, squeezing the logit distribution and reducing model confidence, similarly to temperature scaling. The authors of Stolfo et al. (2024) have hinted that induction heads activate entropy neurons, but validation of this fact was limited to controlled induction settings with synthetic exact token repetition and short naturally occurring n-gram repetitions of fixed length leaving open whether this mechanism generalizes to broader contexts where the model must reconcile retrieved external knowledge with its own parametric memory. 3 Empirical Study: Entropy-based Uncertainty Estimation Underperforms in RAG In this section, we investigate why entropy-based uncertainty estimates tend to underperform in RAG settings. Our evidence suggests that this failure may be linked to the dual role played by induction heads, which we analyze in detail below. 3.1 Entropy-based Uncertainty Estimation Although entropy-based methods are a widely adopted baseline in hallucination detection Nikitin et al. (2024); Fadeeva et al. (2025); Qiao et al. (2026), their effectiveness in RAG settings remains limited. To illustrate this, we evaluate the performance of the Maximum Entropy Fomicheva et al. (2020) on the MS MARCO subset of the RAGTruth dataset Niu et al. (2024) and XSum Narayan et al. (2018). The results are shown in Figure 2. Although offering a weak signal regarding generation errors, the distributions of scores for hallucinated and non-hallucinated responses overlap heavily, demonstrating that entropy-based metrics provide unreliable uncertainty estimates in RAG settings. Figure 2: Maximum Entropy score distributions for hallucinated (pink) and grounded (blue) samples heavily overlap, demonstrating that Maximum Entropy fails to reliably separate hallucinated from grounded LLM responses. 3.2 Dual Role of Induction Heads Induction heads play a dual role in RAG response generation: they both boost and lower the model’s confidence in the correct response. On one hand, induction heads are responsible for copying relevant tokens from the context, directly contributing to the generation of faithful responses, increasing confidence. On the other hand, they activate entropy neurons, which inflate the entropy of the output distribution, lowering the model’s confidence. This dual role may help explain why entropy-based uncertainty estimates tend to be unreliable in RAG settings. 3.2.1 Induction Heads Pull Confidence Up… Correlation. To investigate whether induction head activation is predictive of hallucination in RAG-based responses, we compute the average SinkRate for the top-5 induction heads222We compute the InductionScore (1) for all heads and select the 5 heads with the greatest scores. and evaluate its ability to discriminate between hallucinated and grounded responses using the AUROC score. For an attention matrix A∈ℝN×NA ^N× N, where N is the total sequence length, the SinkRate333This definition is motivated by our empirical observation that attention frequently sinks to utility tokens (e.g., punctuation marks and brackets) that are not necessarily located at the beginning of the prompt (see Appendix B). is defined as: SinkRate⁡(A,n)=maxj⁡∑i=N−nNαi,jwj,SinkRate(A,n)= _j _i=N-n^N _i,jw_j, (5) where n is the response length and wj=min⁡(n,N−j+1)w_j= (n,N-j+1) is a normalization factor that accounts for the number of response tokens able to attend to position j. A greater SinkRate indicates deactivation of the induction head, suggesting that induction has not occurred Elhage et al. (2021). As shown in Figure 3, hallucinated samples consistently exhibit higher SinkRate across all four settings, with statistically significant differences (p<10−4p<10^-4, Mann-Whitney U-test) and AUROC scores ranging from 0.63 to 0.74, implying that the deactivation of induction heads may serve as a potential signal of hallucination in RAG settings. Figure 3: SinkRate scores distributions: hallucinated (pink) vs grounded (blue). AUROC scores and p-values of the Mann-Whitney U-tests are provided in the legend. Figure 4: (a)–(c) Percentage change in Negative Log-Likelihood (NLL) and Entropy following mean ablation of induction heads (blue) versus random heads (green). Ablating induction heads results in significantly higher uncertainty and loss across all models. (d)–(e) Spearman rank correlation (ρ) between the Sink Rate of induction heads and the activations of the top-15 neurons with the lowest LogitVar scores; this indicates that induction head behavior is associated with the activity of these neurons. Causality. To establish a causal link between induction head activation and correct answer generation, we ran a mean ablation experiment on the CoQA dataset (Reddy et al., 2019). Mean ablation replaces a head’s output with its mean activation computed over a reference dataset, effectively removing the head’s contribution to the forward pass. For correctly answered examples with the greatest induction head activation (SinkRate below the 0.33 quantile), we ablated the top-10 induction heads and compared the effect to ablating 10 randomly chosen heads. Figure 4a)-c) shows that ablating induction heads sharply increases loss and entropy, drastically increasing uncertainty compared to randomly chosen heads. This asymmetry indicates that induction heads play a causal role in context retrieval, rather than just correlating with it, and contribute substantially to accurate RAG generation. 3.2.2 …and Push It Back Down Correlation. To validate the connection between entropy neurons and induction heads in RAG settings, we examine the correlations between the average SinkRate (over the top-10 induction heads) and the activation scores of the top-15 entropy neurons444We compute the LogitVar (4) for all neurons in the MLP layer of the last transformer block and select the 15 neurons with the lowest scores., where the activation score refers to the maximum activation value of a neuron taken across the tokens of the generated response. Figure 4d)-e) shows that for both Qwen3-8B and Llama-3.1-8B, most entropy neurons have a negative Spearman correlation with the SinkRate. This supports the idea that induction heads drive entropy neuron excitation, confirming that the findings of Stolfo et al. (2024) translate from controlled setups to real scenarios. Figure 5: (a) Drop in entropy neuron activations (Δℓ2 Post−Pre _2 Post-Pre) when ablating induction heads (blue), confirming causal dependence. (b) Change in entropy (Pre−PostPre-Post) after ablating entropy neurons; the positive values indicate that activations of these neurons inflate model confidence. Causality. To establish the causal influence of induction heads on entropy neurons, we performed mean ablation of the top-k (k=4k=4 for Gemma; k=1k=1 for Llama) induction heads and measured the resulting change in entropy neuron activations via the l2l_2-norm — a natural choice given that entropy neurons act by scaling the LayerNorm denominator (3). As a control, we repeated the same procedure for k randomly selected heads. Figure 5a) shows that ablating induction head consistently decreases the l2l_2-norm of entropy neuron activations for Llama-3.1-8B and Mistral-7B, while this observation doesn’t stand for random heads. This confirms a causal link between induction head activity and entropy neuron excitation. Impact on entropy scores. We further investigated whether identified entropy neurons causally influence model entropy-based uncertainty by performing targeted ablation experiments on the CoQA dataset. We compared the effect of mean-ablating the top-30 entropy neurons with the ablation of 50 randomly selected neurons within the same MLP layer. Figure 5b) illustrates that ablating entropy neurons yields a substantial increase in generation entropy, while random ablation produces no significant change. These findings confirm that these specific neurons actively modulate the model’s predictive confidence. 4 Mechanistic Uncertainty Quantification Our findings indicate that standard entropy-based estimators overlook the controversial impact of induction heads activation, a mechanism critical for faithful RAG generation. To address this, we propose INTRYGUE (Induction-Aware Entropy Gating for Uncertainty Estimation). Our method uses aggregated SinkRate (5), which quantifies activity of induction heads, as a gating term; it modulates the raw entropy score, amplifying the estimated uncertainty when the model fails to mechanistically attend to the retrieved context. Let P denote the input prompt and R=(r1,…,rn)R=(r_1,…,r_n) be the generated response sequence of tokens. At each generation step t, the model outputs a probability distribution over the vocabulary V. We derive the sequence ℰ=ett=1nE=\e_t\_t=1^n by calculating the entropy of these distributions to quantify the uncertainty of each predicted token: et=−∑v∈P(v∣P,R<t)log⁡P(v∣P,R<t).e_t=- _v P(v P,R_<t) P(v P,R_<t). (6) Simultaneously, we extract a set of SinkRates =sii=1kS=\s_i\_i=1^k defined in Eq. 5 corresponding to top-k induction heads with greatest InductionScore (1) values. These rates serve as a mechanistic proxy for quantifying the extent of induction during the generation process. The hyperparameter k is selected on a validation set. MS MARCO CNN/DM Mistral-7B Llama-2-7B Llama-2-13B Mistral-7B Llama-2-7B Llama-2-13B INTRYGUEmin-maxINTRYGUE_min-max 0.77 ± 0.03 0.72 ± 0.04 0.73 ± 0.04 0.69 ± 0.03 0.60 ± 0.03 0.60 ± 0.08 INTRYGUEmeanINTRYGUE_mean 0.71 ± 0.06 0.67 ± 0.02 0.67 ± 0.04 0.65 ± 0.03 0.62 ± 0.08 0.58 ± 0.03 Attention Score 0.49 ± 0.05 0.47 ± 0.07 0.51 ± 0.05 0.57 ± 0.07 0.52 ± 0.03 0.51 ± 0.06 Focus 0.67 ± 0.06 0.63 ± 0.03 0.61 ± 0.03 0.62 ± 0.04 0.51 ± 0.09 0.50 ± 0.06 LN-Entropy 0.65 ± 0.06 0.63 ± 0.03 0.62 ± 0.04 0.64 ± 0.04 0.62 ± 0.09 0.56 ± 0.05 LSC 0.68 ± 0.04 0.57 ± 0.03 0.57 ± 0.05 0.58 ± 0.05 0.49 ± 0.07 0.52 ± 0.07 MaxEntropy 0.67 ± 0.03 0.65 ± 0.06 0.62 ± 0.05 0.61 ± 0.04 0.60 ± 0.04 0.54 ± 0.09 MaxProb 0.75 ± 0.04 0.69 ± 0.03 0.67 ± 0.04 0.61 ± 0.07 0.51 ± 0.07 0.53 ± 0.08 Perplexity 0.64 ± 0.05 0.48 ± 0.03 0.46 ± 0.03 0.49 ± 0.08 0.52 ± 0.04 0.48 ± 0.04 RAUQ 0.56 ± 0.06 0.53 ± 0.06 0.54 ± 0.04 0.49 ± 0.08 0.52 ± 0.07 0.52 ± 0.05 ReDeEP 0.72 ± 0.03 0.63 ± 0.05 0.66 ± 0.03 0.56 ± 0.06 0.57 ± 0.06 0.52 ± 0.03 EigenScore 0.60 ± 0.04 0.58 ± 0.03 0.58 ± 0.04 0.63 ± 0.08 0.59 ± 0.03 0.57 ± 0.08 SAR 0.67 ± 0.04 0.55 ± 0.02 0.48 ± 0.03 0.57 ± 0.04 0.48 ± 0.10 0.52 ± 0.05 Semantic Density 0.55 ± 0.02 0.55 ± 0.05 0.53 ± 0.02 0.56 ± 0.08 0.52 ± 0.04 0.57 ± 0.07 Semantic Entropy 0.60 ± 0.05 0.53 ± 0.04 0.62 ± 0.04 0.54 ± 0.04 0.50 ± 0.06 0.54 ± 0.03 Table 1: AUROC scores for hallucination detection on subsamples of RAGTruth for QA (MS MARCO) and Summarization (CNN/DM). Bold and underline indicate the best and second-best results, respectively. All results are averaged over 5 runs. Let f and g be aggregation functions (e.g., mean or max) that map these sequences to scalars. We employ the aggregated SinkRate as a coefficient that gates the predictive entropy: when induction fails (high SinkRate), the uncertainty signal is amplified; when induction is successful (low SinkRate), the signal is dampened. We thus formulate our score, INTRYGUE, as: INTRYGUE⁡(P,R)=f(sii=1k)⏟InductionActivity⋅g(ett=1n)⏟PredictiveUncertainty.INTRYGUE(P,R)= f (\s_i\_i=1^k )_ subarraycInduction\\ Activity subarray· g (\e_t\_t=1^n )_ subarraycPredictive\\ Uncertainty subarray. (7) In practice, the choice of aggregation functions f and g allows the metric to be tuned to capture either the average uncertainty or the most severe failure points across the generated response. 5 Results 5.1 Experiment setting Datasets and models. We evaluate on four question answering (QA) and summarization tasks: long-form QA (MS MARCO) and summarization (CNN/DM) from RAGTruth Niu et al. (2024), conversational QA (CoQA Reddy et al. (2019)), and extreme summarization (XSum Narayan et al. (2018)). Additional details are in Appendix C. We evaluate six popular open-source LLMs that provide access to their internal states: gemma-3-4b-it (Team, 2025a), LLaMA-2-7B-chat, LLaMA-2-13B-chat (Touvron et al., 2023), Mistral-7B-Instruct-v0.1 (Jiang et al., 2023), LLaMA-3.1-8B-Instruct (Weerawardhena et al., 2025), and Qwen3-8B (Team, 2025b). As annotated responses for gemma-3-4b-it, LLaMA-3.1-8B, and Qwen3-8B are not available in the RAGTruth dataset, we limit the evaluation of these models to CoQA and XSum. Baselines. We compare our method against a comprehensive set of baselines, including information-based: AttentionScore (Sriramanan et al., 2024), Focus (Zhang et al., 2023), Length-Normalized Entropy (Malinin and Gales, ), Lowest Span Confidence (LSC) (Qiao et al., 2026), MaxEntropy (Fomicheva et al., 2020), MaxProb (Fomicheva et al., 2020), Perplexity (Fomicheva et al., 2020), and RAUQ (Vazhentsev et al., 2025); sampling diversity based: EigenScore (Chen et al., ), SAR (Duan et al., 2024), Semantic Density (Qiu and Miikkulainen, 2024), and Semantic Entropy (Kuhn et al., 2023); mechanistic intepretability-based: ReDeEP (Sun et al., 2025). Implementation details are available in Appendix D. 5.2 Main results CoQA Mistral-7B Llama-2-7B Llama-2-13B Llama-3.1-8B Qwen3-8B gemma-3-4b INTRYGUEmin-maxINTRYGUE_min-max 0.79 ± 0.04 0.76 ± 0.04 0.72 ± 0.05 0.78 ± 0.08 0.77 ± 0.07 0.73 ± 0.05 INTRYGUEmeanINTRYGUE_mean 0.82 ± 0.04 0.79 ± 0.03 0.81 ± 0.03 0.81 ± 0.07 0.80 ± 0.06 0.76 ± 0.04 Attention Score 0.55 ± 0.03 0.54 ± 0.06 0.60 ± 0.06 0.56 ± 0.04 0.58 ± 0.09 0.51 ± 0.10 Focus 0.53 ± 0.05 0.33 ± 0.02 0.31 ± 0.03 0.53 ± 0.09 0.42 ± 0.04 0.48 ± 0.03 LN-Entropy 0.83 ± 0.03 0.74 ± 0.03 0.80 ± 0.04 0.81 ± 0.08 0.79 ± 0.05 0.75 ± 0.04 LSC 0.72 ± 0.03 0.70 ± 0.03 0.55 ± 0.05 0.77 ± 0.07 0.75 ± 0.06 0.58 ± 0.09 MaxEntropy 0.78 ± 0.03 0.60 ± 0.03 0.65 ± 0.06 0.75 ± 0.06 0.73 ± 0.05 0.70 ± 0.07 MaxProb 0.72 ± 0.05 0.54 ± 0.03 0.60 ± 0.08 0.68 ± 0.09 0.50 ± 0.09 0.58 ± 0.06 Perplexity 0.59 ± 0.04 0.52 ± 0.05 0.58 ± 0.10 0.50 ± 0.08 0.43 ± 0.07 0.51 ± 0.04 RAUQ 0.40 ± 0.02 0.41 ± 0.05 0.35 ± 0.02 0.45 ± 0.06 0.36 ± 0.04 0.41 ± 0.05 ReDeEP 0.62 ± 0.04 0.67 ± 0.06 0.77 ± 0.03 0.67 ± 0.05 0.59 ± 0.01 0.59 ± 0.05 EigenScore 0.69 ± 0.05 0.55 ± 0.05 0.60 ± 0.03 0.73 ± 0.05 0.70 ± 0.05 0.71 ± 0.05 SAR 0.72 ± 0.02 0.68 ± 0.04 0.76 ± 0.06 0.64 ± 0.06 0.63 ± 0.07 0.58 ± 0.06 Semantic Density 0.61 ± 0.03 0.59 ± 0.05 0.63 ± 0.04 0.59 ± 0.08 0.63 ± 0.06 0.58 ± 0.08 Semantic Entropy 0.79 ± 0.03 0.64 ± 0.05 0.72 ± 0.05 0.82 ± 0.06 0.72 ± 0.05 0.70 ± 0.05 XSum INTRYGUEmin-maxINTRYGUE_min-max 0.73 ± 0.07 0.66 ± 0.07 0.63 ± 0.05 0.66 ± 0.02 0.68 ± 0.06 0.62 ± 0.06 INTRYGUEmeanINTRYGUE_mean 0.70 ± 0.03 0.70 ± 0.03 0.62 ± 0.03 0.64 ± 0.03 0.66 ± 0.04 0.58 ± 0.06 Attention Score 0.59 ± 0.03 0.50 ± 0.08 0.56 ± 0.04 0.51 ± 0.05 0.46 ± 0.06 0.50 ± 0.02 Focus 0.63 ± 0.08 0.65 ± 0.06 0.58 ± 0.04 0.62 ± 0.05 0.61 ± 0.03 0.61 ± 0.04 LN-Entropy 0.69 ± 0.03 0.61 ± 0.02 0.62 ± 0.04 0.64 ± 0.03 0.65 ± 0.04 0.57 ± 0.06 LSC 0.64 ± 0.05 0.57 ± 0.05 0.58 ± 0.04 0.59 ± 0.08 0.62 ± 0.04 0.53 ± 0.07 MaxEntropy 0.67 ± 0.02 0.60 ± 0.06 0.57 ± 0.07 0.62 ± 0.02 0.67 ± 0.06 0.58 ± 0.06 MaxProb 0.60 ± 0.07 0.58 ± 0.05 0.58 ± 0.04 0.62 ± 0.04 0.63 ± 0.04 0.53 ± 0.05 Perplexity 0.58 ± 0.05 0.56 ± 0.05 0.53 ± 0.04 0.61 ± 0.05 0.63 ± 0.03 0.47 ± 0.07 RAUQ 0.46 ± 0.06 0.43 ± 0.04 0.47 ± 0.08 0.46 ± 0.07 0.43 ± 0.04 0.50 ± 0.01 ReDeEP 0.62 ± 0.04 0.61 ± 0.04 0.60 ± 0.04 0.63 ± 0.02 0.69 ± 0.05 0.62 ± 0.06 EigenScore 0.60 ± 0.08 0.60 ± 0.05 0.52 ± 0.03 0.66 ± 0.04 0.62 ± 0.06 0.61 ± 0.08 SAR 0.66 ± 0.03 0.60 ± 0.03 0.58 ± 0.05 0.63 ± 0.04 0.64 ± 0.03 0.50 ± 0.05 Semantic Density 0.63 ± 0.03 0.63 ± 0.06 0.59 ± 0.05 0.55 ± 0.06 0.60 ± 0.07 0.58 ± 0.06 Semantic Entropy 0.64 ± 0.05 0.63 ± 0.03 0.62 ± 0.02 0.61 ± 0.05 0.67 ± 0.04 0.62 ± 0.06 Table 2: Hallucination detection results (AUROC score) on CoQA and XSum datasets. Bold and underline indicate the best and second-best results, respectively. All results are averaged over 5 runs. The main experimental results are presented in Tables 1-2. We report two implementations of our proposed method: INTRYGUEmin-maxINTRYGUE_min-max (where f and g are the sequence minimum and maximum, respectively) and INTRYGUEmeanINTRYGUE_mean (where both are the average). As shown in Tables 1 and 2, INTRYGUE consistently matches or outperforms both attention-based (e.g., RAUQ, Attention Score) and logit-based baselines (e.g., MaxProb, LN-Entropy) across all evaluated benchmarks. This suggests that integrating distinct uncertainty signals, internal induction dynamics and predictive entropy, enables more robust uncertainty quantification. Importantly, the optimal aggregation variant depends heavily on response length. For the long, detailed generations of RAGTruth (MS MARCO, CNN/DM) and XSum, averaging over the sequence blurs localized hallucination signals. Consequently, INTRYGUEmin-maxINTRYGUE_min-max dominates these settings, achieving the top score in 9 out of 12 corresponding configurations. Conversely, INTRYGUEmeanINTRYGUE_mean proves most effective for CoQA with responses consisting of only a few tokens and averaging is more appropriate. Illustrative study. Figures 6b)-c) compare MaxEntropy and mean SinkRate predictions on CoQA (thresholds for both methods are selected on the same validation set to maximize accuracy). Quadrants show sample counts in bold and subset accuracies percentages. Notably, the bottom-left quadrant reveals that when Entropy predicts “hallucination” but SinkRate predicts “grounded”, Entropy significantly underperforms (e.g., 8.1% accuracy for Qwen). This false uncertainty is likely caused by induction head dynamics (Section 3). INTRYGUE effectively mitigates this by lowering entropy scores wherever the SinkRate is low. While the reverse scenario (top-right quadrant) also occurs, it affects a smaller fraction of the dataset (8–10% for both models). 5.3 Ablation studies Figure 6: (a) Performance stability of INTRYGUEmeanINTRYGUE_mean across different values of k (number of induction heads). (b-c) LN-Entropy vs. SinkRate prediction accuracy on CoQA. The bottom-left quadrant demonstrates SinkRate’s ability to correct LN-Entropy errors, a mechanism directly exploited by INTRYGUE. Figure 7: Impact of various non-linear transformations (f) on INTRYGUE performance. Higher AUROC scores are mapped closer to the center. Applying non-linear transformations does not allow for significant improvement in performance. Figure 8: Computational complexity. Runtime in seconds of our method compared to information- and sampling-based baselines. Results are obtained using 20 samples from the XSum dataset with the average over ten runs presented. Gating functions We tested whether applying non-linear transformations (e.g., Tanh, Softsign) to the aggregated SinkRates could improve the INTRYGUE score (7). The results in Figure 7 (and Appendix B) demonstrate that these adjustments offered no significant benefit over using the raw values directly. This confirms that our method works effectively without the need for added complexity. INTRYGUEmin-maxINTRYGUE_min-max vs. isolated signals Table 4 (Appendix B) compares our method to isolated mechanistic (SinkRate) and uncertainty (MaxEntropy) signals. As individual metric performance varies by dataset, single-signal detection proves unreliable. By fusing both features, INTRYGUE consistently outperforms the strongest baselines, demonstrating that these signals are insufficient alone but highly effective combined. Number of induction heads Figure 6a demonstrates the stability of INTRYGUEmeanINTRYGUE_mean across different values of the number of induction heads k. Despite high run-to-run variance, the average AUROC remains stable. Tables 1-2 confirm this variance being inherent evaluation noise shared by all baselines. Thus, INTRYGUE can be employed without hyperparameter tuning. Efficiency To highlight the efficiency of our method, we compare its computational complexity with that of the Entropy and Semantic Entropy baselines in Figure 8. INTRYGUE achieves a runtime comparable to LN-Entropy, while remaining substantially faster than the sampling-based Semantic Entropy. Similar runtimes are observed for other information-based baselines, including RAUQ (2.74±0.032.74± 0.03s) and Perplexity (2.31±0.012.31± 0.01s), with differences within a few tenths of a second. 6 Conclusion In this work, we investigated the mechanistic limitations of entropy-based uncertainty estimators in RAG scenarios. Previous evidence suggested that for highly controlled settings induction heads can activate entropy neurons; our analysis verifies that this exact mechanism occurs in a realistic RAG. While induction heads are crucial for copying relevant information from context and supporting grounded generation, their concurrent triggering of entropy neurons inflates predictive entropy, suppressing model confidence. Thus, a standard entropy-based score is unreliable with common cases of inappropriately high uncertainty to context-grounded answers. To resolve this issue, our INTRYGUE, a training-free uncertainty estimation method, combines predictive entropy with a mechanistic score of induction head activity. By gating entropy with an estimate of induction strength, we compensate for induction-driven entropy inflation to yield a more reliable hallucination score. Experiments across four benchmarks spanning question answering and summarization, evaluated on six open-source LLMs, demonstrate that our method consistently performs on par with or better than a broad set of baselines. Ultimately, we demonstrate that RAG uncertainty quantification must account for output distributions and the internal mechanisms of evidence utilization. The findings motivate a promising future direction: integration of mechanistic interpretability with uncertainty estimation for building more accurate and explainable methods. Limitations While INTRYGUE provides a mechanistically grounded approach to uncertainty estimation in RAG, we identify the following limitations: • Requirement of White-Box Access: INTRYGUE relies on extracting internal attention matrices during the forward pass to compute the SinkRate and monitor induction heads. Consequently, it cannot be applied to proprietary, black-box large language models restricted behind APIs like GPT-4 or Claude. • Transformer Architecture: Because our findings are fundamentally rooted in standard Transformer attention mechanics, the method’s applicability to alternative architectures (such as state-space models) remains an open question. • Task-Dependent Aggregation: As demonstrated in our results from Section 5.2, the optimal sequence aggregation strategy depends on the generation length. While INTRYGUEmin-maxINTRYGUE_min-max is highly effective for long-form generation (MS MARCO, CNN/DM), INTRYGUEmeanINTRYGUE_mean is required for short responses (CoQA). This prevents the score from being entirely plug-and-play across varying output lengths, requiring practitioners to select the appropriate aggregation heuristic for their specific RAG use case. • Faithfulness vs. Objective Factuality: Our mechanistic proxy evaluates how successfully the model attends to and copies from the retrieved context. Therefore, INTRYGUE strictly measures faithfulness to the provided documents, not objective truth. If the upstream retrieval pipeline fetches false or conflicting information and the LLM successfully grounds its answer in that text, INTRYGUE will report low uncertainty, which users must not confuse with factual correctness. References A. Alansari and H. Luqman (2025) Large language models hallucination: a comprehensive survey. arXiv preprint arXiv:2510.06265. Cited by: §1. T. Amorese, C. Greco, M. Cuciniello, R. Milo, O. Sheveleva, N. Glackin, et al. (2023) Automatic speech recognition (asr) with whisper: testing performances in different languages. In CEUR WORKSHOP PROCEEDINGS, Vol. 3574, p. 1–8. Cited by: §1. A. Azaria and T. Mitchell (2023) The internal state of an LLM knows when it’s lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali (Eds.), Singapore, p. 967–976. Cited by: 1st item. S. Bach, A. Binder, G. Montavon, F. Klauschen, K. Müller, and W. Samek (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one 10 (7), p. e0130140. Cited by: Appendix A. [5] C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye INSIDE: llms’ internal states retain the power of hallucination detection. In The Twelfth International Conference on Learning Representations, Cited by: 1st item, §5.1. J. Duan, H. Cheng, S. Wang, A. Zavalny, C. Wang, R. Xu, B. Kailkhura, and K. Xu (2024) Shifting attention to relevance: towards the predictive uncertainty quantification of free-form large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 5050–5063. Cited by: Appendix A, §5.1. N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah (2021) A mathematical framework for transformer circuits. Note: Transformer Circuits Thread External Links: Link Cited by: §1, §3.2.1. S. Es, J. James, L. E. Anke, and S. Schockaert (2024) Ragas: automated evaluation of retrieval augmented generation. In Proceedings of the 18th conference of the european chapter of the association for computational linguistics: system demonstrations, p. 150–158. Cited by: §1. E. Fadeeva, A. Rubashevskii, D. Piatrashyn, R. Vashurin, S. Dhuliawala, A. Shelmanov, T. Baldwin, P. Nakov, M. Sachan, and M. Panov (2025) Faithfulness-aware uncertainty quantification for fact-checking the output of retrieval augmented generation. arXiv preprint arXiv:2505.21072. Cited by: §1, §3.1. W. Fan, Y. Ding, L. Ning, S. Wang, H. Li, D. Yin, T. Chua, and Q. Li (2024) A survey on RAG meeting LLMs: towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, p. 6491–6501. Cited by: §1. J. Ferrando, G. Sarti, A. Bisazza, and M. R. Costa-Jussà (2024) A primer on the inner workings of transformer-based language models. arXiv preprint arXiv:2405.00208. Cited by: Appendix A. M. Fomicheva, S. Sun, L. Yankovskaya, F. Blain, F. Guzmán, M. Fishel, N. Aletras, V. Chaudhary, and L. Specia (2020) Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics 8, p. 539–555. Cited by: §1, §3.1, §5.1. R. Friel and A. Sanyal (2023) Chainpoll: a high efficacy method for llm hallucination detection. arXiv preprint arXiv:2310.18344. Cited by: §1. W. Gurnee, T. Horsley, Z. C. Guo, T. R. Kheirkhah, Q. Sun, W. Hathaway, N. Nanda, and D. Bertsimas (2024) Universal neurons in GPT2 language models. Transactions on Machine Learning Research. Note: External Links: ISSN 2835-8856, Link Cited by: §2.2. H. Hu, C. He, X. Xie, and Q. Zhang (2024) Lrp4rag: detecting hallucinations in retrieval-augmented generation via layer-wise relevance propagation. arXiv preprint arXiv:2408.15533. Cited by: Appendix A. Y. Jiang, X. Li, G. Zhu, H. Li, J. Deng, K. Han, C. Shen, Q. Shi, and R. Zhang (2023) 6G non-terrestrial networks enabled low-altitude economy: opportunities and challenges. arXiv preprint arXiv:2311.09047. Cited by: §5.1. M. Joshi, E. Choi, D. S. Weld, and L. Zettlemoyer (2017) Triviaqa: a large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1601–1611. Cited by: Appendix A. L. Kuhn, Y. Gal, and S. Farquhar (2023) Semantic uncertainty: linguistic invariances for uncertainty estimation in natural language generation. In The Eleventh International Conference on Learning Representations, Cited by: §1, §5.1. T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, et al. (2019) Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7, p. 453–466. Cited by: Appendix A. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. (2020) Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in neural information processing systems 33, p. 9459–9474. Cited by: §1. Z. Li, J. Xiong, F. Ye, C. Zheng, X. Wu, J. Lu, Z. Wan, X. Liang, C. Li, Z. Sun, et al. (2024) Uncertaintyrag: span-level uncertainty enhanced long-context modeling for retrieval-augmented generation. arXiv preprint arXiv:2410.02719. Cited by: §1. S. Lin, J. Hilton, and O. Evans (2022) Truthfulqa: measuring how models mimic human falsehoods. In Proceedings of the 60th annual meeting of the association for computational linguistics (volume 1: long papers), p. 3214–3252. Cited by: Appendix A. [23] A. Malinin and M. Gales Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representations, Cited by: §5.1. S. Narayan, S. B. Cohen, and M. Lapata (2018) Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 conference on empirical methods in natural language processing, p. 1797–1807. Cited by: 3rd item, §3.1, §5.1. A. Nikitin, J. Kossen, Y. Gal, and P. Marttinen (2024) Kernel language entropy: fine-grained uncertainty quantification for llms from semantic similarities. Advances in Neural Information Processing Systems 37, p. 8901–8929. Cited by: §1, §3.1. C. Niu, Y. Wu, J. Zhu, S. Xu, K. Shum, R. Zhong, J. Song, and T. Zhang (2024) Ragtruth: a hallucination corpus for developing trustworthy retrieval-augmented language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 10862–10878. Cited by: 1st item, §3.1, §5.1. C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, et al. (2022) In-context learning and induction heads. arXiv preprint arXiv:2209.11895. Cited by: Appendix A, §2.1. L. Perez-Beltrachini and M. Lapata (2025) Uncertainty quantification in retrieval augmented question answering. Transactions on Machine Learning Research. Cited by: Appendix A, §1. Y. Qiao, L. Pan, Y. Mi, L. Liu, Y. Shen, F. Sun, and Z. Chu (2026) Lowest span confidence: a zero-shot metric for efficient and black-box hallucination detection in llms. arXiv preprint arXiv:2601.19918. Cited by: Appendix A, §1, §3.1, §5.1. X. Qiu and R. Miikkulainen (2024) Semantic density: uncertainty quantification for large language models through confidence measurement in semantic space. Advances in neural information processing systems 37, p. 134507–134533. Cited by: §1, §5.1. S. Reddy, D. Chen, and C. D. Manning (2019) Coqa: a conversational question answering challenge. Transactions of the Association for Computational Linguistics 7, p. 249–266. Cited by: 2nd item, §3.2.1, §5.1. C. Sky, B. Van Durme, J. Eisner, and C. Kedzie (2024) Do androids know they’re only dreaming of electric sheep?. In Findings of the Association for Computational Linguistics ACL 2024, p. 4401–4420. Cited by: 1st item. H. Soudani, E. Kanoulas, and F. Hasibi (2025) Why uncertainty estimation methods fall short in rag: an axiomatic analysis. In Findings of the Association for Computational Linguistics: ACL 2025, p. 16596–16616. Cited by: Appendix A, §1. G. Sriramanan, S. Bharti, V. S. Sadasivan, S. Saha, P. Kattakinda, and S. Feizi (2024) LLM-check: investigating detection of hallucinations in large language models. Advances in Neural Information Processing Systems 37, p. 34188–34216. Cited by: §5.1. A. Stolfo, B. P. Wu, W. Gurnee, Y. Belinkov, X. Song, M. Sachan, and N. Nanda (2024) Confidence regulation neurons in language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §1, §2.2, §3.2.2. Z. Sun, X. Zang, K. Zheng, J. Xu, X. Zhang, W. Yu, Y. Song, and H. Li (2025) ReDeEP: detecting hallucination in retrieval-augmented generation via mechanistic interpretability. In The Thirteenth International Conference on Learning Representations, Cited by: Appendix A, §5.1. [37] L. Tan, K. Huang, J. Shi, and K. Wu InterpDetect: interpretable signals for detecting hallucinations in retrieval-augmented generation. In Mechanistic Interpretability Workshop at NeurIPS 2025, Cited by: Appendix A. G. Team (2025a) Gemma 3. External Links: Link Cited by: §5.1. Q. Team (2025b) Qwen3 technical report. External Links: 2505.09388, Link Cited by: §5.1. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. (2023) Llama 2: open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. Cited by: §5.1. A. Vazhentsev, L. Rvanova, G. Kuzmin, E. Fadeeva, I. Lazichny, A. Panchenko, M. Panov, T. Baldwin, M. Sachan, P. Nakov, et al. (2025) Uncertainty-aware attention heads: efficient unsupervised uncertainty quantification for llms. arXiv preprint arXiv:2505.20045. Cited by: Appendix A, §1, §5.1. S. Weerawardhena, P. Kassianik, B. Nelson, B. Saglam, A. Vellore, A. Priyanshu, S. Vijay, M. Aufiero, A. Goldblatt, F. Burch, et al. (2025) Llama-3.1-foundationai-securityllm-8b-instruct technical report. arXiv preprint arXiv:2508.01059. Cited by: §5.1. J. Wei, N. Karina, H. W. Chung, Y. J. Jiao, S. Papay, A. Glaese, J. Schulman, and W. Fedus (2024) Measuring short-form factuality in large language models. arXiv preprint arXiv:2411.04368. Cited by: Appendix A. Q. Xie, Q. Chen, A. Chen, C. Peng, Y. Hu, F. Lin, X. Peng, J. Huang, J. Zhang, V. Keloth, et al. (2024) Me-llama: foundation large language models for medical applications. Research square, p. rs–3. Cited by: §1. T. Zhang, L. Qiu, Q. Guo, C. Deng, Y. Zhang, Z. Zhang, C. Zhou, X. Wang, and L. Fu (2023) Enhancing uncertainty-based hallucination detection with stronger focus. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 915–932. Cited by: §5.1. Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen, et al. (2025) Siren’s song in the ai ocean: a survey on hallucination in large language models. Computational Linguistics 51 (4), p. 1373–1418. Cited by: §1. Appendix A Related works CoQA XSum Model Total Hal. Rate (%) Total Hal. Rate (%) gemma-3-4b-it 703 22.1 650 31.3 Qwen3-8B 698 14.6 750 24.3 Llama-3.1-8B-Instruct 691 22.1 650 28.9 Llama-2-7b-chat-hf 741 34.7 746 34.9 Llama-2-13b-chat-hf 661 23.9 730 32.3 Mistral-7B-Instruct-v0.1 1525 30.8 749 39.4 Table 3: Hallucination statistics for CoQA and XSum tasks (generated at t=1.0t=1.0 and annotated using GPT-4.1). UQ for Hallucination Detection in RAG scenario. While many UQ-based hallucination detection methods have been developed Duan et al. (2024); Vazhentsev et al. (2025); Qiao et al. (2026), they have mainly been evaluated within closed-book settings (Joshi et al., 2017; Lin et al., 2022; Kwiatkowski et al., 2019; Wei et al., 2024). Existing work on uncertainty quantification in RAG is very limited, with recent works highlighting that standard UQ-based scores are unfit for RAG scenarios because their design does not account for the specific properties of a RAG setup Soudani et al. (2025). While some methods have been proposed to address this issue, their computational overhead and external dependencies make them highly impractical Soudani et al. (2025); Perez-Beltrachini and Lapata (2025). Hallucination Detection Through Mechanistic Interpretability. Mechanistic interpretability (MI) Ferrando et al. (2024) seeks to explain how individual internal components of LLMs contribute to the final prediction. MI has previously been applied to hallucination detection in a few works: Hu et al. (2024) proposes comparing representations of external and parametric knowledge using layer-wise relevance propagation Bach et al. (2015), while Sun et al. (2025); Tan et al. suggest that hallucinations occur when feedforward layers overemphasize parametric knowledge in the residual stream, while Copying Heads Olsson et al. (2022) fail to effectively employ external knowledge from retrieved content. However, the existing approaches rather design novel mechanistic features for hallucination detection than address the downsides of classic methods by considering their mechanistic limitations. Appendix B Additional results B.1 Ablation: gating functions The additional results on applying non-linear mappings to the SinkRate are demonstrated in Figures 11 and 12. They confirm that additional transformations are uncecessary for our method. CoQA Metric Llama-2-13B Qwen3-8B Gemma-3-4B SinkRate (min) 0.64±0.030.64± 0.03 0.57±0.040.57± 0.04 0.63±0.040.63± 0.04 MaxEntropy 0.65±0.060.65± 0.06 0.73±0.050.73± 0.05 0.70±0.070.70± 0.07 INTRYGUEmin-maxINTRYGUE_min-max 0.72±0.050.72± 0.05 0.77±0.070.77± 0.07 0.73±0.050.73± 0.05 MS MARCO Metric Mistral-7B Llama-2-7B Llama-2-13B SinkRate (min) 0.73±0.030.73± 0.03 0.70±0.020.70± 0.02 0.72±0.040.72± 0.04 MaxEntropy 0.67±0.030.67± 0.03 0.65±0.060.65± 0.06 0.62±0.050.62± 0.05 INTRYGUEmin-maxINTRYGUE_min-max 0.77±0.030.77± 0.03 0.72±0.040.72± 0.04 0.73±0.040.73± 0.04 Table 4: Performance comparison of minimum Sink Rate, maximum Entropy, and INTRYGUE across selected models on the CoQA and MS MARCO datasets. INTRYGUE outperforms both individual baseline metrics, demonstrating that combining uncertainty estimates with mechanistic signals yields superior detection compared to relying on either signal alone. B.2 Sink tokens Figures 10 and 10 illustrate the sink rate patterns observed in Mistral-7B. The title of each plot lists the top three tokens with the highest sink rates, formatted as (token, position, sink rate). Notably, attention sinks not only to the initial token but also to other utility tokens (e.g., <0x0A> in these examples). Figure 9: Attention matrix example with the sink patterns. Model: Mistral-7B, dataset: CoQA, head: L18H2. Figure 10: Attention matrix example with the sink patterns. Model: Mistral-7B, dataset: CoQA, head: L18H3. Appendix C Datasets To evaluate the effectiveness of our approach across diverse RAG scenarios, we conduct experiments on four established benchmarks spanning both question answering (QA) and summarization. These datasets present varying levels of difficulty for context grounding and hallucination detection: • RAGTruth (MS MARCO & CNN/DM): We utilize the long-form QA (MS MARCO) and standard summarization (CNN/DM) tasks from the RAGTruth benchmark Niu et al. (2024). These subsets provide high-quality annotated data specifically designed for evaluating hallucinations in retrieval-augmented generation. • CoQA: To test uncertainty quantification in interactive, context-dependent settings, we evaluate on the Conversational Question Answering (CoQA) dataset Reddy et al. (2019), which requires models to ground responses over multi-turn dialogues. To construct our evaluation instances for this task, we generated model responses at a temperature of t=1.0t=1.0 and subsequently annotated them for hallucinations using GPT-4.1. • XSum: Finally, we include the Extreme Summarization dataset (XSum) Narayan et al. (2018). Because XSum requires highly abstractive, single-sentence summaries, models are notoriously prone to hallucination here, providing a rigorous stress-test for our uncertainty estimator. As with CoQA, the summaries used for evaluation were generated at t=1.0t=1.0 and automatically annotated using GPT-4.1. The statistics of the CoQA and XSum datasets are provided in Table 3. Appendix D Implementation details In this section, we describe the key implementation choices: • For EigenScore, we used the representation of the last token to embed sentences, as suggested by Chen et al. . We extracted outputs from the 16th layer, as middle layers have been shown to contain the most factual information Sky et al. (2024); Azaria and Mitchell (2023). • All methods based on multiple generations employed 55 additional model generations, with the exception of Semantic Density, which employed 1010, as a lower number of samples failed to achieve better-than-random prediction quality. • The hyperparameter α for the RaUQ method was set to 0.20.2 for the QA task and 0.00.0 for the summarization task, following the original paper settings. • For a fair comparison with other baselines and our INTRYGUE score, the hyperparameters for the ReDeEP method were selected using response-wise labels (instead of token-wise labels, as proposed in the original paper). All experiments were carried out across 5 random splits, corresponding to seeds [42,…,46][42,…,46]. The train/val/test split fractions were set to 0.4/0.4/0.2, respectively. We used NVidia L40 and NVidia H100 GPUs. Appendix E Use of scientific artifacts The CoQA dataset contains passages from seven domains operating under various licenses: literature and Wikipedia (C BY-SA 4.0), MCTest children’s stories (MSR-LA), RACE exam passages (custom license), and DeepMind CNN news passages (Apache). All dataset artifacts were utilized strictly following their intended licensing terms. The considered datasets contain no personally identifiable information or offensive content. Finally, the text of this paper was proofread and refined for clarity using Gemini-3.1-Pro. Appendix F Potential risks Faithfulness vs. Factual Truth. INTRYGUE measures how well an output is grounded in the retrieved text, not its objective truth. If a RAG pipeline retrieves false, biased, or malicious context, the model may faithfully reproduce it. INTRYGUE will accurately report low uncertainty in these cases, which end-users could dangerously misinterpret as a guarantee of factual correctness. Adversarial Vulnerabilities. Tying uncertainty scores to a specific mechanistic proxy introduces a theoretical attack vector. Adversaries could craft prompt injections specifically designed to artificially spike induction head activity, potentially tricking the gating mechanism into heavily suppressing the uncertainty score for manipulated outputs. (a) Qwen3-8B | CoQA (b) Qwen3-8B | XSum (c) Llama-3.1-8B-Instruct | CoQA (d) Llama-3.1-8B-Instruct | XSum Figure 11: Mean entropy aggregation. Ablation of non-linear mappings that project the SinkRate to the [0,1][0,1] interval before multiplying it with entropy (Eq. 7). Trivial denotes that the SinkRate is set to 1, while Identity indicates that no mapping is applied. The heatmap shows AUC scores, where the center corresponds to AUC=1. Mappings are applied after Sink aggregation, and results are reported for both Mean and Min aggregations. (a) Qwen3-8B | CoQA (b) Qwen3-8B | XSum (c) Llama-3.1-8B-Instruct | CoQA (d) Llama-3.1-8B-Instruct | XSum Figure 12: Max entropy aggregation. Ablation of non-linear mappings that project the SinkRate to the [0,1][0,1] interval before multiplying it with entropy (Eq. 7). Trivial denotes that the SinkRate is set to 1, while Identity indicates that no mapping is applied. The heatmap shows AUC scores, where the center corresponds to AUC=1. Mappings are applied after Sink aggregation, and results are reported for both Mean and Min aggregations.

Instant research discovery