Paper deep dive

Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits

Amirhosein Ghasemabadi, Di Niu

Year: 2024Venue: arXiv preprintArea: Mechanistic Interp.Type: EmpiricalEmbeddings: 65

Models: Qwen3-1.7B, Qwen3-4B, Qwen3-8B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 3/11/2026, 1:11:37 AM

Summary

Gnosis is a lightweight, self-awareness mechanism that enables frozen Large Language Models (LLMs) to perform intrinsic self-verification by decoding signals from hidden states and attention patterns. By compressing internal traces into fixed-budget descriptors, Gnosis predicts correctness with negligible inference cost and outperforms external judges and reward models across various benchmarks.

Entities (6)

Gnosis · mechanism · 100%LLM · model · 100%Amirhosein Ghasemabadi · researcher · 99%Di Niu · researcher · 99%Attention Circuit Encoder · component · 95%Hidden Circuit Encoder · component · 95%

Relation Signals (4)

Amirhosein Ghasemabadi → authored → Gnosis

confidence 95% · We introduce Gnosis... Amirhosein Ghasemabadi

Gnosis → enables → Intrinsic Self-Verification

confidence 95% · Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to perform intrinsic self-verification

Gnosis → uses → Hidden Circuit Encoder

confidence 95% · Gnosis taps hidden states... Hidden Circuit Encoder

Gnosis → uses → Attention Circuit Encoder

confidence 95% · Gnosis taps... attention maps... Attention Circuit Encoder

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models (LLMs) generate fluent and complex outputs but often fail to recognize their own mistakes and hallucinations. Existing approaches typically rely on external judges, multi-sample consistency, or text-based self-critique, which incur additional compute or correlate weakly with true correctness. We ask: can LLMs predict their own failures by inspecting internal states during inference? We introduce Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to perform intrinsic self-verification by decoding signals from hidden states and attention patterns. Gnosis passively observes internal traces, compresses them into fixed-budget descriptors, and predicts correctness with negligible inference cost, adding only ~5M parameters and operating independently of sequence length. Across math reasoning, open-domain question answering, and academic knowledge benchmarks, and over frozen backbones ranging from 1.7B to 20B parameters, Gnosis consistently outperforms strong internal baselines and large external judges in both accuracy and calibration. Moreover, it generalizes zero-shot to partial generations, enabling early detection of failing trajectories and compute-aware control. These results show that reliable correctness cues are intrinsic to generation process and can be extracted efficiently without external supervision.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

64,316 characters extracted from source content.

Expand or collapse full text

Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits Amirhosein Ghasemabadi University of Alberta, Canada ghasemab@ualberta.ca Di Niu University of Alberta, Canada dniu@ualberta.ca Abstract Large language models (LLMs) generate flu- ent and complex outputs but often fail to rec- ognize their own mistakes and hallucinations. Existing approaches typically rely on external judges, multi-sample consistency, or text-based self-critique, which incur additional compute or correlate weakly with true correctness. We ask: can LLMs predict their own failures by inspecting internal states during inference? We introduce Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to per- form intrinsic self-verification by decoding sig- nals from hidden states and attention patterns. Gnosis passively observes internal traces, com- presses them into fixed-budget descriptors, and predicts correctness with negligible inference cost, adding only 5M parameters and operat- ing independently of sequence length. Across math reasoning, open-domain question answer- ing, and academic knowledge benchmarks, and over frozen backbones ranging from 1.7B to 20B parameters, Gnosis consistently outper- forms strong internal baselines and large ex- ternal judges in both accuracy and calibration. Moreover, it generalizes zero-shot to partial generations, enabling early detection of failing trajectories and compute-aware control. These results show that reliable correctness cues are intrinsic to generation process and can be ex- tracted efficiently without external supervision. Code and models:§ Gnosis Github. 1 Introduction Large language models (LLMs) have achieved re- markable performance in open-ended generation and multi-step reasoning, yet they remain unreli- able at assessing the correctness of their own out- puts(Kalai et al., 2025; Huang et al., 2025b). They frequently produce confident but incorrect answers, failing to detect reasoning errors or hallucinations even when such failures are evident to external eval- uators(Kirichenko et al., 2025; Kamoi et al., 2024). This gap between strong generation and weak self- verification limits the reliability, safety, and effi- ciency of LLM deployment, particularly in settings that require long-horizon reasoning or compute- aware control. A fundamental open question is whether LLMs can anticipate their own failures by examining the internal dynamics that govern their generation process. Prior work on LLM self-evaluation and halluci- nation detection largely follows three paradigms. Text-based self-critique and confidence estima- tion (Kadavath et al., 2022; Ulmer et al., 2024; Huang et al., 2025a) infer correctness from gen- erated text or token probabilities, often tracking linguistic fluency rather than reasoning validity and degrading on long or compositional tasks. Multi- sample consistency methods (Sriramanan et al., 2024a; Pawitan and Holmes, 2025) estimate con- fidence from agreement across multiple genera- tions, improving robustness at the cost of infer- ence that scales linearly with the number of sam- ples. External judges and reward models (Sti- ennon et al., 2020; Ouyang et al., 2022; Zheng et al., 2024; Wang et al., 2024b; Liu et al., 2025) train large auxiliary models to evaluate responses, providing strong signals but requiring costly super- vision, additional decoding passes, and substantial inference overhead. Despite their differences, these approaches rely on signals external to the model’s own internal dynamics, leaving open whether cor- rectness can be predicted directly from the genera- tion process itself. In this paper, we demonstrate that large language models can reliably predict their own failures by leveraging signals intrinsic to the generation pro- cess. We introduce Gnosis, a lightweight self- awareness mechanism that endows frozen LLMs with intrinsic self-verification, eliminating the need for external judges. By extracting reliability cues directly from model-internal dynamics during infer- ence, Gnosis produces accurate and well-calibrated arXiv:2512.20578v2 [cs.CL] 4 Jan 2026 Correctness Score N× Prompt LLM Head❄ LLM Response N S×D Hidden Circuit Encoder S×S + LLM❄ Attention Circuit Encoder Self-Attention Feed Forward Gnosis Self-Awareness Mechanism Hidden State Attn Maps H Judgment Head    Figure 1: Overview of our Gnosis self-awareness mechanism and its performance. Left: Gnosis taps hidden states and attention maps from a frozen LLM, learns to compress them into hidden/attention descriptors, and predicts a scalar correctness (hallucination) score with only∼5 million extra parameters and essentially zero added inference cost. Right: Gnosis outperforms 8B Skywork reward models and a Gemini 2.5 Pro judge in AUROC on Math-Reasoning (AMC12 + AIME24/25 + HMMT Feb 2025), Open-Domain QA (TriviaQA), and Academic Knowledge (MMLU-Pro); scores are averaged over the frozen backbones listed in Table 1. correctness estimates with negligible computa- tional overhead. This intrinsic capability enables early detection of failing reasoning trajectories, ef- ficient scaling across model sizes and domains, and practical deployment of compute-aware and reliability-critical language systems. Our main con- tributions are: •Intrinsic, Trajectory-Level Self-Awareness. We introduce Gnosis, a lightweight mecha- nism that enables frozen LLMs to predict the correctness of their own generations by decod- ing signals intrinsic to the inference process. Unlike prior internal-signal methods that rely on statistical features (Geng et al., 2023; Wang et al., 2025; Zhang et al., 2025b) or single- token indicators(Zhang et al., 2025a), Gnosis leverages the full spatiotemporal structure of internal dynamics across an entire generation trajectory. • Dual-Stream Introspection from Hidden States and Attention. Gnosis jointly models hidden-state evolution and attention-routing patterns through a compact, fixed-budget ar- chitecture that operates independently of se- quence length, extracting rich reliability cues with negligible inference overhead. •Cross-Scale Transfer and Early Failure Detection. Gnosis generalizes beyond self- judgment: a head trained on a small backbone model transfers zero-shot to larger variants, and predicts failures reliably from partial rea- soning and generations, enabling early termi- nation and compute-aware control. •State-of-the-Art Performance at Minimal Scale. With only∼5M added parameters, Gnosis is orders of magnitude smaller than external verifiers yet outperforms billion- parameter reward models and proprietary judges on math reasoning, open-domain QA, and academic benchmarks. It works reliably across diverse frozen backbones with negligi- ble latency overhead. 2 Related Work Methods for assessing LLM correctness and hallu- cination risk largely fall into four families: Text- based confidence & self-critique, Internal signal- based indicators and linear probes, external reward models and judge LLMs, and multi-sample self- consistency methods. External Reward Models and Judge LLMs. External verifiers train separate models to score response quality, factuality, or step-wise correct- ness. Outcome and Process Reward Models (OR- M/PRM) are widely used for ranking, hallucination detection, and guiding test-time search (Stiennon et al., 2020; Ouyang et al., 2022; Zheng et al., 2024; Wang et al., 2023; Zhang et al., 2025c). Recent sys- tems emphasize large, carefully curated datasets over architectural novelty: HelpSteer2 combines Likert ratings, pairwise preferences, and extrapola- tion to sharpen discrimination (Wang et al., 2024b), while Skywork-Reward-V2 scales human–AI cura- tion to tens of millions of preference pairs and leads on RewardBench-style suites (Liu et al., 2025). These models provide strong signals but incur sub- stantial annotation cost and add inference latency and deployment overhead by requiring a large aux- iliary model at serving time. Text-Based Confidence & Self-Critique. Text- based approaches aim to estimate correctness from the generated text and token probabilities. Training- free indicators use entropy or max probability as uncertainty proxies but struggle with confident hallucinations and out-of-distribution shifts (Geng et al., 2023; Sriramanan et al., 2024b; Pawitan and Holmes, 2025). Prompt-based calibration elicits verbalized confidence or self-critique, improving ECE but often tracking stylistic fluency more than reasoning validity and requiring extra passes (Ka- davath et al., 2022; Ulmer et al., 2024). Generative and distillation-based calibrators predict correct- ness in a single forward pass, e.g., APRICOT trains a calibrator LLM (Ulmer et al., 2024), and Self- Calibration distills self-consistency signals to en- able early stopping and confidence-weighted sam- pling (Huang et al., 2025a). These methods reduce dependence on external judges but may require full- model fine-tuning, add training cost, and remain brittle across domains and sequence lengths. Internal signal-based indicators and linear probes. Glass-box signals exploit logits, hidden states, and attention routing. Prior work shows hidden activations diverge between correct and hal- lucinated outputs (Duan et al., 2024), with factu- ality cues concentrated in middle/deep layers yet sensitive to domain shift (Zhang et al., 2025b). Token-wise hidden-state entropy and information density can outperform perplexity-based failure prediction (Chen et al., 2024). Trajectory/spectral views analyze how representations evolve across layers (e.g., Chain-of-Embedding, stability of la- tent paths) and relate angular/magnitude changes to correctness (Wang et al., 2025). Attention statistics provide lightweight reliability cues (Huang et al., 2024). A complementary line trains simple probes (shallow MLPs) on final-token states (Azaria and Mitchell, 2023; Burns et al., 2022; Zhang et al., 2025a). However, these approaches consistently yield low accuracy across diverse benchmarks. By relying on fragile heuristics or single-token snap- shots, they miss the generation’s full spatiotempo- ral structure, resulting in performance that falls far short of Gnosis. Multi-Sample Self-Consistency and Test- Time Scaling. Multi-sample self-consistency in- fers confidence from agreement across sampled rationales, boosting robustness but incurring in- ference cost that scales with the number of sam- ples and often saturating on long, compositional tasks (Sriramanan et al., 2024b). Recent cost-aware test-time scaling uses internal signals to prune search or adapt compute, reducing dependence on large external verifiers while retaining some bene- fits of multi-sample reasoning (Huang et al., 2025a; Ghasemabadi et al., 2025). 3 The Gnosis Mechanism We introduce Gnosis, a lightweight self-awareness mechanism designed to retrofit frozen LLMs with introspection capabilities. Gnosis operates on the intuition that a model’s internal traces, its evolving hidden states and attention routing patterns, carry distinctive “fingerprints” of hallucination and rea- soning errors. Unlike external judges that require separate, expensive decoding passes, Gnosis is a passive observer: it compresses the backbone’s in- ternal signals into compact descriptors and fuses them to predict a scalar correctness score. The ar- chitecture is explicitly designed so that its inference cost is independent of the sequence length, adding negligible overhead even for very long contexts. 3.1 Problem Setup and Length-Invariant Inputs Letxdenote an input prompt of lengthS x andˆy the generated response of lengthS y . The input to Gnosis is the concatenated sequence with a total size ofS = S x + S y tokens." The backbone has hidden dimensionD,Ldecoder layers, andHat- tention heads per layer. During generation, we read only the final-layer hidden statesH last ∈ R S×D and the attention mapsA = A ℓ,h ℓ=1..L,h=1..H , where eachA ℓ,h ∈ R S×S is the attention map of head h in layer ℓ. Gnosis learns a verification function: ˆp = f φ H last ,A ∈ [0, 1],(1) whereˆpis the estimated probability that the gen- erated answer is correct andφare the parameters of Gnosis. The backbone LLM remains frozen throughout. Fixed-Budget Compression. To decouple com- putational cost from sequence lengthS, we use a projection operatorΠthat maps variable-length traces into fixed-size tensors: •Hidden States. The sequenceH last ∈ R S×D is interpolated and adaptively pooled along the sequence dimension to a fixed budgetK hid Table 1: Correctness/hallucination detection across domains. For each model, columns (left to right) are: AUROC / AUPR-c / AUPR-e / BSS / ECE. MethodQwen3 1.7B-HybridQwen3 4B-ThinkingQwen3 4B-InstructOpenAI gpt-oss-20B (AUROC↑ / AUPR-c↑ / AUPR-e↑ / BSS↑ / ECE↓) Domain I: Math-Reasoning(AMC12 + AIME24/25 + HMMTFeb2025) Logit Entropy (2024a).79 .73 .82 .25 .05.80 .80 .77 .23 .12.83 .79 .82.32.74.80 .79 .81 .32.07 Mean Token Prob (2024a).78 .71 .82 .23 .06.80 .79 .76 .21 .13.82 .79 .82.31 .08.79 .78 .80 .30 .07 Attn Eigenvalue Score (2024a).61 .52 .63 -.13 .17.55 .60 .46 -.28 .23.72 .66 .75 .11 .13.72 .66 .75 .11 .13 CoE–R (2025) .59 .51 .63 -.17 .20.56 .60 .55 -.26 .24.66 .65 .59 -.02 .15.66 .66 .60 -.02 .15 CoE–C (2025).60 .53 .64 -.14 .18.53 .57 .52 -.32 .26.66 .66 .59 -.02 .14.66 .66 .59 -.02 .14 SkyworkRM-Llama3.1-8B (2025).88 .88 .88 .38 .10.87 .93 .80.24 .18.83 .90.72 .06 .22.81 .80 .83 .29 .10 SkyworkRM-Qwen3-8B (2025).90 .92.89.39 .14.89 .94 .77 .10 .22.83 .89 .75 -.49 .40.81 .79 .84-.12 .30 Gemini 2.5 Pro (2025) .91.88 .86 .50.11.92.97.68 .46.15.84.88 .66 .31 .18.92 .98 .64 -1.09 .10 Gnosis(Ours).95.95.94.59.09.96.98.91.65.05.93.96.89.51.08.85.86.86.38.04 Domain I: Open-Domain QA(TriviaQA) Logit Entropy.64 .53 .73 -.16 .19.68 .70 .63 .02 .13.71 .75.64 .74 .11.79 .85 .63 .05.21 Mean Token Prob.63 .52 .72 -.17 .19.67 .70 .63 .00 .14.71 .75.64 .71.11.79 .86 .62 .05.20 Attn Eigenvalue Score.52 .40 .62 -.39 .27.57 .61 .50 -.20 .19.59 .65 .51 -.16 .19.65 .81 .40 -.30 .21 CoE–R .59 .44 .70 -.25 .22.53 .57 .49 -.29 .24.57 .62 .48 -.23 .19.78 .86 .61 -.03 .21 CoE–C.58 .42 .70 -.28 .22.59 .61 .54 -.17 .20.52 .57 .44 -.32 .22.78 .87 .61 -.03 .21 SkyworkRM-Llama3.1-8B.83 .74.87 .00 .25.75 .74 .74 -.13 .28.69 .73 .60 -.47 .37.79 .88.59 -1.11 .52 SkyworkRM-Qwen3-8B .84 .73 .89 .00 .23.73 .89 .43 -.05 .17.67 .71 .57 -.82 .47.82.90 .64-1.54 .59 Gemini 2.5 Pro .90 .79 .91.40 .11.84.80.86.33.14.75.75.67-.02 .23.74 .83 .54 -.01 .20 Gnosis(Ours).87.79.92.34.10.89.89.88.45.05.86.87.84.38.05.83.90.73.19.17 Domain I: Academic Knowledge-Reasoning(MMLU-Pro) Logit Entropy.73 .86 .49 -.11 .20.74.90.41 -.31 .25.70.82 .45 -.22 .21.61 .75 .39 -.36 .23 Mean Token Prob .73 .86 .49 -.11 .19.74.89 .42 -.30 .25.70.83.48-.17 .23.65 .78 .31 -.24 .20 Attn Eigenvalue Score.61 .78 .36 -.35 .22.51 .78 .24 -.76 .29.59 .80 .32 -.47 .23.52 .72 .31 -.50 .26 CoE–R.55 .72 .34 -.47 .25.59 .80 .32 -.60 .30.52 .72 .28 -.64 .29.61 .77 .38 -.33 .22 CoE–C.55 .72 .34 -.47 .25.60 .81 .32 -.58 .28.52 .73 .29 -.62 .29.60 .76 .38 -.34 .22 SkyworkRM-Llama3.1-8B.65 .79 .46 .01 .10.61 .82 .37 -.03 .10.61 .80 .38 -.11 .15.71 .82 .53 -.39 .33 SkyworkRM-Qwen3-8B .76.87.53 .01 .17.73 .88 .43 -.05 .17.66 .82 .45 -.57 .35.75.86 .57-1.02 .50 Gemini 2.5 Pro.76.83 .68 .12.16.70 .87 .49-.01.18.67 .83.37 -.28 .24.78 .84.70 .22 .15 Gnosis(Ours).80.90.56.15.11.82.93.55.21.05.74.87.51.10.05.75.84.51.07.06 Table 2: Comparison with an MLP-Prob baseline on Qwen3 1.7B. MethodMathTriviaQAMMLU-Pro (AUROC↑, AUPR-c↑, ECE↓) MLP-Prob (2025a).86 .85 .19.71 .58 .21.69 .79 .23 Gnosis.95.95.09.87.79.10.80.90.11 (e.g.,192), yielding ̃ H = Π hid (H last )∈ R K hid ×D .(2) • Attention Maps. Each attention mapA ℓ,h ∈ R S×S is downsampled via adaptive pooling to a fixed grid sizek× k(e.g.,k = 256), giving a standardized set ̃ A = ̃ A ℓ,h ℓ,h , ̃ A ℓ,h ∈ R k×k .(3) All downstream encoders operate only on ̃ H and ̃ A with fixed dimensions(K hid ,D)and (L,H,k,k), so the computational cost of Gnosis does not grow withSand is negligible compared to the backbone; see Appendix A for architectural details. 3.2 Hidden-State Circuit Encoder Standard confidence methods often rely on to- ken probabilities (logits), which are poorly cal- ibrated and only weakly aligned with correct- ness (Ghasemabadi et al., 2025). Gnosis instead learns from the backbone’s internal representation, extracting correctness cues directly from the final- layer latent representations. A small encoderρ hid maps this latent trace into a compact descriptor: z hid = ρ hid ( ̃ H)∈ R D HID .(4) Local Temporal Encoder. We treat ̃ H as a tem- poral signal and apply a lightweight multi-scale 1D depthwise convolution over the sequence dimen- sion to capture local dependencies and irregulari- ties in the hidden trajectory. Table 3: Sibling-model judgment across domains. Each triplet of columns shows AUROC / AUPR-c / ECE. Gnosis-SelfJudge: Gnosis head trained on each back- bone and judging its own generations. Gnosis-RM: a single Gnosis head trained on Qwen3 1.7B-Hybrid and used as a reward model for the other models. Judge / Model Qwen3 1.7B-Hybrid Qwen3 4B-Thinking Qwen3 8B-Hybrid (AUROC↑, AUPR-c↑, ECE↓) Domain I: Math(AMC12 + AIME24/25 + HMMTFeb2025) SkyworkRM-Qwen3-8B.90 .92 .14.89 .94 .22.86 .95 .23 Gnosis-SelfJudge .95 .95 .09 .96 .98 .05.97 .97.08 Gnosis-Qwen1.7B as RM.93.97.18.97 .99 .07 Domain I: Open-Domain QA(TriviaQA) SkyworkRM-Qwen3-8B.84 .73 .23.73 .89 .17.72 .78 .32 Gnosis-SelfJudge .87 .79 .10 .89 .89 .05.86 .90 .09 Gnosis-Qwen1.7B as RM .86.86.04.84.88.12 Domain I: Academic Knowledge-Reasoning(MMLU-Pro) SkyworkRM-Qwen3-8B.76 .87 .17.73 .88.17.73 .89.19 Gnosis-SelfJudge .80 .90 .11 .82 .93 .05.83 .94 .07 Gnosis-Qwen1.7B as RM.81.93 .16.83 .94 .15 Table 4: Correctness detection on Math-reasoning for Qwen3 1.7B-Hybrid backbone under two max re- sponse lengths (12k, 24k). We compare Gnosis with SkyworkRM-Qwen3-8B, highlighting Gnosis’s near- constant latency and large speedups as response length increases. MethodMax lenLatency(ms)↓ AUROC↑ SkyworkRM-Qwen3-8B12k930.90 SkyworkRM-Qwen3-8B 24k2465.88 Gnosis12k25 (×37).95 Gnosis24k25 (×99).94 Global Set Encoder.To summarize the sequence into a compressed representation, we then apply a Set Transformer–style encoder (Lee et al., 2019): Set Attention Blocks (SAB) followed by a Pooling- by-Multihead-Attention (PMA) block. This en- ables global interaction across all positions and ag- gregates the sequence into a small set of summary tokens, which are flattened and linearly projected to form the final hidden descriptorz hid . Figure 2 il- lustrates the detailed architecture design of Hidden Circuit Encoder. Appendix A.1 details the encoder architecture, while Appendix B.3 presents full de- sign ablations. 3.3 Attention Circuit Encoder The attention stream ̃ A reveals layer- and head- level routing patterns that can indicate brittle reasoning or unstable focus, complementing the hidden-state descriptor. Rather than feeding raw attention weights into a large network, Gnosis sum- marizes each downsampled attention map ̃ A ℓ,h ∈ R k×k into a compact feature vector: v ℓ,h = Φ( ̃ A ℓ,h )∈ R d grid .(5) HereΦdenotes our per-map feature extractor, which outputs ad grid -dimensional summary for each attention map. Per-map Feature Extraction. We implement Φusing two complementary approaches: (i) a lightweight CNN that treats each attention map as an image and learns local-to-global patterns, and (i) an interpretable statistics-based extractor that summarizes how attention is distributed, where it concentrates, and how local or long-range it is. Concretely, the statistics include simple measures of spread and texture (e.g., entropy- and frequency- based features), diagonal and near-diagonal mass to capture locality, and lightweight center-and-spread measures that describe the average location of at- tention and how widely it is dispersed. We ablate each variant and their hybrid in Ap- pendix B.2 (Table 6). While the two extractors are individually competitive, the hybrid is the most consistent across benchmarks; we therefore adopt [Φ cnn ; Φ stat ]in the final design. Full definitions of the statistics are provided in Appendix A.2. Cross-Head and Cross-Layer Encoding. We arrange the per-head summaries into anL × H layer–head grid: G∈ R L×H×d grid , G ℓ,h,: =v ℓ,h .(6) We add learned layer and head embeddings to pre- serve depth and head identity. We then treat the L× Hentries as grid tokens. A lightweight en- coderρ attn mixes information across layers and heads using a few axial convolutional layers. This design is substantially cheaper than full global self- attention over the grid. Finally, we apply Pooling- by-Multihead-Attention (PMA) to aggregate the grid into a single descriptor: z attn = ρ attn (G)∈ R D ATT .(7) Because this stage operates on fixed dimensions (L,H,d grid ), the Gnosis-side compute is indepen- dent of the original sequence lengthS. Figure 2 illustrates the detailed architecture design of Atten- tion Circuit Encoder. Detailed architecture choices and ablations are deferred to Appendix B.2. (B, S, D) Input: Hidden State (B, S, d hid ) LayerNorm + Linear Proj (B, d hid, k hid ) Adaptive Avg Pooling 1D Multi-Scale Dilated Convs Gated Mixing Learnable gate params SE Block (Squeeze-Excite) Refined features Set Attention Block(SAB) (B , k hid, d hid ) Queries Learnable Seeds(K) (B, K, d hid ) Pooling MultiHead Attention (PMA) (B, K*d hid ) Flatten Output MLP z hid ∈ ℝ D HID Position Emb (B, L, H, S, S) Input: Attention Maps (B*L*H, K, K) Reshape Compute attention Statistical Features Learnable Visual patterns (Convolutional N) (B, L, H, d grid ) Linear Proj to d grid (B, L, H, d grid ) Head/Layer Position Emb Refined features Axial Grid Processor (Row/Col Convs) (B, K, dgrid) Pooling MultiHead Attention (PMA) Queries Learnable Seeds(K) (B, K*dgrid) Flatten Output MLP z attn ∈ ℝ D ATTN Local Temporal Feature Mixing Final Feature Aggregation (B, L, H, K, K) Downsample Feature Extarction Final Feature Aggregation Hidden Circuit EncoderAttention Circuit Encoder Figure 2: Gnosis Encoder Architecture Details. Hidden Circuit (left): project and adaptively pool the hidden- state trace, apply multi-scale dilated temporal mixing, then use lightweight attention-based pooling (SAB→PMA) to produce a compact descriptorz hid . Attention Circuit (right): downsample each layer–head attention map to a fixedk×kgrid, extract per-map CNN+statistics features, mix across the layer×head grid with a lightweight axial processor, and pool (PMA) to obtainz attn . Appendix A gives a detailed description of the encoder design, and Appendix B includes the complete set of architecture and design ablations. 3.4 Gated Fusion and Correctness Prediction Gnosis fuses the hidden and attention descriptors into a single vector and maps it to a correctness probability. We concatenate the two descriptors z = [z hid ;z attn ],(8) and feed the result into a small gated MLP head. The final correctness estimate is ˆp = σ GatedMLP φ (z) ,(9) whereGatedMLP φ is a lightweight gated MLP andσis the sigmoid. This head lets Gnosis adap- tively weight hidden versus attention features on a per-example basis(e.g., leaning more on atten- tion for reasoning traces and more on hidden states for factual recall). The architecture is intentionally small: Gnosis adds only∼5M parameters, mak- ing it∼1000×smaller than 8B reward models and dramatically smaller than Gemini 2.5 pro as judge. 3.5 Training A key advantage of Gnosis is that it can be trained without costly-annotated data. For each backbone, we generate answers on the training sets and label correctness by comparing predictions to ground- truth answers. This yields a binary classification dataset: D =(H last i ,A i ,y i ) N i=1 , wherey i ∈ 0, 1indicates whether the verifier judged thei-th generation as correct. Gnosis is trained to minimize binary cross-entropy: L(φ) =−E (H last ,A,y)∼D y log ˆp+(1−y) log(1− ˆp) , withˆp = f φ (H last ,A). The backbone is frozen; gradients flow only through the Gnosis encoders and fusion head. 4 Experiments and Results We evaluate Gnosis in three practical regimes: (i) self-judgment, where each Gnosis head scores gen- erations from its own frozen backbone; (i) sibling- model judgment, where a small head serves as a lightweight reward model for larger family mem- bers; and (i) early correctness prediction, where Gnosis is queried on partial completions to support compute-aware control. 4.1 Experimental Setup Backbones. We apply Gnosis to five frozen LLMs: Qwen3 1.7B-Hybrid, Qwen3 4B-Thinking, Qwen3 4B-Instruct, Qwen3 8B-hybrid(Yang et al., 2025) and OpenAI gpt-oss-20B(Agarwal et al., 2025). The backbone weights and decoding settings are never updated. Training Data. We train one correctness head per backbone on a mixed math–trivia corpus to cover both multi-step reasoning and open-domain factual recall. For math, we use the English por- tion of DAPO-Math-17k (∼14k competition-style (a) Math-Reasoning(b) TriviaQA Figure 3: Early Correctness Prediction on Math-Reasoning. Gnosis (red) achieves higher accuracy and better calibration than both MLP-Prob (blue) and a reward model SKYWORKRM-QWEN3-8B (yellow). Notably, after seeing 40% of the completion, Gnosis already matches the full-solution performance of the other methods. Figure 4: 2D Embeddings of Features Learned by Gnosis on Math-Reasoning. We show dimensionality-reduced embeddings of hidden-state features (left), attention features (middle), and their merged features (right), with KDE contours and marginal densities for wrong (red) and correct (blue) answers. Hidden features exhibit the clearest separation, attention features show a weaker but still clear separation, and the merged space yields the sharpest overall discrimination between correct and wrong solutions. problems with numeric or symbolic answers(Yu et al., 2025)). For QA, we subsample 40k ques- tions from a 118k-item TriviaQA training set(Joshi et al., 2017) to retain broad coverage while keeping training compact. We generate two completions per math prompt to capture diverse reasoning tra- jectories and increase correct/incorrect label variety under the same question, and one completion per trivia prompt since answers are shorter and often less ambiguous. We extract final answer, label cor- rectness by comparing to the ground-truth, and dis- card outputs without valid answers. This yields a balanced, fully automated training set that requires no human annotation. Training Details and Cost. We train each head for two epochs over this mixed dataset using Adam with a learning rate of1× 10 −4 . Because the back- bone is frozen and all feature extractors operate at a fixed budget independent of sequence length, training is lightweight. For the largest backbone (gpt-oss-20B MoE), the full pipeline, data gener- ation and training finishes in roughly 12 hours on 2×A100 80 GB GPUs, corresponding to $25 in cloud cost. Smaller backbones train faster. Benchmarks. For each benchmark, we prompt each frozen backbone to generate a solution with a maximum budget of 12k tokens, and re- tain only question–answer pairs with a valid fi- nal answer for evaluation.We evaluate Gno- sis on three disjoint domains: Math-Reasoning (AMC12 2022/2023(AI-MO Team, 2024), AIME 2024/2025(Zhang and Math-AI, 2024, 2025), HMMT Feb 2025(Balunovi ́ c et al., 2025)), Open- Domain QA (18k held-out TriviaQA questions with no overlap with training), and Academic Knowledge Reasoning (MMLU-Pro(Wang et al., 2024a)). Together, these benchmarks stress multi- step reasoning, hallucination detection on short factoid answers, and out-of-distribution general- ization. We report detailed backbone-level out- come statistics (accuracy, hallucination, and non- response rates) in Appendix D. Additional bench- mark details are provided in Appendix C. Metrics. We treat correctness prediction as bi- nary classification and report AUROC and AUPR under two complementary labelings (AUPR-c: cor- rect as positive; AUPR-e: incorrect as positive), together with calibration metrics Brier Skill Score (BSS) and Expected Calibration Error (ECE). AUROC/AUPR measure discriminative ranking un- der class imbalance, whereas BSS/ECE assess the quality and calibration of predicted probabilities. See Appendix C for extended interpretations. Baselines.We compare against four base- line families. (1) Statistical internal scores are training-free indicators computed from the back- bone’s own outputs, reported in Table 1 as Logit Entropy, Mean Token Prob, and Attn Eigenvalue Score(Sriramanan et al., 2024a). (2) Trajecto- ry/spectral internal indicators summarize cross- layer hidden-state dynamics, reported as CoE–R and CoE–C(Wang et al., 2025). (3) External judges include two open-source reward models that are state-of-the-art on public reward-model bench- marks(Malik et al., 2025), SkyworkRM-Llama3.1- 8B and the family-aligned SkyworkRM-Qwen3- 8B(Liu et al., 2025), as well as Gemini 2.5 Pro used as a judge (the Gemini judging prompt is provided in the Appendix E); all are reported in Table 1. (4) A Learnable probe(Zhang et al., 2025a) that ob- serves only the final answer token’s hidden state is reported separately on Qwen3 1.7B in Table 2 to isolate the limitations of single-token probing. 4.2 Self-Judgment We evaluate Gnosis in the standard self-judgment setting: for each backbone, the model generates answers to the benchmark questions, and the ver- ification method predicts the correctness of these specific generations. As shown in Tables 1 and 2, Gnosis consistently outperforms training-free baselines and large external judges across all tested domains. Superiority Over Internal Baselines and Probes.Across Math Reasoning and Open- Domain QA, Gnosis effectively solves the miscal- ibration of standard confidence metrics. It consis- tently lifts AUROC from the mid-0.7s (typical of Logit Entropy) to 0.95–0.96 while roughly dou- bling the BSS, turning negative calibration scores into strongly positive ones. Crucially, Gnosis out- performs the learned MLP-Prob final-token probe by 7–18 AUROC points across benchmarks. This consistent gap confirms that correctness is a prop- erty of the full generation trajectory, specifically the distributed hidden-state dynamics and attention patterns, rather than a state localized to the final token. Efficiency vs. Scale: With only∼5M parame- ters and negligible overhead from its fixed-budget projection, Gnosis matches or exceeds state-of-the- art Skywork 8B Reward Models (∼1000×larger) and the proprietary Gemini 2.5 Pro. This is no- table because Gnosis adds no independent world knowledge. Rather than fact-checking with mas- sive parametric memory, it detects the signatures of hallucination and reasoning error in the backbone’s internal traces. On complex Math Reasoning, Gno- sis surpasses both large judges. It also outperforms Gemini on MMLU-Pro, a domain it was not explic- itly trained on, suggesting that it learns transferable error patterns instead of task-specific cues. Addi- tionally, Gnosis maintains near-constant∼25ms la- tency and achieves roughly37×and99×speedups over the 8B reward model when judging answers of length 12k and 24k tokens, respectively (Table 4). These results show that intrinsic self-verification can be both more scalable and far cheaper than external oversight. Figure 5 compares the predicted correctness score distributions of Gnosis and the Skywork 8B Reward Model. Gnosis shows sharp, bimodal peaks near 0 (incorrect) and 1 (correct), whereas Skywork produces broader, overlapping scores that often cluster around 0.5–0.6. This separation aligns with Gnosis’s stronger calibration (BSS) and its ten- dency to assign more decisive probabilities. 4.3Cross-Scale: Zero-Shot Reward Modeling We introduce “Sibling Modeling”, where we train Gnosis on a small, cheap backbone and deploy it to judge larger family members without fine-tuning. Table 3 highlights a striking outcome: a head trained on a 1.7B backbone transfers effectively to 4B and 8B siblings across all evaluated domains. On Math Reasoning, for instance, it achieves 0.93 AUROC, nearly matching the 0.96 achieved by a self-trained head. Notably, this transferred 1.7B head still consistently outperforms the Skywork 8B Reward Model across all tested backbones, prov- ing that our tiny zero-shot verifier is more reliable than a massive external judge. This broad trans- ferability implies that hallucination manifests as a structural invariant across model scales, offering a “free lunch” where a single small head serves as a supervisor for an entire model family. We observe that this transfer is most effective when models share a similar generation style; while Gnosis ro- bustly handles differences in size, it performs best when the models also align in their formatting (e.g., transferring between thinking models rather than thinking-to-Instruct). 4.4 Early Error Detection Because Gnosis processes internal traces into fixed- size descriptors, it can evaluate partial generations natively. Crucially, this capability is emergent: al- though Gnosis is trained exclusively on complete trajectories, it generalizes zero-shot to partial pre- fixes without any additional fine-tuning. Figure 3 illustrates that on both Math Reasoning and Trivi- aQA, Gnosis reaches near-peak accuracy and pos- itive BSS after observing only 40% of the gen- eration. In contrast, external reward models and single-token learnable probe typically require the full response to stabilize. This enables aggres- sive compute-aware control policies: generated chains-of-thought can be terminated immediately if the internal “hallucination alarm” triggers, prevent- ing wasted compute on failing paths, or the system can automatically escalate the query to a stronger model upon detecting that the current backbone is incapable of answering correctly. 5 Ablations and Analysis We highlight the key ablation insights that motivate Gnosis design, and defer comprehensive studies to Appendix A. Hidden vs. Attention Circuits. Gnosis fuses a hidden-state and an attention circuit. Appendix Ta- ble 5 shows that both streams help, but attention is most useful for long-form reasoning: on Trivi- aQA, hidden-only dominates and fusion adds little, indicating hidden states carry most short factual reliability; on Math Reasoning and MMLU-Pro, attention-only is strong (even slightly better than hidden-only on MMLU-Pro), and fusion yields the best overall performance, suggesting complemen- tary structural cues that emerge in longer reasoning. We visualize this behavior on Math-Reasoning in Figure 4; analogous feature-distribution plots for the other domains are provided in the appendix Figure 7. Taken together, these results support that hidden states provide a broad, robust signal across domains, while attention routing contributes more strongly on reasoning-heavy tasks and less on short factual QA; combining both is the most reliable overall. Attention Map Extractor. We ablate how to summarize each downsampled attention map with a lightweight CNN, predefined fixed statistics, and their hybrid. Appendix Table 6 shows that the three variants perform similarly on Math, while on TriviaQA the CNN-based variants are stronger than statistics alone, and on MMLU-Pro the CNN+Stats hybrid is the most consistent. Based on these, we adopt the CNN+Stats design in the final model. Additional Ablations. We further validate the design path behind Gnosis with targeted ablations across both streams. On the attention stream, Appendix B.2 examines how grid mixing, iden- tity embeddings, pooling strategy, layer selection, and map downsampling affect performance (Ap- pendix Tables 7 and 8). On the hidden stream, we isolate architectural value by including a simple pooled-MLP baseline that naively pools final-layer hidden states before an MLP (Row G in Table 9), alongside broader studies of the local–global en- coder design and sizing trade-offs (Appendix Ta- bles 9 and 10). 6 Discussion and Limitations Gnosis provides a highly efficient framework for self-evaluation and “sibling modeling,” where a small head trained on a compact model effectively judges larger models within the same family. This architecture further supports compute-aware con- trol by enabling early error detection on partial generation traces. However, a key limitation is that Gnosis is designed as a self-awareness mechanism rather than a general-purpose reward model; while it transfers robustly to siblings, it is not capable of acting as a universal zero-shot judge for every model, particularly those with unrelated architec- tures or differing generation styles (e.g., transfer- ring between Thinking and Instruct models). 7 Conclusion We introduced Gnosis, a lightweight mechanism that allows frozen LLMs to detect their own er- rors by interpreting internal hidden and attention traces rather than relying on external judges. De- spite adding only∼5M parameters, Gnosis consis- tently outperforms billion-parameter reward mod- els and Large proprietary models like Gemini 2.5 Pro, demonstrating that high-fidelity correctness signals are intrinsic to the generation process. This approach establishes a new standard for compute- efficient reliability, enabling self-verifying systems that can detect failing trajectories with negligible overhead. References Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Alt- man, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925. AI-MO Team. 2024. Aimo validation set - amc sub- set.https://huggingface.co/datasets/AI-MO/ aimo-validation-amc. Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when it’s lying. arXiv preprint arXiv:2304.13734. Mislav Balunovi ́ c, Jasper Dekoninck, Ivo Petrov, Nikola Jovanovi ́ c, and Martin Vechev. 2025. Matharena: Evaluating llms on uncontaminated math competi- tions. Collin Burns, Haotian Ye, Dan Klein, and Jacob Stein- hardt. 2022. Discovering latent knowledge in lan- guage models without supervision. arXiv preprint arXiv:2212.03827. Siyi Chen, Aoran Zhang, and He He. 2024. Let’s measure information step-by-step: Llm-based eval- uation metrics with hidden states. arXiv preprint arXiv:2508.05469. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Mar- cel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261. Hong Duan, Yuxin Yang, and Kam-Yiu Tam. 2024. Do llms know about hallucination? an empirical investigation of llm’s hidden states. arXiv preprint arXiv:2402.09733. Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, Preslav Nakov, and Iryna Gurevych. 2023. A sur- vey of confidence estimation and calibration in large language models. arXiv preprint arXiv:2311.08298. Amirhosein Ghasemabadi, Keith G Mills, Baochun Li, and Di Niu. 2025. Guided by gut: Efficient test-time scaling with reinforced intrinsic confidence. arXiv preprint arXiv:2505.20325. Chengsong Huang, Langlin Huang, Jixuan Leng, Ji- acheng Liu, and Jiaxin Huang. 2025a. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031. Hui Huang, Yingqi Qu, Jing Liu, Muyun Yang, Bing Xu, Tiejun Zhao, and Wenpeng Lu. 2024. Self-evaluation of large language model based on glass-box features. arXiv preprint arXiv:2403.04222. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 oth- ers. 2025b. A survey on hallucination in large lan- guage models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1–55. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. triviaqa: A Large Scale Distantly Supervised Challenge Dataset for Reading Compre- hension. arXiv e-prints, arXiv:1705.03551. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, and 1 others. 2022. Language mod- els (mostly) know what they know. arXiv preprint arXiv:2207.05221. Adam Tauman Kalai, Ofir Nachum, Santosh S Vem- pala, and Edwin Zhang. 2025. Why language models hallucinate. arXiv preprint arXiv:2509.04664. Ryo Kamoi, Sarkar Snigdha Sarathi Das, Renze Lou, Jihyun Janice Ahn, Yilun Zhao, Xiaoxin Lu, Nan Zhang, Yusen Zhang, Ranran Haoran Zhang, Su- jeeth Reddy Vummanthala, and 1 others. 2024. Eval- uating llms at detecting errors in llm responses. arXiv preprint arXiv:2404.03602. Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J Bell. 2025. Abstentionbench: Rea- soning llms fail on unanswerable questions. arXiv preprint arXiv:2506.09038. Juho Lee, Yoonho Lee, Jungtaek Kim, Adam Ko- siorek, Seungjin Choi, and Yee Whye Teh. 2019. Set transformer: A framework for attention-based permutation-invariant neural networks. In Interna- tional conference on machine learning, pages 3744– 3753. PMLR. Chris Yuhao Liu, Liang Zeng, Yuzhen Xiao, Jujie He, Ji- acai Liu, Chaojie Wang, Rui Yan, Wei Shen, Fuxiang Zhang, Jiacheng Xu, and 1 others. 2025. Skywork- reward-v2: Scaling preference data curation via human-ai synergy. arXiv preprint arXiv:2507.01352. Saumya Malik, Valentina Pyatkin, Sander Land, Ja- cob Morrison, Noah A Smith, Hannaneh Hajishirzi, and Nathan Lambert. 2025. Rewardbench 2: Ad- vancing reward model evaluation. arXiv preprint arXiv:2506.01937. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. Training language models to follow in- structions with human feedback. Advances in neural information processing systems, 35:27730–27744. Yudi Pawitan and Chris Holmes. 2025. Confidence in the reasoning of large language models. Harvard Data Science Review, 7(1). Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. 2024a. Llm-check: Investigating detection of hallucinations in large language mod- els. In Advances in Neural Information Processing Systems (NeurIPS). Gaurang Sriramanan, Siddhant Bharti, Vinu Sankar Sadasivan, Shoumik Saha, Priyatham Kattakinda, and Soheil Feizi. 2024b. Llm-check: Investigating detection of hallucinations in large language models. Advances in Neural Information Processing Systems, 37:34188–34216. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learn- ing to summarize with human feedback. Advances in neural information processing systems, 33:3008– 3021. Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, and Seong Joon Oh. 2024. Calibrating large language models using their generations only. arXiv preprint arXiv:2403.05973. Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2023. Math-shepherd: Verify and reinforce llms step- by-step without human annotations. arXiv preprint arXiv:2312.08935. Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, and Rui Wang. 2025. Latent space chain-of- embedding enables output-free llm self-evaluation. arXiv preprint arXiv:2410.13640. ICLR 2025. Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, and 1 others. 2024a. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Ad- vances in Neural Information Processing Systems, 37:95266–95290. Zhilin Wang, Alexander Bukharin, Olivier Delal- leau, Daniel Egert, Gerald Shen, Jiaqi Zeng, Olek- sii Kuchaiev, and Yi Dong. 2024b.Helpsteer2- preference: Complementing ratings with preferences. arXiv preprint arXiv:2410.01257. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025.Qwen3 technical report. arXiv preprint arXiv:2505.09388. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025. Dapo: An open-source llm reinforcement learning system at scale. Preprint, arXiv:2503.14476. Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Au- rojit Panda, Jinyang Li, and He He. 2025a. Rea- soning models know when they’re right: Probing hidden states for self-verification. arXiv preprint arXiv:2504.05419. Aoran Zhang, Yuhan Chen, Jiamin Pan, Chen Zhao, Ananya Panda, Jiawei Li, and He He. 2025b. Are the hidden states hiding something? testing the lim- its of llm factuality self-evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL). Yifan Zhang and Team Math-AI. 2024. American invi- tational mathematics examination (aime) 2024. Yifan Zhang and Team Math-AI. 2025. American invi- tational mathematics examination (aime) 2025. Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jin- gren Zhou, and Junyang Lin. 2025c. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301. Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jin- gren Zhou, and Junyang Lin. 2024. Processbench: Identifying process errors in mathematical reasoning. arXiv preprint arXiv:2412.06559. Math-ReasoningTriviaQAMMLUPro Gnosis SkyworkRM-Qwen3-8B Figure 5: Predicted Correctness Score Distributions. Gnosis (top) displays sharp, bimodal separation between correct (blue) and wrong (red) answers. In contrast, the larger Skywork model (bottom) exhibits diffuse distributions with significant overlap, reflecting higher uncertainty. Table 5: Impact of Dual-Stream Architecture across Benchmarks. Comparison of single-stream variants against the Full Gnosis model. While hidden states provide a strong signal, fusing them with the attention circuit consistently yields the best performance. Math ReasoningTriviaQAMMLU-Pro Model VariantInput Signals AUROC AUPR-c AUPR-eAUROC AUPR-c AUPR-eAUROC AUPR-c AUPR-e BOTH (GNOSIS)Fused ([z hid ; z attn ])0.950.950.940.870.790.920.800.900.56 ATTENTION-ONLY Attention Maps (z attn )0.920.930.900.780.660.860.800.900.56 HIDDEN-ONLYHidden States (z hid ) 0.920.920.910.870.770.920.780.890.53 A Architecture Overview Figure 2 illustrates the detailed internal components of the Gnosis Mechanism. The architecture consists of two parallel streams that process the frozen backbone’s internal traces to extract reliability signals efficiently: A.1 Hidden Circuit Encoder. This stream processes the sequence of hidden statesH last ∈ R S×D . To avoid the added cost and memory of storing intermediate states for every token, we use only the final-layer hidden state. We show this choice remains strongly predictive. To handle variable lengths while maintaining a fixed compute budget, the sequence is first projected and pooled into a fixed number of tokens. It then passes through a Local Temporal Feature Mixing stage (Phase 1) using multi-scale dilated convolutions and Squeeze-and- Excitation (SE) blocks to capture local dependencies and reweight informative channels. Finally, a Global Set Encoder (Phase 2) utilizes a Set Attention Block (SAB) followed by Pooling by Multihead Attention (PMA) to aggregate the sequence into a compact descriptor z hid ∈ R D HID . A.2 Attention Circuit Encoder. This stream processes the collection of attention mapsA ℓ,h ℓ=1..L,h=1..H from a frozen backbone. To make computation invariant to the original context length, we downsample each map to a fixed grid ̃ A ℓ,h ∈ R k×k . We then summarize each downsampled map into a compact per-head descriptor v ℓ,h = Φ( ̃ A ℓ,h )∈ R d grid . These per-head descriptors are arranged as a layer–head grid and processed by an Axial Grid Processor to model inter-layer and inter-head dependencies, followed by PMA to obtain the final attention descriptor z attn ∈ R D ATT . Per-Map Feature Extraction Variants. As described in Section 3.3, we implementΦusing two alternatives: (i) a lightweight CNN that treats each attention map as an image and learns local-to-global patterns, and (i) an interpretable statistics-based extractor that computes predefined structural descriptors. We compare these variants and their hybrid in Table 6. While CNN-only and Stats-only are individually competitive, the hybrid is the most consistent across domains. Unless otherwise stated, we therefore adopt CNN+Stats as the default Gnosis configuration. The Stats-only variant remains a strong, more interpretable alternative. Interpretable Attention Statistics.The statistics branch summarizes how attention is distributed, how local or long-range it is, and where it tends to concentrate on the map, using a small set of predefined descriptors. Concretely, for each downsampled map ̃ A ℓ,h we compute: •Entropy. We compute map entropy together with row and column entropies. Map entropy reflects the overall dispersion of attention mass (focused vs. diffuse). Row and column entropies provide axis-specific views of this dispersion, indicating whether spread is driven primarily by query positions (rows) or key positions (columns). Together, these metrics offer a direct, interpretable summary of attentional diffusion and stability. •Spectral Texture. We compute spectral entropy and the relative energy from the 2D Fourier spectrum of ̃ A ℓ,h . These features summarize whether attention exhibits coherent, structured patterns (low- frequency dominance) or becomes fragmented and noisy (elevated high-frequency energy and higher spectral entropy). •Locality via Diagonal Structure. We measure diagonal and near-diagonal mass through the diagonal ratio and diagonal-band energies. This provides a simple proxy for locality versus longer-range routing, which often correlates with coherent step-wise reasoning behavior. • Center and Spread on the Map. We compute lightweight center-and-width measures to describe the average location of attention mass and how widely it is dispersed across the grid. The two descriptors are concatenated and passed through a lightweight gated MLP to produce a scalar correctness logit, which is converted to a probability via a sigmoid. Because both encoders operate on fixed-size summaries, Gnosis runs at effectively constant cost in sequence length and can be queried on partial chains of thought. B Comprehensive Ablations and Analysis B.1 Hidden vs. Attention Circuits To quantify the distinct contributions of the internal representations, we trained single-stream variants of Gnosis and compared them to the full dual-stream model on Qwen3 1.7B. As reported in Table 5, the HIDDEN-ONLY model already provides a strong correctness signal across benchmarks. On Math Reasoning, both single-stream variants reach 0.92 AUROC, while fusingz hid andz attn improves performance to 0.95 AUROC. On TriviaQA, the hidden stream remains stronger than attention alone, and fusion achieves the best overall performance. On MMLU-Pro, attention-only is slightly stronger than hidden-only, while fusion matches the best result. These results confirm that attention contributes complementary structural cues that help maximize performance when combined with hidden representations. B.2 Attention Circuit Encoder Ablations To determine the optimal architecture for the attention circuit, we systematically investigated three key design components: the input feature representation for each attention map, the grid topology for mixing information across layers and heads, and the final aggregation strategy. Tables 6, 7, and 8 detail this investigation. We select the configuration highlighted in Row A of Tables 7 and 8 as it achieves the best balance of accuracy and parameter efficiency. Feature Input Representation. We first assessed how best to encode individual attention maps. Table 6 compares a lightweight learned CNN, predefined statistics, and their combination across benchmarks. The predefined statistics remain competitive with the CNN (e.g., identical 0.92 AUROC on Math Reasoning), Table 6: Impact of Per-Map Feature Extractor Choices across Benchmarks. Comparison of learnable CNN, predefined statistics, and their combination for the attention per-map extractorΦ. Across benchmarks, the CNN and statistics variants are competitive, while their hybrid is the most consistent overall; we adopt the CNN+Stats design in our final model. Math ReasoningTriviaQAMMLU-Pro Model VariantAUROC AUPR-c AUPR-eAUROC AUPR-c AUPR-eAUROC AUPR-c AUPR-e CNN+STATS (FINAL)0.920.930.900.780.650.860.800.900.56 CNN-ONLY0.920.930.880.790.660.860.790.900.56 STATS-ONLY 0.920.930.900.750.620.830.760.880.53 Table 7: Attention Circuit: Components & Topology. We ab- late the attention-circuit design by varying grid mixing, identity embeddings, and aggregation. Row A is our default configura- tion. Axial Conv performs lightweight row/column mixing over the (Layer, Head) grid, offering a cheaper alternative to full global self-attention. ID Configuration / Change#Params AUROC AGnosis (Axial + PMA)1.4M0.92 Grid Topology & Identity BRemove Axial Conv (Linear Proj Only)1.1M0.84 CReplace Axial Conv w/ Global Transformer4.5M0.92 DRemove Layer/Head Embeddings1.4M0.90 Aggregation Strategy HReplace PMA w/ Mean Pool (Axial→ Mean)0.9M0.85 Table 8: Attention Circuit: Hyperparam- eters. Impact of layer selection stride and initial map downsampling size (k grid ). (Pro- posed: Select 1 every 5 layers;k grid = 256). Note: These choices mainly affect inference speed/memory rather than parameter count. Higher AUROC is better. ID VariationAUROC AGnosis0.92 Layer Selection Strategy E1 All Layers (Every Map)0.91 E2 First & Last Layers Only0.64 Downsampling Size (k grid ) F1 Small Grid (64)0.68 F2 Large Grid (512)0.92 while providing a more interpretable per-map representation. Combining CNN and statistics yields comparable performance overall and modest gains on MMLU-Pro (0.80 AUROC). Grid Topology and Layer/Head Identity. We next evaluated how to process the collection of extracted map features. As shown in Table 7, removing the grid mixing entirely (Row B) reduces performance, indicating that individual attention heads are not independent predictors; their interactions matter. Replacing our lightweight Axial Convolutions with a heavy Global Transformer (Row C) increases parameters four-fold without improving AUROC, validating the efficiency of the axial design. Furthermore, removing the learned layer and head embeddings (Row D) degrades performance, confirming that the model relies on knowing where a specific activation pattern occurred within the LLM’s depth and breadth. Aggregation Strategy. Finally, we analyzed how to summarize the grid into a fixed vector. Replacing our query-based Pooling by Multihead Attention (PMA) with simple mean pooling (Row H in Table 7) causes a sharp drop in accuracy. This suggests that Gnosis benefits from learning specific "reliability prototypes" (via PMA seeds) rather than uniformly averaging all attention circuits, likely because only a small subset of heads carry high-fidelity correctness signals. B.3 Hidden-State Circuit Encoder Ablations To identify the optimal architecture for the hidden-state circuit, we conducted a comprehensive ablation study investigating feature dimensionality, local temporal processing, and global aggregation strategies. Table 9 & Table 10 details this investigation. The final design (Row A) provides the best trade-off between reliability estimation (AUROC) and computational efficiency. Dimensionality and Sizing. We first investigated the information bottleneck size (d tok ,k hid ). Compar- ing Rows H and I against our proposed model (Row A) reveals a clear performance plateau at size192. Reducing the size to96causes a sharp performance drop (−0.04AUROC), likely due to information loss during the initial pooling. Conversely, scaling to384triples the parameter count without any accuracy Table 9: Hidden-state Circuit: Components & Baselines. We ab- late the hidden-state encoder by isolating the roles of local temporal mixing (Phase 1) and global set aggregation (Phase 2). Row A is our default configuration. Removing Phase 1 (B–D) or weakening global aggregation (E–F) consistently reduces AUROC, and a sim- ple pooled MLP baseline (G) performs substantially worse. ID Configuration / Change#Params AUROC∆ AGnosis2.6M0.92– Phase 1: Local Processing B Remove Phase 1 (Raw Seq→ Set Enc)2.4M0.89-0.03 C Remove Gating & SE (Sum only)2.6M0.91-0.01 D Remove Multi-scale (Dilations→ 1)2.4M0.90-0.02 Phase 2: Global Aggregation E Remove SAB (CNN→ PMA only)1.3M0.85-0.07 F Replace PMA w/ Mean Pool2.1M0.89-0.03 Architectural Baselines G Pooled MLP (GlobalPool→ MLP)0.7M0.82-0.1 Table 10: Hidden-state Circuit: Hyper- parameters.Impact of varying Feature Dimension (d tok ), Pooled Sequence Length (k hid ), and SAB Depth. (Gnosis used settings: d tok = 192,k hid = 192, SAB = 3) ID Variation#Params AUROC AGnosis2.6M0.92 Feature Dimension (d tok ) H1 Small Width (96)0.8M0.88 H2 Large Width (384)9.3M0.92 Pooled Seq Length (k hid ) I1 Short Seq (96)2.6M0.89 I2 Long Seq (384)2.6M0.92 SAB Layers (Default: 3) J1 Fewer Layers (1)1.7M0.89 J2 More Layers (5)3.5M0.93 gain. We therefore fix the dimensions to 192 for efficiency. Local Temporal Encoder (Phase 1). We investigated whether explicit local feature extraction is necessary before global processing. Row B shows that feeding raw pooled sequences directly to the set encoder reduces performance by0.04AUROC, confirming that Phase 1 acts as a critical denoising stage. Inside Phase 1, we found that architectural complexity matters: removing the Squeeze-and-Excite gating (Row C) or replacing the multi-scale dilated convolutions with a standard convolution (Row D) both degrade performance. This suggests the model relies on capturing multi-scale temporal signals (via dilation) and dynamic feature reweighting (via SE/Gating). Global Set Encoder (Phase 2). Finally, we analyzed the global aggregation stage. We found that simply averaging the features (Row F) or removing the global self-attention refinement (Row E) consistently hurts performance. This validates the use of the SAB+PMA stack to capture global context and learn specific reliability prototypes. Notably, our hybrid design significantly outperforms a simple pooled MLP baseline (Row G). C Additional Experimental Setup C.1 Benchmarks (extended). We evaluate on three disjoint domains. For math reasoning, we use a combined test set of AMC12 2022, AMC12 2023(AI-MO Team, 2024), AIME 2024(Zhang and Math-AI, 2024), AIME 2025(Zhang and Math-AI, 2025), and HMMT February 2025(Balunovi ́ c et al., 2025). These competition-style problems span a wide range of difficulty and require multi-step symbolic and numeric reasoning. For open-domain QA, we use an 18k-question held-out trivia subset drawn from the same distribution as our training corpus but with no overlapping items. This benchmark emphasizes short factoid answers and directly evaluates hallucination detection and knowledge grounding. For academic knowledge reasoning, we use MMLU-Pro(Wang et al., 2024a), an out-of-distribution evaluation spanning 14 diverse domains (e.g., math, physics, law, psychology) that combines domain knowledge with multi-step reasoning, providing a broad test of generalization beyond the training mix. C.2 Metrics (extended). We frame correctness prediction as binary classification and report both ranking and calibration quality. AUROC measures how well a method ranks correct completions above incorrect ones (0.5 = chance, 1.0 = perfect). AUPR is reported with two complementary positive classes: AUPR-c treats correct completions as positive and summarizes how well a method recovers correct answers with high precision across Table 11: Backbone outcome rates (%) across evaluation domains. For each frozen backbone, we re- port the fraction of instances that are correct (Accuracy), incorrect (Hallucination), or no-answer (“I don’t know”/refusal/empty). No-answer cases are shown for completeness but are filtered out from our correctness- prediction evaluation (Table 1). BackboneDomain Backbone Accuracy Backbone Hallucination Backbone “I don’t know” Qwen3 1.7B-HybridMath-Reasoning44.87%47.76%7.37% Qwen3 1.7B-HybridTriviaQA33.54%55.67%10.79% Qwen3 1.7B-HybridMMLU-Pro66.09%31.53%2.39% Qwen3 4B-Thinking-2507Math-Reasoning63.27%25.31%11.42% Qwen3 4B-Thinking-2507TriviaQA53.73%45.27%0.99% Qwen3 4B-Thinking-2507MMLU-Pro72.45%24.75%2.79% Qwen3 4B-Instruct-2507Math-Reasoning57.51%29.26%13.22% Qwen3 4B-Instruct-2507TriviaQA56.89%42.98%0.13% Qwen3 4B-Instruct-2507MMLU-Pro71.69%27.46%0.85% Qwen3 8B-HybridMath-Reasoning49.42%14.16%36.42% Qwen3 8B-HybridTriviaQA62.22%36.05%1.74% Qwen3 8B-HybridMMLU-Pro71.93%22.29%5.77% OpenAI gpt-oss-20BMath-Reasoning52.53%42.63%4.84% OpenAI gpt-oss-20BTriviaQA63.70%27.67%8.62% OpenAI gpt-oss-20BMMLU-Pro68.74%30.83%0.44% recall; AUPR-e treats incorrect completions as positive and summarizes how well a method detects errors/hallucinations, which is often the more safety-relevant viewpoint under class imbalance. For probability quality, we report Brier Skill Score (BSS), where BSS> 0indicates improvement over a prevalence baseline, and Expected Calibration Error (ECE), where lower values indicate better alignment between predicted correctness probabilities and empirical accuracy. D Backbone Outcome Statistics To contextualize the results in Table 1, we report the raw outcome breakdown of each frozen backbone on the three evaluation domains. While Table 1 focuses on the quality of correctness prediction (AU- ROC/AUPR/BSS/ECE) for different judges and internal methods, this table provides the underlying base behavior of each backbone: how often it is correct, how often it hallucinates, and how often it produces no valid final answer. Importantly, non-response cases are reported here for completeness but are filtered out and not used in our correctness-prediction evaluation. Thus, the metrics in Table 1 are computed over the subset of generations with a valid, parsable answer. This table therefore serves two purposes: (i) it clarifies the intrinsic difficulty and failure modes of each backbone across domains, and (i) it provides the context needed to interpret domain-dependent shifts in AUPR and calibration metrics reported in Table 1, which are more sensitive to the underlying prevalence of correct vs. incorrect answered generations. E Gemini As judge. For the LLM-as-judge baseline, we use Gemini 2.5 pro as a generative reward model. Given a question and a model-generated answer, the judge produces a brief correctness analysis and then emits a scalar score in[0, 1]at the end of the response. We parse the score from the required<score>tags. Evaluating Gemini via the API on the three benchmarks reported in Table 1 cost approximately $500 in total. Figure 6 shows the prompt for using Gemini as the judge. Figure 6: Gemini judge system prompt used for the LLM-as-judge baseline. SYSTEM_PROMPT You are an objective, expert evaluator. Your task is to review a Question and a proposed Answer. Determine if the Answer is correct, and accurate based on the Question. Assign a correctness score between 0.0 (completely wrong) and 1.0 (perfectly correct). IMPORTANT: Output the final score inside <score> tags. Example: <score>0.95</score> or <score>0.0</score>. Figure 7: 2D PCA scatter plots of features learned by Gnosis across three domains. Rows: Math-Reasoning, TriviaQA, and MMLU-Pro. Columns: hidden-state features (z hid ), attention features (z attn ), and their merged representation. We show PCA scatter plots with KDE contours and marginal densities for wrong (red) and correct (blue) answers. Across domains, the merged features provide the clearest overall class separation, illustrating the complementarity of hidden and attention signals.

Instant research discovery