Paper deep dive
Where Facts Go Missing: A Layerwise Taxonomy and Per-Layer Attribution of Information Omission in Air-Gapped LLMAgent Pipelines
Santhiya Rajan
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 8/1/2026, 12:05:11 PM
Summary
This paper introduces a nine-layer taxonomy (L0-L8) to attribute information omission in air-gapped LLM agent pipelines, distinguishing between deterministic software losses (L0-L3) and behavioral non-retrieval (L4-L8). Through 75,476 synthetic trials and a 372-trial real-agent pilot, the study finds an overall omission rate of 0.574, with L0-L3 accounting for 73.4% of weighted loss. Key factors increasing omission include context length (OR=7.43), q4 KV cache quantization, and scaled RoPE. The research highlights that omission is a pipeline-level property, not just a model failure, and proposes an instrumented attribution harness to diagnose failures at specific boundaries.
Entities (19)
Relation Signals (18)
Nine-layer taxonomy → includes → L4-L8 Layers
confidence 95% · Layers L4–L8 are behavioral
Nine-layer taxonomy → includes → L0-L3 Layers
confidence 95% · We present a nine-layer taxonomy (L0-L8)... Layers L0–L3 are deterministic
L0-L3 Layers → accountfor → Omission Rate
confidence 93% · deliberately injected deterministic faults at L0-L3 account for 73.4% of weighted loss
Omission Waterfall → distinguishes → behavioral non-retrieval
confidence 93% · conditional omission waterfall that distinguishes deterministic software loss from behavioral non-retrieval
Omission Waterfall → distinguishes → deterministic software loss
confidence 93% · conditional omission waterfall that distinguishes deterministic software loss from behavioral non-retrieval
Context Length → increases → Omission Rate
confidence 92% · Increasing context length is most strongly associated with omission (odds ratio 7.43)
q4 KV Cache → increases → Omission Rate
confidence 90% · Completed server-profile analyses associate q4 KV cache and scaled RoPE with higher omission.
Scaled RoPE → increases → Omission Rate
confidence 90% · Completed server-profile analyses associate q4 KV cache and scaled RoPE with higher omission.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Air-gapped and on-premises language-model agents can silently omit decision-critical facts at any boundary between source ingestion and final answer generation. We present a nine-layer taxonomy (L0-L8), an instrumented attribution harness, and a conditional omission waterfall that distinguishes deterministic software loss from behavioral non-retrieval. We analyze 75,476 controlled synthetic trials spanning five open-weight model configurations and two inference engines, together with a separate 372-trial real-agent pilot covering FHIR, PubMed, and SEC-EDGAR sources with LangChain and ADK orchestration. The weighted synthetic benchmark yields an omission rate of 0.574 (95% CI: 0.571-0.578); deliberately injected deterministic faults at L0-L3 account for 73.4% of weighted loss under the benchmark allocation. Increasing context length is most strongly associated with omission (odds ratio 7.43, 95% CI: 5.44-10.15). Completed server-profile analyses associate q4 KV cache and scaled RoPE with higher omission. In the real-agent pilot, 57.8% of traces are unsuccessful overall and 50.9% remain unsuccessful after excluding execution errors. These results establish pipeline-level attribution in a controlled stress test, but benchmark allocations, confounded model comparisons, and heuristic behavioral labels do not measure production prevalence or causal architectural effects.
Tags
Links
- Source: https://arxiv.org/abs/2607.22448v2
- Canonical: https://arxiv.org/abs/2607.22448v2
Trouble viewing inline? Open PDF directly →
Full Text
46,254 characters extracted from source content.
Expand or collapse full text
Where Facts Go Missing: A Layerwise Taxonomy and Per-Layer Attribution of Information Omission in Air-Gapped LLM Agent Pipelines Santhiya Rajan Abstract Air-gapped and on-premises language-model agents can silently omit decision-critical facts at any boundary between source ingestion and final answer generation. We present a nine-layer taxonomy (L0–L8), an instrumented attribution harness, and a conditional omission waterfall for separating byte-level software loss from behavioral non-retrieval. The committed aggregate artifacts describe 75,476 synthetic stress-test trials across five open-weight model configurations and two inference engines, plus a separate 372-trial real-data pilot. The displayed layer rates imply benchmark omission OR=0.574O_R=0.574 (95% CI [0.571, 0.578]); under the experiment’s deliberately weighted fault allocation, L0–L3 contribute 73.4% of waterfall loss. This is a controlled benchmark decomposition, not an estimate of omission prevalence in production. Context length has the largest reported behavioral association (OR=7.43OR=7.43, 95% CI [5.44, 10.15]). Completed server-profile follow-ups associate q4 KV cache and scaled RoPE with higher omission. In the real-data pilot, 0.578 of all trials were unsuccessful; after excluding 52 execution errors, 0.509 of the 320 outcome-coded trials remained unsuccessful. These results support pipeline-level diagnosis while also exposing important limits: deterministic fault injection drives the benchmark allocation, cross-model comparisons are confounded, and the raw main-sweep trials are absent from this checkout. Introduction Regulated and sovereign organizations increasingly deploy language-model agents that may never leave the building. A hospital that exposes patient records over a FHIR server, a law firm reviewing privileged discovery, or a national agency handling classified material cannot send tokens to a frontier API. The practical consequence is a distinctive stack: a quantized 4–8B open-weights model served by llama.cpp or vLLM on a single 16–24 GB GPU, wrapped by a Model Context Protocol (MCP) tool server that pages documents out of an internal system, and driven by a lightweight orchestration loop. This is a different reliability regime from the API-hosted frontier agent, and its dominant failure mode is different too. The well-studied failure of language models is hallucination: the assertion of something false. Its dual—omission, the silent absence of a fact that should have been surfaced—is comparatively understudied, even though reviews of clinical summarization note that omission detection remains far less developed than hallucination detection (Croxford et al. 2025). Omission is more dangerous precisely because it is silent. A hallucinated lab value can be caught by a reader who knows the range; an omitted critical value produces a fluent, confident, and complete-looking report that simply never mentions it. The canonical incident in an on-prem agent is coverage collapse: the tool returns 400 observations across paginated pages, the agent reads the first 20, and reports “no anomalies found”—a conclusion that is locally faithful to what the model saw and globally catastrophic. Our central claim is that omission in a deployed agent can be a property of the whole pipeline, not only of the model. A needle can be deleted by a PHI redactor before inference, dropped because pagination was not followed, evicted under context pressure, suppressed during generation, or lost when the orchestrator compacts history. Output-only evaluation cannot distinguish these mechanisms. We therefore take the pipeline as the unit of diagnosis and ask which layer is consistent with each observed failure. The controlled benchmark estimates attribution within its designed cell allocation; establishing production prevalence requires a representative field study. Contributions. • A nine-layer taxonomy of omission vectors (L0–L8) spanning the entire on-prem agent pipeline, from ingestion and tool protocol to the agent loop, unifying mechanisms that prior work treats in isolation (Section 3). • An attribution methodology that separates deterministic layers (audited at checkpoint taps) from behavioral layers (measured by controlled contrasts and a heuristic forced-logprob probe), formalized as an omission waterfall (Sections 3–4). • An open cross-architecture harness that runs the identical needle protocol across a sliding-window-hybrid target, a full-attention control, and an SSM-hybrid contrast; across llama.cpp and vLLM; and across no-framework, LangChain, and ADK orchestration against the same endpoint (Section 5). We separate the completed measurement populations. The aggregate reports cover a 75,476-trial synthetic sweep over five model configurations and two engines. A distinct 372-trial pilot covers two model configurations, LangChain and ADK, and five source/provider conditions. Completed KV-cache and RoPE profiles cover all five model configurations. Experiments without complete evidence are reserved for Future Scope rather than mixed into the reported findings. Related Work We organize prior work into five threads and, for each, state the gap this paper fills. Across all five, prior work operates at the output level—detecting whether a generated summary omits a fact—or benchmarks a single layer in isolation. Our contribution is orthogonal: pipeline-level attribution that assigns omission to its originating layer. Long-context degradation. Retrieval accuracy is position-dependent (“lost in the middle,” a U-shaped curve with >>30% swing (Liu et al. 2024; Gupte et al. 2025)) and, critically, length-dependent even when retrieval is perfect (Du et al. 2025): adding context degrades reasoning that does not depend on the added tokens. Benchmarks beyond single-needle retrieval—RULER (Hsieh et al. 2024), NoLiMa’s non-literal needles (Modarressi et al. 2025), and ∞Bench (Zhang et al. 2024)—show accuracy collapsing at long context even for models advertising large windows, a pattern that holds in medical QA (AlMannaa et al. 2025). Recent work extends this to agents under extreme context growth (Zeng et al. 2026), classifier-monitored context rot (Martin and Roger 2026; Xia et al. 2026), and even the destabilization of safety refusals in long context (Hadeliya et al. 2025). Gap: these characterize that long context hurts; they do not attribute a given production omission to the attention layer versus the engine or the orchestrator. Quantization, including the KV cache. Systematic evaluation finds that long-context tasks (≥ 4k tokens) are more sensitive to KV-cache quantization than to weight-only quantization (Li et al. 2024), because outlier structures in KV activations accumulate error across stored tokens as sequence length grows (Hooper et al. 2024). This is precisely hardware-budget-induced attention omission, and it is exactly the knob an on-prem operator turns to fit a longer window into fixed VRAM. Follow-on work refines KV quantization (Muller et al. 2026; Gokhale et al. 2025) and characterizes the fragility of KV eviction (Feng et al. 2025). Gap: this literature measures accuracy under a quantization setting; it does not separate “token never resided in cache” from “resident token was corrupted” within a running agent. Tool and MCP layer faults. Production agent harnesses truncate tool output at arbitrary character or token limits—e.g., ∼ 25K-token caps with 2KB previews (Anthropic 2026; Morph 2025)—and pay a tool-schema “context tax” of 5–50K tokens for tool definitions alone. Emerging taxonomies catalogue real and runtime faults in MCP software (Taraghi et al. 2026; Owotogbe et al. 2026), smelly tool descriptions (Hasan et al. 2026), and dynamic tool construction (Fei et al. 2025); the industry consensus mitigation is to pass resources/IDs rather than raw data (Microsoft 2025). Gap: these describe the fault modes; we fold them into a single attribution frame (L1) and measure their share of end-to-end omission. Knowledge conflict and prior override. When context contradicts a model’s parametric memory, stronger models persist in wrong internal memory even with correct evidence in context, and fine-tuning does not fully fix it (Sun et al. 2025; Zhao et al. 2026). Benchmarks (NQ-Swap (Longpre et al. 2021), ConflictBank (Su et al. 2024), ClashEval (Wu et al. 2024)) and decoder-side mitigations (Zhang et al. 2025; Jiang et al. 2026; Peng et al. 2026) isolate this behavior. This confirms that a needle test alone cannot isolate prior-induced omission (L6): only a conflict needle can. Gap: conflict benchmarks measure a behavior; we use the literal-vs-conflict contrast as an attribution instrument within the waterfall. Omission/faithfulness detection and compaction loss. Named methods detect omission at the output level: MED-OMIT weights omitted facts by clinical impact (Schumacher et al. 2023); an EMNLP-2025 industry detector targets medical summaries (Oukelmoun et al. 2025); CARE gives conformal omission/hallucination flags with coverage guarantees (Bedi et al. 2026); AgenticSum uses verify-then-answer loops (Piya and Beheshti 2026); and event-based recall benchmarks clinical time-series summaries (Shukla et al. 2026). Faithfulness taxonomies (Si et al. 2025; Ding et al. 2025) and medication-safety reviews (Normand et al. 2025) refine the target. A parallel line shows that history compaction silently erases information—safety constraints (Chen 2026), financial detail (Lee et al. 2026)—and studies the rate–distortion trade-off of what to keep (Colaco and Lahjouji 2026; Li et al. 2026). Gap: all of these are output-level detectors or single-stage studies; none attributes an omission to a pipeline layer, which is what an operator needs to act. A Layerwise Taxonomy of Omission We model the on-prem agent pipeline as an ordered sequence of nine layers, each of which can delete or corrupt a decision-critical fact. Table 1 lists the layers and their concrete mechanisms; below we expand the mechanisms that are specific to the air-gapped stack. Table 1: The nine-layer omission taxonomy (L0–L8), in pipeline order. Layers L0–L3 are deterministic: a fact either survives the byte/token stream at a checkpoint or it does not, so losses are counted exactly. Layers L4–L8 are behavioral: losses are estimated by controlled ablation and logit decomposition. # Layer Key omission mechanisms Class L0 Source / ingestion & redaction OCR table-structure loss, PHI/PII de-identification stripping values, local guardrail rewrites Determ. L1 Tool protocol & schema MCP transport size caps, unfollowed pagination (Bundle.link.next), tool-call parse failures, tool-list/schema truncation (context tax) Determ. L2 Orchestrator middleware string slicing, metadata stripping, history/state condensation Determ. L3 Serialization / tokenizer / template special-token injection (⟨ _of_turn⟩ ), template drift, tool-role folding, digit/code tokenization Determ. L4 Engine runtime & memory context shifting without --keep, VRAM limits, weight & KV-cache quantization, vLLM preemption Behav. L5 Attention / positional sliding-window gaps, lost-in-the-middle, RoPE misconfiguration Behav. L6 Priors / training bias prior override of anomalous facts, synthetic-data fragility, frequency smoothing Behav. L7 Decoding & structured output greedy/top-p pruning, repetition penalty, grammar/schema-constrained generation (GBNF) Behav. L8 Agent loop & output iteration caps, history compaction, context-budget policy, max_tokens/stop-sequence truncation Behav. The tool-protocol layer (L1) is layer zero for real agents. Treating “the tool returns a payload” as atomic hides the single most common omission in llama.cpp-class agents. A FHIR MCP server can expose 30+ tools whose JSON schemas alone consume 5–15k tokens; when the orchestrator caps the tool list, the model never learns a retrieval tool exists—omission by unreachable capability, invisible to any needle test. Small local models emit imperfect JSON, and harnesses that silently drop unparseable tool calls produce turns where the tool was never invoked. And pagination is rarely followed past page one: reading 20 of 400 observations and reporting “no anomalies” is an L1 loss, not a model error. Tokenizer and template (L3). A tool payload containing a literal ⟨ _of_turn⟩ string—plausible in scraped legal text or free-text notes—can prematurely terminate the model’s view of context depending on the --special flag, a silent truncation and a prompt-injection surface at once. Digit and code tokenization (4.8 mEq/L, ICD-10, NDC numbers) fragments unusually, and quantized small models reproduce these verbatim less reliably. Quantization as a first-class engine term (L4). Weight quantization (Q4/Q3) degrades long-context retrieval and exact numeric recall disproportionately to short-context chat quality, and KV-cache quantization—enabled precisely to fit a larger window into 16–24 GB—directly corrupts stored mid-context representations (Li et al. 2024; Hooper et al. 2024). The core on-prem dilemma is that a bigger window bought with heavier quantization can retrieve less than a smaller window at higher precision. Attention and configuration (L5). Sliding-window-hybrid stacks (local + global layers) and the lost-in-the-middle curve both live here, but so does a purely configurational failure: an operator who applies --rope-freq-scale or YaRN to stretch context past training length can scramble positional geometry and reproduce “mid-context blindness” that looks architectural but is a config bug. The two measurement classes. The nine layers split cleanly. Layers L0–L3 are deterministic software: a fact survives the byte/token stream at a checkpoint or it does not, so each loss is exact, per-payload, and attributable with certainty by inspection—no inference, no statistics. Layers L4–L8 are behavioral: the same tokens are present but the model may or may not use them, so we hold everything fixed, vary one factor, and estimate the change in retrieval rate over many trials. The omission waterfall. For a coherent cascade, let NiN_i be the number of evaluation opportunities entering layer i and LiL_i the number first lost there. Define the conditional layer rate ωi=Li/Ni _i=L_i/N_i, with Ni+1=Ni−LiN_i+1=N_i-L_i. The unconditional contribution of layer i is ci=Li/N0c_i=L_i/N_0. Therefore ci=ωi∏j<i(1−ωj),OR=∑i=08ci=1−∏i=08(1−ωi).c_i= _i\! _j<i(1- _j), O_R= _i=0^8c_i=1- _i=0^8(1- _i). (1) Equation 1 distinguishes conditional rates ωi _i from their shares of total loss, ci/ORc_i/O_R. We apply it to the experiment’s fixed, deliberately weighted benchmark allocation. It is consequently a diagnostic decomposition of that allocation, not an estimate of layer prevalence under an unknown production workload. Wilson intervals describe binomial sampling uncertainty within cells; they do not capture uncertainty from cell weighting, model selection, or heuristic attribution. The fixed order in Table 1 prevents a failure from being counted at more than one upstream checkpoint. Attribution Methodology Table 2: Checkpoint taps T0–T7. Each tap records the raw bytes or token IDs crossing a boundary, keyed by trial ID; the needle carries a unique canary (e.g., LAB-7Q4X9) so its presence at T1–T5 is checkable by exact match with no judge. T4 is detokenized and checked; T5 is read from engine logs. Tap Between What it captures T0 — raw source record (needle known) ↓ L0 ingestion / OCR / redaction T1 L0→ 1 post-ingestion text ↓ L1 MCP transport, pagination, parsing T2 L1→ 2 tool result received by orchestrator ↓ L2 formatting / truncation T3 L2→ 3 compiled prompt string ↓ L3 chat template + tokenizer T4 L3→ 4 final token-ID sequence (detokenize & check) ↓ L4 KV admission, context shift T5 L4→ 5 tokens resident in KV cache (from events) ↓ L5–L7 attention, priors, decoding T6 L7→ 8 generated output ↓ L8 agent loop, multi-turn T7 — final agent answer Instrumentation. Before any trials we add logging taps at every pipeline boundary (Table 2). Every needle carries a unique alphanumeric canary (e.g., LAB-7Q4X9) alongside its clinical or legal value, so presence at T1–T5 is a fully offline exact-match check. Each needle must round-trip the tokenizer losslessly (encode→ == original) before use, or L3 losses masquerade as L5 failures. Following needle-design practice (Gao et al. 2025; Wang et al. 2024; Ebrahimzadeh and Salili 2026), we build three families—literal (exact value), paraphrase (NoLiMa-style: query shares no surface words with the needle), and conflict (NQ-Swap-style: needle contradicts a strong prior, e.g., lithium 4.8 mEq/L against the model’s ∼ 0.8 expectation)—needles are inserted at sentence boundaries, and plausible distractors are included in half of all trials, since distractor-free noise overstates retrieval (Ebrahimzadeh and Salili 2026). Engine events (context shift, SWA checkpoint restore, vLLM preemption) are captured as per-trial covariates: these are the L4 attribution signals. Phase A — deterministic audit (no inference). For ∼ 500 synthetic payloads spanning 2k–64k tokens, we pass each through the real production components and check needle presence at T1–T4. A1 toggles the PHI redactor (L0); A2 places the needle on page 3 of 5 and checks whether the harness fetches past page 1 (L1); A3 sweeps payload sizes across orchestrator truncation limits (L2); A4 injects template special-token strings and detokenizes T4 (L3); A5 sweeps registered tool count (5/15/30) to measure schema tax (L1). Output: exact ω0..ω3 _0.. _3 as counts, each individually explainable. Phase B — engine isolation (L4). The goal is to separate “tokens never resident in cache” from “model failed to use resident tokens.” B1 sizes payloads at 0.5×,0.9×,1.1×,1.5×0.5×,0.9×,1.1×,1.5× of -c with --keep -1 on/off; when a context-shift event fires and the needle’s positions fall in the evicted range, failure attributes to L4 by the event log. B2 compares f16 with q4_0 KV-cache precision. B3 runs 5-turn conversations with the needle introduced at turn 1 and queried at turn 5 on the SWA-hybrid stack, targeting the documented checkpoint-restoration bug class (ggml-org 2026b, a, c). The engine version is pinned and recorded, and reruns follow version changes. Table 3: Completed Phase-C main-effect design: one factor at a time from the gold-path baseline (16k, depth 0.5, literal, Q4_K_M weights, f16 KV, greedy, native RoPE), with n=100n=100 per core main-effect cell. Separate server-profile reports provide the completed KV-cache and RoPE contrasts. Factor Levels Layer Context length 2k, 8k, 16k, 32k L5 Needle depth 0.1, 0.5, 0.9 L5 Needle family literal / paraphrase / conflict L5 vs. L6 KV-cache type f16 / q4_0 L4→ 5 Sampler greedy / t=0.7t=0.7 / rep 1.3 L7 RoPE config native / unnecessary scaling L5 Phase C — behavioral factorial (L5, L6, L7). From a verified gold path (deterministic layers lossless, no engine events fired), we run the completed main-effect matrix of Table 3 one factor at a time from a fixed baseline. Power analysis for proportions—detecting a 15-point drop from a 0.8 baseline at α=0.05α=0.05, power 0.8, needs ≈ 85 trials/arm—sets n=100n=100 per core main-effect cell (Gao et al. 2025; Zhao et al. 2025). Attribution reads: depth/length effects on literal needles → L5; the (conflict −- literal) gap at matched depth/length → L6; sampler effects on identical prompts → L7. Phase D — logit decomposition (separates L5/L6 from L7). For eligible failed Phase-B/C trials that survive T4, the implementation scores the expected continuation and records mean/minimum forced log probability plus a divergence index. A failure with mean forced log probability above the configured threshold −2.0-2.0 is labeled pruned_L7; otherwise a conflict needle is labeled unsurfaced_prior_L6, and other families are labeled unsurfaced_positional_L5. Thus Phase D is a rule-based diagnostic heuristic, not a causal per-token decomposition: the recorded rank does not currently determine the label, and attention versus prior is inferred from needle family. Updated Phase-D records are appended, so any row-level reanalysis must retain the latest record per trial ID. Phase E — agent loop (L8). On the best configuration from B–D we run end-to-end multi-step scenarios: E1 a pagination task requiring 5 tool calls with max-iterations ∈3,10∈\3,10\ (coverage == fraction of records referenced or explicitly dismissed); E2 history compaction on/off across a 10-turn session with the needle introduced early and queried late; E3 a max_tokens/stop-sequence sweep on answer-side truncation. We set ω8 _8 to end-to-end omission minus the single-turn omission predicted by the B–D model on matched tasks. Analysis. We report (i) the waterfall ω0,…,ω8 _0,…, _8 with 95% Wilson intervals; (i) ordinary fixed-effects logistic regression on gold-path Phase-C rows, retrieved ∼ C(factor) + …, with tabled contrasts oriented so OR>1OR>1 denotes higher omission; and (i) a mutually exclusive failure ledger. The code does not fit the previously specified random intercept for needle ID, and its confidence intervals are model-based rather than cluster-robust. Repeated needles and heterogeneous model-specific effects therefore remain sources of dependence. Odds ratios are associations conditional on the included factors, not percentages of omission caused. Ledger labels are deterministic for L0–L3 but heuristic for L5–L7 as described above. Experimental Setup Table 4: Cross-model matrix. The labels describe context-handling mechanisms, but the rows also differ in training data, tokenizer, scale, instruction tuning, and templates. Results are therefore descriptive cross-model contrasts, not isolated architecture effects. Slot Model Architecture (context handling) Role swa_hybrid Gemma 4 E4B it hybrid sliding-window + global attention target under test full_attn Qwen3-8B dense full attention + GQA, 32k native full-attention control ssm_hybrid Granite 4.0-H-Tiny Mamba-2 / transformer hybrid (SSM state) recurrent-state contrast gemma_moe Gemma 4 26B-A4B it SWA pattern + 128-expert MoE + vision exploratory (MoE) qwen_linear_moe Qwen3.5-35B-A3B gated linear attention + 256-expert MoE exploratory (linear attn.) Models. Table 4 defines three primary and two exploratory configurations. Matching prompts across rows improves comparability, but it does not isolate attention architecture: model family, tokenizer, parameter count, training, and chat template change simultaneously. We therefore report model-specific omission rates as descriptive contrasts and reserve causal architectural claims for a future within-family or weight-matched study. Engines, profiles, and frameworks. KV-cache type and RoPE scaling are llama-server start flags rather than request parameters, so the completed follow-up runs restart the server and stamp the profile on each trial. Both contrasts cover all five model configurations. The core engine axis compares llama.cpp with vLLM where supported. The separate real-data pilot crosses LangChain and ADK with two served model configurations; because it has no no-framework arm and contains framework-specific execution errors, it estimates end-to-end framework-conditioned performance rather than an isolated orchestration effect. Sources, domains, and offline evaluation. The synthetic sweep uses generated clinical-style payloads and local inference. The real-data pilot covers FHIR, PubMed, and SEC-EDGAR material through three injected-document sources and two live MCP-provider conditions. Model serving and answer scoring remain local, but selected source acquisition uses public online services; the experiment is therefore air-gap-compatible at inference time, not fully offline end to end. All reported retrieval labels use case-normalized expected-answer substring matching, including paraphrase trials. The pilot used vLLM 0.25.1 with tensor parallelism over two L4 GPUs per model configuration. Results Table 5: Allocation-weighted omission waterfall (Eq. 1). Conditional rates imply OR=0.574O_R=0.574; intervals are Wilson intervals from the committed aggregate. Layer ωi _i 95% CI Label source L0 ingestion/redaction 0.160 [0.158, 0.163] checkpoint L1 tool protocol 0.175 [0.172, 0.178] checkpoint L2 orchestrator 0.165 [0.162, 0.168] checkpoint L3 tokenizer/template 0.000 [0.000, 0.000] checkpoint L4 engine/memory 0.000 [0.000, 0.000] telemetry L5 attention/position 0.189 [0.185, 0.192] heuristic L6 priors 0.012 [0.011, 0.014] heuristic L7 decoding 0.018 [0.017, 0.019] heuristic L8 agent loop 0.065 [0.062, 0.067] task contrast Total ORO_R 0.574 [0.571, 0.578] implied by rates The committed aggregate reports describe 75,476 main-sweep trials over five model configurations and the supported llama.cpp/vLLM combinations. Server-profile follow-ups and the real-data pilot are separate populations and are not pooled into the main waterfall. The raw main-sweep and profile trial files are absent from this checkout, so the following audit reconciles the committed aggregate values rather than claiming an independent row-level reproduction. Waterfall. Substitution into Eq. 1 gives OR=0.5743O_R=0.5743 before rounding. The weighted contributions of L0–L3 sum to 0.42130.4213, or 73.4% of waterfall loss; L5 is the largest displayed behavioral conditional rate (ω5=0.189 _5=0.189). Because Phase A deliberately injects redaction, pagination, and truncation failures, 73.4% describes this benchmark allocation. It does not show that 73.4% of naturally occurring production omissions are caused by middleware. Table 6: Omission-direction odds ratios (>1>1 means higher omission). Core Phase-C contrasts use the fixed-effects model; profile contrasts come from separately launched server-profile follow-ups against matched gold-path subsets. Factor (contrast) OR 95% CI Core Phase C Length (32k vs. 2k) 7.43 [5.44, 10.15] Depth (0.5 vs. 0.1) 1.38 [1.23, 1.54] Paraphrase vs. literal 3.65 [3.31, 4.03] Conflict vs. literal 1.42 [1.28, 1.59] Rep. penalty 1.3 vs. greedy 0.32 [0.26, 0.41] Separate profile follow-ups q4 KV vs. f16 KV (5/5) 2.25 [2.07, 2.45] Scaled vs. native RoPE (5/5) 1.99 [1.83, 2.16] Behavioral and profile associations. Context length has the largest reported core association: the omission odds at 32k are 7.43 times those at 2k. Paraphrase scoring is also strongly associated with failure, but the use of literal substring matching makes this contrast partly a measurement effect rather than pure semantic retrieval. The repetition-penalty setting is associated with lower omission (OR=0.32OR=0.32); this is not evidence of a general decoding benefit beyond the tested prompts. The completed matched profile reports associate q4 KV cache with higher omission than f16 (OR=2.25OR=2.25) and scaled RoPE with higher omission than native RoPE (OR=1.99OR=1.99). These are server-launch configuration associations estimated across all five model configurations. Table 7: Descriptive total omission in the synthetic sweep. Model rows are not causal architecture comparisons; engine rows aggregate only supported combinations. Axis Level ORO_R Model swa_hybrid 0.549 full_attn 0.716 ssm_hybrid 0.489 gemma_moe 0.528 qwen_linear_moe 0.474 Engine llama.cpp 0.593 vLLM 0.537 Cross-model and engine contrasts. The full-attention configuration has the highest descriptive omission (0.7160.716), while the SSM-hybrid configuration is lower (0.4890.489). This reverses the anticipated rank ordering but cannot establish that dense attention causes more omission, because every model row changes multiple architectural and training variables. Likewise, the aggregate llama.cpp–vLLM difference (0.5930.593 vs. 0.537) is descriptive because engine support is unbalanced across model configurations. Table 8: Audited aggregate endpoints: (a) real-pilot failure, (b) source/provider rates, and (c) the allocation-weighted synthetic failure ledger. Panel (c) is ordered by the waterfall layers and uses the same checkpoint-first unconditional contributions as Table 5; source and domain are confounded, and behavioral labels remain heuristic. (a) Real-pilot endpoints Group n Rate 95% CI All, end-to-end 372 0.578 [0.527, 0.627] LangChain 186 0.602 [0.530, 0.670] ADK 186 0.554 [0.482, 0.623] Outcome-coded 320 0.509 [0.455, 0.564] (b) Real-pilot source/provider Source n Rate 95% CI FHIR injected 108 0.148 [0.093, 0.227] SEC injected 108 0.639 [0.545, 0.723] PubMed injected 108 0.889 [0.816, 0.935] SEC MCP 24 0.500 [0.314, 0.686] PubMed MCP 24 0.917 [0.742, 0.977] (c) Synthetic failure ledger (checkpoint-first) Attributed cause Fraction Software loss (L0–L3) 0.421 Evicted (L4) 0.000 Unsurfaced-positional (L5) 0.109 Unsurfaced-prior (L6) 0.006 Pruned heuristic (L7) 0.008 Loop loss (L8) 0.030 Real-data pilot. The pilot crosses two served model configurations, two frameworks, and five source/provider conditions. Of 372 trials, 157 received a final success attribution, 163 were outcome-coded as non-success, and 52/372 (14.0%) remained unknown: 44 LangChain recursion-limit errors and eight SEC filing-not-found errors. Eleven of the 163 non-successes contain the expected substring but remain loop failures under the pilot endpoint, so this measure is broader than literal omission. Counting unknown errors as unsuccessful yields 0.578; excluding them gives 0.509 non-success among 320 outcome-coded trials. Framework rates include their execution errors and should be read as end-to-end reliability, not as isolated framework effects. Table 8 shows large source differences, but each domain is tied to its source and sample construction, so a domain effect is not separately identified. Failure ledger. Panel (c) sums to 0.574 after rounding and is the checkpoint-first, mutually exclusive rendering of the same unconditional contributions shown in the waterfall. Software loss (L0–L3, 0.421) is the largest benchmark category, followed by positional non-surfacing (0.109); loop loss (L8, 0.030) is smaller. These rankings prioritize debugging within this stress-test design; they should not be transferred to production without representative workload weights and validation of the behavioral labels. Limitations and Future Scope The reported findings above use only completed experiments. Partial, unexecuted, or unevaluated work is confined to the following future scope. Complete the server-profile matrix. The partial Q8-versus-Q4_K_M weight study is not included in Results. It must be completed for gemma_moe and qwen_linear_moe, then repeated with balanced cells, identical model artifacts, launch-order randomization, and per-model estimates. Planned Q3 weight and q8 KV-cache conditions should be executed in the same balanced design. Validate semantic scoring and Phase-D labels. Current outcomes use expected-answer substring matching. A blinded human set and a local semantic scorer should be calibrated by family and domain, with inter-annotator agreement and sensitivity analysis. Separately, the −2.0-2.0 forced-logprob rule should be validated against labeled attention, prior, and decoding failures. Phase-D append-only records must be deduplicated before reanalysis. Run interaction and dependence analyses. KV-cache×length and depth×family interactions should be run and reported with model×factor terms and needle-level clustered or hierarchical uncertainty. A within-family, weight-matched comparison is required for causal architecture attribution. Expand and repair the real-data pilot. A rerun should eliminate the 52 execution errors, add a no-framework baseline and llama.cpp where supported, and increase the 24-trial live-provider cells. Crossing multiple sources within each domain is necessary to separate source/provider effects from domain effects. Measure production prevalence and runtime-detector utility. A representative deployment study is needed to replace benchmark cell weights with workload frequencies. Candidate detectors—canary injection, coverage accounting, forced citation, logprob monitoring, two-pass cross-checks, and engine-telemetry alarms—should be compared with output-level detectors (Schumacher et al. 2023; Bedi et al. 2026) on precision, recall, false-alarm cost, and latency. Pass-by-reference should be evaluated as a mitigation while controlling for context-length degradation (Du et al. 2025). Archive complete artifacts. The raw 75,476-trial main sweep and raw server-profile trials must be versioned with deduplication rules, model hashes, engine commits, launch commands, and an environment manifest. Without those files, the aggregate tables can be checked for internal arithmetic but not independently regenerated. Conclusion This study reframes omission as a pipeline-diagnosis problem and provides a nine-layer taxonomy, instrumented checkpoints, and an auditable waterfall. Within the deliberately weighted synthetic stress test, the displayed rates imply OR=0.574O_R=0.574, with 73.4% of loss assigned to L0–L3. The operational lesson is conditional but useful: verify transport, pagination, truncation, and loop coverage before attributing a missing fact to model cognition. The evidence does not yet establish production prevalence or causal architecture effects; completing the experiments in Section Limitations and Future Scope is necessary for those stronger claims. References F. AlMannaa, T. Tseriotou, J. Chim, and M. Liakata (2025) Investigating LLM Capabilities on Long Context Comprehension for Medical Question Answering. External Links: 2510.18691 Cited by: Long-context degradation.. Anthropic (2026) Tool results capped at ∼ 25k tokens with a 2kb preview. Note: https://github.com/anthropics/claude-code/issues/45770GitHub issue #45770. Accessed: 2026-07-16 Cited by: Tool and MCP layer faults.. S. Bedi, B. Lin, A. Y. Zhou, C. O. Stanwyck, J. A. Jindal, S. Koyejo, D. Stutz, and N. H. Shah (2026) CARE: A Conformal Safety Layer for Medical Summarization. External Links: 2606.08969 Cited by: Omission/faithfulness detection and compaction loss., Measure production prevalence and runtime-detector utility.. S. Chen (2026) Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents. External Links: 2606.22528 Cited by: Omission/faithfulness detection and compaction loss.. A. G. Colaco and N. Lahjouji (2026) What to Keep, What to Forget: A Rate–Distortion View of Memory Compaction in LLMs and Agents. External Links: 2607.08032 Cited by: Omission/faithfulness detection and compaction loss.. E. Croxford, Y. Gao, N. Pellegrino, K. Wong, G. Wills, E. First, F. Liao, C. Goswami, B. Patterson, and M. Afshar (2025) Current and future state of evaluation of large language models for medical summarization tasks. npj Health Systems. Note: https://w.nature.com/articles/s44401-024-00011-2 Cited by: Introduction. Q. Ding, L. Luo, Y. Cao, and P. Luo (2025) The Gray Zone of Faithfulness: Taming Ambiguity in Unfaithfulness Detection. External Links: 2510.21118 Cited by: Omission/faithfulness detection and compaction loss.. Y. Du, M. Tian, S. Ronanki, S. Rongali, S. Bodapati, A. Galstyan, A. Wells, R. Schwartz, E. A. Huerta, and H. Peng (2025) Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. External Links: 2510.05381 Cited by: Long-context degradation., Measure production prevalence and runtime-detector utility.. A. Ebrahimzadeh and S. M. Salili (2026) Not All Needles Are Found: How Fact Distribution and Don’t Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs. External Links: 2601.02023 Cited by: Instrumentation.. X. Fei, X. Zheng, and H. Feng (2025) MCP-Zero: Active Tool Discovery for Autonomous LLM Agents. External Links: 2506.01056 Cited by: Tool and MCP layer faults.. Y. Feng, H. Guo, J. Lv, S. K. Zhou, and X. Xie (2025) Taming the Fragility of KV Cache Eviction in LLM Inference. External Links: 2510.13334 Cited by: Quantization, including the KV cache.. Y. Gao, Y. Xiong, W. Wu, Z. Huang, B. Li, and H. Wang (2025) U-NIAH: unified RAG and LLM evaluation for long context needle-in-a-haystack. ACM Transactions on Information Systems. Note: https://doi.org/10.1145/3786609 Cited by: Instrumentation., Phase C — behavioral factorial (L5, L6, L7).. ggml-org (2026a) Context shift not working with gemma-4 and quantized KV cache. Note: https://github.com/ggml-org/llama.cpp/issues/21379llama.cpp issue #21379. Accessed: 2026-07-16 Cited by: Phase B — engine isolation (L4).. ggml-org (2026b) Gemma-4 SWA checkpoint restoration discards mid-conversation context. Note: https://github.com/ggml-org/llama.cpp/issues/21769llama.cpp issue #21769. Accessed: 2026-07-16 Cited by: Phase B — engine isolation (L4).. ggml-org (2026c) SWA models in RAG pipelines enter continuous checkpoint-invalidation loops. Note: https://github.com/ggml-org/llama.cpp/issues/24587llama.cpp issue #24587. Accessed: 2026-07-16 Cited by: Phase B — engine isolation (L4).. S. Gokhale, D. Das, R. Patwari, A. Sirasao, and E. Delaye (2025) KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference. External Links: 2512.01953 Cited by: Quantization, including the KV cache.. M. Gupte, E. Dixit, M. Tayyab, and A. Adiththan (2025) What Works for ’Lost-in-the-Middle’ in LLMs? A Study on GM-Extract and Mitigations. External Links: 2511.13900 Cited by: Long-context degradation.. T. Hadeliya, M. A. Jauhar, N. Sakpal, and D. Cruz (2025) When Refusals Fail: Unstable Safety Mechanisms in Long-Context LLM Agents. External Links: 2512.02445 Cited by: Long-context degradation.. M. M. Hasan, H. Li, G. K. Rajbahadur, B. Adams, and A. E. Hassan (2026) Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptions. External Links: 2602.14878 Cited by: Tool and MCP layer faults.. C. Hooper, S. Kim, H. Mohammadzadeh, M. W. Mahoney, Y. S. Shao, K. Keutzer, and A. Gholami (2024) KVQuant: towards 10 million context length LLM inference with KV cache quantization. External Links: 2401.18079 Cited by: Quantization, including the KV cache., Quantization as a first-class engine term (L4).. C. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg (2024) RULER: what’s the real context size of your long-context language models?. External Links: 2404.06654 Cited by: Long-context degradation.. R. Jiang, T. Wu, Y. Wang, B. Zhu, and L. Huang (2026) From Context-Aware to Conflict-Aware: Generalizing Contrastive Decoding for Knowledge Conflict in LLMs. External Links: 2606.10298 Cited by: Knowledge conflict and prior override.. H. Lee, S. Park, S. Lee, J. Seo, J. Lee, S. Yoo, M. Kim, C. Na, Z. Wang, Z. Golkhou, M. Kim, S. Sabanis, A. Lopez-Lira, D. Mehta, S. Lee, C. Choi, W. Ahn, and Y. Lee (2026) When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysis. External Links: 2606.29251 Cited by: Omission/faithfulness detection and compaction loss.. S. Li, X. Ning, L. Wang, T. Liu, X. Shi, S. Yan, G. Dai, H. Yang, and Y. Wang (2024) Evaluating quantized large language models. External Links: 2402.18158 Cited by: Quantization, including the KV cache., Quantization as a first-class engine term (L4).. Y. Li, Z. Hou, Y. Jing, J. Tang, and Y. Dong (2026) CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents. External Links: 2607.05378 Cited by: Omission/faithfulness detection and compaction loss.. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024) Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, p. 157–173. Cited by: Long-context degradation.. S. Longpre, K. Perisetla, A. Chen, N. Ramesh, C. DuBois, and S. Singh (2021) Entity-based knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Note: arXiv:2109.05052 Cited by: Knowledge conflict and prior override.. S. Martin and F. Roger (2026) Classifier Context Rot: Monitor Performance Degrades with Context Length. External Links: 2605.12366 Cited by: Long-context degradation.. Microsoft (2025) Pass MCP resources, not raw data. Note: https://microsoft.github.io/mcscatblog/posts/mcp-resources-as-tool-inputs/Accessed: 2026-07-16 Cited by: Tool and MCP layer faults.. A. Modarressi, H. Deilamsalehy, F. Dernoncourt, T. Bui, R. A. Rossi, S. Yoon, and H. Schütze (2025) NoLiMa: long-context evaluation beyond literal matching. External Links: 2502.05167 Cited by: Long-context degradation.. Morph (2025) MCP output too large. Note: https://w.morphllm.com/mcp-output-too-largeAccessed: 2026-07-16 Cited by: Tool and MCP layer faults.. L. K. Muller, P. Bich, C. Boretti, H. Chang, J. Zhuang, and L. Cavigelli (2026) KVarN: Variance-Normalized KV-Cache Quantization Mitigates Error Accumulation in Reasoning Tasks. External Links: 2606.03458 Cited by: Quantization, including the KV cache.. O. Normand, E. Borsi, M. Fruin, L. E. Walker, J. Heagerty, C. C. Holmes, A. J. Avery, I. E. Buchan, and H. Coppock (2025) A Real-World Evaluation of LLM Medication Safety Reviews in NHS Primary Care. External Links: 2512.21127 Cited by: Omission/faithfulness detection and compaction loss.. A. Oukelmoun, N. Semmar, G. de Chalendar, C. Cormi, M. Oukelmoun, E. Vibert, and M. Allard (2025) Detecting omissions in LLM-generated medical summaries. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, p. 325–337. Note: https://aclanthology.org/2025.emnlp-industry.22.pdf Cited by: Omission/faithfulness detection and compaction loss.. J. Owotogbe, I. Kumara, W. van den Heuvel, D. A. Tamburri, A. K. Iannillo, and R. Natella (2026) A Taxonomy of Runtime Faults in Model Context Protocol Servers. External Links: 2606.05339 Cited by: Tool and MCP layer faults.. H. Peng, J. Tang, W. Zeng, H. Xu, and X. Zhao (2026) Navigating Unreliable Parametric and Contextual Knowledge: Explicit Knowledge Conflict Resolution for LLM Inference. External Links: 2606.20245 Cited by: Knowledge conflict and prior override.. F. L. Piya and R. Beheshti (2026) AgenticSum: An Agentic Inference-Time Framework for Faithful Clinical Text Summarization. External Links: 2602.20040 Cited by: Omission/faithfulness detection and compaction loss.. E. Schumacher, D. Rosenthal, D. Naik, V. Nair, L. Price, G. Tso, and A. Kannan (2023) Extrinsically-Focused Evaluation of Omissions in Medical Summarization. External Links: 2311.08303 Cited by: Omission/faithfulness detection and compaction loss., Measure production prevalence and runtime-detector utility.. A. Shukla, Y. Yuan, B. Tamo, Y. Wang, M. Nnamdi, S. Tan, J. Li, B. Marteau, B. Willingham, and M. Wang (2026) Benchmarking LLM Summaries of Multimodal Clinical Time Series for Remote Monitoring. External Links: 2603.01557 Cited by: Omission/faithfulness detection and compaction loss.. S. Si, Q. Wang, H. Zhao, Y. Bai, G. Chen, K. Luo, G. Chen, F. Qi, M. Zhang, B. Chang, and M. Sun (2025) FaithLens: Detecting and Explaining Faithfulness Hallucination. External Links: 2512.20182 Cited by: Omission/faithfulness detection and compaction loss.. Z. Su, J. Zhang, X. Qu, T. Zhu, Y. Li, J. Sun, J. Li, M. Zhang, and Y. Cheng (2024) ConflictBank: a benchmark for evaluating the influence of knowledge conflicts in LLMs. External Links: 2408.12076 Cited by: Knowledge conflict and prior override.. K. Sun, F. Bai, and M. Dredze (2025) Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict. External Links: 2506.06485 Cited by: Knowledge conflict and prior override.. M. Taraghi, M. M. Morovati, and F. Khomh (2026) Real Faults in Model Context Protocol (MCP) Software: a Comprehensive Taxonomy. External Links: 2603.05637 Cited by: Tool and MCP layer faults.. W. Wang, S. Zhang, Y. Ren, Y. Duan, T. Li, S. Liu, M. Hu, Z. Chen, K. Zhang, L. Lu, X. Zhu, P. Luo, Y. Qiao, J. Dai, W. Shao, and W. Wang (2024) Needle in a multimodal haystack. External Links: 2406.11230 Cited by: Instrumentation.. K. Wu, E. Wu, and J. Zou (2024) ClashEval: quantifying the tug-of-war between an LLM’s internal prior and external evidence. External Links: 2404.10198 Cited by: Knowledge conflict and prior override.. S. Xia, Y. Wang, Z. Huang, and P. Liu (2026) Diagnosing and Mitigating Context Rot in Long-horizon Search. External Links: 2606.29718 Cited by: Long-context degradation.. W. Zeng, Y. Huang, and J. He (2026) LOCA-bench: benchmarking language agents under controllable and extreme context growth. External Links: 2602.07962 Cited by: Long-context degradation.. Q. Zhang, Z. Xiang, Y. Xiao, L. Wang, J. Li, X. Wang, and J. Su (2025) FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation. External Links: 2506.08938 Cited by: Knowledge conflict and prior override.. X. Zhang, Y. Chen, S. Hu, Z. Xu, J. Chen, M. K. Hao, X. Han, Z. L. Thai, S. Wang, Z. Liu, and M. Sun (2024) ∞bench: Extending long context evaluation beyond 100k tokens. External Links: 2402.13718 Cited by: Long-context degradation.. T. Zhao, J. Chen, S. Zhang, H. Zhu, Q. Lin, and J. Liu (2026) Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method. External Links: 2604.11209 Cited by: Knowledge conflict and prior override.. Y. Zhao, W. Chen, Z. Xu, M. Patwardhan, C. Wang, Y. Liu, L. Vig, and A. Cohan (2025) AbGen: evaluating large language models in ablation study design and evaluation for scientific research. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12479–12491. Note: https://aclanthology.org/2025.acl-long.611/ Cited by: Phase C — behavioral factorial (L5, L6, L7)..