Paper deep dive
What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions
Yichao Gao, Yumo Zhang, Yunhao Yao, Haohua Du, Puhan Luo, Ruiqi Li, Zhiqiang Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/26/2026, 5:22:21 AM
Summary
The paper introduces AttnLocate, a runtime framework designed to detect and localize behavior-guiding instructions within Large Language Model (LLM) agents that are vulnerable to injection attacks. By treating the localization of influential context spans as an object detection problem, AttnLocate utilizes a multi-head, multi-layer attention aggregation scheme to create token-level features. These features are processed by a 1-D U-Net with an anchor-free detection head to identify specific text spans that influence tool-calling decisions. The system then adjudicates the legitimacy of these instructions based on the authority of the external provider, effectively mitigating indirect prompt injection and tool poisoning attacks without requiring model retraining.
Entities (8)
Relation Signals (6)
AttnLocate → mitigates → Injection Attacks
confidence 95% · AttnLocate dynamically adjudicates malicious invocation attempts... covering scenarios involving indirect prompt injection and tool poisoning.
AttnLocate → uses → 1-D U-Net
confidence 95% · a 1-D U-Net equipped with an anchor-free detection head is deployed to detect these spans.
AttnLocate → employs → Attention Aggregation
confidence 93% · we design a multi-head, multi-layer attention aggregation scheme to construct a token-level feature space tailored for object detection.
LLM Agents → vulnerableto → Injection Attacks
confidence 92% · LLM agents integrated with external resources... make them vulnerable to injection attacks
AttnLocate → evaluatedon → MCPTox
confidence 90% · We evaluate AttnLocate... against different injection attacks: MCPTox... for tool poisoning
AttnLocate → evaluatedon → InjecAgent
confidence 90% · and InjecAgent... for indirect prompt injection
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:LLM agents integrated with external resources gain complex task capabilities, yet the unified natural-language context channel makes them vulnerable to injection attacks: untrusted external data may be dynamically parsed as behavior-guiding instructions during LLM inference, thereby subverting the agent's decision. Existing defenses focus on static detection or isolation of malicious content at the input/output level, remains insufficient for detecting such dynamic inducements that arise during model reasoning. We propose Attnlocate, a runtime framework for fine-grained localization of context spans that genuinely influence tool-calling decisions, i.e., behavior-guiding instructions. Attnlocate casts this localization problem as an object detection task, aiming to detect the distinctive activation traces induced by behavior-guiding instructions within the attention matrix. Specifically, we design a multi-head, multi-layer attention aggregation scheme to construct a token-level feature space tailored for object detection. Then, a 1-D U-Net equipped with an anchor-free detection head is deployed to detect these spans. Finally, based on the authority of the provider from which the detected behavior-guiding spans originate, Attnlocate dynamically adjudicates malicious invocation attempts. We evaluate Attnlocate across ten agent configurations from five LLM families, covering scenarios involving indirect prompt injection and tool poisoning. Attnlocate achieves a mean IoU of 0.743, an average AUROC of 0.956, and a 0.934 true-positive rate at 0.067 false-positive rate. It also transfers effectively across unseen models and supports authority policy adaptation without retraining.
Tags
Links
- Source: https://arxiv.org/abs/2608.24022v1
- Canonical: https://arxiv.org/abs/2608.24022v1
Trouble viewing inline? Open PDF directly →
Full Text
45,078 characters extracted from source content.
Expand or collapse full text
What Guides the Agent? Adjudicating Unauthorized Behavior via Localizing Behavior-Guiding Instructions Yichao Gao Yumo Zhang Yunhao Yao Haohua Du Puhan Luo Ruiqi Li Zhiqiang Wang †thanks: Corresponding author. Abstract LLM agents integrated with external resources gain complex task capabilities, yet the unified natural-language context channel makes them vulnerable to injection attacks: untrusted external data may be dynamically parsed as behavior-guiding instructions during LLM inference, thereby subverting the agent’s decision. Existing defenses primarily rely on static input/output-level detection or isolation of overtly malicious content, yet fall short of identifying injections that, though seemingly benign, actively subvert the model’s decision-making process. We propose AttnLocate, a runtime framework for fine-grained localization of context spans that genuinely influence tool-calling decisions, i.e., behavior-guiding instructions. AttnLocate casts this localization problem as an object detection task, aiming to detect the distinctive activation traces induced by behavior-guiding instructions within the attention matrix. Specifically, we design a multi-head, multi-layer attention aggregation scheme to construct a token-level feature space tailored for object detection. Then, a 1-D U-Net equipped with an anchor-free detection head is deployed to detect these spans. Finally, based on the authority of the provider from which the detected behavior-guiding spans originate, AttnLocate dynamically adjudicates malicious invocation attempts. We evaluate AttnLocate across ten agent configurations from five LLM families, covering scenarios involving indirect prompt injection and tool poisoning. AttnLocate achieves a mean IoU of 0.743, an average AUROC of 0.956, and a 0.934 true-positive rate at 0.067 false-positive rate. It also transfers effectively across unseen models and supports authority policy adaptation without retraining. 1University of Science and Technology of China 2University of Washington 3Beihang University gyc77@mail.ustc.edu.cn, zhiqiang.wang@mail.ustc.edu.cn 1 Introduction Figure 1: Illustration of an external resource injection attack and AttnLocate’s defense mechanism. AttnLocate localizes behavior-guiding spans and then adjudicates unauthorized behaviors based on its origin. Large language model (LLM) agents have rapidly evolved from conversational chatbots to autonomous decision-making systems that interact with external resources, e.g., tools, databases, and APIs. By integrating natural language reasoning with tool invocation capabilities, these agents achieve remarkable performance across complex tasks (Zhao et al. 2023). However, the very interface that enables this flexibility, i.e., the unified natural-language context channel, also introduces a critical vulnerability: injection attacks. In such attacks, untrusted external content (e.g., tool outputs, metadata fields) is dynamically parsed by the LLM as part of its input context and may be misinterpreted as behavior-guiding instructions, thereby subverting the agent’s intended decision and causing unauthorized tool execution (OWASP GenAI Security Project 2025; OWASP Foundation 2025). For example, as illustrated in Fig. 1, an attacker injects a malicious instruction into an external resource, causing the agent to disregard the user’s benign request to read /tmp/hello.txt and instead invoke ReadFile on the sensitive SSH private key ~/.ssh/id_rsa. Extensive work has been proposed to defend against such injection attacks: Static scanning reduces exposure by detecting or removing suspicious content before it enters the model’s input context, structurally separating instructions from data (Chen et al. 2025; AI 2025; He et al. 2025) Architectural defenses provide system-level containment by isolating untrusted components, explicitly separating control and data flows (Debenedetti et al. 2025; Wu et al. 2025), or decoupling trusted planning from constrained execution (Li et al. 2026). Behavior-auditing methods mediate agent executions and constraining unsafe actions under explicit policies (Jing et al. 2025). While these methods can mitigate attack exposure or reject unsafe outputs, they are incapable of identifying context spans that, despite appearing benign, are nonetheless interpreted as behavior-guiding instructions and consequently distort the model’s decision-making process. Recently, attribution-based methods (Wang et al. 2026b; Wang et al. 2025c; Wang et al. 2026a) have been proposed to estimate influence at fixed granularities (e.g., passages or metadata fields), but their coarse units entangle a short malicious instruction with its benign carrier. Therefore, a robust defense must go beyond detecting overtly malicious content and perform fine-grained identification of both the span that is functionally interpreted as an instruction and its originating provider. Thus, we propose AttnLocate, a runtime monitoring framework that performs fine-grained localization of behavior-guiding instructions within the agent context and adjudicates unauthorized invocation behavior based on the origin of the behavior-guiding span (Figure 1). AttnLocate is built on a key observation: during LLM generation, tokens that genuinely influence the decision exhibit distinctive activation patterns in the attention matrix, patterns that are qualitatively different from general contextual salience or attention sinks. Challenge. Three challenges must be addressed to achieve a robust localization and adjudication system. C1: attention does not directly indicate behavioral influence, as it is corrupted by noise and exhibits substantial variance across different heads/layers.C2: instruction spans have uncertain boundaries and variable lengths, making token classification inadequate for localization. C3: The legitimacy of an behavior depends on policy configuration and the authority of the external resource provider, thus requiring policy-adaptive adjudication mechanism. AttnLocate Design. We first design a multi-head, multi-layer attention aggregation mechanism to extracts token-level feature vectors capturing complementary dependence patterns while suppressing head/layer variance (Addressing C1). AttnLocate then casts the behavior-guiding instruction localization as an object detection task over the attention matrix, employing a 1D U-Net backbone with an anchor-free detection head for variable-length span boundary prediction. This is further reinforced by a sink-aware regularization to mitigate the impact of attention noise (Addressing C2). Finally, an authority arbiter adjudicates unauthorized behavior based on the source provider of the localized span and a configurable authority policy, enabling policy-adaptive adjudications for different scenarios (Addressing C2). We evaluate AttnLocate across ten agent configurations from six model families against different injection attacks: MCPTox(Wang et al. 2026c) for tool poisoning(Beurer-Kellner and Fischer 2025) and InjecAgent(Zhan et al. 2024) for indirect prompt injection(OWASP GenAI Security Project 2025). AttnLocate achieves an average mIoU of 0.7430.743 for localizing behavior-guiding instructions and attains 0.9560.956 AUROC and 0.9340.934 recall for adjudicating unauthorized behavior. AttnLocate also generalizes to unseen models and supports policy adaptation without retraining. Our main contributions are as follows: ∙ We formulate the detection of unauthorized agent behavior as a dynamic instruction-localization problem and demonstrate that attention patterns provide evidence for identifying the exact context spans that guide tool-call decisions. ∙ We propose AttnLocate, a novel framework combining attention aggregation, a 1-D U-Net with sink-aware objectness regularization, and an authority-based arbiter to jointly localize and adjudicate unauthorized instructions. ∙ We conduct extensive experiments on two injection scenarios, demonstrating superior localization (behavior-guiding span) and adjudication (unauthorized behavior) performance, strong cross-model generalization, and configurable authority policy adaption. Figure 2: Attention activation distribution for a successful attack. The attention exhibits pronounced activation peaks over those input spans that determine the final tool-calling decision (i.e., behavior-guiding instructions), including the invoked tool, the user query (benign), and the externally injected instruction that is resolved into a behavior-guiding instruction (malicious). 2 Background and Related Work 2.1 Attention Preliminaries. Transformer-based LLMs generate decisions through self-attention (Vaswani et al. 2023): Given an input context X=(x1,…,xn)X=(x_1,…,x_n) and a generated decision Y=(y1,…,ym)Y=(y_1,…,y_m), the attention from yiy_i to xjx_j at layer ℓ and head h is Ai,jℓ,h=softmax(Qℓ,hKℓ,h⊤/dh)n+i,jA_i,j ,h=softmax(Q ,hK ,h / d_h)_n+i,j. It controls the contribution of the value representation of xjx_j to the hidden representation of yiy_i. Aggregating such weights across output tokens, heads, and layers therefore produces token-level signals that indicate which input context spans influence the decision, i.e., the behavior-guiding instructions (Abnar and Zuidema 2020; Metzger et al. 2022). Recent methods consequently process or learn from attention to support token attribution (Cohen-Wang et al. 2025), context traceback (Wang et al. 2026a), and decision-provenance inspection (Wang et al. 2026b). Following this line, AttnLocate uses aggregated attention as decision provenance features, rather than directly interpreting individual weights as explanations, and learns to localize the context spans that guide the decision. 2.2 Defenses against Injection Attacks. Existing defenses against injection attacks can be organized into four broad categories. Static scanning prevents untrusted content from affecting model inference. Some methods detect or filter suspected injections before they reach the model (AI 2025; He et al. 2025; Wang et al. 2025d). Architectural defenses provide system-level containment. One line of work isolates untrusted components (Wu et al. 2025). Another separates trusted control flow from untrusted data flow (Debenedetti et al. 2025). A third decouples trusted planning from capability-constrained execution (Li et al. 2026). Behavior-auditing methods monitor agent actions and enforce runtime constraints such as contextual integrity, access control, or policy compliance (Jing et al. 2025; Wang et al. 2025a; Shi et al. 2026; Zhao et al. 2026). Although these defenses reduce exposure or block unsafe actions, they rarely identify the exact context span that was functionally interpreted as behavior-guiding instruction. Attribution-based methods are most closely related to our work, as they trace decisions back to influential inputs and provide reasoning-time evidence. ContextCite (Cohen-Wang et al. 2024) and TracLLM (Wang et al. 2025c) estimate influence through perturbation; AttnTrace (Wang et al. 2026a) improves efficiency via attention aggregation and context subsampling; MindGuard (Wang et al. 2026b) attributes tool calls to metadata fields using attention-derived dependency graphs. However, these methods operate on coarse-grained units (e.g., passages, documents, or metadata fields), which prevents precise identification of behavior-guiding instructions and consequently hampers subsequent adjustments and selective intervention. 3 Threat Model Figure 3: System design of AttnLocate. The Attention Aggregation and Instruction Localization jointly implement dynamic behavior-guiding instruction localization ℒ(⋅)L(·), while the Unauthorized Behavior Adjudication implements policy-based unauthorized behavior Adjudication (⋅)A(·). 3.1 Trust Model As shown in Figure 1, an LLM-based agent typically consists of three parts: User. Users are assumed to be trusted, issuing a query q that specifies the task to be completed by the agent. External Context Providers. External context providers include third-party resources (e.g., databases and document repositories), tools (e.g., MCP servers), APIs, and other services that incorporate external information into the agent’s reasoning context. These third-party providers are untrustworthy, since they operate outside the direct control of the user, thus constituting an injection surface that enables attackers to inject malicious commands. LLM Agent. The LLM agent determines and executes tool invocations based on the user query q and the external context C: y=F(q,C),y=F(q,C), (1) where C may comprise metadata injected by providers or execution results returned from prior operations. Following prior work (Wang et al. 2026b), the LLM agent is considered honest but vulnerable. Specifically, although the agent does not intentionally violate user goals or system-level constraints, it may misinterpret malicious content injected into the external context as behavior-guiding instructions, thereby causing the agent to perform unintended tool invocations. 3.2 Attack Model Attacker Goals. The attacker’s goal is to induce the agent to execute malicious attacker-expected behaviors by injecting malicious span m into the context used by LLM reasoning. The attack succeeds if m is incorrectly parsed by the model as a behavior-guiding instruction, which influences the model’s decision y, and ultimately causes the execution to deviate from the user’s intent. Attacker Capabilities. We consider a strong attacker with complete control over the external resources under their administration. The attacker may arbitrarily inject, remove, or modify content at any position in the context supplied by these resources, such as metadata and returned payloads. Attackers influences the agent exclusively through the untrusted external context, without directly modify the trusted components of the workflow, such as User and LLM Agnet. 3.3 AttnLocate Defense Defense Goals. As discussed above, injection arises when untrusted external data is interpreted by the LLM as a behavior-guiding instruction. Accordingly, AttnLocate pursues two complementary goals: 1. Dynamic Instruction Localization (ℒ(⋅)L(·)). AttnLocate monitors the agent’s inference process to dynamically identify external context spans that are truly interpreted as behavior-guiding instructions, rather than on whether it merely appeals malicious. ℒ(q∥C,y)=τ^t,L(q C,y)= τ_t, (2) where τ^t=m τ_t=m if the candidate instruction m truly influences the model’s decision, and τ^t=∅ τ_t= otherwise. 2. Unauthorized Behavior Adjudication ((⋅)A(·)). AttnLocate evaluates the identified instruction against configurable security policies to adjudicate unauthorized behavior. When an instruction originates from a context provider that lacks sufficient authority (according to a custom policy Π ), AttnLocate identifies the invocation as malicious: (τ^t,Π)=maliciousif τ^t≠∅∧Π(ρ(τ^t))<Ry,A( τ_t, )=malicious τ_t≠ \; \; (ρ( τ_t) )<R_y, (3) where ρ(τ^t)ρ( τ_t) denotes the provider to which τ^t τ_t belongs, and RyR_y is the minimum authority required to execute the current decision y. Core Idea. During the LLM’s generation process, the model attends to different parts of its input context. AttnLocate’s core idea is to identify precisely those spans that genuinely influence the model’s decision-making behavior, i.e., the behavior-guiding instructions. The attention mechanism, which reflects token-to-token dependencies, naturally provides a fine-grained, internal signal that indicates which context segments contribute to the construction of a decision. As illustrated in Figure 2, context items that impact the final invocation decision (such as the user query, the metadata of the invoked tool, and the injected instruction parsed as a behavior-guiding instruction) exhibit significantly high activations in the attention matrix. AttnLocate formally exploits this attention-activation property by reformulating the b-g instruction identification problem as a target detection task within the attention matrix (i.e., detecting regions of high activation). Defender Capabilities. AttnLocate requires white-box access to the underlying model to retrieve the attention matrices generated during inference, but it imposes no modification to the existing agent workflow and does not invoke any external LLMs. The primary deployment scenarios of AttnLocate are as follows: • Self-hosted agent systems (e.g., OpenHands deployment using a local Llama (Wang et al. 2025b)); • Security value-added services built into provider platforms (e.g., activation-probing guardrails used by Anthropic’s Fable5 (Anthropic 2026)). 4 AttnLocate Design As illustrated in Figure 3, AttnLocate consists of three part: Attention Aggregation converts the multi-layer, multi-head attention generated during inference into a token-level feature space tailored for object detection. Instruction Localization then applies a 1-D U-Net and an anchor-free detection head to identify the context span that the model interprets as a behavior-guiding instruction. Unauthorized Behavior Adjudication resolves the localized span’s provider and evaluates its authority under the configurable policy Π . 4.1 Attention Aggregation This module aggregates attention across heads and layers to derive token-level features that characterize the attention relationships between the current decision y and the input context. Such aggregation integrates attention signals across different heads and model depths, thereby providing a more robust representation for subsequent instruction localization. Head Aggregation. For layer l∈1,…,Ll∈\1,…,L\ and head h∈1,…,Hh∈\1,…,H\, let (l,h)A^(l,h) be the corresponding attention matrix. AttnLocate first average the attention weights within each head: A¯(l)=1H∑h=1H(l,h). A^(l)= 1H _h=1^HA^(l,h). (4) Gaussian Layer Weighting. After head averaging, different layers still encode complementary contextual signals. We therefore aggregate the layer-wise matrices using normalized Gaussian weights centered at the upper-middle model depth: A=∑l=1LwlA¯(l),wl=exp(−(l−μ)22σ2)∑r=1Lexp(−(r−μ)22σ2),A= _l=1^Lw_l\, A^(l), w_l= \! (- (l-μ)^22σ^2 ) _r=1^L \! (- (r-μ)^22σ^2 ), (5) Typically, AttnLocate uses μ≈2L3μ≈ 2L3 and σ=2σ=2 to give higher weight to task-relevant attention signals in the middle and upper layers (Wang et al. 2026b). Decision-Conditioned Slicing. AttnLocate then selects the attention matrix AyA_y only associated with the final decision y and the external context CexternalC_external: Ay=A[P(y),P(Cexternal)],A_y=A[P(y),P(C_external)], (6) where P(⋅)P(·) is the token position mapping function. Hence, the subsequent localization is solely conditioned on the contextual dependencies of y to support the CoT paradigm: restricting attention to the final decision-making call while leaving the intermediate reasoning tokens unaccounted for. Feature Extraction. For each input context position j, we derive a feature vector zjz_j from Ay[:,j]A_y[:,j] using its mean, maximum, standard deviation, and a learned attention-pooling operator. These statistics capture complementary dependence patterns, ranging from diffuse influence across the decision to sharp reliance on an individual context token. The resulting sequence Z=(z1,…,z|Cexternal|)Z=(z_1,…,z_|C_external|) preserves token-level feature for instruction localization. 4.2 Instruction Localization This module localizes behavior-guiding instructions from attention-derived features. AttnLocate formulate this problem as object detection task rather than independent token classification, thereby avoiding fragmented predictions and unstable boundaries. 1-D U-net Backbone. AttnLocate encode Z using a one-dimensional U-Net as backbone model: 1) The Encoder progressively doubles the channel width through convolutions with a stride of 2, while halving the temporal resolution, thereby capturing dependencies between distant context regions. The Decoder combines coarse features with encoder features through skip connections and restores token-level resolution for accurate boundary recovery. Anchor-free Detection. An anchor-free detection head adapted from FCOS (Tian et al. 2019) is then added to AttnLocate. At each position i, the head predicts an objectness logit oio_i, left and right boundary offsets (li,ri)(l_i,r_i) to the left and right span edges, and a centerness score cic_i to suppress off-center predictions. A sample-level head over pooled features additionally produces a scalar logit s for binary detection. At inference time, the sample-level probability determines whether the current decision depends on any behavior-guiding context span: i∗ i^* =argmaxiσ(oi)σ(ci), = _iσ(o_i)σ(c_i), (7) τ^t τ_t =∅,σ(s)<δ,[i∗−li∗,i∗+ri∗+1),otherwise. = cases ,&σ(s)<δ,\\ [i^*-l_i^*,\,i^*+r_i^*+1 ),&otherwise. cases where the decoded interval is clipped to [0,Tx)[0,T_x). Consequently, ℒ(q∥C,y)L(q C,y) returns a span only when that span is associated with the construction of y; the mere presence of instruction-like or malicious-looking text is insufficient. Sink-Aware Regularization. To mitigate the influence of noise such as attention sinking, we assign greater training weight to high-attention background positions. Let bi∈0,1b_i∈\0,1\ be the objectness label for context position i, and let a¯i a_i denote its attention averaged over the decision-token axis. We define the sink set as sink=i|bi=0,a¯i>βTx.S_sink= \\,i\; |\;b_i=0,\ a_i> βT_x \. (8) The relative threshold adapts to the context length and selects only background positions whose attention exceeds the uniform level by a factor of β. We then assign αi=αsink>1 _i= _sink>1 for i∈sinki _sink and αi=1 _i=1 otherwise. The weighted objectness object is obj=−∑iαi[bilogσ(oi)+(1−bi)log(1−σ(oi))].J_obj=- _i _i [b_i σ(o_i)+(1-b_i) \! (1-σ(o_i) ) ]. (9) Applying focal modulation to this objective yields focalJ_focal, which explicitly trains the model to distinguish attention-sink background tokens from genuine behavior-guiding spans. Training objective. We denote the training objective by J, combining three terms: =focal+λgiougiou+λclsbce,J=J_focal+ _giouJ_giou+ _clsJ_bce, (10) where giouJ_giou is the one-dimensional generalized IoU loss (Rezatofighi et al. 2019) over foreground positions, and bceJ_bce supervises the sample-level existence head. Dataset Model Localization Adjudication mIoU↑ Hit@0.5↑ Hit@0.7↑ AUROC↑ AP↑ FPR↓ TPR↑ MCPTox Qwen3-8B 0.720 0.909 0.727 0.949 0.891 0.059 0.910 Qwen3-8B† 0.774 0.927 0.830 0.988 0.976 0.056 0.927 Qwen3-14B 0.708 0.900 0.600 0.944 0.824 0.010 0.900 Qwen3-14B† 0.730 0.905 0.738 0.950 0.940 0.082 0.894 DeepSeek-R1-Qwen-14B† 0.748 0.910 0.785 0.975 0.951 0.060 0.918 DeepSeek-R1-Qwen3-8B 0.721 0.906 0.742 0.962 0.928 0.075 0.902 Phi-4† 0.858 0.938 0.914 0.944 0.979 0.087 0.938 LLaMA-2-7B 0.739 0.813 0.697 0.936 0.883 0.120 0.950 Mistral-7B 0.701 0.830 0.632 0.941 0.909 0.068 0.907 Gemma2-9B 0.692 0.848 0.661 0.951 0.937 0.092 0.908 InjecAgent DeepSeek-R1-Qwen-14B† 0.718 0.890 0.746 0.976 0.948 0.045 0.951 DeepSeek-R1-Qwen3-8B† 0.747 0.821 0.652 0.949 0.914 0.069 0.967 Phi-4† 0.792 0.858 0.711 0.966 0.936 0.053 0.967 LLaMA-2-7B 0.706 0.874 0.784 0.927 0.876 0.090 0.967 Mistral-7B 0.734 0.809 0.631 0.943 0.905 0.073 0.951 Gemma2-9B 0.801 1.000 0.858 0.989 0.971 0.029 0.984 Table 1: Overall performance of AttnLocate across all agents and datasets. † denotes think mode. Some model configurations are not included in InjecAgent, as they yield a zero attack success rate. 4.3 Unauthorized Behavior Adjudication The adjudication (τ^t,Π)A( τ_t, ) determines whether a localized instruction is unauthorized by comparing its provider’s authority with that required for the current decision. Provenance Resolution. At inference step t, AttnLocate derives a provenance map Γt _t from the agent context. Using existing structural delimiters and provenance annotations, Γt _t maps each token position to the provider of the corresponding context item. The resolver ρ(τ^t)ρ( τ_t) then returns the provider to which the localized span belongs. Because provenance is recovered from the runtime context, this process does not require context components to be manually isolated in advance. Authority Arbiter. The configurable policy Π assigns an authority level to each provider according to deployment- and task-specific requirements. By default, it instantiates the trust hierarchy defined in Section 3, assigning higher authority to the system and user channels than to external-context providers. Let RyR_y denote the minimum authority required to execute the current decision y. The arbiter implements the unauthorized behavior adjudication as (τ^t,Π)=malicious,τ^t≠∅∧Π(ρ(τ^t))<Ry,benign,otherwise.A( τ_t, )= casesmalicious,& τ_t≠ \ \ \! (ρ( τ_t) )<R_y,\\ benign,&otherwise. cases (11) This separation between dynamic instruction localization and unauthorized behavior adjudication is central to AttnLocate. The localization identifies what actually guides the current decision, while the adjudication determines whether the corresponding provider is authorized to exert that influence. Because neither stage relies on attack-specific lexical patterns, AttnLocate generalizes across indirect prompt injection and tool poisoning. Moreover, each alert includes the localized span, its resolved provider, and the violated authority requirement, making the defense decision auditable. 5 Evaluation 5.1 Experimental Setup Datasets. We evaluate tool poisoning attacks on MCPTox (Wang et al. 2026c), in which the malicious payload is embedded in tool metadata, and indirect prompt injection attacks on InjecAgent (Zhan et al. 2024), in which the malicious payload is embedded in the execution output of tools. LLM agents. We evaluate AttnLocate across ten agent configurations from the Qwen (Bai et al. 2023), DeepSeek (DeepSeek-AI et al. 2024), Phi (Abdin et al. 2024), LLaMA (Touvron et al. 2023), Mistral (Jiang et al. 2023) and Gemma (Team et al. 2024) families, covering both standard and thinking modes. Training and calibration. For the main evaluation, we train and test a separate AttnLocate for each model–dataset pair. Object detection employs a depth-4 1-D U-Net, with μ=2L/3μ=2L/3, σ=L/6σ=L/6, β=10β=10, and αsink=3 _sink=3. We set λcls=1 _cls=1 and λgiou=2 _giou=2. Baselines. We compare AttnLocate against three representative paradigms: Static scanning baselines include LLM-Guard (AI 2025) and LLM Detector (He et al. 2025); behavior-auditing approaches include MCIP (Jing et al. 2025); attribution-based monitors include MindGuard (Wang et al. 2026b) and TracLLM (Wang et al. 2025c). Metrics. Corresponding to the two defense goals of localization and adjudication introduced in §3.3, we evaluate the following two aspects of objectives: 1) Localization. For successful attacks with behavior-guiding span τt≠∅ _t≠ , we report mean intersection-over-union (mIoU) and Hit@θ, where θ∈0.5,0.7θ∈\0.5,0.7\. Hit@θ is the fraction of examples satisfying IoU(τ^t,τt)≥θIoU( τ_t, _t)≥θ. 2) Adjudication. An execution is positive (i.e., successful attack) if its behavior is guided by a behavior-guiding instruction whose provider lacks the authority required for y, i.e., τt≠∅ _t≠ and Π(ρ(τt))<Ry (ρ( _t))<R_y. We report AUROC curve and average precision (AP), together with the false-positive rate (FPR) and true-positive rate (TPR) for this task. 5.2 Main Results Table 1 reports the localization and adjudication performance of AttnLocate across different LLM agents. For localization, AttnLocate yields consistently strong span overlaps, with mIoU spanning from 0.6920.692 to 0.8580.858. AttnLocate exhibits notably superior performance in the think mode, improving by approximately 5.4%5.4\% over the baseline on the Qwen3 models. Given the localized spans, adjudication AUROC ranges between 0.9270.927 and 0.9890.989. On InjecAgent, all detectors surpass 0.950.95 TPR, and Gemma2-9B stands out with an AUROC of 0.9890.989 and a TPR of 0.9840.984. Paradigm Method Qwen3-8B† Qwen3-8B Phi-4† TPR↑ FPR↓ TPR↑ FPR↓ TPR↑ FPR↓ Static Scan LLM-Guard 0.513 0.374 0.547 0.369 0.520 0.411 LLM Detector 0.600 0.403 0.673 0.426 0.587 0.469 Behavior Audit MCIP 0.560 0.551 0.547 0.531 0.540 0.560 Attribution-based MindGuard 0.813 0.117 0.807 0.094 0.747 0.151 TracLLM 0.900 0.129 0.887 0.174 0.873 0.149 AttnLocate (Ours) 0.927 0.056 0.910 0.059 0.947 0.070 Table 2: Comparation with existing works. Best per column bolded, and second-best underlined. Table 2 compares the adjudication performance of different defense paradigms across three LLM settings. AttnLocate substantially outperforms evaluted baselines, achieving TPRs>0.9100.910 with FPRs<0.0700.070. Furthermore, it consistently exceeds the performance of existing attention-based source analysis baselines, ie.e, TracLLM and MindGuard, across all model backbones. On Phi-4†, it achieves the highest TPR (0.9470.947) and the lowest FPR (0.0700.070). 5.3 Analysis and Ablations Training Agent Target Agent AUROC↑ mIoU↑ Hit@0.5↑ Qwen3-8B† Qwen3-8B† 0.988 0.774 0.927 Qwen3-8B 0.867 0.797 0.909 Qwen3-14B 0.909 0.719 0.818 Qwen3-14B† 0.904 0.730 0.881 DeepSeek-R1-Qwen-14B† 0.921 0.736 0.902 Phi-4† 0.859 0.658 0.888 Table 3: Zero-shot cross-model transfer of AttnLocate on MCPTox. Training data collected from on Qwen3-8B† reasoning and transferred without retraining to various LLMs. Cross-model generalization. We train AttnLocate using the attention obtained from Qwen3-8B† and transfer it to an unseen agent without retraining. As shown in Table 3, Localization remains effective on the unseen models, it obtains mIoU values of 0.7360.736 and 0.6580.658 on DeepSeek-R1-Qwen-14B† and Phi-4†, respectively, while retaining Hit@0.5 near 0.900.90. Adjudication AUROCs are 0.8930.893 on average across unseen Qwen3 agents, 0.9210.921 on DeepSeek-R1-Qwen-14B†, and 0.8590.859 on Phi-4†. These results show that both localization and its downstream adjudication transfer across agent architectures. (a)mIOU performance. (b)TPR/FPR performance. Figure 4: Performance across different input context length: longer context length monotonically lower mIoU, TPR, and FPR. Input context length. Fig. 4 examines performance across context lengths. Both mIoU and TPR monotonically decrease with length, from 0.81050.8105/0.97220.9722 at ≤256≤ 256 to 0.69510.6951/0.85710.8571 at >2048>2048, while FPR concurrently declines from 0.07140.0714 to 0.01590.0159, indicating that longer inputs dilute salient signals. (a)Effect on mIOU. (b)Effect on TPR and FPR. Figure 5: Ablation of the Gaussian center μ: μ=2L/3μ=2L/3 yields the highest mIoU with high TPR and markedly lower FPR. Gaussian weighting center. As shown in Figure 5, weighting shallow layers inflates high adjudication FPR. Centering at μ=2L/3μ=2L/3 yields the highest mIoU of 0.7740.774 with high TPR and markedly lower FPR. Moving further toward the deeper layers lowers FPR but degrades both mIoU and TPR. We therefore use μ=2L/3μ=2L/3 as the default. (a)Effect on mIOU. (b)Effect on TPR and FPR. Figure 6: Ablation of sink-aware regularization: β=10β=10 gives best mIoU, high TPR, low FPR. Sink-aware regularization. Figure 6 ablates the sink-aware regularization weight β. Without sink suppression, AttnLocate retains a TPR of 0.9460.946 but incurs an FPR of 0.4170.417. Setting β=10β=10 suppresses these false activations, achieving the peak mIoU of 0.7740.774 with a TPR of 0.9270.927. Further increasing β to 1212 over-regularizes and degrades both metrics. We thus adopt β=10β=10 as default. Policy Authorized Sources TPR↑ FPR↓ Principal-only System/User instr. 0.949 0.086 Tool-authorized⋆ + Tool directives 0.927 0.056 Result-authorized + Result sources 0.909 0.048 Table 4: Authority-policy analysis. Each row adds the corresponding source to the authority set. Configurability of the authority policy. Varying the authority policy (Principal-only → Tool-authorized → Result-authorized) yields stable performance, with TPR above 0.9090.909 and FPR below 0.0860.086, demonstrating that AttnLocate can adapt to configurable authority policy. 6 conclusion We present AttnLocate, a runtime framework that localizes behavior-guiding instructions from attention patterns and adjudicates unauthorized invocation behaviors based on the origin of the localized spans. Extensive evaluations across ten agent configurations from six model families against tool poisoning and indirect prompt injection demonstrate AttnLocate’s strong localization and high adjudication performance, and it can transfer to unseen models and adapt to authority policy changes without retraining. Our findings highlight that localizing the exact behavior-guiding instructions enables fine-grained, provenance-aware policy enforcement, a capability absent in prior detection-based or coarse-grained attribution defenses. References Abdin et al. (2024) M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, J. R. Lee, Y. T. Lee, Y. Li, W. Liu, C. C. T. Mendes, A. Nguyen, E. Price, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, X. Wang, R. Ward, Y. Wu, D. Yu, C. Zhang, and Y. Zhang Phi-4 technical report. External Links: 2412.08905 Cited by: §5.1. Abnar and Zuidema (2020) S. Abnar and W. Zuidema Quantifying attention flow in transformers. External Links: 2005.00928 Cited by: §2.1. AI (2025) P. AI LLM guard: the security toolkit for llm interactions. Note: https://github.com/protectai/llm-guard Cited by: §1, §2.2, §5.1. Anthropic (2026) Anthropic Claude fable 5 & claude mythos 5 system card. System Card Anthropic. External Links: Link Cited by: 2nd item. Bai et al. (2023) J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y. Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y. Zhang, Z. Zhang, C. Zhou, J. Zhou, X. Zhou, and T. Zhu Qwen technical report. External Links: 2309.16609 Cited by: §5.1. Beurer-Kellner and Fischer (2025) L. Beurer-Kellner and M. Fischer MCP security notification: tool poisoning attacks. Invariant Labs. Note: https://invariantlabs.ai/blog/mcp-security-notification-tool-poisoning-attacks Cited by: §1. Chen et al. (2025) S. Chen, J. Piet, C. Sitawarin, and D. A. Wagner StruQ: defending against prompt injection with structured queries. In 34th USENIX Security Symposium, USENIX Security 2025, Seattle, WA, USA, August 13-15, 2025, p. 2383–2400. Cited by: §1. Cohen-Wang et al. (2025) B. Cohen-Wang, Y. Chuang, and A. Madry Learning to attribute with attention. External Links: 2504.13752 Cited by: §2.1. Cohen-Wang et al. (2024) B. Cohen-Wang, H. Shah, K. Georgiev, and A. Madry ContextCite: attributing model generation to context. External Links: 2409.00729 Cited by: §2.2. Debenedetti et al. (2025) E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tramèr Defeating prompt injections by design. CoRR abs/2503.18813. External Links: Link, Document, 2503.18813 Cited by: §1, §2.2. DeepSeek-AI et al. (2024) DeepSeek-AI, :, X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu, H. Gao, K. Gao, W. Gao, R. Ge, K. Guan, D. Guo, J. Guo, G. Hao, Z. Hao, Y. He, W. Hu, P. Huang, E. Li, G. Li, J. Li, Y. Li, Y. K. Li, W. Liang, F. Lin, A. X. Liu, B. Liu, W. Liu, X. Liu, X. Liu, Y. Liu, H. Lu, S. Lu, F. Luo, S. Ma, X. Nie, T. Pei, Y. Piao, J. Qiu, H. Qu, T. Ren, Z. Ren, C. Ruan, Z. Sha, Z. Shao, J. Song, X. Su, J. Sun, Y. Sun, M. Tang, B. Wang, P. Wang, S. Wang, Y. Wang, Y. Wang, T. Wu, Y. Wu, X. Xie, Z. Xie, Z. Xie, Y. Xiong, H. Xu, R. X. Xu, Y. Xu, D. Yang, Y. You, S. Yu, X. Yu, B. Zhang, H. Zhang, L. Zhang, L. Zhang, M. Zhang, M. Zhang, W. Zhang, Y. Zhang, C. Zhao, Y. Zhao, S. Zhou, S. Zhou, Q. Zhu, and Y. Zou DeepSeek llm: scaling open-source language models with longtermism. External Links: 2401.02954 Cited by: §5.1. He et al. (2025) P. He, C. Li, B. Zhao, T. Du, and S. Ji Automatic red teaming llm-based agents with model context protocol tools. External Links: 2509.21011 Cited by: §1, §2.2, §5.1. Jiang et al. (2023) A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed Mistral 7b. External Links: 2310.06825 Cited by: §5.1. Jing et al. (2025) H. Jing, H. Li, W. Hu, Q. Hu, X. Heli, T. Chu, P. Hu, and Y. Song MCIP: protecting MCP safety via model contextual integrity protocol. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 1177–1194. Cited by: §1, §2.2, §5.1. Li et al. (2026) E. Li, T. Mallick, E. Rose, W. K. Robertson, A. Oprea, and C. Nita-Rotaru ACE: A security architecture for llm-integrated app systems. In 33rd Annual Network and Distributed System Security Symposium, NDSS 2026, San Diego, California, USA, February 23-27, 2026, Cited by: §1, §2.2. Metzger et al. (2022) N. Metzger, C. Hahn, J. Siber, F. Schmitt, and B. Finkbeiner Attention flows for general transformers. External Links: 2205.15389 Cited by: §2.1. OWASP Foundation (2025) OWASP Foundation MCP Tool Poisoning. Note: https://owasp.org/w-community/attacks/MCP_Tool_Poisoning Cited by: §1. OWASP GenAI Security Project (2025) OWASP GenAI Security Project LLM01:2025 Prompt Injection. Note: https://genai.owasp.org/llmrisk/llm01-prompt-injection/ Cited by: §1, §1. Rezatofighi et al. (2019) H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese Generalized intersection over union: a metric and a loss for bounding box regression. External Links: 1902.09630 Cited by: §4.2. Shi et al. (2026) T. Shi, J. He, Z. Wang, H. Li, L. Wu, W. Guo, and D. Song Progent: securing ai agents with privilege control. External Links: 2504.11703, Link Cited by: §2.2. Team et al. (2024) G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivière, M. S. Kale, J. Love, P. Tafti, L. Hussenot, P. G. Sessa, A. Chowdhery, A. Roberts, A. Barua, A. Botev, A. Castro-Ros, A. Slone, A. Héliou, A. Tacchetti, A. Bulanova, A. Paterson, B. Tsai, B. Shahriari, C. L. Lan, C. A. Choquette-Choo, C. Crepy, D. Cer, D. Ippolito, D. Reid, E. Buchatskaya, E. Ni, E. Noland, G. Yan, G. Tucker, G. Muraru, G. Rozhdestvenskiy, H. Michalewski, I. Tenney, I. Grishchenko, J. Austin, J. Keeling, J. Labanowski, J. Lespiau, J. Stanway, J. Brennan, J. Chen, J. Ferret, J. Chiu, J. Mao-Jones, K. Lee, K. Yu, K. Millican, L. L. Sjoesund, L. Lee, L. Dixon, M. Reid, M. Mikuła, M. Wirth, M. Sharman, N. Chinaev, N. Thain, O. Bachem, O. Chang, O. Wahltinez, P. Bailey, P. Michel, P. Yotov, R. Chaabouni, R. Comanescu, R. Jana, R. Anil, R. McIlroy, R. Liu, R. Mullins, S. L. Smith, S. Borgeaud, S. Girgin, S. Douglas, S. Pandya, S. Shakeri, S. De, T. Klimenko, T. Hennigan, V. Feinberg, W. Stokowiec, Y. Chen, Z. Ahmed, Z. Gong, T. Warkentin, L. Peran, M. Giang, C. Farabet, O. Vinyals, J. Dean, K. Kavukcuoglu, D. Hassabis, Z. Ghahramani, D. Eck, J. Barral, F. Pereira, E. Collins, A. Joulin, N. Fiedel, E. Senter, A. Andreev, and K. Kenealy Gemma: open models based on gemini research and technology. External Links: 2403.08295 Cited by: §5.1. Tian et al. (2019) Z. Tian, C. Shen, H. Chen, and T. He FCOS: fully convolutional one-stage object detection. External Links: 1904.01355 Cited by: §4.2. Touvron et al. (2023) H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample LLaMA: open and efficient foundation language models. External Links: 2302.13971 Cited by: §5.1. Vaswani et al. (2023) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin Attention is all you need. External Links: 1706.03762 Cited by: §2.1. Wang et al. (2025a) H. Wang, C. M. Poskitt, and J. Sun AgentSpec: customizable runtime enforcement for safe and reliable LLM agents. CoRR abs/2503.18666. External Links: Link, Document, 2503.18666 Cited by: §2.2. Wang et al. (2025b) X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y. Shao, N. Muennighoff, Y. Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig OpenHands: an open platform for ai software developers as generalist agents. External Links: 2407.16741 Cited by: 1st item. Wang et al. (2026a) Y. Wang, R. Geng, Y. Chen, and J. Jia AttnTrace: contextual attribution of prompt injection and knowledge corruption. In IEEE Symposium on Security and Privacy, SP 2026, San Francisco, CA, USA, May 18-21, 2026, p. 4320–4338. Cited by: §1, §2.1, §2.2. Wang et al. (2025c) Y. Wang, W. Zou, R. Geng, and J. Jia TracLLM: A generic framework for attributing long context llms. In 34th USENIX Security Symposium, USENIX Security 2025, Seattle, WA, USA, August 13-15, 2025, p. 3845–3864. Cited by: §1, §2.2, §5.1. Wang et al. (2025d) Y. Wang, S. Chen, R. Alkhudair, B. Alomair, and D. A. Wagner Defending against prompt injection with datafilter. CoRR abs/2510.19207. External Links: Link, Document, 2510.19207 Cited by: §2.2. Wang et al. (2026b) Z. Wang, H. Du, G. Shi, J. Zhang, H. Cheng, Y. Yao, K. Guo, and X. Li MindGuard: intrinsic decision inspection for securing llm agents against metadata poisoning. External Links: 2508.20412, Link Cited by: §1, §2.1, §2.2, §3.1, §4.1, §5.1. Wang et al. (2026c) Z. Wang, Y. Gao, Y. Wang, S. Liu, H. Sun, H. Cheng, G. Shi, H. Du, and X. Li MCPTox: A benchmark for tool poisoning on real-world MCP servers. In Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, p. 35811–35819. Cited by: §1, §5.1. Wu et al. (2025) Y. Wu, F. Roesner, T. Kohno, N. Zhang, and U. Iqbal IsolateGPT: an execution isolation architecture for llm-based agentic systems. In 32nd Annual Network and Distributed System Security Symposium, NDSS 2025, San Diego, California, USA, February 24-28, 2025, Cited by: §1, §2.2. Zhan et al. (2024) Q. Zhan, Z. Liang, Z. Ying, and D. Kang InjecAgent: benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Findings of ACL, Vol. ACL 2024, p. 10471–10506. Cited by: §1, §5.1. Zhao et al. (2023) P. Zhao, Z. Jin, and N. Cheng An in-depth survey of large language model-based artificial intelligence agents. External Links: 2309.14365, Link Cited by: §1. Zhao et al. (2026) W. Zhao, Z. Li, P. Zhang, and J. Sun ClawGuard: A runtime security framework for tool-augmented LLM agents against indirect prompt injection. CoRR abs/2604.11790. External Links: Link, Document, 2604.11790 Cited by: §2.2.