Paper deep dive
ASCon: A Direction-Aware Reciprocal Agent--Step Contextualization Model for Failure Attribution in Multi-Agent Systems
Shuyu Jiang, Yue Ran, Kaiyu Xu, Xingshu Chen, Yi Zhang, Hao Ren, Rui Tang, Tianwei Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/13/2026, 3:34:08 AM
Summary
The paper introduces ASCon, a direction-aware reciprocal Agent-Step Contextualization model for failure attribution in LLM-based multi-agent systems. ASCon unifies the identification of faulty agents, erroneous steps, and failure modes by aggregating trajectory evidence into unified agent and step representations using direction-aware graph attention and masked step-to-agent attention. It demonstrates significant improvements in detection accuracy over existing methods.
Entities (9)
Relation Signals (7)
ASCon → uses → Direction-Aware Graph Attention
confidence 95% · ASCon introduces direction-aware graph attention to model execution context
ASCon → uses → Masked Step-to-Agent Attention
confidence 95% · ASCon introduces... masked step-to-agent attention to construct behavior-aware agent representations
ASCon → improves → Faulty-Agent Detection
confidence 90% · Experiments show that ASCon can improve faulty-agent detection by 5.83"+ in micro-accuracy
ASCon → improves → Faulty-Step Detection
confidence 90% · Experiments show that ASCon can improve... faulty-step detection by 10.63"+ in micro-accuracy
ASCon → improves → Failure-Mode Detection
confidence 90% · Experiments show that ASCon can improve... failure-mode detection by 14.73"+ in Macro-F1.
ASCon → evaluatedon → TracerTraj
confidence 85% · We evaluate ASCon on TracerTraj and Aegis-Bench
ASCon → evaluatedon → Aegis-Bench
confidence 85% · We evaluate ASCon on TracerTraj and Aegis-Bench
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Failure attribution in LLM-based multi-agent systems (MAS) aims to answer who caused failures, when they occurred, and why by identifying responsible targets including faulty agents, erroneous steps, and failure modes. Existing methods have primarily focused on developing dedicated models for specific attribution targets, with limited attention to the evidential dependencies among them. Despite these attribution targets are different, they rely on common diagnostic evidence from MAS trajectories, including task constraints, agent roles, behavioral histories and inter-agent interactions. This commonality motivates us to develop a unified representation model that aggregates the trajectory evidence into individual agent and step representations, which can subsequently be adapted to different attribution targets. Accordingly, we propose ASCon, a direction-aware reciprocal \textbf{A}gent--\textbf{S}tep \textbf{Con}textualization model for multiple failure attribution targets. ASCon introduces direction-aware graph attention to model execution context, masked step-to-agent attention to construct behavior-aware agent representations, and agent-conditioned step contextualization to incorporate agent context back into step representations. The resulting contextualized representations enable different attribution targets through lightweight target-specific heads. Experiments show that ASCon can improve faulty-agent detection by 5.83\%+ in micro-accuracy, faulty-step detection by 10.63\%+ in micro-accuracy, and failure-mode detection by 14.73\%+ in Macro-F1. Meanwhile, it can also substantially enhance the LLM-based methods' attribution capabilities in out-of-domain scenarios.
Tags
Links
- Source: https://arxiv.org/abs/2608.10646v1
- Canonical: https://arxiv.org/abs/2608.10646v1
Trouble viewing inline? Open PDF directly →
Full Text
69,815 characters extracted from source content.
Expand or collapse full text
ASCon: A Direction-Aware Reciprocal Agent–Step Contextualization Model for Failure Attribution in Multi-Agent Systems Shuyu Jiang1,2, Yue Ran1, Kaiyu Xu1, Xingshu Chen 1, Yi Zhang1, Hao Ren 1, Rui Tang1, Tianwei Zhang2 1Sichuan University, China 2Nanyang Technological University, Singapore Abstract Failure attribution in LLM-based multi-agent systems (MAS) aims to answer who caused failures, when they occurred, and why by identifying responsible targets including faulty agents, erroneous steps, and failure modes. Existing methods have primarily focused on developing dedicated models for specific attribution targets, with limited attention to the evidential dependencies among them. Despite these attribution targets are different, they rely on common diagnostic evidence from MAS trajectories, including task constraints, agent roles, behavioral histories and inter-agent interactions. This commonality motivates us to develop a unified representation model that aggregates the trajectory evidence into individual agent and step representations, which can subsequently be adapted to different attribution targets. Accordingly, we propose ASCon, a direction-aware reciprocal Agent–Step Contextualization model for multiple failure attribution targets. ASCon introduces direction-aware graph attention to model execution context, masked step-to-agent attention to construct behavior-aware agent representations, and agent-conditioned step contextualization to incorporate agent context back into step representations. The resulting contextualized representations enable different attribution targets through lightweight target-specific heads. Experiments show that ASCon can improve faulty-agent detection by 5.83%+ in micro-accuracy, faulty-step detection by 10.63%+ in micro-accuracy, and failure-mode detection by 14.73%+ in Macro-F1. Meanwhile, it can also substantially enhance the LLM-based methods’ attribution capabilities in out-of-domain scenarios. Code is available at: https://github.com/Shuyu-07/ASCon Introduction Large language model (LLM)-based multi-agent systems (MAS) have emerged as a promising paradigm for complex problem solving by coordinating agents with specialized roles and tool-use capabilities across tasks such as software development, information retrieval, and scientific analysis (Fourney et al., 2024; Ghareeb et al., 2026; Qian et al., 2024). Meanwhile, their growing capabilities are also accompanied by more complex failure modes like inter-agent misalignment, making system debugging and error correction increasingly difficult (Qi et al., 2026; Yu et al., 2025). Diagnosing such failures requires developers to inspect lengthy traces with domain knowledge, often taking several to tens of minutes per instance (Cemri et al., 2025; Epperson et al., 2025; Zhang et al., 2025). This substantial diagnostic burden made automated failure attribution increasingly important for scalable MAS debugging, maintenance, and subsequent optimizations (Ma et al., 2026; Wang et al., 2026a; In et al., 2026). Figure 1: (a) MAS failure attribution are commonly formulated at separate agent, step, and failure-mode levels. (b) These targets share the same trajectory evidence, which ASCon models for unified attribution across all three levels. As shown in Figure 1, MAS failure attribution refers to the identification of key failure factors from an execution trace, including: who – the agents responsible for the failure; when – the steps at which the failure occurs or first manifests; and why – the corresponding failure modes or error modes (Liu et al., 2026). Existing studies typically develop separate methods for different attribution targets. For step-level attribution, failures are usually located through LLM-based search and ranking, or by modeling anomalous temporal changes in behavioral sequences (Zhang et al., 2025; Zhu et al., 2026a; Sun et al., 2026). Agent-level methods instead determine responsibility by examining whether an agent’s behavioral history is consistent with its intended goal or assigned role (Geng et al., 2026; Wu et al., 2026). Failure-mode attribution methods commonly abstract these anomalous behaviors into predefined error categories through LLM inference (Kong et al., 2026). Although agent, step, and failure-mode attribution are often formulated separately in these valuable studies, they draw on closely related evidence from the same execution trajectory. For example, whether a candidate step is faulty is determined from itself as well as its preceding and subsequent execution actions, the task constraints, and the role of the executing agent (Qiao et al., 2026; Chen et al., 2026). Likewise, attributing responsibility to an agent requires aggregating evidence from its historical actions and examining whether they remain consistent with its role and the task constraints (Ge et al., 2026). On this basis, failure-mode identification further characterizes how the observed anomalous behavior deviates from the exception (Kong et al., 2026). Therefore, task constraints, agent role settings, behavioral history, and inter-agent interactions jointly provide the contextual evidence for different failure attribution targets. Motivated by this observation, we propose ASCon, a direction-aware reciprocal agent-step contextualization model for multi-agent failure attribution, which unifies step-, agent-, and failure-mode attribution within a single framework. ASCon first introduces a direction-aware graph attention to separately model the preceding and subsequent execution contexts. It then uses masked step-to-agent attention to aggregate the steps performed by each agent into a behavior-aware agent representation, which is further contextualized over an agent interaction graph to capture cross-agent collaboration. Finally, the agent representations are injected back into their corresponding step representations, forming a reciprocal contextualization process between steps and agents. Through this reciprocal information flow, ASCon integrates task constraints, behavioral context, and collaboration patterns into agent and step representations, thereby enabling flexible adaptation to different supervision schemes through lightweight prediction heads. We evaluate ASCon on TracerTraj and Aegis-Bench, which together cover agent-level, step-level, and failure-mode attribution. Experimental results show that ASCon achieves improvements of 5.83%+ in agent-level micro-accuracy, 10.63%+ in step-level micro-accuracy, and 14.73%+ in failure-mode macro-F1. Moreover, when incorporated into LLM-based attribution methods, ASCon improves out-of-domain agent-level micro-accuracy by 8.69%+, step-level micro-accuracy by 12.50%+ and failure-mode-level macro-F1 by 3.15%+. Our main contributions are as follows: • We propose ASCon, a reciprocal agent-step contextualization framework for multi-agent failure attribution, which supports agent-, step-, and failure-mode attribution under heterogeneous supervision settings. • We introduce direction-aware graph attention for modeling preceding and subsequent step dependencies, and masked step-to-agent attention for selectively aggregating agent-specific behavioral evidence. • Comprehensive experiments indicate that ASCon consistently improves performance across attribution targets and effectively enhances out-of-domain generalization. Related Works MAS failure attribution aims to identify responsible agents, erroneous steps, or failure modes from execution trajectories (Zhang et al., 2025; Chen et al., 2026; Kong et al., 2026). Existing methods can be broadly divided into prompt-based and learning-based failure attributions. Prompt-based methods. Prompt-based methods directly prompt LLMs to inspect execution trajectories and localize suspicious behaviors without training a dedicated attribution model. Specifically, Who&When studies All-at-Once, step-by-step, and binary-search strategies (Zhang et al., 2025); SDBL narrows the candidate scope before localization (Sun et al., 2026); and RAFFLES iteratively refines fault hypotheses through judge-evaluator reasoning (Zhu et al., 2026a). Other studies incorporate counterfactual analysis or causal structures to guide the reasoning process (West et al., 2025; Wang et al., 2026b; Ge et al., 2026). While these methods offer flexible trajectory analysis, their attribution quality is largely determined by the reasoning capability of the underlying LLM, especially for long and structurally complex traces. Learning-based methods. Learning-based methods construct task-specific supervision and learn representations of failed trajectories. For instance, AgentTracer generates annotated traces through counterfactual replay and programmed fault injection, and trains a dedicated failure tracer (Zhang et al., 2026). Aegis introduces context-aware error generation and annotations of faulty agents and error modes, supporting supervised fine-tuning and reinforcement learning (Kong et al., 2026). StepFinder models trajectories as temporal semantic sequences for step-level localization (Zhu et al., 2026b), and reduces reliance on inference-time reasoning by learning attribution patterns directly from execution data. Figure 2: The framework of our ASCon model. Problem Formulation An LLM-based multi-agent system consists of a set of agents =a1,a2,…,aNaA=\a_1,a_2,…,a_N_a\ that collaborate to complete a task. Its execution is recorded as a trajectory τ=s1,s2,…,sNsτ=\s_1,s_2,…,s_N_s\, where each step is represented as st=(t,ϕ(t),ct)s_t=(t,φ(t),c_t). Here, t is the step index, ϕ(t)∈1,…,Naφ(t)∈\1,…,N_a\ identifies the agent executing the step, and ctc_t denotes its textual execution record, such as an agent message, intermediate reasoning, tool invocation, or tool feedback. Given a failed trajectory τ, MAS failure attribution aims to determine who is responsible for the failure, when the failure occurs, and why it occurs. Let =m1,m2,…,mMY=\m_1,m_2,…,m_M\ denote the set of failure modes. The ground-truth attribution is represented as (τ)=(ai∗,i∗,i∗)i=1KG(τ)= \ (a_i^*,S_i^*,Y_i^* ) \_i=1^K, where ai∗∈a_i^* is a faulty agent, i∗⊆τS_i^* τ contains its faulty steps, and i∗⊆Y_i^* contains the corresponding failure modes. The objective is to learn an attribution function fθf_θ that maps the failed trajectory to a structured prediction: fθ:τ↦^(τ)≈(τ)f_θ:τ G(τ) (τ). Accordingly, the predicted agents, steps, and failure modes constitute the who, when, and why of MAS failures. Methodology As illustrated in Figure 2, ASCon learns mutually contextualized representations of steps and agents from an MAS execution trajectory. It first introduces a direction-aware graph attention mechanism that separately captures preceding and succeeding context in both step execution and agent interaction structures. Then it contextualizes individual steps along the execution trajectory, aggregates their behavioral evidence to construct agent representations, and conditions each step on the representation of its executing agent. Finally, target-specific prediction layers map the resulting agent and agent-conditioned step representations to the required failure-attribution outputs. Direction-aware Graph Attention To capture the evolving dependencies in MAS execution, where earlier actions influence subsequent decisions and agent behaviors are coordinated through interactions, ASCon models each trajectory as a directed step graph and a directed agent graph. For the step graph, the user query is prepended as step s0s_0 to provide task constraints for evaluating subsequent behaviors, yielding the augmented execution sequence ~=(s0,s1,…,sNs). S=(s_0,s_1,…,s_N_s). Each step in ~ S is treated as a graph node, forming the node set V(s)=s0,s1,…,sNs.V^(s)=\s_0,s_1,…,s_N_s\. Adjacent steps in the execution sequence are connected according to their temporal order, forming the directed step graph G(s)=(V(s),ℰ(s))G^(s)=(V^(s),E^(s)), where ℰ(s)=(st,st+1)∣0≤t<NsE^(s)=\(s_t,s_t+1) 0≤ t<N_s\. Each directed edge represents a direct execution dependency between consecutive actions, preserving the temporal flow of the trajectory. For the agent graph, ASCon introduces a user node a0a_0 corresponding to s0s_0 with ϕ(0)=0φ(0)=0. The augmented agent node set is defined as V(a)=~=a0,a1,…,aNaV^(a)= A=\a_0,a_1,…,a_N_a\. Agents responsible for adjacent steps are connected following the same execution direction, yielding the directed agent graph G(a)=(V(a),ℰ(a))G^(a)=(V^(a),E^(a)), where ℰ(a)=(aϕ(t),aϕ(t+1))∣0≤t<NsE^(a)=\(a_φ(t),a_φ(t+1)) 0≤ t<N_s\. In failure attribution, the preceding and succeeding node neighbors in G(s)G^(s) and G(a)G^(a) provide asymmetric evidence: the former provide the context for assessing whether a node is anomalous, whereas the latter reveal its potential downstream effects. Treating both directions identically may therefore blur the distinction between a root failure and its consequences. Thus, ASCon introduces a direction-aware graph attention network (DGAT) that separately models incoming, outgoing, and self information. Let G=(,ℰ)G=(V,E) uniformly denote the constructed G(s)G^(s) or G(a)G^(a), and iu_i denote the input feature of node viv_i. Its incoming and outgoing neighborhoods are defined as in(i)=vj∣(vj,vi)∈ℰN_in(i)=\v_j (v_j,v_i) \, and out(i)=vj∣(vi,vj)∈ℰN_out(i)=\v_j (v_i,v_j) \. At the l-th layer, DGAT first computes attention scores for incoming and outgoing edges separately as follows: eji,inl e_ji,in^l =LeakyReLU(inl⊤[inlil;inljl]), =LeakyReLU (a_in^l [W_in^lu_i^l;W_in^lu_j^l ] ), (1) eij,outl e_ij,out^l =LeakyReLU(outl⊤[outlil;outljl]). =LeakyReLU (a_out^l [W_out^lu_i^l;W_out^lu_j^l ] ). inlW_in^l and outlW_out^l are learnable direction-specific transformation matrices. inla_in^l and outla_out^l are learnable attention vectors. The attention scores are normalized separately along the incoming and outgoing directions, and the direction-specific messages im_i are aggregated by taking an attention-weighted sum of the corresponding neighborhoods as follows: i,inl _i,in^l =∑vj∈in(i)αji,inlinljl, = _v_j _in(i) _ji,in^lW_in^lu_j^l, (2) i,outl _i,out^l =∑vj∈out(i)αij,outloutljl. = _v_j _out(i) _ij,out^lW_out^lu_j^l. αji,inl _ji,in^l =softmaxj∈in(i)(eji,inl), =softmax_j _in(i)(e_ji,in^l), (3) αij,outl _ij,out^l =softmaxj∈out(i)(eij,outl). =softmax_j _out(i)(e_ij,out^l). For the next-layer, the node representation is updated by combining self, incoming, and outgoing information: ~il+1=ELU(selflil+i,inl+i,outl), u_i^l+1=ELU (W_self^lu_i^l+m_i,in^l+m_i,out^l ), (4) where selflW_self^l is the self-transformation matrix, and ELU(⋅)ELU(·) is the nonlinear activation function. Contextualized Step Representation Since the correctness of steps depends not only on its own content but also on its consistency with task constraints and surrounding actions, ASCon learns contextualized step representations over the directed step graph G(s)G^(s), whose edges record inter-step execution dependencies. Each node in ~ S is first encoded into a semantic embedding t(s)=fLM(ct)e_t^(s)=f_LM(c_t), where fLMf_LM denotes a pretrained text encoder and ctc_t is the step content. Then, ASCon applies LsL_s DGAT layers to G(s)G^(s) and its step embeddings (s)=[0(s);1(s);…;Ns(s)]X^(s)=[e_0^(s);e_1^(s);…;e_N_s^(s)] to get the contextualized step representation (s)H^(s) as follows: (s)=DGATLs(s)((s),G(s)),H^(s)=DGAT_L_s^(s) (X^(s),G^(s) ), (5) where t(s)h_t^(s) is the contextualized representation of step sts_t. Agent-conditioned step representation. Beyond the task constraints and surrounding actions encoded above, the correctness of a step also depends on its consistency with the setting of the executing agent. Therefore, ASCon constructs an agent-conditioned step representation by incorporating the behavioral state of the responsible agent. For step sts_t, let aϕ(t)a_φ(t) be the agent associated with this step. The behavior-aware agent representation ϕ(t)(a)h_φ(t)^(a), computed in the next subsection, is projected into the step representation space: t(a)=a2sϕ(t)(a).r_t^(a)=W_a2sh_φ(t)^(a). (6) The agent-conditioned step representation is computed as t(a−s)=LayerNorm(MLP([t(s);t(s)⊙t(a)])),h_t^(a-s)=LayerNorm (MLP ( [h_t^(s);h_t^(s) _t^(a) ] ) ), (7) where ⊙ denotes element-wise multiplication. Method Code Math Agentic Micro-Accuracy Macro-Accuracy Agent Step Agent Step Agent Step Agent Step Agent Step DeepSeek-V4-Pro 33.07 11.81 53.97 25.40 58.83 37.83 54.30 32.66 48.62 25.01 Gemini-3.1-Flash 74.80 24.41 50.79 42.86 60.50 21.00 62.03 23.29 62.03 29.42 All-at-Once 66.14 7.87 34.92 30.16 61.00 22.17 59.75 20.51 54.02 20.07 Step-by-Step 65.35 10.24 57.14 36.51 69.67 23.50 67.98 22.41 64.05 23.42 Binary Search 48.82 2.36 42.86 28.57 53.00 20.50 51.52 18.23 48.23 17.14 SDBL (EASD) 70.08 7.09 44.44 38.10 64.83 35.67 64.05 31.27 59.78 26.95 Qwen-SFT 44.09 5.51 49.21 22.22 61.33 38.00 57.59 31.52 51.54 21.91 AgentTracer 83.46 7.09 36.51 42.86 61.00 23.50 62.66 22.41 60.32 24.48 StepFinder 81.89 41.73 79.37 74.60 73.33 52.67 75.18 52.66 78.20 56.33 ASCon 85.04 52.76 79.37 74.60 80.33 64.33 81.01 63.29 81.58 63.90 Table 1: Agent- and step-level attribution accuracy (%) on the root-fault attribution task. Best results are in bold, with best baselines underlined. Columns 2∼ 7 report detailed sub-test set results and the last four columns report overall average accuracies. Behavior-aware Agent Representation After step-level encoding, ASCon lifts behavioral evidence from steps to agents. Since an agent is characterized by its executed steps and different steps contribute unequally to fault diagnosis, we introduce masked step-to-agent attention to restrict aggregation to the agent’s own steps and emphasize the most informative ones. Masked step-to-agent attention. To incorporate the task constraint into the agent aggregation, we introduce a user agent a0a_0 associated only with step s0s_0. The augmented agent node set is ~=a0,a1,…,aNa A=\a_0,a_1,…,a_N_a\. Let k(a)e_k^(a) denote the role embedding of agent aka_k, where 0≤k≤Na0≤ k≤ N_a. We define an agent-step assignment matrix ∈0,1(Na+1)×(Ns+1)M∈\0,1\^(N_a+1)×(N_s+1), where Mkt=1M_kt=1 if step sts_t is associated with agent aka_k, and Mkt=0M_kt=0 otherwise. For each agent aka_k, ASCon first computes a local behavioral summary from its associated steps: ¯k(s)=∑t=0NsMktt(s)∑t=0NsMkt. h_k^(s)= _t=0^N_sM_kth_t^(s) _t=0^N_sM_kt. (8) This summary provides an agent-specific context for constructing the pooling query. Then, the query for step-to-agent attention aka_k is generated as k=q[k(a);¯k(s)]q_k=W_q[e_k^(a); h_k^(s)], while each step representation is projected into a key vector t=Kt(s)k_t=W_Kh_t^(s). The agent-step relevance is then measured by scaled dot-product attention as Akt=k⊤t/dqA_kt=q_k k_t/ d_q. The assignment mask further restricts each agent only attention to its own steps as follows: βkt=softmaxt(A~kt),A~kt=Akt,Mkt=1,−∞,Mkt=0, _kt=softmax_t( A_kt),\\ A_kt= casesA_kt,&M_kt=1,\\ -∞,&M_kt=0, cases (9) where βkt _kt is the attention score. The behavior-aware representation of agent k is then obtained by aggregating its step-level evidence: k(a)=∑t=0Nsβktt(s)g_k^(a)= _t=0^N_s _kth_t^(s). Agent graph contextualization. To complement agent-specific evidence with the overall task context, ASCon further computes a global trajectory representation (s)g^(s) by mean-pooling all step representations. The initial agent representations are formed by combining agent role, behavior-aware evidence, and global trajectory context: k(a)=LayerNorm(MLP([k(a);k(a);(s)])).z_k^(a)=LayerNorm (MLP ([e_k^(a);g_k^(a);g^(s)] ) ). (10) Similarly, ASCon applies LaL_a DGAT layers on the agent graph G(a)G^(a) to learn the final agent representation (a)H^(a), as follows: (a)=DGATLa(a)((a),G(a)),H^(a)=DGAT_L_a^(a) (Z^(a),G^(a) ), (11) where (a)=[0(a);1(a);…;Na(a)]Z^(a)=[z_0^(a);z_1^(a);…;z_N_a^(a)], and k(a)∈(a)h_k^(a) ^(a) serves as a contextual anchor for the failure attribution task. Target-specific Failure Attribution The contextualized step and agent representations learned above provide a common basis for identifying faulty agents (who), faulty steps (when), and failure modes (why). Specifically, agent representations support faulty agent and failure mode identification, agent-conditioned step representations support faulty step localization. Generally, existing MAS failure-attribution studies primarily instantiate these targets in two representative forms: (a) root-fault attribution, which identifies one root responsible agent and one key failure step, and (b) failure-mode attribution, which identifies faulty agents and their corresponding failure modes. Accordingly, ASCon employs the following two prediction heads for these two settings. Root-fault attribution. For root-fault attribution, ASCon independently predicts whether each agent and step is faulty using two binary classification heads. The fault probabilities are computed as y^k(a)=σ(MLPa(k(a))) y_k^(a)=σ(MLP_a(h_k^(a))) and y^t(s)=σ(MLPs(t(a−s))) y_t^(s)=σ(MLP_s(h_t^(a-s))), where σ(⋅)σ(·) denotes the sigmoid function. Let ^(a) y^(a) and ^(s) y^(s) collect the predicted probabilities of all agents and steps, with (a)y^(a) and (s)y^(s) denoting their binary labels. The training objective is ℒr=∑q∈a,sBCE(^(q),(q)).L_r=Σ _q∈\a,s\BCE ( y^(q),y^(q) ). (12) During inference, the highest-probability agent and step are selected as the attribution result. Failure-mode attribution. As the relevant diagnostic evidence of an agent’s failure mode is distributed across its steps, ASCon combines each contextualized agent representation with associated steps. For agent aka_k, the step-level evidence is pooled as ¯k(a−s)=∑t=1NsMktt(a−s)/∑t=1NsMkt h_k^(a-s)= _t=1^N_sM_kth_t^(a-s)/ _t=1^N_sM_kt and fused with its agent representation: k(a)=LayerNorm(MLPf([k(a);¯k(a−s)])).o_k^(a)=LayerNorm(MLP_f([h_k^(a); h_k^(a-s)])). (13) ASCon then predicts the faulty-agent probability y^k(a)=σ(MLPa(k(a))) y_k^(a)=σ(MLP_a(o_k^(a))) and failure-mode distribution ^k(m)=softmax(MLPm(k(a))) b_k^(m)=softmax(MLP_m(o_k^(a))). Let ^(a) y^(a) denote the faulty-agent probabilities and (a)y^(a) binary labels, while ^ℱ(m) B_F^(m) and ℱ(m)b_F^(m) denote the predicted distributions and labels of the faulty agents in set ℱ=k∣yk(a)=1F=\k y_k^(a)=1\. The training objective is ℒm=BCE(^(a),(a))+CE(^ℱ(m),ℱ(m)).L_m=BCE ( y^(a),y^(a) )+CE ( B_F^(m),b_F^(m) ). (14) Experiments and Analysis Experimental Settings Datasets. For root-fault and failure-mode attributions, we select TracerTraj (Zhang et al., 2026) and Aegis-Bench (Kong et al., 2026) as the corresponding evaluation benchmarks. TracerTraj contains trajectories generated by six MASs across coding, mathematical reasoning, and general agentic tasks. We use its extended version as the benchmark, which comprises 3,208 training and 790 test trajectories (127 code, 63 math, and 600 agentic test instances). Aegis-Bench contains over 9K trajectories collected from six MASs and covers 14 failure modes, with each trajectory potentially involving multiple faulty agents and failure modes. Following its default setup, we use 600 trajectories for testing and split the remainder 8:2 into training and validation sets. Baselines. We compare ASCon with several advanced prompt-based and learning-based failure attribution methods. For root-fault attribution, the prompt-based baselines include All-at-Once, Step-by-Step, and Binary Search (Zhang et al., 2025), as well as SDBL (EASD) (Sun et al., 2026), all instantiated with GPT-4o-mini following their original settings. We additionally evaluate DeepSeek-V4-Pro and Gemini-3.1-Flash-lite with the All-at-Once prompt. The learning-based baselines include StepFinder (Zhu et al., 2026b), AgentTracer (Zhang et al., 2026), and Qwen-SFT, fine-tuned from Qwen2.5-14B. For failure-mode attribution where related baselines remain limited, we evaluate the same commercial LLMs using the prompt template from Aegis (Kong et al., 2026), and include Aegis-SFT (fine-tuned with LoRA), as the learning-based baseline. All prompt templates used in our evaluation are provided in Appendix A. Metrics. Following prior studies (Kong et al., 2026; Zhu et al., 2026b), we evaluate root-fault attribution using agent- and step-level accuracies, which measures whether the top-ranked prediction matches the ground-truth. For failure-mode attribution, we report Micro-F1(μ 1) and Macro-F1(MF1). μ 1 reflects overall performance across all instances, whereas MF1 computes the F1-score for each class independently and then takes the unweighted average. Parameters. We set LsL_s and LaL_a to 2, the step and agent representation dimensions to 256, the attention dimension to 128, and the dropout rate to 0.1. Qwen3-Embedding-0.6B is used to encode steps and agent roles. All models are trained for 12 epochs with AdamW, using a learning rate of 5×10−55× 10^-5 and a weight decay of 10−510^-5. The ASCon model is implemented in PyTorch and runs on an Intel Core Ultra 7 265KF CPU with an NVIDIA GeForce RTX 5060 GPU. Main Results Performance on root-fault attribution. Table 1 shows that ASCon achieves the best overall performance, with macro-accuracies of 81.58% for faulty agent attribution and 63.90% for faulty step attribution. Compared with StepFinder, the strongest baseline, ASCon improves the two metrics by 3.38% and 7.57%, respectively. Across different task domains, ASCon achieves the best results on code and agentic tasks and matches StepFinder on math, with particularly large step-level gains of 11.03% on code and 11.66% on agentic tasks. The consistently lower step-level scores than agent-level scores indicate that locating the exact faulty action is more challenging than identifying the responsible agent. Across the baselines, the agent-step macro-accuracy gap ranges from 21.87% to 40.63%, whereas ASCon reduces it to 17.68%. This is mostly because ASCon incorporates direction-aware step context and agent-level behavioral context into each step representation, thereby providing stronger evidence for fine-grained fault localization. Model Pair Agent Error μ 1 MF1 μ 1 MF1 μ 1 MF1 DeepSeek-V4-Pro 10.70 4.70 73.40 46.78 22.04 16.19 Gemini-3.1-Flash 12.67 5.97 71.76 43.65 23.34 17.11 GPT-4o-mini 9.86 3.14 65.28 27.72 20.81 16.32 Qwen3.5-Flash 11.49 4.70 67.69 40.50 23.61 16.35 Aegis-SFT 17.85 9.89 86.58 68.21 26.08 22.96 Our Method 31.80 20.98 86.48 71.31 38.27 37.69 Table 2: Results for the failure-mode attribution task on the Aegis-Bench. Performance on failure-mode attribution We evaluate failure-mode attribution at three levels: Agent, which identifies faulty agents while ignoring failure modes; Error, which identifies failure modes while ignoring agents; and Pair, which requires the correct agent-error association. Table 2 shows that ASCon achieves the best overall performance at all level attributions. Compared with Aegis-SFT, the strongest baseline, ASCon improves pair-level μ 1 and MF1 by 13.95% and 11.09%, respectively, while increasing error-level μ 1 and MF1 by 12.19% and 14.73%. At the agent level, ASCon improves MF1 by 3.10% while maintaining comparable μ 1, with a difference of only 0.10%. These results indicate that ASCon is more effective at jointly identifying faulty agents and their associated error types, while the consistent MF1 gains suggest improved performance across less frequent classes rather than only on dominant categories. Root-fault attribution Model Agent μ Agent MAcc Step μ Step Acc Lightweight models (<10<10M) StepFinder 42.93 42.52 19.02 15.28 ASCon 45.65 46.36 20.10 17.94 LLM-based Models (≥ 8B) DeepSeek-V4-Pro 58.70 55.42 22.28 17.20 +ASCon 67.39 66.89 43.48 40.59 Gemini-3.1-Flash 53.80 49.99 22.83 18.06 +ASCon 60.87 57.94 36.41 31.24 SDBL (EASD) 57.07 56.09 30.98 27.27 +ASCon 57.07 60.28 34.24 31.05 Qwen-SFT 50.00 49.07 27.17 25.42 +ASCon 53.80 52.78 32.07 29.93 AgentTracer 50.00 43.95 26.63 21.77 +ASCon 55.43 51.64 30.43 26.41 Failure-mode attribution Model Agent μ 1 Agent MF1 Error μ 1 Error MF1 ASCon 44.76 26.43 7.37 5.95 Deepseek-V4-pro 48.10 29.60 12.64 7.57 +ASCon 51.14 38.23 13.82 7.76 Gemini-3.1-Flash 50.63 34.47 14.13 8.09 +ASCon 53.23 35.08 15.61 11.21 GPT-4o-mini 49.34 31.61 6.99 5.16 +ASCon 53.90 38.74 10.33 8.71 Qwen3.5-Flash 46.04 28.97 9.22 6.46 +ASCon 52.52 31.85 11.89 9.34 Aegis-SFT 31.78 16.48 5.39 2.85 +ASCon 42.73 27.09 9.28 6.43 Table 3: Out-of-domain evaluation results on Who&When dataset.“+ASCon” denotes the use of ASCon predictions as auxiliary attribution evidence. μ and MAcc represent micro-accuracy and macro-accuracy, respectively. Setting Root-Fault Attribution Failure-Mode Attribution Step Agent Agent Error Pair μ MAcc μ MAcc μ 1 MF1 μ 1 MF1 μ 1 MF1 ASCon 66.20+0.00 64.50+0.00 82.41+0.00 86.60+0.00 86.48+0.00 71.31+0.00 38.27+0.00 37.69+0.00 31.80+0.00 20.98+0.00 DGAT→ 61.14-5.06 59.81-4.69 79.37-3.04 82.24-4.36 83.10-3.38 66.08-5.23 33.67-4.60 32.49-5.20 26.06-5.74 18.28-2.70 DGAT→ 64.94-1.26 58.57-5.93 81.27-1.14 84.42-2.18 82.65-3.84 62.17-9.14 32.70-5.57 31.98-5.71 24.90-6.90 15.87-5.11 w/o MSTAA 62.03-4.17 61.68-2.82 76.84-5.57 82.87-3.74 85.13-1.35 68.88-2.43 36.94-1.33 35.70-1.99 29.12-2.68 18.54-2.44 MSTAA→ 61.39-4.81 64.17-0.33 80.76-1.65 85.67-0.93 85.98-0.50 68.49-2.82 34.79-3.48 34.38-3.31 27.82-3.98 17.89-3.09 w/o ACSR 61.14-5.06 62.62-1.88 82.03-0.38 86.92+0.32 85.69-0.79 62.58-8.73 36.73-1.54 35.50-2.19 29.16-2.64 18.66-2.32 Table 4: Ablation results of ASCon. Subscripts denote changes relative to the full model. Underlined values indicate the largest degradation for each metric. Generalization to Out-of-Domain Scenarios To evaluate out-of-domain generalization, we test ASCon on Who&When benchmark without further training. Beyond default baselines, we incorporate ASCon predictions into several strong LLM-based attribution methods to examine whether its fault evidence can complement their reasoning under distribution shift. Specifically, for SDBL, we replace its original fault-range prediction with the candidates ranked highest by ASCon, selecting the top-3 agents and the top-5/10 steps for the Algorithm-Generated/Handcrafted subset, while retaining its subsequent reasoning process. For other LLM-based baselines, ASCon predictions are appended to the input. Root-fault attribution uses step and agent fault probabilities, while failure-mode attribution uses agent-level fault probabilities and top-five candidate fault types. Detailed formats, settings, and results are provided in Appendix B. As shown in Table 3, ASCon outperforms StepFinder on all agent- and step-level metrics, indicating stronger out-of-domain transfer among lightweight attribution models. Some LLM-based methods achieve higher performance, which is expected given their substantially larger model capacity and broad pretrained knowledge. Nevertheless, incorporating ASCon predictions further improves these methods on all granularity metrics. These results show that ASCon captures transferable fine-grained fault evidence that effectively complements LLM-based reasoning in unseen scenarios. Ablations To assess the contribution of each component in ASCon, we construct the following five variants. DGAT→ and DGAT→ replace DGAT with a standard GAT and a bidirectional GRU, respectively, to evaluate the necessity of direction-aware relational modeling. w/o MSTAA and MSTAA→ remove masked step-to-agent attention and replace its learned weighting with mean pooling over agent-owned steps, respectively, to examine the importance of selective behavioral aggregation. Finally, w/o ACSR removes the agent-conditioned step representation to assess the contribution of agent context to step-level attribution. As shown in Table 4, all variants exhibit overall performance degradation, confirming that the three components provide complementary attribution evidence. Replacing DGAT consistently reduces performance, while BiGRU causes the largest degradation on most failure-mode metrics, including a 9.14% drop in Agent MF1 and a 6.90% drop in Pair MF1. This suggests that sequential encoding alone is insufficient, and that explicitly modeling directional relations among steps provides more discriminative structural evidence. The degradation of the MSTAA variants shows that learned step-to-agent attention is more effective than simple averaging, as different steps contribute unequally and irrelevant or propagated behaviors should be suppressed. Removing ACSR substantially reduces root-fault Step micro-accuracy by 5.06% and consistently harms Error and Pair attribution, showing that conditioning individual steps on their responsible agents improves fine-grained agent–error association. Overall, the ablation results support the complementary roles of directional step modeling, selective step-to-agent aggregation, and agent-conditioned step refinement. Can ASCon Capture Potential Dependencies? Figure 3: Performance comparison between vanilla graphs and LLM-inferred graphs. To assess whether ASCon can implicitly learn useful inter-step and inter-agent dependencies from the vanilla graph, we replace its original edges with dependencies inferred by Deepseek-V4-pro with the prompt in Appendeix A.7. As shown in Figure 3, the LLM-inferred graph provides no consistent advantage. In root-fault attribution, it improves code agent accuracy by 3.15% and yields marginal gains on agentic tasks, but reduces step accuracy by 6.30% on code and 6.35% on math. In failure-mode attribution, the two graph constructions perform comparably, with the LLM-inferred graph only slightly lower. These results suggest that ASCon can already capture attribution-relevant dependencies through supervised representation learning and directional graph aggregation. In contrast, LLM-inferred edges may introduce noisy or overly broad connections, which are particularly harmful to fine-grained step localization and failure-mode discrimination. Therefore, the simpler vanilla graph in ASCon is both sufficient and more reliable. Conclusion This paper presented ASCon, a direction-aware reciprocal agent-step contextualization model for MAS failure attribution. By jointly modeling directional action context, agent behavioral histories, and inter-agent interactions, ASCon learns contextualized representations that support faulty-agent, faulty-step, and failure-mode attribution. Experiments demonstrate ASCon made consistent improvements over strong baselines, and can also enhance existing attribution methods in out-of-domain scenarios. These findings highlight ASCon can learn effective agent and step contextualization representations for fine-grained and transferable MAS failure diagnosis. References M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran, M. A. Zaharia, J. E. Gonzalez, and I. Stoica (2025) Why do multi-agent LLM systems fail?. In Advances in Neural Information Processing Systems, Vol. 38. External Links: Link Cited by: Introduction. M. Chen, J. Wang, F. Mu, Y. Wang, Z. Liu, H. Feng, and Q. Wang (2026) Seeing the whole elephant: a benchmark for failure attribution in llm-based multi-agent systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 19888–19905. Cited by: Introduction, Related Works. W. Epperson, G. Bansal, V. C. Dibia, A. Fourney, J. Gerrits, E. Zhu, and S. Amershi (2025) Interactive debugging and steering of multi-agent ai systems. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, p. 1–15. Cited by: Introduction. A. Fourney, G. Bansal, H. Mozannar, C. Tan, E. Salinas, E. Zhu, F. Niedtner, G. Proebsting, G. Bassman, J. Gerrits, J. Alber, P. Chang, R. Loynd, R. West, V. Dibia, A. Awadallah, E. Kamar, R. Hosn, and S. Amershi (2024) Magentic-one: a generalist multi-agent system for solving complex tasks. External Links: 2411.04468, Link Cited by: Introduction. Y. Ge, L. Xie, Z. Li, Y. Pei, and T. Zhang (2026) Spectrum-based failure attribution for multi-agent systems. In Proceedings of the ACM on Software Engineering, Vol. 3, p. 1888–1909. Cited by: Introduction, Prompt-based methods.. M. Geng, S. Gu, Z. Liu, C. Xu, Z. Qu, and H. Wang (2026) Failure localization in multi-agent code generation via knowledge-guided and transferable reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 318–326. Cited by: Introduction. A. E. Ghareeb, B. Chang, L. Mitchener, A. Yiu, C. J. Szostkiewicz, D. Shved, G. J. Gyimesi, J. M. Laurent, S. M. Wright, M. T. Razzak, et al. (2026) A multi-agent system for automating scientific discovery. Nature, p. 1–3. Cited by: Introduction. Y. In, M. Tanjim, J. Subramanian, S. Kim, U. Bhattacharya, W. Kim, S. Park, S. Sarkhel, and C. Park (2026) Rethinking failure attribution in multi-agent systems: a multi-perspective benchmark and evaluation. In Workshop on Failure Modes of Agentic AI at ICML 2026, Cited by: Introduction. F. Kong, R. Zhang, H. Yin, G. Zhang, X. Zhang, Z. Chen, Z. Zhang, X. Zhang, S. Zhu, and X. Feng (2026) Aegis: automated error generation and attribution for multi-agent systems. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Appendix B, Introduction, Introduction, Learning-based methods., Related Works, Datasets., Baselines., Metrics.. J. Liu, H. Xi, S. Zhang, Y. Zeng, T. Yue, C. Wang, J. Kang, Q. Wu, and H. Wang (2026) Who&When pro: can llms really attribute failures in ai agents?. External Links: 2607.09996 Cited by: Introduction. M. Ma, J. Zhang, F. Yang, Y. Kang, Q. Lin, S. Rajmohan, and D. Zhang (2026) DoVer: intervention-driven auto debugging for LLM multi-agent systems. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Introduction. S. Qi, J. Ma, R. Xing, W. Guo, X. Huang, Z. Gao, J. Deng, J. Liu, L. Zhang, B. Wei, B. Yang, P. Wang, J. Sun, J. Tao, Y. Wu, H. Liu, Y. Yao, and T. Liu (2026) Beyond individual intelligence: surveying collaboration, failure attribution, and self-evolution in LLM-based multi-agent systems. External Links: 2605.14892, Link Cited by: Introduction. C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, J. Xu, D. Li, Z. Liu, and M. Sun (2024) ChatDev: communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 15174–15186. External Links: Document, Link Cited by: Introduction. H. Qiao, H. Tong, E. Lim, B. Liu, and G. Pang (2026) VerifyMAS: hypothesis verification for failure attribution in LLM multi-agent systems. External Links: 2605.17467, Link Cited by: Introduction. K. Sun, W. Li, B. Dong, Y. Lin, J. Zhang, and B. Shi (2026) Scope delineation before localization: a two-stage framework for enhancing failure attribution in multi-agent systems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 33108–33116. Cited by: Introduction, Prompt-based methods., Baselines.. X. Wang, M. Yin, E. Koh, and M. D. Dogan (2026a) XAgen: an explainability tool for identifying and correcting failures in multi-agent workflows. In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems, p. 1–11. Cited by: Introduction. Y. Wang, W. Wu, J. Wang, and Q. Wang (2026b) From flat logs to causal graphs: hierarchical failure attribution for LLM-based multi-agent systems. External Links: 2602.23701, Link Cited by: Prompt-based methods.. A. West, Y. Weng, M. Zhu, Z. Lin, and Y. Zhang (2025) Abduct, act, predict: scaffolding causal inference for automated failure attribution in multi-agent systems. External Links: 2509.10401, Link Cited by: Prompt-based methods.. H. Wu, W. Zheng, M. Zhang, X. Bai, M. Pu, L. Sun, J. Liu, and Y. Xu (2026) Intention-behavior consistency-based automated failure attribution for llm-driven multi-agent systems. Expert Systems with Applications, p. 133044. Cited by: Introduction. M. Yu, S. Wang, G. Zhang, J. Mao, C. Yin, Q. Liu, K. Wang, Q. Wen, and Y. Wang (2025) NetSafe: exploring the topological safety of multi-agent system. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 2905–2938. External Links: Link, Document, ISBN 979-8-89176-256-5 Cited by: Introduction. G. Zhang, J. Wang, J. Chen, W. Zhou, K. Wang, and S. YAN (2026) AgenTracer: who is inducing failure in the LLM agentic systems?. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Learning-based methods., Datasets., Baselines.. S. Zhang, M. Yin, J. Zhang, J. Liu, Z. Han, J. Zhang, B. Li, C. Wang, H. Wang, Y. Chen, et al. (2025) Which agent causes task failures and when? on automated failure attribution of llm multi-agent systems. In International Conference on Machine Learning, p. 76583–76599. Cited by: Appendix A, Appendix B, Introduction, Introduction, Prompt-based methods., Related Works, Baselines.. C. Zhu, S. Hong, J. Wu, K. Chawla, Y. Tang, Y. Yin, N. Wolfe, E. Babinsky, and D. Liu (2026a) RAFFLES: reasoning-based attribution of faults for LLM systems. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7659–7688. External Links: Document, Link Cited by: Introduction, Prompt-based methods.. T. Zhu, Y. Wu, W. Jin, Y. Li, and G. Huang (2026b) StepFinder: a temporal semantic framework for failure attribution in multi-agent systems. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, External Links: Document, Link Cited by: Learning-based methods., Baselines., Metrics.. Appendix Appendix A A. Prompt Templates Used in This Paper A.1 All-at-Once Attribution Prompt Template for Root-Fault Attribution This prompt template comes frome the All-at-Once baseline (Zhang et al., 2025), and is also used by Qwen-SFT and the LLM-based baselines for root-fault attribution. You are an AI assistant tasked with analyzing a multi-agent conversation history when solving a real-world problem. The problem is: problem. Identify which agent made an error, at which step, and explain the reason for the error. Here is the conversation: chat_content Based on this conversation, please predict the following: 1. The name of the agent who made a mistake and should be directly responsible for the incorrect solution to the real-world problem. If no agent makes an obvious mistake, select the single most likely responsible agent. Directly output the name of the agent. 2. The step at which the responsible agent first made a mistake. For example, consider the following conversation: “agent a”: “x”, “agent b”: “x”, “agent c”: “x”, “agent a”: “x”.\``agent a′: ``x′, ``agent b′: ``x′, ``agent c′: ``x′, ``agent a′: ``x′\. Each entry represents one step in which an agent provides an output. If the mistake occurs in agent c’s output, the step number is 2. If it occurs in the second output of agent a, the step number is 3. 3. The reason for your prediction. Please answer in valid JSON format as follows: "agent_name": "Your predicted root-cause faulty agent name", "step_number": 1, "reason_for_mistake": "Your reason" A.2 Prompt of AgentTracer You are AgentTracer, a failure attribution model for LLM-based agentic systems. Given a failed multi-agent trajectory, identify the earliest decisive error: the first step whose correction would be sufficient to avoid the final failure. Return exactly the following format: <think>brief evidence-based reasoning</think> <answer>agent_name|step_number</answer> The agent_name must match one of the agents in the trajectory, and step_number must be an integer step ID. A.3 Prompt of ASCon-Enhanced SDBL Method You are an AI assistant tasked with analyzing a multi-agent conversation history when solving a real-world problem. The problem is: problem. Identify which agent made an error, at which step, and explain the reason for the error. Here is the conversation: chat_content The following agents and steps are flagged for special attention. These are the candidates identified by our automated procedure as the most likely fault locations. Please focus on these high-confidence candidates and prioritize them when locating the root cause: reference_content Based on this conversation, please predict the following: 1. The name of the agent directly responsible for the incorrect solution. If no agent makes an obvious mistake, select the single most likely responsible agent and output its name. 2. The step at which the responsible agent first made a mistake. For example, in the conversation “agent a”: “x”, “agent b”: “x”, “agent c”: “x”, “agent a”: “x”,\``agent a′: ``x′, ``agent b′: ``x′, ``agent c′: ``x′, ``agent a′: ``x′\, each entry represents one step. If the mistake occurs in agent c’s output, the step number is 2. If it occurs in the second output of agent a, the step number is 3. 3. The reason for your prediction. Please answer in valid JSON format as follows: "agent_name": "Your predicted root-cause faulty agent name", "step_number": 1, "reason_for_mistake": "Your reason" A.4 Prompt of ASCon-Enhanced LLM-Based Attribution Methods You are an AI assistant tasked with analyzing a multi-agent conversation history when solving a real-world problem. The problem is: problem. Identify which agent made an error, at which step, and explain the reason for the error. Here is the conversation: chat_content Based on this conversation, please predict the following: 1. Each step is associated with an auxiliary fault probability. First examine the high-probability steps and their neighboring context to determine whether they contain the root-cause mistake. 2. The name of the agent directly responsible for the incorrect solution. If no agent makes an obvious mistake, select the single most likely responsible agent and output its name. 3. The step at which the responsible agent first made a mistake. For example, in the conversation “agent a”: “x”, “agent b”: “x”, “agent c”: “x”, “agent a”: “x”,\``agent a′: ``x′, ``agent b′: ``x′, ``agent c′: ``x′, ``agent a′: ``x′\, each entry represents one step. If the mistake occurs in agent c’s output, the step number is 2. If it occurs in the second output of agent a, the step number is 3. 4. The reason for your prediction. Please answer in valid JSON format as follows: "agent_name": "Your predicted root-cause faulty agent name", "step_number": 1, "reason_for_mistake": "Your reason" A.5 Prompt Template for Failure-Mode Attribution ## ROLE AND GOAL You are a meticulous Multi-Agent System (MAS) Quality Assurance analyst. Your sole purpose is to analyze conversation logs to identify and categorize agent errors based on a strict set of definitions. ## ERROR DEFINITIONS WITH EXAMPLES You MUST use the exact error codes provided below. ### Functional Mistakes (FM-1.x - Task Execution Errors): - FM-1.1: **Task specification deviation** - Agent deviates from specified task requirements (e.g., was asked to write code in Python, but used JavaScript). - FM-1.2: **Role specification deviation** - Agent acts outside its designated role (e.g., a ’CodeWriter’ agent starts criticizing other agents’ work, which is the ’Critic’s’ role). - FM-1.3: **Add redundant steps** - Agent adds unnecessary or duplicate steps (e.g., imports a library that was already imported in a previous step). - FM-1.4: **Remove conversation history** - Agent ignores or removes important context from previous turns (e.g., ignores a user’s correction from the previous message). - FM-1.5: **Remove termination conditions** - Agent fails to define proper stopping criteria, leading to loops or unfinished tasks (e.g., writes a recursive function with no base case). ### Functional Mistakes (FM-2.x - Communication & Coordination Errors): - FM-2.1: **Repeat handled tasks** - Agent redundantly handles already completed tasks (e.g., re-writes a piece of code that was already finalized and approved). - FM-2.2: **Make request ambiguous** - Agent provides unclear or confusing instructions to other agents (e.g., asks another agent to ”handle the data” without specifying how). - FM-2.3: **Deviate from main goal** - Agent pursues objectives unrelated to the main task (e.g., starts discussing the history of programming languages in the middle of a coding task). - FM-2.4: **Hide important information** - Agent withholds crucial information needed by other agents (e.g., knows a library has a bug but doesn’t mention it). - FM-2.5: **Ignore other agents** - Agent fails to consider input, corrections, or questions from other agents. - FM-2.6: **Inconsistent reasoning** - Agent’s logic contradicts its own previous statements (e.g., in step 2 agent says ’option A is best’, but in step 4 says ’option A is a bad choice’ without new information). ### Functional Mistakes (FM-3.x - Quality & Verification Errors): - FM-3.1: **Premature termination** - Agent stops or declares the task complete before all requirements are met. - FM-3.2: **Remove verification steps** - Agent skips necessary validation or testing steps (e.g., writes code but doesn’t write any unit tests for it). - FM-3.3: **Incorrect verification** - Agent performs flawed or wrong verification (e.g., writes a test that doesn’t actually check for the correct condition). ## ANALYSIS WORKFLOW Please follow these steps carefully: ### Step 1: Agent Summary First, analyze and summarize what each agent has done throughout the conversation: - List each agent that appears in the conversation - For each agent, summarize their main actions, decisions, and contributions - Note any patterns or recurring behaviors ### Step 2: Error Analysis For each agent identified in Step 1: - Carefully examine their actions against each error definition - Look for violations of task requirements, role boundaries, communication issues, or quality problems - Note any potential errors with specific reasoning ### Step 3: Final Judgment Based on your analysis in Steps 1 and 2: - Determine which agents (if any) committed errors - Assign the appropriate error code(s) to each faulty agent - Ensure agent names match exactly as they appear in the conversation log ## REQUIRED OUTPUT FORMAT Your response must contain: 1. **Agent Summary**: A brief analysis of what each agent did 2. **Error Analysis**: Your reasoning for identifying errors 3. **Final Answer**: A valid JSON object with your conclusions **JSON Format:** ”faulty_agents”: [”agent_name”: ”X”, ”error_type”: ”FM-X.X”] **Examples:** - Multiple Errors: ”faulty_agents”: [”agent_name”: ”X1”, ”error_type”: ”FM-1.1”, ”agent_name”: ”X2”, ”error_type”: ”FM-3.2”, ”agent_name”: ”X3”, ”error_type”: ”FM-2.5”] - No Errors: ”faulty_agents”: [] **Important:** Make sure the agent names you output exactly match those in the conversation log. Do not fabricate names. ## CONVERSATION TO ANALYZE: conversation_text ## YOUR ANALYSIS: A.6 Prompt Template for ASCon-enhaced Failure-Mode Attribution ## ROLE AND GOAL You are a meticulous Multi-Agent System (MAS) Quality Assurance analyst. Your sole purpose is to analyze conversation logs to identify and categorize agent errors based on a strict set of definitions. ## ERROR DEFINITIONS WITH EXAMPLES You MUST use the exact error codes provided below. ### Functional Mistakes (FM-1.x - Task Execution Errors): - FM-1.1: **Task specification deviation** - Agent deviates from specified task requirements (e.g., was asked to write code in Python, but used JavaScript). - FM-1.2: **Role specification deviation** - Agent acts outside its designated role (e.g., a ’CodeWriter’ agent starts criticizing other agents’ work, which is the ’Critic’s’ role). - FM-1.3: **Add redundant steps** - Agent adds unnecessary or duplicate steps (e.g., imports a library that was already imported in a previous step). - FM-1.4: **Remove conversation history** - Agent ignores or removes important context from previous turns (e.g., ignores a user’s correction from the previous message). - FM-1.5: **Remove termination conditions** - Agent fails to define proper stopping criteria, leading to loops or unfinished tasks (e.g., writes a recursive function with no base case). ### Functional Mistakes (FM-2.x - Communication & Coordination Errors): - FM-2.1: **Repeat handled tasks** - Agent redundantly handles already completed tasks (e.g., re-writes a piece of code that was already finalized and approved). - FM-2.2: **Make request ambiguous** - Agent provides unclear or confusing instructions to other agents (e.g., asks another agent to ”handle the data” without specifying how). - FM-2.3: **Deviate from main goal** - Agent pursues objectives unrelated to the main task (e.g., starts discussing the history of programming languages in the middle of a coding task). - FM-2.4: **Hide important information** - Agent withholds crucial information needed by other agents (e.g., knows a library has a bug but doesn’t mention it). - FM-2.5: **Ignore other agents** - Agent fails to consider input, corrections, or questions from other agents. - FM-2.6: **Inconsistent reasoning** - Agent’s logic contradicts its own previous statements (e.g., in step 2 agent says ’option A is best’, but in step 4 says ’option A is a bad choice’ without new information). ### Functional Mistakes (FM-3.x - Quality & Verification Errors): - FM-3.1: **Premature termination** - Agent stops or declares the task complete before all requirements are met. - FM-3.2: **Remove verification steps** - Agent skips necessary validation or testing steps (e.g., writes code but doesn’t write any unit tests for it). - FM-3.3: **Incorrect verification** - Agent performs flawed or wrong verification (e.g., writes a test that doesn’t actually check for the correct condition). ## ANALYSIS WORKFLOW Please follow these steps carefully: ### Step 1: Agent Summary First, analyze and summarize what each agent has done throughout the conversation: - List each agent that appears in the conversation - For each agent, summarize their main actions, decisions, and contributions - Note any patterns or recurring behaviors ### Step 2: Error Analysis For each agent identified in Step 1: - Carefully examine their actions against each error definition - Look for violations of task requirements, role boundaries, communication issues, or quality problems - Note any potential errors with specific reasoning ### Step 3: Final Judgment Based on your analysis in Steps 1 and 2: - Determine which agents (if any) committed errors - Assign the appropriate error code(s) to each faulty agent - Ensure agent names match exactly as they appear in the conversation log ### AUXILIARY AGENT FAULT PROFILE Each conversation will be followed by an auxiliary fault profile for every agent, including: - auxiliary fault probability: the probability that the agent is faulty in this task; - top 5 likely fault types: the five most likely fault types predicted for that agent. Please use this auxiliary information to assist your judgment of faulty agents and fault types. ## REQUIRED OUTPUT FORMAT Your response must contain: 1. **Agent Summary**: A brief analysis of what each agent did 2. **Error Analysis**: Your reasoning for identifying errors 3. **Final Answer**: A valid JSON object with your conclusions **JSON Format:** ”faulty_agents”: [”agent_name”: ”X”, ”error_type”: ”FM-X.X”] **Examples:** - Multiple Errors: ”faulty_agents”: [”agent_name”: ”X1”, ”error_type”: ”FM-1.1”, ”agent_name”: ”X2”, ”error_type”: ”FM-3.2”, ”agent_name”: ”X3”, ”error_type”: ”FM-2.5”] - No Errors: ”faulty_agents”: [] **Important:** Make sure the agent names you output exactly match those in the conversation log. Do not fabricate names. ## CONVERSATION TO ANALYZE: conversation_text ## YOUR ANALYSIS: A.7 Prompt of LLM-Based Agent/Step Graph Inference You are an expert in Multi-Agent System behavior analysis. I will provide you with a trajectory of agent behaviors consisting of multiple numbered steps. Each step contains information such as the executing agent, actions, and dialogues. You need to complete two tasks and output the results strictly in the specified format. Task 1: Construct the Step Dependency Graph Nodes are step numbers, i.e., Step 1, Step 2, …. For any two steps SiS_i and SjS_j, typically with i<ji<j, establish a directed edge Si→SjS_i→ S_j if any of the following conditions is satisfied: 1. Content Dependency: The output of SjS_j retains, integrates, references, or is directly derived from the concrete output of SiS_i. This includes data, instructions, assumptions, intermediate conclusions, code snippets, or task descriptions. 2. Goal/Context Dependency: Step SiS_i establishes the problem definition, task goal, execution context, role assignment, or operational constraints used by SjS_j. 3. Feedback Dependency: Step SjS_j explicitly responds to, evaluates, or revises the output of step SiS_i. Important notes for Task 1: • Add only one edge when the same pair satisfies multiple conditions. • A step may depend on multiple preceding steps. • Do not omit dependencies originating from initialization, problem-definition, or task-assignment steps. • Use the step numbers exactly as they appear in the trajectory. Task 2: Construct the Agent Interaction Graph Nodes are agent names. Add a directed edge between two agents when: • information, messages, tasks, requests, feedback, or intermediate conclusions are transmitted between them; • one message or task explicitly targets multiple agents, in which case an edge is added from the sender to each recipient; • a step dependency Si→SjS_i→ S_j exists, where SiS_i is performed by agent A and SjS_j by agent B, with A≠BA≠ B. Do not add duplicate edges. Output Format Output only a valid JSON object. Agent names must be copied from the name attribute of the corresponding history steps. "agent_interactions": [ "from": "<Agent name>", "to": "<Agent name>" ], "step_causalities": [ "from": <step_number>, "to": <step_number> ] Here is the agent behavior trace to be analyzed: trajectory A.8 Real-Time Failure Detection Prompt You are a real-time failure detector for a multi-agent system. We will provide you with the original user task and the execution trajectory observed so far. Your task is to determine, based on the observed agent actions and intermediate states, whether a failure has occurred. A failure occurs when an observed action or intermediate result is inconsistent with the user task, established task constraints, valid prior information, or the requirements of the current execution step. Do not report a failure solely because the task is unfinished or the trajectory contains exploration, uncertainty, or a later correction. Return only: "fault\_detected": true or false, "fault\_step": the earliest identifiable faulty step ID or null, "fault\_agent": the responsible agent or null, "reason": "A brief explanation based on the observed trajectory." Here is the user task and the execution trajectory. [USER TASK] task [OBSERVED TRAJECTORY] trajectory Appendix B B. Detailed Generalization Performance Comparison Results Model HC AG Micro-Accuracy Macro-Accuracy Agent Step Agent Step Agent Step Agent Step Lightweight models (<10<10M) StepFinder 41.38 5.17 43.65 25.40 42.93 19.02 42.52 15.28 ASCon 48.28 12.07 44.44 23.81 45.65 20.11 46.36 17.94 LLM-based models (≥8≥ 8B) Gemini-3.1-Flash 39.66 5.17 60.32 30.95 53.80 22.83 49.99 18.06 GPT-4o-mini 58.62 6.90 53.97 11.90 55.44 10.32 56.30 9.40 Step-by-Step 31.03 12.07 24.60 14.29 26.63 13.59 27.82 13.18 Binary Search 20.17 10.17 27.78 10.32 25.38 10.27 23.98 10.25 DeepSeek-V4-Pro 46.55 3.45 64.29 30.95 58.70 22.28 55.42 17.20 +ASCon 65.52+18.97 32.76+29.31 68.25+3.96 48.41+17.46 67.39+8.69 43.48+21.20 66.89+11.47 40.59+23.39 Gemini-3.1-Flash 39.66 5.17 60.32 13.49 53.80 10.87 49.99 9.33 +ASCon 50.00+10.34 17.24+12.07 65.87+5.55 45.24+31.75 60.87+7.07 36.41+25.54 57.94+7.95 31.24+21.91 SDBL (EASD) 53.45 17.24 58.73 37.30 57.07 30.98 56.09 27.27 +ASCon 68.97+15.52 22.41+5.17 51.59-7.14 39.68+2.38 57.07+0.00 34.24+3.26 60.28+4.19 31.05+3.78 Qwen-SFT 46.55 20.69 51.59 30.16 50.00 27.17 49.07 25.02 +ASCon 50.00+3.45 24.14+3.45 55.56+3.97 35.71+5.55 53.80+3.80 32.07+4.90 52.78+3.71 29.93+4.91 AgentTracer 27.59 8.62 60.32 34.92 50.00 26.63 43.95 21.77 +ASCon 41.38+13.79 15.52+6.90 61.90+1.58 37.30+2.38 55.43+5.43 30.43+3.80 51.64+7.69 26.41+4.64 Table 5: Detailed results on the out-of-domain root-fault attribution task. Micro-accuracy is weighted by the HC and AG test-set sizes, whereas macro-accuracy assigns equal weight to both test sets. Subscripts in the “+ASCon” rows denote absolute percentage-point changes from the corresponding base methods. The best result in each column is shown in bold. Out-of-Domain Evaluation Setup. We evaluate the out-of-domain generalization of ASCon on the Who&When benchmark without any further training or parameter updates. The root-fault attribution test set follows the original Who&When dataset (Zhang et al., 2025). The failure-mode attribution test set is adopted from (Kong et al., 2026), which extends the same Who&When trajectories with additional failure-type annotations. In addition to evaluating ASCon directly, we incorporate its predictions into several strong LLM-based attribution methods to examine whether the learned fault evidence can complement LLM reasoning under distribution shift. For root-fault attribution, we replace SDBL’s original first-stage fault-range prediction with the candidates ranked highest by ASCon, while retaining its subsequent reasoning procedure. Specifically, we select the top-3 agents together with the top-5 and top-10 steps for the Algorithm-Generated (AG) and Handcrafted (HC) subsets, respectively. For the remaining LLM-based baselines, the predicted fault probability predicted by ASCon is attached to each step using the following format to enhance the root-fault attribution task: step_id: x, content: x, fault_probability: x. The augmented trajectory is then provided to the prompt templates in Appendices A.3 and A.4. For failure-mode attribution, ASCon evidence is appended after the complete conversation history. Each agent is associated with its predicted fault probability and five most likely failure modes using the following format: agent_name: x, fault_probability: x, top5_likely_failure_modes: x. The resulting input is evaluated using the prompt template in Appendix A.6. Out-of-Domain Results. Tables 5 and 6 report the out-of-domain results for root-fault and failure-mode attribution. On root-fault attribution, ASCon consistently outperforms StepFinder and substantially improves most enhanced baselines, especially for step localization. With ASCon assistance, DeepSeek-V4-Pro gains 21.20 and 23.39 percentage points in step-level micro- and macro-accuracy, respectively, and achieves the best results on seven of the eight metrics. These gains indicate that ASCon provides transferable fault evidence that helps LLMs identify where failures emerge under distribution shift. For failure-mode attribution, LLMs assisted by ASCon achieve the best agent- and error-level results. Specifically, GPT-4o-mini with ASCon obtains the highest agent-level scores, while Gemini-3.1-Flash with ASCon achieves the best error-level performance. Aegis-SFT also receives the largest agent-level gains, improving by 10.95 and 10.61 percentage points in micro- and macro-F1. Overall, ASCon generalizes beyond its training distribution and effectively complements existing attribution methods. Model Agent μ 1 Agent MF1 Error μ 1 Error MF1 Pair μ 1 Pair MF1 Lightweight models (<10<10M) ASCon 44.76 26.43 7.37 5.95 1.87 1.12 LLM-based models (≥8≥ 8B) DeepSeek-V4-Pro 48.10 29.60 12.64 7.57 7.05 3.16 +ASCon 51.14+3.04 38.23+8.63 13.82+1.18 7.76+0.19 8.24+1.19 4.85+1.69 Gemini-3.1-Flash 50.63 34.47 14.13 8.09 8.41 4.85 +ASCon 53.23+2.60 35.08+0.61 15.61+1.48 11.21+3.12 7.64-0.77 3.15-1.70 GPT-4o-mini 49.34 31.61 6.99 5.16 3.40 1.70 +ASCon 53.90+4.56 38.74+7.13 10.33+3.34 8.71+3.55 4.19+0.79 2.51+0.81 Qwen3.5-Flash 46.04 28.97 9.22 6.46 6.42 3.06 +ASCon 52.52+6.48 31.85+2.88 11.89+2.67 9.34+2.88 6.69+0.27 3.23+0.17 Aegis-SFT 31.78 16.48 5.39 2.85 2.34 1.19 +ASCon 42.73+10.95 27.09+10.61 9.28+3.89 6.43+3.58 3.96+1.62 1.98+0.79 Table 6: Detailed results on the out-of-domain failure-mode attribution task. Appendix C C. Can ASCon Perform Real-time Fault Detection? To evaluate fault detection capabilities in an online setting, we construct a real-time detection experiment on the TracerTraj test set. Each trajectory is truncated at its first annotated faulty step, and only the user query and the interaction history observed up to that point are provided as input. We compare ASCon with four LLM-based reasoning baselines, including GPT-4o-mini, DeepSeek-V4-Pro, Gemini-3.1-Flash, and Qwen3.5-Flash, using the prompt provided in Appendix A.8. A detection is considered as a correct alert only when the predicted faulty position exactly matches the ground-truth faulty position; both premature detection and missed detection are counted as errors. Figure 4 shows that ASCon achieves the highest correct alert rate of 58.23%, outperforming Qwen3.5-Flash by 1.27% and Gemini-3.1-Flash by 5.57%. This result indicates that ASCon can identify the first faulty action from partial trajectories without relying on future observations. Figure 4: Performance comparison on the alert accuracy (%).