Paper deep dive
ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems
Weilin Jin, Mingyu Wang, Taiyu Zhu, Ziqi Zhou, Wenbo Li, Haoyang Huang, Nan Duan, Yifan Wu, Ying Li, Zhonghai Wu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/9/2026, 8:42:16 AM
Summary
The paper introduces ReCast, a method for failure attribution in LLM-based multi-agent systems. It addresses the limitation of existing methods that use raw hidden states or text embeddings, which show limited separation between root-cause steps and other steps. ReCast learns attribution-oriented step representations by selecting relevant LLM layers, constructing pattern and deviation features via sparse projection, and training a bidirectional encoder with contrastive and ranking losses. The method achieves state-of-the-art performance on four benchmarks.
Entities (14)
Relation Signals (12)
RECAST → evaluatedon → AFTraj-2K
confidence 95% · Experiments on four benchmarks—... AFTraj-2K... show that ReCast consistently achieves the best Hit@1
RECAST → evaluatedon → WHO&WHEN PRO
confidence 95% · Experiments on four benchmarks—... and Who&When Pro... show that ReCast consistently achieves the best Hit@1
RECAST → evaluatedon → Who&When
confidence 95% · Experiments on four benchmarks—Who&When... show that ReCast consistently achieves the best Hit@1
RECAST → evaluatedon → TraceElephant
confidence 95% · Experiments on four benchmarks—... TraceElephant... show that ReCast consistently achieves the best Hit@1
RECAST → trainedon → ReCast-2K
confidence 95% · We also introduce ReCast-2K, a training dataset for failure attribution... Experiments on four benchmarks... show that ReCast consistently achieves the best Hit@1
RECAST → uses → Johnson–Lindenstrauss projection
confidence 95% · a fixed sparse Johnson–Lindenstrauss projection (Li et al., 2006) transforms the selected features
RECAST → developedby → Tsinghua University
confidence 90% · Ziqi Zhou Affiliation: Tsinghua University
RECAST → developedby → Peking University
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In LLM-based agent systems, failures can originate from early steps whose effects propagate through subsequent interactions, making their origins difficult to identify. To trace such failures back to their origin, failure attribution has been formulated as the task of identifying the earliest step responsible for the failure. Recent methods leverage LLM internal signals for failure attribution, typically using hidden states as step representations. We therefore conduct an empirical study to evaluate how effectively these representations distinguish root-cause steps from other steps and find limited separation. Motivated by this observation, we propose ReCast, a step representation learning method that transforms hidden states from a frozen LLM into attribution-oriented step representations. ReCast first selects attribution-relevant layers, then constructs complementary pattern and deviation features, and finally learns contextualized step representations through an encoder trained with contrastive and ranking objectives. We also introduce ReCast-2K, a training dataset for failure attribution. ReCast achieves the best Hit@1 across four benchmarks, surpassing the strongest baseline by 5.65 and 9.19 pp on Who&When Algorithm and Handcrafted, respectively. Code is available at this https URL .
Tags
Links
- Source: https://arxiv.org/abs/2610.11334v1
- Canonical: https://arxiv.org/abs/2610.11334v1
Trouble viewing inline? Open PDF directly →
Full Text
102,921 characters extracted from source content.
Expand or collapse full text
ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems Weilin Jin Affiliation: Peking University, Beijing, China Affiliation: Joy Future Academy, Beijing, China Mingyu Wang Affiliation: Peking University, Beijing, China Affiliation: Joy Future Academy, Beijing, China Taiyu Zhu Affiliation: Peking University, Beijing, China Ziqi Zhou Affiliation: Tsinghua University, Beijing, China Wenbo Li †thanks: Project leader. Affiliation: Joy Future Academy, Beijing, China Haoyang Huang Affiliation: Joy Future Academy, Beijing, China Nan Duan Affiliation: Joy Future Academy, Beijing, China Yifan Wu †thanks: Corresponding authors. Affiliation: Peking University, Beijing, China Ying Li22footnotemark: 2 Affiliation: Peking University, Beijing, China Zhonghai Wu Affiliation: Peking University, Beijing, China Abstract In LLM-based agent systems, failures can originate from early steps whose effects propagate through subsequent interactions, making their origins difficult to identify. To trace such failures back to their origin, failure attribution has been formulated as the task of identifying the earliest step responsible for the failure. Recent methods leverage LLM internal signals for failure attribution, typically using hidden states as step representations. We therefore conduct an empirical study to evaluate how effectively these representations distinguish root-cause steps from other steps and find limited separation. Motivated by this observation, we propose ReCast, a step representation learning method that transforms hidden states from a frozen LLM into attribution-oriented step representations. ReCast first selects attribution-relevant layers, then constructs complementary pattern and deviation features, and finally learns contextualized step representations through an encoder trained with contrastive and ranking objectives. We also introduce ReCast-2K, a training dataset for failure attribution. ReCast achieves the best Hit@1 across four benchmarks, surpassing the strongest baseline by 5.65 and 9.19 p on Who&When Algorithm and Handcrafted, respectively. Code is available at https://anonymous.4open.science/r/ReCast-5FB6. 1 Introduction Large language model (LLM) based multi-agent systems (MASs) coordinate specialized agents, external tools, and environment interactions to solve increasingly complex tasks (Wu et al., 2024; Fourney et al., 2024; Song et al., 2025). Each execution produces a trace of agent activity, such as messages and tool calls, providing a basis for understanding system behavior. Despite their growing capabilities, MASs remain vulnerable to errors such as reasoning errors and incorrect tool use (Liu et al., 2026a). These errors can propagate through subsequent interactions and ultimately cause task failure, limiting the reliability of MASs in real-world applications. Diagnosing why a multi-agent execution failed is considerably harder than detecting the failure itself. A task may admit many valid solution paths, and stochastic agent decisions can produce substantially different traces even under the same system and task. More importantly, the root cause may initially appear plausible, since its consequences emerge only after several downstream steps. Failure attribution therefore requires identifying the earliest decisive error from a long sequence of actions, rather than merely detecting anomalous content or locating the final visible symptom (Zhang et al., 2025a). Accurate step-level attribution is essential for debugging agent behaviors, improving orchestration strategies, and constructing more reliable multi-agent systems. Existing methods such as CHIEF (Wang et al., 2026) use LLMs to analyze execution traces, but autoregressive decoding can incur substantial inference latency. To reduce this overhead, recent methods explore failure attribution using LLM internal signals: MASPrism (Liu et al., 2026b) derives negative log-likelihood from hidden states and combines it with attention. StepFinder (Zhu et al., 2026) instead learns attribution from text embeddings. These approaches involve different choices about how to represent each step for attribution. However, a key question remains underexplored: what step representations are most suitable for failure attribution? To investigate this question, we empirically assess how well text embeddings and hidden states distinguish the root-cause step from other steps. Figure 1 reveals limited separation in these representations, motivating the construction and learning of attribution-oriented step representations. Figure 1: Comparison of text embeddings and last-layer hidden-state representations on failed trajectories from ReCast-2K and the Handcrafted and Algorithm subsets of Who&When. Lower ARS/GRS and higher SI indicate better separation. See Appendix A for metric definitions. To address this gap, we propose ReCast, a step representation learning method that integrates feature selection, feature construction, and contextual learning for failure attribution. The method consists of three modules. First, an attribution-aware probe selects internal features according to their ability to distinguish the root-cause step from other steps. Second, a fixed sparse Johnson–Lindenstrauss projection (Li et al., 2006) transforms the selected features into complementary pattern and deviation representations, capturing each step’s internal activation structure and its departure from successful execution behavior. Third, a bidirectional encoder learns contextualized step representations from the complete trajectory. Training jointly optimizes root and non-root contrastive losses alongside a trajectory-level ranking loss. Together, these three stages determine which internal signals are retained, how they are represented, and how they are learned for attribution. We also introduce ReCast-2K, a training dataset containing 1,0441,044 successful and 1,4961,496 failed trajectories. Experiments on four benchmarks—Who&When (Zhang et al., 2025a), TraceElephant (Chen et al., 2026), AFTraj-2K (Zhang et al., 2026a), and Who&When Pro (Liu et al., 2026a)—show that ReCast consistently achieves the best Hit@1, surpassing the strongest baseline by 5.65 and 9.19 p on Who&When Algorithm and Handcrafted, respectively. Ablation studies further support the contributions of its main components. Our main contributions are as follows: • To the best of our knowledge, we are the first to systematically study the discriminative ability of step representations for multi-agent failure attribution, using quantitative metrics to assess separation within and across trajectories. • We propose ReCast, a step representation learning method that combines feature selection, feature construction, and contextual learning to produce attribution-oriented step representations. • We construct ReCast-2K, a diverse source training dataset, and evaluate ReCast on four benchmarks, where it outperforms all compared baselines in Hit@1 accuracy. ReCast also enables efficient attribution from precomputed features. 2 Related Work Black-box Failure Attribution Methods. These methods attribute failures from observable trajectories through structured reasoning or learned models. Among training-free methods, CHIEF (Wang et al., 2026) constructs hierarchical causal graphs from execution logs for oracle-guided backtracking and counterfactual attribution. FALAT (Rafi et al., 2026) formulates attribution as hierarchical dependency-guided search. Distinguishing root causes from noisy or propagated symptoms remains challenging for these methods. Learning-based methods train attribution models from trajectories. AgenTracer (Zhang et al., 2026b) generates annotated failures through counterfactual replay and fault injection and trains AgenTracer-8B with reinforcement learning. AgentForesight (Zhang et al., 2026a) detects decisive errors from trajectory prefixes for online auditing. StepFinder (Zhu et al., 2026) models cross-step dependencies in semantic sequences for root-cause ranking. ASCon (Jiang et al., 2026) jointly models step context and agent interactions for failure attribution. These methods primarily use external trajectory representations, leaving internal diagnostic signals underexplored. White-box Failure Attribution Methods. These methods exploit LLM internal signals. MASPrism (Liu et al., 2026b) identifies symptom steps using negative log-likelihood (NLL) derived from prefill-stage hidden states, then traces potential root causes using attention weights. OAT (Yeh et al., 2026) learns successful hidden-state dynamics with neural controlled differential equations and attributes failures to steps that deviate from these dynamics. Rather than directly scoring internal signals or modeling successful dynamics, ReCast learns attribution-oriented step representations. 3 PROBLEM FORMULATION 3.1 Definition Background. Consider a multi-agent system with an agent set ℳ=m1,…,mNM=\m_1,…,m_N\ interacting with an environment ℰE. Given a task query q∈q , the system produces an execution trace τ=(q,s0,a1,s1,a2,…,at,st,…,aT,sT),τ=(q,s_0,a_1,s_1,a_2,…,a_t,s_t,…,a_T,s_T), (1) where sts_t is the system state after step t, ata_t is the candidate model action, and T is the final step of the execution. Here, we only consider model actions as candidates for failure attribution. Let B index a batch of traces, with τi _i denoting the trace indexed by i∈Bi∈ B and TiT_i its terminal step. Let Z(τ)∈0,1Z(τ)∈\0,1\ denote the task-specific success evaluator for task q, where Z(τ)=1Z(τ)=1 indicates successful completion. We then define the successful and failed trace sets as succ=τ∣Z(τ)=1T^succ=\τ Z(τ)=1\ and fail=τ∣Z(τ)=0T^fail=\τ Z(τ)=0\. Objective. Following Who&When (Zhang et al., 2025a), consider a failed trace τi _i with Z(τi)=0Z( _i)=0. Suppose that the action ata_t taken by an active agent at step t is replaced with a feasible corrective action a~t a_t, while the execution history before step t remains fixed. The system then re-executes the trajectory from step t under the corrected action, producing a counterfactual trace τ~i τ_i. If Z(τ~i)=1Z( τ_i)=1, timestamp t is defined as a decisive error. We use the earliest decisive error as the unique root-cause target for each failed trace τi _i: ti∗=mint∈1,…,Ti:t is a decisive error in τi.t_i^*= \t∈\1,…,T_i\:t is a decisive error in _i\. (2) The root-cause timestamp ti∗t_i^* is unique by definition. We call all remaining candidate timestamps non-root steps and write (τi)=1,…,Ti∖ti∗N( _i)=\1,…,T_i\ \t_i^*\. This label identifies steps other than the attribution target and does not imply that they are error-free. A successful trace has no root cause, so all its candidate timestamps belong to (τi)N( _i). Failure attribution ranks candidate timestamps to identify the unique target ti∗t_i^*. 3.2 Observation Some existing failure attribution methods use step representations derived from text embeddings or LLM hidden states. We conduct an empirical study to evaluate how effectively these representations distinguish the root-cause step from other steps. Specifically, we examine Qwen3 text embeddings (Zhang et al., 2025b), following StepFinder (Zhu et al., 2026), and last-layer hidden states from a frozen LLM, motivated by MASPrism’s use of internal signals (Liu et al., 2026b). We assess these representations from two perspectives: (1) Within-trace separation. The root-cause step should be distinguishable from non-root-cause steps within the same trace. (2) Cross-trace separation. Root-cause representations should also be distinguishable from non-root representations across traces. Based on these criteria, we define three metrics. Adjacent Root Similarity (ARS) and Global Root Similarity (GRS) measure local and overall within-trace separation, respectively, while the cosine Silhouette Index (SI) measures cross-trace separation. Using these metrics, we compare Qwen3 text embeddings with mean-pooled and last-token representations from the final hidden-state layer on the failed trajectories in our training dataset and the Handcrafted and Algorithm subsets of Who&When. See Appendix A for metric and representation details. Figure 1 shows that the evaluated representations exhibit limited separation under the proposed metrics. High ARS and GRS indicate that the root-cause step remains similar to its predecessor and other steps in the same trace, while low SI indicates weak separation across traces. Although their relative performance varies, these representations exhibit limited separation between the root-cause step and other steps under these metrics. This observation motivates investigating whether attribution-oriented step representation learning can improve failure attribution. 4 METHODOLOGY Figure 2: Overview of ReCast, which learns attribution-oriented step representations from frozen LLM hidden states for failure attribution. Motivated by our empirical study, we develop ReCast to learn attribution-oriented step representations from hidden states extracted by prefilling trajectories with a frozen LLM. As shown in Figure 2, ReCast selects attribution-relevant layers and constructs complementary pattern and deviation features. An encoder is then jointly trained with a scoring head under root and non-root contrastive losses and a trajectory-level ranking loss to learn contextualized step representations. 4.1 Feature Selection and Construction Hidden-state extraction. We use a frozen LLM with hidden-state dimension dhd_h to extract step representations by prefilling the trajectory. For each model-generated action ai,ta_i,t in trajectory τi _i, we obtain the layer-ℓ step feature by mean-pooling its dhd_h-dimensional token-level hidden states, conditioned on the preceding execution prefix: i,t(ℓ)=MeanPooltokens(ai,t)(LLM(ℓ)(ai,t∣τi,<t)),i,t(ℓ)∈ℝdh.h_i,t^( )=MeanPool_tokens(a_i,t) (LLM^( )(a_i,t _i,<t) ), _i,t^( ) ^d_h. (3) Attribution-aware layer selection. Transformer layers encode information at different levels of abstraction and are not equally informative for failure attribution. We therefore use a lightweight probe to select layers according to their ability to distinguish the root-cause step from all other steps. For each layer, we apply layer normalization (LN) to obtain normalized step features i,t(ℓ)u_i,t^( ). We then construct a difference vector for each failed trajectory by subtracting the mean non-root feature from the root-cause feature: i,t(ℓ)=LN(i,t(ℓ)),i(ℓ)=i,ti∗(ℓ)−1|(τi)|∑t∈(τi)i,t(ℓ),i,t(ℓ),i(ℓ)∈ℝdh.u_i,t^( )=LN (h_i,t^( ) ), _i^( )=u_i,t_i^*^( )- 1|N( _i)| _t ( _i)u_i,t^( ), _i,t^( ),d_i^( ) ^d_h. (4) Using the difference vectors i(ℓ)d_i^( ), we assess each layer’s attribution capability through five-fold cross-validation on failed training trajectories. In each round, we compute (ℓ),(ℓ)∈ℝdh μ^( ),v^( ) ^d_h as the mean and variance vectors of i(ℓ)d_i^( ) over the trajectories in four folds. These statistics define a scoring direction that emphasizes large and consistent differences: (ℓ)=(ℓ)/((ℓ)+η(ℓ))‖(ℓ)/((ℓ)+η(ℓ))‖2,(ℓ)∈ℝdh.w^( )= μ^( )/(v^( )+η^( )) \| μ^( )/(v^( )+η^( )) \|_2, ^( ) ^d_h. (5) Here η(ℓ)=λridgedh−1∑j=1dhvj(ℓ)η^( )= _ridged_h^-1 _j=1^d_hv_j^( ) is a scalar ridge term. We then evaluate this direction on the remaining fold by scoring each candidate step as ri,t(ℓ)=⟨i,t(ℓ),(ℓ)⟩r_i,t^( )= _i,t^( ),w^( ) . We aggregate the evaluation results across all five folds and rank layers by their root-cause identification performance. Detailed selection criteria are provided in Appendix B.1. Using this ranking, we select k layers while ensuring coverage across the model depth. We partition all layers into k contiguous groups and retain the highest-ranked layer from each group. Let ℓ1,…,ℓk _1,…, _k denote these layers in depth order; their step features are concatenated as i,t=[i,t(ℓ1);…;i,t(ℓk)],i,t∈ℝkdh.h_i,t= [h_i,t^( _1);…;h_i,t^( _k) ], _i,t ^kd_h. (6) Dual-branch projection. To reduce the dimensionality of the concatenated hidden state, we construct two complementary branches using a fixed sparse Johnson–Lindenstrauss projection (Li et al., 2006). The pattern branch applies layer-wise normalization to preserve relative activation patterns. For the deviation branch, we introduce successful trajectories as a reference to capture how each step differs from successful execution. The branch outputs are computed as i,tpat=LN(JL[LN(i,t(ℓj))]j=1k)∈ℝdp,i,tdev=LN(JL(i,t−succ))∈ℝdp.x_i,t^pat\!=\!LN\! (W_JL [LN\! (h_i,t^( _j) ) ]_j=1^k ) ^d_p, _i,t^dev\!=\!LN\! (W_JL (h_i,t- μ^succ ) ) ^d_p. (7) Here, succ∈ℝkdh μ^succ ^kd_h is the mean concatenated hidden state over all steps in successful training trajectories, and JL∈ℝdp×kdhW_JL ^d_p× kd_h is the sparse projection matrix shared by both branches. In the pattern branch, normalization is applied separately to each selected layer before concatenation; the deviation branch operates directly on the concatenated hidden state. The outputs are concatenated as i,t=[i,tpat;i,tdev]∈ℝ2dpx_i,t=[x_i,t^pat;x_i,t^dev] ^2d_p. Both the projection matrix and reference mean are fixed during training. Further details are provided in Appendix B.2. 4.2 Attribution-Oriented Step Representation Learning View augmentation. Before encoding, we independently augment each projected trajectory i=(i,t)t=1TiX_i= (x_i,t )_t=1^T_i twice to obtain two aligned views ~iv X_i^\,v, where v∈f1,f2v∈\f_1,f_2\. The input and augmented views share the same dimensions: i,~iv∈ℝTi×2dpX_i, X_i^\,v ^T_i× 2d_p. For timestamp t and feature coordinate d∈1,…,2dpd∈\1,…,2d_p\, the augmented feature is x~i,t,dv=ki,tvϕi,t,dvxi,t,d+ϵi,t,dv,ki,tv,ϕi,t,dv∈0,1,ϵi,t,dv∈ℝ, x_i,t,d^\,v=k_i,t^v _i,t,d^vx_i,t,d+ _i,t,d^v, k_i,t^v, _i,t,d^v∈\0,1\, _i,t,d^v , (8) where ki,tvk_i,t^v and ϕi,t,dv _i,t,d^v are the step and feature masks, respectively, and ϵi,t,dv _i,t,d^v is additive Gaussian noise. These perturbations are sampled independently for the two views. We protect the root-cause step at ti∗t_i^* from step masking while allowing feature-level perturbations. The augmentation settings are reported in Appendix E.2. Encoding. A bidirectional encoder fθf_θ maps each augmented view to dzd_z-dimensional step representations using the complete trajectory: iv=fθ(~iv)=(i,tv)t=1Ti∈ℝTi×dz.Z_i^v=f_θ( X_i^\,v)= (z_i,t^v )_t=1^T_i ^T_i× d_z. (9) A scoring head gψ:ℝdz→ℝg_ψ:R^d_z produces the attribution score αi,tv=gψ(i,tv) _i,t^v=g_ψ(z_i,t^v) from each step representation i,tv∈ℝdzz_i,t^v ^d_z, providing step-level supervision during encoder training. Root-cause contrastive objective. For each root-cause step, we use its representation in view f1f_1 as the anchor and the corresponding representation in view f2f_2 as the sole positive, since both represent the same underlying step under different perturbations. All non-root steps in view f2f_2 across the batch serve as negatives, promoting separation from diverse non-root contexts. The positive and negative cosine similarities are ρi,ti∗+=cos(i,ti∗f1,i,ti∗f2),ρi,ti∗,j,t−=cos(i,ti∗f1,j,tf2). _i,t_i^*^+= (z_i,t_i^*^f_1,z_i,t_i^*^f_2), _i,t_i^*,j,t^-= (z_i,t_i^*^f_1,z_j,t^f_2). (10) Let Bfail=i∈B:τi∈failB_fail=\i∈ B: _i ^fail\ index the failed trajectories in the batch. We optimize these pairs with an InfoNCE objective: ℒroot=−1|Bfail|∑i∈Bfaillogexp(ρi,ti∗+/κ)exp(ρi,ti∗+/κ)+∑j∈B∑t∈(τj)exp(ρi,ti∗,j,t−/κ).L_root=- 1|B_fail| _i∈ B_fail ( _i,t_i^*^+/κ) ( _i,t_i^*^+/κ)+ _j∈ B _t ( _j) ( _i,t_i^*,j,t^-/κ). (11) Here κ>0κ>0 is the temperature parameter. For successful trajectories, (τi)=1,…,TiN( _i)=\1,…,T_i\, so all their candidate steps contribute to the negative pool. Non-root-cause contrastive objective. To learn discriminative non-root representations, we select anchors from the steps preceding the root cause. For each failed trajectory, let i=t∈(τi):t<ti∗P_i=\t ( _i):t<t_i^*\. When iP_i is nonempty, the anchor timestamps are selected as α¯i,t=αi,tf1+αi,tf22,i=maxi,argmaxt∈iα¯i,t. α_i,t= _i,t^f_1+ _i,t^f_22, _i= \ _i,\ *arg\,max_t _i α_i,t \. (12) The latest step provides a nearby reference, while the highest-scoring step is most likely to be mistaken for the root cause. We retain one anchor when selections coincide and choose the earliest step when scores tie. For each t∈it _i, view f1f_1 supplies the anchor, while view f2f_2 supplies the same step as the positive and that trajectory’s root-cause step as the sole negative: ρi,t+=cos(i,tf1,i,tf2),ρi,t−=cos(i,tf1,i,ti∗f2). _i,t^+= (z_i,t^f_1,z_i,t^f_2), _i,t^-= (z_i,t^f_1,z_i,t_i^*^f_2). (13) Let Bpre=i∈Bfail:i≠∅B_pre=\i∈ B_fail:P_i≠ \ index the eligible failed trajectories. We average the InfoNCE losses first over the selected anchors within each trajectory and then over these trajectories: ℒnon-root=−1|Bpre|∑i∈Bpre1|i|∑t∈ilogexp(ρi,t+/κ)exp(ρi,t+/κ)+exp(ρi,t−/κ).L_non -root=- 1|B_pre| _i∈ B_pre 1|S_i| _t _i ( _i,t^+/κ) ( _i,t^+/κ)+ ( _i,t^-/κ). (14) This reduction gives each eligible trajectory equal weight, regardless of whether it contributes one or two anchors. If BpreB_pre is empty, the loss is zero. Listwise responsibility. For each i∈Bfaili∈ B_fail and view v, we normalize step scores and maximize the root-cause probability at ti∗t_i^*. The ranking loss averages the two view-specific losses: ℒrankv=−1|Bfail|∑i∈Bfaillogexp(αi,ti∗v)∑t=1Tiexp(αi,tv),ℒrank=12(ℒrankf1+ℒrankf2).L_rank^v=- 1|B_fail| _i∈ B_fail ( _i,t_i^*^v) _t=1^T_i ( _i,t^v), _rank= 12 (L_rank^f_1+L_rank^f_2 ). (15) Training objective. The final encoder objective combines the trajectory-level ranking loss, the root-cause contrastive objective, and the non-root-cause contrastive objective: ℒ=λrankℒrank+λrootℒroot+λnon-rootℒnon-root,L= _rankL_rank+ _rootL_root+ _non -rootL_non -root, (16) where λrank _rank, λroot _root, and λnon-root _non -root are the loss weights. At inference, we encode the unaugmented trajectory and use the scoring head to identify the highest-scoring step as the predicted root cause. The specific optimization settings are reported in Appendix E.2. 5 Experiments 5.1 Experimental Setup Benchmarks. We evaluate our method on four failure-attribution benchmarks: Who&When (Zhang et al., 2025a), Who&When Pro (Liu et al., 2026a), AFTraj-2K (Zhang et al., 2026a), and TraceElephant (Chen et al., 2026). Our evaluation includes 124 algorithm-generated (AG) and 58 handcrafted (HC) trajectories from Who&When, 1,251 test trajectories from an 8:2 task-grouped split of the Who&When Pro text subset, 163 failed test trajectories from AFTraj-2K, and 175 valid trajectories from the Captain-Agent (85) and Magentic-One (90) subsets of TraceElephant. All TraceElephant results use these two subsets. Further benchmark details are provided in Appendix C. Training datasets. We use different training datasets for different target benchmarks. Specifically, for Who&When and TraceElephant, we train on our ReCast-2K dataset. For AFTraj-2K, we train on its official training split. For Who&When Pro, we use the training partition described above. See Appendix D for construction and annotation details. Evaluation protocol. We report Hit@k for k∈1,3,5k∈\1,3,5\, Mean Reciprocal Rank (MRR), AUROC, and AUPRC. Hit@k measures whether the annotated root-cause step appears among the top-k predictions, while MRR captures its overall ranking position. AUROC and AUPRC assess score-level separation between root-cause and non-root steps, with AUPRC being particularly informative under class imbalance. All methods use the same three seeds, and we report their mean and standard deviation. See Appendix E.1 for detailed definitions. Baselines. We group the baselines into three categories. (1) Heuristic baselines. First selects the first candidate step as the predicted root cause, whereas Random uniformly samples one candidate step. (2) LLM-based methods. We evaluate the all-at-once and step-by-step protocols from Who&When (Zhang et al., 2025a) using Qwen3.5-27B (Qwen Team, 2026), GPT-5.6 Luna (OpenAI, 2026b), and DeepSeek V4 Flash (Xu et al., 2026), including both open and closed models. We also evaluate CHIEF (Wang et al., 2026) with DeepSeek V4 Flash and include the reported results of AgenTracer (Zhang et al., 2026b). (3) Feature-based methods. We compare with OAT (Yeh et al., 2026), StepFinder (Zhu et al., 2026), MASPrism (Liu et al., 2026b), and ASCon (Jiang et al., 2026). In each in-domain setting, the supervised feature-based baselines use the same failed-trajectory training split as our method. OAT instead follows its original positive-only training protocol. Detailed baseline settings are provided in Appendix F. Implementation details. We extract step representations using Qwen3.5-27B and set the projection dimension of each branch to dp=2,048d_p=2,048. The bidirectional Transformer encoder uses four layers with dz=128d_z=128. Training uses AdamW for up to 50 epochs with a batch size of 32 and a learning rate of 2×10−42× 10^-4. The InfoNCE temperature is κ=0.12κ=0.12, and the loss weights are λrank=1.0 _rank=1.0, λroot=0.95 _root=0.95, and λnon-root=0.25 _non -root=0.25. Additional settings are provided in Appendix E.2. Table 1: Hit@1 attribution accuracy (%) on Who&When and TraceElephant. Results from three random seeds are summarized as mean ± population standard deviation. The LLMs are Qwen3.5-27B (Qwen), DeepSeek V4 Flash (DS), and GPT-5.6 Luna (GPT). Best results are shown in bold, and second-best results are underlined. Category Method Who&When TraceElephant Algorithm Handcrafted Captain Magentic Heuristic First 16.13± 0.00 1.72± 0.00 7.06± 0.00 6.67± 0.00 Random 16.40± 4.85 5.75± 0.81 12.55± 6.10 5.93± 2.62 LLM-based All-at-Once (Qwen) 26.61± 0.00 3.45± 0.00 7.06± 0.00 4.44± 0.00 All-at-Once (DS) 22.04± 0.76 10.34± 2.44 12.55± 1.11 9.63± 0.52 All-at-Once (GPT) 27.42± 0.66 18.39± 5.69 14.90± 4.74 8.15± 0.52 Step-by-Step (Qwen) 32.26± 0.00 17.24± 0.00 25.88± 0.00 18.89± 0.00 Step-by-Step (DS) 29.84± 0.66 21.26± 2.15 23.92± 2.93 11.11± 2.40 Step-by-Step (GPT) 31.99± 0.76 18.97± 3.72 29.41± 2.54 15.93± 0.52 CHIEF (DS) 39.78± 3.25 24.14± 3.72 23.53± 2.54 25.19± 1.89 AgenTracer† 37.30 20.68 – – Feature-based OAT 9.68± 0.66 5.17± 5.08 3.53± 0.00 17.41± 1.39 MASPrism 33.87± 0.00 5.17± 0.00 18.82± 0.00 7.78± 0.00 StepFinder 22.85± 2.12 18.97± 2.44 27.84± 2.00 19.26± 0.52 ASCon 26.34± 1.01 21.84± 0.81 30.98± 1.11 25.93± 0.52 Ours ReCast 45.43± 2.01 33.33± 1.63 35.69± 2.93 27.04± 1.89 † AgenTracer-8B is not publicly available; therefore, we cannot report multi-seed results. 5.2 Main Results Comparison on Who&When and TraceElephant. As shown in Table 1, our method achieves the best Hit@1 accuracy across all four subsets of Who&When and TraceElephant. On Who&When, it reaches 45.43%45.43\% on Algorithm and 33.33%33.33\% on Handcrafted, exceeding the second-best method, CHIEF, by 5.65 and 9.19 percentage points, respectively. On TraceElephant, our method reaches 35.69%35.69\% on Captain-Agent and 27.04%27.04\% on Magentic-One, outperforming the second-best method, ASCon, by 4.71 and 1.11 percentage points, respectively. These improvements support the effectiveness of learning attribution-oriented step representations for failure attribution. Table 2: Hit@1 attribution accuracy (%) on the two in-domain benchmark settings. Results from three random seeds are summarized as mean ± population standard deviation. Best results are shown in bold, and second-best results are underlined. Category Method AFTraj-2K Who&When Pro Heuristic First 0.00± 0.00 7.35± 0.00 Random 11.45± 0.77 25.69± 0.69 LLM-based All-at-Once (Qwen) 20.86± 0.00 21.10± 0.00 All-at-Once (DS) 30.47± 5.02 25.95± 1.29 All-at-Once (GPT) 19.02± 0.87 33.33± 1.27 Step-by-Step (Qwen) 51.53± 0.00 60.33± 0.04 Step-by-Step (DS) 53.99± 1.74 62.16± 0.26 Step-by-Step (GPT) 54.40± 0.77 63.58± 0.37 Feature-based OAT 31.49± 1.04 16.47± 0.99 MASPrism 22.09± 0.00 10.87± 0.00 StepFinder 73.82± 1.61 69.81± 0.61 ASCon 83.03± 0.29 82.71± 0.21 Ours ReCast 84.05± 0.87 91.42± 0.21 Comparison on AFTraj-2K and Who&When Pro. Table 2 reports the Hit@1 results on AFTraj-2K and Who&When Pro. On AFTraj-2K, our method achieves 84.05%84.05\% Hit@1, outperforming GPT-5.6 Luna with Step-by-Step and ASCon by 29.65 and 1.02 percentage points, respectively. On Who&When Pro, it reaches 91.42%91.42\%, exceeding the same baselines by 27.84 and 8.71 points. These gains show that our method remains effective when trained on other datasets, suggesting that its benefits are not limited to ReCast-2K. Figure 3: Cumulative Hit@k accuracy on Who&When and TraceElephant. Stacked segments denote ranks 1, 2–3, and 4–5; error bars show the population standard deviation over three seeds. Hit@k comparison. Figure 3 compares cumulative Hit@1, Hit@3, and Hit@5 across methods on Who&When and TraceElephant. We make two observations: (1) Strong top-k coverage. Our method achieves the highest Hit@5 accuracy across all four benchmark settings, maintaining its advantage under a more relaxed ranking criterion. (2) Better top-rank prioritization. On Captain-Agent, our method reaches 78.04%78.04\% Hit@5, exceeding MASPrism’s 75.29%75.29\%, while achieving a substantially higher Hit@1 of 35.69%35.69\% compared with 18.82%18.82\% for MASPrism. These results indicate that our method ranks the root-cause step more effectively, placing it first more often while maintaining strong top-five coverage. Learned representation separation. To assess whether our method addresses the limited separation identified in Section 3.2, we evaluate the learned step representations using the same metrics. Compared with the basic representations, the learned representations achieve lower ARS and GRS and higher SI on average across seeds. On Handcrafted and Algorithm, mean SI improves from the best basic-representation values of 0.170.17 and 0.030.03 to 0.590.59 and 0.290.29, respectively. These results indicate improved within-trace and cross-trace separation, supporting our goal of learning attribution-oriented step representations. See Appendix A.3 for the full comparison. 6 Analysis Table 3: Ablation results on Who&When. Hit@1 (%) over all trajectories is reported as mean ± standard deviation across three seeds. Δ Hit@1 reports the change from the full-model mean in percentage points (p). Component Variant Hit@1 Δ Hit@1 (p) None Full model 41.58± 1.04 — Layer selection All 64 layers 38.83± 2.30 −2.75-2.75 Uniform 8 38.10± 2.55 −3.48-3.48 Last 8 37.55± 0.93 −4.03-4.03 Random 8 37.18± 0.69 −4.40-4.40 Global probe top-8 38.83± 1.13 −2.75-2.75 Projection branches Pattern only 40.11± 1.96 −1.47-1.47 Deviation only 41.03± 2.92 −0.55-0.55 Encoder configuration Per-step MLP 40.48± 2.12 −1.10-1.10 No augmentation 37.36± 0.90 −4.22-4.22 Non-root anchor selection Latest only 39.93± 0.93 −1.65-1.65 Highest-score only 39.74± 0.52 −1.84-1.84 Objective Rank only 37.73± 0.26 −3.85-3.85 Rank + root 40.48± 0.69 −1.10-1.10 Rank + non-root 37.73± 1.04 −3.85-3.85 Ablation Study. As summarized in Table 3, we conduct ablation experiments on the Who&When benchmark. Each ablation varies one component while keeping the remaining settings fixed. The full model achieves the highest mean Hit@1 of 41.58%41.58\%. Attribution-aware layer selection outperforms five alternatives by 2.75–4.40 percentage points, while the joint loss improves over ranking-only training by 3.85 points. Removing augmentation reduces Hit@1 by 4.22 points, highlighting its contribution to representation learning. Ablations of the projection branches, encoder, and non-root anchor selection also yield lower accuracy, supporting the overall design. Backbone and efficiency analyses. In Appendix G.3, we compare Qwen3.5 backbones with 0.8B, 9B, and 27B parameters while keeping the downstream architecture fixed. The 27B model achieves the highest Hit@1 on Who&When, AFTraj-2K, and Who&When Pro, whereas 0.8B and 9B perform best on Captain-Agent and Magentic-One, respectively. These results indicate that larger backbones can improve attribution accuracy, but increasing model size does not consistently yield better performance. Appendix G.4 further compares accuracy and processing time, demonstrating efficient attribution with ReCast when step features are precomputed. 7 Conclusion This work explores step representation learning for failure attribution in LLM-based agent systems. To understand how well existing representations distinguish root-cause steps from other steps, we conduct an empirical study of text embeddings and hidden states, finding limited separation under the evaluated metrics. Motivated by this observation, ReCast transforms frozen LLM hidden states into attribution-oriented step representations through layer selection, complementary pattern and deviation feature construction, and contextual representation learning with contrastive and ranking objectives. We introduce ReCast-2K to support training. ReCast achieves the best Hit@1 across four benchmarks, while representation analysis shows improved within-trace and cross-trace separation. These results highlight the value of learning step representations tailored to failure attribution. AI use statement In this work, we used generative AI tools for dataset construction and method implementation. We have not used generative AI tools for writing mathematical proofs, and tasks involving critical ingredients for proofs are not applicable to this work. Additionally, we used generative AI tools for drafting sections of the manuscript, language editing, and reference formatting. We have reviewed all AI-assisted work. We checked the accuracy and consistency of AI-assisted content against experimental records and the implementation. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI. Reproducibility statement We provide our code to support reproducibility. Appendices C and D describe the datasets, candidate-step processing, and training-data construction. Method implementation and evaluation settings, including metrics, hyperparameters, and random seeds, are detailed in Appendices B and E. Baseline configurations and LLM attribution prompts appear in Appendices F and I, respectively. References Anthropic (2026) Anthropic Introducing claude opus 4.6. Note: Accessed: 2026-09-15 External Links: Link Cited by: §D.2. Chen et al. (2026) M. Chen, J. Wang, F. Mu, Y. Wang, Z. Liu, H. Feng, and Q. Wang Seeing the whole elephant: a benchmark for failure attribution in llm-based multi-agent systems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 19888–19905. Cited by: §C.4, §D.1, §1, §5.1. Fourney et al. (2024) A. Fourney, G. Bansal, H. Mozannar, C. Tan, E. Salinas, F. Niedtner, G. Proebsting, G. Bassman, J. Gerrits, J. Alber, et al. Magentic-one: a generalist multi-agent system for solving complex tasks. arXiv preprint arXiv:2411.04468. Cited by: §C.1, §C.4, §D.1, §1. Google (2025) Google Introducing gemini 3. Note: Accessed: 2026-09-15 External Links: Link Cited by: §D.2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874. Cited by: §C.3. Hong et al. (2024) S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber MetaGPT: meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations, Vol. 2024, p. 23247–23275. Cited by: §C.3. Hurst et al. (2024) A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276. Cited by: §C.1, §D.1. Jiang et al. (2026) S. Jiang, Y. Ran, K. Xu, X. Chen, Y. Zhang, H. Ren, R. Tang, and T. Zhang ASCon: a direction-aware reciprocal agent–step contextualization model for failure attribution in multi-agent systems. arXiv preprint arXiv:2608.10646. Cited by: §F.3, §2, §5.1. Li et al. (2006) P. Li, T. J. Hastie, and K. W. Church Very sparse random projections. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, p. 287–296. Cited by: §1, §4.1. Liu et al. (2026a) J. Liu, H. Xi, S. Zhang, Y. Zeng, T. Yue, C. Wang, J. Kang, Q. Wu, and H. Wang Who&When pro: can llms really attribute failures in ai agents?. arXiv preprint arXiv:2607.09996. Cited by: §C.2, §1, §1, §5.1. Liu et al. (2023) J. Liu, C. S. Xia, Y. Wang, and L. Zhang Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems, Vol. 36, p. 21558–21572. Cited by: §C.3. Liu et al. (2026b) Y. Liu, H. Feng, J. Pu, and Z. Chen MASPrism: lightweight failure attribution for multi-agent systems using prefill-stage signals. arXiv preprint arXiv:2605.07509. Cited by: 2nd item, §F.3, §1, §2, §3.2, §5.1. Mialon et al. (2024) G. Mialon, C. Fourrier, T. Wolf, Y. LeCun, and T. Scialom Gaia: a benchmark for general ai assistants. In International Conference on Learning Representations, Vol. 2024, p. 9025–9049. Cited by: §C.1, §C.3, §C.4, §D.1. OpenAI (2026a) OpenAI GPT-5.5 system card. Note: Accessed: 2026-09-15 External Links: Link Cited by: §D.2. OpenAI (2026b) OpenAI GPT-5.6 system card. Note: Published July 9, 2026 External Links: Link Cited by: §F.2, §5.1. Qwen Team (2026) Qwen Team Qwen3.5: towards native multimodal agents. External Links: Link Cited by: §E.2, §F.2, §F.3, §5.1. Rafi et al. (2026) M. N. Rafi, M. Ahasanuzzaman, D. J. Kim, Z. Wang, and T. Chen FALAT: tracing failures in llm agent trajectories via dependency-guided search. arXiv preprint arXiv:2606.00765. Cited by: §2. Roucher et al. (2025) A. Roucher, A. V. del Moral, T. Wolf, L. von Werra, and E. Kaunismäki smolagents: a smol library to build great agentic systems. Cited by: §C.3. Rousseeuw (1987) P. J. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20, p. 53–65. Cited by: §A.1. Song et al. (2025) L. Song, J. Liu, J. Zhang, S. Zhang, A. Luo, S. Wang, Q. Wu, and C. Wang Adaptive in-conversation team building for language model agents. arXiv preprint arXiv:2405.19425. Cited by: §C.4, §D.1, §1. Wang et al. (2024) C. Wang, Q. Wu, and the AG2 Community AG2: open-source agentos for ai agents. Note: Available at https://docs.ag2.ai/ External Links: Link Cited by: §C.1. Wang et al. (2026) Y. Wang, W. Wu, J. Wang, and Q. Wang From flat logs to causal graphs: hierarchical failure attribution for llm-based multi-agent systems. arXiv preprint arXiv:2602.23701. Cited by: §F.2, §1, §2, §5.1. Wu et al. (2024) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. AutoGen: enabling next-gen llm applications via multi-agent conversation. In First Conference on Language Modeling, Cited by: §C.3, §1. Xu et al. (2026) A. Xu, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, C. Ling, et al. Deepseek-v4: towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348. Cited by: §D.1, §F.2, §5.1. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §D.1, §F.3. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, p. 2369–2380. Cited by: §C.3. Yeh et al. (2026) S. Yeh, Y. Zhu, S. Deep, and S. Li Tracing agentic failure from the flow of success. arXiv preprint arXiv:2607.12747. Cited by: §F.3, §2, §5.1. Yoran et al. (2024) O. Yoran, S. J. Amouyal, C. Malaviya, B. Bogin, O. Press, and J. Berant Assistantbench: can web agents solve realistic and time-consuming tasks?. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 8938–8968. Cited by: §C.1, §C.4, §D.1. Zhang et al. (2026a) B. Zhang, J. Zhu, Z. Shi, D. Liu, and R. Tang AgentForesight: online auditing for early failure prediction in multi-agent systems. arXiv preprint arXiv:2605.08715. Cited by: §C.3, §1, §2, §5.1. Zhang et al. (2026b) G. Zhang, J. Wang, J. Chen, W. Zhou, K. Wang, and S. Yan Agentracer: who is inducing failure in the llm agentic systems?. In International Conference on Learning Representations, Vol. 2026, p. 11377–11399. Cited by: §F.2, §2, §5.1. Zhang et al. (2025a) S. Zhang, M. Yin, J. Zhang, J. Liu, Z. Han, J. Zhang, B. Li, C. Wang, H. Wang, Y. Chen, and Q. Wu Which agent causes task failures and when? on automated failure attribution of llm multi-agent systems. In Proceedings of the 42nd International Conference on Machine Learning, Cited by: §C.1, §F.2, Appendix I, §1, §1, §3.1, §5.1, §5.1. Zhang et al. (2025b) Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, et al. Qwen3 embedding: advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Cited by: §F.3, §3.2. Zhu et al. (2026) T. Zhu, Y. Wu, W. Jin, Y. Li, and G. Huang StepFinder: a temporal semantic framework for failure attribution in multi-agent systems. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, p. 6946–6957. Cited by: §F.3, §1, §2, §3.2, §5.1. Appendix Contents A Empirical Similarity Study.A A.1 Evaluation Metrics.A.1 A.2 Representations.A.2 A.3 Complete Results.A.3 B Method Implementation Details.B B.1 Attribution-Aware Layer Selection.B.1 B.2 Dual-branch Projection.B.2 C Benchmark Details.C C.1 Who&When.C.1 C.2 Who&When Pro.C.2 C.3 AFTraj-2K.C.3 C.4 TraceElephant.C.4 C.5 Data Processing.C.5 D Training Dataset Construction.D D.1 Task Sampling and Trajectory Generation.D.1 D.2 Root-Cause Annotation.D.2 D.3 Final Dataset Statistics.D.3 E Evaluation Details and Hyperparameters.E E.1 Evaluation Metrics.E.1 E.2 Hyperparameter Settings.E.2 F Baseline Settings.F F.1 Heuristic Baselines.F.1 F.2 LLM-Based Methods.F.2 F.3 Feature-Based Methods.F.3 G Additional Experimental Results.G G.1 MRR Across Rank Cutoffs.G.1 G.2 Score-Level Discrimination.G.2 G.3 Hidden-State Backbone Comparison.G.3 G.4 Time Efficiency Analysis.G.4 H Case Study.H I Prompt Templates.I I.1 All-at-Once Prompt.I.1 I.2 Step-by-Step Prompt.I.2 Appendix A Empirical Similarity Study A.1 Evaluation Metrics In this subsection, i,tx_i,t denotes the representation being evaluated for step t in failed trace τi _i, including text embeddings, hidden states, projected features, or learned representations. We use the root-cause timestamps ti∗t_i^* and non-root sets (τi)N( _i) defined in Section 3.1. Let pre=τi∈fail:ti∗>1T_pre=\ _i ^fail:t_i^*>1\ contain the traces whose root cause has a preceding step. We define Adjacent Root Similarity (ARS) as ARS=1|pre|∑τi∈precos(i,ti∗,i,ti∗−1).ARS= 1|T_pre| _ _i _pre \! (x_i,t_i^*,x_i,t_i^*-1 ). (17) Global Root Similarity (GRS) is defined as GRS=1|fail|∑τi∈fail1|(τi)|∑t∈(τi)cos(i,ti∗,i,t).GRS= 1|T^fail| _ _i ^fail 1|N( _i)| _t ( _i) \! (x_i,t_i^*,x_i,t ). (18) For the cosine Silhouette Index (SI) (Rousseeuw, 1987), we pool the root-cause representations into root=i,ti∗:τi∈failX_root=\x_i,t_i^*: _i ^fail\ and the non-root representations into non=i,t:τi∈fail,t∈(τi)X_non=\x_i,t: _i ^fail,\,t ( _i)\. For each representation x, we define its same-class and opposite-class sets as ((),())=(root,non),∈root,(non,root),∈non. (S(x),O(x) )= cases (X_root,X_non ),&x _root,\\ (X_non,X_root ),&x _non. cases Using cosine distance dcos(,)=1−cos(,)d_ (x,z)=1- (x,z), we compute a() a(x) =1|()|−1∑∈()∖dcos(,), = 1|S(x)|-1 _z (x) \x\d_ \! (x,z ), (19) b() b(x) =1|()|∑∈()dcos(,). = 1|O(x)| _z (x)d_ \! (x,z ). The resulting SI is SI=1|root|+|non|∑∈root∪nonb()−a()max(a(),b()).SI= 1|X_root|+|X_non| _x _root _non b(x)-a(x) \! (a(x),b(x) ). (20) Together, these metrics assess two aspects of representation quality: within-trace and cross-trace separation. Within a trace, ARS measures similarity between the root-cause step and its immediately preceding step, while GRS measures its average similarity to all other steps; lower values indicate stronger separation. Across traces, SI measures the separation between root-cause and non-root representations, with higher values indicating better separation. A.2 Representations We compare the following three representations of candidate model-action steps: • Text embedding. Following StepFinder, we encode the serialized text of each candidate model-action step with Qwen3-Embedding-0.6B, producing a 1,0241,024-dimensional vector. • Mean last-layer hidden state. Following MASPrism (Liu et al., 2026b) in using LLM internal signals, we average the Qwen3.5-27B last-layer hidden states over action tokens at each candidate timestamp, producing a 5,1205,120-dimensional vector. • Last-token last-layer hidden state. We use the Qwen3.5-27B last-layer hidden state of the final token at each candidate timestamp, producing a 5,1205,120-dimensional vector. A.3 Complete Results Table 4: Separation of step representations measured by ARS, GRS, and SI. ReCast results show mean ± population standard deviation across seeds 66, 2020, and 4242; other representations are fixed. Best results are bolded for each dataset and metric, based on mean values for ReCast. Dataset Representation ARS ↓ GRS ↓ SI ↑ Training Text embedding 0.54780.5478 0.61730.6173 0.14820.1482 Hidden[-1] (Mean) 0.72540.7254 0.76300.7630 0.01310.0131 Hidden[-1] (Last) 0.78320.7832 0.76860.7686 0.02280.0228 Projected features 0.57630.5763 0.60010.6001 0.05660.0566 ReCast −0.2590±0.2116-0.2590± 0.2116 −0.2749±0.1977-0.2749± 0.1977 0.7032±0.18040.7032± 0.1804 Handcrafted Text embedding 0.51950.5195 0.58850.5885 0.09590.0959 Hidden[-1] (Mean) 0.77100.7710 0.76910.7691 0.04640.0464 Hidden[-1] (Last) 0.70190.7019 0.67600.6760 0.17350.1735 Projected features 0.60010.6001 0.61270.6127 0.05930.0593 ReCast 0.4157±0.07810.4157± 0.0781 0.4235±0.09330.4235± 0.0933 0.5913±0.12330.5913± 0.1233 Algorithm Text embedding 0.83340.8334 0.72230.7223 0.03090.0309 Hidden[-1] (Mean) 0.89690.8969 0.84170.8417 −0.0409-0.0409 Hidden[-1] (Last) 0.80490.8049 0.79440.7944 −0.0137-0.0137 Projected features 0.76580.7658 0.68880.6888 0.00050.0005 ReCast 0.4087±0.03420.4087± 0.0342 0.3878±0.04670.3878± 0.0467 0.2873±0.09860.2873± 0.0986 We evaluate the three basic representations alongside our encoder inputs and learned step representations on the same failed trajectories using the metrics defined above. In Table 4, Projected features denotes i,t∈ℝ2dpx_i,t ^2d_p, the concatenated pattern and deviation features from Eq. 7 before augmentation and encoding, with 2dp=4,0962d_p=4,096. ReCast denotes the learned contextualized step representations before the scoring head, with dz=128d_z=128. Among the basic representations, text embeddings perform best in most comparisons, although their advantage varies by dataset and metric. All three representations exhibit substantial within-trace similarity, reflected in high ARS and GRS, and limited cross-trace separation, reflected in low SI. This pattern is consistent with the limited separation observed under these metrics in Section 3.2. In contrast, ReCast achieves the lowest mean ARS and GRS and the highest mean SI on every dataset. Its mean SI reaches 0.59130.5913 on Handcrafted and 0.28730.2873 on Algorithm, exceeding the best basic-representation values of 0.17350.1735 and 0.03090.0309, respectively. The gains over projected features further show that representation learning improves separation beyond feature construction. These improvements within and across trajectories provide quantitative support for learning attribution-oriented step representations. Appendix B Method Implementation Details B.1 Attribution-Aware Layer Selection Section 4.1 presents the overall attribution-aware layer-selection procedure. Here, we detail fold construction, probe statistics, and layer ranking based on held-out evaluation results. We partition the failed training trajectories failT^fail into five folds using a deterministic hash of each trace identifier. For fold f, let ℐfI^f denote the held-out trajectories and ℐ−f=fail∖ℐfI^-f=T^fail ^f the trajectories in the other four folds. The probe is fitted on ℐ−fI^-f and evaluated on ℐfI^f. The mean and sample-variance vectors used in Eq. 5 are (ℓ)=1|ℐ−f|∑τi∈ℐ−fi(ℓ),(ℓ)=1|ℐ−f|−1∑τi∈ℐ−f(i(ℓ)−(ℓ))⊙(i(ℓ)−(ℓ)). μ^( )= 1|I^-f| _ _i ^-fd_i^( ), ^( )= 1|I^-f|-1 _ _i ^-f (d_i^( )- μ^( ) ) (d_i^( )- μ^( ) ). (21) Using the score ri,t(ℓ)r_i,t^( ) defined in the main text, the within-trajectory rank of the labeled root cause is ranki(ℓ)=1+∑t∈(τi)[ri,t(ℓ)>ri,ti∗(ℓ)],∀τi∈ℐf.rank_i^( )=1+ _t ( _i)1 [r_i,t^( )>r_i,t_i^*^( ) ], ∀\, _i ^f. (22) Thus, ranki(ℓ)=1rank_i^( )=1 when no non-root step has a strictly larger score than the root cause, with tied scores leaving the rank unchanged. We pool the out-of-fold results to evaluate each layer by Hit@1, MRR, and mean root margin: Hit@1(ℓ) @1^( ) =1∑f=15|ℐf|∑f=15∑τi∈ℐf[ranki(ℓ)=1], = 1 _f=1^5|I^f| _f=1^5 _ _i ^f1 [rank_i^( )=1 ], (23) MRR(ℓ) ^( ) =1∑f=15|ℐf|∑f=15∑τi∈ℐf1ranki(ℓ), = 1 _f=1^5|I^f| _f=1^5 _ _i ^f 1rank_i^( ), (24) Margin(ℓ) ^( ) =1∑f=15|ℐf|∑f=15∑τi∈ℐf(ri,ti∗(ℓ)−maxt∈(τi)ri,t(ℓ)). = 1 _f=1^5|I^f| _f=1^5 _ _i ^f (r_i,t_i^*^( )- _t ( _i)r_i,t^( ) ). (25) To ensure coverage across model depth, we partition the L layers into k contiguous groups: b=⌊(b−1)Lk⌋+1,…,⌊bLk⌋,b=1,…,k.G_b= \ (b-1)Lk +1,…, bLk \, b=1,…,k. (26) Within each group, we rank layers by Hit@1, breaking ties by MRR and then mean root margin, and select the highest-ranked layer: ℓb=argmaxℓ∈b(Hit@1(ℓ),MRR(ℓ),Margin(ℓ)). _b= *arg\,max_ _b (Hit@1^( ),MRR^( ),Margin^( ) ). (27) The resulting layers and probe settings are reported in Appendix E.2. B.2 Dual-branch Projection Equation 7 defines the pattern and deviation branches. Here we provide the implementation details not specified in the main text: how the sparse projection is sampled and the exact normalization and reference statistics. We sample a sparse projection over all L layers and retain the columns corresponding to the selected layers, ensuring that layer-selection variants share the same projection coordinates. The random entries follow (JL)a,b∼1δdp,with probability δ2,0,with probability 1−δ,−1δdp,with probability δ2.δ=1Ldh. (W_JL )_a,b cases 1 δ d_p,&with probability δ2,\\[5.0pt] 0,&with probability 1-δ,\\[5.0pt] - 1 δ d_p,&with probability δ2. cases δ= 1 Ld_h. (28) Here, JL∈ℝdp×kdhW_JL ^d_p× kd_h, where L is the total number of backbone layers (L=64L=64 for Qwen3.5-27B), dhd_h is the feature dimension of one hidden layer, dpd_p is the projection dimension, and δ is the projection density, i.e., the probability that an entry of JLW_JL is nonzero. The full-layer matrix has shape dp×Ldhd_p× Ld_h and is generated with random state 42. Its random entries are sampled independently of both the attribution labels and the hidden-state covariance, and the retained matrix is fixed and shared by both branches and across training seeds. All instances of LNLN in Eq. 7 use parameter-free normalization. For a vector ∈ℝnv ^n, where n=dhn=d_h for a layer feature and n=dpn=d_p for a projected branch output, we compute LN()=−[]Var()+ϵ.LN(v)= v-E[v] Var(v)+ε. (29) Here, the expectation and variance are computed independently for each step over its n feature coordinates, and ϵε is a numerical-stability constant. The normalization has no learned scale or bias. The successful-reference mean succ∈ℝkdh μ^succ ^kd_h used in Eq. 7 is computed over all candidate steps in the successful training trajectories: succ=∑τi∈succ∑t=1Tii,t∑τi∈succTi. μ^succ= _ _i ^succ _t=1^T_ih_i,t _ _i ^succT_i. (30) Using the fixed projection matrix JLW_JL and successful-reference mean succ μ^succ, Eq. 7 produces the pattern and deviation features i,tpatx_i,t^pat and i,tdevx_i,t^dev, which are concatenated directly without additional weighting or branch-specific scoring heads. The remaining dimensions and hyperparameters are provided in Appendix E.2. Appendix C Benchmark Details C.1 Who&When Who&When (Zhang et al., 2025a) contains 184 failed trajectories generated under two MAS construction settings and evaluated on two task benchmarks. The algorithm-generated (AG) setting uses CaptainAgent from the AG2 library (Wang et al., 2024) to construct a task-specific MAS for each query, whereas the handcrafted (HC) setting uses the fixed, manually designed Magentic-One system (Fourney et al., 2024); both settings use GPT-4o (Hurst et al., 2024) as the base language model. The underlying tasks are drawn from GAIA (Mialon et al., 2024), which evaluates agents on real-world questions requiring reasoning and tool use, and AssistantBench (Yoran et al., 2024), which focuses on realistic and time-consuming web tasks. The AG subset contains 126 failed trajectories, including 98 generated on GAIA and 28 on AssistantBench. The HC subset contains 58 failed trajectories produced by Magentic-One, including 30 trajectories from GAIA and 28 from AssistantBench. Each trajectory is annotated with the failure-responsible agent, the decisive error step, and a natural-language explanation. Because our candidate set is restricted to model-generated actions, we exclude two AG trajectories whose annotated decisive errors correspond to outputs from the Computer_terminal tool rather than model-action steps, leaving 124 AG trajectories and all 58 HC trajectories for evaluation. C.2 Who&When Pro Who&When Pro (Liu et al., 2026a) is a large-scale failure-attribution benchmark containing 12,32612,326 failed trajectories drawn from 26 source benchmarks, nine task categories, and 15 agent frameworks. It covers text, image, and video modalities as well as both single-agent and multi-agent execution settings. The benchmark defines 18 error modes spanning perception, reasoning, planning, action, verification, and coordination failures, and provides gold-standard labels for the responsible agent, the decisive error step, and the corresponding error mode. Rather than inferring failure causes retrospectively, Who&When Pro constructs each failed trajectory by injecting a controlled error into a successful execution and retaining only interventions that change the final outcome from success to failure, providing causally grounded labels for the responsible agent and decisive step. Our experiments use all 6,2576,257 text trajectories, which span 16 source benchmarks, eight agent frameworks, and 14 represented error modes; Table 5 summarizes their category-level composition and trajectory lengths. To keep trajectories from the same task together, we group them by benchmark, task query, and reference answer and assign entire groups to an 8:2 training–test split using seed 42. The partitions contain 5,0065,006 and 1,2511,251 trajectories, respectively, with no shared task groups or normalized queries. Since the training partition contains only failed trajectories, we use 1,0441,044 successful trajectories from ReCast-2K to compute the deviation reference mean and provide contrastive negatives. Table 5: Composition of the Who&When Pro text-modality subset used in our experiments. Step counts refer to aligned model-action steps, and token counts are computed over the corresponding inputs. Task Category # Trajectories # Steps # Tokens Total Train Test Avg. Max Avg. Max Embodied 311 249 62 30.0 30 1,444.6 1,636 STEM 1,517 1,213 304 5.6 30 4,199.5 23,786 Data Science 2,446 1,958 488 3.7 17 609.1 6,861 Deep Search 1,641 1,313 328 5.6 31 1,551.9 21,948 Coding 342 273 69 3.0 3 3,127.6 7,490 Total 6,257 5,006 1,251 5.9 31 1,906.0 23,786 C.3 AFTraj-2K AFTraj-2K (Zhang et al., 2026a) contains 2,2762,276 multi-agent trajectories across the Coding, Math, and Agentic domains, including 1,1621,162 successful trajectories and 1,1141,114 failed trajectories with decisive-error annotations. Each domain pairs representative task benchmarks with suitable multi-agent frameworks. The Coding domain uses HumanEval+ and MBPP+ (Liu et al., 2023), with trajectories generated by AutoGen (Wu et al., 2024) and MetaGPT (Hong et al., 2024). The Math domain uses MATH-500 (Hendrycks et al., 2021) with AutoGen, while the Agentic domain uses GAIA (Mialon et al., 2024) and HotpotQA (Yang et al., 2018) with Smolagents (Roucher et al., 2025). Across the three domains, trajectories contain 11.5 turns on average. Within AFTraj-2K, successful trajectories are selected through strict checks for outcome correctness, execution integrity, and step coherence. Failed trajectories come from two sources: controlled error injections and naturally failed runs. For injected failures, the modified agent and step serve as direct labels; for natural failures, the earliest decisive error is identified through multi-LLM proposal and verification. The dataset is divided into 1,9441,944 training and 332 test trajectories, with each successful trajectory and its injected variants kept in the same partition. For the in-domain setting, we train on the AFTraj-2K training split and evaluate on its test split. C.4 TraceElephant TraceElephant (Chen et al., 2026) is a failure-attribution benchmark with fully observable execution traces and reproducible environments. For every agent action, it records the input context, model output, and associated tool interactions, together with trace-level information such as the task instruction, agent configuration, and system architecture. Our evaluation uses two multi-agent system subsets. Captain-Agent (Song et al., 2025) dynamically assembles task-specific teams and contributes 85 failed trajectories, including 73 from GAIA (Mialon et al., 2024) and 12 from AssistantBench (Yoran et al., 2024). Magentic-One (Fourney et al., 2024) uses a fixed multi-agent architecture; after excluding one AssistantBench trajectory lacking a root-cause label, its evaluation subset contains 90 failed trajectories, including 74 from GAIA and 16 from AssistantBench. Together, these two subsets contain 175 failed trajectories, with an average of 25.1 and a maximum of 59 candidate steps per trajectory. None of these trajectories exactly overlaps with the failed trajectories in our training corpus. C.5 Data Processing To standardize the heterogeneous trace formats across benchmarks, we use the recorded step roles to retain model-generated actions as attribution candidates and exclude environment observations, tool-result messages, and other non-model events. Among the failed AFTraj-2K test trajectories, this processing reduces the pooled candidate population from 1,711 to 1,548 steps while retaining all 163 trajectories and their annotated root-cause steps. All compared feature-based methods use the same processed candidate set. Appendix D Training Dataset Construction Table 6: Composition of our source training dataset. Initial runs are the aligned executions before filtering. The final total consists of retained successful trajectories and failed trajectories with root-cause annotations. Additional generation runs are grouped under Others. Benchmark MAS Model Initial Final Dataset Runs Succ. Fail. Total AssistantBench Captain-Agent DeepSeek V4 Flash 165 20 53 73 GPT-4o-0806 165 14 38 52 Qwen3-235B 165 20 53 73 Subtotal 495 54 144 198 GAIA Captain-Agent DeepSeek V4 Flash 451 257 189 446 GPT-4o-0806 421 79 186 265 Qwen3-235B 449 160 195 355 Others 41 6 0 6 Subtotal 1,362 502 570 1,072 Magentic-One DeepSeek V4 Flash 449 177 258 435 GPT-4o-0806 460 132 291 423 Qwen3-235B 456 179 233 412 Subtotal 1,365 488 782 1,270 Total – – 3,222 1,044 1,496 2,540 D.1 Task Sampling and Trajectory Generation We construct ReCast-2K using the TraceElephant data-collection pipeline (Chen et al., 2026). Captain-Agent (Song et al., 2025) trajectories are collected on GAIA (Mialon et al., 2024) and AssistantBench (Yoran et al., 2024), and Magentic-One (Fourney et al., 2024) trajectories on GAIA. Tasks are drawn from the validation splits. Executions primarily use DeepSeek V4 Flash (Xu et al., 2026), GPT-4o-0806 (Hurst et al., 2024), and Qwen3-235B (Yang et al., 2025). The collection contains 3,2223,222 trajectories before filtering, as summarized in Table 6. Captain-Agent generation. Captain-Agent dynamically constructs a task-specific team for every execution. Its builder is instructed to include capabilities for information retrieval, Python-based file processing and calculation, and answer verification, and selects concrete experts from the Captain-Agent agent library. Each run is allowed at most 30 group-chat rounds. Generation uses temperature 0.1, top-p 0.95, and a maximum of 4,096 output tokens per model call; failed or empty executions are retried up to five times. Magentic-One generation. Magentic-One uses its fixed centrally orchestrated team: an Orchestrator maintains the plan and delegates to WebSurfer, FileSurfer, Coder, and ComputerTerminal. WebSurfer operates a browser, FileSurfer reads local artifacts, Coder proposes programs, and ComputerTerminal executes them in an isolated run directory. As with Captain-Agent, attached GAIA files are exposed through their resolved local paths, and each execution is capped at 30 rounds. D.2 Root-Cause Annotation Before annotation, we filter incomplete or malformed traces, normalize agent identities and metadata, and extract the final answer from each run. We determine execution outcomes by matching the extracted answers against benchmark references using both exact and semantic criteria. Failure preprocessing removes traces without a usable answer or with at most one step and applies answer-based deduplication within each task and source configuration. Table 6 reports the final retained trajectories. For each retained failed trajectory, we apply an all-at-once annotation protocol using three independent LLM judges: GPT-5.5 (OpenAI, 2026a), Gemini 3 Pro Preview (Google, 2025), and Claude Opus 4.6 (Anthropic, 2026). Each judge predicts a root-cause step, and we aggregate valid returned predictions by step-level voting. In the final corpus, 1,112 trajectories carry consensus labels: 766 have unanimous agreement among valid predictions, and 346 have a two-of-three majority. The remaining 384 trajectories undergo manual review to determine their final root-cause labels. Together, these groups account for all 1,496 failed trajectories. The responsible agent is the agent acting at the root-cause step. D.3 Final Dataset Statistics Table 7 summarizes the final dataset by generation model and trajectory type. It contains 1,0441,044 successful and 1,4961,496 failed trajectories with substantial variation in step and token counts. Two patterns emerge from the collected data: (1) Failed trajectories are generally longer than successful ones. They average 25.9 versus 17.2 steps and 8,346.68,346.6 versus 5,605.55,605.5 tokens. (2) Trajectory characteristics vary noticeably across generation models. GPT-4o-0806 produces failed trajectories with the highest average number of steps (32.0), whereas DeepSeek V4 Flash yields the highest average and maximum token counts among failed trajectories. Together, these variations provide broad coverage of trajectory lengths and execution patterns in the training data. Table 7: Statistics of ReCast-2K. Steps count recorded model responses, while token counts cover the question prefix and serialized trajectory using the Qwen3.5-27B tokenizer. Overall averages are computed over all trajectories. Model Type # Trajectories # Steps # Tokens Avg. Max Avg. Max DeepSeek V4 Flash Successful 454 15.7 61 7,005.5 59,153 Failed 500 21.6 55 10,603.5 74,773 GPT-4o-0806 Successful 225 18.5 60 4,460.0 22,757 Failed 515 32.0 61 7,996.1 29,094 Qwen3-235B Successful 359 18.5 62 4,568.5 21,378 Failed 481 24.0 62 6,375.8 23,621 Others Successful 6 4.8 7 4,677.5 6,729 Overall Successful 1,044 17.2 62 5,605.5 59,153 Failed 1,496 25.9 62 8,346.6 74,773 Appendix E Evaluation Details and Hyperparameters E.1 Evaluation Metrics Using the root-cause timestamp ti∗t_i^* defined in Section 3.1, let ranki(ti∗)rank_i(t_i^*) denote the predicted rank of the root cause in failed trajectory τi _i. Hit@k and MRR over the evaluated failed trajectories are computed as Hit@k=1|fail|∑τi∈fail[ranki(ti∗)≤k],MRR=1|fail|∑τi∈fail1ranki(ti∗).Hit@k= 1|T^fail| _ _i ^failI\! [rank_i(t_i^*)≤ k ], = 1|T^fail| _ _i ^fail 1rank_i(t_i^*). We report Hit@k for k∈1,3,5k∈\1,3,5\. Results are reported as mean ± population standard deviation over three seeds. For score-based evaluation, we use the attribution score αi,t _i,t assigned to timestamp t. The root-cause step ti∗t_i^* is the sole positive candidate; all timestamps in (τi)N( _i) are negative candidates. Their total numbers over the evaluated failed trajectories are C+=|fail|,C−=∑τi∈fail|(τi)|.C_+=|T^fail|, C_-= _ _i ^fail|N( _i)|. AUROC and AUPRC pool these candidate-level predictions across trajectories rather than averaging per-trajectory values. AUROC compares every annotated root-cause step with every non-root step across the evaluated trajectories: AUROC=1C+C−∑τi∈fail∑τj∈fail∑t∈(τj)[(αi,ti∗>αj,t)+12(αi,ti∗=αj,t)].AUROC= 1C_+C_- _ _i ^fail _ _j ^fail _t ( _j) [I( _i,t_i^*> _j,t)+ 12I( _i,t_i^*= _j,t) ]. (31) For AUPRC, we aggregate the attribution scores of all candidate steps across the evaluated trajectories and sort the J distinct scores in descending order as θ1>θ2>⋯>θJ _1> _2>·s> _J. At threshold θj _j, every candidate step with αi,t≥θj _i,t≥ _j is treated as a positive prediction. Such a prediction is a true positive if t=ti∗t=t_i^* and a false positive otherwise. The corresponding counts are TPj=∑τi∈fail(αi,ti∗≥θj),FPj=∑τi∈fail∑t∈(τi)(αi,t≥θj).TP_j= _ _i ^failI( _i,t_i^*≥ _j), _j= _ _i ^fail _t ( _i)I( _i,t≥ _j). Precision and recall are Precj=TPjTPj+FPj,Recj=TPjC+.Prec_j= TP_jTP_j+FP_j, _j= TP_jC_+. We compute AUPRC using the non-interpolated average-precision formulation: AUPRC=∑j=1J(Recj−Recj−1)Precj,Rec0=0.AUPRC= _j=1^J (Rec_j-Rec_j-1 )Prec_j, _0=0. (32) E.2 Hyperparameter Settings Table 8: Hyperparameter settings for our method. Component Hyperparameter Value Backbone Hidden-state dimension dhd_h 5,120 Layer selection Cross-validation folds 5 Layer budget k 8 Ridge ratio 0.10 Selected layers 7,15,19,32,33,47,49,57\7,15,19,32,33,47,49,57\ Projection Dimension per branch dpd_p 2,048 Final step-feature dimension 2dp2d_p 4,096 Random state 42 Encoder Transformer layers / attention heads 4 / 4 Hidden dimension 128 Output dimension dzd_z 128 Feed-forward dimension 384 Maximum sequence length (steps) 512 Dropout 0.10 Augmentation Step / feature masking probability 0.10 / 0.10 Gaussian-noise standard deviation 0.01 Objective InfoNCE temperature κ 0.12 (λrank,λroot,λnon-root)( _rank, _root, _non -root) (1.0,0.95,0.25)(1.0,0.95,0.25) Optimization Optimizer AdamW Learning rate / weight decay 2×10−42× 10^-4 / 10−410^-4 Batch size 32 Maximum epochs 50 Gradient clipping 1.0 Training seeds 6, 20, 42 Table 8 summarizes the settings used with Qwen3.5-27B (Qwen Team, 2026) as the hidden-state extraction backbone. The selected eight layers provide features for the pattern and deviation branches, each with dimension dp=2,048d_p=2,048. A four-layer bidirectional Transformer maps their concatenation to dz=128d_z=128 step representations. The backbone, projection matrix, and successful-reference mean remain fixed while the encoder and scoring head are jointly optimized. Appendix F Baseline Settings F.1 Heuristic Baselines First. First is a deterministic position-only baseline that ranks candidate steps by their step order and selects the earliest step as the predicted root cause. It has no trainable parameters and does not use the trajectory text, agent identity, or benchmark labels. Its standard deviation across seeds is zero because the ranking is deterministic. Random. Random produces a complete ranking by uniformly shuffling all candidate steps. For a fair comparison, we evaluate it with the same three seeds as the other methods and report the mean and population standard deviation. Both First and Random operate directly on each benchmark’s original candidate set. F.2 LLM-Based Methods Common protocol. We adopt two standard LLM-based attribution protocols from Who&When (Zhang et al., 2025a). Appendix I provides the complete prompts used in all evaluations, including the task instruction, ground-truth answer, and serialized multi-agent conversation. Each run returns at most one predicted step. A prediction is counted as correct only when the parsed step maps to a valid candidate and matches the annotated root-cause step exactly; an unparsable response, an out-of-candidate step, or a run that finds no error is counted as incorrect. Since these protocols return a single step rather than a complete ranking, we report only Hit@1. The two protocols operate as follows: • All-at-Once. The model receives the complete conversation in one request and is asked to return the responsible agent, the first erroneous step, and a short explanation. • Step-by-Step. The model processes growing trajectory prefixes in step order. At each step, the prompt asks whether the most recent action contains an error that could derail the execution. Evaluation stops at the first response parsed as “Yes,” and that step becomes the prediction. If all steps receive “No” or no valid decision is produced, the trajectory is marked as having no predicted root cause. This protocol uses more calls than All-at-Once but prevents later trajectory content from influencing an earlier decision. Models and inference settings. We evaluate both protocols with Qwen3.5-27B (Qwen Team, 2026), DeepSeek V4 Flash (Xu et al., 2026), and GPT-5.6 Luna (OpenAI, 2026b). Qwen3.5-27B is served locally with vLLM using bfloat16 weights, greedy decoding with temperature zero, thinking disabled, and a configured 131K-token context window. Across all four benchmarks, all three models use a maximum output length of 16,384 tokens under both protocols. DeepSeek V4 Flash uses explicitly enabled thinking, while GPT-5.6 Luna retains its default configuration. Both protocols are evaluated with each of the three models using inference seeds 6,20,42\6,20,42\, and results are reported as mean ± population standard deviation. AgenTracer. AgenTracer (Zhang et al., 2026b) constructs attribution training data through counterfactual replay and fault injection and trains an 8B attribution model with reinforcement learning. Since the model is not publicly available, we directly use its reported Who&When results, without re-evaluation on our filtered Algorithm subset. These are single reported values and do not have a three-seed standard deviation. CHIEF. CHIEF (Wang et al., 2026) constructs hierarchical causal graphs for failure attribution. We follow its released six-stage pipeline, retaining the original prompts, parsers, and retrieval setup. Inference uses DeepSeek V4 Flash with thinking enabled, a maximum output length of 16,384 tokens per call, and seeds 6,20,42\6,20,42\. Retrieval uses the supplied knowledge bases and all-MiniLM-L6-v2 embeddings. Following the original protocol, prompts include task reference answers but not root-cause annotations. We evaluate the final predicted step using Hit@1. F.3 Feature-Based Methods Table 9: Optimization settings for trainable feature-based baselines. Method LR Weight decay Batch Budget OAT 10−410^-4 10−510^-5 32 300 epochs StepFinder 10−310^-3 10−510^-5 16 50 epochs ASCon 5×10−55×10^-5 10−510^-5 1 12 epochs OAT. OAT (Yeh et al., 2026) formulates unsupervised failure attribution as one-class learning: it learns the normal execution dynamics of successful trajectories and treats deviations from these dynamics as failure signals. We extract the query and candidate-action representations from the final layer of Qwen3.5-27B, using mean pooling over the corresponding tokens. For each seed, the successful trajectories are randomly divided into 80% training and 20% validation partitions. PCA is fitted only on the training partition to reduce the 5,120-dimensional representations to 64 dimensions; the projected features are then standardized by subtracting the training-partition mean and dividing by its standard deviation. The query representation initializes the trajectory dynamics, and step timestamps are normalized to [0,1][0,1]. The predictor models each successful trajectory with a Neural CDE whose projected input and latent-state dimensions are both 64. We directly use the released OAT implementation, which employs a three-layer MLP vector field, cubic interpolation, the Euler solver, and a four-layer control gate with hidden width 12. The query-conditioned initializer and hidden-state bridge are three-layer MLPs with hidden width 64; adjoint computation and initial-state norm regularization are disabled. We optimize the model with AdamW using the settings in Table 9. At evaluation time, the squared reconstruction error et=∥t−^t∥22e_t= _t- h_t _2^2 is used as the anomaly score for step t, and steps with larger scores are ranked earlier. StepFinder. StepFinder (Zhu et al., 2026) is a supervised lightweight attribution model that represents a trajectory as parallel sequences of step-content and agent-identity embeddings. We separately encode the content and agent name of each candidate step with Qwen3-Embedding-0.6B (Zhang et al., 2025b), using last-token pooling and a maximum input length of 8,192 tokens, and retain 128 and 32 dimensions, respectively. A two-layer BiLSTM with hidden size 64 per direction and dropout 0.5 captures the evolution of the trajectory. The resulting step representations are processed by a two-head agent-aware interaction module with head dimension 32. This module augments global step attention with an agent-similarity bias and a trajectory-level agent gate, both controlled by α=0.1α=0.1. StepFinder first predicts a learned error logit for each step and then refines it using normalized differences between steps at scales 1,2\1,2\ and a linearly decaying position bias that favors earlier root causes. We set the corresponding weights to β=0.9β=0.9 and γ=0.4γ=0.4. Training minimizes trajectory-level cross-entropy over the candidate steps together with an auxiliary loss for predicting the next-step representation; the auxiliary loss weight is λ=0.9λ=0.9. We use AdamW, gradient clipping at 1.0, and the optimization settings in Table 9. The same configuration is used across all reported benchmarks. Candidate steps are ranked by the resulting refined error scores. MASPrism. MASPrism (Liu et al., 2026b) is a training-free and oracle-free attribution method that extracts diagnostic signals during two prefill passes of a frozen language model. We follow its original two-stage scoring protocol without training an attribution head or using root-cause annotations. The Filtering stage uses step-level negative log-likelihood to identify the top 20% downstream symptom steps and attention from these symptoms to retain up to five earlier candidate sources. The Diagnosis stage restores the selected symptom and candidate contents, recomputes the prefill signals, and combines normalized attention, NLL contrast, and multi-symptom consensus to rank the candidate steps. We use a consensus top-55 cutoff and set the consensus coefficient to 0.3. We use the frozen Qwen3-0.6B (Yang et al., 2025) in bfloat16. Each trajectory is serialized with the model’s chat template without the ground-truth answer, and the maximum input length is set to 16,384 tokens. Following the original layer-selection rule, we average attention over all heads and the final 20% of the 28 Transformer blocks, corresponding to layers 22–27 under zero-based indexing. The Filtering stage retains up to 512 prefix tokens per step. During Diagnosis, the selected symptom and candidate steps are restored in full, while each remaining ordinary step retains up to 128 prefix tokens. These larger input budgets reduce truncation as a potential confound. The method performs only two prefill passes, requires no decoding, and is deterministic; the three reported seeds therefore produce identical predictions. ASCon. ASCon (Jiang et al., 2026) jointly contextualizes step and agent representations to support failure attribution. We use its step-level predictions under the failure attribution setting to localize the root-cause step. In our implementation, candidate-step text and each unique agent name are encoded separately with Qwen3.5-27B (Qwen Team, 2026) using last-token pooling to obtain 5,120-dimensional embeddings. Feature extraction uses bfloat16 weights, a maximum input length of 1,024 tokens, and an embedding batch size of 4. Consecutive candidate steps form a directed step graph, while transitions between their recorded actors form a directed agent graph. Under our unified candidate representation, the graphs contain only the retained candidate-step nodes and their corresponding actor nodes; the user query and user role are not instantiated as additional graph nodes. A two-layer direction-aware graph attention network separately aggregates preceding, succeeding, and self information to obtain contextualized step representations. ASCon next aggregates the contextualized representations of each agent’s steps through masked step-to-agent attention, producing a behavior-aware agent representation. It then incorporates the agent identity and global trajectory context and propagates information over the directed agent graph. The resulting representation of each agent is fused back into the representations of its steps, allowing step-level predictions to incorporate agent-specific behavioral context. We use two direction-aware graph attention layers, 256-dimensional step representations, 128-dimensional agent representations, a 128-dimensional step-to-agent attention space, and dropout 0.1. Separate step and agent classifiers are trained jointly with class-balanced binary cross-entropy using loss weights λstep=0.3 _step=0.3 and λagent=1.0 _agent=1.0. Optimization uses AdamW for 12 epochs with gradient clipping at 1.0 and the remaining settings in Table 9. Candidate steps are ranked by the step-classifier logits. Appendix G Additional Experimental Results This section collects additional experimental results omitted from the main text, including complete evaluation metrics, detailed subset analyses, backbone comparisons, and time efficiency analysis. G.1 MRR Across Rank Cutoffs Figure 4: MRR@K across rank cutoffs K=1,…,5K=1,…,5 on the four benchmarks. Curves show the cumulative reciprocal-rank score as the cutoff increases; error bars denote the population standard deviation over three seeds. MRR@K complements Hit@K by assigning a reciprocal-rank score to the root cause when it appears within the top K positions and zero otherwise, rewarding methods that place the root cause earlier within a fixed inspection budget. Figure 4 reports this metric for rank cutoffs from 1 to 5. ReCast achieves the highest MRR@K among the compared methods at every cutoff across all four benchmarks. On TraceElephant, ReCast reaches MRR@1 of 0.3124 and MRR@5 of 0.4341, compared with 0.2838 and 0.3860 for ASCon. G.2 Score-Level Discrimination The rank-based results above measure whether each trajectory’s root cause is prioritized over the other steps in the same trajectory. We additionally evaluate whether the raw attribution scores consistently separate root-cause steps from non-root steps after pooling all candidate steps within each benchmark’s evaluation population. Figures 5 and 6 show the resulting ROC and PR curves, respectively. The corresponding AUROC and AUPRC follow the definitions in Appendix E.1. Figure 5: Receiver operating characteristic (ROC) curves on the four benchmarks. Each curve is computed by pooling the candidate-step predictions within a benchmark. Solid lines show the mean over three seeds, and shaded regions indicate the population standard deviation. Figure 6: Precision–recall (PR) curves on the four benchmarks. Each curve is computed by pooling the candidate-step predictions within a benchmark. Solid lines show the mean over three seeds, and shaded regions indicate the population standard deviation. ReCast achieves the highest AUROC and AUPRC on all four benchmarks. Its advantage in AUPRC is particularly meaningful under class imbalance, as PR curves capture the trade-off between recovering root-cause steps and avoiding false positives. On Who&When and TraceElephant, it obtains AUROC values of 0.81390.8139 and 0.75830.7583 and AUPRC values of 0.29640.2964 and 0.21250.2125, respectively. On AFTraj-2K and Who&When Pro, ReCast reaches AUROC values of 0.97750.9775 and 0.98870.9887 together with AUPRC values of 0.90820.9082 and 0.96670.9667. Together, the MRR@K, AUROC, and AUPRC results support the effectiveness of ReCast for failure attribution. Its advantage extends from prioritizing the root-cause step within each trajectory to distinguishing root-cause steps from other candidates across trajectories. G.3 Hidden-State Backbone Comparison Table 10: Hidden-state backbone configurations. Each backbone retains eight layers selected independently using the same source-only procedure. Backbone # Layers Hidden size Selected layers Qwen3.5-0.8B 24 1,024 3, 5, 8, 11, 13, 18, 19, 22 Qwen3.5-9B 32 4,096 3, 7, 11, 16, 17, 23, 26, 31 Qwen3.5-27B 64 5,120 7, 15, 19, 32, 33, 47, 49, 57 Protocol. To examine whether attribution performance scales monotonically with backbone size, we compare Qwen3.5 models with 0.8B, 9B, and 27B parameters while holding the downstream ReCast pipeline fixed. For each backbone, we independently select eight layers using attribution-aware layer selection on ReCast-2K, set the projection dimension of each branch to dp=2,048d_p=2,048, and retain the same encoder architecture. Training and evaluation follow the same settings as the main experiments. Results are reported as mean ± population standard deviation over seeds 6, 20, and 42. Table 10 summarizes the backbone configurations and selected layers. Table 11: Effect of the hidden-state extraction backbone. Results report mean ± population standard deviation over three encoder seeds. Bold indicates the best mean within each evaluation population, determined before rounding. Evaluation population Backbone Hit@1 (%) Hit@3 (%) Hit@5 (%) MRR (%) AUROC (%) AUPRC (%) Who&When Handcrafted Qwen3.5-0.8B 30.46± 2.15 41.95± 2.15 55.75± 0.81 41.80± 1.07 80.00± 2.12 19.09± 0.84 Qwen3.5-9B 29.89± 3.54 44.83± 3.72 52.87± 4.30 41.61± 0.83 79.94± 2.41 18.55± 2.91 Qwen3.5-27B 33.33± 1.63 45.98± 0.81 57.47± 0.81 44.21± 0.77 79.22± 0.91 21.88± 0.92 Who&When Algorithm Qwen3.5-0.8B 38.17± 2.31 74.73± 2.31 91.94± 1.14 59.20± 2.00 72.00± 0.38 33.42± 0.43 Qwen3.5-9B 38.98± 0.38 74.46± 2.01 92.74± 2.37 59.85± 0.67 73.64± 1.58 35.10± 0.47 Qwen3.5-27B 45.43± 2.01 72.31± 1.01 92.47± 1.66 63.28± 1.42 74.75± 1.00 38.52± 1.84 TraceElephant Captain-Agent Qwen3.5-0.8B 38.43± 2.42 62.35± 4.19 78.82± 2.54 55.12± 2.14 81.07± 2.40 26.82± 2.49 Qwen3.5-9B 37.25± 2.00 62.35± 2.54 80.00± 1.66 54.54± 1.19 80.92± 2.03 27.42± 0.37 Qwen3.5-27B 35.69± 2.93 61.57± 1.47 78.04± 2.42 52.90± 1.13 78.30± 2.00 25.30± 0.75 TraceElephant Magentic-One Qwen3.5-0.8B 27.78± 0.91 42.22± 1.81 50.00± 1.57 39.81± 0.31 71.69± 1.92 18.87± 0.52 Qwen3.5-9B 28.89± 0.91 43.70± 2.92 54.81± 3.43 41.07± 1.30 71.61± 3.56 16.51± 1.76 Qwen3.5-27B 27.04± 1.89 46.67± 1.57 52.96± 2.10 40.52± 1.16 72.78± 0.58 17.98± 1.99 AFTraj-2K Qwen3.5-0.8B 82.82± 1.81 94.68± 0.29 97.96± 0.77 89.05± 1.06 97.63± 0.08 88.05± 1.38 Qwen3.5-9B 83.64± 1.61 94.89± 1.26 98.16± 0.50 89.72± 0.95 97.88± 0.24 90.54± 0.75 Qwen3.5-27B 84.05± 0.87 95.09± 1.00 97.96± 0.29 89.99± 0.49 97.75± 0.21 90.82± 0.58 Who&When Pro Qwen3.5-0.8B 87.93± 0.49 95.31± 0.14 98.13± 0.19 92.09± 0.29 97.84± 0.11 94.25± 0.21 Qwen3.5-9B 89.90± 0.04 96.16± 0.36 98.29± 0.08 93.40± 0.09 98.35± 0.06 95.61± 0.07 Qwen3.5-27B 91.42± 0.21 96.64± 0.26 98.53± 0.14 94.36± 0.11 98.87± 0.12 96.67± 0.17 Results. Table 11 shows that the benefits of a larger backbone vary across benchmarks and metrics. Qwen3.5-27B achieves the highest Hit@1 on both Who&When subsets, AFTraj-2K, and Who&When Pro, with all six metrics improving with backbone size on Who&When Pro. However, smaller backbones remain competitive: 0.8B and 9B achieve the highest Hit@1 on Captain-Agent and Magentic-One, respectively. On AFTraj-2K, their Hit@1 scores are within 1.23 percentage points of 27B, and 9B achieves the highest Hit@5 and AUROC. These results suggest that larger backbones can improve attribution performance, although the gains vary across benchmarks and metrics. With attribution-aware layer selection and representation learning, smaller backbones can also provide effective signals for failure attribution. G.4 Time Efficiency Analysis Figure 7: Hit@1 accuracy (%) versus measured time per trajectory on Who&When and TraceElephant. Points show three-seed means; time is in seconds on a logarithmic scale. Feature-based methods report attribution time excluding feature extraction, while LLM methods report inference time excluding backend initialization. Upper-left positions indicate higher accuracy at lower measured cost. A and S denote All-at-Once and Step-by-Step. Protocol. We compare Hit@1 accuracy with the average processing time per trajectory on Who&When and TraceElephant. ReCast, StepFinder, ASCon, and OAT use precomputed features, excluding feature extraction and embedding-cache preparation from attribution time. For MASPrism, we report attribution time excluding model initialization, tokenization, and prefill-stage signal extraction. For LLM-based methods, inference time excludes model initialization but includes prompt preparation and API delays and retries where applicable. We divide the total measured time by the number of evaluated trajectories and report the mean over three seeds in Figure 7. Results. ReCast achieves the highest Hit@1 across all four subsets with mean attribution times ranging from 0.11 to 8.28 ms per trajectory. It has the lowest mean attribution time among the compared feature-based methods on HC, AG, and Magentic-One. On Captain-Agent, StepFinder and ASCon are faster, but ReCast achieves higher accuracy. In contrast, the LLM-based baselines shown in Figure 7 require 2.88–30.50 seconds per trajectory under the measured configurations while attaining lower Hit@1. This comparison highlights the efficiency of ReCast’s attribution stage when step features are available. Appendix H Case Study Figure 8: Step-level attribution scores on three HC trajectories (a–c) and three AG trajectories (d–f). Original zero-based step indices are retained; gaps indicate events without candidate scores. Figure 8 presents six cases from the Handcrafted and Algorithm subsets of Who&When. The curves show ReCast’s attribution scores for all candidate steps, with higher scores indicating higher ranks within each trajectory. In these selected cases, ReCast ranks the annotated root-cause step above other high-scoring candidates, providing qualitative support for the effectiveness of its learned step representations in failure attribution. HC-50, restaurant search (Figure 8(a)). The task asks for dine-in restaurants within one block of Washington Square Park that serve vegan main dishes for under $15. At step 5, a location-only search returns general listings without verifying the remaining constraints; the dataset annotates this retrieval as the root cause. Later steps also receive elevated scores. For example, the summary at step 106 presents proximity and dine-in service as confirmed while leaving vegan options and prices for further verification. ReCast nevertheless assigns its highest score to step 5, ranking the earlier annotated error above this later competing peak. HC-37, missing a source constraint (Figure 8(b)). The task asks for an organism’s maximum body length in meters, as reported by the Monterey Bay Aquarium website. At step 4, the agent searches for National Geographic videos without including the required source, an omission annotated as the root cause. At step 22, it opens a Human Origins video whose displayed content does not provide the requested measurement. Despite other local score peaks, ReCast ranks step 4 first, prioritizing the earlier annotated omission over the later browsing event. HC-42, premature completion (Figure 8(c)). The task asks which word was deleted in a federal rule amendment documented on Cornell Law School’s website. Browsing the Federal Rules index at step 16 and the Witnesses article at step 24 produces local score peaks. At step 29, however, the Orchestrator declares the task complete while acknowledging that the retrieved page does not identify the requested word. This premature completion is the annotated root cause. ReCast assigns its highest score to step 29, ranking the unsupported completion decision above the earlier browsing steps. AG-116, substituting simulated data (Figure 8(d)). The task asks for the lowest sale price of a single-family house in Queen Anne in January 2023. With the transaction file unavailable, step 3 proposes analysis using an assumed CSV path and receives an elevated score. At step 7, the agent constructs simulated transactions instead of obtaining the actual records; the dataset annotates this substitution as the root cause. ReCast assigns its highest score to step 7, ranking the use of simulated data above the earlier analysis based on an assumed file path. AG-109, incorrect distance conversion (Figure 8(e)). The task seeks supermarkets within two blocks of Lincoln Park, Chicago, that offer ready-to-eat salads for under $15. At step 1, the agent converts miles to blocks using an assumed rate of 20 blocks per mile, which the dataset identifies as incorrect for Chicago. A later discussion of candidate stores at step 5 also produces a local score peak. ReCast nevertheless ranks step 1 highest, selecting the annotated conversion error rather than the subsequent store recommendations. AG-120, unverified restaurant claims (Figure 8(f)). AG-120 addresses the same restaurant-search task as HC-50 through a different execution. At step 7, a missing Google Maps API key blocks automated verification, and the agent proposes manual checks. At step 8, another agent presents distances, dining options, and menu prices as verified without performing the required checks, which the dataset annotates as the root cause. ReCast’s score rises at step 7 but peaks at step 8, prioritizing the unsupported claims over the preceding verification obstacle. Appendix I Prompt Templates We directly use the Who&When implementation (Zhang et al., 2025a) and present its prompt templates below. The red fields denote trajectory-specific values inserted at inference time. The ground_truth field contains the benchmark-provided task answer, not the root-cause annotation. Line wrapping below is typographical and does not change the prompt content. I.1 All-at-Once Prompt All-at-Once presents the complete trajectory in one request and asks the model to identify the responsible agent and the first erroneous step. All-at-Once Evaluation Template SYSTEM MESSAGE You are an AI assistant skilled in analyzing conversations. USER MESSAGE You are an AI assistant tasked with analyzing a multi-agent conversation history when solving a real world problem. The problem is: problem The Answer for the problem is: ground_truth Identify which agent made an error, at which step, and explain the reason for the error. Here’s the conversation: conversation Based on this conversation, please predict the following: 1. The name of the agent who made a mistake that should be directly responsible for the wrong solution to the real world problem. If there are no agents that make obvious mistakes, decide one single agent in your mind. Directly output the name of the Expert. 2. In which step the mistake agent first made mistake. For example, in a conversation structured as follows: "agent a": "x", "agent b": "x", "agent c": "x", "agent a": "x" , each entry represents a ‘step’ where an agent provides input. The ‘x’ symbolizes the speech of each agent. If the mistake is in agent c’s speech, the step number is 2. If the second speech by ‘agent a’ contains the mistake, the step number is 3, and so on. Please determine the step number where the first mistake occurred. 3. The reason for your prediction. Please answer in the format: Agent Name: (Your prediction) Step Number: (Your prediction) Reason for Mistake: I.2 Step-by-Step Prompt Step-by-Step evaluates growing trajectory prefixes. At each step, the model receives the history up to and including that step. Evaluation terminates at the first response parsed as “Yes”; otherwise, it advances to the next prefix. Step-by-Step Evaluation Template SYSTEM MESSAGE You are a precise step-by-step conversation evaluator. USER MESSAGE AT STEP step You are an AI assistant tasked with evaluating the correctness of each step in an ongoing multi-agent conversation aimed at solving a real-world problem. The problem being addressed is: problem. The Answer for the problem is: ground_truth Here is the conversation history up to the current step: conversation The most recent step (step) was by ’agent’. Your task is to determine whether this most recent agent’s action (Step step) contains an error that could hinder the problem-solving process or lead to an incorrect solution. Please respond with ’Yes’ or ’No’ and provide a clear explanation for your judgment. Note: Please avoid being overly critical in your evaluation. Focus on errors that clearly derail the process. Respond ONLY in the format: 1. Yes/No. 2. Reason: [Your explanation here]