Paper deep dive
PATH-Bench: Path-Dependent Evaluation of Lifelong Agents
Xidong Yang, Xingyi Zhang, Wenhao Li, Wenyan Liu, Junjie Sheng, Yun Hua, Wei Yin, Tao Fang, Chuyun Shen, Xiangfeng Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Lifelong LLM agents increasingly adapt through external learning states that store past interactions as retrievable memories or reusable skills, yet existing benchmarks rarely account for how the path of accumulated experience shapes what agents transfer and retain. In this work, we establish PATH-Bench, a benchmark for path-dependent evaluation of lifelong agents. PATH-Bench estimates directed task relationships via multi-model in-context learning, constructs probe-centered sequences with controlled helpful and interfering histories, and repeatedly evaluates probe tasks to measure average performance, forward transfer, backward transfer, and forgetting. We evaluate eight representative agents on single-turn code generation and multi-turn tool-use tasks under positive- and negative-dominant histories. Benchmark results show that experience utility depends jointly on how experience is represented and on the task's interaction structure, that strong transfer does not ensure retention, and that later experience can reshape gains acquired earlier in the learning path. Based on these findings, we propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task, admitting helpful items while filtering out potential interference. SEU consistently reduces forgetting while improving forward transfer in the majority of settings. The PATH-Bench provides both a controlled evaluation framework and actionable guidance for designing more selective and robust lifelong agents.
Tags
Links
- Source: https://arxiv.org/abs/2608.01149v1
- Canonical: https://arxiv.org/abs/2608.01149v1
Trouble viewing inline? Open PDF directly →
Full Text
58,709 characters extracted from source content.
Expand or collapse full text
PATH-Bench: Path-Dependent Evaluation of Lifelong Agents Xidong Yang 1∗† , Xingyi Zhang 1∗ , Wenhao Li 2 , Wenyan Liu 3 , Junjie Sheng 4 , Yun Hua 5 , Wei Yin 6 , Tao Fang 3 , Chuyun Shen 7 , Xiangfeng Wang 1,8† 1 East China Normal University 2 Tongji University 3 Ant Group 4 Independent Researcher 5 Shanghai Jiao Tong University 6 Bank of Communications 7 Shanghai University of International Business and Economics 8 Shenzhen Loop Area Institute xdyang@stu.ecnu.edu.cn, xfwang@cs.ecnu.edu.cn Abstract Lifelong LLM agents increasingly adapt through external learning states that store past interactions as retrievable mem- ories or reusable skills, yet existing benchmarks rarely ac- count for how the path of accumulated experience shapes what agents transfer and retain. In this work, we establish PATH-Bench, a benchmark for path-dependent evaluation of lifelong agents. PATH-Bench estimates directed task rela- tionships via multi-model in-context learning and constructs probe-centered sequences with controlled helpful and interfer- ing histories, scoring all tasks for average performance while tracking recurring probes to measure forward transfer, back- ward transfer, and forgetting. We evaluate eight representative agents on single-turn code generation and multi-turn tool-use tasks under positive- and negative-dominant histories. Bench- mark results show that experience utility depends jointly on how experience is represented and on the task’s interaction structure, that strong transfer does not ensure retention, and that later experience can reshape gains acquired earlier in the learning path. Based on these findings, we propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task, ad- mitting helpful items while filtering out potential interference. SEU consistently reduces forgetting while improving forward transfer in the majority of settings. PATH-Bench provides both a controlled evaluation framework and actionable guidance for designing more selective and robust lifelong agents. 1 Introduction Across the AI industry, a growing vision is to develop per- sistent agents that learn from experience, build memories, and improve over time across repeated interactions (OpenAI 2026b). This motivates lifelong LLM agents that accumulate and preserve useful experience over time (Zheng et al. 2026; Asawa et al. 2026; Xu et al. 2026b). One approach is to inter- nalize such experience through continual parameter updates, but doing so at LLM scale is costly and risks catastrophic for- getting (Li et al. 2024; Zheng et al. 2025b; Shi et al. 2025). ∗ These authors contributed equally. † Corresponding author. APFWTBWTFGT BGE-M3 RAG HippoRAG- v2 Clin AWM 4→34→21→12→2 3→22→13→31→1 1→41→32→43→4 2→13→44→24→3 −3−2−10123 Rank improvement (Path 1 rank − Path 2 rank) Figure 1: Agent-rank changes across two task paths on Life- longAgentBench (Zheng et al. 2025a). Each cell shows the transition from Path 1 to Path 2 for average performance (AP), forward transfer (FWT), backward transfer (BWT), and for- getting (FGT); color encodes its direction and magnitude. To avoid these limitations, many lifelong agents keep the backbone frozen and instead realize adaptation through the agent harness, which maintains an external learning state by storing past interactions as retrievable memories or reusable skills (Hu et al. 2025b; Yang et al. 2026b,a). However, their effectiveness depends on what is retained, how it is repre- sented, and the path through which experience accumulates. Despite its central role in continual learning (Bell and Lawrence 2022; Benavides-Prado and Riddle 2022), path dependence remains largely overlooked in evaluations of lifelong LLM agents. Specifically, existing benchmarks do not control the task histories over which the agent har- ness updates its external learning state before evaluating later tasks (Wei et al. 2025; Asawa et al. 2026; Xu et al. 2026b). Positive experience may enable positive transfer, whereas misleading memories, outdated strategies, or con- flicting skills may induce persistent interference. To probe this sensitivity, we conduct a preliminary study on Lifelon- gAgentBench (Zheng et al. 2025a), the first unified bench- mark for systematically assessing LLM agents as lifelong arXiv:2608.01149v1 [cs.AI] 2 Aug 2026 learners, and find that different task paths over an identical task set can change the relative ranking of agents (Figure 1; see Appendix A for details). Consequently, evaluations over fixed or randomized paths cannot reveal how the accumu- lation of prior experience shapes later performance (Baker et al. 2023). Characterizing this dependence requires tracing perfor- mance along controlled histories with metrics such as for- ward transfer, backward transfer, and forgetting (Zuffer, Burke, and Harandi 2025; Wołczyk et al. 2021). However, applying these analyses at scale to LLM agents is compu- tationally challenging. Evaluating a single agent task may already involve multi-step reasoning, tool use, environment interaction, and feedback (He, Treude, and Lo 2025; Hu et al. 2025a), while traditional lifelong learning metrics require re- peatedly reevaluating the full task set throughout the learning trajectory (Pan et al. 2025). The key challenge is to efficiently evaluate how task path dependence shapes the lifelong per- formance of LLM agents, which is essential for assessing their ability to learn and adapt reliably from accumulated experience. To address this challenge, we introduce PATH-Bench, a benchmark for path-dependent evaluation of lifelong agents. The core idea is to replace exhaustive enumeration of task his- tories with sampling guided by task relationships, and to re- place repeated evaluation of the full task set with longitudinal tracking of a single probe. This design enables PATH-Bench to efficiently evaluate how task path dependence shapes the lifelong performance of LLM agents. Across eight representative lifelong agents spanning retrieval-augmented generation (RAG), agentic memory, and skill-based approaches, we find that accumulated experience does not consistently improve performance, and its utility de- pends on whether the representation of experience matches the task’s interaction structure. We further observe a dis- sociation between transfer and retention, as strong imme- diate transfer can coexist with substantial forgetting. Con- trolled path-transition experiments further show that later experience can reshape gains acquired earlier in the learn- ing path. Guided by these findings, we propose Selective Experience Use (SEU), an agent harness that regulates how path-accumulated experience influences each new task, ad- mitting helpful items in concrete or abstracted form while filtering out potential interference. SEU consistently reduces forgetting across agents, datasets, and transfer histories, while improving forward transfer in the majority of evaluated set- tings, showing how PATH-Bench diagnostics can guide the design of lifelong agent harnesses that regulate the use of accumulated experience. Overall, the main contributions are summarized as follows: 1) PATH-Bench is introduced, to the best of our knowledge the first benchmark for tracing transfer and retention in LLM agents under controlled histories: it estimates task relations with multi-model in-context learning (ICL), samples con- figurable relation-constrained sequences, and quantifies the resulting performance changes; 2) A controlled empirical study of eight representative lifelong agents is conducted, disentangling how the external learning state, its representa- tion, and the learning path jointly shape transfer and reten- tion; 3) Guided by these findings, we propose SEU, an agent harness that regulates how path-accumulated experience in- fluences each new task, reducing forgetting and improving forward transfer in the majority of evaluated settings. 2 Related Work Lifelong Learning. Lifelong learning studies how sys- tems acquire knowledge from task streams while balanc- ing plasticity and stability, commonly analyzed through knowledge transfer and forgetting (Parisi et al. 2019; Wang et al. 2024a; Lopez-Paz and Ranzato 2017). Classical ap- proaches rely on regularization, experience replay, or param- eter isolation (Kirkpatrick et al. 2017; Ye and Bors 2023, 2022), but frequent online updates to large language mod- els are costly and may destabilize their general capabili- ties (Luo et al. 2025). LLM agents adapt by evolving ex- ternal states—memories, workflows, and skills—around a frozen backbone (Park et al. 2023; Zhang et al. 2026b; Yang et al. 2026b; Ma et al. 2026). Evolution of LLM Agents. Memory and skills are two primary substrates for experience-driven agent evolu- tion (Zhong et al. 2024; Sumers et al. 2024). Memory meth- ods differ in representation and update strategy: AWM (Wang et al. 2024b) extracts reusable workflows, Clin (Majumder et al. 2023) distills causal rules, SimpleMem (Liu et al. 2026) compresses interactions into structured guidance, and MemRL (Zhang et al. 2026b) updates memory utility from environmental rewards; broader frameworks study memory formation, revision, and retrieval over time (Zhang et al. 2025; Li et al. 2025; Du et al. 2025; Cai et al. 2025). Skill- based agents abstract trajectories into reusable skills that can be discovered, routed, and refined without modifying the backbone (Yang et al. 2026b; Ma et al. 2026; Alzubi et al. 2026; Zhang et al. 2026a). Lifelong Learning Benchmarks. Existing LLM-agent life- long learning benchmarks span three related directions. Long-horizon memory benchmarks evaluate whether mod- els retain and integrate information distributed across dia- logues or agent-environment trajectories (Maharana et al. 2024; Castillo-Bolado et al. 2024; Wan and Ma 2025; Zhao et al. 2026). Sequential adaptation benchmarks test whether agents improve from feedback streams, using performance trajectories, transfer, retention, or gains beyond underlying model capability (Wu et al. 2024; Zheng et al. 2025a; Ai et al. 2025; Asawa et al. 2026), with EdgeBench further scaling this setting to ultra-long real-world tasks that accu- mulate environment interaction (Zhu et al. 2026). Dynamic and self-evolving benchmarks further study environment changes, memory updates, and capability erosion across mul- tiple forms of agent state (Xu et al. 2026b; Yu et al. 2026b). Despite this breadth, existing benchmarks generally can not measure how task path dependence shapes both transfer and retention, while PATH-Bench is designed to close the gap. 3 The PATH-Bench Benchmark The overview framework of PATH-Bench is provided in Fig- ure 2. PATH-Bench uses an ICL-based transfer gain matrix Figure 2: Overview of PATH-Bench. The framework filters both task pools for measurable headroom, estimates directed task relationships with a multi-model in-context transfer gain matrix, samples probe-centered sequences under positive- and negative- dominant histories, and evaluates agents that adapt through an external learning state with average performance, forward transfer, backward transfer, and forgetting. to estimate directed task relationships and construct probe- centered sequences with configurable transfer patterns. By repeatedly evaluating probes under controlled histories, it measures average sequence performance and characterizes learning dynamics through forward transfer, backward trans- fer, and forgetting. 3.1 Problem Formulation Consider an LLM agent that solves a stream of tasks while accumulating experience from the tasks it has processed. Its behavior on any task therefore depends not only on that task but on the ordered history of tasks encountered before it. Let q be a fixed probe task, H = (h 1 ,...,h k ) an ordered history of intervening tasks that the agent processes before q, andA H denotes the agent after processing H. The expected performance ofA H on q is defined as: P q (H) = E ρ(y,q)|A H ,(1) where y is the agent’s response to q and ρ(y,q) scores its success. Performance is path-dependent when P q (H) varies with the content and order of H. In a lifelong sequence, the i-th occurrence of the probe sees a history prefixH i , yielding the trajectory p probe i = P q (H i ). PATH-Bench evaluates an agent by contrasting this probe trajectory under controlled histories. 3.2 Benchmark Construction Pipeline Task Selection. To evaluate the lifelong learning capabilities of LLM agents across distinct capability domains, we select two representative public benchmarks with substantially dif- ferent task characteristics. For code generation, we employ BigCodeBench (Zhuo et al. 2025), a single-turn benchmark that evaluates practical program synthesis under complex in- structions and diverse function calls, scoring each task by whether it is completed. For tool use, we select WildTool- Bench (Yu et al. 2026a), a multi-turn benchmark grounded in real-world user behavior and targeting planning and exe- cution on complex, cross-tool tasks, scoring each task by the proportion of successfully completed subtasks. To ensure that lifelong agents have a meaningful opportunity to improve, we follow the headroom principle in continual learning bench- mark design (Asawa et al. 2026): we estimate the baseline performance of strong LLMs on candidate tasks and filter out those already saturated or offering little observable learning signal. From the resulting headroom-aware pool, we sample D = 120 tasks from each benchmark to form the evaluation pool. Task Relationship. Existing agent evaluations often rely on randomized task sequences that ignore inter-task relation- ships, making it difficult to study how earlier experience facilitates or interferes with later tasks. PATH-Bench instead models these relationships to construct evaluation sequences that probe transfer and forgetting. Because directly estimating pairwise transfer through repeated agent evolution is compu- tationally expensive (Wołczyk et al. 2021), we use one-shot in-context learning effects as a practical proxy for inter-task transferability (Zhao et al. 2025). For each ordered pair of pool tasks (t i ,t j ), t i serves as a one-shot demonstration for a frozen LLM, and the in-context transfer gain of t i on t j is defined as M i,j = S(t j | t i )− S(t j | ∅),(2) where S(t j | t i ) and S(t j | ∅) denote the demonstration- conditioned and zero-shot performance on t j , respectively. A gain matrixM ∈ R D×D over the D tasks in the pool is estimated independently for DeepSeek-V4-Flash (Xu et al. 2026a), GPT-5.4-mini (OpenAI 2026a), and GLM-5.0 (Zeng et al. 2026), with every entry averaged over five random seeds. Each model then casts a vote for the pair according to the sign of its own averaged gain: positive when M i,j > 0, neutral when M i,j = 0, and negative when M i,j < 0. Any relation supported by at least two models fixes the final entry V i,j , while pairs receiving one vote of each type are labeled mixed and excluded from relation-constrained sampling. Ap- pendix B reports two complementary validations of the vot- ing matrixV: cross-model consistency, and preservation of the one-shot signal under multi-task composition. 3.3 Evaluation Protocol Sequence Sampling. The voting matrixV supports a config- urable, probe-centered sequence construction protocol. The sampler first selects an eligible task T probe = t j as the probe task and partitions the remaining tasks t i into positive- , neutral-, and negative-transfer sets according to V i,j . It then constructs an intervening-task history with the speci- fied length and transfer composition and reinserts T probe at configured probe intervals. Only the intervening tasks up- date the agent’s external learning state; repeated occurrences of T probe serve exclusively as evaluation probes. Changes in probe performance therefore reveal how the accumulated history facilitates or interferes with the same task. Average performance summarizes the score across all tasks in the se- quence, whereas forward transfer, backward transfer, and for- getting are read from the probe trajectory alone. This allows users to define evaluation scenarios through transfer proper- ties rather than specific task identities; in our experiments, we instantiate positive- and negative-dominant histories. Let p probe i denote the probe score at its i-th occurrence, out of n occurrences in total. Each generated sequence opens with a probe before any sequence-specific experience, giv- ing the cold-start baseline p cold ≡ p probe 1 . A short positive- transfer warm-up block then lets the agent acquire useful probe-related experience, and the second probe yields the post-warm-up reference p warm ≡ p probe 2 . The remaining oc- currences i = 3,...,n, interleaved with intervening tasks at the configured intervals, are the measurement probes: for- ward transfer is scored against the cold start p cold , while backward transfer and forgetting are scored against the post- warm-up reference p warm . Evaluation Metrics. We evaluate agents using one over- all performance metric and three lifelong learning metrics. A lifelong sequence comprises L task executions in total, counting the k intervening tasks together with all n occur- rences of the probe; let p ℓ denote the performance score obtained by an agent on the ℓ-th execution. - Average Performance (AP): AP denotes the agent’s average performance score across all task executions, including both probe and intervening tasks, while higher values indicate stronger overall task-solving ability, i.e., AP = 1 L L X ℓ=1 p ℓ . - Forward Transfer (FWT): FWT denotes whether the inter- vening task history improves later occurrences of the probe task relative to its initial performance, while higher values indicate stronger positive transfer, i.e., FWT = 1 n− 2 n X i=3 (p probe i − p cold ). - Backward Transfer (BWT): BWT denotes the beneficial retroactive effect of later experience, averaging the amount by which subsequent probes rise above the post-warm-up ref- erence, while higher values indicate stronger positive back- ward transfer, i.e., BWT = 1 n− 2 n X i=3 max 0, p probe i − p warm . - Forgetting (FGT): FGT measures how much of the acquired probe-related capability is lost, averaging how far subsequent probes fall short of the post-warm-up reference, while lower values indicate better knowledge retention, i.e., FGT = 1 n− 2 n X i=3 max 0, p warm − p probe i . 4 Experiments PATH-Bench is instantiated on two task datasets: Big- CodeBench for code generation and WildToolBench for tool- use scenarios. For each dataset, we construct sequences under positive- and negative-dominant transfer histories. Following the protocol in Section 3.3, we set the positive-transfer warm- up length to five tasks, sample an intervening queue of 100 tasks, and use probe intervals of 6-12 intervening tasks. Each condition draws 70% of intervening tasks from its dominant transfer set (positive or negative) and the remainder at ran- dom from the other two sets. For each dataset and condition, we sample 50 lifelong evaluation sequences and evaluate each sequence over five runs. Furthermore, DeepSeek-V4- Flash is employed as the base inference model for all evalu- ated agents. Three families of lifelong agents are compared: - RAG Agents: include BGE-M3 RAG (Chen et al. 2024) and HippoRAG-v2 (Gutiérrez et al. 2025). BGE-M3 RAG is a simple retrieval baseline that encodes historical interac- tion trajectories with BGE-M3 model and retrieves the most similar ones by cosine similarity; - Agentic Memory Agents: include Clin (Majumder et al. 2023), AWM (Wang et al. 2024b), SimpleMem (Liu et al. 2026), and MemRL (Zhang et al. 2026b), which maintain and evolve structured memory representations; - Skill Agents: include AutoSkill (Yang et al. 2026b) and SkillClaw (Ma et al. 2026), which accumulate reusable pro- cedural capabilities. Dataset Method Positive DominantNegative Dominant AP (↑) FWT (↑) BWT (↑) FGT (↓) AP (↑) FWT (↑) BWT (↑) FGT (↓) BigCode Bench DeepSeek-V4-Flash 70.18–67.67– BGE-M3 RAG69.97 0.706.21 11.51 67.37 5.063.13 18.06 HippoRAG-v270.52 4.5910.31 5.32 67.85 7.035.99 16.15 Clin68.92 3.227.75 10.13 67.11 6.996.36 16.16 AWM65.56 −4.67 5.828.50 61.95 −8.36 5.38 12.53 SimpleMem68.98 1.449.727.88 66.44 5.816.92 12.70 MemRL71.02 0.776.179.00 68.89 5.357.97 10.61 AutoSkill73.93 6.676.917.04 71.01 6.737.437.90 SkillClaw70.75 −1.89 6.507.99 69.46 −4.74 7.57 11.10 WildTool Bench DeepSeek-V4-Flash 42.80–43.00– BGE-M3 RAG45.52 −1.48 3.704.88 45.32 −3.09 2.394.57 HippoRAG-v241.70 −2.61 2.953.86 41.67 −0.54 2.023.66 Clin46.29 0.733.062.63 46.40 −0.63 3.072.30 AWM42.86 0.013.102.69 42.84 2.023.251.83 SimpleMem46.18 0.092.573.28 46.21 −0.62 2.942.75 MemRL44.37 −3.44 2.215.65 44.82 −5.32 2.264.58 AutoSkill42.38 −5.08 2.144.72 42.71 −3.89 1.183.96 SkillClaw41.11 −0.77 2.913.28 40.85 −1.21 2.503.71 Table 1: Lifelong learning performance on BigCodeBench and WildToolBench under positive- and negative-dominant interfer- ence. DeepSeek-V4-Flash serves as the memory-free baseline. AP denotes Average Performance (%), while FWT, BWT, and FGT are reported in percentage points (p). Arrows indicate whether higher or lower values are better. Best results within each category are bolded. 4.1 Main Results Table 1 summarizes the lifelong learning performance of the evaluated agents across both datasets and transfer-history conditions. The main experiment compares lifelong agent harnesses across controlled experience paths, examining how overall performance, transfer, and retention vary across task settings and harness designs. ① External experience does not universally improve lifelong learning performance. Figure 4 reports each agent’s AP change relative to the memory-free baseline. The memory-free baseline achieves 70.18% and 67.67% AP on BigCodeBench, and 42.80% and 43.00% on WildTool- Bench, under positive- and negative-dominant histories, re- spectively. On BigCodeBench, BGE-M3 RAG, Clin, AWM, and SimpleMem underperform the baseline in both con- ditions; on WildToolBench, HippoRAG-v2, AutoSkill, and SkillClaw do so in both conditions, while AWM falls slightly below it under negative-dominant histories. Because under- performance occurs across RAG agents, agentic memory agents, and skill agents alike, the bottleneck is not the ca- pacity to store experience but the ability to judge whether stored experience is useful for the current task. Irrelevant, conflicting, or overly task-specific experience can offset the benefits of reuse, so the central challenge is to prevent un- suitable experience from influencing current decisions rather than to accumulate more of it. ② The useful representation of experience depends on the interaction structure of the task. On the single- turn BigCodeBench, agents that retain task-specific expe- rience dominate: AutoSkill achieves the highest AP in both conditions and the strongest positive-dominant FWT (6.67 p), HippoRAG-v2 obtains the best positive-dominant BWT (10.31 p) and the lowest positive-dominant FGT (5.32 p), and MemRL, which stores utility-weighted interaction expe- rience, ranks second in AP (71.02%). The pattern reverses on the multi-turn WildToolBench, where agents that distill experience into compact abstrac- tions lead: Clin, which maintains causal rules, achieves the highest AP in both conditions, while AWM, which extracts reusable workflows, obtains the best negative-dominant FWT (2.02 p) and BWT (3.25 p). HippoRAG-v2, AutoSkill, and SkillClaw, which all improve over the memory-free baseline on BigCodeBench, fall below it on WildToolBench. The per- metric best values partition accordingly: no agent holds a best value on both datasets. Task-specific experience grounded in concrete interactions is thus more effective for single-turn code tasks, whereas higher-level abstractions detached from individual trajectories are more effective for multi-turn tool tasks. ③ Transfer and retention dissociate, and agents without experience selection are most vulnerable to forgetting. Traditional continual learning treats transfer and retention as complementary dimensions (Lopez-Paz and Ranzato 2017); we observe the same dissociation when a frozen-backbone LLM adapts through an external learning state. On Big- CodeBench, HippoRAG-v2 achieves the highest negative- dominant FWT (7.03 p) but still exhibits substantial forget- ting (16.15 p), whereas AutoSkill and MemRL forget far 20 10 0 10 20 BGE-M3 RAGHippoRAG-v2ClinAWM 23456789101112 20 10 0 10 20 SimpleMem 23456789101112 MemRL 23456789101112 AutoSkill 23456789101112 SkillClaw Change from post-warm-up reference (p) Probe occurrence PositiveNegative Figure 3: Probe reappearance dynamics on BigCodeBench under positive- and negative-dominant histories. Curves report the change from the post-warm-up reference p warm (in percentage points) at successive probe occurrences. −6−303 ΔAP vs. memory-free baseline (p) BGE-M3 RAG HippoRAG-v2 Clin AWM SimpleMem MemRL AutoSkill SkillClaw BCB · Pos. BCB · Neg. WTB · Pos. WTB · Neg. Figure 4: Change in average performance relative to the memory-free DeepSeek-V4-Flash baseline. BCB and WTB denote BigCodeBench and WildToolBench, respectively. Positive values indicate improvement. less (7.90 p and 10.61 p) while transferring less strongly (6.73 p and 5.35 p). On WildToolBench, BGE-M3 RAG and MemRL exceed the baseline in AP yet record nega- tive FWT and the two highest FGT values, so their overall advantage does not reflect stable probe-task gains. Under negative-dominant histories on BigCodeBench, AutoSkill and MemRL, both of which apply a selection step before injection, forget less than the two RAG agents, which pass retrieved trajectories through unfiltered. However, MemRL’s high FGT on WildToolBench shows that selection alone is insufficient when the representation does not match the task structure (Finding②), indicating that both representation and selection jointly determine retention along the learning path. 4.2 Path-Dependent Dynamics ① Probe reappearance dynamics. Figure 3 traces how each agent’s probe-task performance on BigCodeBench evolves relative to the post-warm-up reference p warm as intervening experience accumulates. The horizontal axis starts at occur- rence 2 (p warm ) and covers the first twelve occurrences shared by all 50 sequences; Appendix C reports the corresponding WildToolBench curves. Probe-task performance erodes along the learning path, even when the path is predominantly helpful. Ev- ery curve starts at p warm by construction. Under positive- dominant histories, five of eight agents still drift below p warm despite most intervening tasks being helpful; only HippoRAG-v2 and SimpleMem stay clearly above it (5.16 p and 1.96 p). Under negative-dominant histories the erosion is stronger and universal: every agent ends belowp warm , with BGE-M3 RAG declining by 14.28 p and AutoSkill the most stable at 0.80 p. This within-path degradation shows that accumulating experience progressively reshapes an agent’s external state, and that acquiring a useful probe-related ca- pability does not guarantee retaining it as the path continues. ② Controlled path-transition analysis. To isolate path- transition effects beyond the sustained histories above, we construct matched sequences on BigCodeBench:X → I 1 → X → I 2 → X, where X is the probe task and each five-task interval is sampled entirely from its positive- or negative-transfer set. This yields four transition conditions: Pos→Pos, Pos→Neg, Neg→Pos, and Neg→Neg, all shar- ing the same probe task so that cross-condition comparisons are controlled. We evaluate each agent on 30 matched se- quences with five runs per sequence. Figure 5 reports the mean changes p probe 2 − p probe 1 and p probe 3 − p probe 2 . Agents are more susceptible to recent experience, and positive transfer is easier to restore than to preserve. A positive-transfer interval consistently improves all eight agents at the next probe, confirming that the transfer gain matrix identifies genuinely helpful experience. A negative- transfer interval produces agent-dependent responses: Clin, 402002040 Mean effect (p) BGE-M3 RAG HippoRAG-v2 Clin AWM SimpleMem MemRL AutoSkill SkillClaw Initial positive interval p probe 2 p probe 1 (PosPos / PosNeg) +5.0 +7.0 +5.0 +0.7 +12.7 +10.3 +10.0 +2.3 402002040 Mean effect (p) Initial negative interval p probe 2 p probe 1 (NegPos / NegNeg) -14.7 -6.7 +4.7 -7.7 -1.3 +5.3 +8.0 +2.3 402002040 Mean effect (p) Positive-to-negative transition p probe 3 p probe 2 (PosNeg) -2.7 -6.7 -0.7 -1.3 -10.0 +0.0 +7.3 +0.7 402002040 Mean effect (p) Negative-to-positive transition p probe 3 p probe 2 (NegPos) +12.0 +7.3 +4.7 -2.0 +2.0 +4.7 +6.7 +2.0 Mean > 0Mean < 0Mean = 0 Figure 5: Controlled path-transition effects on BigCodeBench. Points and error bars denote means and 95% confidence intervals. AgentHist. ∆AP↑ ∆FWT↑ ∆BWT↑ ∆FGT↓ BigCodeBench HippoRAG-v2 Pos. +1.82 +2.07 −5.50 −3.57 Neg. +1.70 +3.32 −1.51 −16.84 AWM Pos. +2.93 +8.39 −0.03 −5.42 Neg. +4.77 +1.73 +0.66 −13.07 AutoSkill Pos. −2.18 +0.39 +0.02 −4.37 Neg. −3.32 +2.38 −4.68 −4.05 WildToolBench HippoRAG-v2 Pos. +0.62 +3.68 +1.54 −2.39 Neg. +0.47 −1.31 +0.43 −2.02 AWM Pos. −0.01 +0.40 −1.37 −0.52 Neg. +0.46 −2.17 −0.67 −0.50 AutoSkill Pos. +0.58 +4.87 −2.08 −2.95 Neg. −0.09 +3.56 +0.80 −3.76 Table 2: Effect of SEU across datasets and transfer histories. Values denote paired percentage-point changes relative to the original agent; positive values indicate improvement, except for FGT, where negative values are favorable. Boldface de- notes favorable changes. MemRL, AutoSkill, and SkillClaw still improve, whereas both RAG agents, AWM, and SimpleMem decline, with BGE-M3 RAG showing the largest drop (−14.7 p). Across the switching conditions, the most recent interval dominates: when the path switches from negative to positive transfer, seven of eight agents recover at the next probe; when it switches from positive to negative, five of eight agents lose part of their initial gain. This recency effect means that an agent’s probe-task state reflects the transfer character of its recent history more than its cumulative history, explaining both the erosion observed in sustained paths and why later helpful experience can partially reverse earlier interference. 4.3 Selective Experience Use The preceding findings suggest that agents should not in- ject every retrieved experience indiscriminately. We there- fore implement Selective Experience Use, a lightweight agent harness that regulates how path-accumulated experience in- fluences each new task, operating between an agent’s native retrieval module and its inference model. Each retrieved item is judged against the current task and admitted as concrete detail when it is directly reusable (preserve), as compact task- relevant guidance when only its higher-level pattern transfers (abstract), or not at all (ignore); implementation details are provided in Appendix D. We evaluate representative agents with SEU on the same subset of 20 sequences drawn from the 50 sequences used in the main experiments. Table 2 re- ports the paired changes across datasets and transfer-history conditions. SEU consistently reduces forgetting across agents, datasets, and transfer histories, while improving forward transfer in the majority of settings. As shown in Table 2, SEU reduces FGT in every condition across all three agents and both datasets, confirming that filtering experience at the point of use curbs interference from the accumulated path. The retention benefit is more pronounced on BigCodeBench, where HippoRAG-v2 and AWM see FGT reductions of up to 16.84 and 13.07 p, than on WildToolBench (0.50–3.76 p). AP and FWT also improve in most conditions, with AWM gaining the most on BigCodeBench (+4.77 p AP, +8.39 p FWT). BWT shows no consistent trend, suggesting that SEU primarily mitigates interference from new experience rather than strengthening backward consolidation. 5 Conclusion We introduced PATH-Bench for evaluating lifelong LLM agents under controlled learning paths. By estimating task relationships via multi-model in-context learning and con- structing probe-centered sequences, PATH-Bench efficiently traces performance, transfer, and forgetting. Experiments on code-generation and tool-use tasks across eight agents with diverse harness designs show that experience utility depends on how it is represented and on the task’s interaction struc- ture, that immediate transfer does not ensure retention, and that later experience can reshape earlier gains. These find- ings motivate Selective Experience Use (SEU), which filters experience at inference time to consistently reduce forgetting while improving forward transfer in most settings. Nonethe- less, PATH-Bench has limitations: the current benchmark covers two task domains, single-turn code generation and multi-turn tool use, and all lifelong agent evaluations use a single backbone LLM. Extending PATH-Bench to broader task domains and diverse backbone models is a key direction for future work. References Ai, Q.; Tang, Y.; Wang, C.; Long, J.; Su, W.; and Liu, Y. 2025. MemoryBench: A Benchmark for Memory and Continual Learning in LLM Systems. arXiv preprint arXiv:2510.17281. Alzubi, S.; Provenzano, N.; Bingham, J.; Chen, W.; and Vu, T. 2026. EvoSkill: Automated Skill Discovery for Multi- Agent Systems. arXiv preprint arXiv:2603.02766. Asawa, P.; Glaze, C. M.; Orlanski, G.; Ramakrishnan, R.; Xu, B.; Biswal, A.; Chen, V. S.; Sala, F.; Zaharia, M.; and Gonzalez, J. E. 2026. Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments. arXiv preprint arXiv:2606.05661. Baker, M. M.; New, A.; Aguilar-Simon, M.; Al-Halah, Z.; Arnold, S. M.; Ben-Iwhiwhu, E.; Brna, A. P.; Brooks, E.; Brown, R. C.; Daniels, Z.; et al. 2023. A domain-agnostic approach for characterization of lifelong learning systems. Neural Networks, 160(C): 274–296. Bell, S. J.; and Lawrence, N. D. 2022. The effect of task order- ing in continual learning. arXiv preprint arXiv:2205.13323. Benavides-Prado, D.; and Riddle, P. 2022. A theory for knowledge transfer in continual learning. In Conference on Lifelong Learning Agents (CoLLAs). Cai, Z.; Guo, X.; Pei, Y.; Feng, J.; Su, J.; Chen, J.; Zhang, Y.-Q.; Ma, W.-Y.; Wang, M.; and Zhou, H. 2025. FLEX: Continuous Agent Evolution via Forward Learning from Ex- perience. arXiv preprint arXiv:2511.06449. Castillo-Bolado, D.; Davidson, J.; Gray, F.; and Rosa, M. 2024. Beyond prompts: Dynamic conversational benchmark- ing of large language models. In Advances in Neural Infor- mation Processing Systems (NeurIPS). Chen, J.; Xiao, S.; Zhang, P.; Luo, K.; Lian, D.; and Liu, Z. 2024. M3-Embedding: Multi-Linguality, Multi- Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. In Annual Meeting of the As- sociation for Computational Linguistics (ACL). Du, X.; Li, L.; Zhang, D.; and Song, L. 2025. MemR 3 : Memory Retrieval via Reflective Reasoning for LLM Agents. arXiv preprint arXiv:2512.20237. Gutiérrez, B. J.; Shu, Y.; Qi, W.; Zhou, S.; and Su, Y. 2025. From RAG to Memory: Non-Parametric Contin- ual Learning for Large Language Models. arXiv preprint arXiv:2502.14802. He, J.; Treude, C.; and Lo, D. 2025. LLM-Based Multi- Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead. ACM Transactions on Software Engineering and Methodology, 34(5): 1–30. Hu, Y.; Cai, Y.; Du, Y.; Zhu, X.; Liu, X.; Yu, Z.; Hou, Y.; Tang, S.; and Chen, S. 2025a. Self-evolving multi-agent collaboration networks for software development. In Inter- national Conference on Learning Representations (ICLR). Hu, Y.; Liu, S.; Yue, Y.; Zhang, G.; Liu, B.; Zhu, F.; Lin, J.; Guo, H.; Dou, S.; Xi, Z.; Jin, S.; Tan, J.; Yin, Y.; Liu, J.; Zhang, Z.; Sun, Z.; Zhu, Y.; Sun, H.; Peng, B.; Cheng, Z.; Fan, X.; Guo, J.; Yu, X.; Zhou, Z.; Hu, Z.; Huo, J.; Wang, J.; Niu, Y.; Wang, Y.; Yin, Z.; Hu, X.; Liao, Y.; Li, Q.; Wang, K.; Zhou, W.; Liu, Y.; Cheng, D.; Zhang, Q.; Gui, T.; Pan, S.; Zhang, Y.; Torr, P.; Dou, Z.; Wen, J.-R.; Huang, X.; Jiang, Y.-G.; and Yan, S. 2025b. Memory in the Age of AI Agents. arXiv preprint arXiv:2512.13564. Kirkpatrick, J.; Pascanu, R.; Rabinowitz, N.; Veness, J.; Des- jardins, G.; Rusu, A. A.; Milan, K.; Quan, J.; Ramalho, T.; Grabska-Barwinska, A.; et al. 2017. Overcoming catas- trophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13): 3521–3526. Li, H.; Ding, L.; Fang, M.; and Tao, D. 2024. Revisiting catastrophic forgetting in large language model tuning. In Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP). Li, L.; Shi, D.; Zhou, J.; Wei, X.; Yang, M.; Jin, S.; and Yang, S. 2025. Retrieval feedback memory enhancement large model retrieval generation method. arXiv preprint arXiv:2508.17862. Liu, J.; Su, Y.; Xia, P.; Han, S.; Zheng, Z.; Xie, C.; Ding, M.; and Yao, H. 2026. SimpleMem: Efficient Lifelong Memory for LLM Agents. arXiv preprint arXiv:2601.02553. Lopez-Paz, D.; and Ranzato, M. 2017. Gradient episodic memory for continual learning. In Advances in Neural Infor- mation Processing Systems (NeurIPS). Luo, Y.; Yang, Z.; Meng, F.; Li, Y.; Zhou, J.; and Zhang, Y. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. IEEE Transactions on Audio, Speech and Language Processing, 33: 3776–3786. Ma, Z.; Yang, S.; Ji, Y.; Wang, X.; Wang, Y.; Hu, Y.; Huang, T.; and Chu, X. 2026. SkillClaw: Let Skills Evolve Collectively with Agentic Evolver. arXiv preprint arXiv:2604.08377. Maharana, A.; Lee, D.-H.; Tulyakov, S.; Bansal, M.; Barbieri, F.; and Fang, Y. 2024. Evaluating very long-term conversa- tional memory of LLM agents. In Annual Meeting of the Association for Computational Linguistics (ACL). Majumder, B. P.; Mishra, B. D.; Jansen, P.; Tafjord, O.; Tandon, N.; Zhang, L.; Callison-Burch, C.; and Clark, P. 2023. CLIN: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134. OpenAI. 2026a.Introducing GPT-5.4 Mini and Nano. https://openai.com/zh-Hans-CN/index/introducing- gpt-5-4-mini-and-nano/. OpenAI. 2026b. Introducing OpenAI Frontier. https: //openai.com/index/introducing-openai-frontier/. Pan, C.; Yang, X.; Li, Y.; Wei, W.; Li, T.; An, B.; and Liang, J. 2025. A Survey of Continual Reinforcement Learning. arXiv preprint arXiv:2506.21872. Parisi, G. I.; Kemker, R.; Part, J. L.; Kanan, C.; and Wermter, S. 2019. Continual lifelong learning with neural networks: A review. Neural networks, 113: 54–71. Park, J. S.; O’Brien, J.; Cai, C. J.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2023. Generative agents: Interactive simulacra of human behavior. In ACM Symposium on User Interface Software and Technoloy (UIST). Shi, H.; Xu, Z.; Wang, H.; Qin, W.; Wang, W.; Wang, Y.; Wang, Z.; Ebrahimi, S.; and Wang, H. 2025. Continual learning of large language models: A comprehensive survey. ACM Computing Surveys, 58(5): 1–42. Sumers, T. R.; Yao, S.; Narasimhan, K.; and Griffiths, T. L. 2024. Cognitive Architectures for Language Agents. Trans- actions on Machine Learning Research, 2024. Wan, L.; and Ma, W. 2025. StoryBench: A Dynamic Bench- mark for Evaluating Long-Term Memory with Multi Turns. arXiv preprint arXiv:2506.13356. Wang, L.; Zhang, X.; Su, H.; and Zhu, J. 2024a. A compre- hensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 46(8): 5362–5383. Wang, Z. Z.; Mao, J.; Fried, D.; and Neubig, G. 2024b. Agent workflow memory. arXiv preprint arXiv:2409.07429. Wei, T.; Sachdeva, N.; Coleman, B.; He, Z.; Bei, Y.; Ning, X.; Ai, M.; Li, Y.; He, J.; Chi, E. H.; et al. 2025. Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self- Evolving Memory. arXiv preprint arXiv:2511.20857. Wołczyk, M.; Zając, M.; Pascanu, R.; Kuciński, Ł.; and Miłoś, P. 2021. Continual world: A robotic benchmark for continual reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS). Wu, C.-K.; Tam, Z. R.; Lin, C.-Y.; Chen, Y.-N.; and Lee, H.- y. 2024. StreamBench: Towards benchmarking continuous improvement of language agents. In Advances in Neural Information Processing Systems (NeurIPS). Xu, A.; Lin, B.; Xue, B.; Wang, B.; Xu, B.; Wu, B.; Zhang, B.; Lin, C.; Dong, C.; Ling, C.; et al. 2026a. Deepseek-v4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348. Xu, J.; Li, Q.; Wu, J.; Lan, Y.; Li, S. S.; Zhou, H.; Jiang, B.; Wang, L.; Wang, J.; Luu, A. T.; Xiong, C.; Park, H. W.; Hooi, B.; and Hu, Z. 2026b. EvoArena: Tracking Memory Evo- lution for Robust LLM Agents in Dynamic Environments. arXiv preprint arXiv:2606.13681. Yang, X.; Li, W.; Sheng, J.; Hua, Y.; Chen, H.; Shen, C.; and Wang, X. 2026a. Agentic episodic control. In Annual Meeting of the Association for Computational Linguistics (ACL). Yang, Y.; Li, J.; Pan, Q.; Zhan, B.; Cai, Y.; Du, L.; Zhou, J.; Chen, K.; Chen, Q.; Li, X.; et al. 2026b. Au- toSkill: Experience-Driven Lifelong Learning via Skill Self- Evolution. arXiv preprint arXiv:2603.01145. Ye, F.; and Bors, A. G. 2022. Lifelong generative modelling using dynamic expansion graph model. In AAAI Conference on Artificial Intelligence (AAAI). Ye, F.; and Bors, A. G. 2023. Continual variational autoen- coder via continual generative knowledge distillation. In AAAI Conference on Artificial Intelligence (AAAI). Yu, P.; Liu, W.; Yang, Y.; Li, J.; Zhang, Z.; Feng, X.; et al. 2026a. Benchmarking LLM Tool-Use in the Wild. In Inter- national Conference on Learning Representations (ICLR). Yu, Y.; Yuan, X.; Jin, H.; Liu, H.; Yu, Y.; and Wang, H. 2026b. Do Self-Evolving Agents Forget? Capability Degra- dation and Preservation in Lifelong LLM Agent Adaptation. arXiv preprint arXiv:2605.09315. Zeng, A.; Lv, X.; Hou, Z.; Du, Z.; Zheng, Q.; Chen, B.; Yin, D.; Ge, C.; Huang, C.; Xie, C.; et al. 2026. GLM- 5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763. Zhang, H.; Long, Q.; Bao, J.; Feng, T.; Zhang, W.; Yue, H.; and Wang, W. 2026a. MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents. arXiv preprint arXiv:2602.02474. Zhang, S.; Wang, J.; Zhou, R.; Liao, J.; Feng, Y.; Zhang, W.; Wen, Y.; Li, Z.; Xiong, F.; Qi, Y.; Tang, B.; and Wen, M. 2026b. MemRL: Self-Evolving Agents via Runtime Re- inforcement Learning on Episodic Memory. arXiv preprint arXiv:2601.03192. Zhang, Z.; Dai, Q.; Li, R.; Bo, X.; Chen, X.; and Dong, Z. 2025. Learn to memorize: Optimizing LLM-based agents with adaptive memory framework. arXiv preprint arXiv:2508.16629. Zhao, H.; Li, M.; Sun, L.; and Zhou, T. 2025. Benchmark Reduction with In-Context Transferability. In International Conference on Learning Representations (ICLR). Zhao, Y.; Yuan, B.; Huang, J.; Yuan, H.; Yu, Z.; Xu, H.; Hu, L.; Shankarampeta, A.; Huang, Z.; Ni, W.; Tian, Y.; and Zhao, J. 2026. AMA-Bench: Evaluating Long- Horizon Memory for Agentic Applications. arXiv preprint arXiv:2602.22769. Zheng, J.; Cai, X.; Li, Q.; Zhang, D.; Li, Z.; Zhang, Y.; Song, L.; and Ma, Q. 2025a. LifelongAgentBench: Eval- uating LLM Agents as Lifelong Learners. arXiv preprint arXiv:2505.11942. Zheng, J.; Qiu, S.; Shi, C.; and Ma, Q. 2025b. Towards lifelong learning of large language models: A survey. ACM Computing Surveys, 57(8): 1–35. Zheng, J.; Shi, C.; Cai, X.; Li, Q.; Zhang, D.; Li, C.; Yu, D.; and Ma, Q. 2026. Lifelong learning of large language model based agents: A roadmap. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(05): 5552–5571. Zhong, W.; Guo, L.; Gao, Q.; Ye, H.; and Wang, Y. 2024. MemoryBank: Enhancing large language models with long- term memory. In AAAI Conference on Artificial Intelligence (AAAI). Zhu, D.; Zhou, X.; Qin, S.; Zhu, X.; Ding, H.; Zhong, S.; Wen, Z.; Xie, Z.; Gou, C.; Ren, L.; et al. 2026. EdgeBench: Unveiling Scaling Laws of Learning from Real-World Envi- ronments. arXiv preprint arXiv:2607.05155. Zhuo, T. Y.; Chien, V. M.; Chim, J.; Hu, H.; Yu, W.; Widyasari, R.; Yusuf, I. N. B.; Zhan, H.; He, J.; Paul, I.; et al. 2025. BigCodeBench: Benchmarking Code Genera- tion with Diverse Function Calls and Complex Instructions. In International Conference on Learning Representations (ICLR). Zuffer, A.; Burke, M.; and Harandi, M. 2025. Advance- ments and Challenges in Continual Reinforcement Learning: A Comprehensive Review. arXiv preprint arXiv:2506.21899. Appendix Contents A Motivation Experiment B Transfer Matrix Validation C Probe Reappearance on WildToolBench D Selective Experience Use E Benchmark Statistics F Use of AI Systems A Motivation Experiment To probe path sensitivity, we conduct a preliminary study on LifelongAgentBench (Zheng et al. 2025a), the first unified benchmark for systematically assessing LLM agents as life- long learners. We randomly sample a common set of 50 tasks and evaluate four agents (AWM, Clin, BGE-M3 RAG, and HippoRAG-v2) under two task paths over the same task set. Each path is evaluated with three random seeds. We com- pute Average Performance (AP), Forward Transfer (FWT), Backward Transfer (BWT), and Forgetting (FGT) from the standard performance matrix a i,j (Lopez-Paz and Ranzato 2017; Pan et al. 2025), which records the score on task j after the agent has been trained through task i, requiring all tasks to be reevaluated at each training step. Different task paths over an identical task set produce sub- stantially different relative assessments. For example, Clin moves from first to fourth in AP, while AWM moves from sec- ond to first and BGE-M3 RAG from fourth to third. The rank changes are also visible for the lifelong metrics: Clin moves from first to third in FWT, whereas BGE-M3 RAG moves from fourth to second. This preliminary result shows that different task paths over an identical task set can change the relative ranking of agents, motivating PATH-Bench’s con- trolled construction of learning paths. B Transfer Matrix Validation As described in Section 3.2, PATH-Bench uses one-shot in- context learning effects as a practical proxy for inter-task transferability. It estimates a gain matrixM independently for DeepSeek-V4-Flash, GPT-5.4-mini, and GLM-5.0, with each entry averaged over five random seeds, and aggregates their signs by majority voting intoV. We validate this consensus signal through cross-model sign consistency and preserva- tion of its directional bias under multi-task composition. Cross-model sign consistency. Each model-specific trans- fer sign is compared with the corresponding entry ofV over off-diagonal task pairs, excluding self-transfer pairs to avoid trivially positive entries. On BigCodeBench, 12,279 of the 14,280 off-diagonal pairs (85.99%) receive a non-mixed re- lation; on WildToolBench the corresponding count is 12,038 pairs (84.30%). The remaining mixed-disagreement pairs, where no two models agree on the transfer direction, are ex- cluded from relation-constrained sampling because they in- dicate that the inter-task relationship has not been reliably es- tablished. As shown in Table 3, the agreement rates between each model andV span 75.12–77.47% on BigCodeBench and 66.37–79.33% on WildToolBench. The wider spread on WildToolBench originates almost entirely from GPT- 5.4-mini (66.37%), while the other two backbones remain BackboneAgreement (%) BigCodeBench WildToolBench DeepSeek-V4-Flash75.1279.33 GPT-5.4-mini77.4766.37 GLM-5.076.8177.09 Table 3: Sign consistency between the gain matrix of each backbone and the voting matrixV over non-mixed pairs. Self-transfer pairs are excluded, and the three backbones jointly constructV. above 77%; the multi-model voting mechanism mitigates such single-model variation by requiring at least two models to agree before fixing a relation. Sign preservation under multi-task composition. The gain matrixM is estimated from one-shot demonstrations, whereas the sampled lifelong sequences interleave multiple tasks between successive probe occurrences. This validation tests whether the transfer directions captured by one-shot estimation remain informative under such multi-task com- position. We select 50 BigCodeBench target tasks t j that have both positive and negative incoming relations inV and consider prefix lengths N ∈ 3, 6, 9, 12. For each target and N, positive prefixes C + contain only positive-transfer sources, negative prefixesC − contain only negative-transfer sources, and random prefixes are drawn from all non-target tasks. For each combination of 50 targets, 4 prefix lengths, and 3 prefix types (positive, negative, and random), we sample 3 sequences with replacement (50 × 4 × 3 × 3 = 1,800 sequences in total). Each prefix C is provided as an N-shot context to DeepSeek-V4-Flash, and the target task t j is evaluated over five seeds. The realized transfer gain is computed following the same definition as in Section 3.2, extended from one-shot to N-shot prefixes. As shown in Figure 7, positive prefixes yield average gains of 8.8–13.7 percentage points across the four lengths, while negative prefixes yield losses of 9.9–15.9 points; random pre- fixes remain close to zero, with gains of 0.1–3.9 points. The aggregate directional bias ofV thus persists when multiple selected tasks are combined, confirming the property that relation-constrained sampling relies on. C Probe Reappearance on WildToolBench Figure 6 reports probe reappearance dynamics on WildTool- Bench under the same protocol as the BigCodeBench analy- sis in the main text. Two differences from BigCodeBench stand out. First, the dynamic range is considerably narrower, with every curve staying within roughly five percentage points ofp warm across all twelve occurrences. Second, the two conditions are no longer consistently ordered: Clin, AWM, and SimpleMem place their negative-dominant curve at or above the positive- dominant one over most of the sequence, so for these agents a negative-dominant history does not visibly cost more than 5 0 5 BGE-M3 RAGHippoRAG-v2ClinAWM 23456789101112 5 0 5 SimpleMem 23456789101112 MemRL 23456789101112 AutoSkill 23456789101112 SkillClaw Change from post-warm-up reference (p) Probe occurrence PositiveNegative Figure 6: Probe reappearance dynamics on WildToolBench under positive- and negative-dominant histories. Curves report the change from the post-warm-up reference p warm (in percentage points) at successive probe occurrences. 4681012 Multi-task prefix length N −40 −20 0 20 40 Mean transfer gain (p) PositiveNegativeRandom Figure 7: Multi-task sign-preservation validation on Big- CodeBench. Curves report mean realized transfer gain over 50 target tasks; bands are 95% confidence intervals computed across target-level means. For each target and prefix length, three sequences per group were sampled and evaluated over five seeds. a positive-dominant one on WildToolBench. The erosion it- self is still present, most clearly for MemRL and AutoSkill, whose curves drift steadily below p warm under both condi- tions. The within-path erosion that PATH-Bench isolates on BigCodeBench therefore also appears on WildToolBench, but with a smaller magnitude and a correspondingly weaker separation between the two history conditions. D Selective Experience Use SEU operates between an agent’s native retrieval module and its inference model. Before each task, the agent retrieves ex- perience through its own mechanism (e.g., similarity-based passage retrieval for RAG agents, accumulated workflow pat- terns for AWM, reinforced interaction records for MemRL). SEU issues a single LLM call that evaluates the retrieved 120 tasks BCB Domain Computation (55) General (25) System (17) Visualization (7) Cryptography (7) Time (5) Network (4) 480 turns WTB Task Type Single (124) Chat (119) Clarify (118) Parallel (73) Mixed (37) Sequential (9) Figure 8: Task composition of the BigCodeBench (BCB) and WildToolBench (WTB) subsets used in PATH-Bench. BigCodeBench tasks are grouped by primary domain (left); WildToolBench turns are grouped by task type (right). memory bundle against the current task description and se- lects one of three actions: preserve retains the bundle un- changed when it is relevant, non-conflicting, and concise enough for direct injection; abstract replaces it with a com- pact summary retaining only transferable information rele- vant to the current task; ignore suppresses the bundle en- tirely when it has no concrete reusable connection or risks misleading the agent. The compiled result is ephemeral and is not written back to the agent’s long-term memory. The full prompt is shown in Figure 11; the compiler uses DeepSeek- V4-Flash as the backbone. E Benchmark Statistics Figure 8 shows the compositional breakdown of the two benchmark subsets used in PATH-Bench. BigCodeBench. The 120-task subset is drawn from BigCodeBench-v0.1.4 and spans seven domains, each deter- mined by the primary library of each task. Computation is the 010203040 Number of tasks string csv datetime json scipy itertools seaborn re collections sklearn os random numpy matplotlib pandas 6 6 7 9 9 11 11 13 14 17 19 23 35 38 39 BigCodeBench — Top 15 Libraries Figure 9: Fifteen most frequently required libraries in the BigCodeBench subset. The 120 tasks involve 55 unique li- braries in total. 050100150200250300350 Number of transitions 171 (48%) 110 (31%) 79 (22%) WildToolBench — Instruction Transition Coreferential ReferencePartial InformationLong-Range Dependency Figure 10: Distribution of instruction transition types across consecutive turns in the WildToolBench subset. dominant domain, accounting for 55 tasks (45.8%), followed by General (25, 20.8%) and System (17, 14.2%); Visualiza- tion and Cryptography each contribute 7 tasks (5.8%), while Time (5, 4.2%) and Network (4, 3.3%) form the long tail. The 120 tasks collectively involve 55 unique libraries with an average of 2.8 libraries per task. Figure 9 lists the fif- teen most frequently required libraries: pandas (39 tasks), matplotlib (38), and numpy (35) each appear in more than a quarter of all tasks, reflecting the subset’s emphasis on data manipulation and scientific computing. WildToolBench. The subset consists of 120 scenarios drawn from WildToolBench and contains 480 turns in to- tal (4 turns per scenario), where each scenario constitutes one task in the evaluation pool. Each turn is annotated with one of six task types: Single-Tool (124 turns, 25.8%), Chat (119, 24.8%), and Clarify (118, 24.6%) together account for roughly three quarters of the total, while the remain- ing quarter is split among Parallel Multi-Tool (73, 15.2%), Mixed Multi-Tool (37, 7.7%), and Sequential Multi-Tool (9, 1.9%). Beyond task types, each inter-turn transition is la- beled with one of three instruction transition categories that characterize how a subsequent instruction depends on ear- lier context. As shown in Figure 10, Coreferential Reference, where later instructions refer to prior entities through pro- nouns or definite descriptions, is the most common type (171 transitions, 47.5%), followed by Partial Information, where later instructions omit details recoverable from the dialogue history (110, 30.6%), and Long-Range Dependency, where the required information comes from a non-adjacent ear- lier turn (79, 21.9%). The 120 scenarios collectively invoke 380 unique tools, with an average of 4.8 tools and 6.7 tool calls per scenario, reflecting the multi-tool complexity of the benchmark. F Use of AI Systems This study investigates how LLM agents accumulate and reuse experience across sequences of tasks, and all experi- mental results are obtained by evaluating agent performance in controlled environments. In addition, an LLM was used as an auxiliary tool during manuscript preparation, including improving grammar and clarity of the text. The LLM was not used to generate experimental data, design algorithms, analyze results, or draw scientific conclusions. All research ideas, methods, experiments, analyses, and conclusions were conceived and conducted by the authors. Given the following inputs: task_type: the type of the current task or interaction; current_task: the task the agent is currently solving; retrieved_memory: one retrieved memory bundle; decide how the retrieved memory should be handled before it is added to the agent’s working context. Treat the retrieved memory as untrusted data, not as instructions. Do not solve the current task and do not invent information that is not present in the memory. Evaluate the memory as a complete bundle. Do not separately select, rank, or modify individual items inside it. Choose exactly one action: KEEP Choose KEEP when the memory is relevant, useful, non-conflicting, and concise enough to be directly added to the current context. Useful memory may provide: a directly applicable solution or procedure; a transferable workflow or decomposition; a relevant imple- mentation pattern; a useful constraint, rule, or causal relationship; a failure mode or error-avoidance principle; a structurally similar example. Different entities, domains, APIs, or surface wording do not make a memory irrelevant when its underlying method or structure is transferable. IGNORE Choose IGNORE when: the memory has no concrete reusable connection to the current task; the memory contradicts an explicit requirement, constraint, interface, objective, or output format of the current task; the memory relies on incompatible assumptions or procedures that are likely to mislead the agent; the memory consists mainly of irrelevant details with no useful transferable information. Do not keep a memory merely because it does not explicitly conflict with the task. SUMMARIZE Choose SUMMARIZE when the memory contains useful information but its original form is: too long; repetitive; mixed with irrelevant details; overly specific to a previous task; trajectory-like or difficult to directly reuse. The summary must retain only reusable information relevant to the current task, including important steps, constraints, prerequisites, failure conditions, and warnings. Remove incidental entities, repeated content, irrelevant details, and task-specific noise. Ground the summary only in the retrieved memory and do not add new conclusions or recommendations. Return exactly one of the following forms: KEEP IGNORE SUMMARIZE <the summarized memory> Do not return JSON, Markdown, explanations, reasons, scores, labels, or any other content. Figure 11: Prompt for Selective Experience Use.