Paper deep dive
Capability-Driven Self-Evolution of Agent Memory
Yaoqi Chen, Yuru Feng, Qianxi Zhang, Baotong Lu, Jianan Lu, Zhirui Wang, Shusen Xu, Zewen Jin, Zengzhong Li, Cheng Li, Qi Chen
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Memory self-evolution uses task feedback to iteratively improve executable memory programs that store and retrieve information from past interactions. Existing approaches typically adopt holistic evolution, deriving revision directions from mixed feedback and judging progress by overall performance. This can obscure optimization directions and hide capability-specific gains offset by regressions elsewhere, leaving promising directions underexplored. We introduce capability-driven evolution, which extends search guidance from overall performance to individual capability dimensions, preserving promising revisions and expanding exploration beyond the boundaries of holistic evolution. We propose PrisMem, which uses dependency-aware capability selection to prioritize targets with potential cross-capability benefits and history-guided diagnosis to refine capability specialists. Trace-guided integration compares evaluated programs on paired differential cases, using their behavioral differences to consolidate complementary gains into a unified memory program. Experiments show that PrisMem outperforms the strongest baselines by 10.54 and 7.83 percentage points on BEAM-1M and LongMemEval-M, respectively, demonstrating its effectiveness on million-token histories.
Tags
Links
- Source: https://arxiv.org/abs/2610.06361v1
- Canonical: https://arxiv.org/abs/2610.06361v1
Trouble viewing inline? Open PDF directly â
Full Text
126,653 characters extracted from source content.
Expand or collapse full text
Capability-Driven Self-Evolution of Agent Memory Yaoqi Chen 1,2 * , Yuru Feng 2,3 * , Qianxi Zhang 2 , Baotong Lu 2 , Jianan Lu 2 , Zhirui Wang 2 , Shusen Xu 2 , Zewen Jin 1 , Zengzhong Li 2 , Cheng Li 1 , Qi Chen 2 1 University of Science and Technology of China 2 Microsoft 3 University of California, San Diego Abstract Memory self-evolution uses task feedback to iteratively improve executable memory pro- grams that store and retrieve information from past interactions. Existing approaches typi- cally adopt holistic evolution, deriving revision directions from mixed feedback and judg- ing progress by overall performance. This can obscure optimization directions and hide capability-specific gains offset by regressions elsewhere, leaving promising directions un- derexplored. We introduce capability-driven evolution, which extends search guidance from overall performance to individual capability dimensions, preserving promising revi- sions and expanding exploration beyond the boundaries of holistic evolution. We propose PRISMEM, which uses dependency-aware capability selection to prioritize targets with po- tential cross-capability benefits and history-guided diagnosis to refine capability specialists. Trace-guided integration compares evaluated programs on paired differential cases, using their behavioral differences to consolidate complementary gains into a unified memory pro- gram. Experiments show that PRISMEM outperforms the strongest baselines by 10.54 and 7.83 percentage points on BEAM-1M and LongMemEval-M, respectively, demonstrating its effectiveness on million-token histories. 1 Introduction Memory has become a key component of large language model (LLM) agents, enabling them to retain and leverage information from past interactions (Packer et al., 2023; Zhong et al., 2024; Kang et al., 2025; Salama et al., 2025). These interactions contain diverse information, including explicit facts, implicit preferences and dynamically changing states. Moreover, different requests call for diverse capabilities, such as factual retrieval, temporal tracking, preference recognition, and cross-session synthesis (Maharana et al., 2024; Wu et al., 2025; Tavakoli et al., 2026). Effective memory is therefore inherently multidimensional, requiring a memory program to support distinct yet interconnected capabilities across diverse tasks and evolving interactions (Xu et al., 2025; Rasmussen et al., 2025; Zhang et al., 2026b; Salama et al., 2025; Ye et al., 2026). Self-evolving memory offers a promising solution by representing agent memory as an executable program, evaluating it on training tasks, and iteratively refining its implementation by an LLM based on task feed- back (Pan et al., 2026; Liu et al., 2026b; Zhang et al., 2026a). A key challenge is identifying useful revision directions and determining which improvements are worth pursuing. Existing methods generally adopt holis- tic evolution, deriving optimization directions from mixed feedback across memory capabilities and judging progress mainly by overall task performance. Such compressed scalar guidance can confine the search to a lim- ited region of capability space, with overall performance plateauing while some capability directions remain underexplored (Fig. 1(c)). This limitation arises from two issues: ambiguous optimization directions and hidden capability gains. First, mixed feedback combines failures from tasks requiring different capabilities, which may suggest multiple possible revisions for the same memory component with different effects. For example, broader retrieval may improve multi-session reasoning but increase over-personalization by introducing irrelevant personal memo- ries (Hu et al., 2026b; Pulipaka et al., 2026; Cao et al., 2026). Our analysis in Fig. 1(b) shows that such uneven â Equal contribution. 1 arXiv:2610.06361v1 [cs.AI] 5 Oct 2026 Exploration boundary (a) Hidden capability gains F A T M P (b) Uneven capability changes (c) Illustration of holistic evolution (d) Illustration of capability-driven evolution F A T M P F-Factual retrieval; T -Temporal tracking; P-Preference extraction; M-Multi-session synthesis; A -Adversarial Figure 1: Motivation for capability-driven evolution. Analysis of M â and EvolveMem evolution traces on BEAM. (a) Overall and maximum capability-level score changes for each revision. Many revisions improve at least one capability despite lower overall scores (upper-left region). (b) Among revisions that improve one capability (row), the percentage for which another capabilityâs score decreases (column). (c) Holistic evolution confines exploration to a limited region. (d) Capability-driven evolution enables broader exploration through explicit capability directions. effects are common. Before these directions are implemented and evaluated, their actual effects across capa- bilities remain unknown, making it difficult to determine which direction to pursue for overall improvement. Second, a revisionâs gains on one capability can be offset by regressions in others, leaving the overall score unchanged or even lower. Fig. 1(a) shows that 80.5% of revisions without an overall gain still improve at least one capability. This causes useful mechanisms to be overlooked before their benefits can be further refined, integrated, and reflected in overall performance. This issue is particularly pronounced near a performance plateau, where marginal gains are easily masked, further confining the search around the current solution. Our key insight is to elevate the evolution space from a single overall-performance dimension to multiple capability-level dimensions. This motivates capability-driven evolution: Capability-specific feedback sug- gests focused diagnosis and guides explicit directions, while capability-level evaluation preserves promising revisions even without overall gains, enabling further refinement and broader exploration along individual ca- pability dimensions (Fig. 1(d)). Crucially, this process also turns broader optimization directions into evaluated memory programs whose implementations and execution results reveal how differently these programs behave on the same tasks. These references provide guidance for combining useful mechanisms into a stronger unified memory program, rather than choosing among unexplored directions from mixed feedback. We propose PRISMEM, which decomposes holistic memory evolution into capability-specific directions, anal- ogous to a prism separating light. To achieve broader gains from each capability refinement step, dependency- aware capability selection exploits dependencies among capabilities to prioritize those whose improvement can also benefit others. Refinement starts from the best-performing program for the selected capability (specialist), allowing local gains to develop even when obscured by the overall score and expanding the search boundary. History-guided diagnosis then identifies informative failures that past programs handled well, while discount- ing repeatedly examined cases and capability combinations to broaden diagnostic coverage and uncover fur- ther revision opportunities. Finally, trace-guided integration consolidates capability specialists evolved along different directions. Using paired differential cases, it compares the base program with each specialist to ex- pose their behavioral differences and guide how complementary gains should be incorporated. This translates broader capability-level exploration into a stronger unified memory program. Our contributions are: (1) We introduce capability-driven evolution, a new paradigm for memory self-evolution that lifts search guidance to the capability dimension, expanding the exploration boundary. (2) We propose PRISMEM, which combines dependency-aware capability selection, history-guided diagnosis, and trace-guided integration to develop capability-level advances into a stronger memory program. (3) We evolve on shorter- context splits of BEAM and LongMemEval and test on their 1M-token versions, achieving improvements of 7.83â10.54 percentage points over the best-performing baselines and demonstrating effective generalization to million-token histories. 2 Background and Related Work Diverse Memory Capabilities. Agent memory enables agents to retain and reuse interaction history for reasoning, decision-making, and long-term interaction (Park et al., 2023; Packer et al., 2023). Diverse histori- 2 cal information and user needs impose heterogeneous requirements on memory systems, where design choices that benefit one capability may adversely affect others. For instance, personalized assistance requires lever- aging a userâs preferences, while general information requests should remain unaffected by unrelated personal details (Zhong et al., 2024; Hu et al., 2026b). Appendix E provides more examples. These diverse requirements motivate different approaches to memory design, including static and self-evolving approaches. Static Agent Memory. Static memory systems use predefined structures and fixed strategies for stor- ing and retrieving information, with manually designed mechanisms for different memory capabilities. Mem0 (Chhikara et al., 2025) and SimpleMem (Liu et al., 2026a) emphasize preserving and consolidating salient information for factual retrieval. Temporal context receives particular attention in Zep (Rasmussen et al., 2025), Hindsight (Latimer et al., 2025) and TiMem (Li et al., 2026), supporting temporal tracking of events and changing states. MemoryBank (Zhong et al., 2024), MemInsight (Salama et al., 2025), Memo- ryOS (Kang et al., 2025), O-Mem (Wang et al., 2025), and EverMemOS (Hu et al., 2026a) draw on accumu- lated user information to support preference extraction and personalized responses. A-MEM (Xu et al., 2025), GAM (Yan et al., 2025), MRAgent (Ji et al., 2026), MemWeaver (Ye et al., 2026), and Amory (Zhou et al., 2026) emphasize connecting and combining information across interactions for multi-session synthesis. Self-Evolving Agent Memory. Self-evolving memory methods use task feedback to iteratively refine memory structures and management strategies by an LLM, finally producing a fixed memory program. Evolve- Mem (Liu et al., 2026b) focuses on configuration-level optimization of retrieval strategies, while M â (Pan et al., 2026), MemEvolve (Zhang et al., 2026a) and MemPro (Liu et al., 2026c) pursue program-level evolution of full memory structures and processing logic. These methods rely on holistic evolution driven by overall scores. This mixed feedback can blur optimization directions, while aggregate scores can mask capability gains that are offset by regressions elsewhere, leading to stagnant evolution. 3 PrisMem: Capability-Driven Memory Evolution Problem Formulation.PRISMEM aims to improve how an agent stores and retrieves conversational history by optimizing its executable memory program P . Let x = (H,q,y,E) denote a task instance with history H, question q, reference answer y, and reference evidence E. The program constructs memory from H and retrieves content for answering q, receiving a task score s(P ;x)â [0, 1]. We denote the average score of P over a task set S as s(P ;S). The task set is split into S train , S valid , and S test . Memory self-evolution iteratively evaluates P on S train , leverages an LLM to diagnose problems and refine P , and pursues improvements on S valid . After reaching maximum iterations, the best-performing program is frozen for evaluation onS test . Fine-grained Memory Interface. The memory interface determines the scope within which the memory program can be modified during self-evolution. We decompose the memory program into five components: Ex- traction, Indexing, Planning, Retrieval, and Answer. Compared with the commonly used âaddâ and âretrieveâ abstraction in agent memory systems, this decomposition provides more localized modification boundaries, helping the coding model identify which stage is responsible for a failure and revise the corresponding compo- nent more precisely. Capability Set for Our Study.PRISMEM uses capabilities to organize evolution and guide program re- finement. In our experiments, we instantiate the framework with five representative agent memory capabilities, denoted by C (Table 1). We select these capabilities through a broad review of widely used memory bench- marks (Maharana et al., 2024; Wu et al., 2025; Tavakoli et al., 2026) and existing memory systems (Zhong et al., 2024; Rasmussen et al., 2025; Chhikara et al., 2025; Xu et al., 2025), focusing on recurring memory requirements across these settings 1 . Each capability captures a distinct and essential memory requirement. We offline annotate tasks inS train andS valid using an LLM to assign each task one primary capability â(x)âC and up to two secondary capabilities Ď(x) â C\â(x). These capability labels organize feedback and guide evolution but are never provided to the memory program during evaluation, ensuring a fair comparison with baselines. LetS c =xâS : â(x) = c denote the subset of tasks associated with primary capability c. 3.1 Capability-Driven Evolution PRISMEM organizes evolution into three stages to expand the search boundary along individual capability di- mensions. First, the cold start stage addresses broad weaknesses in the seed program. Second, the capability 1 Table 5 in Appendix illustrates how different question types from some benchmarks relate to the chosen capabilities. Hereafter, we use F, T, P, M, and A as shorthand for the full capability names. 3 Table 1: Five memory capabilities used in our experiments. CapabilityMemory requirement Factual retrieval (F)Preserve and retrieve explicit facts, values, and other localized details. Temporal tracking (T)Order events, determine recency, track states and knowledge updates. Preference extraction (P)Recover and apply user preferences, instructions and constraints. Multi-session synthesis (M)Aggregate and synthesize evidence across multiple sessions or events. Adversarial (A)Abstain when memory lacks supporting evidence, or cautiously identify and resolve contradictory or misleading evidence. refinement stage performs dependency-aware capability selection to prioritize capabilities and history-guided diagnosis to provide focused revisions, while capability-level evaluation preserves promising variations. Fi- nally, trace-guided integration consolidates these specialized programs into a stronger, unified memory pro- gram. Figure 2 illustrates the evolution process, and Algorithm 1 in Appendix presents the pseudocode. 3.1.1 Stage I: Cold Start The initial seed memory program P 0 is deliberately simple, providing a broad search space but potentially leaving basic processing weaknesses across multiple capabilities. Cold start jointly considers capability-wise feedback across all capabilities over T 1 = 5 rounds to quickly establish a balanced base program for subse- quent capability refinement. The program with the highest validation score becomes the starting point for the subsequent stages, while the evolution history is retained to guide later capability selection and refinement. To guide these early revisions, we construct a metric table that aggregates evidence-based diagnostics by capa- bility, preserving capability-specific differences and reducing diagnostic ambiguity. The diagnostics measure semantic similarity between reference evidence and (i) extracted memory units, (i) content retrieved using the evidence as a query cue, and (i) content retrieved for the actual question. These metrics assess infor- mation preservation during extraction, accessibility through indexing, and retrieval effectiveness. Comparing these capability-by-component patterns helps the model localize broad processing weaknesses and prioritize concrete revisions in early evolution. 3.1.2 Stage I: Capability Refinement Next, we refine individual capabilities of the memory program obtained from cold start stage, retaining valuable capability-specific breakthroughs when they are obscured by overall performance. Let P t denote the set of candidate programs retained up to iteration t. For each capability c, we track its best-performing program as the specialist P â t (c), and denote the globally best program on the complete task set by G t . When a target capability c t is selected, refinement is initialized from its specialist, i.e., P t = P â t (c t ). Each retained candidate is evaluated on tasks across all capabilities, so a single program can simultaneously serve as the specialist for multiple capabilities. Dependency-Aware Capability Selection. Selecting a capability for the next refinement is impor- tant because it determines how effectively the limited refinement budget expands the capability boundary. We prioritize capabilities that are weak themselves or appear to be bottlenecks for other capabilities, while down- weighting those that have already received substantial refinement budget. A capability may exhibit not only a direct deficit on tasks where it is the primary requirement, but also latent error pressure on tasks where it appears as a secondary requirement. For example, a deficiency in temporal tracking may affect tasks primarily targeting multi-session synthesis when they also require temporal information. To quantify this latent pressure, we consider tasks with primary capability c p and secondary capability c s . Since such co-occurrence does not necessarily imply that deficiencies in c s affect c p , we first define a structural base-rate prior: D(c p âc s ) = P xâS c p train 1[c s â Ď(x)] |S c p train | ,(1) which measures how often primary-c p tasks also require c s . At iteration t, let e t,c s (x) = 1â s(P â t (c s );x), Ěe t,c s (c p ) = 1 |S c p train | X xâS c p train e t,c s (x)(2) 4 Figure 2: Overview of PRISMEM. Stage I: Cold start quickly establishes a balanced program and initial evo- lution history. Stage I: Capability refinement uses dependency-aware capability selection and history-guided case selection to target capabilities and diagnose informative failures, respectively, refining their specialists through reflection and code revision. Stage I: Trace-guided integration compares the base program (â) and other capability specialists on paired differential cases, using their behavioral differences to guide integration into a unified memory program. denote the instance-level and average error of the c s specialist on primary-c p tasks. We then measure how disproportionately this error is concentrated on tasks that require c s : F t,c s (c p âc s ) = P xâS c p train e t,c s (x)1[c s â Ď(x)] P xâS c p train e t,c s (x) ,(3) with F t,c s (c p âc s ) = 0 when the denominator vanishes. Intuitively, if c s is not a bottleneck for c p , its error should be distributed roughly according to the base rate, i.e., F â D. Thus, an excess gap FâD > 0 indicates that deficiencies in c s are disproportionately associated with errors on c p . Scaling this gap by the primary task error yields the incoming error pressure: I t (c p âc s ) = Ěe t,c s (c p ) F t,c s (c p âc s )â D(c p âc s ) + ,(4) where [z] + = max(z, 0). For a candidate capability c, we combine its direct deficiency with its strongest cross-capability impact: U t (c) = Ěe t,c (c) + max c p âC\c I t (c p âc).(5) Crucially, using the maximum rather than a sum avoids degree-centrality bias and focuses the scheduler on the most acute structural bottleneck. Finally, to avoid repeatedly allocating the search budget to plateaued capabilities, we discount urgency by the historical allocation count n t (c): Ď t (c) = U t (c) 1 + n t (c) , c t = arg max câC Ď t (c).(6) Thus, a capability c t is prioritized either for its direct deficiency or for its strongest upstream impact on other capabilities, while its priority naturally decreases with repeated exploration. History-Guided Diagnosis. In later-stage evolution, the memory program is relatively mature, so further improvements require fine-grained diagnosis of individual failures. Inspecting failure cases through their exe- cution results helps pinpoint whether an error arises from noisy extraction, defective memory representation, retrieval failure, or answer misinterpretation, translating behavioral symptoms into actionable code revisions. However, diagnostic cases can be lengthy. We therefore select K = 5 cases that are severe, historically ac- tionable, and collectively diverse, allowing each refinement step to explore a broader and more valuable set of optimization directions. 5 Specifically, we first consider training task instances whose primary capability is the selected target c t and filter out those with scores above δ = 0.8, retaining cases that are not fully correct. We then rank the remaining failures by a history-guided priority weight: w t (i) = 1 + (1/|P t (c t )|) P PâP t (c t ) s(P ;x i ) (1 + m t (i))(1 + m t (Ď(x i ))) .(7) Here,P t (c t ) contains valid programs produced in previous iterations that also targeted c t . The numerator mea- sures historical performance; a high historical score for a current failure identifies a regression witness, whose contrast with the current failure helps localize the effects of recent revisions. The denominator discounts cases that have been selected frequently (m t (i)) and secondary-capability combinations that have been repeatedly covered (m t (Ď(x i ))), promoting diverse coverage and reducing redundant optimization of similar failures. This enables the limited refinement budget to explore broader and more informative optimization directions. 3.1.3 Stage I: Trace-guided Integration While capability specialists extend the performance boundary along different dimensions, their improvements may remain isolated. The integration stage consolidates these distributed gains into a unified memory program, starting from the overall best program so far and progressively incorporating capability specialists from other directions. A planning model determines the integration order based on the lineage topology, processing spe- cialists from those closest to the base program to those farther away, and produces a step-by-step integration plan. A coding model then executes the plan. This integration is enabled by the evaluation results of retained programs. Since all programs are evaluated on the same training task set, we can compare revisions on the same cases and trace their effects through the evolution lineage, revealing their capability-specific gains and side effects. We therefore introduce a paired differential cases strategy to guide integration. For each capability, we select the training case with the largest score gap between the base program and the specialist being integrated. Comparing their execution results exposes the specialistâs behavioral differences, helping the planning model determine how its gains and side effects should be incorporated. The resulting plan is executed for up to T 3 = 3 rounds. If integration fails to improve the overall score or causes any capability to decrease by more than θ, the failed implementation and its execution results on the same differential cases are returned to the coding model for another integration. Overall, this process aims to translate complementary capability-specific improvements into stronger overall performance while mitigating regressions across capabilities. 3.2 Efficient Evolution Self-evolution repeatedly revises and evaluates memory programs, incurring substantial inference costs. We reduce this cost through compact diagnostic contexts and revision-aware execution reuse. First, diagnostic cases provide the information needed to identify program failures, including the question, reference answer and evidence, retrieved content, and model prediction. Full traces can be lengthy, so we construct compact diagnostic contexts by removing redundant content and marking omissions while retaining snippets relevant to the question and answer. This reduces diagnostic-case token cost by 44% on average while preserving key information for diagnosis. Second, evaluating each candidate program requires executing the memory pipeline, but revisions may affect only a subset of its components. Using the fine-grained memory interface, we compare each candidate with its parent and reuse outputs from unaffected components. For example, a retrieval-only revision can reuse the constructed memory and skip extraction and indexing. Selectively reusing completed results reduces redundant processing and accelerates program evaluation. 4 Experiments We evaluate PRISMEM on four questions: Q1: How does PRISMEM perform compared with existing methods? Q2: What is the evolution cost of PRISMEM? Q3: How much does each component contribute to PRISMEMâs performance? Q4: Can PRISMEM expand the capability boundary and integrate the resulting improvements into the final program? 6 4.1 Experimental Setup Benchmarks. We evaluate on two widely used agent memory benchmarks, BEAM (Tavakoli et al., 2026) and LongMemEval (Wu et al., 2025), which cover diverse question types over long, multi-session histo- ries. All evolution methods train their memory programs for 20 iterations on the smaller splits, BEAM- 100K and LongMemEval-S, and are then tested on the longer splits with different content, BEAM-1M and LongMemEval-M, with over 1 million tokens. Dataset details can be found in Appendix B.1. This protocol tests whether the learned design choices generalize to longer contexts. Baselines. We compare PRISMEM with seven representative agent memory methods, including static and self-evolving approaches. Static baselines include Mem0 (Chhikara et al., 2025) and A-MEM (Xu et al., 2025), which organize and update structured memories; HippoRAG2 (Guti Ě errez et al., 2025), which supports memory- enhanced retrieval with graphs; and SimpleMem (Liu et al., 2026a) and LightMem (Fang et al., 2026), which emphasize efficient memory construction and retrieval. Self-evolving methods include M â (Pan et al., 2026), which evolves executable memory programs through population-based reflective code search, and Evolve- Mem (Liu et al., 2026b), which uses failure diagnosis to adapt retrieval infrastructure and answer-generation configurations. Both adopt holistic evolution, using aggregate task performance as the primary signal for ex- ploration. Models. We use Qwen3.8-27B (Qwen, 2026) as the task, reflection, coding, and judging model, and Qwen3- Embedding-8B (Qwen, 2025) as the embedding model. Inference runs on NVIDIA A100 GPUs (NVIDIA, 2020) with vLLM 0.20.0 (Kwon et al., 2023) under Python 3.12. We also report results with GPT-5.5 (OpenAI, 2026) on BEAM in Appendix C.1. Metrics. Task performance is measured by the LLM judge scores provided by each benchmark. Overall score is averaged over all questions, while capability-level scores are averaged over questions grouped by the native question-type mapping in Table 5. Inference cost is measured by the average number of task LLM input and output tokens per question. For self-evolving methods, we additionally report the token consumption of the whole evolution procedure. Each experiment is run three times, and results are reported as mean Âą standard deviation. 4.2 Overall Performance (Q1) Table 2 shows the main results on two million-token memory benchmarks. Static methods show different capa- bility strengths: HippoRAG2 leads in factual retrieval, while A-MEM is competitive in preference extraction. Holistic evolution yields only limited gains over static methods, with M â improving over the strongest static baseline by 0.99 p on BEAM and EvolveMem by 0.34 p on LongMemEval. This limited improvement re- flects how mixed feedback can obscure optimization directions and aggregate scores can mask capability gains offset by regressions elsewhere. In contrast, PRISMEM achieves the best performance in nearly all capability comparisons, improving overall scores over the strongest static and self-evolving baselines by 8.17â11.53 p and 7.83â10.54 p, respectively. These results demonstrate the effectiveness of capability-driven evolution: capability refinement expands ex- ploration along individual dimensions and develops useful revisions beyond holistic evolution, while trace- guided integration consolidates these gains into a stronger program. These gains are achieved with token usage comparable to that of other methods. PRISMEM also generalizes well across context lengths and underlying models. Programs evolved on BEAM- 100K and LongMemEval-S retain their advantages on the longer BEAM-1M and LongMemEval-M splits. With GPT-5.5, PRISMEM also outperforms the baselines by 6.17â20.41 p on BEAM-1M (Appendix C.1), showing that capability-driven evolution can produce effective memory programs across different underlying models. Additional cross-model and cross-dataset transfer experiments further support the transferability of the evolved programs (Appendix C.2). 4.3 Evolution Efficiency (Q2) Table 3: Evolution tokens (millions, â) with Qwen3.8-27B on two benchmarks. MethodBEAMLongMemEval M â 76.5Âą 7.1202.3Âą 10.3 EvolveMem69.2Âą 7.3102.7Âą 8.4 PRISMEM73.7Âą 6.4143.3Âą 9.6 We compare the token consumption across self-evolving methods under the same 20-round evolution budget. The re- sults are summarized in Table 3. LongMemEval incurs higher evolution costs because each question has an independent history, resulting in higher per-round extraction costs than BEAM. Overall, EvolveMem consumes fewer tokens because 7 Table 2: Performance on BEAM-1M and LongMemEval-M with Qwen3.8-27B. âScoreâ denotes the benchmark-provided LLM judge score, and âAvg. #Tokensâ denotes the task LLMâs average token usage per question. All results are reported as meanÂą std. Best results are in bold. Method Score (%)â Avg. #Tokens (K)â FactualTemporal Preference Multi-session AdversarialOverall retrievaltrackingextractionsynthesis BEAM-1M Mem054.74Âą 3.06 46.51Âą 0.79 60.48Âą 2.3938.02Âą 0.6349.11Âą 1.24 48.95Âą 0.61190.28Âą 0.20 A-MEM56.21Âą 1.83 46.22Âą 0.62 65.39Âą 0.5253.87Âą 0.8641.13Âą 0.51 51.56Âą 0.62428.95Âą 3.58 HippoRAG2 65.24Âą 1.26 47.06Âą 1.85 66.43Âą 1.5744.83Âą 0.6044.82Âą 2.28 51.86Âą 0.87207.70Âą 0.19 SimpleMem 48.55Âą 3.33 36.82Âą 0.47 48.85Âą 1.1921.87Âą 2.0050.68Âą 0.40 40.18Âą 0.7695.72Âą 0.24 LightMem46.66Âą 0.82 33.40Âą 0.65 36.64Âą 0.6316.70Âą 0.8955.45Âą 0.94 36.44Âą 0.3886.20Âą 0.05 M â 61.29Âą 5.14 48.98Âą 5.87 71.91Âą 3.4042.19Âą 4.8846.02Âą 7.52 52.85Âą 2.4989.47Âą 5.11 EvolveMem 41.80Âą 4.77 39.74Âą 3.15 52.21Âą 1.5321.67Âą 3.0745.67Âą 3.50 40.01Âą 1.4698.40Âą 4.67 PRISMEM74.17Âą 4.8757.26Âą 2.1575.58Âą 3.9154.12Âą 3.3764.32Âą 6.5163.39Âą 1.02100.59Âą 5.02 LongMemEval-M Mem069.79Âą 1.80 63.54Âą 0.90 43.33Âą 0.0063.54Âą 1.8063.33Âą 5.77 62.50Âą 0.50 5772.67Âą 4.12 A-MEM87.50Âą 1.56 59.90Âą 0.90 44.44Âą 1.9254.17Âą 1.8096.67Âą 5.77 67.33Âą 0.76 8873.92Âą 14.45 HippoRAG2 89.58Âą 1.80 47.92Âą 0.90 24.44Âą 1.9240.63Âą 3.1390.00Âą 10.00 58.67Âą 1.26 6185.48Âą 5.01 SimpleMem 82.81Âą 1.56 74.48Âą 2.39 11.11Âą 1.9267.71Âą 1.8073.33Âą 11.55 66.50Âą 0.50 2624.68Âą 7.90 LightMem47.92Âą 0.90 74.48Âą 0.90 24.44Âą 1.9258.33Âą 1.8096.67Âą 5.77 57.00Âą 0.87 1888.30Âą 1.28 M â 76.04Âą 5.49 47.40Âą 3.93 12.22Âą 5.0967.71Âą 4.7793.33Âą 11.55 56.83Âą 3.21 1528.94Âą 30.34 EvolveMem 78.13Âą 3.13 69.79Âą 2.39 42.22Âą 3.8565.63Âą 3.1370.00Âą 20.00 67.67Âą 1.53 2749.94Âą 27.18 PRISMEM82.29Âą 3.9375.00Âą 3.1347.78Âą 1.9282.29Âą 3.6196.67Âą 5.7775.50Âą 1.802232.65Âą 32.72 it evolves only the retrieval and answer configurations, whereas M â and PRISMEM evolve the full memory program and incur additional extraction costs. Despite this, PRISMEM reduces evolution cost by 4%â30% over M â , enabled by compact diagnostic contexts and revision-aware execution reuse introduced in Section 3.2. 4.4 Ablation Study (Q3) Table 4: Ablations with Qwen3.8-27B on BEAM-1M. â is relative to full PRISMEM. Variant Overall scoreâ (%)â(p) Full PRISMEM63.39Âą 1.02â w/o capability evolution53.24Âą 2.38 â10.15 w/o trace-guided integration60.23Âą 1.86 â3.16 Round-robin target selection60.84Âą 1.13 â2.55 Random case selection58.13Âą 1.25 â5.26 To examine the contribution of each design choice, we compare four variants of PRISMEM on BEAM with Qwen3.8-27B. The results are shown in Ta- ble 4. Removing capability-driven evolution falls back to conventional holistic evolution and reduces the score by 10.15 p. Without capability-level opti- mization, mixed feedback can blur optimization di- rections, while useful capability-specific revisions may not receive further refinement when they do not improve overall performance. This large drop high- lights capability-driven evolution as a primary source of improvement. Removing trace-guided integration reduces the score by 3.16 p, but the variant still outperforms the baselines. This shows that optimizing along explicit capability directions already improves evolution by reducing mixed feedback and focusing revisions, while integration guided by evaluated programs and paired differential cases across capabilities provides additional gains in overall performance. Using a fixed round-robin schedule instead of dependency-aware capability selection reduces the score by 2.55 p, supporting targeted refinement of weak capabilities and potential bottlenecks for multi-capability tasks. Random case selection reduces the score by 5.26 p, highlighting the importance of matching diagnostic cases to each stageâs objective. Our case selection provides capability-relevant failures for focused refinement and contrasting execution results that expose useful revisions and potential side effects during integration. 8 05060708091011121314151617 F T P M A Target 05060707070707071313131313 05050707070707070707070717 05050707090909090909151515 05050505051010101010101010 05050508080808080808080817 âMTAMPTAMTPFA Iteration 0 2% 4% 6% 8% 10% Score delta from Iteration 05 (a) Score gains during capability refinement stage M* EvolveMem PrisMem peak final select F T PM A 10 30 50 70 90 (b) Peak vs. final performance Figure 3: Evolution trace analysis. (a) Gains of each Stage I iteration relative to program 05 obtained after cold start. Cell numbers are the creation iteration of the retained specialist for each capability; dots mark specialist updates. Scores improve intermittently across capabilities as refinement progresses. (b) Capability- wise scores of M â , EvolveMem, and PRISMEM. Solid lines show the final selected programs, while dashed lines show the best capability-wise score reached across all iterations. PRISMEM outperforms holistic evolution across capabilities, expanding the performance boundary, while its final integrated program closely matches its historical capability-wise best. 4.5 Evolution Trace Analysis (Q4) We analyze PRISMEMâs evolution trajectories on BEAM with Qwen3.8-27B. In Figure 3a, capability scores improve intermittently throughout Stage I, showing refinements progressively expand performance along each capability dimension. Refining one capability can also update specialists for others, reflecting dependencies among capabilities and our dependency-aware selection of targets that can benefit multiple capabilities simul- taneously. Figure 3b shows that capability-driven evolution expands beyond the performance boundary of holistic evolution, uncovering better optimization directions. Trace-guided integration then successfully con- solidates these gains, with the final program closely matching the peak capability-wise performance reached during evolution. 5 Conclusion We introduced capability-driven evolution to expand memory program optimization along explicit capabil- ity dimensions. Capability-specific feedback clarifies revision directions, while capability-level evaluation preserves gains obscured by overall scores. PRISMEM combines dependency-aware capability selection and history-guided diagnosis to refine capability specialists, then uses trace-guided integration to consolidate their complementary gains into a unified memory program. On BEAM and LongMemEval, PRISMEM outperforms the strongest baselines by 7.83â10.54 p on million-token histories. We hope this work encourages further exploration of capability-driven evolution for building stronger agent memory. References Ruike Cao, Fanyu Zhao, Fugen Yao, Liang Dong, Jian Xu, Guanjun Jiang, Yifei Zhao, Han Zhang, and Li Xiao. MemCalib: Benchmarking and optimizing memory use in LLM agents. arXiv preprint arXiv:2609.24259, 2026. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production- ready AI agents with scalable long-term memory. arXiv preprint arXiv:2504.19413, 2025. Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. LightMem: Lightweight and efficient memory- augmented generation. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=dyJ0GWpjJB. Bernal Jim Ě enez Guti Ě errez, Yiheng Shu, Weijian Qi, Sizhe Zhou, and Yu Su. From RAG to memory: Non- parametric continual learning for large language models. In Proceedings of the 42nd International Confer- ence on Machine Learning, volume 267, p. 21497â21515, 2025. Chuanrui Hu, Xingze Gao, Zuyi Zhou, Dannong Xu, Yi Bai, Xintong Li, Hui Zhang, Tong Li, Chong Zhang, Lidong Bing, and Yafeng Deng. EverMemOS: A self-organizing memory operating system for structured 9 long-horizon reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, p. 45836â45853, 2026a. Yulin Hu, Zimo Long, Jiahe Guo, Xingyu Sui, Xing Fu, Weixiang Zhao, Yanyan Zhao, and Bing Qin. OP- Bench: Benchmarking over-personalization for memory-augmented personalized conversational agents. arXiv preprint arXiv:2601.13722, 2026b. Shuo Ji, Yibo Li, and Bryan Hooi. Memory is reconstructed, not retrieved: Graph memory for LLM agents. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/ forum?id=xRVWftS3ES. Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory OS of AI agent. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 25961â25970, 2025. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAt- tention. In Proceedings of the 29th Symposium on Operating Systems Principles, p. 611â626, 2023. Chris Latimer, Nicol Ě o Boschi, Andrew Neeser, Chris Bartholomew, Gaurav Srivastava, Xuan Wang, and Naren Ramakrishnan. Hindsight is 20/20: Building agent memory that retains, recalls, and reflects. arXiv preprint arXiv:2512.12818, 2025. Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, and Jie Tan. TiMem: Temporal-hierarchical memory consolidation for long-horizon conversational agents. In Findings of the Association for Computational Linguistics: ACL 2026, p. 21700â 21720, 2026. Jiaqi Liu, Yaofeng Su, Peng Xia, Siwei Han, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. Sim- pleMem: Efficient lifelong memory for LLM agents. In Forty-third International Conference on Machine Learning, 2026a. URL https://openreview.net/forum?id=oBgLvd5YC6. Jiaqi Liu, Xinyu Ye, Peng Xia, Zeyu Zheng, Cihang Xie, Mingyu Ding, and Huaxiu Yao. EvolveMem: Self- evolving memory architecture via AutoResearch for LLM agents. arXiv preprint arXiv:2605.13941, 2026b. Qingshan Liu, Guoqing Wang, Wen Wu, Jingqi Huang, Xinqi Tao, Dejia Song, Jie Zhou, and Liang He. MemPro: Agentic memory systems as evolvable programs. arXiv preprint arXiv:2606.00619, 2026c. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, p. 13851â13870, 2024. NVIDIA. NVIDIA A100 Tensor Core GPU. https://w.nvidia.com/en-us/data-center/ a100/, 2020. Accessed: 2025-04-01. OpenAI. Introducing GPT-5.5. https://openai.com/index/introducing-gpt-5-5/, 2026. Ac- cessed: 2026-08-30. Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560, 2023. Wenbo Pan, Shujie Liu, Xiangyang Zhou, Shiwei Zhang, Wanlu Shi, Mirror Xu, and Xiaohua Jia. M â : Every task deserves its own memory harness. arXiv preprint arXiv:2604.11811, 2026. Joon Sung Park, Joseph OâBrien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bern- stein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, p. 1â22, 2023. Sidharth Pulipaka, Oliver Chen, Manas Sharma, Taaha Saleem Bajwa, Vyas Raina, and Ivaxi Sheth. Persist- Bench: When should long-term memories be forgotten by LLMs? In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=Z7Rhzk13NT. Qwen. Qwen3-Embedding-8B. https://huggingface.co/Qwen/Qwen3-Embedding-8B, 2025. Accessed: 2026-05-01. Qwen. Qwen3.8-27B. https://huggingface.co/Qwen/Qwen3.8-27B, 2026. Accessed: 2026-08- 30. 10 Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. Zep: A temporal knowl- edge graph architecture for agent memory. arXiv preprint arXiv:2501.13956, 2025. Rana Salama, Jason Cai, Michelle Yuan, Anna Currey, Monica Sunkara, Yi Zhang, and Yassine Benajiba. MemInsight: Autonomous memory augmentation for LLM agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 33136â33152, 2025. Mohammad Tavakoli, Alireza Salemi, Carrie Ye, Mohamed Abdalla, Hamed Zamani, and J. Ross Mitchell. Be- yond a million tokens: Benchmarking and enhancing long-term memory in LLMs. In The Fourteenth Inter- national Conference on Learning Representations, 2026. URL https://openreview.net/forum? id=y59hf5lrMn. Piaohong Wang, Motong Tian, Jiaxian Li, Yuan Liang, Yuqing Wang, Qianben Chen, Tiannan Wang, Zhicong Lu, Jiawei Ma, Yuchen Eleanor Jiang, and Wangchunshu Zhou. O-Mem: Omni memory system for person- alized, long horizon, self-evolving agents. arXiv preprint arXiv:2511.13593, 2025. Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. LongMemEval: Bench- marking chat assistants on long-term interactive memory. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=pZiyCaVuti. Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-Mem: Agentic memory for LLM agents. In Advances in Neural Information Processing Systems, volume 38, p. 17577â17604, 2025. B. Y. Yan, Chaofan Li, Hongjin Qian, Shuqi Lu, and Zheng Liu. General agentic memory via deep research. arXiv preprint arXiv:2511.18423, 2025. Juexiang Ye, Xue Li, Xinyu Yang, Chengkai Huang, Lanshun Nie, Lina Yao, and Dechen Zhan. MemWeaver: Weaving hybrid memories for traceable long-horizon agentic reasoning. In Findings of the Association for Computational Linguistics: ACL 2026, p. 12928â12956, 2026. Guibin Zhang, Haotian Ren, Chong Zhan, Junhao Wang, He Zhu, Wangchunshu Zhou, and Shuicheng Yan. MemEvolve: Meta-evolution of agent memory systems. In Forty-third International Conference on Machine Learning, 2026a. URL https://openreview.net/forum?id=qpkG0eKx4v. Weizhi Zhang, Xinyang Zhang, Chenwei Zhang, Liangwei Yang, Jingbo Shang, Zhepei Wei, Henry Peng Zou, Zijie Huang, Zhengyang Wang, Yifan Gao, Xiaoman Pan, Lian Xiong, Jingguo Liu, Philip S. Yu, and Xian Li. PersonaAgent: Bridging memory and action for personalized LLM agents. In Findings of the Association for Computational Linguistics: ACL 2026, p. 26421â26439, 2026b. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. MemoryBank: Enhancing large lan- guage models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, p. 19724â19731, 2024. Yue Zhou, Xiaobo Guo, Belhassen Bayar, and Srinivasan H. Sengamedu. Amory: Building coherent narrative- driven agent memory through agentic reasoning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics, p. 3926â3938, 2026. 11 A Implementation Details A.1 Capability Selection and Offline Annotation Selecting Capabilities for Our Study. Capability-driven evolution needs a vocabulary that separates recurring memory requirements and makes their progress observable. As an initial step toward capability- driven evolution, we derive five representative capabilities in Table 1 from a broad review of agent memory benchmarks and memory systems (Maharana et al., 2024; Wu et al., 2025; Tavakoli et al., 2026; Zhong et al., 2024; Rasmussen et al., 2025; Chhikara et al., 2025; Xu et al., 2025). Explicit fact recovery, changing states, persistent user constraints, evidence distributed across sessions, and insufficient or conflicting support recur across these benchmarks and systems. These requirements provide a concrete instantiation of the vocabulary used in our study, covering factual retrieval, temporal tracking, preference extraction, multi-session synthesis, and adversarial. Table 5 connects this vocabulary to existing benchmark question types. Table 5: Correspondence between benchmark question types and the memory capabilities used in our study. The entries describe primary requirements; individual questions may involve additional capabilities. Question typePrimary memory capabilities LoCoMo (Maharana et al., 2024) Single-hop; Open-domain knowledge 1 Factual retrieval Multi-hopMulti-session synthesis Temporal reasoningTemporal tracking AdversarialAdversarial LongMemEval (Wu et al., 2025) Single-session user; single-session assistantFactual retrieval Single-session preferencePreference extraction Multi-sessionMulti-session synthesis Temporal reasoning; knowledge updateTemporal tracking Abstention variants 2 Adversarial BEAM (Tavakoli et al., 2026) Information extractionFactual retrieval Event ordering; temporal reasoning; knowledge update Temporal tracking Preference following; instruction followingPreference extraction Multi-session reasoning; summarizationMulti-session synthesis Contradiction resolution; AbstentionAdversarial 1 LoCoMoâs open-domain questions require retrieving information about the speaker and combining it with external knowledge. We therefore map this question type to factual retrieval. 2 The question identifier ending in abs. An individual task can require several capabilities. For example, in BEAM (Tavakoli et al., 2026), resolving conflicting statements about whether a user has implemented a project feature requires factual retrieval to recover both claims and adversarial handling to recognize their inconsistency and request clarification. In LongMemEval (Wu et al., 2025), counting the distinct museums or galleries visited in a given month requires multi-session synthesis to combine reported visits and temporal tracking to exclude visits outside that month. These examples motivate the primaryâsecondary annotation below. OfflineCapabilityAnnotation. Question text alone is insufficient for reliably identifying a taskâs memory requirements. For example, a BEAM question asks about a userâs education and specialization in psychology, yet neither detail is provided in the history. The task therefore requires adversarial abstention despite its factual wording. We tested question-only annotation with GPT-5.5 on 700 BEAM-1M questions. The predicted pri- mary capability matched the benchmark-type mapping in Table 5 for only 337 questions (48.14%), indicating the limited reliability of question-only annotation in this setting. We therefore provide the annotator with the question, reference answer and evidence. The answer clarifies the expected response, while the evidence identifies the information needed to support it. We use LLM-based annotation so that the procedure also applies when benchmark-specific question types are unavailable. Given these inputs and the capability definitions, we use GPT-5.5 to assign one primary capability and up to two distinct secondary capabilities using the prompt template below. The primary label captures the requirement 12 whose failure would most directly prevent a correct answer; secondary labels capture additional requirements. We reuse valid annotations across runs. Capability labels are used only to aggregate capability-level feedback and select the next capability to evolve; they are never passed to the memory program. We therefore annotate only the training and validation sets, with no annotation needed for the test set. At test time, the evolved program receives the history and question, preserving the same task information available to the baselines for a fair comparison. Native benchmark types will be used to group the test set for capability-level reporting, as detailed in Section B.1. Prompt for offline capability annotation System. You are a classifier agent. Classify the question according to the memory-related capabilities it requires. Return exactly the requested JSON object. Task. Determine which long-term-memory capabilities are required to answer the current question correctly. Use the question, golden evidence, and golden answer together to identify the memory operation the question tests. Capabilities. ⢠factual retrieval: locate and preserve an explicit fact, name, exact value, list item, or other localized detail. Use it as primary when direct information recovery is the central requirement. ⢠temporal: order events, compare times, determine recency, or track an old state, a new state, and a knowledge update. ⢠preferenceextraction: recover and apply a userâs lasting preference, prior instruction, con- straint, or standing need. ⢠multisessionsynthesis: connect, aggregate, or summarize evidence distributed across multi- ple sessions or events. ⢠adversarial: abstain when memory lacks supporting evidence, or cautiously identify and resolve contradictory or misleading evidence. Choose exactly one primary capability: the capability whose failure would most directly prevent a correct answer. Add zero, one, or two distinct secondary capabilities only when they are also genuinely required. For example, a contradiction whose resolution depends on recency can use adversarial as primary and temporal as secondary; a changed preference can use preference extraction and temporal. Question: [question] Golden evidence: [reference snippets serialized as JSON] Golden answer: [reference answer] Output. Return exactly one JSON object and no other text. Allowed labels are the five identifiers above; secondary labels must be distinct and different from the primary label. "primary": "<one allowed capability>", "secondary": ["<zero to two other capabilities>"] A.2 Evidence-Based Diagnostic Metrics During the cold-start stage, we diagnose the memory pipeline by tracing each reference evidence snippet from its source dialogue segment through extraction, indexing, and retrieval. Specifically, we examine whether the relevant information is preserved in the extracted memory units, effectively represented and accessible through the index, and ultimately retrieved for the actual question. These diagnostics allow us to distinguish weaknesses in Extraction, Indexing, Planning and Retrieval across capabilities. The three diagnostics use the same embedding-based matching function. For each task x, let E x denote the set of distinct reference evidence snippets annotated by the dataset. Each e â E x can be uniquely mapped to its source dialogue segment after text normalization, from which we can obtain the corresponding extracted 13 memory units U e . To measure how well a set of text items preserves or exposes a reference evidence snippet, we compare the evidence with the most similar item in the set. Specifically, for an L2-normalized embedding function Ď and a set of text items Z, we define the maximum matching similarity as: sim max (e,Z) = max zâZ Ď(e) ⤠Ď(z), Z ̸=â , 0,Z =â . (8) This score measures whether the information in evidence e is represented by at least one item in Z. Let R e denote the text blocks retrieved from the memory index when the original evidence e is used as an oracle query, following the standard query planning and retrieval procedure. Let R x denote the text blocks retrieved by the memory program for the actual question x. We then compute: d ext (x) = 1 |E x | X eâE x sim max (e,U e ),(9) d idx (x) = 1 |E x | X eâE x sim max (e,R e ),(10) d ret (x) = 1 |E x | X eâE x sim max (e,R x ).(11) Here, d ext measures whether the relevant evidence is preserved after extraction, d idx measures whether the preserved information can be effectively accessed through the index using an oracle evidence query, and d ret measures whether the memory program retrieves the relevant evidence for the actual question. We report these diagnostics separately for each capability. Comparing them helps localize weaknesses to memory extraction, indexing, or query planning and retrieval. For example, a high d ext but substantially lower d idx suggests that the evidence is preserved in memory units but not effectively indexed or accessed. A high d idx but low d ret instead points to problems in query formulation, query planning, or retrieval. Two length statistics provide additional context: the average number of characters of the memory units and retrieved content. The former helps assess whether extraction produces overly verbose memory units, while the latter characterizes the amount of information exposed to the answer model and provides context for inter- preting retrieval quality. A.3 Diagnostic Case Presentation and Compression Diagnostic cases provide the context needed to understand and diagnose errors in the memory pipeline, espe- cially in later evolution iterations. To support detailed analysis, each case presents a comprehensive view of the information relevant to the task. Each case includes the primary and secondary capability labels, question, reference answer, predicted answer, task score, query plan, retrieved text, and a reference evidence view. The evidence view pairs each reference evidence snippet with its original source text and the memory units in which it is represented. This pairing helps distinguish missing evidence from evidence that was preserved in memory but poorly indexed, retrieved, or interpreted. Diagnosis and coding receive the same compact case material, allowing each proposed change to be traced back to the evidence that motivates it. Full source segments and retrieved passages can be lengthy, making diagnostic cases unnecessarily large. We therefore compress the material presented to the evolution model while preserving the evidence needed for diagnosis. We use a budget of 2400 characters per displayed excerpt and retain shorter texts in full. For longer texts, we divide them into source records, paragraphs, or sentences, while preserving session, date, and role headers and keeping fenced code blocks intact. A greedy selection procedure then selects spans that maximize coverage of the question and reference answer, corresponding reference evidence, and the model prediction. Span relevance combines semantic cosine similarity and stemmed non-stopword Dice overlap, weighted at 0.75 and 0.25, respectively. The questionâanswer cue has weight 1, the evidence snippets have a total weight of 1, and the prediction has weight 0.5. Including the prediction helps retain evidence that explains an incorrect answer, such as conflicting numeric values. Selected spans are restored to their original source order and separated by explicit omission markers (...). For extracted memory, we always retain at least the best-matching unit for each reference evidence snippet, add other relevant units when space permits, and deduplicate repeated displays. These choices preserve the connections among reference evidence, its memory representation, and the model prediction while removing repeated or unrelated context. We evaluate the effect of context reduction on PRISMEMâs real traces from BEAM and find that the average case-context size decreases from 16,776 to 9,428 tokens, corresponding to a 43.8% reduction. 14 A.4 Validate Generated Programs Generated memory program revisions can fail before they can be evaluated. We check program completeness, interface compatibility, and basic execution behavior before allocating task model calls, so readily detectable errors can be corrected early. The checks proceed from inexpensive source inspection to bounded synthetic execution. 1. Source and module completeness. The patched source must be nonempty, parse as Python, use allowed imports, and avoid evaluation-side interfaces. The module must load successfully and define MemoryUnit, Query, Memory, and the three nonempty extraction, planning, and answer instruction strings. A patch must make an actual source change. 2. Schema and signature compatibility. MemoryUnit and Query must be nonempty dataclasses with supported JSON-compatible fields: strings, integers, floats, booleans, string lists, and their optional forms. The constructor, index, and retrieve must preserve the required synchronous signatures. Structured outputs are checked for missing or extra fields, invalid types, and nonfinite numeric values. 3. Functional smoke tests. A local stub backend exercises construction, indexing a session with synthetic units, indexing an empty-unit session, and retrieval with a valid query. Indexing must return None; retrieval must return a string within the 16K-character context limit. These tests require no task-LLM inference. 4. Synthetic complexity checks. A larger probe indexesâź2K synthetic units across 100 batches and executes two queries with 4096-dimensional embeddings. It checks local computation and backend-call counts. A local CPU-work budget and an outer process timeout bound stalled or unexpectedly expensive execution. These checks catch malformed schemas, incomplete patches, interface mismatches, and basic execution failures before full task evaluation. A.5 Memory Interface. The memory interface defines the scope of modifications during self-evolution. We decompose the memory program into five components: Extraction, Indexing, Planning, Retrieval, and Answer, enabling precise prob- lem localization and targeted modifications by the model. The five components are defined as follows: Extraction converts history segments into structured memory units U = P.extract(H); Indexing organizes these units U into persistent memory M = P.index(U,H); Planning translates the question q into a structured query unit Q = P.plan(q); Retrieval executes the query Q against persistent memory M to return relevant content R = P.retrieve(M,Q); and Answer instructs the task model to use the retrieved content R and question q to generate a high-quality answer Ëy = P.answer(q,R). Evolution can modify the schemas of memory and query units, as well as the code and instructions for these five memory components. Here are the memory interface definitions provided in the prompt. Memory interface definition You are designing a Memory Program that implements three classes: 1. MemoryUnit (dataclass): Defines what information is captured as memory units from source text. ⢠Must be a @dataclass with typed fields ⢠An external LLM will populate several instances by generating JSON matching your field definitions ⢠Field types MUST be JSON-compatible: use only str, int, float, bool, list[str], and their Optional forms ⢠Do NOT use datetime, tuple, bytes, dict, nested dataclasses, or custom objects â the fixed harness cannot construct them from the generated JSON ⢠Use field(metadata="description": "...") to describe fields â descriptions are shown to the LLM that populates instances 2. Query (dataclass): Defines what parameters are used when retrieving from memory. ⢠Must be a @dataclass with typed fields ⢠An external LLM will populate one instance by generating JSON matching your field definitions ⢠Same JSON-compatible type restriction and field description support as MemoryUnit 3. Memory (class): The core memory system. 15 __init__(self, backend) Receives a Backend with: backend.ask(prompt, system_prompt=None, json_object=False) -> str LLM for reasoning, summarization, or reranking backend.embed(texts: list[str]) -> np.ndarray One L2-normalized embedding row per input text index(self, units: list[MemoryUnit], raw_text: str) -> None Incrementally index one source session. units contains all valid MemoryUnits generated from that ses- sion and may be empty. raw text is the raw session text with a timestamp. Calls occur in chronological order. In this function, you may merge true duplicates, normalize references to the same entity, preserve timestamps and the order of distinct, updated, or conflicting facts, and build dense, lexical, exact-value, graph, metadata, or combined indexes for retrieval. retrieve(self, query: Query) -> str Retrieve relevant information as a string. Use the Query fields to search the relevant dense, lexical, exact- value, graph, or metadata indexes, rank and combine matches, remove repeated results, and return the useful context within OUT CHARBUDGET. Allowed imports: json, re, math, hashlib, collections, dataclasses, typing, datetime, textwrap, numpy. The execution environment provides OUTCHARBUDGET, the enforced retrieval-character limit. Runtime constraints These limits are enforced during evaluation. Violating them makes the evaluation fail. ⢠retrieve() output limit: Memory.retrieve() must return at most 16000 characters. ⢠retrieve() is called once per question, and must be read-only and independent of prior query calls. Instruction constants (required) Three module-level string constants provide the natural-language instructions used in task-LLM prompts: ⢠MemoryExtractionInstruction: Instruction for MemoryUnit generation. Tells the task LLM to extract atomic facts, procedures, relationships, decisions, and preferences from each segment, re- solve references, and preserve names, exact values, dates/times, and lists. The MemoryUnit schema is provided separately â no need to describe field names or types here. ⢠QueryPlanningInstruction: Instruction for Query generation. Tells the task LLM how to for- mulate retrieval queries and tailor them to the capabilities required by the question, including splitting multi-aspect questions when useful. The Query schema is provided separately â no need to describe field names or types here. ⢠AnswerInstruction: Instruction for answer generation. Controls answer format, length, and style, and guides how retrieved content is interpreted and used. Apply question-specific strategies when needed, including distinguishing user statements from assistant suggestions, resolving updates or con- tradictions using source, time, and scope, counting distinct events rather than memory units or repeated mentions, and synthesizing facts across sessions. Preserve relevant chronology and specific changes for progression questions. Ensure that information needed for source, time, or event identification is preserved in memory and exposed during retrieval. MemoryExtractionInstruction, QueryPlanningInstruction, and AnswerInstruction must not be empty. A.6 Revision-Aware Execution Reuse A local revision often leaves most of the memory pipeline unchanged. We compare the generated candidateâs components with those of its parent and begin re-execution at the earliest affected stage, as shown in Table 6. For example, a new ranking rule can reuse extracted units and the built memory, whereas a changed extrac- tion instruction requires new units and all downstream computations. This dependency-aware reuse avoids unnecessary LLM calls while preserving evaluation coverage. Reusable state is copied before being attached to the candidate, keeping parent and child memory states sepa- rate. Changes to shared helpers invalidate indexing conservatively because those helpers may affect memory 16 Table 6: Re-execution boundaries for a candidate relative to its parent. For changes spanning several compo- nents, the earliest affected stage determines what can be reused. Changed componentRe-execution begins atReusable work Memory-unit schema or extraction stage ExtractionSegmentation Constructor, index, or helpersIndexingSegmentation and extracted units Query schema, planning, or retrieval Query planning Segmentation, extraction, and indexed memory Answer instruction onlyAnswer generation Built memory, query plan, and retrieved context construction. Reuse follows the programâs execution dependencies and the actual inputs to cached computa- tions. A.7 Seed Memory Program The initial seed memory program is deliberately simple, providing a broad search space. It appends ex- tracted units in input order and returns their concatenation, truncated to the context budget, without query- dependent ranking. The execution environment fixes the output character budget at 16K (OUT CHARBUDGET = 16000) across evolution runs. The implementation is shown below. Initial seed memory program used for PRISMEM from dataclasses import dataclass, field MemoryExtractionInstruction = "Extract the key information from the text into several memory units." QueryPlanningInstruction = "Given the following question, generate a query to retrieve relevant memory." AnswerInstruction = "Based on the retrieved memory and the original question, provide a short answer without explanation." @dataclass class MemoryUnit: memory_text: str = field( metadata="description": "One piece of information worth remembering from the text" ) @dataclass class Query: query_text: str = field( metadata="description": "A query for retrieving information relevant to the question" ) class Memory: def __init__(self, backend): self.backend = backend self.memories: list[str] = [] def index(self, units: list[MemoryUnit], raw_text: str) -> None: self.memories.extend(unit.memory_text for unit in units) def retrieve(self, query: Query) -> str: return " ".join(self.memories)[:OUT_CHAR_BUDGET] 17 A.8 Pseudocode of PrisMem We provide the pseudocode of PRISMEM below, illustrating its execution flow. Algorithm 1 PRISMEM: Capability-Driven Memory Evolution Require: Seed P 0 ; datasets S train ,S valid ; capability set C; iterations for three stages T 1 ,T 2 ,T 3 with T = T 1 + T 2 + T 3 ; case budget K; thresholds δ,θ. Ensure: Final memory program G T . 1: W âP 0 ; EVALUATE(P 0 ,S train ,S valid ) 2: for t = 1,...,T 1 do⡠Stage I: Cold Start 3: P â SELECT(W,t); M â COMPUTEMETRICS(P,S train ) 4: dâ DIAGNOSE(P,M ); P Ⲡâ REVISE(P,d,M ) 5:if P Ⲡ̸=⼠and EVALUATE(P Ⲡ,S train ,S valid ) succeeds then 6: W âW âŞP Ⲡ7:end if 8: end for 9: P base â arg max PâW s(P ;S valid );P T 1 âP base ; G T 1 â P base 10: P â T 1 (c)â P base for every câC 11: for t = T 1 + 1,...,T 1 + T 2 do⡠Stage I: Capability Refinement 12: c t â SELECTCAPABILITY(P â tâ1 (c) câC ,t)⡠Priority Ď t (c) via Equation (6) 13: P t â P â tâ1 (c t ) 14: B t â TOPK(x i âS c t train : s(P t ;x i ) < δ,w t ,δ,K)⡠Curate via Equation (7) 15: Z t âCASE(P t ,x)| xâB t 16: d t â DIAGNOSE(P t ,c t ,Z t ) 17: P Ⲡâ REVISE(P t ,d t ,Z t ) 18:if P Ⲡ̸=⼠and s(P Ⲡ;S c t train ) > s(P t ;S c t train ) then 19:if EVALUATE(P Ⲡ,S train ,S valid ) succeeds then⡠Full-suite evaluation 20:P t âP tâ1 âŞP Ⲡ21:else 22:P t âP tâ1 23:end if 24:else 25: P t âP tâ1 26:end if 27: P â t (c)â arg max PâP t s(P ;S c valid ) for every câC 28: G t â arg max PâP t s(P ) 29: end for 30: E ââ ;B ââ ; T Ⲡâ T 1 + T 2 ⡠Stage I: Integration 31: G T â G T Ⲡ;T â EVOLUTIONTRACE(P T Ⲡ); 32: for all câC where s(P â T Ⲡ(c);S c train ) > s(G T Ⲡ;S c train ) do 33:Ëx c â arg max xâS c train s(P â T Ⲡ(c);x)â s(G T Ⲡ;x) 34: E âE âŞCASE(P â T Ⲡ(c), Ëx c ), CASE(G T Ⲡ, Ëx c ) 35: B âBâŞËx c 36: end for 37: ifE ̸=â then 38:Î â PLANINTEGRATION(G T Ⲡ,P â T Ⲡ(c) câC ,T ,E )⡠Generate plan 39:for t = T Ⲡ+ 1,...,T do 40:P t â EXECUTEINTEGRATE(G T Ⲡ, Î ,E )⡠Coding with plan and paired cases 41:if P t ̸=⼠and s(P t ) > s(G T Ⲡ) and max câC [1â s(P t ;S c train )/s(G T Ⲡ;S c train )]⤠θ then 42:G T â P t 43:break 44:else 45:E âE âŞCASE(P t ,x)| xâB⡠Accumulate cases 46:end if 47:end for 48: end if 49: return G T ⡠Final unified program 18 B Experimental Details B.1 Dataset Splits and Capability Composition We evolve memory programs on the shorter-history BEAM-100K and LongMemEval-S splits (âź100K tokens) and evaluate their frozen implementations on the million-token BEAM-1M and LongMemEval-M splits. This setup reduces the risk of overestimating evolutionary performance due to overfitting and provides a more direct evaluation of the evolution methodâs ability to generalize to million-token histories. For BEAM, the evolution split contains 160 training questions from 8 conversations and 120 validation ques- tions from 6 conversations, while the test set contains all 700 BEAM-1M questions from another 35 conversa- tions. For LongMemEval, each question is associated with a specific conversation. To reduce evaluation cost while maintaining a balanced question-type distribution, we stratify by question type and randomly sample 60 questions for training and 60 for validation in total. Following the same stratified sampling procedure, we construct a 200-question subset of LongMemEval-M with different histories for testing. The frozen program constructs memory from each test history with its implementation held fixed. Table 7 reports the capability counts and average history length of each split. Training and validation labels are assigned by the LLM annotator in Section A.1. Test questions are grouped using the native question-type mapping in Table 5. Table 7: Sizes, primary capability composition, and average history length of each dataset split. F, T, P, M, and A denote factual retrieval, temporal tracking, preference extraction, multi-session synthesis, and adversarial, respectively. Average history length is measured with the Qwen3.8-27B tokenizer. DatasetSplitFTPMATotalAvg. tokens (K) BEAM-100KTrain2043323332160125.11 BEAM-100KValidation1832242224120146.83 BEAM-1MTest702101401401407001,089.57 LongMemEval-STrain1917119460109.40 LongMemEval-SValidation201788760109.40 LongMemEval-MTest64643032102001,079.04 B.2 Parameter Settings Model Inference Parameters. For task models, we disable thinking or use low thinking effort with a maximum of 8,192 tokens. For diagnosis and coding models, we enable thinking with medium reasoning effort and a maximum of 65,536 tokens. All methods use the same inference settings. For benchmark judging, we use the same model as the task model and set the temperature to 0 for deterministic outputs. We test Qwen3.8-27B and GPT-5.5 in our experiments. Table 8: Parameter settings of PRISMEM we used in experiments. ParameterValue Stage I: Capability refinement Case score threshold δ0.8 Refinement case budget K5 Stage I: Integration Capability-wise score degradation ratio θ0.1 Integration case budget1 per capability Algorithm Parameters. We use an evolution budget of T = 20 rounds for all evolutionary methods. For PRISMEM, we allocate T 1 = 5 rounds to cold start, T 2 = 12 to capability refinement, and T 3 = 3 to final integration. Table 8 lists the parameter values used for stages I and I. 19 C Additional Experimental Results C.1 Results on GPT-5.5 Table 9 shows the performance of different methods on BEAM using GPT-5.5. The results preserve the same trends observed with Qwen3.8-27B (Table 2), demonstrating the robustness of PRISMEM across different un- derlying models. Table 9: Performance on BEAM with GPT-5.5. Method Overall ScoreAvg. #Tokens (%)â(K)â Mem057.77Âą 0.17180.21Âą 0.12 A-MEM60.92Âą 0.41352.11Âą 0.25 HippoRAG257.73Âą 0.50193.52Âą 0.18 SimpleMem52.36Âą 0.55121.39Âą 0.69 LightMem46.68Âą 0.0594.05Âą 0.19 M â 60.75Âą 2.5397.47Âą 4.75 EvolveMem46.77Âą 1.5694.39Âą 3.08 PRISMEM67.09Âą 1.7999.36Âą 3.62 C.2 Transfer Experiments We examine whether PRISMEMâs evolved memory programs transfer beyond the model and dataset used during evolution. We evaluate memory programs evolved with Qwen3.8-27B on BEAM-100K. For model transfer, we evaluate these programs with GPT-5.5 on BEAM-1M. For dataset transfer, we retain Qwen3.8-27B and evaluate on LongMemEval-M. The other settings remain the same as those in the main experiments. The results are summarized in Table 10 and Table 11. Table 10: Model transfer evaluated with GPT-5.5 on BEAM-1M. All programs are evolved on BEAM-100K. Overall scores (%) are reported as âmeanÂą stdâ over three runs. Method Evolved with Qwen3.8-27B (transfer) Evolved with GPT-5.5 (target model) PRISMEM67.25Âą 1.3567.09Âą 1.79 Table 11: Dataset transfer evaluated with Qwen3.8-27B on LongMemEval-M. All programs are evolved with Qwen3.8-27B. Overall scores (%) are reported as âmeanÂą stdâ over three runs. Method Evolved on BEAM (transfer) Evolved on LongMemEval (target dataset) PRISMEM76.50Âą 1.6775.50Âą 1.80 These results show that the evolved memory programs maintain strong performance even when transferred to different models and datasets without additional program optimization, demonstrating the robustness and generalizability of the capability-driven evolution approach. C.3 Capability Router Experiments We compare trace-guided integration with routing among capability-best programs from the same PRISMEM evolution trajectory on BEAM. The question-only router uses Qwen3.8-27B to predict the primary capability from the question alone, since reference answers and evidence are unavailable at test time. The oracle-label router uses the benchmark question types mapped to capabilities in Table 5. Both routers select the corre- sponding capability-best program from the evolution run. In contrast, integration uses a single program for all questions without capability labels at test time. Overall scores are computed over all 700 questions in BEAM-1M. 20 Table 12: Routing versus integration on BEAM-1M with Qwen3.8-27B. StrategyOverall Score (%) Question-only router57.74Âą 1.21 Oracle-label router62.27Âą 1.14 PRISMEM63.39Âą 1.02 Table 12 shows that integration outperforms question-only routing by 5.65 p and oracle-label routing by 1.12 p. Question-only routing is limited by the difficulty of inferring memory requirements from the question alone during test time, without access to the reference answer and supporting evidence; its predicted labels match the oracle mapping for only 48% of the total 700 questions. Although oracle-label routing uses accurate capability labels, it still selects individual programs optimized primarily along specific capability directions. Since a question may require multiple capabilities, these programs may provide unbalanced coverage of its requirements. In contrast, trace-guided integration combines complementary mechanisms within a shared pro- gram and reconciles their conditions of use, resulting in stronger overall performance. These results support the value of integrating useful capability-specific revisions into a shared memory program. D Detailed Evolution Trace Table 13 summarizes key evolution iterations of PRISMEM evolved on BEAM with Qwen3.8-27B, including the diagnosed failure, resulting revision, and modified memory program components. Blue tags denote the capability targeted by each evolution step, while gray letter boxes mark capabilities for which the resulting node achieves the best score (specialist). Table 13: BEAM evolution traces with failure diagnoses, revision summaries, and current specialist. F: factual retrieval, T: temporal tracking, P: preference extraction, M: multi-session synthesis, A: adversarial. Evolution Trace Failure diagnosisRevision summaryCurrent best for Stage I: Cold-start revisions n05 A single flat query could not capture multiple semantic aspects of synthesis questions, causing retrieval to over-concentrate on one topic while missing complementary evidence scattered across sessions. Decompose multi-aspect questions into focused sub-queries and retrieve each aspect independently with proportional budget allocation and cross-query deduplication. schemaplanningretrievalanswering MT AP F Stage I: Capability-directed refinement n05ân06 Synthesis Over-atomized memory units fragmented each logical item into multiple small facts, causing related evidence to appear as an unstructured list and hindering distinct-item counting and cross-session progression synthesis. Add a topic label to each memory unit and group retrieved evidence by topic, so related facts are presented as coherent blocks for counting and multi-session synthesis. schemaextractionindexingretrieval answering F n05ân07 Temporal Session-level temporal provenance was lost during indexing, so retrieved memories lacked discussion-time context and the model could not reliably distinguish statement order from event dates. Add session date to each memory unit, recover it from the source session during indexing, and expose explicit date markers in retrieved evidence while preserving relevance-based ranking. schemaextraction indexing retrieval answeringhelpers FT P 21 Evolution Trace Failure diagnosisRevision summaryCurrent best for n05ân08 Adversarial Memory units lacked speaker provenance, causing user-confirmed facts to be conflated with assistant recommendations or hypotheticals and obscuring genuine contradictions between user statements. Add explicit speaker labels to memory units and propagate them through retrieval, then constrain answering to treat user-sourced units as direct evidence and surface conflicting user statements without resolution. schemaextractionindexingretrieval answering A n05ân09 Synthesis Per-unit relevance ranking favored locally similar facts while dropping weaker but essential progression evidence, and the remaining units were returned as a flat list, preventing reconstruction of coherent cross-session threads. Add topic-level thread labels, group memories by topic and session order, retrieve matched threads as chronological blocks, and use relevance ranking only to fill remaining uncovered evidence. schemaextractionindexingretrieval answeringhelpers P n09ân10 Preference Standing preferences were retrieved only through topical relevance, so cross-cutting user constraints were often omitted when the current query concerned a different domain. Add an explicit standing-preference flag and reserve retrieval capacity for these units, ensuring persistent user constraints are always exposed alongside task-specific evidence. schemaextractionindexingretrieval answeringhelpers M n07ân13 Synthesis The AnswerInstruction lacked explicit synthesis rules for organizing retrieved evidence across topics and time, leading to fragmented narratives and inconsistent handling of counts, cumulative values, and distinct focus areas. Add concise answer-side guidance to build chronological topic narratives, distinguish duplicates from cumulative or additive values, and enumerate distinct areas consistently. answering F n07ân15 Preference Retrieved user preferences were treated as passive context rather than binding response constraints, causing the answer LLM to ignore required formatting and presentation behaviors. Strengthen the AnswerInstruction to enforce retrieved standing preferences as mandatory constraints on response structure and content. answering P n07ân17 Adversarial Existence questions were planned primarily around affirmative evidence, causing explicit negations to receive low retrieval priority and making user-level contradictions easy to miss. Make existence-query planning contrastive by retrieving both affirmative and negative evidence, and require the answer LLM to surface both sides when they conflict. planninganswering AT Stage I: Final integration n20 Capability specialists captured complementary strengths, but their mechanisms were not directly composable: aggressive topic grouping, preference enforcement, and aggregation rules could degrade relevance, temporal ordering, or contradiction handling when combined. Merge the complementary mechanisms conservatively by adding topic and standing-preference metadata, lightweight topic-aware scoring, and concise synthesis/preference rules while preserving flat relevance retrieval and adversarial safeguards. schemaextraction indexing retrieval answering helpers 22 E Cases and Suggested Revisions on Different Capabilities We examine three cases from memory evolution trajectories on BEAM, each contrasting two revisions with different capability effects and a third revision that addresses both needs. These cases show that different ca- pabilities may suggest different revisions, yet they are not inherently incompatible: by comparing the strengths and limitations of revisions across capabilities, integration can identify and combine complementary improve- ments. PRISMEM leverages evolved programs and their evaluation results across capabilities to guide this integration and improve overall performance. E.1 Case 1: Store each interaction as a whole versus each fact separately. Temporal tracking questions compare events or values at different points in a userâs history, requiring the mem- ory to preserve relationships among related details. The diagnosis found that a single request or calculation was split across multiple memory units, causing its steps, values, and outcome to be treated as separate interactions. The model then revised the extraction instruction to keep each complete interaction in one unit. This improved temporal tracking by preserving relationships across events. However, grouping related information also com- bined target statements with long, less informative context, causing the answer LLM to overlook contradictions with opposing statements elsewhere in the history. Temporal tracking increased by 3.41 p, while adversarial handling decreased by 14.06 p. Adversarial questions require identifying conflicting claims, such as a user stating both that they have practiced a topic and that they have never practiced it. The diagnosis found that distinct factual statements were some- times omitted during extraction, motivating a revision to the extraction instruction that preserves each distinct fact independently and recovers missing denials in adversarial questions. However, separating facts also dis- persed related values across memory units, making it harder to associate them with the events that produced them. Adversarial handling increased by 12.89 p, while temporal tracking decreased by 6.90 p. These cases show that temporal tracking benefits from preserving relationships among events and values, while contradiction detection requires opposing claims to remain independently accessible. One can refine the ex- traction instruction to ask the model to preserve a complete plan, process, or timeline within one memory unit while storing contradictory claims separately. For example, a problem count and its associated score remain together, while a denial of prior practice remains separate from evidence of completed practice. This revision improved both capabilities: temporal tracking increased by 0.87 p, and adversarial handling increased by 2.73 p. E.2 Case 2: Reconstruct chronological order versus require explicit causal links. Temporal tracking questions require reconstructing the order of events across conversations. The diagnosis found that answers grouped concrete events into broad activity categories, obscuring their sequence. The model revised the answer instruction to organize specific events in chronological session order, making the sequence clearer. However, this sometimes led the model to infer a causal relationship from temporal order, treating an event that occurred earlier as having caused a later event. This caused adversarial performance to decrease by 9.38 p, while temporal tracking increased by 5.61 p. Adversarial questions about causal influence require checking whether the history explicitly connects a stated cause to its effect. The diagnosis found that chronological ordering of related events was sometimes treated as evidence of causal influence. The model revised the answer instruction to require an explicit causal link and abstain when such evidence was absent. However, this overly strict criterion caused some temporal questions requiring an explicitly stated sequence to be incorrectly rejected. Adversarial handling increased by 8.20 p, while temporal tracking decreased by 4.25 p. These cases highlight two distinct requirements for relating events: temporal tracking can reconstruct order from dated records, whereas causal influence requires explicit evidence connecting cause and effect. A good answer instruction revision should ask the model to organize answers to temporal and progress questions around the userâs specific events in chronological session order, while checking for an explicit connection between cause and effect in causal and influence questions. This revision improved both capabilities: temporal tracking increased by 2.71 p, and adversarial handling increased by 8.59 p. E.3 Case 3: Increase retrieval diversity versus tighten relevance filtering. Multi-session synthesis questions require covering different relevant aspects of a userâs history. The diagnosis found that retrieval repeatedly selected similar records from one dominant topic, leaving other aspects un- derrepresented. The model therefore revised retrieval to use maximal marginal relevance (MMR), balancing 23 relevance to the question with diversity among selected records. This broadened the evidence available for synthesis, but could also introduce records from other events, making it harder to associate dates and updates with the target event in temporal questions. Thus, multi-session synthesis increased by 7.08 p, while temporal tracking decreased by 5.95 p. Temporal tracking questions instead require closely related records about the target event to compare dates and track updates. The diagnosis found that records about nearby events interfered with identifying the correct timeline. The model raised the candidate relevance threshold for MMR (e.g., 0.6â 0.7), focusing retrieval on records more closely related to the question. However, stricter filtering also removed records covering other relevant aspects, narrowing the evidence available for broad synthesis. Temporal tracking therefore increased by 5.87 p, while multi-session synthesis decreased by 5.62 p. These cases highlight two retrieval requirements: broad coverage for multi-session synthesis and focused re- trieval of related records for temporal tracking. A good retrieval revision should be able to adjust the MMR balance according to the predicted question type: synthesis questions favor greater diversity (e.g., 0.6 MMR weight), while temporal and progression questions favor relevance to retain closely related records (e.g., 0.7 MMR weight). This revision improved both capabilities: multi-session synthesis increased by 3.92 p, and temporal tracking increased by 3.70 p. F Prompt Catalog We present the reflection, refinement coder, planning, and integration coder prompts used in PRISMEM. The reflection and refinement coder templates are used in Stage I (cold start) and Stage I (capability refinement), while the planning and integration coder templates are used in Stage I (integration). Reflection prompt template used in Stages I and I System. You are an expert in long-term memory systems for LLM agents. Think carefully, ground every conclusion in the supplied implementation and measurements, and return exactly the requested format. Task. You are diagnosing a memory system serving many long dialogues between a user and an assis- tant. The system uses an external LLM to generate multiple MemoryUnits from each dialogue segment. It passes all units from each source session and the raw session text with timestamp to Memory.index in chronological order, building memory incrementally across sessions. For each question, the external LLM generates a typed Query, which is passed to Memory.retrieve once. The retrieved text and An- swerInstruction are then provided to the answer LLM. The goal is to maximize answer accuracy within a 16000-character retrieval budget. Consider the recall-precision trade-off throughout the entire memory pipeline: each memory unit should retain sufficient factual information while avoiding unnecessary noise. Too little retrieval may miss key evidence, while too much may interfere with answering. The memory system is evaluated across five capabilities: ⢠factual retrieval: preserve and locate explicit facts, names, exact values, lists, and local details; ⢠temporal: distinguish states of the same fact at different times, order events, determine recency, and track updates over time; ⢠preferenceextraction: recover and apply long-term user preferences, instructions, constraints, and needs; ⢠multisessionsynthesis: connect and summarize evidence distributed across multiple sessions; ⢠adversarial: recognize when the dialogue contains no relevant information and abstain, or detect and cautiously handle contradictory or misleading evidence. Different capabilities may require different memory designs. You can use the Query Planning stage to identify the capabilities required by the question and, when appropriate, adapt the query and retrieval strategy accordingly to preserve accuracy across capabilities. Stage I: cold start Analyze the provided measurements and cases carefully. Identify the capability with the largest accuracy gap, diagnose its most important cause in the current implementation, and propose exactly one diagnosis and implementation change. Stage I: capability refinement For the target capability, diagnose the main cause of errors in the current implementation and propose exactly one diagnosis and implementation change. Target capability: [Target capability] 24 Consider both dimensions before choosing that single change: (A) Prompt Optimization: How should MemoryExtractionInstruction, QueryPlanningInstruction, or An- swerInstruction change to guide the task LLM better? (B) Memory Design: How should the MemoryUnit/Query schemas or Memory.index/retrieve logic change to store and retrieve more useful information? Memory abstractions. Insert the memory-interface prompt. Current implementation. [Complete parent source] Current measurements. Training accuracy by capability and average score: [Training capability profile and average score] Stage I only: evidence-based metrics Below are diagnostic metrics computed using semantic similarity at different memory stages. The table below reports the average answer accuracy (Acc) and evidence-similarity diagnostics for each capability. Use these metrics to help locate problems in the current implementation. Semantic similarity is robust to paraphrasing, but may remain high even when exact details are missing or changed. These metrics are imperfect intermediate indicators, so analyze them together with representative cases rather than optimizing them directly. ⢠evidence extracted: how closely golden evidence is represented by generated MemoryUnits from its source segment ⢠evidence oracleretrieved: how reachable golden evidence is through Query generation and Memory.retrieve when the evidence itself is used as the question ⢠evidenceretrieved: how much golden evidence Memory.retrieve returns for the real question [Table with columns: Capability, Acc, evidenceextracted, evidenceoracleretrieved, evidenceretrieved; one row per capability] Below are the length-diagnostic measurements across all questions. ⢠chars perunit: the average generated MemoryUnit JSON length; larger and smaller values each have recall and precision trade-offs ⢠retrievedchars: the average real-question retrieval length; larger and smaller values each have recall and precision trade-offs [Length measurements with name, definition, and current value] Representative cases. Below are representative cases for reference. âdiagnosticsâ contains the measurement results for each case. âprimaryâ is the primary capability; âsecondaryâ lists the secondary capabilities.âoriginaltextâ shows the original raw text of each golden evidence. âextractedasâ shows how golden evidence text is extracted and stored in MemoryUnits by the current implementation. âretrievedâ shows text returned by Memory.retrieve for the question. If âoriginaltextâ, âextractedasâ, or âretrievedâ is too long, it may be truncated to retain information relevant to answering the question. [Representative cases serialized as JSON: one per capability in Stage I, or up to five target-capability cases in Stage I] Previously explored revisions (when available). Stage I: cold start Here are previously explored revisions from the same current implementation. You can refer to them and avoid repeating unhelpful revision directions. [Base and child revisions with summaries, modified components, capability scores, and average score] Stage I: capability refinement Here are previously committed versions for the target capability. Use them to understand which changes have already been explored and avoid reverting useful improvements. [JSON list with node, summary, components, and training profile] 25 Output. Analyze the provided measurements and cases to identify the main reasons why the current im- plementation fails to produce correct results. Ground the diagnosis in the implementation, measurements, previously explored revisions, repeated nearby unsuccessful directions when shown, and representative cases. Return one diagnosis with one concrete implementation change. Answer with exactly one JSON object: "diagnosis": "<one code-and-case-grounded problem and concrete implementation change>" Refinement Coder prompt template used in Stages I and I System. You are an expert in long-term memory systems for LLM agents. Think carefully, ground every conclusion in the supplied implementation and measurements, and return exactly the requested format. Task. You are improving a memory system serving many long dialogues between a user and an assis- tant. The system uses an external LLM to generate multiple MemoryUnits from each dialogue segment. It passes all units from each source session and the raw session text with timestamp to Memory.index in chronological order, building memory incrementally across sessions. For each question, the external LLM generates a typed Query, which is passed to Memory.retrieve once. The retrieved text and An- swerInstruction are then provided to the answer LLM. The goal is to maximize answer accuracy within a 16000-character retrieval budget. Consider the recall-precision trade-off throughout the entire memory pipeline: each memory unit should retain sufficient factual information while avoiding unnecessary noise. Too little retrieval may miss key evidence, while too much may interfere with answering. The test questions cover temporal reasoning, evidence connected across sessions, facts updated over time, contradictions that must be surfaced, abstention when dialogue contains no answer, and long-term prefer- ences or instructions that should guide an answer. Implement the requested change based on the diagnosis feedback for the current implementation; do not broaden it into an unrelated redesign. Implement the requested algorithm efficiently without sacrificing its intended functionality or accuracy. Memory abstractions. Insert the memory-interface prompt. Current implementation. [Complete fixed parent source] Diagnosis. [Diagnosis] Representative cases. Use the following cases to understand the diagnosis and implement the re- quested change. âprimaryâ is the primary capability; âsecondaryâ lists the secondary capabil- ities. âoriginaltextâ shows the original raw text of each golden evidence. âextractedasâ shows how golden evidence text is extracted and stored in MemoryUnits by the current implementation. âretrievedâ shows text returned by Memory.retrieve for the question. If âoriginaltextâ, âextractedasâ, or âretrievedâ is too long, it may be truncated to retain information relevant to answering the question. [The same compact cases used for diagnosis, serialized as JSON] Revision history (when available). Stage I: cold start The following are recent, similar revision directions that did not improve accuracy. Do not repeat these directions. Use the diagnosis to make a change in a different component or another part of the memory pipeline. [Revision summaries and modified components] Stage I: capability refinement Here are previously committed versions for the target capability. Use them to understand which changes have already been explored and avoid reverting useful improvements. [JSON list with node, summary, components, and training profile] How to write the patch. The patcher locates hunks from quoted context rather than trusting line numbers. Copy context exactly, begin every changed region with an @@ hunk header, and use separate hunks for non-contiguous edits. The patch must compile, make a real change, and preserve the required schemas, 26 instructions, and three Memory signatures. Return the diff against Current implementation, not against a previous attempt. Example: @@ -MemoryExtractionInstruction = "Extract the key information from the text into several memory units." +MemoryExtractionInstruction = "Extract distinct facts, updates, and preferences from the text into several memory units." Previous-attempt feedback (when available). If the previous attempt failed during patching, checks, or execution: Previous failed attempt: [Previous summary, code, and error] Repair this error. Return a complete diff against the original Current implementation. If the previous attempt was measured but not accepted: Previous measured attempt: [Previous reply JSON with summary and code] [Measurement JSON with parent, candidate, and reasons] Analyze this failure and revise the original parent following the diagnosis. Output. Answer with exactly one JSON object: "summary": "<one concise change description>", "code": "<unified diff against Current implementation>" Planning prompt template used in Stage I System. You are an expert in long-term memory systems for LLM agents. Think carefully, ground every conclusion in the supplied implementation and measurements, and return exactly the requested format. Task. Plan one integrated Memory Program using Current implementation as the sole base. Compare the supplied capability specialists using their implementations, training gains over the base, and paired cases. Select compatible improvements, retain useful base behavior, and resolve conflicts and shared dependencies. Return an implementation-ready plan: specify what to change, where, and how, including required schema, instruction, indexing, retrieval, and helper updates. Memory abstractions. Insert the memory-interface prompt. Current implementation. [Complete overall-best base source] Capability specialist sources. [JSON object identifying base-equivalent versions and distinct specialist sources, with their capability roles and training gains over the base] Current measurements. Training accuracy by capability and average score: [Base capability-profile JSON] Length diagnostics: ⢠chars perunit: the average generated MemoryUnit JSON length; larger and smaller values each have recall and precision trade-offs ⢠retrievedchars: the average real-question retrieval length; larger and smaller values each have recall and precision trade-offs [JSON list with name, definition, and value for each length diagnostic] Representative cases. Compare base with capability specialist outputs; * parent denotes a specialistâs earlier parent for retention comparison. âoriginaltextâ shows the original raw text of each golden evidence. âextractedasâ shows how golden evidence text is extracted and stored in MemoryUnits by the current implementation. âretrievedâ shows text returned by Memory.retrieve for the question. If âoriginal textâ, âextractedasâ, or âretrievedâ is too long, it may be truncated to retain information relevant to answering the question. 27 [Paired-case JSON: question, goldenanswer, originaltext, and the available base, capabilitycase, and capability parent outputs; each output contains predictionanswer, answerscore, queryplan, retrieved, and extractedas] Output. Return exactly one JSON object: "Step-by-step plan": "<concrete integration plan specifying changes and dependencies>" Integration Coder prompt template used in Stage I System. You are an expert in long-term memory systems for LLM agents. Think carefully, ground every conclusion in the supplied implementation and measurements, and return exactly the requested format. Task. Implement the integration plan against Current implementation, using the supplied capability spe- cialist sources. Integrate required schema, instruction, indexing, retrieval, and helper dependencies to- gether; resolve conflicts and preserve compatible base behavior. Use measured case changes to refine the integration, not benchmark-specific rules. Memory abstractions. Insert the memory-interface prompt. Current implementation. [Complete fixed overall-best base source] Capability specialist sources. [JSON list of distinct specialist sources, their capability case version names, and traininggainoverbase] Merge plan. [JSON object containing the integration plan] Representative cases. âoriginaltextâ shows the original raw text of each golden evidence. âextractedasâ shows how golden evidence text is extracted and stored in MemoryUnits by the current implementation. âretrievedâ shows text returned by Memory.retrieve for the question. If âoriginaltextâ, âextractedasâ, or âretrievedâ is too long, it may be truncated to retain information relevant to answering the question. [Paired-case JSON with the shared question, golden answer, original evidence, and base and capability specialist outputs] How to write the patch. The patcher locates hunks from quoted context rather than trusting line numbers. Copy context exactly, begin every changed region with an @@ hunk header, and use separate hunks for non-contiguous edits. The patch must compile, make a real change, and preserve the required schemas, instructions, and three Memory signatures. Return the diff against Current implementation, not against a previous attempt. Example: @@ -MemoryExtractionInstruction = "Extract the key information from the text into several memory units." +MemoryExtractionInstruction = "Extract distinct facts, updates, and preferences from the text into several memory units." Previous attempts (when available). Compare each attemptâs case outputs with the corresponding base and capability specialist outputs. Use the implementations to identify integration gains, conflicts, and regressions. Missing views are unavailable, not unchanged. Return a complete diff against Current imple- mentation. [JSON object keyed by base attempt1, baseattempt2, etc.; each entry contains the available source or patch, training result, error, and per-case candidate outputs] Output. Return exactly one JSON object: "summary": "<one concise change description>", "code": "<unified diff against Current implementation>" 28 G Evolved Memory Program In this section, we present the final memory programs evolved by PRISMEM on BEAM and LongMemEval. G.1 BEAM Final Memory Program The program evolved by PRISMEM on BEAM augments dated atomic memories with topic identifiers and a flag for broadly applicable user preferences or standing instructions. ⢠Extraction assigns related facts to a shared topic and distinguishes cross-topic preferences from project- specific constraints. ⢠Indexing stores the embeddings and metadata, recovers missing session dates from the input header, and collects distinct standing preferences separately. ⢠Planning splits multi-aspect questions into subqueries and adds a query for negative evidence when checking whether an action occurred. ⢠Each subquery ranks memory units using cosine similarity with weight 0.7 and lexical query-token coverage with weight 0.3, plus a topic-based bonus of 0.05. Within the 16000-character context budget, retrieval reserves space for a standing-preference block capped at 400 characters and divides the remaining space equally among subqueries. It removes exact-text duplicates across subqueries and appends the preference block after the date-prefixed results. ⢠The answering instruction distinguishes statement time from event time, organizes chronological summaries, separates distinct-event sums from running totals, and applies retrieved preferences to the response. It calls for clarification when affirmative and negative evidence conflict, and abstention when evidence is insufficient. Final memory program evolved by PRISMEM for BEAM import re import numpy as np from dataclasses import dataclass, field MemoryExtractionInstruction = ( "Extract distinct atomic facts, decisions, preferences, " "instructions, and temporal events from the text. " "Resolve pronouns and references to full names. Preserve " "exact values, names, dates, times, and lists verbatim. " "If the text contains a user preference or standing " "instruction, capture it explicitly as its own memory " "unit. For cross-cutting preferences or standing " "instructions that apply broadly regardless of the " "specific topic or project (e.g. âalways format code " "with syntax highlightingâ, âprefer minimal " "dependenciesâ, âalways compare multiple solution " "methodsâ), set standing_preference to True. For " "preferences tied to a specific project, feature, or " "domain (e.g. âprefers Bootstrap for this websiteâ), set " "standing_preference to False. Capture any " "contradictions or updates to previously stated facts. " "For each fact, assign a concise topic identifier " "(entity+thread in lowercase hyphenated form, e.g. " "âweather-app-autocompleteâ, " "âgreg-research-collaborationâ, " "âtriangle-classification-progressâ) that groups all " "related facts about the same ongoing thread, project, " "or person across sessions. Set session_date to the " "current conversation session date in Y-M-D format " "(the date the user discussed the topic in chat, NOT any " "project or schedule date in the content)." ) QueryPlanningInstruction = ( "Given the question, decompose it into one or more " "retrieval sub-queries. For single-topic questions, " "provide exactly one sub-query. For multi-aspect, " 29 "synthesis, or multi-part questions, provide one " "sub-query per distinct topic or aspect so that all " "relevant evidence scattered across sessions is " "retrieved independently. Each sub-query should be " "concise and contain the key entities, specific time " "references, and the particular aspect being asked " "about. For temporal questions include relevant date or " "event words. For preference or instruction questions " "include the domain and the type of constraint. For " "existence or experience questions (e.g. âHave I ever " "done Xâ, âHave I completed Xâ, âDid I do X beforeâ, " "âHave I used Xâ), include an additional sub-query " "targeting negation or absence (e.g. ânever Xâ, ânot " "done X beforeâ, âno prior experience with Xâ, âhas not " "attempted Xâ) so that explicit denial or inexperience " "statements are retrieved with equal prominence " "alongside affirmative evidence." ) AnswerInstruction = ( "Based on the retrieved memory context and the original " "question, provide a concise and accurate answer. Each " "entry is prefixed with [Y-M-D] showing the session " "date when the user discussed that topic. For questions " "about the order or sequence in which the user raised " "topics, use these session-date prefixes to determine " "chronological order. Distinguish statement time (the " "[Y-M-D] prefix) from event time (dates within the " "entry text, e.g. project schedules). For synthesis or " "summary questions, group entries by topic and present a " "unified chronological narrative of how the userâs " "project or understanding progressed, describing how " "earlier sessions set the stage for later developments. " "For count or total questions about progress across " "sessions, sum values reported in separate distinct " "events (e.g. âcompleted 10â in one session then " "âcompleted 15â in a later session = 25 total); use the " "latest value only when entries track a single running " "total. However, if two values appear to be " "contradictory statements about the same quantity in the " "same session or context, do NOT sum them-surface the " "contradiction instead. If the context contains " "contradictory information, explicitly surface the " "contradiction and note the conflicting details. If the " "context does not contain sufficient information to " "answer, state that clearly rather than guessing. If any " "retrieved entry states a user preference or standing " "instruction (e.g., âalways include...â, âI prefer...â, " "âmake sure to...â), you MUST apply that constraint to " "the structure and content of your answer-not merely " "acknowledge it. If a [Standing Preferences] section " "appears in the retrieved context, apply every " "preference listed there. CRITICAL for existence or " "experience questions (e.g. âHave I ever Xâ, âDid I do " "Xâ, âHave I completed X beforeâ): if the retrieved " "context contains BOTH affirmative evidence that the " "user did, completed, or used X AND explicit statements " "that the user never did X, has not done X, or has no " "prior X, your answer MUST lead with the contradiction. " "Present both the affirmative evidence and the negation " "statement explicitly, briefly quote or paraphrase each, " "and ask the user to clarify which is correct. Do NOT " "resolve the contradiction by picking one side, " "dismissing one as outdated, or explaining away the " "other." ) 30 @dataclass class MemoryUnit: memory_text: str = field( metadata="description": "One atomic piece of information worth remembering: a fact, event, preference, decision, or instruction. Include specific names, values, and dates." ) session_date: str = field( default="", metadata="description": "The conversation session date in Y-M-D format. This is the date of the current chat session, not any project or schedule date mentioned in the content." ) topic: str = field( default="", metadata="description": "A concise topic identifier in lowercase- hyphenated form (entity+thread) grouping related facts across sessions. Examples: âweather-app-autocompleteâ, âtriangle-classification-progressâ, âportfolio-website-sprint-1â. Use âgeneralâ if no clear thread." ) standing_preference: bool = field( default=False, metadata="description": "True ONLY for cross-cutting user preferences or standing instructions that apply broadly regardless of topic (e.g. â always use tree diagrams for probabilityâ, âprefer minimal dependenciesâ). False for topic-specific facts or one-off requests." ) @dataclass class Query: query_texts: list[str] = field( metadata="description": "A list of retrieval sub-queries. Each sub- query targets one distinct topic or aspect. For single-topic questions use one entry. For multi-aspect or synthesis questions, decompose into one sub-query per distinct topic so that all relevant evidence is retrieved. Each should contain key entities, time references, and the specific aspect being asked about." ) class Memory: def __init__(self, backend): self.backend = backend self.memories: list[str] = [] self.embeddings: list[np.ndarray] = [] self.session_dates: list[str] = [] self.topics: list[str] = [] self.standing_prefs: list[str] = [] self.topic_to_indices: dict[str, list[int]] = _STOP = frozenset( "the a an is are was were in on at to for of and or but if how what when where who which did do does have has had i my me you your we our they their it its this that with from by as be been being am will would can could should may might shall not no yes any all each every some there here so than too very just also only own same out up down over under again further then once your you my me i we they he she it its his her their our your mine yours his hers ours theirs".split() ) @staticmethod def _parse_session_date(raw_text: str) -> str: header = raw_text[:500] m = re.search(râ( 4)-( 2)-( 2)â, header) 31 if m: return f"m.group(1)-m.group(2)-m.group(3)" m = re.search(râ( 1,2)/( 1,2)/( 4)â, header) if m: return f"m.group(3)-int(m.group(1)):02d-int(m.group(2)):02d" return "" @staticmethod def _content_tokens(text: str) -> set[str]: return t for t in re.findall(r" + ", text.lower()) if t not in Memory._STOP and len(t) > 1 @staticmethod def _normalize_topic(topic: str) -> str: return re.sub(râ[Ëa-z0-9]+â, â-â, topic.lower()).strip(â-â) or " general" def _topic_tokens(self, topic: str) -> set[str]: normalized = self._normalize_topic(topic) parts = set(normalized.split(â-â)) parts.add(normalized) return t for t in parts if len(t) > 2 def index(self, units: list[MemoryUnit], raw_text: str) -> None: if not units: return parsed_date = self._parse_session_date(raw_text) texts = [u.memory_text for u in units] embs = self.backend.embed(texts) for i, u in enumerate(units): sd = getattr(u, âsession_dateâ, â) or parsed_date topic = getattr(u, âtopicâ, â) or â sp = getattr(u, âstanding_preferenceâ, False) self.memories.append(texts[i]) self.embeddings.append(embs[i]) self.session_dates.append(sd) self.topics.append(topic) norm_topic = self._normalize_topic(topic) if norm_topic not in self.topic_to_indices: self.topic_to_indices[norm_topic] = [] self.topic_to_indices[norm_topic].append(i) if sp and texts[i] not in self.standing_prefs: self.standing_prefs.append(texts[i]) def retrieve(self, query: Query) -> str: if not self.memories: return "" sub_queries = query.query_texts if not sub_queries: return "" # Build standing preferences block standing_prefs_block = "" if self.standing_prefs: prefs_lines = [f"- p" for p in self.standing_prefs] standing_prefs_block = "[Standing Preferences] " + " ".join( prefs_lines) if len(standing_prefs_block) > 400: standing_prefs_block = standing_prefs_block[:400] standing_overhead = (len(standing_prefs_block) + 4) if standing_prefs_block else 0 main_budget = OUT_CHAR_BUDGET - standing_overhead if main_budget < 800: main_budget = OUT_CHAR_BUDGET standing_overhead = 0 32 standing_prefs_block = "" n = len(sub_queries) per_query_budget = main_budget // n all_embs = np.stack(self.embeddings) seen_texts: set[str] = set() selected: list[tuple[float, str, str]] = [] # (score, session_date, text) for sq in sub_queries: query_emb = self.backend.embed([sq])[0] cos_scores = all_embs @ query_emb q_tokens = self._content_tokens(sq) kw_scores = np.zeros(len(self.memories), dtype=np.float64) if q_tokens: for i, txt in enumerate(self.memories): t_tokens = self._content_tokens(txt) kw_scores[i] = len(q_tokens & t_tokens) / len(q_tokens) # Topic boost topic_boost = np.zeros(len(self.memories), dtype=np.float64) if q_tokens: for topic, indices in self.topic_to_indices.items(): t_toks = self._topic_tokens(topic) if not t_toks: continue overlap = q_tokens & t_toks if len(overlap) >= 2: for idx in indices: topic_boost[idx] = 0.05 elif len(overlap) == 1: for tok in overlap: if len(tok) > 4: for idx in indices: topic_boost[idx] = 0.05 break scores = 0.7 * cos_scores + 0.3 * kw_scores + topic_boost order = np.argsort(-scores) used = 0 for idx in order: idx = int(idx) txt = self.memories[idx] if txt in seen_texts: continue sd = self.session_dates[idx] if idx < len(self.session_dates) else "" prefix_len = len(sd) + 3 if sd else 0 cost = len(txt) + prefix_len + 2 if used + cost > per_query_budget: continue selected.append((float(scores[idx]), sd, txt)) seen_texts.add(txt) used += cost if not selected: return standing_prefs_block if standing_prefs_block else "" # Keep relevance order; format with date prefix for temporal context parts: list[str] = [] total = 0 for score, sd, txt in selected: prefix = f"[sd] " if sd else "" 33 formatted = prefix + txt sep = " " if parts else "" if total + len(sep) + len(formatted) > main_budget: break parts.append(formatted) total += len(sep) + len(formatted) result = " ".join(parts) if standing_prefs_block: result += " " + standing_prefs_block return result[:OUT_CHAR_BUDGET] G.2 LongMemEval Final Memory Program The program evolved by PRISMEM on LongMemEval uses a flat dense index of dated atomic memories. ⢠Extraction preserves exact names, values, dates, lists, and user preferences, resolves references to explicit entity names, and records the source session date. ⢠Indexing stores each unitâs text, normalized embedding, and date in aligned lists. ⢠Planning generates a single query tailored to the questionâs entities, factual qualifiers, time references, or preference cues. ⢠Retrieval ranks all memory units by cosine similarity, keeps those scoring at least 0.25, and packs date- prefixed units in relevance order within a 16000-character context budget. ⢠The answering instruction applies remembered constraints to recommendations, uses date prefixes for tem- poral calculations, enforces factual qualifiers, aggregates distinct occurrences, and requests abstention when a specific fact is unsupported. Final memory program evolved by PRISMEM for LongMemEval from dataclasses import dataclass, field import numpy as np MemoryExtractionInstruction = ( "Extract distinct atomic facts, procedures, " "relationships, decisions, and preferences from the text " "into several memory units. Resolve pronouns and " "references to explicit entity names. Preserve exact " "names, numeric values, dates/times, and lists verbatim. " "For each unit, also set session_date to the ISO date " "(Y-M-D) of the conversation, taken from the " "timestamp in the conversation header." ) QueryPlanningInstruction = ( "First, classify the question into one of these types: " "(A) preference/suggestion/advice/recommendation, (B) " "temporal/ordering/sequence, (C) factual retrieval with " "specific qualifiers, (D) multi-session synthesis or " "aggregate/count, (E) verification or potential " "abstention. Then write query_text tailored to that " "type. (A) For preference/suggestion/advice questions, " "do NOT write only a surface-topic query (e.g., " "âactivities to do during commuteâ). Instead, construct " "a query that explicitly targets the userâs stated " "preferences, interests, exclusions, constraints, and " "relevant personal history in the domain. Combine domain " "keywords from the question (e.g., âcommuteâ, âpodcastâ, " "âlisteningâ, âhotelâ) with preference-signal terms " "(e.g., âuser prefersâ, âinterested inâ, âwants to " "exploreâ, âbranch outâ, âdoes not wantâ, âexclusionâ, " "âgenreâ, âspecifically likesâ, âpersonal historyâ) so " 34 "that memory units recording what the user likes, " "dislikes, wants to try next, and their background in " "that domain rank highest. (B) For temporal/ordering " "questions, include the specific entities and any time " "references so dated events are retrieved. (C) For " "factual retrieval, include exact entity names and ALL " "qualifying criteria from the question. (D) For " "multi-session synthesis or aggregate questions, name " "each distinct component or entity mentioned so evidence " "across sessions is retrieved. (E) For verification or " "abstention questions, include the specific role, title, " "or condition named in the question to confirm whether " "it exists. Keep the query concise but information-dense." ) AnswerInstruction = ( "First, identify the question type and follow the " "corresponding strategy. Then answer using ONLY the " "retrieved memory text. === RECOMMENDATION / " "SUGGESTION / ADVICE / OPINION QUESTIONS (HIGHEST " "PRIORITY) === the question asks for a " "recommendation, suggestion, advice, or opinion (e.g., " "ârecommend resourcesâ, âwhat should I tryâ, âshould I " "attend Xâ, âwhat would be good for meâ, âdo you think " "it would be a good ideaâ, âwhat should I doâ, âany tips " "or ideasâ): - The userâs stated preferences, " "interests, skill level, software/tools they use, " "exclusions, and personal history found in the retrieved " "memory ARE the answer. You MUST generate a concrete " "recommendation or suggestion explicitly shaped by those " "preferences. - It is NEVER correct to answer " "âinformation not availableâ, âno specific resources " "foundâ, or âthe specific information is not availableâ " "when the retrieved memory contains ANY domain " "preference, interest, tool, or relevant personal " "context for the user. The preference itself IS the " "useful information. Do not treat the userâs preferences " "as âsimilar but different factsâ to be rejected. - " "Example: If asked to recommend video editing learning " "resources and the memory says the user enjoys Adobe " "Premiere Pro and wants to learn advanced settings, the " "answer must recommend Adobe Premiere Pro advanced " "tutorials or resources, NOT say âno specific resources " "are available.â - Enumerate every explicitly stated " "preference, constraint, exclusion, and relevant " "personal fact from the retrieved memory. A preference " "stored without an explicit location, date, or occasion " "scope is a GENERAL preference that applies to any " "suggestion in the same domain. Apply all as binding " "constraints. Generate suggestions that respect all " "listed constraints, explicitly note exclusions, and " "reference the userâs personal history as supporting " "evidence. Never ignore a stated exclusion. === " "TEMPORAL / ORDERING / DURATION / âHOW MANY DAYS OR " "WEEKSâ QUESTIONS === - The [Y-M-D] prefix on each " "memory line IS the date of the event or conversation " "that produced that memory. Treat it as the event date " "for that fact. Use these dates to compute differences. " "- When the question asks for a duration, difference, or " "ordering between two or more events, identify the " "[Y-M-D] date for each event from the retrieved " "memory, then compute the exact numerical difference in " "days (or convert to weeks if the question asks for " "weeks). Show the calculation briefly. - Example: If " "one event is dated [2023-03-17] and another is dated " "[2023-04-23], the difference is 37 days. State the " "exact number. - For ordering questions, sort all dated " 35 "events chronologically by their [Y-M-D] prefix and " "present the ordered list using exact entity names as " "written in the memory. === FACTUAL RETRIEVAL WITH " "SPECIFIC QUALIFIERS === - When the question contains " "multiple qualifying criteria, select the single memory " "fact that satisfies ALL of them. Do not pick a " "partially matching fact. - Exactness: Reproduce entity " "names, dates, times, and numeric values exactly as they " "appear in the retrieved memory. Never paraphrase or " "approximate. For example, use âModern Art Museumâ not " "âModern Art Galleryâ. - Near-miss/abstention: If the " "retrieved memory mentions a similar but different fact " "than what the question specifically asks about, do NOT " "substitute it. State that the specific information is " "not available. For instance, if asked about an " "âundergrad course research projectâ poster but the " "memory only records a âthesis researchâ poster, answer " "that the specific information is not available. This " "rule does NOT apply to recommendation or suggestion " "questions covered above. === MULTI-SESSION SYNTHESIS " "/ AGGREGATE / COUNT / SUM QUESTIONS === - Identify " "EVERY relevant numeric value or duration across all " "retrieved memory units and sum or count them to produce " "a single final answer. - Include values that refer to " "the same entity but represent distinct occurrences " "(e.g., the same game played on two different " "difficulties counts as two separate values). - " "Example: If asked for the total weeks to complete two " "activities, and memory shows 2 weeks for one and 1.5 " "weeks for the other, the answer is 3.5 weeks, not two " "separate answers. === VERIFICATION / ABSTENTION " "QUESTIONS === - Include the specific role, title, or " "condition named in the question to confirm whether it " "exists in the retrieved memory. - If the retrieved " "memory does not contain the specific information asked " "about, state clearly that it is not available. Do not " "substitute a similar fact. === GENERAL RULES (apply " "to all types) === - Keep the answer concise. Do not " "add information not present in the retrieved memory. - " "Use exact entity names, dates, and values as written in " "the retrieved memory." ) @dataclass class MemoryUnit: memory_text: str = field( metadata="description": "One piece of information worth remembering from the text" ) session_date: str = field( default="", metadata="description": "The ISO date (Y-M-D) of the session from which this memory was extracted, taken from the conversation timestamp" ) @dataclass class Query: query_text: str = field( metadata="description": "A query for retrieving information relevant to the question" ) 36 class Memory: def __init__(self, backend): self.backend = backend self.memories: list[str] = [] self.embeddings: list[np.ndarray] = [] self.session_dates: list[str] = [] def index(self, units: list[MemoryUnit], raw_text: str) -> None: texts = [unit.memory_text for unit in units] dates = [unit.session_date for unit in units] if texts: vecs = self.backend.embed(texts) self.memories.extend(texts) self.session_dates.extend(dates) self.embeddings.extend(list(vecs)) def retrieve(self, query: Query) -> str: if not self.memories: return "" q_vec = self.backend.embed([query.query_text])[0] all_vecs = np.stack(self.embeddings) scores = all_vecs @ q_vec sorted_indices = np.argsort(-scores) selected: list[str] = [] total_chars = 0 MIN_SIMILARITY = 0.25 for idx in sorted_indices: if scores[idx] < MIN_SIMILARITY: break text = self.memories[idx] date = self.session_dates[idx] tagged = f"[date] text" if date else text extra = len(tagged) + (2 if selected else 0) if total_chars + extra > OUT_CHAR_BUDGET: continue selected.append(tagged) total_chars += extra return " ".join(selected) 37