Paper deep dive

RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents

Fanyu Zhao, Ruike Cao, Liang Dong, Fugen Yao, Jian Xu, Guanjun Jiang, Han Zhang, Yifei Zhao, Yinsheng Li

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 142

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 9/23/2026, 3:49:27 AM

Summary

The paper introduces RPMem, a lifecycle-independent parametric memory framework for LLM agents that persists and evolves across sessions. It uses a two-stage architecture: single-session compilation into model-independent latent memory via a hypernetwork, and cross-session consolidation using a recurrent gate. The consolidated memory is mapped to backbone-specific LoRA parameters, enabling memory reuse when the backbone model is replaced. RPMem outperforms text-based and parametric baselines on long-term memory benchmarks.

Entities (11)

RPMem · method · 100%LoRA · technique · 95%PERMA · benchmark · 95%Qwen3-8b · model · 95%Alibaba · organization · 90%Fudan University · organization · 90%HyperNetwork · component · 90%Metis-9B · model · 90%PersonaMem-v2 · benchmark · 90%PrefEval · benchmark · 90%Recurrent Gate · component · 90%

Relation Signals (7)

RPMem → achievesbestscoreon → PERMA

confidence 95% · With Qwen3-8B on PERMA, RPMem reaches 85.52%, outperforming... baselines

RPMem → outperforms → Full Context

confidence 95% · outperforming the strongest... text-based baselines by 12.98 percentage points

RPMem → outperforms → Metis-9B

confidence 95% · RPMem reaches 85.52%, outperforming the strongest parametric... baselines by 5.32... percentage points

RPMem → uses → LoRA

confidence 95% · The consolidated memory is then mapped to backbone-specific low-rank adaptation (LoRA) parameters

RPMem → developedby → Fudan University

confidence 90% · Authors affiliated with Fudan University

RPMem → developedby → Alibaba

confidence 90% · Authors affiliated with Qwen Business Unit of Alibaba

RPMem → enables → lifecycle_independence

confidence 90% · These results establish RPMem as a lifecycle-independent parametric memory framework

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Long-running LLM agents require memory that persists and evolves across sessions. Text-based memory retrieves and reconstructs past interactions at every query, making long-horizon performance increasingly dependent on retrieval quality and contextual reasoning as histories grow. Parametric memory encodes experience directly into model computation, but existing approaches provide limited support for cross-session memory evolution. Their coupling to a specific backbone further restricts memory reuse after model replacement. We introduce RPMem, a two-stage architecture that compiles each session into a model-independent latent memory through forward computation and selectively integrates it with retained memory via a task-trained recurrent gate. The consolidated memory is then mapped to backbone-specific low-rank adaptation (LoRA) parameters, allowing the encoding capability to transfer when the backbone is replaced. Evaluation across three long-term memory benchmarks and five diverse backbones demonstrates broad generalization with near-constant update cost and memory footprint. With Qwen3-8B on PERMA, RPMem reaches 85.52%, outperforming the strongest parametric and text-based baselines by 5.32 and 12.98 percentage points, respectively. Ablations validate the complementary roles of session compilation and cross-session consolidation, while dynamics analyses reveal that the gate acquires task-specific memory integration strategies. These results establish RPMem as a lifecycle-independent parametric memory framework that maintains evolving cross-session memory that remains reusable across backbone replacements. Our implementation is available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

138,776 characters extracted from source content.

Expand or collapse full text

RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents Fanyu Zhao* Affiliation: Fudan University Affiliation: Qwen Business Unit of Alibaba Ruike Cao* Affiliation: Qwen Business Unit of Alibaba Liang Dong† Affiliation: Qwen Business Unit of Alibaba Fugen Yao Affiliation: Qwen Business Unit of Alibaba Jian Xu Affiliation: Qwen Business Unit of Alibaba Guanjun Jiang Affiliation: Qwen Business Unit of Alibaba Han Zhang Affiliation: Fudan University Yifei Zhao Affiliation: Fudan University Yinsheng Li† Corresponding author: fyzhao20@fudan.edu.cn Affiliation: Fudan University Abstract Long-running LLM agents require memory that persists and evolves across sessions. Text-based memory retrieves and reconstructs past interactions at every query, making long-horizon performance increasingly dependent on retrieval quality and contextual reasoning as histories grow. Parametric memory encodes experience directly into model computation, but existing approaches provide limited support for cross-session memory evolution. Their coupling to a specific backbone further restricts memory reuse after model replacement. We introduce RPMem, a two-stage architecture that compiles each session into a model-independent latent memory through forward computation and selectively integrates it with retained memory via a task-trained recurrent gate. The consolidated memory is then mapped to backbone-specific low-rank adaptation (LoRA) parameters, allowing the encoding capability to transfer when the backbone is replaced. Evaluation across three long-term memory benchmarks and five diverse backbones demonstrates broad generalization with near-constant update cost and memory footprint. With Qwen3-8B on PERMA, RPMem reaches 85.52%, outperforming the strongest parametric and text-based baselines by 5.32 and 12.98 percentage points, respectively. Ablations validate the complementary roles of session compilation and cross-session consolidation, while dynamics analyses reveal that the gate acquires task-specific memory integration strategies. These results establish RPMem as a lifecycle-independent parametric memory framework that maintains evolving cross-session memory that remains reusable across backbone replacements. Our implementation is available at code. 1 Introduction LLM agents increasingly operate across multiple sessions, requiring continuity in both interaction and task execution. Sustained user interaction depends on retaining relevant historical context [3, 50, 18]. Task execution across episodes similarly requires preserving prior decisions, constraints, and outcomes [29, 28, 51]. Long-term memory has therefore become an essential capability for agent architectures [15]. The dominant approach to long-term memory stores past interactions as external text and retrieves relevant information at query time. Recent work has refined this approach by extracting salient information and consolidating it into compact, retrievable records [7, 9]. Adaptive memory organization further links related entries and revises existing records as new information arrives [44]. Reinforcement learning has also been introduced to optimize memory operations and the selection of retrieved information for downstream tasks [45]. This text-based formulation keeps stored memories inspectable and separate from the backbone. Despite improvements in organization and management, using these memories still requires retrieving relevant records and integrating their contents into the input context at each query. When evidence is distributed across sessions, incomplete retrieval may omit information necessary for reasoning. Expanding the retrieved context can improve coverage, but does not ensure that the model effectively uses the available evidence [25]. Parametric memory offers an alternative by encoding experience directly into model computation, allowing memory to inform future reasoning without occupying the input context [4, 27, 36, 49]. However, deploying parametric memory in long-lived agents requires more than encoding a single context into parameters. Memory updates through forward computation are desirable for scalable deployment, as they avoid the overhead of repeated gradient-based optimization. Methods such as Doc-to-LoRA and SHINE achieve this by mapping bounded text to low-rank adaptation (LoRA) parameters [14] in a single pass [4, 27], but each session produces a static snapshot without a mechanism for incremental cross-session updates. Selectively retaining, revising, and integrating information across sessions raises a more challenging requirement. MEMORYLLM and M+ maintain updatable memory within the language model [39, 40], while Metis learns forward-updated memory matrices within a memory-augmented backbone [49]. Yet in existing parametric approaches, memory representations are defined by and coupled to a specific backbone architecture. In deployed systems, the serving model is routinely replaced, and under this coupling accumulated memory cannot be directly reused. Existing adapter-transfer methods support reuse of learned task adaptations across model changes [38, 10, 19]. Persistent agent memory additionally requires continuity of cross-session updates through those changes. This lifecycle dependence is, we argue, the central unsolved barrier to persistent parametric memory. We define lifecycle independence as preserving and continuing to use accumulated memory across serving-backbone replacements. We introduce RPMem (Recurrent Parametric Memory), a two-stage recurrent architecture that addresses these requirements jointly, as illustrated in Figure 1. During memory compilation, a shared hypernetwork [11] encodes each session into a model-independent latent memory through forward computation. During memory consolidation, a lightweight recurrent gate trained with downstream-task supervision selectively integrates each incoming session with the accumulated memory while keeping its size fixed. A model-specific decoder learned during compilation converts the consolidated memory into LoRA parameters for the serving backbone, enabling reuse of the same memory across backbones. Figure 1: Overview of the RPMem architecture. (a) Single-session compilation trains the Perceiver-based resampler and LoRA decoder to match context-conditioned responses, with the context encoder and backbone frozen. (b) Cross-session consolidation trains only the recurrent gate to integrate session memories under task supervision. (c) At inference, accumulated memory is updated recurrently and decoded into backbone-specific LoRA parameters. Adapted decoders enable reuse of the same memory across backbones; all modules are frozen. Our contributions are summarized below: 1. We formulate the problem of lifecycle-independent parametric memory and identify the requirements it imposes on memory writing, cross-session evolution, and backbone decoupling. 2. We introduce RPMem, to our knowledge the first lifecycle-independent parametric memory architecture, which maintains evolving cross-session memory in a model-independent latent space that is decoded into LoRA parameters for the current frozen backbone. 3. Experiments across three benchmarks and five backbones, together with ablation and memory-dynamics analyses, validate the two-stage design and demonstrate that parametric memory can be decoupled from the backbone lifecycle. 2 Method 2.1 Overview We consider an agent that receives an ordered stream of bounded interaction sessions 1:T=(c1,…,cT)C_1:T=(c_1,…,c_T) and later answers a query x. Here, T is the number of observed sessions, t∈1,…,Tt∈\1,…,T\ indexes their arrival order, and ctc_t denotes the t-th session. A frozen autoregressive language model fθf_θ, whose backbone parameters are denoted by θ, generates a response y according to p(y∣x,1:T)=∏j=1Np(yj∣x,y<j,1:T),p(y x,C_1:T)= _j=1^Np(y_j x,y_<j,C_1:T), (1) where N=|y|N=|y| is the response length, j∈1,…,Nj∈\1,…,N\ is the generation position, yjy_j is the token generated at position j, and y<jy_<j is its preceding response prefix. Our goal is a fixed-size persistent memory that carries information from 1:TC_1:T into generation and incorporates each new session without reprocessing earlier ones. RPMem organizes memory processing into session encoding, recurrent consolidation, and model-specific decoding. Single-Session Memory Compilation (Figure 1(a)) learns to encode each session into fixed-shape latent memory and decode it into LoRA parameters that preserve the session’s influence on the frozen backbone’s responses. Cross-Session Memory Consolidation (Figure 1(b)) then uses downstream supervision to learn how to selectively integrate successive session memories through a recurrent gate. The model-specific decoder maps the consolidated memory to LoRA parameters for response generation. 2.2 Single-Session Memory Compilation The memory compiler is a hypernetwork comprising a session encoder and a model-specific LoRA decoder. The session encoder combines a frozen context encoder with a trainable Perceiver-based resampler [16], which compresses variable-length context features into fixed-shape session memory. For a tokenized session ctc_t, these operations are Xt X_t =Eξ(ct)∈ℝL×nt×de, =E_ξ(c_t) ^L× n_t× d_e, (2) qt q_t =Gϕ(Xt)∈ℝL×M×r×d. =G_φ(X_t) ^L× M× r× d. (3) Here, the frozen context encoder EξE_ξ, with parameters ξ and hidden width ded_e, extracts features for the ntn_t tokens of ctc_t from L evenly spaced depths, one per adapted layer of the compilation backbone. The axes of XtX_t index encoder depth, token position, and feature. The resampler GϕG_φ, with trainable parameters ϕφ, compresses each layer’s token features independently through shared cross-attention blocks. In its output qtq_t, M counts the types of linear modules receiving LoRA updates within each adapted layer. The symbols r and d denote the LoRA rank and latent feature dimension, respectively. Each slice qt,ℓ,m,k∈ℝdq_t, ,m,k ^d represents rank component k∈1,…,rk∈\1,…,r\ for module type m∈1,…,Mm∈\1,…,M\ at layer ℓ∈1,…,L ∈\1,…,L\. The overall shape of qtq_t is therefore independent of the session length ntn_t. A decoder maps qtq_t to a model-specific adapter collection and applies the resulting LoRA updates to the corresponding frozen backbone weights: Dβ(qt) D_β(q_t) =Λt=At,ℓ,m,Bt,ℓ,mℓ,m, = _t=\A_t, ,m,B_t, ,m\_ ,m, (4) W~t,ℓ,m W_t, ,m =Wℓ,m+γBt,ℓ,mAt,ℓ,m. =W_ ,m+γ B_t, ,mA_t, ,m. (5) Here, DβD_β denotes the decoder with trainable parameters β, and Λt _t collects the generated factor pairs across layers ℓ and module types m. At,ℓ,m∈ℝr×dℓ,minA_t, ,m ^r× d^in_ ,m is the down-projection factor and Bt,ℓ,m∈ℝdℓ,mout×rB_t, ,m ^d^out_ ,m× r is the up-projection factor generated from session ctc_t. The symbols dℓ,mind^in_ ,m and dℓ,moutd^out_ ,m denote the input and output widths of the original linear map Wℓ,m∈ℝdℓ,mout×dℓ,minW_ ,m ^d^out_ ,m× d^in_ ,m. The fixed coefficient γ scales the LoRA update, and W~t,ℓ,m W_t, ,m denotes the resulting session-adapted map. We write θ⊕Λtθ _t for the frozen backbone augmented with the generated adapter collection. We train the memory compiler to encode session memories in latent space and make them available through generated LoRA parameters. For each session, we construct query–response pairs. Given the same query and reference response prefix, we align the next-token distribution of the frozen backbone equipped with the generated LoRA with that of the same backbone reading the session context without an adapter: pi,jctx(v) p^ctx_i,j(v) =pθ(v∣c,xi,yi,<j), =p_θ(v c,x_i,y_i,<j), (6) pi,jmem(v) p^mem_i,j(v) =pθ⊕Λ⁡(c)(v∣xi,yi,<j). =p_θ (c)(v x_i,y_i,<j). (7) Here, each session c has Q query–response pairs (xi,yi)i=1Q\(x_i,y_i)\_i=1^Q, indexed by i, with a query xix_i and a fixed reference response yi=(yi,1,…,yi,Ni)y_i=(y_i,1,…,y_i,N_i) of length NiN_i. At response position j∈1,…,Nij∈\1,…,N_i\, yi,jy_i,j denotes the token and yi,<j=(yi,1,…,yi,j−1)y_i,<j=(y_i,1,…,y_i,j-1) its preceding prefix. The collection Λ⁡(c)=Dβ(Gϕ(Eξ(c))) (c)=D_β(G_φ(E_ξ(c))) is the adapter generated from session c. The symbols pi,jctxp^ctx_i,j and pi,jmemp^mem_i,j denote the context-conditioned and adapter-conditioned next-token distributions, respectively, while v indexes a candidate token in their shared output vocabulary. Construction and split details are provided in Appendix B. We retain the K highest-probability tokens under the context-conditioned distribution. Both distributions use this same token set and aggregate the remaining probability mass into a single tail category. The selected set, tail mass, and token-level forward-KL loss are Si,j=TopKv⁡(pi,jctx(v),K),τi,ja=1−∑v∈Si,jpi,ja(v), S_i,j=TopK_v\! (p^ctx_i,j(v),K ), τ^a_i,j=1- _v∈ S_i,jp^a_i,j(v), (8) ℓi,j=∑v∈Si,jpi,jctx(v)log⁡pi,jctx(v)pi,jmem(v)+τi,jctxlog⁡τi,jctxτi,jmem. _i,j= _v∈ S_i,jp^ctx_i,j(v) p^ctx_i,j(v)p^mem_i,j(v)+τ^ctx_i,j τ^ctx_i,jτ^mem_i,j. (9) Here, TopKv⁡(⋅,K)TopK_v(·,K) returns the set of K token indices with the highest probabilities, so |Si,j|=K|S_i,j|=K. The superscript a∈ctx,mema∈\ctx,mem\ identifies one of the two distributions, τi,jaτ^a_i,j is its probability mass outside Si,jS_i,j, and ℓi,j _i,j is the forward-KL loss at position j of reference response yiy_i. The compiler averages ℓi,j _i,j over positions within each response and then over query–response pairs to obtain the session loss ℒFKL(c)L_FKL(c) and the regularized compilation objective ℒcompL_comp: ℒFKL(c) _FKL(c) =1Q∑i=1Q1Ni∑j=1Niℓi,j, = 1Q _i=1^Q 1N_i _j=1^N_i _i,j, (10) ℒcomp _comp =c[ℒFKL(c)+λℛ1(c)]. =E_c\! [L_FKL(c)+ _1(c) ]. (11) Here, cE_c averages over sessions in the compilation corpus, ℛ1(c)R_1(c) averages the absolute magnitudes of all LoRA factors decoded from c, and λ is the regularization weight. Optimization updates ϕφ and β while holding ξ and θ fixed. The separation between GϕG_φ and DβD_β supports adaptation to a target backbone fθ′f_θ , whose frozen parameters are denoted by θ′θ . We retain GϕG_φ, instantiate a target-specific decoder Dβ′D_β with parameters β′β , and optimize β′β with the same fixed-reference forward-KL objective while the memory encoder and target backbone remain fixed. The target decoder accepts the unchanged memory representation and generates LoRA factors matching the target backbone’s layer count and module dimensions. 2.3 Cross-Session Memory Consolidation Compilation places every qtq_t in the same fixed-shape memory space. Let ht∈ℝL×M×r×dh_t ^L× M× r× d denote the accumulated memory after the first t sessions. We set h0=0h_0=0 before any session arrives and initialize the memory directly from the first session as h1=q1h_1=q_1. For each subsequent session, a lightweight recurrent consolidation gate combines the accumulated memory with the incoming representation through a retention tensor: zt=σ⁡([ht−1;qt]Wg+bg),t≥2.z_t=σ\! ([h_t-1;q_t]W_g+b_g ), t≥ 2. (12) The tensor zt∈ℝL×M×r×dz_t ^L× M× r× d contains a d-dimensional retention vector at every layer, module, and rank index. The operator [;][\,;\,] denotes concatenation, Wg∈ℝ2d×dW_g ^2d× d is the shared gate projection, bg∈ℝdb_g ^d is its bias, and σ is the element-wise sigmoid. Concatenation and the shared linear projection operate along the latent feature dimension. The gate updates the persistent state as ht=zt⊙ht−1+(1−zt)⊙qt,t≥2,h_t=z_t h_t-1+(1-z_t) q_t, t≥ 2, (13) where ⊙ denotes element-wise multiplication and 11 is an all-ones tensor with the same shape as ztz_t. Thus, ztz_t weights the previous state and 1−zt1-z_t weights the incoming session representation. The same gate is shared across every layer, module, and rank index, yielding 2d2+d2d^2+d parameters independent of L, M, and r. Compilation learns to encode individual sessions as parametric memory through distribution matching. We separate consolidation learning from memory compilation to tailor memory retention and updating to the information needs and interaction patterns of the target application scenario. The second stage trains the gate with supervision from that scenario, and the learned policy is reused across subsequent interactions after deployment. For a training example (1:T,x,y)(C_1:T,x,y), the compiler produces the ordered representations q1,…,qTq_1,…,q_T, the recurrence forms the final persistent state hTh_T, and the decoder produces the corresponding adapter collection ΛT=Dβ(hT) _T=D_β(h_T). We collect the trainable gate parameters as ψ=Wg,bgψ=\W_g,b_g\ and optimize the expected loss over the target application’s training distribution: ℒcon(ψ)=(1:T,x,y)∼task[ℓtask(pθ⊕ΛT(⋅∣x),y)],L_con(ψ)=E_(C_1:T,x,y) _task [ _task\! (p_θ _T(\,· x),y ) ], (14) where taskD_task is the training-data distribution of the target application, pθ⊕ΛT(⋅∣x)p_θ _T(\,· x) is the memory-conditioned response distribution, and ℓtask _task is a differentiable prediction loss that evaluates this prediction against the target y. Gradients propagate through the frozen backbone and decoder to update ψ, while θ, ξ, ϕφ, and β remain fixed. The frozen compiler also allows the qtq_t representations to be precomputed during training. At deployment (Figure 1(c)), all modules are frozen. The first session initializes memory, and subsequent sessions update it through encoding and recurrent consolidation. Queries access the retained memory through the current backbone’s decoder. Backbone replacement preserves the accumulated memory, session encoder, and learned gate, switching only to the new backbone and its adapted decoder. Subsequent sessions use the same memory space and consolidation policy. 3 Experiments We evaluate three aspects of RPMem: cross-session memory performance and generalization, the contributions of its two training stages, and the lifecycle properties required for long-term deployment. 3.1 Experimental Setup Benchmarks and metrics. PERMA [26] evaluates memory formation, revision, and intervention across ordered sessions; we report its four clean and noisy single-domain (SD) and multi-domain (MD) settings and their macro-average. PersonaMem-v2 [18] derives independent queries from long user histories; we report Overall, Self, and Current accuracy. PrefEval [50] measures memory after 10, 70, and 300 intervening turns, averaged across its three information forms. Detailed task construction and metric definitions appear in Appendix D.1. Baselines. Textual-memory methods include Full Context, RAG [21] with M3-Embedding [5], Rolling Summary, Mem0 [7], and LightMem [9]. Parametric baselines include SFT, trained on the same downstream split with the visible history, and the released Metis-9B recurrent-memory model [49]. No Context and Gold State provide query-only and annotated-memory references. Implementations and method-specific hyperparameters are documented in Appendix D.2. Protocol. In the main comparison, RPMem and our baseline implementations use Qwen3-8B [46] with thinking disabled. We directly evaluate the released Metis-9B model. Generalization and transfer span five backbones across model families, scales, and architectures. PERMA uses ten-fold leave-one-user-out evaluation, while PersonaMem-v2 and PrefEval follow their official splits. For these benchmarks, we train the consolidation gate on each training split with the compiler from subsection 2.2 and answer-label cross-entropy. Full training and inference protocols appear in Appendix D.3. 3.2 Main Results Table 1compares all methods. Bold and underlined values mark the best and second-best non-oracle results, excluding Gold State. Table 1: Main comparison on cross-session memory benchmarks (accuracy, %). Type Method PERMA PersonaMem-v2 PrefEval SD-C SD-N MD-C MD-N Core Overall Self Current 10 70 300 Reference No Context 66.73 66.73 54.60 54.60 60.66 25.74 25.28 25.42 36.67 39.82 39.07 Gold State 72.70 72.23 66.06 65.47 69.12 57.90 64.05 69.16 80.18 80.74 80.00 Textual Rolling Summary 39.57 40.27 23.34 27.41 32.65 26.58 25.46 25.32 44.26 45.19 45.19 LightMem 56.63 56.20 40.72 40.20 48.44 30.46 31.42 34.51 81.30 79.07 73.52 Mem0 57.57 58.57 41.03 42.10 49.82 30.02 30.26 31.90 73.15 65.19 61.67 RAG (BGE-M3) 63.00 62.53 49.16 47.46 55.54 30.32 31.13 31.70 59.26 54.63 50.74 Full Context 76.47 73.03 70.72 69.95 72.54 31.10 32.72 33.97 60.00 45.00 40.56 Parametric SFT 78.00 71.47 49.30 42.61 60.35 31.64 33.65 35.47 57.22 40.00 38.15 Metis-9B 80.87 80.47 80.17 79.29 80.20 29.64 30.24 31.32 43.89 39.07 39.82 RPMem (ours) 90.57 91.20 79.67 80.62 85.52 40.22 43.61 47.28 69.81 72.41 74.81 Across PERMA and PersonaMem-v2, RPMem obtains the strongest non-oracle aggregate performance under two independently constructed cross-session evaluations. On PERMA, it reaches 85.52%, exceeding Metis-9B and Full Context by 5.32 and 12.98 percentage points (p), respectively, and leads three of the four core settings. The advantage over Metis also holds with Qwen3.5-9B for both methods (Appendix G.1). RPMem also surpasses SFT by 25.17 p under the same downstream split and visible-history boundary, extending its advantage to a parametric baseline trained on in-domain histories. RPMem also achieves the highest average across all seven historical-context variants at 86.53% (Appendix D.6). On PersonaMem-v2, it achieves 40.22% Overall accuracy, surpassing SFT by 8.58 p and Full Context by 10.89 and 13.31 p on Self and Current. The consistent advantage across benchmark constructions shows that the learned memory remains effective while tracking currently valid user information over long histories. PrefEval further reveals how the methods respond as the intervening history grows. RPMem ranks third at 10 turns and second at 70 turns, then achieves the best 300-turn result at 74.81%, surpassing LightMem by 1.29 p. Over the same range, LightMem declines from 81.30% to 73.52%, whereas RPMem rises from 69.81% to 74.81%. Its strongest relative performance at the longest interval shows that consolidated memory remains effective when the preference-bearing interaction is separated from the query by a long history. Figure 2: PERMA generalization across five backbones. Left: seven-variant average; right: SD and MD averages (accuracy, %). Performance across language models. We instantiate RPMem on five backbones and compare it with matched No Context and Full Context references. Figure 2 shows consistent gains across model families, parameter scales, and dense and MoE architectures, with overall improvements of 5.95–20.41 p over the stronger reference. The complete values underlying the figure are reported in Appendix D.7. These results show that the memory mechanism generalizes across heterogeneous language-model backbones. 3.3 Ablation Studies RPMem learns memory at two levels: the compiler encodes each session, and the recurrent consolidation module integrates the resulting memories over time. We isolate these components by varying the compiler objective under a fixed consolidation protocol and then varying the consolidation rule with the selected compiler. Table 2 reports both ablations with SD and MD macro-averages; complete definitions and setting-level results appear in Appendix C. Table 2: Ablations on PERMA (accuracy, %): (a) compilation objectives and (b) consolidation rules. Core SD and Core MD average the clean and noisy settings. FKL/RKL denote forward/reverse KL; bold and underline mark the best and second-best results in each panel. Variant Core SD Core MD Core Avg. (a) Memory compilation objective Compiler SFT 72.20 50.23 61.21 Sampled RKL (top-K) 74.08 59.73 66.91 Sampled RKL (K3) 70.82 63.22 67.02 Top-K CE 82.99 72.34 77.66 Sampled FKL 88.37 79.14 83.75 Fixed FKL 90.89 80.15 85.52 (b) Cross-session consolidation Rank concatenation 14.97 12.71 13.84 Latent averaging 62.40 28.64 45.52 Latest Session 67.72 39.40 53.56 Factor averaging 69.44 38.99 54.21 Learned gate (ours) 90.89 80.15 85.52 Compilation objective. Fixed FKL reaches 85.52%, improving over Top-K CE by 7.86 p and Compiler SFT by 24.31 p. The 16.45 p gain from hard labels to selected-token probabilities shows that the relative likelihoods of plausible continuations carry information induced by the session. Retaining the probability mass outside the selected support contributes a further 7.86 p, showing that faithful compilation also depends on distributional coverage. Sampled FKL remains within 1.77 p of Fixed FKL, whereas reversing the KL direction reduces accuracy by more than 16 p under sampled-response training. Together, these results favor matching the full context-conditioned distribution along a fixed, grounded response trajectory. Objective definitions and checkpoint-wise training dynamics are provided in Appendix C.1 and Appendix C.2. Cross-session consolidation. The learned gate reaches 85.52%, exceeding the strongest evaluated fixed rule, LoRA factor averaging, by 31.31 p. Factor averaging improves over retaining only the latest session by 0.65 p, latent averaging performs 8.04 p worse, and rank concatenation falls to 13.84% despite retaining every decoded session adapter. The learned gate gains 21.45 p over factor averaging in SD histories and 41.16 p in MD histories, localizing its largest benefit to sequences containing heterogeneous memory. These comparisons show the benefit of task-supervised, coordinate-wise integration over the evaluated fixed rules for combining accumulated and incoming information. Further definitions and data-efficiency results appear in Appendix C.3 and Appendix C.4. Additional experiments on Clean SD and Clean MD show consistent gains over update averaging, tuned EMA, and a task-trained scalar gate (Appendix C.3). 3.4 Lifecycle Properties We first test the defining lifecycle claim that accumulated memory remains reusable after backbone replacement. We then evaluate its deployment viability through resource scaling and general-capability retention. Cross-backbone reuse. We reuse the frozen Qwen3-8B session encoder with Qwen3-4B, Ministral-3-8B [24], Qwen3.5-9B, and Qwen3.5-35B-A3B [30], adapting only the model-specific decoder during compilation for each target. Under the same target-side training budget, transfer improves all 28 backbone–setting combinations over compilers trained from scratch and raises the cross-model average from 75.97% to 89.64%, showing that one parametric memory representation can serve different backbones through target-specific decoding. Complete results and adaptation costs appear in Appendix F. Streaming deployment. We profile update latency, persistent-memory size, and historical query tokens from one to 64 sessions against Full Context, Mem0, and uncompressed adapter rank concatenation. Table 3 reports the endpoint and growth; metric definitions and the full protocol appear in Appendix D.5. Table 3: Online deployment efficiency at 64 accumulated sessions. Type Method Update Memory Hist. tokens ↓ Time (s) ↓ Growth ↓ Size (MiB) ↓ Growth ↓ Textual Full Context – – 0.138 78.09×78.09× 28,498.4 Mem0 12.554 1.10×1.10× 20.041 62.11×62.11× 441.7 Parametric Adapter rank concatenation 0.948 29.46×29.46× 183.738 5.10×5.10× 0 RPMem (ours) 0.043 1.03×1.03× 36.562 1.00×1.00× 0 At 64 sessions, RPMem writes a session in 0.043 seconds, approximately 22 times faster than rank concatenation and 292 times faster than Mem0. Across the full profile, update and memory growth remain 1.03×1.03× and 1.00×1.00×, respectively, with no historical query tokens. Avoiding history reprocessing also reduces query cost: over the 5,000-question PersonaMem-v2 test set with approximately 32K-token histories, mean language-model forward time is 0.055 seconds, compared with 4.287 seconds for Full Context. Training the shared compiler requires an estimated 338 GPU-hours, whereas consolidation for a representative PERMA fold takes 13 minutes on one A800-80GB GPU; complete costs appear in Appendix D.5. General-capability retention. Finally, we evaluate the unmodified backbone (Base LLM), Latest Session, and RPMem on MMLU [13], GSM8K [8], and IFEval [52], averaging the two memory methods over ten user-specific adapters. Complete protocols and user-level results appear in Appendix D.5. Table 4: General-capability retention (accuracy, %). Method MMLU GSM8K IFEval Base LLM 71.96 75.21 81.70 Latest Session 69.85 73.15 77.38 RPMem (ours) 70.13 73.78 77.89 RPMem remains within 1.83 p of Base LLM on MMLU, 1.43 p on GSM8K, and 3.81 p on IFEval, while improving over Latest Session on all three benchmarks. Cross-session memory therefore largely preserves the backbone’s general capabilities. 4 Analysis of Learned Consolidation Dynamics The preceding experiments demonstrate the effectiveness of RPMem’s cross-session memory. We next examine how the learned consolidation mechanism organizes memory as sessions accumulate. We study the Qwen3-8B configuration of RPMem under the PERMA ten-fold protocol. For every held-out user in Clean SD and Noisy SD, we replay the complete interaction history through the recurrent gate and trace how each session changes the memory, interacts with previously retained information, and survives subsequent updates. 4.1 Semantically Structured Memory Updates At each session, the recurrent gate assigns a coordinate-wise retention coefficient to the accumulated memory, while the remaining weight determines how much of the incoming session memory is written. We summarize the complete update from one semantic event by its write magnitude, defined as the mean incoming-memory weight after composing all compiler segments in that event. We analyze fusion steps following direct first-session initialization and compare write magnitudes between domain-emergence events, which establish a memory domain, and supplements, which add evidence to an existing domain. Emergence produces larger writes for every held-out user, and the emergence–supplement write difference is approximately five times larger than the difference between matched Clean and Noisy inputs (Figure 3(a–b)). Write magnitude therefore follows an event’s contribution to the memory timeline while remaining stable under surface perturbations. To examine domain structure, we measure the cosine similarity of centered write-weight vectors and the fraction of a historical source’s contribution retained after one update. Same-domain events have more similar write patterns (Figure 3(c)), while a new event attenuates same-domain historical contributions more strongly than cross-domain contributions (Figure 3(d)). Both paired differences hold for every held-out user (Figure 3(e)), indicating that learned consolidation combines domain-specific revision with cross-domain preservation. Figure 3: Learned consolidation dynamics on PERMA. (a–b) User-mean write magnitudes by event role and noise condition; diamonds mark across-user means, and dashed diagonals indicate equality. (c–d) Same- versus cross-domain write-pattern similarity and one-step source survival; violins show matched-pair distributions and points show user means. (e) Paired differences: Same minus Cross for similarity, Cross minus Same for survival. (f) Source survival over subsequent updates on a logarithmic scale. Statistical details, including interval definitions, appear in Appendix E. 4.2 Long-Term Survival of Memory Sources We next examine whether these differentiated updates preserve memory over longer horizons. Unfolding the recurrence defines source survival as the fraction of a session’s original contribution remaining after subsequent updates. The survival advantage of domain-emergence events grows from 1.31×1.31× after one update to 6.64×6.64× after 60 (Figure 3(f)). Functional retention is evaluated separately: across seven PERMA variants, accuracy is 86.7% immediately after the target memory event and 86.5% after subsequent intervening sessions (Appendix E). Across the observed histories, fixed-size memory exhibits differentiated writing and retention, with domain-establishing events receiving stronger writes and persisting longer than supplements. A representative held-out trajectory in Appendix I traces this organization through a complete interaction history. 5 Related Work Text-Based Agent Memory. Most agent-memory systems maintain interaction history as external textual records and retrieve selected entries for each query. Generative Agents [29] organizes experience as a memory stream, MemGPT [28] manages hierarchical storage, and Mem0 and LightMem [7, 9] extract structured records from conversations. A-MEM [44] constructs linked notes, while Infini Memory [17] consolidates observations into topic documents. Memory-R1 and AgeMem [45, 48] learn memory management through reinforcement learning, with AgeMem jointly controlling external memory and the active context. GRU-Mem [32] learns when to update textual memory and stop reading further context. At the infrastructure level, MemOS [23] unifies the organization and lifecycle management of textual, activation, and parametric memories; RPMem addresses the learned representation and integration of cross-session parametric memory. These advances improve storage and retrieval, yet cross-session knowledge must still be selected and reconstructed within a finite context on every call, making long-horizon performance jointly dependent on retrieval quality, compression fidelity, and contextual reasoning. Parametric Agent Memory. Parametric memory makes prior information available through model computation rather than context injection. PAM [36] internalizes long-horizon agent experience into LoRA adapters through fine-tuning; TMEM [31] accumulates online LoRA gradient updates from extracted experience. SELF-PARAM [41] integrates successive contexts into model weights by matching context-conditioned response distributions. WISE and ELDER [37, 22] manage sequential knowledge edits through routed side memory and mixtures of LoRA adapters, respectively. EVAF [12] selects events using surprise and valence signals, then updates a LoRA adapter with buffered replay. These approaches use gradient-based writes; RPMem learns compilation and consolidation offline and performs subsequent memory updates through forward computation. ParamMem [47] encodes cross-sample reflection patterns into model parameters. MLP Memory [42] distills a kkNN retriever’s output distribution and combines its predictions with the language model’s. Titans [1] updates a neural memory module at test time, while GradMem [20] optimizes prefix memory tokens using a context-reconstruction objective with the language model frozen. Architecture-native methods maintain mutable memory inside the language model: MEMORYLLM [39] introduces a self-updatable latent memory pool, M+ [40] extends it with scalable long-term retrieval, and Metis [49] learns forward-updated memory matrices through memory-specific mid-training. Metis represents the closest concurrent work: it satisfies both forward-only writing and cross-session evolution through gated memory dynamics, yet its memory dimensions, update projections, and read-write mechanisms are defined by the specific backbone architecture, preventing memory reuse after model replacement. These architecture-native approaches couple memory to the backbone that produced it. Context-to-Adapter Generation and Transfer. Context distillation optimizes parameters to reproduce behavior induced by a given context [33]. MAC [34] amortizes documents into a memory bank and aggregates their modulations for each query. RPMem consolidates incoming sessions into persistent, fixed-size memory before a query arrives. Doc-to-LoRA [4] amortizes document-specific distillation into a shared Perceiver hypernetwork that generates LoRA parameters in one forward pass; SHINE [27] scales this mapping to multi-layer adapters, and Profile-to-PEFT [35] generates personalized adapters from textual user profiles. These methods establish forward mappings from bounded text to model parameters, but each updated source produces a new static adapter without a mechanism for cross-session memory evolution. Generative Adapter [6] supports forward-only adaptation to streaming context through backbone-specific weight updates, and LoRA-Gen [43] generates adapters for a target model from task descriptions. Trans-LoRA, Trans-PEFT, and TiTok [38, 10, 19] transfer adapters learned from fixed task data across backbones, addressing model portability for static adapters rather than evolving memory. Memory Decoder [2] learns a retriever-derived prediction distribution that can be combined with backbones sharing a tokenizer, enabling cross-model reuse of domain memory. RPMem complements this portability with recurrent cross-session integration and model-specific decoding of accumulated memory into LoRA parameters. 6 Discussion and Conclusion The results show how cross-session parametric memory can remain useful as interaction histories and serving models evolve. Across three memory benchmarks, RPMem achieves strong performance with fixed-size memory and no historical text in the query context. Ablations establish the complementary roles of session compilation and learned consolidation, while the dynamics analysis reveals how the gate organizes accumulated memory: writes reflect semantic role, updates selectively revise related domains, and domain-establishing events persist over longer horizons. These findings link effective cross-session memory to learned integration within fixed capacity. Across five backbones, model-specific decoding enables accumulated memory to be reused after backbone replacement. Deployment measurements further show approximately constant update cost and per-user memory size as histories grow, with limited degradation in general capabilities. RPMem thus provides a lifecycle-independent memory architecture in which experience accumulates across sessions and remains available as the serving backbone evolves. 7 Limitations Application-specific consolidation aligns memory retention and updating with the target scenario’s requirements, but the learned policy’s applicability to scenarios with substantially different memory needs remains to be studied. Future work could explore cross-scenario learning that balances shared consolidation capabilities with application-specific adaptation. Acknowledgments This work was supported by Qwen Business Unit through Alibaba Research Intern Program. References [1] A. Behrouz, P. Zhong, and V. Mirrokni (2025) Titans: learning to memorize at test time. In Advances in Neural Information Processing Systems, Vol. 38, p. 113506–113543. External Links: Document Cited by: §5. [2] J. Cao, J. Wang, R. Wei, Q. Guo, K. Chen, B. Zhou, and Z. Lin (2025) Memory Decoder: a pretrained, plug-and-play memory for large language models. In Advances in Neural Information Processing Systems, Vol. 38, p. 115487–115510. External Links: Document Cited by: §5. [3] R. Cao, S. Bai, F. Yao, L. Dong, J. Xu, and L. Xiao (2026) ATPO: adaptive tree policy optimization for multi-turn medical dialogue. In International Conference on Learning Representations, Vol. 2026, p. 31272–31292. External Links: Link Cited by: §1. [4] R. Charakorn, E. Cetin, S. Uesaka, and R. T. Lange (2026) Doc-to-LoRA: learning to instantly internalize contexts. In Forty-third International Conference on Machine Learning, External Links: Link Cited by: §1, §5. [5] J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu (2024) M3-embedding: multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics: ACL 2024, p. 2318–2335. External Links: Document Cited by: §3.1. [6] T. Chen, H. Fang, P. Xia, X. Liu, B. Van Durme, L. Zettlemoyer, J. Gao, and H. Cheng (2025) Generative Adapter: contextualizing language models in parameters with A single forward pass. In International Conference on Learning Representations, Vol. 2025, p. 27450–27470. External Links: Link Cited by: §5. [7] P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav (2025) Mem0: building production-ready AI agents with scalable long-term memory. In ECAI 2025—28th European Conference on Artificial Intelligence, Frontiers in Artificial Intelligence and Applications, Vol. 413, p. 2993–3000. External Links: Document Cited by: §1, §3.1, §5. [8] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021) Training verifiers to solve math word problems. CoRR abs/2110.14168. External Links: Link Cited by: §D.5, §3.4. [9] J. Fang, X. Deng, H. Xu, Z. Jiang, Y. Tang, Z. Xu, S. Deng, Y. Yao, M. Wang, S. Qiao, H. Chen, and N. Zhang (2026) LightMem: lightweight and efficient memory-augmented generation. In International Conference on Learning Representations, Vol. 2026, p. 98706–98729. External Links: Link Cited by: §1, §3.1, §5. [10] N. Gu, P. Fu, X. Liu, K. Ma, Z. Lin, and W. Wang (2025) Adapt once, thrive with updates: transferable parameter-efficient fine-tuning on evolving base models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria, p. 14765–14783. External Links: Document Cited by: §1, §5. [11] D. Ha, A. M. Dai, and Q. V. Le (2017) HyperNetworks. In International Conference on Learning Representations, External Links: Link Cited by: §1. [12] H. Han (2026) Memory depth, not memory access: selective parametric consolidation for long-running language agents. CoRR abs/2606.26806. External Links: Link Cited by: §5. [13] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2021) Measuring massive multitask language understanding. In International Conference on Learning Representations, External Links: Link Cited by: §D.5, §3.4. [14] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: §1. [15] Y. Hu, S. Liu, Y. Yue, G. Zhang, B. Liu, F. Zhu, J. Lin, H. Guo, S. Dou, Z. Xi, S. Jin, J. Tan, Y. Yin, J. Liu, Z. Zhang, Z. Sun, Y. Zhu, H. Sun, B. Peng, Z. Cheng, X. Fan, J. Guo, X. Yu, Z. Zhou, Z. Hu, J. Huo, J. Wang, Y. Niu, Y. Wang, Z. Yin, X. Hu, Y. Liao, Q. Li, K. Wang, W. Zhou, Y. Liu, D. Cheng, Q. Zhang, T. Gui, S. Pan, Y. Zhang, P. Torr, Z. Dou, J. Wen, X. Huang, Y. Jiang, and S. Yan (2025) Memory in the age of AI agents. CoRR abs/2512.13564. External Links: Link Cited by: §1. [16] A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira (2021) Perceiver: general perception with iterative attention. In Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 139, p. 4651–4664. External Links: Link Cited by: §2.2. [17] S. Ji, B. Wu, Z. Wang, L. Xia, Q. Li, R. Wang, W. Ding, Z. Zhu, B. Li, G. Dai, and Y. Wang (2026) Infini Memory: maintainable topic documents for long-term LLM agent memory. CoRR abs/2606.10677. External Links: Link Cited by: §5. [18] B. Jiang, Y. Yuan, M. Shen, Z. Hao, Z. Xu, Z. Chen, Z. Liu, A. R. Vijjini, J. He, H. Yu, R. Poovendran, G. Wornell, L. Ungar, D. Roth, S. Chen, and C. J. Taylor (2025) PersonaMem-v2: towards personalized intelligence via learning implicit user personas and agentic memory. CoRR abs/2512.06688. External Links: Link Cited by: §1, §3.1. [19] C. Jung and J. Kim (2026) TiTok: transfer token-level knowledge via contrastive excess to transplant LoRA. In International Conference on Learning Representations, Vol. 2026, p. 123084–123113. External Links: Link Cited by: §1, §5. [20] Y. Kuratov, M. Kairov, A. Bulatov, I. Rodkin, and M. Burtsev (2026) GradMem: learning to write context into memory with test-time gradient descent. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: §5. [21] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2020) Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems, Vol. 33, p. 9459–9474. External Links: Link Cited by: §3.1. [22] J. Li, Q. Wang, Z. Wang, Y. Zhang, and Z. Mao (2025) ELDER: enhancing lifelong model editing with Mixture-of-LoRA. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 24440–24448. External Links: Document Cited by: §5. [23] Z. Li, S. Song, H. Wang, S. Niu, D. Chen, J. Yang, C. Xi, H. Lai, J. Zhao, Y. Wang, J. Ren, Z. Lin, J. Huo, T. Chen, K. Chen, K. Li, Z. Yin, Q. Yu, B. Tang, H. Yang, Z. J. Xu, and F. Xiong (2025) MemOS: an operating system for memory-augmented generation (MAG) in large language models. CoRR abs/2505.22101. External Links: Link Cited by: §5. [24] A. H. Liu, K. Khandelwal, S. Subramanian, V. Jouault, A. Rastogi, A. Sadé, A. Jeffares, A. Jiang, A. Cahill, A. Gavaudan, A. Sablayrolles, A. Héliou, A. You, A. Ehrenberg, A. Lo, A. Eliseev, A. Calvi, A. Sooriyarachchi, B. Bout, B. Rozière, B. De Monicault, C. Lanfranchi, C. Barreau, C. Courtot, D. Grattarola, D. Dabert, D. d. l. Casas, E. Chane-Sane, F. Ahmed, G. Berrada, G. Ecrepont, G. Guinet, G. Novikov, G. Kunsch, G. Lample, G. Martin, G. Gupta, J. Ludziejewski, J. Rute, J. Studnia, J. Amar, J. Delas, J. S. Roberts, K. Yadav, K. Chandu, K. Jain, L. Aitchison, L. Fainsin, L. Blier, L. Zhao, L. Martin, L. Saulnier, L. Gao, M. Buyl, M. Jennings, M. Pellat, M. Prins, M. Poirée, M. Guillaumin, M. Dinot, M. Futeral, M. Darrin, M. Augustin, M. Chiquier, M. Schimpf, N. Grinsztajn, N. Gupta, N. Raghuraman, O. Bousquet, O. Duchenne, P. Wang, P. von Platen, P. Jacob, P. Wambergue, P. Kurylowicz, P. R. Muddireddy, P. Chagniot, P. Stock, P. Agrawal, Q. Torroba, R. Sauvestre, R. Soletskyi, R. Menneer, S. Vaze, S. Barry, S. Gandhi, S. Waghjale, S. Gandhi, S. Ghosh, S. Mishra, S. Aithal, S. Antoniak, T. L. Scao, T. Cachet, T. S. Sorg, T. Lavril, T. N. Saada, T. Chabal, T. Foubert, T. Robert, T. Wang, T. Lawson, T. Bewley, T. Edwards, U. Jamil, U. Tomasini, V. Nemychnikova, V. Phung, V. Maladière, V. Richard, W. Bouaziz, W. Li, W. Marshall, X. Li, X. Yang, Y. E. Ouahidi, Y. Wang, Y. Tang, and Z. Ramzi (2026) Ministral 3. CoRR abs/2601.08584. External Links: Link Cited by: §3.4. [25] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024) Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, p. 157–173. External Links: Document Cited by: §1. [26] S. Liu, J. Zhu, L. Shu, J. Lin, Y. Chen, H. Zhang, C. Zhang, D. Xu, J. Li, B. Tang, Z. Li, F. Xiong, E. Chen, and T. Xu (2026) PERMA: benchmarking personalized memory agents via event-driven preference and realistic task environments. CoRR abs/2603.23231. External Links: Link Cited by: Appendix H, §3.1. [27] Y. Liu, X. Wang, Y. Mao, Y. Gelberg, H. Maron, and M. Zhang (2026) SHINE: a scalable in-context hypernetwork for mapping context to LoRA in a single pass. In Proceedings of the 43rd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 306. External Links: Link Cited by: §1, §5. [28] C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez (2023) MemGPT: towards LLMs as operating systems. CoRR abs/2310.08560. External Links: Link Cited by: §1, §5. [29] J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein (2023) Generative agents: interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, p. 2:1–2:22. External Links: Document Cited by: §1, §5. [30] Qwen Team (2026) Qwen3.5: towards native multimodal agents. External Links: Link Cited by: §3.4. [31] T. Ren, W. Luo, H. Yang, R. Zhu, X. Huang, Y. Wu, B. Chou, J. Ye, J. Liang, Y. Li, and Y. Peng (2026) Scaling self-evolving agents via parametric memory. CoRR abs/2606.04536. External Links: Link Cited by: §5. [32] L. Sheng, Y. Zhang, W. Ma, Y. Shi, T. Huang, X. Wang, A. Zhang, K. Shen, and T. Chua (2026) When to memorize and when to stop: gated recurrent memory for long-context reasoning. In Proceedings of the 43rd International Conference on Machine Learning, External Links: Link Cited by: §5. [33] C. Snell, D. Klein, and R. Zhong (2022) Learning by distilling context. CoRR abs/2209.15189. External Links: Link Cited by: §5. [34] J. Tack, J. Kim, E. Mitchell, J. Shin, Y. W. Teh, and J. R. Schwarz (2024) Online adaptation of language models with a memory of amortized contexts. In Advances in Neural Information Processing Systems, Vol. 37, p. 130109–130135. External Links: Document Cited by: §5. [35] Z. Tan, Z. Zhang, H. Wen, Z. Li, R. Zhang, P. Chen, F. Mo, Z. Liu, Q. Zeng, Q. Yin, and M. Jiang (2026) Instant personalized large language model adaptation via hypernetwork. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), San Diego, California, United States, p. 23557–23580. External Links: Document Cited by: §5. [36] Z. Tang, F. Wei, Z. Tang, P. Dong, X. Liu, Q. Wang, X. Chu, and B. Li (2026) Parameters as agentic memory: internalizing long-horizon memories for efficient LLM agents. In Second Workshop on Agents in the Wild: Safety, Security, and Beyond at ICML, External Links: Link Cited by: §1, §5. [37] P. Wang, Z. Li, N. Zhang, Z. Xu, Y. Yao, Y. Jiang, P. Xie, F. Huang, and H. Chen (2024) WISE: rethinking the knowledge memory for lifelong model editing of large language models. In Advances in Neural Information Processing Systems, Vol. 37, p. 53764–53797. External Links: Document Cited by: §5. [38] R. Wang, S. Ghosh, D. Cox, D. Antognini, A. Oliva, R. Feris, and L. Karlinsky (2024) Trans-LoRA: towards data-free transferable parameter efficient finetuning. In Advances in Neural Information Processing Systems, Vol. 37, p. 61217–61237. External Links: Document Cited by: §1, §5. [39] Y. Wang, Y. Gao, X. Chen, H. Jiang, S. Li, J. Yang, Q. Yin, Z. Li, X. Li, B. Yin, J. Shang, and J. McAuley (2024) MEMORYLLM: towards self-updatable large language models. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 50453–50466. External Links: Link Cited by: §1, §5. [40] Y. Wang, D. Krotov, Y. Hu, Y. Gao, W. Zhou, J. McAuley, D. Gutfreund, R. Feris, and Z. He (2025) M+: extending MemoryLLM with scalable long-term memory. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 63308–63323. External Links: Link Cited by: §1, §5. [41] Y. Wang, X. Liu, X. Chen, S. O’Brien, J. Wu, and J. McAuley (2025) Self-updatable large language models by integrating context into model parameters. In International Conference on Learning Representations, Vol. 2025, p. 16961–16979. External Links: Link Cited by: §5. [42] R. Wei, J. Cao, J. Wang, J. Kai, Q. Guo, B. Zhou, and Z. Lin (2026) MLP Memory: a retriever-pretrained memory for large language models. In International Conference on Learning Representations, Vol. 2026, p. 132772–132795. External Links: Link Cited by: §5. [43] Y. Xiao, L. Song, R. Yang, C. Cheng, Y. Ge, X. Li, and Y. Shan (2025) LoRA-Gen: specializing large language model via online LoRA generation. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 68459–68471. External Links: Link Cited by: §5. [44] W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang (2025) A-MEM: agentic memory for LLM agents. In Advances in Neural Information Processing Systems, Vol. 38, p. 17577–17604. External Links: Document Cited by: §1, §5. [45] S. Yan, X. Yang, Z. Huang, E. Nie, Z. Ding, Z. Li, X. Ma, J. Bi, K. Kersting, J. Z. Pan, H. Schütze, V. Tresp, and Y. Ma (2026) Memory-R1: enhancing large language model agents to manage and utilize memories via reinforcement learning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), San Diego, California, United States, p. 12805–12825. External Links: Document Cited by: §1, §5. [46] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025) Qwen3 technical report. CoRR abs/2505.09388. External Links: Link Cited by: §3.1. [47] T. Yao, Y. Chen, Y. Zheng, P. Li, Z. Shen, and K. Zhang (2026) ParamMem: augmenting language agents with parametric reflective memory. CoRR abs/2602.23320. External Links: Link Cited by: §5. [48] Y. Yu, L. Yao, Y. Xie, Q. Tan, J. Feng, Y. Li, and L. Wu (2026) Agentic memory: learning unified long-term and short-term memory management for large language model agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 21457–21483. External Links: Document Cited by: §5. [49] Z. Zhang, Z. Guo, Y. Sun, X. Zhang, X. Hao, Z. Lin, Y. Zhang, X. Zhao, T. Shen, B. Tang, Z. J. Xu, J. Yan, H. Wang, X. Chen, F. Xiong, Z. Li, and T. Chua (2026) Metis: memory foundation model. CoRR abs/2607.26760. External Links: Link Cited by: §1, §1, §3.1, §5. [50] S. Zhao, M. Hong, Y. Liu, D. Hazarika, and K. Lin (2025) Do LLMs recognize your preferences? evaluating personalized preference following in LLMs. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Vol. 2025, p. 15888–15931. External Links: Link Cited by: §1, §3.1. [51] W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang (2024) MemoryBank: enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38, p. 19724–19731. External Links: Document Cited by: §1. [52] J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou (2023) Instruction-following evaluation for large language models. CoRR abs/2311.07911. External Links: Link Cited by: §D.5, §3.4. Appendix Appendix A Algorithms The following algorithms specify the two training stages and the online memory interface using the notation of section 2. The context encoder EξE_ξ and answering backbone fθf_θ are frozen throughout. Compilation optimizes the Perceiver-based resampler parameters ϕφ and decoder parameters β; consolidation subsequently optimizes the gate parameters ψ=Wg,bgψ=\W_g,b_g\. The training data are constructed in Appendix B, and numerical configurations are given in Appendix D.4. We write AdamW⁡(w,g,s)AdamW(w,g;s) for an optimizer step with parameters w, gradient g, and optimizer state s, returning the updated parameters and state. The state contains the moment estimates and step count and is initialized to zero. All optimizer hyperparameters follow the configuration appendix. Session compilation. Let compD_comp denote the collection of training sessions and their fixed query–response pairs. For each pair, the reference model receives the session as text and is evaluated on the fixed response prefix. In Algorithm A.1, TopKv⁡(p,K)TopK_v(p,K) returns the indices of the K largest probabilities in distribution p. The projection ΠS(p)=((p⁡(v))v∈S, 1−∑v∈Sp⁡(v)) _S(p)=((p(v))_v∈ S,\,1- _v∈ Sp(v)) collects the probabilities on support S and their complementary tail mass. The cache ⁡(c,i,j)T(c,i,j) associates session c, pair i, and response position j with this support and projected reference distribution; its probabilities are recovered from the stored reference log-probabilities. We denote the cached projected distribution by p¯i,jctx p^ctx_i,j. The symbols ℬB, ℒcL_c, and ℒ^comp L_comp denote a mini-batch, its individual session objectives, and their mean, respectively. The number of compilation epochs is denoted by EcompE_comp; the reference cache is prepared once and reused across these epochs. Factor regularization ℛ1R_1 follows Equation D.1. Algorithm A.1 Single-session memory compilation 1: comp,Eξ,fθ,Gϕ,Dβ,K,λ,EcompD_comp,E_ξ,f_θ,G_φ,D_β,K,λ,E_comp 2: ϕ,βφ,β 3: scomp←0s_comp← 0 4: for (c,(xi,yi)i=1Q)∈comp(c,\(x_i,y_i)\_i=1^Q) _comp do 5: for i=1,…,Q;j=1,…,Nii=1,…,Q;\ j=1,…,N_i do 6: pi,jctx←pθ(⋅∣c,xi,yi,<j)p^ctx_i,j← p_θ(· c,x_i,y_i,<j) 7: Si,j←TopKv⁡(pi,jctx,K)S_i,j _v(p^ctx_i,j,K) 8: ⁡(c,i,j)←(Si,j,ΠSi,j(pi,jctx))T(c,i,j)←(S_i,j, _S_i,j(p^ctx_i,j)) 9: end for 10: end for 11: for e=1,…,Ecompe=1,…,E_comp do 12: for each mini-batch ℬ⊂compB _comp do 13: for (c,(xi,yi)i=1Q)∈ℬ(c,\(x_i,y_i)\_i=1^Q) do 14: q←Gϕ(Eξ(c))q← G_φ(E_ξ(c)) 15: Λ(c)←Dβ(q) (c)← D_β(q) 16: for i=1,…,Q;j=1,…,Nii=1,…,Q;\ j=1,…,N_i do 17: (Si,j,p¯i,jctx)←⁡(c,i,j)(S_i,j, p^ctx_i,j) (c,i,j) 18: pi,jmem←pθ⊕Λ⁡(c)(⋅∣xi,yi,<j)p^mem_i,j← p_θ (c)(· x_i,y_i,<j) 19: ℓi,j←DKL(p¯i,jctx∥ΠSi,j(pi,jmem)) _i,j← D_KL( p^ctx_i,j\,\|\, _S_i,j(p^mem_i,j)) 20: end for 21: ℒc←Q−1∑i=1QNi−1∑j=1Niℓi,j+λℛ1(c)L_c← Q^-1 _i=1^QN_i^-1 _j=1^N_i _i,j+ _1(c) 22: end for 23: ℒ^comp←|ℬ|−1∑c∈ℬℒc L_comp←|B|^-1 _c L_c 24: ((ϕ,β),scomp)←AdamW⁡((ϕ,β),∇ϕ,βℒ^comp,scomp)((φ,β),s_comp) ((φ,β), _φ,β L_comp;s_comp) 25: end for 26: end for Cross-session consolidation. Training examples are drawn from the target application’s distribution taskD_task. For each example (1:T,x,y)(C_1:T,x,y), the frozen session encoder produces ordered memories q1,…,qTq_1,…,q_T. These tensors may be cached and shared by examples with the same history segments. We denote the ordered cache for a history by (1:T)Q(C_1:T), the configured initial gate by ψ0 _0, and the number of training epochs by EconE_con. The operator ShuffleShuffle permutes training examples using the configured random seed. Algorithm A.2 resets the recurrent memory for each example, initializes it from the first session, and differentiates the task loss through the subsequent recurrence. The symbol ℓ denotes the loss of the current example. Freezing decoder and backbone parameters preserves differentiation with respect to their memory-conditioned inputs. Algorithm A.2 Application-specific cross-session memory consolidation 1: task,Eξ,Gϕ,Dβ,fθ,ℓtask,ψ0,EconD_task,E_ξ,G_φ,D_β,f_θ, _task, _0,E_con 2: ψ=Wg,bgψ=\W_g,b_g\ 3: ψ←ψ0;scon←0ψ← _0;\ s_con← 0 4: for (1:T,x,y)∈task(C_1:T,x,y) _task do 5: (1:T)←(Gϕ(Eξ(ct)))t=1TQ(C_1:T)←(G_φ(E_ξ(c_t)))_t=1^T 6: end for 7: for e=1,…,Econe=1,…,E_con do 8: for (1:T,x,y)∈Shuffle(task)(C_1:T,x,y) (D_task) do 9: (q1,…,qT)←(1:T)(q_1,…,q_T) (C_1:T) 10: h0←0;h1←q1h_0← 0;\ h_1← q_1 11: for t=2,…,Tt=2,…,T do 12: zt←σ⁡([ht−1;qt]Wg+bg)z_t←σ([h_t-1;q_t]W_g+b_g) 13: ht←zt⊙ht−1+(1−zt)⊙qth_t← z_t h_t-1+(1-z_t) q_t 14: end for 15: ΛT←Dβ(hT) _T← D_β(h_T) 16: ℓ^←ℓtask(pθ⊕ΛT(⋅∣x),y) ← _task(p_θ _T(· x),y) 17: (ψ,scon)←AdamW⁡(ψ,∇ψℓ^,scon)(ψ,s_con) (ψ, _ψ ;s_con) 18: end for 19: end for Online memory and backbone replacement. Algorithm A.3 separates arrival of a session from answering a query. All module parameters are fixed during deployment. The retained memory hTh_T also serves as the input to an adapted decoder Dβ′D_β for a target backbone fθ′f_θ . Decoder adaptation uses the compilation objective with target-backbone reference distributions, holding the memory encoder and target backbone fixed, as described in subsection 2.2. This adaptation precedes deployment of the replacement backbone; the accumulated memory, session encoder, and learned gate are retained across the switch. The target decoder maps the unchanged memory representation to the new backbone’s layer and module dimensions. The stream ℰE contains typed events Session⁡(c)Session(c), Query⁡(x)Query(x), and Replace⁡(Dβ′,fθ′)Replace(D_β ,f_θ ). The operators DecodeDecode and EmitEmit generate a response under the chosen decoding policy and return it to the caller, respectively. Queries in this algorithm follow the first memory write. Algorithm A.3 Streaming memory updates and query-time decoding 1: Eξ,Gϕ,Dβ,fθ,ψ,ℰE_ξ,G_φ,D_β,f_θ,ψ,E 2: T←0;h0←0T← 0;\ h_0← 0 3: for e∈ℰe do 4: if e=Session⁡(c)e=Session(c) then 5: q←Gϕ(Eξ(c))q← G_φ(E_ξ(c)) 6: if T=0T=0 then 7: hT+1←qh_T+1← q 8: else 9: z←σ⁡([hT;q]Wg+bg)z←σ([h_T;q]W_g+b_g) 10: hT+1←z⊙hT+(1−z)⊙qh_T+1← z h_T+(1-z) q 11: end if 12: T←T+1T← T+1 13: else if e=Query⁡(x)e=Query(x) then 14: ΛT←Dβ(hT) _T← D_β(h_T) 15: y←Decode(pθ⊕ΛT(⋅∣x))y (p_θ _T(· x)) 16: Emit⁡(y)Emit(y) 17: else if e=Replace⁡(Dβ′,fθ′)e=Replace(D_β ,f_θ ) then 18: (Dβ,fθ)←(Dβ′,fθ′)(D_β,f_θ)←(D_β ,f_θ ) ⊳ hT,ψh_T,ψ unchanged 19: end if 20: end for Appendix B Training Data Construction Compilation trains the memory compiler to encode individual sessions in latent memory, and consolidation learns a memory-integration policy for a target application scenario. The two stages use separate data constructions: query–response pairs for individual sessions, followed by ordered histories paired with task queries and targets. Their training procedures are specified in Appendix A. B.1 Single-Session Compilation Data Sources and normalization. The source mixture covers conversational interactions, task-oriented and tool-use dialogues, coding-agent trajectories, and synthetic QA. Table B.1 lists the configured sources and domain-level sampling targets. These percentages are token-budget targets used during construction. The final corpus is filtered jointly across sources. Table B.1: Compilation sources and target shares of context tokens. Domain Sources Target share Conversation UltraChat-200k 35% Task and tool use Taskmaster-2; Schema-Guided Dialogue; Toucan SFT 25% Coding SWE-Zero OpenHands trajectories 20% Synthetic QA D2L synthetic QA, selected level-1 training shards 20% Source adapters map records to an ordered event stream. Each event retains its identifier, speaker role, event type, and content; tool calls additionally retain available names, arguments, and results. This representation preserves the evidence needed to associate each pair with the corresponding interaction. For Toucan and SWE-Zero, normalization removes system messages, assistant reasoning fields, and designated reasoning-tool calls. The synthetic source is decoded with its source tokenizer before conversion to the same schema. Long records are segmented at event boundaries under the 4,096-token budgets of both the Qwen3-8B and ModernBERT tokenizers, with one event of overlap between adjacent segments. Filtering. Duplicate detection applies Unicode normalization and case folding before hashing word-token sequences. Exact duplicates share a SHA-256 hash; near-duplicate detection uses 64-bit SimHash over three-token shingles, rejecting signatures within Hamming distance three for texts containing at least 20 normalized tokens. Filtering is applied across the accepted corpus before the training/validation assignment. The source configuration excludes designated downstream benchmark sources, and coding trajectories are filtered against an explicit evaluation-repository exclusion list. Source identifiers, revisions, and record provenance are retained in the data manifests. Table B.2lists the repositories and revision prefixes specified in the source configuration. The coding exclusion list contains the 100 repositories represented by the 500 tasks in the verified split of SWE-bench-Live/SWE-bench-Live, revision a637bd46829f. This repository-level filter is applied during source normalization. The synthetic subset selects level-1 training shards and decodes their token sequences with the Mistral-7B-Instruct-v0.2 tokenizer. Table B.2: Configured compilation data sources. Revision identifiers are abbreviated to 12 characters. Repository and subset Revision HuggingFaceH4/ultrachat_200k 8049631c405a DeepPavlov/TaskMaster2 333d425d1f5 google-research-datasets/ dstc8-schema-guided-dialogue, training dialogues e852981ae349 Agent-Ark/Toucan-1.5M, SFT subset 0df3cf37f2ab nvidia/SWE-Zero-openhands-trajectories 7b3cd106d00f SakanaAI/self_gen_qa_d2l, level-1 training shards Selected shards Query–response pairs. Every accepted session has exactly ten query–response pairs. A pair contains a query and a fixed reference response; answerable pairs also cite supporting session events. Existing source-grounded pairs are preserved, and missing pairs are generated and validated against the same evidence-linking contract. For answerable questions, the generation prompt requests concise responses grounded in explicit event content, with evidence identifiers copied from the session. The preparation pipeline also supports source-derived response pairs and deterministic event-recall fallbacks to complete pair coverage. Validation checks the pair schema, nonempty reference responses, answerability flags, evidence identifiers, exact pair counts, and preservation of the underlying session content. These checks establish structural and evidence-link consistency; reference responses remain the fixed trajectories used in distribution matching. Table B.3specifies the documented reproduction configuration for query–response generation. The generator returns a compact JSON record containing the query, question type, answerability flag, reference response, and evidence-event identifiers. Answerable pairs must cite valid events; the protocol also permits unanswerable pairs with an empty evidence list. All retained pairs receive the same training treatment, without answerability-based filtering or weighting at training time. The parser removes duplicate query–response pairs and validates each accepted pair against its session. Generation continues for sessions with incomplete coverage, up to the configured round limit. Event-grounded fallbacks then complete the ten pairs, and a final per-source validation checks coverage and preservation of session content. Source-derived and fallback pairs follow the same schema as generated pairs. Prompt examples are provided in Appendix B.3. Table B.3: Documented query–response generation configuration. Setting Value Generator Locally hosted Qwen3-8B Generation policy grounded_compact_v3 Pairs per session 10 Temperature 0 Maximum generated tokens per request 768 Generation batch size 64 Maximum generation rounds 3 Additional source-derived pairs per session 0 by default Event-grounded fallback Enabled Corpus freeze and splits. The resulting frozen corpus contains 671,030 sessions, 6,710,300 pairs, and 525,898,190 backbone-tokenized context tokens. Its deterministic split assigns eight pairs from each of 664,128 sessions to training, reserves two disjoint pairs from those sessions for query-level validation, and holds out another 6,902 sessions for session-level validation. The session-level assignment uses a seeded hash of the session identifier, with a 1% holdout probability. Within training sessions, a second seeded hash orders pair identifiers to select the two query-held-out pairs. Both assignments use seed 42. Session-level validation evaluates unseen session content, whereas query-level validation evaluates disjoint pairs over training-session content. Table B.4gives the pair counts for each partition. Training and query-level validation share session content and use disjoint pair sets; session-level validation is disjoint in session content. Table B.4: Compilation data partitions. Partition Sessions Pairs per session Total pairs Training 664,128 8 5,313,024 Query-level validation 664,128 2 1,328,256 Session-level validation 6,902 10 69,020 Reference probability targets. For every fixed reference response, the frozen base language model is evaluated with the corresponding session present in its textual context. At each response position, the data pipeline stores the model’s top-K token log-probabilities; the remaining mass is reconstructed as the tail category in Equation 9. The compiler is trained from these offline probability targets using Equation 11. The adapter-conditioned model receives the same query and response prefix, with historical information supplied through the generated adapter. Only response positions contribute to distribution matching. Averaging first within each pair and then within each session gives equal session weight despite variable response lengths. Reference support size, sequence limits, and optimizer settings are specified in Appendix D.4. B.2 Cross-Session Consolidation Data Training-example construction. Each downstream example comprises the chronological history available at its query boundary, the query, candidate answers, and the correct answer label. The history is converted to bounded inputs using the compilation-stage event representation. Native session boundaries are preserved where available; long sessions are divided into chronological segments with a 4,096-token budget and one-event overlap. Oversized events are split into token chunks to retain their content. The session encoder consumes the history segments, while the query and candidate answers are supplied to the answering backbone. The correct label supplies task supervision. Frozen session encodings are cached with their order and segmentation metadata so that multiple questions can reuse the same encoded history. PERMA. For every evaluated history variant, each question is paired with its query-specific visible session prefix and official A–H candidate set. Ten leave-one-user-out folds assign nine users to gate training and the remaining user to evaluation. All three task types of the training users contribute supervision. Each example starts a fresh recurrence over its visible history, with the first segment initializing memory and subsequent segments processed in chronological order. The held-out user’s history is encoded at evaluation using the fixed compiler and trained gate. PersonaMem-v2. Construction starts from the pinned 32K-Text release and its training, validation, and benchmark partitions. Training and validation rows belonging to benchmark personas are excluded. Two known invalid training rows are removed: one has no distractor answers and one includes the correct answer among the distractors. Null distractor entries are discarded with the repair recorded. The resulting partitions contain 18,527 training examples, 2,059 validation examples, and all 5,000 benchmark questions. Each example resolves its linked chat history into a chronological message stream, which is segmented under the shared token budget. Candidate answers are deterministically shuffled using the instance identifier, and the same ordering is used by every method. Training uses the training partition; the final-epoch gate is evaluated on the benchmark partition. PrefEval. Each base example is represented in three evidence forms: an explicit statement, a choice-based interaction, and a persona-based interaction. The corresponding evidence sessions are followed by the official intervening dialogue prefix for each of the 10-, 70-, and 300-turn settings. The frozen preparation contains 1,000 base examples and 3,000 form-specific examples before expansion over these three history lengths. The topic split follows the official training implementation with seed 42: 16 topics are used for training, while transportation, technology shopping, education resources, and motor shopping are held out. One gate is trained jointly across training topics, evidence forms, and history lengths. The split contains 820 training and 180 held-out base examples, yielding 2,460 and 540 form-specific examples, respectively. Expansion over the three history lengths produces 7,380 training instances and 1,620 evaluation instances; each history-length setting evaluates the same 540 held-out form-specific examples. Base examples and topics are disjoint across the training and evaluation partitions. Shared intervening sessions reuse their cached encodings. Answer options are deterministically permuted with seed 42 and the instance identifier, consistently across methods. Supervision and optimization. For these multiple-choice experiments, the task loss in Equation 14 supplies cross-entropy supervision on the correct answer label after option permutation. On PERMA, training uses the full-vocabulary next-token logits with the correct answer-label token as the target; evaluation selects the highest-logit legal option. Gradients pass through the frozen answering backbone and decoder to the recurrent gate. Across training examples, this supervision learns a shared integration policy for the application’s memory requirements. The policy is reused across subsequent interactions with its parameters fixed after deployment. Training parameters and checkpoint selection are specified in Appendix D.4; evaluation metrics and aggregation are specified in Appendix D.1. B.3 Query–Response Generation Prompts The following prompt examples illustrate grounded query–response generation from session histories. Wording and output-format notation are simplified for readability while preserving the generation requirements. Query–Response Generation: System Prompt Create concise question–answer pairs grounded in the provided agent session. Requirements 1. Use only facts explicitly present in the session history. 2. For each answerable question, provide a reference answer and cite one or more supporting events. Copy event identifiers exactly from the history; do not invent identifiers or cite the session as a whole. 3. Mark unanswerable questions accordingly and leave their supporting event list empty. 4. Keep questions and reference answers concise. Output Return one compact JSON object containing each question, its type, answerability, reference answer, and supporting event identifiers. Do not include additional prose, Markdown, memory summaries, or reasoning. Query–Response Generation: User Prompt Generate the remaining grounded question–answer pairs so that the session has exactly ten pairs in total. Do not repeat any existing questions. Existing questions, if any: [Existing questions] Session history: [Session events with their identifiers] Appendix C Ablation Details C.1 Compilation Objective Variants The objective ablation in Table C.1 varies two properties of compiler training: the response trajectory used to evaluate the token distributions and the divergence defined on those distributions. We use the notation introduced in subsection 2.2. For pair i and response position j, pi,jctxp^ctx_i,j is the next-token distribution produced when the frozen language model receives the session as textual context, and pi,jmemp^mem_i,j is the distribution produced when the same model receives the compiler-generated adapter. The fixed response token at this position is yi,jy_i,j, and Si,jS_i,j contains the K highest-probability tokens under pi,jctxp^ctx_i,j. Fixed-trajectory objectives. Compiler SFT treats each fixed reference response as a sequence of hard token targets. Its token-level loss is ℓi,jSFT=−log⁡pi,jmem(yi,j). ^SFT_i,j=- p^mem_i,j(y_i,j). (C.1) Here, ℓi,jSFT ^SFT_i,j is the negative log-likelihood of reference token yi,jy_i,j under the adapter-conditioned distribution. Top-K CE replaces the hard target with the probabilities of the selected context-conditioned tokens: ℓi,jTopK=−∑v∈Si,jpi,jctx(v)logpi,jmem(v). ^TopK_i,j=- _v∈ S_i,jp^ctx_i,j(v) p^mem_i,j(v). (C.2) The symbol v indexes a token in the shared vocabulary, and ℓi,jTopK ^TopK_i,j denotes the selected-support cross-entropy. Fixed FKL uses the same fixed response trajectory and selected support, then represents all tokens outside Si,jS_i,j by their aggregate probability mass. Its token-level loss is defined in Equation 9, and the session-balanced aggregation and adapter regularization follow Equation 10 and Equation 11. Sampled-trajectory objectives. The sampled-trajectory variants sample a response y~i=(y~i,1,…,y~i,N~i) y_i=( y_i,1,…, y_i, N_i) from the adapter-conditioned model, where N~i N_i is its generated length and y~i,<j y_i,<j is the prefix preceding position j. Both models are then evaluated along this shared sampled prefix: p~i,jctx(v)=pθ(v∣c,xi,y~i,<j). p^ctx_i,j(v)=p_θ(v c,x_i, y_i,<j). (C.3) p~i,jmem(v)=pθ⊕Λ⁡(c)(v∣xi,y~i,<j). p^mem_i,j(v)=p_θ (c)(v x_i, y_i,<j). (C.4) The tilde distinguishes distributions evaluated on the sampled trajectory from the fixed-trajectory distributions in Equation 6 and Equation 7. Sampled FKL applies the forward-KL construction in Equation 9 to p~i,jctx p^ctx_i,j and p~i,jmem p^mem_i,j. Sampled RKL selects the top-K support S~i,j S_i,j under p~i,jmem p^mem_i,j and reverses the order of the two distributions. For a∈ctx,mema∈\ctx,mem\, let τ~i,ja τ^a_i,j denote the probability mass that p~i,ja p^a_i,j assigns outside S~i,j S_i,j. Its token-level objective is ℓi,jRKL=∑v∈S~i,jp~i,jmem(v)log⁡p~i,jmem(v)p~i,jctx(v)+τ~i,jmemlog⁡τ~i,jmemτ~i,jctx. ^RKL_i,j= _v∈ S_i,j p^mem_i,j(v) p^mem_i,j(v) p^ctx_i,j(v)+ τ^mem_i,j τ^mem_i,j τ^ctx_i,j. (C.5) Here, ℓi,jRKL ^RKL_i,j is the top-K-plus-tail approximation to the reverse KL at the sampled response position. Sampled RKL (K3) estimates the same divergence from the sampled response token. We define the sampled-token log-ratio as ri,j=log⁡p~i,jctx(y~i,j)−log⁡p~i,jmem(y~i,j).r_i,j= p^ctx_i,j( y_i,j)- p^mem_i,j( y_i,j). (C.6) The corresponding K3 loss is ℓi,jK3=exp⁡(ri,j)−ri,j−1, ^K3_i,j= (r_i,j)-r_i,j-1, (C.7) where ri,jr_i,j compares the two models’ log-probabilities for sampled token y~i,j y_i,j and ℓi,jK3 ^K3_i,j is its nonnegative single-sample reverse-KL estimator. All variants average token losses within pairs and then across pairs, and all use the same adapter regularization as Equation 11. Each sampled-trajectory run begins with 10,377 SFT warm-start updates, then switches to its respective objective for the remaining 41,508 updates within the same run, reaching the common 51,885-update budget. Every trained compiler is frozen before fitting an independent consolidation gate with the same downstream architecture and optimization protocol. Table C.1: Effect of compiler training objectives on PERMA accuracy (%). Objective Clean SD Noisy SD Clean MD Noisy MD Avg. Compiler SFT 72.43 71.97 50.47 49.98 61.21 Sampled RKL (top-K) 73.53 74.63 60.49 58.97 66.91 Sampled RKL (K3) 71.03 70.60 63.42 63.02 67.02 Top-K CE 82.10 83.87 73.30 71.37 77.66 Sampled FKL 88.90 87.83 79.23 79.04 83.75 Fixed FKL 90.57 91.20 79.67 80.62 85.52 C.2 Compilation Training Dynamics We evaluate intermediate compilation checkpoints by freezing each compiler and fitting an independent consolidation gate with the same downstream protocol. Figure C.1 reports both the average trajectory over all seven PERMA settings and the corresponding trajectory within each setting. Fixed FKL establishes the highest average downstream accuracy by approximately 15,000 updates and preserves this lead through the 51,885-update budget. The same ordering appears throughout the seven individual settings, while Compiler SFT exhibits an early decrease followed by a gradual recovery. These trajectories show that the endpoint comparison in Table C.1 reflects a stable difference in compilation quality over training. Figure C.1: Compilation training dynamics on PERMA. Panel (a) reports the unweighted average over all seven settings; its shaded regions span the minimum and maximum setting-level accuracy. Panels (b)–(h) report each setting separately; their shaded regions show the standard deviation across the ten held-out-user folds. Every checkpoint is evaluated after independently fitting the same consolidation gate architecture. Daggered variants switch to their respective objectives after 10,377 SFT warm-start updates within each run. C.3 Cross-Session Consolidation Variants The consolidation ablation in Table C.2 fixes the Fixed FKL compiler and changes the operation applied to the chronologically ordered sequence of memory representations. All variants share the compiler checkpoint, input segmentation, visible-history boundary, held-out-user folds, and evaluation protocol. The compiler and backbone remain frozen; the learned rule fits the consolidation gate on the training users of each fold. Factor convention. Let S denote the number of compiler segments in the visible history and s∈1,…,Ss∈\1,…,S\ index their chronological order. For one adapted layer and module, suppressing the indices ℓ,m ,m, let As∈ℝr×dinA_s ^r× d^in and Bs∈ℝdout×rB_s ^d^out× r be the decoded factors of segment s. Here r is the generated factor width, and din,doutd^in,d^out are the target module’s input and output dimensions. The decoder also supplies a shared factor pair A0,B0A_0,B_0, called the head bias in the implementation, which contributes the history-independent update ΔW0=γB0A0 W_0=γ B_0A_0. The coefficient γ is the checkpoint’s LoRA scaling from Equation 5. With head bias enabled, this shared pair occupies one additional rank-r block. Selection and latent averaging. Latest Session retains all compiler segments belonging to the final observed semantic session and combines their decoded factors by rank concatenation. Latent averaging takes the element-wise mean of all S segment representations and decodes that single mean. Learned consolidation instead decodes the accumulated memory produced by subsection 2.3. Both latent averaging and learned consolidation produce one rank-r generated block plus the shared bias block. Rank concatenation. Concatenation stacks the down-projection factors vertically and the up-projection factors horizontally, appending the shared bias pair once: Acat A_cat =[A1⊤,…,AS⊤,A0⊤]⊤, =[A_1 ,…,A_S ,A_0 ] , (C.8) Bcat B_cat =[B1,…,BS,B0], =[B_1,…,B_S,B_0], (C.9) ΔWcat W_cat =γBcatAcat=γ∑s=1SBsAs+ΔW0. =γ B_catA_cat=γ _s=1^SB_sA_s+ W_0. (C.10) The resulting factor width is (S+1)r(S+1)r with head bias enabled; Latest Session uses the same construction over the selected session’s segments. The update is an unnormalized sum under the unchanged coefficient γ. LoRA factor averaging. This rule computes the arithmetic mean of each generated factor separately: A¯ A =1S∑s=1SAs, = 1S _s=1^SA_s, (C.11) B¯ B =1S∑s=1SBs, = 1S _s=1^SB_s, (C.12) ΔWfac W_fac =γB¯A¯+ΔW0=γS2∑s=1S∑u=1SBsAu+ΔW0. =γ B\, A+ W_0= γS^2 _s=1^S _u=1^SB_sA_u+ W_0. (C.13) Here u independently indexes the down-projection factor in the expanded product. The shared bias factors are identical across segments and remain unchanged by averaging, giving a total factor width of 2r2r. Thus this operator includes cross-segment products BsAuB_sA_u. Averaging the induced updates is a distinct operator, γS−1∑sBsAs+ΔW0γ S^-1 _sB_sA_s+ W_0; the reported row evaluates factor averaging. The variants in Table C.2 retain the same checkpoint scaling γ, applied directly to the factor product without division by the assembled factor width. Update norms follow from the specified operators, with no additional post-aggregation norm calibration. Concatenation has history-dependent factor width, whereas latent averaging, factor averaging, and learned consolidation have fixed factor width. Table C.2: Cross-session consolidation ablation with the Fixed FKL compiler on PERMA (accuracy, %). Consolidation Clean SD Noisy SD Clean MD Noisy MD Avg. Rank concatenation 14.83 15.10 12.70 12.71 13.84 Latent averaging 63.13 61.67 28.67 28.61 45.52 Latest Session 68.07 67.37 38.96 39.83 53.56 Factor averaging 69.50 69.37 38.99 38.99 54.21 Learned gate (ours) 90.57 91.20 79.67 80.62 85.52 Additional consolidation experiments. Experimental setup. We evaluate alternative consolidation rules on PERMA Clean SD and Clean MD using Qwen3-8B and the same frozen five-pass memory compiler. Each setting uses ten held-out-user folds, with nine users available for training or coefficient selection and the remaining user reserved for evaluation. All methods share the compiler, decoder, backbone, history segmentation, and evaluation questions. Histories are segmented at 4,096 tokens with one-message overlap; updates operate on these compiler segments. Latent averaging and RPMem are rerun alongside the additional controls. Accuracy is macro-averaged over the ten held-out users, and the two-setting mean gives equal weight to Clean SD and Clean MD. Aggregation rules. Latent averaging takes the arithmetic mean of the segment representations before decoding. Tuned exponential moving averaging (EMA) initializes h1=q1h_1=q_1 and applies ht=αht−1+(1−α)qth_t=α h_t-1+(1-α)q_t thereafter. For each fold and setting, we select α from 0,0.25,0.5,0.75,0.9,0.95,0.99\0,0.25,0.5,0.75,0.9,0.95,0.99\ using macro-average accuracy on the nine training-side users, with ties resolved toward the smaller coefficient. The selected coefficient is fixed for the held-out user. Update averaging decodes each segment separately and averages its induced weight update: ΔW=γS∑s=1SBsAs+ΔW0, W= γS _s=1^SB_sA_s+ W_0, where S is the number of segments, γ is the compiler’s LoRA scale, and ΔW0 W_0 is the shared decoder bias update, included once. This rule averages complete updates rather than the two factors separately. Its factor-concatenation implementation retains a rank that grows with S. Learned consolidation. The scalar gate mean-pools the previous memory and incoming representation over their layer, module, and rank axes, concatenates the resulting two 512-dimensional vectors, and applies a learned linear map and sigmoid. The resulting retention coefficient is shared across all memory coordinates; the gate has 1,025 parameters. RPMem instead uses coordinate-dependent retention. Both learned methods initialize h1=q1h_1=q_1 and apply consolidation from the second segment onward. Both are trained on the other nine users with the same answer-token cross-entropy objective for five epochs, using AdamW with learning rate 10−310^-3, weight decay 0.010.01, gradient clipping at 1.01.0, and seed 42. The compiler and backbone remain frozen, and neither gate receives the evaluation question as an input. Table C.3: Additional consolidation experiments on PERMA (accuracy, %). Each setting averages ten held-out-user folds. Mean averages the two displayed settings; it is distinct from the four-setting Core Avg. in the main results. All values are from the matched evaluation runs described here. Method Task supervision Clean SD Clean MD Mean Latent averaging None 63.13 28.67 45.90 Update averaging None 68.57 38.12 53.34 Scalar gate Answer-token training 67.57 39.21 53.39 Tuned EMA Coefficient selection 68.07 38.96 53.51 RPMem Answer-token training 90.80 79.13 84.97 Results. Update averaging and tuned EMA achieve two-setting means of 53.34% and 53.51%, respectively, compared with 45.90% for latent averaging. EMA selects α=0α=0 in all twenty setting–fold combinations, favoring the most recent segment within the evaluated coefficient family. The learned scalar gate reaches 53.39%, whereas RPMem reaches 84.97%. Under the same task supervision and training budget, coordinate-dependent retention therefore substantially outperforms a single shared retention coefficient on both settings. These comparisons support the value of fine-grained recurrent consolidation beyond uniform averaging and globally weighted updates. C.4 Amount of Consolidation Supervision We vary the number of training users from one to nine and evaluate each gate on users excluded from its training set. For every training-set size, cyclic sampling produces ten balanced train-evaluation partitions, and results are first aggregated within users. As shown in Figure C.2, one-user adaptation reaches 79.84% on Clean SD, and five-user adaptation reaches 89.98%. Accuracy remains between 89.98% and 91.47% from five to nine users. The learning curve demonstrates that the consolidation policy transfers across users with limited downstream supervision. Figure C.2: Data efficiency of consolidation adaptation on PERMA Clean SD. Curves report held-out users, and shaded regions show user-level bootstrap 95% confidence intervals. The horizontal reference marks Full Context at 76.47%. Appendix D Full Experimental Configuration and Hyperparameters D.1 Evaluation Scope and Benchmark Protocols PERMA. PERMA contains ten simulated users with temporally ordered interaction histories. Preferences are introduced, extended, or revised by events in these sessions, and each evaluation query presents eight candidate answers labeled A through H. The benchmark crosses two history structures, single-domain (SD) and multi-domain (MD), with clean and noisy history realizations to form four core settings. Style SD and Style MD rewrite the histories in user-specific linguistic styles, while Style-Long SD adds realistic long-dialogue distractors to the rewritten single-domain history. Core Avg. is the unweighted mean of the four core settings. The seven-setting average is used only in analyses that explicitly report all seven settings. Task-adapted methods use ten-fold leave-one-user-out evaluation: every fold trains on nine users and reports predictions for the remaining user, after which accuracy is averaged over the ten held-out users. PersonaMem-v2. We use the frozen 32K-Text release, comprising 18,527 training examples, 2,059 validation examples, and 5,000 independent benchmark questions. Each example associates a long user history with a multiple-choice query. The annotations identify whether the queried information concerns the focal user or another person and whether the relevant state is current, updated, or marked for forgetting. Overall is the official aggregate accuracy. Self restricts evaluation to information about the focal user, and Current restricts it to the currently valid state. These two slices accompany Overall in the compact main table because they directly expose the user-specific state maintained across the history. The complete ownership and update-status breakdown is retained in the benchmark result artifacts. PrefEval. The evaluation uses the official split of 16 training topics and four held-out topics. It covers explicit statements, implicit choice-based evidence, and implicit persona-driven evidence, each evaluated after 10, 70, and 300 turns of intervening dialogue. For each history length, the main table reports the unweighted mean accuracy over the three evidence forms. Topic-disjoint evaluation measures whether the downstream consolidation policy transfers to semantic domains absent from its training partition. D.2 Baseline Implementations Reference conditions. No Context presents the query and candidate answers without historical information. Gold State converts the state annotation supplied by a benchmark into structured text and places it before the query. The latter condition measures answer selection given direct access to the annotated state; its result also reflects the coverage and granularity of the benchmark annotation. Textual-memory methods. Full Context concatenates all visible sessions in chronological order and places the resulting history before the query. RAG divides the same visible history into retrievable units, embeds them with M3-Embedding (BGE-M3), and injects the ten units with the highest dense-retrieval scores. On PERMA, units are non-overlapping groups of two consecutive messages from the chronologically flattened history, including role labels. Retrieved units are presented in descending relevance order; embedding inputs are truncated at 8,192 tokens. Rolling Summary updates a 1,024-token summary after every session by combining the previous summary with the incoming interaction. The PERMA Mem0 run uses the open-source v2.0.12 pipeline, Qwen3-8B for memory extraction, and BGE-M3 for retrieval. LightMem applies its hierarchical semantic compression and retrieval pipeline to the same visible history. Every method receives an identical query and candidate-answer interface after constructing its textual memory input. SFT. This trained control freezes Qwen3-8B and fits a rank-8 LoRA with scale 32 on the down_proj modules. For each PERMA variant and held-out user, the adapter is trained for five epochs on the raw history sessions of the other nine users with the standard assistant-response objective. The resulting 70 adapters follow the same leave-one-user-out boundary as downstream consolidation. Training supervision consists of dialogue responses from the training histories, while evaluation supplies the held-out user’s complete visible history before each query. The same construction follows the official training partition on the other main benchmarks. Metis. We load the released Metis-9B memory checkpoint with its native Qwen3.5-9B backbone. Historical sessions are written sequentially through the model’s native recurrent memory mechanism, preserving their original order. The memory state is reset at the beginning of every independent benchmark task, and the released checkpoint is evaluated directly through the shared non-thinking answer-selection interface. Results for the released 4B, 9B, and 27B checkpoints are reported in Appendix G.2; the main table uses the strongest overall scale, Metis-9B. D.3 Shared Evaluation and Training Protocol In the main comparison, RPMem and our baseline implementations use Qwen3-8B as the answering model. We directly evaluate the released Metis-9B model. Generalization and transfer use the five backbones reported in Appendix D.7 and Appendix F. Thinking mode is disabled throughout. Within each benchmark, methods receive the same chronological history boundary, query, candidate answers, and output parser. On PERMA, prediction selects the largest next-token logit among the legal labels A through H. Accuracy is first computed for each held-out user and then macro-averaged across the ten folds. PersonaMem-v2 uses its frozen training, validation, and benchmark splits, and PrefEval uses its official topic-disjoint split. The default RPMem run losslessly segments sessions at 4,096 tokens and overlaps one complete event between adjacent segments. Its compiler is the 51,885-update checkpoint trained with the forward-KL objective in Equation 11. Each stored reference position contains the 32 highest-probability context-conditioned tokens and a tail category containing the remaining probability mass. Each accepted compilation session contributes eight training pairs and two disjoint query-level validation pairs. During downstream adaptation, the context encoder, Perceiver-based resampler, LoRA decoder, and answering model remain fixed while the consolidation gate is optimized on the training partition. Table D.1 records the default PERMA gate configuration; compiler architecture, optimization, and the remaining downstream protocols appear in Appendix D.4. Table D.1: Primary Cross-Session Memory Consolidation configuration used for PERMA. Component Setting Backbone Qwen3-8B, non-thinking mode Data split 10-fold leave-one-user-out Compiler Fixed FKL checkpoint 51,885 Maximum segment length 4,096 tokens Adjacent-segment overlap One event Trainable module Consolidation gate only Trainable parameters 524,800 Initial memory h0=0h_0=0; h1=q1h_1=q_1 Gate initialization Weight 0, bias −2.0-2.0 Optimizer AdamW Learning rate 10−310^-3 Weight decay 0.01 Gradient clipping 1.0 Training epochs 5 Batch size 1 Random seed 42 Prediction Maximum A–H next-token logit D.4 Compiler Architecture and Optimization Architecture. The primary Qwen3-8B compiler uses a frozen ModernBERT-base context encoder. Its hidden states are mapped to the 36 target-layer positions by uniformly spaced encoder indices, with repeated indices when the target has more layers than the encoder. LoRA is applied to the feed-forward down-projection (down_proj) in each target layer, giving L=36L=36 and M=1M=1. With r=8r=8 and d=512d=512, each session memory has shape 36×1×8×51236× 1× 8× 512. Table D.2 specifies the encoder and decoder. Table D.2: Primary Qwen3-8B compiler architecture. Component Configuration Frozen models Qwen3-8B; ModernBERT-base Adapted layers / module types L=36L=36; M=1M=1, down_proj Generated factor width / latent width r=8r=8; d=512d=512 Perceiver latent queries 8 Perceiver encoder / output blocks 9 / 1 cross-attention blocks Attention heads / head width 8 / 64 Perceiver feed-forward width 2,048 Additional self-attention blocks 0 Decoder pre-head 4 layer-specific residual MLP blocks Decoder MLP widths 512→2,048→512512→ 2,048→ 512 Decoder output Layer-specific linear factor projections Shared head-bias block Enabled; width 8 LoRA scaling / dropout γ=32γ=32 / 0 Each decoder residual block applies layer normalization, a layer-specific linear expansion, SiLU, a linear contraction, and a second layer normalization before adding the residual. After the four blocks, each latent vector is normalized by its Euclidean norm and projected to the concatenated input- and output-factor dimensions. Splitting this projection yields the two factors, each multiplied by a learned layer- and rank-specific scalar. These scalars are initialized to one for A and zero for B. For the shared head-bias block, entries of A0A_0 are initialized from a zero-mean Gaussian with standard deviation 0.2/rdin0.2/ rd^in, while B0B_0 is initialized to zero. This block is appended once during adapter assembly, as defined in Appendix C.3. The coefficient γ directly multiplies the factor product, using the same scaling convention across assembled factor widths. Factor regularization. For a bounded training session c, the implementation first computes the mean absolute value of each generated factor, adds the A and B terms, and averages over layers and module types: ℛ1(c)=1LM∑ℓ=1L∑m=1M(‖Aℓ,m(c)‖1rdℓ,min+‖Bℓ,m(c)‖1rdℓ,mout).R_1(c)= 1LM _ =1^L _m=1^M ( \|A_ ,m(c)\|_1rd^in_ ,m+ \|B_ ,m(c)\|_1rd^out_ ,m ). (D.1) Here, ∥⋅∥1\|·\|_1 denotes the sum of absolute entries, and Aℓ,m(c),Bℓ,m(c)A_ ,m(c),B_ ,m(c) are the generated factors for session c at layer ℓ and module type m after their learned scalar multipliers. The denominators are the respective entry counts, using the dimensions defined in subsection 2.2. Both factors receive equal weight despite different input and output widths. The penalty is evaluated before shared-bias concatenation and application of γ, and uses λ=0.01λ=0.01 in Equation 11. Compilation optimization. Training updates the Perceiver-based resampler and decoder while freezing both language models. Each of the 664,128 training sessions contributes eight pairs; response-token losses are averaged within pairs and then within sessions. The recorded global batch contains 64 sessions, obtained from eight workers, one session per worker, and eight gradient-accumulation steps. Five passes therefore give 664,128×5/64=51,885664,128× 5/64=51,885 optimizer updates. The final completed checkpoint is used for downstream evaluation. Table D.3 records the primary training configuration. Table D.3: Primary fixed-reference forward-KL compilation configuration. Component Setting Training sessions / pairs per session 664,128 / 8 Retained reference support K=32K=32, with aggregate tail mass Maximum session length 4,096 tokens Maximum query-response sequence 640 tokens Maximum reference-model sequence 4,864 tokens Global batch / accumulation steps 64 sessions / 8 Workers / sessions per worker 8 / 1 Training passes / updates 5 / 51,885 Optimizer AdamW, β1=0.9 _1=0.9, β2=0.999 _2=0.999 Configured learning rate 4×10−54× 10^-5 Warmup updates 500 Weight decay / gradient clipping 0.01 / 1.0 Factor-regularization weight λ=0.01λ=0.01 Random seed 42 Downstream optimization. The PERMA settings appear in Table D.1. For PersonaMem-v2 and PrefEval, the formal training protocol uses the shared configuration in Table D.4. Each training question produces one optimizer update. PersonaMem-v2 shuffles personas and then their training questions using the fixed seed; the chronological order of the history within each question is preserved. PrefEval trains one gate jointly across the 16 training topics, three information forms, and three primary history lengths, and evaluates on the four held-out topics. The data construction and split definitions appear in Appendix B.2. In all cases, optimization updates only the consolidation gate; the compiler and answering backbone remain frozen. The final-epoch gate is used for evaluation. Table D.4: Shared downstream training configuration for PersonaMem-v2 and PrefEval. Setting Value Trainable module Recurrent consolidation gate Initial memory h0=0h_0=0; h1=q1h_1=q_1 Gate weight / bias initialization Wg=0W_g=0; bg=−2b_g=-2 Training objective Cross-entropy over legal answer labels Optimizer / learning rate AdamW / 10−310^-3 Weight decay / gradient clipping 0.01 / 1.0 Training epochs / random seed 5 / 42 Questions per optimizer update 1 Evaluation checkpoint Final epoch D.5 Training and Query-Cost Details Streaming deployment protocol. We profile accumulated source-session counts of 1,2,4,8,16,32,1,2,4,8,16,32, and 6464 on eligible PERMA Clean-SD histories. All models and services are resident before timing. A hot RPMem update includes segmentation and encoding of the newly arrived session, recurrent memory consolidation, and LoRA materialization. For adapter rank concatenation, we concatenate the accumulated history, repartition it into chunks of at most 4,096 tokens, and recompile all chunks at each update. Unlike the session-wise segmentation used in the consolidation ablation (Appendix C.3), these chunks can span session boundaries. The Mem0 timer includes LLM-based memory extraction, embedding, and vector-index updates, while Full Context has no separate write operation. Persistent memory counts per-user data retained for answering queries and processing subsequent updates, excluding shared model parameters such as the compiler and consolidation gate. Historical query tokens count the memory tokens supplied to the answering model. Growth values are the empirical ratios between the 64-session and one-session endpoints rather than asymptotic complexity claims. General-capability retention. We evaluate the unmodified Qwen3-8B backbone (Base LLM, with no user-memory adapter) and ten user-specific memory states on MMLU [13], GSM8K [8], and IFEval [52]. Table D.5 reports the user-level mean and standard deviation for both memory methods. Table D.5: General-benchmark performance (%) of the base model and memory-adapted models. Method MMLU (5-shot) GSM8K (5-shot) IFEval (0-shot) Base LLM 71.96 75.21 81.70 Latest Session 69.85±0.5169.85± 0.51 73.15±1.6373.15± 1.63 77.38±1.2077.38± 1.20 RPMem (ours) 70.13±0.4170.13± 0.41 73.78±1.5373.78± 1.53 77.89±1.4277.89± 1.42 Query cost on PersonaMem-v2. The deployment measurements in the main text are complemented by an independent evaluation over the 5,000-question PersonaMem-v2 test set. Table D.6 reports the mean input length, language-model forward time, and parametric state associated with each method. Table D.6: Mean query cost on the PersonaMem-v2 test set. Method Input tokens Forward time (s) Parametric state Full Context 33,363.71 4.287 – RAG (BGE-M3) 2,757.58 0.253 – Latest Session 337.49 0.056 36 MiB, rank 16 Adapter rank concatenation 337.49 0.056 195.9 MiB, rank 87.1 mean RPMem 337.49 0.055 36 MiB, rank 16 The rank reported in Table D.6 denotes the assembled factor width: eight generated components plus the eight-component shared head-bias block for RPMem and Latest Session (Appendix C.3), not the numerical rank of the update matrix. With an approximately 32K-token user history, RPMem reduces the mean forward time from 4.287 seconds under Full Context to 0.055 seconds while preserving a fixed-rank 36 MiB state. Rank concatenation receives the same query tokens but expands the adapter state with the number of accumulated sessions. At the 64-session endpoint of the deployment experiment, RPMem incurs 0.575 GiB of peak incremental GPU memory, compared with 2.826 GiB for adapter rank concatenation; this measurement is reported here because peak allocation is not available for the textual-memory pipelines. Consolidation adaptation cost. For a representative Clean-SD fold with 630 training examples and 3,150 updates, the 524,800-parameter consolidation gate trains in 13.0 minutes on one NVIDIA A800-80GB and produces a 2.004 MiB checkpoint. The peak GPU-memory increase over the full training run is 12.173 GiB, measured relative to the resident-model baseline before gate training. This increase primarily reflects activations needed to backpropagate through the frozen decoder and language model rather than the gate parameters. Table D.7: Consolidation-stage adaptation cost on a representative PERMA fold. Measurement Value Trainable parameters 524,800 Training examples 630 Parameter updates 3,150 (5 epochs) Training time 13.0 min Peak incremental memory 12.173 GiB Saved checkpoint size 2.004 MiB To measure how adaptation scales with the memory sequence, we select ten Type-3 tasks containing at least 64 cached session representations, truncate each sequence to progressively longer prefixes, and repeat every measurement three times. Peak Δ reports the mean per-step peak memory increase above a warmed-up baseline with the model, gate, optimizer, and query tensors already resident. As shown in Table D.8, increasing the sequence from one to 64 representations raises total step time by 1.39×1.39× and peak incremental memory by 3.6%. Table D.8: Consolidation optimization cost as the number of cached session representations increases. Sessions Load (ms) Compute (ms) Total (ms) Peak Δ (GiB) 1 0.8 159.0 159.8 5.059 2 1.4 161.1 162.5 5.062 4 2.6 161.6 164.3 5.069 8 4.7 161.2 165.9 5.081 16 8.3 164.1 172.4 5.107 32 17.3 169.6 186.9 5.153 64 43.1 179.6 222.7 5.242 Of the additional 62.9 ms at 64 sessions, 42.3 ms is spent loading cached representations and 20.6 ms in GPU computation. The measured scaling therefore localizes the principal growth term to cache I/O, while the recurrent consolidation computation and activation memory remain comparatively stable. Compilation optimization cost. Table D.9 reports measured step times and estimated allocated GPU-hours for the objectives evaluated in Table C.1. A fixed-reference full-vocabulary reverse-KL run (Fixed RKL) is included as an additional runtime comparison. The three 8-GPU configurations have nearly identical step times and an estimated cost of 338–339 GPU-hours for 51,885 updates. Each of the four 24-GPU runs, including Fixed RKL, performs 10,377 SFT warm-start updates followed by 41,508 updates with its respective objective. The total estimate uses separately measured preflight step times for the two phases: GPU-hours=243600(10,377tSFT+41,508tobjective).GPU-hours= 243600 (10,377\,t_SFT+41,508\,t_objective ). Here, both step times are in seconds; only tobjectivet_objective is displayed for these runs in the table. Table D.9: Compilation-stage optimization cost by training objective. Objective Prefix Loss Step (s) Allocated GPUs Est. GPU-hours Top-K CE Reference Top-K CE 2.943 8 339 Fixed FKL Reference FKL 2.932 8 338 Compiler SFT Reference CE 2.940 8 339 Sampled RKL (K3) Student RKL (K3) 13.061 24 3,849 Fixed RKL Reference RKL 9.585 24 2,939 Sampled FKL Student FKL 18.202 24 5,284 Sampled RKL (top-K) Student RKL 15.944 24 4,627 The 24-GPU runs reserve 16 actor and eight teacher GPUs throughout training. Objective-stage timing includes online teacher scoring and, for sampled variants, student generation. Estimates exclude non-update overhead and separate offline reference preparation. Fixed FKL combines the highest downstream accuracy in Table C.1 with the lowest estimated allocated training cost among the evaluated distribution-matching configurations. D.6 Complete Historical-Context Shift Results Table D.10reports the complete method matrix for the three historical-context shift settings summarized in subsection 3.2. Style SD and Style MD rewrite the visible histories in user-specific linguistic styles. Style-Long SD additionally inserts realistic long-dialogue distractors into the rewritten single-domain history. Avg. is the unweighted mean of the three settings. Table D.10: Complete PERMA historical-context shift results (accuracy, %). Method Style SD Style-Long SD Style MD Avg. Full Context 76.40 75.83 71.35 74.53 Gold State 73.50 73.50 63.80 70.27 RAG (BGE-M3) 65.53 65.53 46.47 59.18 Rolling Summary 39.63 39.93 26.66 35.41 Mem0 58.73 59.47 40.35 52.85 LightMem 55.13 56.17 39.34 50.21 SFT 76.73 77.40 56.68 70.27 Metis-9B 79.67 79.97 79.87 79.84 RPMem (ours) 91.37 90.97 81.33 87.89 RPMem leads all three context-shift variants and reaches an average accuracy of 87.89%, exceeding Metis-9B and Full Context by 8.05 and 13.36 percentage points (p), respectively. Relative to the corresponding clean settings, its accuracy changes by +0.80+0.80 p on Style SD, +0.40+0.40 p on Style-Long SD, and +1.66+1.66 p on Style MD. Style-Long SD jointly varies linguistic style and history length, so its result measures stability under the combined shift. Across all seven PERMA variants, RPMem reaches 86.53%, demonstrating that the parametric memory remains reliable as the style and length of the historical context change. D.7 Complete Model-Generalization Results Table D.11reports the aggregate values underlying Figure 2. SD Avg. averages the four single-domain variants, MD Avg. averages the three multi-domain variants, and Overall Avg. averages all seven PERMA variants. Within each backbone, all methods use the same history boundary, non-thinking answer-selection protocol, and held-out-user folds. No Context results are aggregated from independent evaluations of each variant. Table D.11: Complete aggregate results underlying the cross-model generalization comparison (accuracy, %). Backbone Method SD Avg. MD Avg. Overall Avg. Qwen3-4B No Context 65.70 60.97 63.67 Full Context 79.82 73.39 77.06 RPMem (ours) 85.98 79.04 83.01 Qwen3-8B No Context 66.73 54.60 61.53 Full Context 75.43 70.67 73.39 RPMem (ours) 91.03 80.54 86.53 Ministral-3-8B No Context 69.57 74.47 71.67 Full Context 77.05 77.56 77.27 RPMem (ours) 89.48 83.00 86.70 Qwen3.5-9B No Context 29.24 35.63 31.98 Full Context 66.67 84.34 74.25 RPMem (ours) 96.21 92.59 94.66 Qwen3.5-35B-A3B No Context 76.00 84.11 79.48 Full Context 57.72 74.38 64.86 RPMem (ours) 93.65 94.96 94.21 Appendix E Memory Dynamics Analysis Details Table E.1reports the functional lifecycle comparison that motivates the update-dynamics analysis in the main text. T1 precedes the target memory event, T2 immediately follows it, and T3 follows subsequent intervening sessions. The table reports the memory-bearing T2 and T3 checkpoints averaged across all seven PERMA variants. Table E.1: Memory-bearing accuracy (%) across the PERMA lifecycle. Method In-Time (T2) Post-Intervention (T3) T3–T2 (p) Full Context 76.3 77.2 +0.9+0.9 Latent averaging 46.5 45.6 −0.9-0.9 RPMem (ours) 86.7 86.5 −0.2-0.2 E.1 Source Decomposition of the Recurrent State With the direct initialization h1=q1h_1=q_1, unrolling the recurrent update in equation 13 for t≥2t≥ 2 assigns an explicit coordinate-wise weight to every source session: hT h_T =∑i=1Twi,T⊙qi, = _i=1^Tw_i,T q_i, (E.1) w1,T w_1,T =∏k=2Tzk, = _k=2^Tz_k, (E.2) wi,T w_i,T =(1−zi)⊙∏k=i+1Tzk,2≤i≤T. =(1-z_i) _k=i+1^Tz_k, 2≤ i≤ T. (E.3) Here, i indexes the source session, k indexes a subsequent update, and wi,Tw_i,T has the same shape as qiq_i. Its entries are the coefficients of that source in the accumulated memory after step T. All products are element-wise, and an empty product is the all-ones tensor. Thus w1,1=1w_1,1=1, while wi,i=1−ziw_i,i=1-z_i for i≥2i≥ 2. The coefficients are nonnegative and sum to one at every coordinate. They describe the source decomposition along the observed gate trajectory; perturbing a session can also change later gates and therefore requires a new forward replay. E.2 Trajectory Replay and Statistical Protocol The analysis uses the Qwen3-8B compiler trained with fixed-reference forward KL. For each of the ten held-out PERMA users, trajectories are replayed with the consolidation module trained on the other nine users for the corresponding benchmark variant. Clean SD and Noisy SD each contain 423 task-conditioned trajectories and 34,218 session occurrences, giving 846 trajectories and 68,436 session occurrences in total. These counts include shared history appearing in different task-conditioned trajectories; the independent evaluation units are the ten users. Each semantic session in this replay occupies one compiler segment. For matched comparisons, we first average the paired differences within each user and then average the ten user estimates with equal weight. The 95% confidence intervals are the 2.5th and 97.5th percentiles of 10,000 bootstrap means obtained by resampling users with replacement. Matching remains fixed during resampling. These intervals summarize variation across held-out users conditional on the fitted fold models. Reading the main dynamics figure. In Figure 3(a–b), each colored point is one held-out user’s paired-condition mean, and the hollow diamond averages the ten users. The diagonal marks equal write magnitude. In (c–d), violins show the matched-pair distributions, overlaid boxes span the interquartile range with median lines and 5th–95th percentile whiskers, and colored points show user means. Panel (e) reports each user’s paired difference and its across-user mean; horizontal error bars give the user-bootstrap 95% confidence interval. The differences are Same minus Cross for write-pattern similarity and Cross minus Same for one-step survival, so positive values indicate stronger same-domain alignment and revision, respectively. In (f), the shaded bands give user-bootstrap 95% confidence intervals for the mean survival curves. Event-role matching is performed without replacement within the same user, trajectory, and segment count. A minimum-total-cost assignment uses the sum of absolute differences in normalized session position and log token count, each scaled by its within-stratum interquartile range. A zero interquartile range is replaced by one. We report the resulting comparisons as associations along the observed histories and assess residual covariate balance using absolute standardized mean differences. E.3 Session-Level Write Statistics For a semantic session e containing one or more compiler segments, let ZeZ_e denote the coordinate-wise retention of memory present before the event. For events following initialization, Ze=∏k∈ezkZ_e= _k∈ ez_k, where k indexes the event’s compiler segments. The event containing the first segment has Ze=0Z_e=0 because that segment directly initializes memory. We summarize the resulting update by the scalar write magnitude we=1−mean⁡(Ze).w_e=1-mean(Z_e). (E.4) The mean is taken over all layer, module, rank, and latent-feature coordinates of ZeZ_e. Retention and write statistics are computed over fusion steps t≥2t≥ 2, following the direct initialization h1=q1h_1=q_1. Within each trajectory, the write range is the maximum minus the minimum write magnitude over these fusion steps; the reported trace range averages these ranges across trajectories. The untrained gate assigns z=σ⁡(−2)=0.119z=σ(-2)=0.119 to every coordinate of each gated update. Downstream training raises the mean retention to 0.670 and produces an average within-trajectory range of 0.141 in write magnitude, as shown in Table E.2. Table E.2: Consolidation statistics on PERMA Clean SD for fusion steps t≥2t≥ 2. Consolidation Retain mean Retain P5/P50/P95 Write mean Trace range Untrained gate 0.119 0.119/0.119/0.119 0.881 0.000 Learned gate 0.670 0.636/0.672/0.698 0.330 0.141 Domain-emergence and supplement events correspond to PERMA’s preference-emergence and preference-supplement annotations, respectively. The semantic-role comparison uses 810 matched emergence–supplement pairs. The mean write difference is +0.056+0.056 with a user-level bootstrap 95% confidence interval of [0.039,0.073][0.039,0.073]. After matching, the absolute standardized differences are 0.055 for log token count and 0.962 for normalized position. The association therefore reflects semantic role together with its temporal placement in the benchmark. The Clean/Noisy comparison aligns sessions by user, task identifier and type, session index, date, event type, domain, and trajectory role. All 34,218 session occurrences align across the two variants; the reported perturbation comparison selects the 18,234 pairs whose conversation text differs. Noise changes text within the aligned histories while preserving their event structure. Each variant uses its corresponding trained gate, so this comparison measures the behavior of the complete trained recurrence under the two benchmark conditions. The mean Noisy-minus-Clean difference is +0.011+0.011, and the mean absolute per-session drift is 4.8% of the Clean write magnitude. E.4 Domain Updates and Long-Term Survival For the domain analysis, we flatten, center, and normalize the coordinate-wise write pattern 1−Ze1-Z_e and compute cosine similarity for matched same-domain and cross-domain event pairs. One-step survival is the ratio between a historical source coefficient immediately after and before a new update. Same-domain and cross-domain write-pattern similarities are 0.872 and 0.784, respectively; the corresponding one-step survival values are 0.458 and 0.818. Write-pattern pairs are matched within user and trajectory, with equal source and target segment counts. Matching covariates include both event positions, their temporal separation, and both log token counts. Transition comparisons similarly match source age, incoming-event position, and both log token counts. These comparisons characterize domain-associated update patterns in the observed trajectories. At recurrence-step resolution, source i’s survival after a subsequent updates is Li(a)=wi,i+awi,i,L_i(a)= w_i,i+aw_i,i, (E.5) where a is a nonnegative integer and division is element-wise. The source coefficients are defined by Equation E.2 for i=1i=1 and Equation E.3 for i≥2i≥ 2. For either case, Li(a)=∏k=i+1i+azkL_i(a)= _k=i+1^i+az_k, with Li(0)=1L_i(0)=1. For multi-segment semantic sessions, subsequent segment retentions are composed over the corresponding event boundaries. For the scalar survival curves, we divide the coordinate mean of the remaining source coefficient by its coordinate mean at insertion. This weights coordinate-wise survival by the source’s initial write coefficients. At each session age, eligible sources are those whose recorded trajectory extends to that age. Their scalar values are averaged within user and then equally across eligible users. All five checkpoints below include ten users. The emergence-source count is 810 at each checkpoint; supplement-source counts are 2,871 at ages 1, 5, and 10, 2,502 at age 40, and 1,782 at age 60. Table E.3 reports the numerical checkpoints underlying the survival curves in the main text. Table E.3: Mean source survival after subsequent session updates. Source role 1 5 10 40 60 Domain emergence 0.294 0.121 0.081 0.038 0.032 Supplement 0.225 0.058 0.028 0.008 0.005 At segment resolution, the untrained gate follows the constant decay σ(−2)a≈0.119aσ(-2)^a≈ 0.119^a and reaches 2.41×10−52.41× 10^-5 after five gated updates. A semantic-session horizon composes all intervening segment updates. At the five-session checkpoint, the learned recurrence retains 0.121 of emergence sources and 0.058 of supplement sources. Appendix F Full Cross-Backbone Transfer Results F.1 Transfer Protocol and Paired Evaluation Source and target modules. Head Transfer reuses the session encoder from the Qwen3-8B compiler at update 51,885. Its context encoder EξE_ξ and Perceiver-based resampler GϕG_φ remain fixed while a target-specific decoder Dβ′D_β is trained for the frozen target backbone fθ′f_θ . From Scratch initializes the resampler and target decoder anew and optimizes both, with the context encoder and target backbone held fixed. The two conditions therefore compare reuse of the learned memory encoding with target-side learning from random initialization. At backbone replacement, the adapted decoder reads the retained memory hTh_T and produces Dβ′(hT)D_β (h_T) for the new backbone, following Algorithm A.3. The memory shape, session encoder, and learned consolidation gate remain unchanged; the target decoder accommodates the new backbone’s layer count and module dimensions. Target-side training. Both conditions use the same 664,128 compilation sessions and fixed reference responses. Reference distributions are prepared with the corresponding target backbone conditioned on each textual session; the paired runs for that target share these cached distributions. Each run completes one pass, 10,377 updates, with global batch size 64 and the same random seed, using the fixed-reference forward-KL objective. The source compiler’s five-pass training precedes this target-side comparison and supplies the checkpoint shared across transfer targets. The matched budget refers to the one-pass target training stage. The target configurations retain eight Perceiver queries, latent width 512, LoRA rank eight, nine Perceiver encoder blocks, and a four-block decoder pre-head. Dense targets adapt the MLP down-projection in each layer. For Qwen3.5-35B-A3B, the decoder generates factors for the always-active shared expert’s down-projection. The documented optimizer configuration uses AdamW with learning rate 4×10−54× 10^-5, weight decay 0.01, 500 warmup updates, gradient clipping at 1.0, and seed 42. The factor regularization coefficient is 0.01, and the LoRA scaling coefficient is 32. Batch construction and runtime alignment. Qwen3-4B, Ministral-3-8B, and Qwen3.5-9B use eight single-GPU data-parallel replicas, each processing one session per microbatch and accumulating eight microbatches. The Qwen3.5-35B-A3B protocol distributes each frozen backbone replica over eight GPUs and runs four replicas. Each replica accumulates 16 sessions, and trainable-module gradients are averaged across replicas. Both layouts yield 64 sessions per optimizer update, with the same batch construction for Head Transfer and From Scratch. Validation is scheduled every 5,000 updates and at the final epoch boundary; the completed one-pass checkpoint supplies the downstream compiler. For Qwen3.5 targets, reference target preparation and compiler training use matching frozen Transformers and Tokenizers runtimes. Launch-time checks compare these runtimes with the target asset manifest to preserve token and response-position alignment. Downstream evaluation. For the paired compiler comparison, each resulting compiler is frozen before fitting its consolidation gate with the same architecture and ten-fold leave-one-user-out PERMA protocol. Gate training uses nine users in each fold, and the remaining user supplies the reported evaluation examples. This downstream task-adaptation stage is separate from training the target decoder on the compilation corpus. Table F.1 and Table F.2 report the complete paired From Scratch and Head Transfer results over all seven PERMA variants. Figure 2 reports the corresponding RPMem performance for each language model. Figure F.1: Cross-backbone transfer across the PERMA memory lifecycle (accuracy, %; seven-variant average). Vertical axes begin at 60%. Table F.1: Cross-backbone accuracy (%) on the four single-domain PERMA variants. Target backbone Initialization Clean SD Noisy SD Style SD Style-Long SD SD Avg. Qwen3-4B From scratch 75.67 75.27 75.30 74.57 75.20 Head transfer 86.20 85.83 85.83 86.07 85.98 Ministral-3-8B From scratch 76.23 76.40 75.97 76.37 76.24 Head transfer 90.03 89.37 89.27 89.23 89.48 Qwen3.5-9B From scratch 79.10 79.57 78.57 78.53 78.94 Head transfer 96.50 96.50 96.00 95.83 96.21 Qwen3.5-35B-A3B From scratch 80.53 82.57 81.13 81.07 81.33 Head transfer 93.53 93.00 94.37 93.70 93.65 Cross-model average From scratch 77.88 78.45 77.74 77.63 77.93 Head transfer 91.57 91.18 91.37 91.21 91.33 Table F.2: Cross-backbone accuracy (%) on the three multi-domain PERMA variants and all seven variants. Target backbone Initialization Clean MD Noisy MD Style MD MD Avg. Overall Avg. Qwen3-4B From scratch 62.47 60.03 60.95 61.15 69.18 Head transfer 78.89 79.32 78.92 79.04 83.01 Ministral-3-8B From scratch 63.64 63.79 63.79 63.74 70.88 Head transfer 81.70 84.72 82.59 83.00 86.70 Qwen3.5-9B From scratch 83.33 84.63 83.97 83.98 81.10 Head transfer 92.52 92.68 92.57 92.59 94.66 Qwen3.5-35B-A3B From scratch 84.40 86.00 83.22 84.54 82.70 Head transfer 95.17 94.97 94.73 94.96 94.21 Cross-model average From scratch 73.46 73.61 72.98 73.35 75.97 Head transfer 87.07 87.92 87.20 87.40 89.64 F.2 Target-Backbone Adaptation Cost Table F.3reports the target-side compilation stage defined above. Trainable parameters count the optimized decoder weights in Head Transfer and the resampler plus decoder weights in From Scratch. The decoder is a shared adapter-generating network; the per-user memory and its generated LoRA factors are measured separately in Appendix D.5. Time is the recorded wall-clock duration of the one-pass target training run. The first three targets use eight GPUs on one node, while Qwen3.5-35B-A3B uses 32 GPUs across four nodes. Timing comparisons are paired within each target and hardware allocation. Validation loss is the final checkpoint’s fixed-reference forward-KL validation metric on the compilation data. Table F.3: One-pass target-backbone compilation cost. Target backbone Initialization Trainable params GPUs Time (h) Val. loss Qwen3-4B From scratch 580.67M 8 7.64 0.7568 Head transfer 532.40M 8 6.74 0.6759 Ministral-3-8B From scratch 659.72M 8 8.16 0.6827 Head transfer 611.45M 8 7.46 0.5944 Qwen3.5-9B From scratch 589.67M 8 32.27 0.6486 Head transfer 541.40M 8 30.92 0.5815 Qwen3.5-35B-A3B From scratch 437.48M 32 41.66 0.5713 Head transfer 389.21M 32 38.42 0.5121 Freezing the transferred resampler reduces target-side trainable parameters by 7.32% to 11.03% across the four backbones. Adaptation time decreases by 4.17% to 11.74%, and the final Fixed FKL validation loss is lower for every target. Together with the paired downstream results, these measurements show that the retained encoder improves target-side learning under the same target-data exposure. The cost table accounts for target-side training after the source compiler and reference targets have been prepared. Source compilation is the shared upstream training investment described in Appendix D.4. Downstream gate fitting has its own per-fold cost, reported in Appendix D.5, which also defines the forward-only write and query measurements. Thus the reported training durations and deployment latencies refer to distinct stages of the memory lifecycle. Appendix G Additional Comparisons with Metis G.1 Comparison on Qwen3.5-9B To control backbone family and scale in the comparison with Metis, we evaluate No Context, Full Context, Metis-9B, and RPMem on Qwen3.5-9B. All four methods share the PERMA task splits, visible-history boundary, non-thinking mode, and A–H next-token-logit answer selection. RPMem uses the adapted decoder described in Appendix F and fits its consolidation gate on the nine training users of each fold. Metis-9B uses its released memory-model weights without additional PERMA training. Table G.1: Same-backbone comparison on PERMA with Qwen3.5-9B (accuracy, %). Method Clean SD Noisy SD Clean MD Noisy MD Core Avg. 7-Variant Avg. No Context 29.20 29.20 35.63 35.63 32.41 31.98 Full Context 69.27 72.00 85.71 85.68 78.17 74.25 Metis-9B 80.87 80.47 80.17 79.29 80.20 80.04 RPMem (ours) 96.50 96.50 92.52 92.68 94.55 94.66 Core Avg. averages the four displayed settings; 7-Variant Avg. additionally includes Style SD, Style-Long SD, and Style MD. Under this shared backbone and evaluation protocol, RPMem exceeds Metis-9B by 14.35 p on the core average and 14.62 p across all seven variants. G.2 Metis Scale Results We evaluate all three released Metis scales under the same seven-variant PERMA protocol. Table G.2 shows that Metis-9B obtains the highest overall average and is therefore used in the main comparison; Metis-27B is strongest only on Clean MD. Table G.2: PERMA accuracy (%) across released Metis model scales. Model Clean SD Noisy SD Style SD Style-Long SD Clean MD Noisy MD Style MD Core Avg. Metis-4B 73.77 74.67 73.27 73.13 70.81 70.71 68.34 72.49 Metis-9B 80.87 80.47 79.67 79.97 80.17 79.29 79.87 80.20 Metis-27B 78.67 74.87 77.03 77.03 81.55 78.45 78.65 78.39 Appendix H Evaluation Protocol Checks and Supplementary Diagnostics We check prompt reproduction and answer scoring to contextualize the PERMA comparisons. Under the original prompt, Qwen3-32B reproduces the official standalone Type-3 results within 0.7 p in each setting; Table H.1 also reports the results under our prompt. For Qwen3-8B in the No Context condition, API generation and local A–H logits differ by 0.22 p on SD and 1.90 p on MD. Table H.1: Reproduction of PERMA standalone Type-3 accuracy (%) with Qwen3-32B. Setting Reported Official prompt Our prompt Clean SD 87.00 87.00 87.00 Noisy SD 87.70 87.23 87.71 Clean MD 93.60 93.63 87.90 Noisy MD 93.00 92.36 89.17 Macro average 90.33 90.05 87.94 Query-only evaluation provides a reference for performance available without historical information. Qwen3-32B reaches 75.58–75.89% on Type 3 under this condition, and Qwen3-8B reaches 52.70–66.95% depending on domain. Table H.2 supplements these checks with our GPT-4o-mini No Context reproduction and memory-system results reported by PERMA [26]; the table distinguishes their provenance. Together, these results document protocol sensitivity and query-only performance. Attribution of the observed differences to information loss, contextual interference, or answer-option cues requires additional controlled comparisons. Table H.2: Supplementary GPT-4o-mini results on PERMA (accuracy, %). No Context is our reproduction; all other rows are reported results from the benchmark paper. Method Clean SD Noisy SD Clean MD Noisy MD Avg. No Context 73.19 73.19 82.26 82.26 77.73 Full history 78.00 76.60 70.70 72.00 74.33 RAG (BGE-M3) 70.20 71.90 68.20 66.90 69.30 MemOS 81.10 85.30 73.20 75.20 78.70 Mem0 68.60 66.00 65.00 66.20 66.45 LightMem 65.70 67.10 60.50 63.10 64.10 Memobase 73.30 68.30 69.40 64.30 68.83 EverMemOS 72.80 69.50 71.30 73.20 71.70 Supermemory 65.50 67.40 65.60 63.70 65.55 Appendix I Additional Gate Dynamics Visualization and Case Studies Figure I.1presents a deterministically selected representative trajectory: among 423 candidates with at least 40 sessions, three domains, and both emergence and supplement events, it is nearest to the multivariate median of four gate statistics. The trajectory contains 82 sessions for held-out user 507, including ten preference events from seven domains and 72 distractor sessions. The write curve displays sessions 2–82, with the first session omitted. This instance illustrates the aggregate pattern in subsection 4.2: early domain-establishing events can form a long-lived backbone even when later same-domain updates temporarily dominate. Figure I.1: Representative held-out trajectory: event timeline (top), session-wise write magnitude from session 2 onward (middle), and memory-unit source shares (bottom). The timeline and heatmap cover the complete history. E and S denote emergence and supplement events on the task’s preference timeline; I denotes a block of consecutive intervention sessions. An intervention may replay a preference event from another timeline, including a domain’s emergence. For each pair of units, the heatmap gives the earlier unit’s summed source share at the end of the later unit, mirrored across the diagonal. Color intensity uses a logarithmic scale. Circled numbers identify three selected events.

Instant research discovery