Paper deep dive
REMORY: Learning Residual Memory for Context Compaction
Hanchen Xia, Baoyou Chen, Yutang Ge, Naihao Deng, Senqiao Yang, Zilong Dong, Weihao Yuan, Siyu Zhu
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 10/9/2026, 8:34:49 AM
Summary
The paper introduces REMORY, a neural memory network designed to enhance long-horizon AI agents by supplementing textual summaries with bounded sequences of soft memory tokens. This residual memory approach allows frozen Large Language Models (LLMs) to approximate continuations from full history using only a fraction of input positions, significantly improving source attribution, reducing tool errors, and increasing performance on benchmarks like BrowseComp and Terminal-Bench 2.1.
Entities (8)
Relation Signals (6)
Qwen3.8-27B → evaluatedon → BrowseComp
confidence 95% · Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory.
GLM-5.3-Flash → evaluatedon → Terminal-Bench 2.1
confidence 95% · Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory.
REMORY → uses → Soft Memory Tokens
confidence 95% · We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens.
REMORY → targets → Context Compaction
confidence 93% · Title: REMORY: Learning Residual Memory for Context Compaction
REMORY → reduces → Tool Errors
confidence 92% · Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1.
REMORY → improves → Source Attribution
confidence 90% · On SummHay, REMORY improves source attribution at nearly unchanged insight coverage...
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1.
Tags
Links
- Source: https://arxiv.org/abs/2610.11287v1
- Canonical: https://arxiv.org/abs/2610.11287v1
Trouble viewing inline? Open PDF directly →
Full Text
29,574 characters extracted from source content.
Expand or collapse full text
REMORY: LEARNING RESIDUAL MEMORY FOR CONTEXT COMPACTION Hanchen Xia 1 Baoyou Chen 1,2 Yutang Ge 3 Naihao Deng 4 Senqiao Yang 5 Zilong Dong 6 Weihao Yuan 7 Siyu Zhu 2,1,8,* 1 Shanghai Academy of AI for Science 2 Fudan University 3 Shanghai Jiao Tong University 4 University of Michigan 5 The Chinese University of Hong Kong 6 Alibaba Group 7 Nanjing University 8 Shanghai Innovation Institute xiahanchen, chenbaoyou@sais.org.cn * Corresponding author. Abstract Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1. 1 Introduction Long-horizon agents work within a finite context window. As observations, tool outputs, and decisions accumulate, they must compact their history to continue. A common practice today is to replace earlier interaction with a textual summary. For example, popular coding agents such as Claude Code and Codex CLI provide/compactto free context space (Anthropic, 2026; OpenAI, 2026a). The summary becomes a checkpoint from which the agent resumes its work. Input length follows a sawtooth pattern: it grows with interaction, drops at compaction, and grows again (Figure 2a). A textual checkpoint changes how prior context is available to the agent. Preserving the main thread does not guarantee that the summary will support every subsequent decision, and the relevance of earlier context can change as the task unfolds. We do not assume that the information needed beyond the summary can be identified or enumerated in advance. The challenge is to sustain useful access to prior context across compaction without keeping the entire interaction in the active window. Human cognition illustrates how a limited active workspace can support sustained work. While contemporary agent contexts span roughly 0.2–1.0 million token positions, human working memory holds only a few chunks under controlled conditions (Baddeley, 2012; Cowan, 2001). Yet people can pursue complex projects over months or years, repeatedly drawing on details, experience, and learned skills in long-term memory (Ericsson & Kintsch, 1995; Squire & Dede, 2015). This ability depends on access to stored knowledge as well as active maintenance. Hippocampal indexing theory describes one form of neural indexing: partial cues activate an index that reinstates distributed cortical representations of an earlier experience (Teyler & Rudy, 2007). 1 arXiv:2610.11287v1 [cs.CL] 8 Oct 2026 REMORY: Learning Residual Memory for Context CompactionPreprint Qwen3.8 27B GLM-5.3 Flash Gemini 3.1 Pro Claude Mythos 5 GPT-5.6 Sol 0 50 100 Score (%) 74.0 77.0 84.9 89.0 85.9 88.0 90.4 (a) BrowseComp Qwen3.8 27B GLM-5.3 Flash Gemini 3.1 Pro Claude Mythos 5 GPT-5.6 Sol 0 50 100 Score (%) 71.9 76.4 84.3 87.6 70.7 88.0 88.8 (b) Terminal-Bench 2.1 Qwen w/o residualQwen w/ residualGLM w/o residualGLM w/ residual Figure 1: BrowseComp and Terminal-Bench 2.1 scores. Pale and solid bars compare the same local actor without and with residual memory. Gray bars are published frontier references, with their original harnesses and protocols (Google DeepMind, 2026; Terminal-Bench Team, 2026; OpenAI, 2026b); they are not controlled comparisons with our runs. 0200400600800 Request 0 50 100 150 Input tokens (k) (a) Repeated compaction(b) Human memory Figure 2: Context compaction and human memory. (a) Input length across successive requests in an agent run, showing repeated growth and sharp reductions. (b) Memory systems illustrated using the Harvard–Oxford atlas (Harvard–Oxford Atlas Contributors, 2026): prefrontal regions (blue), hippocampus (cyan), and association cortex (purple). Figure 2b illustrates the complementary memory systems involved (Goldman-Rakic, 1995; Kumaran et al., 2016). For agents, this suggests pairing an explicit checkpoint with a learned neural representation that keeps prior context useful as work continues. We introduce REMORY, a neural memory network for context compaction. Given a history and its generated summary, we train the network to generate a bounded sequence of soft memory tokens that can be appended after the summary, helping a frozen LLM approximate the continuation it would produce with the full history. We call this residual memory because the soft tokens are both conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. The summary supplies a readable checkpoint, while the learned tokens supplement it during subsequent decisions. Exact records remain accessible through ordinary archive and file-retrieval tools. On SummHay, residual tokens improve source attribution at nearly unchanged insight coverage. This supports an associative interpretation: the memory helps connect recovered claims to supporting evidence. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Residual memory also reduces repeated tool outputs and tool errors for both models on BrowseComp and Terminal-Bench 2.1. REMORY pushes the 320B-parameter GLM-5.3-Flash (Z.ai, 2026a) towards frontier-level performance (Figure 1). 2 REMORY: Learning Residual Memory for Context CompactionPreprint 2 Residual Memory (a) Codex CLI compaction(b) Residual compensation ┌─┐ ┌─┐ │CURRENT CHAT CONTEXT│ │CURRENT CHAT CONTEXT│ └─┬─┘ └─┬─┬─┘ │ ┌─▼─┐┌─▼─┐ ┌─▼─┐ │/compact│/compact│ ┌─▶│REMORY│ └─┬─┘└─┬─┘ │ └─┬─┘ ┌─▼─┐┌─▼─┐ │ ┌─▼─┐ │ Text summary │ Text summary │─┘ │ Compensation │ └─┬─┘└─┬─┘ └─┬─┘ │└─┬─┘ ┌─▼─┐ ┌─▼─┐ │SUMMARY│ │SUMMARY + COMPENSATION│ └─┬─┘ └─┬─┘ │ ┌─▼─┐ ┌─▼─┐ │NEXT MODEL CALL│ │NEXT MODEL CALL│ └─┘ └─┘ 52k4k Training step 78% 80% 82% (c) Reconstruction 00.75k1.5k Training step 87% 90% 93% 96% (d) Residual Stage-I init. Random init. Figure 3: Residual compensation and training. (a) Codex CLI passes the/compactsummary to the next actor call. (b) The REMORY network uses that summary to condition memory construction from the active context; summary and continuous memory jointly support subsequent actions. Archive access remains an ordinary tool action in either setting. (c,d) Qwen3.8-27B teacher–student top-1 agreement during Stage I reconstruction and Stage I residual training. The residual panel compares reconstruction initialization with random initialization under the same residual objective and data order. Curves are optimization diagnostics from the training streams. 2.1 A memory supplement to the summary Residual memory is constructed by conditioning on a generated summary and appending learned soft tokens after it (Figure 3(a,b)). At compaction boundaryi,H i contains the previous summary and residual memory, when present, together with the interaction since the last boundary. Given the new summaryS i , the memory network constructs M i =C φ (H i ;S i ), |M i |≤ m.(1) M i is a sequence of vectors in the frozen actor’s input space. The concatenation[S i ;M i ]forms an analogue of a residual connection along the sequence dimension. The actor reads persistent instructions, this sequence, and then new interaction. The same process repeats at later compactions, allowing memory to carry information across successive boundaries. Archive excerpts enter through ordinary tool results. The network uses frozen actor features and learned queries to predict memory vectors in parallel, with gated cross-attention to the summary. Each block of up to 1,024 source positions produces 64 soft tokens. Hierarchical compression bounds the final memory at 4,096 slots while preserving source order. Source encoding and intermediate representations may be larger. For Qwen3.8-27B, we adapt the DFlash architecture (Chen et al., 2026); for GLM-5.3-Flash, we use DFlash-2 with grouped dynamic convolutions (Inco AI, 2026). The resulting memory networks have 1.94B and 1.502B trainable parameters, respectively. Their output is continuous memory rather than vocabulary tokens. Gradients through the frozen actor train this interface without changing the actor’s parameters. 2.2 Learning through a frozen actor Figure 3(c,d) shows optimization behavior during the two training stages. 3 REMORY: Learning Residual Memory for Context CompactionPreprint Table 1: SummHay on the same 92 queries. Summary++ adds text under the residual memory’s additional-position budget. Higher is better; all queries, including generation and formatting failures, are retained. RepresentationCoverageCitation F1Joint score Raw context72.4656.6043.41 Summary67.5556.9839.84 Summary++67.5256.9439.78 Summary + REMORY67.9561.0343.39 Stage I: reconstruction. The teacher receives a source passage and an instruction to repeat it. The student replaces the passage with its soft memory and receives the same instruction. Both predict the source text, teaching the memory network to preserve source information. We use text fields from MMFineReason-Full- 2.3M (Lin et al., 2026). Stage I: summary-conditioned continuation. Starting from Stage I, the teacher receives the original interaction history. The student receives the corresponding fixed summary and memory constructed from that history. Both predict the same subsequent assistant response. The memory encoder never receives the future response. This stage uses reasoning and coding trajectories, including prompts from Open- PerfectBlend (Labonne, 2024), with precomputed summaries and Qwen-generated responses. Examples follow a curriculum ordered by preceding-context length. In both stages, the teacher and student share the same frozen actorp θ . For a target responsey, their predictions at position j are p j = p θ (·| C full ,y <j ), q j = p θ (·| C mem ,y <j ),(2) where both contexts include the persistent instructions. We optimize the memory parameters through student-to-teacher reverse KL: L(φ) = P j w j D KL (q j ∥p j ) P j w j .(3) The loss covers the full vocabulary and complete response, with positive weights that mildly emphasize earlier positions. Onlyφis updated. In Stage I, the summary is already available, directing memory toward the remaining prediction gap. Memory may overlap with summary content; it need not reconstruct every omitted fact. 3 Experiments 3.1 Source attribution after compaction SummHay evaluates query-relevant insight recovery and source attribution from long document collec- tions (Laban et al., 2024). We evaluate all 92 queries from ten collections with Qwen3.8-27B under four representations: raw context, summary alone, summary plus an additional text supplement (summary++), and summary plus residual memory. Summary++ uses the same additional-position budget as the memory condition. The actor, query, decoding, and base summary are fixed across the compacted conditions. Answer generation has no tools, archive access, or source retrieval; gold insights and citations are used only for scoring. Coverage measures recovered insights; citation F1 measures attribution; the joint score combines both. We use GPT-5.6-Sol with the benchmark coverage prompts. Coverage and joint score average over all 621 reference insights, while citation F1 averages over covered insights. Adding residual memory increases citation F1 by 4.04 points and joint score by 3.55, with a coverage change of 0.40 (Table 1; gains are computed before 4 REMORY: Learning Residual Memory for Context CompactionPreprint Table 2: Qwen3.8-27B workflow results: mean±standard deviation over five runs. Costs are mean API-equivalent USD per task. AutomationBenchJobBench ConfigurationScoreCostScoreCost w/o residual35.5± 1.3$0.5933.4± 1.25$3.09 w/ residual45.3± 1.0$0.5341.0± 1.41$3.21 Table 3: Terminal-Bench 2.1 and BrowseComp: one run per configuration. Costs are mean API-equivalent USD per task. Repeats and errors are tool-output counts over the full task set. Terminal-Bench 2.1BrowseComp ActorResidual ScoreCost Repeats Errors ScoreCost Repeats Errors Qwen3.8-27Bw/o71.9 $4.2945993374.0 $5.3925,801 25,244 w/76.4 $2.8637756877.0 $4.6918,105 12,698 GLM-5.3-Flash w/o84.3 $3.2733466484.9 $5.658,3167,498 w/87.6 $3.0524049689.0 $5.005,8345,412 rounding). A paired collection bootstrap gives a 95% interval of[+0.74, +6.21]for the joint-score gain, conditional on the generated and judged outputs. On the 553 insights covered by both summary conditions, citation F1 still rises from 57.31 to 61.41. The larger attribution gain at nearly unchanged coverage suggests that memory helps connect recovered claims to evidence. Mean inputs contain 96,872 positions for raw context, 1,393 for summary alone, and 5,024 for summary plus memory. The latter uses 5.2% of the raw-context positions and approaches its joint score, with lower coverage and higher citation F1. The equal-budget text supplement remains close to summary alone. In this setting, the learned supplement preserves attribution cues more effectively than allocating the same extra positions to generated text. 3.2 End-to-end agent performance We evaluate BrowseComp research tasks (Wei et al., 2025), Terminal-Bench 2.1 command-line tasks (Merrill et al., 2026), AutomationBench workflows (Shepard & Salimans, 2026), and JobBench professional tasks (Li et al., 2026b). Each within-actor comparison holds weights, the Codex CLI harness, prompts, tools, decoding, budgets, and compaction policy fixed; both configurations can retrieve exact history. Qwen and GLM use 262,144-position and one-million-position context windows, respectively. The intervention adds up to 4,096 residual slots at compaction. We use the full task sets: 1,266 BrowseComp and 89 Terminal-Bench tasks for both actors (one run), and 600 AutomationBench and 65 JobBench tasks for Qwen (five runs). Terminal-Bench and AutomationBench use task verifiers; BrowseComp and JobBench use matched GPT-5.6-Sol judges. Residual checkpoints complete both training stages. Qwen improves by 9.8 points on AutomationBench and 7.6 on JobBench (Table 2), extending the gains to business workflows and professional-task rubric credit. It gains 4.5 points on Terminal-Bench and 3.0 on BrowseComp; GLM gains 3.3 and 4.1, respectively (Table 3). The improvements therefore also hold for a stronger actor with a larger context window. Published frontier scores in Figure 1 provide context; the controlled evidence comes from the within-actor comparisons. Repeated tool outputs fall by 17.9–29.8% and errors by 25.3–49.7% across the four actor–benchmark pairs in Table 3. Repeats count identical outputs from the same tool within a task, excluding planning and polling; 5 REMORY: Learning Residual Memory for Context CompactionPreprint 12:0012:3013:0013:3014:0014:3015:0015:3016:0016:30 Time (UTC, 2026-09-22) 0 40K 80K 120K 160K 200K Input tokens w/ residualw/o residual w/o residual w/ residual 0 50 100 150 200 250 Time between compacts (min) 56 compacts 45 compacts (a) Live context(b) Compact cadence Figure 4: Live Trading context use. (a) An illustrative September 22 trace. (b) Compaction intervals in the separate September 20–23 evaluation: 56/45 completed compactions and 51/41 within-session intervals without/with residual memory. Intervals do not cross restarts; two residual intervals exceed the display range but remain in the box statistics. errors count failed commands or explicit tool errors. These aggregate reductions are consistent with better use of earlier feedback, although complete trajectories differ in their actions and length. Generation cost falls in every reported comparison except JobBench, where the higher score accompanies a rise from $3.09 to $3.21 per task. The estimates price actor and summary tokens at September 2026 uncached provider rates, counting residual positions as input (Alibaba Cloud, 2026; Z.ai, 2026b). They exclude source-feature encoding, memory-network execution, and tools, so they measure token-priced generation rather than total execution cost. 3.3 The first action after compaction We examine 488 paired next actions from 100 sampled Qwen BrowseComp trajectories, holding the persistent prompt and summary fixed. Following rubric-based planning evaluation (Yu et al., 2026), GPT-5.6-Sol compares anonymized actions against criteria fixed from the preceding history. Candidate reasoning is withheld and counterfactual calls remain unexecuted. Residual actions receive 345 wins (70.7%), 21 ties (4.3%), and 122 losses (25.0%). With the checkpoint fixed, the preference links the memory channel to the first post-compaction decision. It is a judgment of proposed actions, not a measurement of the outcomes those counterfactual actions would have produced. 3.4 Case studies Live Trading.Live Trading follows two concurrent Qwen paper-trading accounts under the same context policy. At the September 23 cutoff, settled ROI is−11.86%without residual and+5.11%with it; logged generation costs are $303.81 and $170.30. ROI counts resolved markets against an initial $20,000 bankroll. These two trajectories differ in market selection and timing, limiting return comparisons. Both accounts retain the growth-and-reset pattern (Figure 4). Compactions fall from 56 to 45 alongside fewer responses (5,336 versus 3,013). Per 1,000 responses, compactions instead rise from 10.5 to 14.9: the lower total accompanies less interaction. Mean live context decreases from 105,372 to 101,185 positions including memory. Fewer calls and smaller contexts explain the lower generation spend in this record. Cell segmentation.In a Terminal-Bench cell-segmentation task, a refactor deletes helper functions before compaction. The summary retains the known error and backup identity but omits the exact edit commands. 6 REMORY: Learning Residual Memory for Context CompactionPreprint Task imageObserved mask overlay Figure 5: Cell segmentation after repair. The task image and the exact overlay viewed by the actor are reproduced without pixel changes. With summary and memory available, the actor rereads both files, restores the helpers, and passes all nine verifier tests (Figure 5). The repair illustrates complementary roles for summary, memory, and retrieval; it does not establish which omitted details came from memory. 4 Related Work Agent memory often combines summaries with retrieval. ReadAgent pairs episode gists with original passages, while MemGPT separates working and archival memory (Lee et al., 2024; Packer et al., 2023). ReSum uses periodic summaries for long-horizon search; AgentFold learns multi-scale context management within agent trajectories (Wu et al., 2025; Ye et al., 2025). REMORY keeps the textual checkpoint and retrieval interface, and learns an additional continuous representation conditioned on the checkpoint. AutoCompressors, Gist Tokens, and ICAE learn continuous representations of longer text (Chevalier et al., 2023; Mu et al., 2023; Ge et al., 2024). LCLMs scale encoder–decoder compression and support selective expansion of compressed segments (Li et al., 2026a). Distribution matching provides a behavioral criterion for prompt compression (Wingate et al., 2022); recurrence and memory tokens retain information across segments (Dai et al., 2019; Bulatov et al., 2022). Our objective applies this predictive criterion to the information gap left by a summary, training the supplement through a frozen actor’s continuation predictions. 5 Conclusion REMORY learns a bounded memory supplement for contextual compaction. The summary provides a readable account of prior work, soft tokens carry additional predictive information, and retrieval supplies exact records when needed. Reconstruction followed by summary-conditioned continuation trains this interface while leaving the actor unchanged. Improved source attribution, higher agent scores, and preferred next actions support the usefulness of this residual formulation. 7 REMORY: Learning Residual Memory for Context CompactionPreprint References Alibaba Cloud. Qwen3.8-27B: Model information and pricing, 2026. URLhttps://w.alibabacloud.com /help/en/model-studio/qwen3-8-27b. Anthropic. Explore the context window: Claude Code docs, 2026. URLhttps://code.claude.com/docs/e n/context-window. Alan Baddeley. Working memory: Theories, models, and controversies. Annual Review of Psychology, 63:1–29, 2012. doi: 10.1146/annurev-psych-120710-100422. Aydar Bulatov, Yury Kuratov, and Mikhail Burtsev. Recurrent memory transformer. In Advances in Neural Information Processing Systems, volume 35, p. 11079–11091. Curran Associates, Inc., 2022. doi:10.52202/068431-0805. Jian Chen, Yesheng Liang, and Zhijian Liu. DFlash: Block diffusion for flash speculative decoding. In Proceedings of the International Conference on Machine Learning, 2026. URL https://arxiv.org/abs/2602.06036. Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. Adapting language models to compress contexts. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 3829–3846. Association for Computational Linguistics, 2023. doi: 10.18653/v1/2023.emnlp-main.232. Nelson Cowan. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1):87–114, 2001. doi: 10.1017/S0140525X01003922. Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, p. 2978–2988. Association for Computational Linguistics, 2019. doi: 10.18653/v1/P19-1285. K. Anders Ericsson and Walter Kintsch. Long-term working memory. Psychological Review, 102(2):211–245, 1995. doi: 10.1037/0033-295X.102.2.211. Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. In International Conference on Learning Representations, p. 2591–2607, 2024. URL https://proceedings.iclr.c/paper_files/paper/2024/hash/0b276510ec2d3f6613a8 b60c41f0438-Abstract-Conference.html. Patricia S. Goldman-Rakic. Cellular basis of working memory. Neuron, 14(3):477–485, 1995. doi:10.1016/0896 -6273(95)90304-6. Google DeepMind. Gemini 3.1 Pro model card, February 2026. URLhttps://deepmind.google/models/m odel-cards/gemini-3-1-pro/. Harvard–Oxford Atlas Contributors. Harvard–Oxford cortical and subcortical structural atlases. Distributed with the FMRIB Software Library, 2026. URLhttps://fsl.fmrib.ox.ac.uk/fsl/docs/other/datasets .html. Inco AI. DFlash 2: Keep drafting parallel. Technical blog, August 2026. URLhttps://inco.ai/blog/dflas h2/. Dharshan Kumaran, Demis Hassabis, and James L. McClelland. What learning systems do intelligent agents need? complementary learning systems theory updated. Trends in Cognitive Sciences, 20(7):512–534, 2016. doi:10.101 6/j.tics.2016.05.004. Philippe Laban, Alexander Fabbri, Caiming Xiong, and Chien-Sheng Wu. Summary of a haystack: A challenge to long-context LLMs and RAG systems. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 9885–9903. Association for Computational Linguistics, 2024. doi:10.18653/v1/20 24.emnlp-main.552. Maxime Labonne. Open-PerfectBlend. Hugging Face dataset, 2024. URLhttps://huggingface.co/datas ets/mlabonne/open-perfectblend. Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. A human-inspired reading agent with gist memory of very long contexts. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, p. 26396–26415. PMLR, 2024. URLhttps: //proceedings.mlr.press/v235/lee24c.html. 8 REMORY: Learning Residual Memory for Context CompactionPreprint Ang Li, Sean McLeish, Haozhe Chen, Nimit Kalra, Zaiqian Chen, Artem Gazizov, Venkata Anoop Suhas Kumar Morisetty, Bhavya Kailkhura, Harshitha Menon, Zhuang Liu, Brian R. Bartoldson, Tom Goldstein, Sanae Lotfi, Micah Goldblum, and Pavel Izmailov. End-to-end context compression at scale. arXiv preprint arXiv:2606.09659, 2026a. URL https://arxiv.org/abs/2606.09659. Yuetai Li, Yichen Feng, Zhangchen Xu, Zixian Ma, Kaiyuan Zheng, Fengqing Jiang, Xinghua Sun, Rulin Shao, Zichen Chen, Yue Huang, Xinyang Han, Brian Lee, Kayla Xu, Shenglai Zeng, Hang Hua, Xiangliang Zhang, Basel Alomair, Ranjay Krishna, Luke Zettlemoyer, Pang Wei Koh, Bhaskar Ramasubramanian, Luyao Niu, Xiang Yue, and Radha Poovendran. JobBench: Aligning agent work with human will. arXiv preprint arXiv:2605.26329, 2026b. URL https://arxiv.org/abs/2605.26329. Honglin Lin, Zheng Liu, Yun Zhu, Chonghan Qin, Juekai Lin, Xiaoran Shang, Conghui He, Wentao Zhang, and Lijun Wu. MMFineReason: Closing the multimodal reasoning gap via open data-centric methods. arXiv preprint arXiv:2601.21821, 2026. URL https://arxiv.org/abs/2601.21821. Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, et al. Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. arXiv preprint arXiv:2601.11868, 2026. URL https://arxiv.org/abs/2601.11868. Jesse Mu, Xiang Li, and Noah Goodman. Learning to compress prompts with Gist Tokens. In Advances in Neural Information Processing Systems, volume 36, p. 19327–19352. Curran Associates, Inc., 2023. doi:10.52202/0 75280-0848. OpenAI. Codex developer commands, 2026a. URLhttps://developers.openai.com/codex/cli/slas h-commands. OpenAI. GPT-5.6: Frontier intelligence that scales with your ambition, July 2026b. URLhttps://openai.com /index/gpt-5-6/. Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, and Joseph E. Gonzalez. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560v1, 2023. URLhttps://arxiv.org/abs/23 10.08560v1. Daniel Shepard and Robin Salimans. AutomationBench. arXiv preprint arXiv:2604.18934, 2026. URLhttps: //github.com/zapier/AutomationBench. Larry R. Squire and Adam J. O. Dede. Conscious and unconscious memory systems. Cold Spring Harbor Perspectives in Biology, 7(3):a021667, 2015. doi: 10.1101/cshperspect.a021667. Terminal-Bench Team. Terminal-Bench 2.1, 2026. URLhttps://w.tbench.ai/news/terminal-ben ch-2-1. Timothy J. Teyler and Jerry W. Rudy. The hippocampal indexing theory and episodic memory: Updating the index. Hippocampus, 17(12):1158–1169, 2007. doi: 10.1002/hipo.20350. Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. BrowseComp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516, 2025. URL https://arxiv.org/abs/2504.12516. David Wingate, Mohammad Shoeybi, and Taylor Sorensen. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, p. 5621–5634. Association for Computational Linguistics, 2022. doi:10.18653/v1/ 2022.findings-emnlp.412. Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Xinmiao Yu, Dingchu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Minhao Cheng, Shuai Wang, Hong Cheng, and Jingren Zhou. ReSum: Unlocking long-horizon search intelligence via context summarization. arXiv preprint arXiv:2509.13313, 2025. URL https://arxiv.org/abs/2509.13313. Rui Ye, Zhongwang Zhang, Kuan Li, Huifeng Yin, Zhengwei Tao, Yida Zhao, Liangcai Su, Liwen Zhang, Zile Qiao, Xinyu Wang, Pengjun Xie, Fei Huang, Siheng Chen, Jingren Zhou, and Yong Jiang. AgentFold: Long- horizon web agents with proactive context management. arXiv preprint arXiv:2510.24699, 2025. URLhttps: //arxiv.org/abs/2510.24699. 9 REMORY: Learning Residual Memory for Context CompactionPreprint Xinmiao Yu, Liwen Zhang, Xiaocheng Feng, Yong Jiang, Bing Qin, Pengjun Xie, and Jingren Zhou. WebAnchor: Anchoring agent planning to stabilize long-horizon web reasoning. arXiv preprint arXiv:2601.03164, 2026. URL https://arxiv.org/abs/2601.03164. Z.ai. GLM-5.3-Flash model card, 2026a. URL https://huggingface.co/zai-org/GLM-5.3-Flash. Z.ai. Model API pricing, 2026b. URL https://docs.z.ai/guides/overview/pricing. 10