Paper deep dive

Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations

Ziyang Liu

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 58

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%

Last extracted: 4/15/2026, 1:37:41 AM

Summary

The paper introduces 'cooperative paging,' a memory management technique for long-horizon LLM conversations that replaces evicted context segments with minimal keyword bookmarks. This approach allows models to use a 'recall()' tool to retrieve full content on demand, outperforming traditional truncation and retrieval-based baselines on the LoCoMo benchmark. The study also explores the paging design space, identifying that page granularity and bookmark specificity are critical factors for performance.

Entities (5)

Cooperative Paging · methodology · 100%LoCoMo · benchmark · 100%recall() · tool · 100%DeepSeek-V3.2 · llm · 95%GPT-4o-mini · llm · 95%

Relation Signals (3)

Cooperative Paging → evaluatedon → LoCoMo

confidence 100% · We evaluate cooperative paging on both controlled experiments and the LoCoMo benchmark

Cooperative Paging → usestool → recall()

confidence 100% · the model is given a recall() tool to retrieve the full content

GPT-4o-mini → implements → Cooperative Paging

confidence 90% · Bookmark+Recall achieves the highest score... on GPT-4o-mini

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:When LLM conversations grow beyond the context window, old content must be evicted -- but how does the model recover it when needed? We propose cooperative paging: evicted segments are replaced with minimal keyword bookmarks ([pN:keywords], ~8-24 tokens each), and the model is given a recall() tool to retrieve full content on demand. On the LoCoMo benchmark (10 real multi-session conversations, 300+ turns), cooperative paging achieves the highest answer quality among six methods -- outperforming truncation, BM25, word-overlap retrieval, a search-tool baseline, and full context -- on four models (GPT-4o-mini, DeepSeek-v3.2, Claude Haiku, GLM-5), confirmed by four independent LLM judges ($p=0.017$, paired bootstrap). We then study the paging design space with a 5x4 ablation over boundary strategies and eviction policies (3,176 synthetic probes, 1,600 LoCoMo probes). Key findings: (1) coarse fixed-size pages (fixed_20) reach 96.7% while content-aware topic_shift collapses to 56.7%; (2) eviction policy choice is data-dependent (FIFO best on synthetic, LFU on LoCoMo); (3) two bookmark generation strategies improve over the heuristic baseline (+4.4 and +8.7 E2E points); (4) the remaining bottleneck is bookmark discrimination -- the model triggers recall() 96% of the time but selects the correct page only 57% when bookmarks are insufficiently distinctive. Keyword specificity alone accounts for a 25 percentage point accuracy difference.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

57,817 characters extracted from source content.

Expand or collapse full text

Cooperative Memory Paging with Keyword Bookmarks for Long-Horizon LLM Conversations Ziyang Liu Independent Researcher ziyang.liu.r@outlook.com Abstract When LLM conversations grow beyond the context window, old content must be evicted— but how does the model recover evicted infor- mation when it needs it? Existing approaches either require the model to self-diagnose in- formation gaps via explicit retrieval calls, detect faults only for tool outputs like file reads, or compress content irreversibly. We propose cooperative paging: evicted con- versation segments are replaced with mini- mal keyword bookmarks ([pN:keywords], ∼8 tokens each), and the model is given a recall() tool to retrieve full content on de- mand. The bookmarks serve as a lightweight table of contents for compressed memory, en- abling the model to decide when retrieval is needed.On the LoCoMo benchmark (10 real multi-session conversations, 300+ turns each), cooperative paging achieves the highest answer quality among six methods tested—outperforming truncation, BM25 re- trieval, word-overlap retrieval, a search-tool baseline, and truncated full context—on four models from three provider families (GPT-4o- mini, DeepSeek-v3.2, Claude Haiku, GLM- 5), a ranking confirmed by four independent LLM judges (p=0.017 vs. BM25, paired boot- strap). We then study the paging design space directly with a turn-by-turn simulator (3,176 probes on synthetic long conversations, 1,600 on LoCoMo): a 5×4 ablation over page- boundary strategies (fixed-size, topic-shift, exchange-count) and eviction policies (FIFO, LRU, LFU, B ́ el ́ ady oracle).We find that (1) page granularity dominates: coarse fixed- size pages (fixed20) reach 96.7% while content-aware topicshift over-fragments and collapses to 56.7%; (2) eviction policies are data-dependent:B ́ el ́ ady upper-bounds online policies by 8–14 points, and which online policy wins flips between forward- moving synthetic data (FIFO) and revisit- heavy LoCoMo (LFU); (3) the bookmark gen- eration bottleneck is partially closable: of six bookmark generation strategies we test, two improve over the heuristic baseline—hybrid (heuristic + LLM-discriminative augmenta- tion) on synthetic data (+4.4 E2E points) and llm-batch (single-call cross-page non-overlap) on real LoCoMo dialogues (+8.7 points, +50 on open-domain questions). The other four strategies, including TF·IDF and per-page LLM, under-perform the heuristic by lower- ing the recall trigger rate—a structural fail- ure mode predicted by the information-gap hy- pothesis; (4) the bookmark is now the bottle- neck: the model triggers recall() 96% of the time but selects the correct page only 57% of the time when bookmarks are insufficiently distinctive. A format ablation on controlled probes (n=22) suggests that minimal keyword bookmarks achieve the best accuracy-to-cost ratio, and keyword specificity alone accounts for a 25 percentage point accuracy difference. 1 Introduction Large language models deployed in multi-turn conversations face a fundamental resource man- agement problem: the context window is finite, but conversation history grows without bound. When the context budget is exceeded, old content must be evicted. The question is: how does the model get evicted content back when it needs it? This problem is the memory management prob- lem that operating systems solved decades ago with virtual memory and demand paging (Den- ning, 1968). In OS paging, the application is un- aware of physical memory limits; the OS transpar- ently loads pages from disk when needed. Recent work has applied this analogy to LLMs: Packer et al. (2023) introduced hierarchical memory tiers with explicit retrieval via function calls, and Ma- son (2026) built a transparent paging proxy for agentic coding sessions. However, existing ap- proaches have fundamental limitations. MemGPT requires the model to self-diagnose information gaps—but models cannot reliably detect what they arXiv:2604.12376v1 [cs.CL] 14 Apr 2026 do not know. Pichay detects faults only through tool-call matching, which is limited to addressable outputs like file reads. Compression approaches like Adaptive Focus Memory (Cruz, 2025) lose details irreversibly. We propose cooperative paging: when conver- sation content is evicted, it is replaced with a min- imal keyword bookmark costing only∼8 tokens, e.g.: [p3: allergy, peanut, budget] The model is given a recall() tool to retrieve the full content of any bookmarked segment on de- mand. Unlike MemGPT, where the model must formulate free-form search queries about what it might be missing, our bookmarks provide a table of contents for compressed memory: the model sees which topics are available and decides when to retrieve. This shifts the cognitive load from “guess what you might need” to “check the index.” We evaluate cooperative paging on both con- trolled experiments and the LoCoMo benchmark (Maharana et al., 2024)—a dataset of real multi- session conversations spanning 300+ turns. Our contributions are: 1. A cooperative paging mechanism using min- imal keyword bookmarks and a recall tool. On controlled scenarios, the model achieves 91% recall accuracy with zero observed false positives (n = 22).On LoCoMo, Book- mark+Recall achieves the highest answer qual- ity among six methods on both GPT-4o-mini and DeepSeek-v3.2 (§4). 2. A paging design study that treats page bound- ary and eviction policy as first-class design choices. Using a turn-by-turn paging simula- tor, we run a full 5×4 ablation (3,176 synthetic probes, 1,600 LoCoMo probes) and find that page granularity dominates (fixed-size 96.7% vs. topic-shift 56.7%), B ́ el ́ ady’s oracle beats FIFO by 14 points, and the remaining bottle- neck has shifted from “when to recall” (now 96% correct) to “which page to recall” (57% correct). To our knowledge, this is the first study of page boundary detection and eviction policies for in-session LLM conversation pag- ing (§5). 3. Design principles for bookmark construction: minimal keyword bookmarks (∼8–24 tokens) achieve the best accuracy-to-cost ratio over richer formats, and keyword specificity is the most impactful design factor in our controlled study, accounting for a 25.7 percentage point improvement (§6). 4. Closing the bookmark bottleneck.We propose two complementary bookmark gen- eration strategies—hybrid (heuristic + LLM- discriminative augmentation) and llm-batch (single-call cross-page non-overlap)—that im- prove end-to-end accuracy by +4.4 points on synthetic data and +8.7 points on real LoCoMo dialogues, with a +50 point improvement on open-domain questions. Critically, four other natural strategies (TF·IDF, per-page LLM, etc.) under-perform the heuristic baseline by lower- ing the recall trigger rate—a structural failure mode we link back to the information-gap hy- pothesis (§7). 2 Why Passive Fault Detection Fails 2.1 Hypothesis A natural approach to detecting memory faults is to monitor the model’s output uncertainty. When generating a response, if the model needs informa- tion that was evicted, its token-level NLL should spike—analogous to a hardware page fault in OS virtual memory. EM-LLM (Fountas et al., 2025) demonstrated that Bayesian surprise (a form of prediction error) effectively segments input text into episodic events. We hypothesized that the same signal, applied during output generation, could detect when the model lacks evicted context. 2.2 Experiment We constructed 5 multi-turn conversations, each containing planted critical information (food al- lergies, budget constraints, scheduling conflicts). For each conversation, we generated responses un- der two conditions: (A) full context with all infor- mation present, and (B) gist-compressed context where the critical information was replaced with a short summary. We measured per-token NLL using logprobs from the model API and com- pared rolling NLL windows (w = 32 tokens) be- tween conditions. 2.3 Results Table 1 and Figure 1 show the results. We test 5 conversations: 4 experimental cases with planted critical information plus 1 control case requiring no evicted content. In two of four experimental cases, the gist condition produces lower NLL than Peanut Budget Rust Meeting Control 0.0 0.1 0.2 0.3 0.4 Mean NLL (a) Full vs Gist Full context Gist context Peanut Budget Rust Meeting Control 0.2 0.1 0.0 0.1 0.2 NLL -0.065 -0.118 +0.027 +0.205 +0.025 (b) No Consistent Signal Figure 1: NLL-based fault detection fails. (a) Mean NLL under full vs. gist context across 5 test cases. (b) ∆NLL (gist− full): two cases show reverse signal (gist is more confident), and the mean experimental ∆NLL (+0.012) is indistinguishable from the control (+0.025). Table 1: NLL comparison: full context vs. gist con- text. ∆NLL = gist− full. Negative values indicate the model is more confident with the gist (less informa- tion). CaseFullGist∆NLL Peanut allergy0.3140.249 −0.065 Budget $500.3230.205 −0.117 Rust preference0.0870.114+0.026 Meeting 2–4pm0.0670.272+0.205 Control0.1920.217+0.025 the full context condition—the opposite of the hy- pothesized signal. The mean ∆NLL across the 4 experimental cases (+0.012) is indistinguishable from the control (+0.025). 2.4 Analysis The root cause is that models do not “struggle” when lacking context.Instead, they generate a different response—one that is simpler, more generic, and often more confident.When the model knows about a peanut allergy, it generates cautious restaurant recommendations with hedg- ing language (higher NLL). Without this knowl- edge, it recommends restaurants directly and flu- ently (lower NLL). A confident wrong answer has low NLL. Design implication. This pilot suggests that NLL-based uncertainty signals are not a reliable basis for detecting memory faults in LLMs—at least under gist compression with GPT-4o-mini. The underlying issue is that models do not “strug- gle” when lacking context; they simply produce a different, often more confident, response. While a more exhaustive study (more models, more com- pression regimes, alternative uncertainty signals) could refine this picture, the failure mode is funda- mental enough to motivate a cooperative design: the system should explicitly inform the model about what was compressed, rather than hoping to detect information gaps from output behavior alone. 3 Related Work OS-Inspired LLM Memory. The analogy be- tween context windows and physical memory has motivated several systems. Kwon et al. (2023) apply OS paging to KV cache management at the hardware level.Packer et al. (2023) pro- pose MemGPT, which organizes LLM memory into hierarchical tiers (main context, recall stor- age, archival storage) and uses function calls for retrieval. Mason (2026) build Pichay, a transparent proxy that implements demand paging for agentic context windows, detecting page faults via tool- call matching. Our work differs in two ways: we operate on conversation content rather than tool outputs, and we use cooperative bookmarks rather than requiring explicit retrieval calls or tool-call matching. ContextCompressionandTruncation. StreamingLLM (Xiao et al., 2024) maintains a sliding window with attention sinks, discard- ing middle content permanently. Infini-attention (Munkhdalai et al., 2024) compresses past context into a fixed-size memory matrix.LLMLingua (Jiang et al., 2023) and Selective Context (Li et al., 2023) prune low-information tokens. Adap- tive Focus Memory (Cruz, 2025) assigns each message one of three fidelity levels (full, com- pressed, placeholder) under a token budget. These approaches perform irreversible compression— once details are lost, they cannot be recovered. Our bookmark mechanism is reversible:the recall() tool restores full content from exter- nal storage. Episodic and Retrieval-Based Memory. EM- LLM (Fountas et al., 2025) uses surprise-based segmentation inspired by human episodic memory to organize tokens into coherent events, achiev- ing strong performance on long-context bench- marks.ReadAgent (Lee et al., 2024) applies episode pagination with gist memory for long doc- ument comprehension. Mem0 (Chhikara et al., 2025) and A-MEM (Xu et al., 2025) build pro- Table 2: Comparison with closest related systems along three design axes. MemGPTPichayOurs In-context stubNoneHandleKeyword Recall triggerModel queryTool matchModel + kw Content scopeMessagesTool outputConversation ReversibleYesYesYes Format studyNoNoYes duction memory systems with dynamic extraction and graph-based storage. MemoryBank (Zhong et al., 2024) adds long-term memory with sum- marization and forgetting for multi-turn conver- sations, while SeCom (Pan et al., 2025) studies memory granularity and segmentation for person- alized conversational agents. These systems fo- cus on what to store and how to retrieve; we focus on the complementary question of how to signal when retrieval is needed—and find that minimal keyword cues suffice. Positioning. Table 2 summarizes the key differ- ences between our approach and the most closely related systems. 4 Cooperative Paging with Keyword Bookmarks 4.1 Mechanism Our approach replaces evicted conversation seg- ments with minimal keyword bookmarks and pro- vides a recall() tool for on-demand retrieval. Figure 2 illustrates the full pipeline. Segmentation. Conversation turns are grouped into pages of 5 turns each. When the context bud- get is exceeded, the oldest pages are compressed. Bookmark generation. Each compressed page is replaced with a keyword bookmark: [p3:allergy,peanut,budget] Keywords are extracted automatically from page content using a simple heuristic: capitalized multi- character tokens, numbers, and dates from the first 3–4 turns of each page, filtered against a 60- word stopword list and limited to 3–5 keywords. No manual curation, LLM calls, or conversation- type-specific rules are used—the same extraction pipeline runs identically for all conversation types. Each bookmark costs∼8–24 tokens (depending on keyword count and format syntax), compared to∼200–500 tokens for the original page content. Table 3: Recall accuracy by critical information type on the controlled evaluation (22 probes, n per type shown). Info TypeRecall Acc.n Budget100%3 Deadline86%7 Allergy100%2 Medical100%6 Preference75%4 Overall90.9%22 Recall tool. The model receives a single tool: recall(pageids=[3,5]) → full content of pages 3 and 5 The system prompt instructs: “If you need specific details from a bookmark, call recall() before answering. Do not guess details you do not have.” Recovery. When the model calls recall(), the proxy injects the full page content from an ex- ternal key-value store. Pages can be compressed and expanded repeatedly—the mechanism is re- versible, unlike summarization-based approaches. Key design insight. Unlike MemGPT, where the model must generate free-form search queries about what it might be missing, our bookmarks provide a table of contents: the model sees exactly which topics are available in each compressed seg- ment. This reduces the retrieval decision from open-ended search to index lookup. 4.2 Controlled Evaluation We generated 10 synthetic multi-turn conversa- tions (20–35 turns each) with planted critical in- formation across 7 types: allergy, budget, dead- line, medical, preference, schedule, and contact details.Each conversation was compressed to bookmarks for early pages, and 22 QA probes tested whether the model correctly recalled needed information. Results. On this controlled set, the book- mark mechanism achieves 90.9% recall accuracy (model calls recall() when needed), 95.2% page selection accuracy (correct page retrieved), and 0 observed false positives out of 22 probes. We note that the sample size is small; the LoCoMo evaluation below provides larger-scale validation. Table 3 shows per-type performance. Figure 2: Cooperative memory paging. As the conversation grows (left), turns are grouped into pages that occupy the limited context window (center, k=5 active slots). When capacity is exceeded, an eviction policy selects a page to compress into a∼8-token keyword bookmark (dashed). When the model later needs an evicted detail— in this example the user’s peanut allergy planted early in the conversation—it calls recall(page ids=[1]) and receives the full page content (right). The bookmarks act as a lightweight table of contents over compressed history, shifting the retrieval decision from “guess what you might be missing” to “check the index.” 4.3 LoCoMo Benchmark We evaluate on LoCoMo (Maharana et al., 2024), a benchmark of 10 real multi-session conver- sations (19–30 sessions, 300+ turns each) with human-annotated QA pairs spanning 5 categories: single-hop, temporal reasoning, multi-hop, open- domain, and unanswerable. We compare against 5 baselines: • Full context: all turns included (truncated to 60 turns to fit context window) • Truncation: last 20 turns only • BM25 retrieval: evicted sessions retrieved by BM25 scoring against the query (top-3) • Word-overlap retrieval: evicted sessions re- trieved by word-overlap similarity against the query (top-3) • Search-tool baseline (inspired by MemGPT): model receives a free-form memory search tool with no bookmarks; the backend ranks evicted sessions by BM25 and returns the top-3. This is not a full MemGPT reimplementation but isolates the effect of free-form search vs. struc- tured bookmarks Table 4 shows the GPT-4o-mini results. Book- mark+Recall scores 2.18/5, the highest among all six methods. We validate this ranking more rigor- ously with multi-judge evaluation and paired boot- strap tests in §4.5. It scores above all retrieval baselines: BM25 retrieval (1.86), word-overlap re- trieval (1.88), Search-tool baseline search (1.90), Table 4: LoCoMo results with GPT-4o-mini (98 QA probes, 10 conversations). Bookmark+Recall achieves the highest score, exceeding all retrieval and memory baselines. 95% confidence intervals shown. MethodScore (1–5)95% CI Truncation (20 turns)1.64[1.43, 1.86] BM25 Retrieval (top-3)1.86[1.61, 2.10] Word-Overlap Retrieval (top-3)1.88[1.63, 2.12] Search-Tool Baseline1.90[1.61, 2.19] Full Context (trunc. 60)2.02[1.75, 2.29] Bookmark+Recall2.18[1.89, 2.48] and truncated full context (2.02). The advantage over Search-tool baseline search (+0.28) is no- table: both methods provide retrieval tools, but bookmarks give the model a structured index to guide retrieval decisions, while MemGPT requires open-ended query formulation. Recall accuracy on LoCoMo is 59.4% (95% CI: [49.4%, 68.7%]), lower than the controlled eval- uation (90.9%) due to the much larger number of sessions (19–30 vs. 3–5 bookmarks). Despite im- perfect recall, partial retrieval of relevant sessions provides meaningful quality gains. Efficiency analysis. Bookmark+Recall requires on average 1.6 LLM calls per probe (1.0 for gen- eration + 0.6 for recall-triggered follow-up). The bookmark overhead per conversation is∼150–250 tokens for all compressed session stubs, compared to 2,000+ tokens for full session content. When the model triggers recall, it adds one round-trip latency (∼1–3s). On probes where recall is not Single- hop TemporalMulti- hop Open- domain Unansw. 0 1 2 3 4 Score (1-5) +0.7 +2.2 +0.2 +1.6 +1.2 Full ContextTruncationBookmark+Recall Figure 3: LoCoMo per-category comparison. Book- mark+Recall outperforms baselines across all 5 QA categories, with the largest gains on temporal reason- ing and open-domain questions, which require access- ing distant conversation history. Table 5: Cross-model comparison on LoCoMo. Book- mark+Recall ranks first on all four models across three provider families. MethodGPTDSHaikuGLM Full Context2.022.581.051.84 Truncation1.642.56– BM25 Retrieval1.862.691.051.72 Word-Overlap1.882.69– Search-tool1.902.16– Bookmark+Recall2.182.741.472.23 GPT = GPT-4o-mini, DS = DeepSeek-v3.2, Haiku = Claude Haiku 4.5, GLM = GLM-5. “–” = not evaluated (top-3 methods only for Haiku/GLM). triggered (∼40% of cases), the overhead is zero— only the lightweight bookmarks are present in con- text. 4.4 Cross-Model Generalization To verify that cooperative paging generalizes across model families, we replicate the Lo- CoMo evaluation on four models spanning three providers (Table 5):GPT-4o-mini (OpenAI), DeepSeek-v3.2, Claude Haiku 4.5 (Anthropic), and GLM-5 (Zhipu). The full 6-method com- parison runs on GPT-4o-mini and DeepSeek- v3.2; for Claude Haiku and GLM-5, we com- pare the three most competitive methods (Book- mark+Recall, Full Context, BM25). The ranking is consistent: Bookmark+Recall ranks first on all four models.The advantage is largest on GLM-5 (+0.39 over Full Context) and smallest on Claude Haiku (+0.42 over tied baselines). Claude Haiku scores are uniformly low across all methods, suggesting that this model struggles with long conversational contexts in general, but Bookmark+Recall still provides the Table 6: Multi-judge validation on LoCoMo (100 probes). Each answer is scored independently by four judge models. Bookmark+Recall ranks first under ev- ery judge. Judge ModelFull CtxBM25Bookmark GPT-4o-mini1.831.792.11 DeepSeek-v3.22.111.982.23 Claude Haiku1.501.391.87 GLM-51.541.541.84 Cross-judge avg.1.751.682.01 largest uplift. On DeepSeek-v3.2, the advantage over the search-tool baseline is even larger (+0.58 vs. +0.28 on GPT-4o-mini), suggesting that the structured bookmark index is particularly helpful for models that are less likely to formulate effec- tive free-form search queries. 4.5 Multi-Judge Validation A potential concern with the LoCoMo evalua- tion is reliance on a single LLM judge. To test whether the ranking is robust to judge choice, we re-generate answers for the three most compet- itive methods (Bookmark+Recall, Full Context, BM25) on 100 probes and independently score each answer with four different judge models: GPT-4o-mini, DeepSeek-v3.2, Claude Haiku 4.5, and GLM-5. Table 6 shows the results. Bookmark+Recall ranks first under all four judges, with a cross-judge average of 2.01 vs. 1.75 (Full Context) and 1.68 (BM25).Paired bootstrap tests (B=10,000, scores averaged across judges) confirm the advantage: Bookmark+Recall vs. BM25, ∆= + 0.34, p=0.017; vs. Full Context, ∆= + 0.27, p=0.047. Inter-judge agreement is moderate to high: pairwise Pearson r ranges from 0.64 to 0.85 across the six judge pairs, indicating that the judges largely agree on which answers are good and which are poor, even though absolute score levels differ (Claude Haiku is the strictest judge, DeepSeek the most lenient). 5 Paging Design Space: Boundaries and Eviction Section 4 showed that cooperative paging works. But how to do the paging is itself a design ques- tion: where to place page boundaries, and which page to evict when the context is full. These two choices are orthogonal to bookmark content and were fixed heuristics in §4. This section studies them directly. We build a paging simulator that processes a conversation turn-by-turn, creates pages according to a boundary strategy, and evicts the oldest/least- useful page when the number of active pages ex- ceeds a fixed context budget (k=5 active pages). At query time, the model sees active pages in full plus bookmarks for all evicted pages and may call recall() as before. 5.1 Experimental Setup Data. We evaluate on two datasets: • Synthetic long conversations (20 conversa- tions, 120–200 turns each). Each conversation contains 6–8 planted facts from 8 categories (al- lergy, budget, contact, deadline, medical, num- ber, preference, schedule), with probe questions interspersed throughout and re-probed at the end. • LoCoMo (Maharana et al., 2024), where we concatenate all sessions of each multi-session conversation into a single continuous stream and place probe questions at the conversation end. This tests our paging design on real, human- generated dialogue. Strategies. We compare five page boundary strategies and four eviction policies in a full cross. Boundaries: fixed 5, fixed10, and fixed20 split every N turns; topicshift splits when the Jaccard word-overlap between the previous 5 turns and the new turn drops below 0.15; exchange 5 splits after five user–assistant exchanges. Eviction: FIFO (oldest first), LRU (evict the page whose content was least recently referenced by a probe or recall; updated at each probe), LFU (evict the page with the fewest cumu- lative references; ties broken by age), and B ́ el ́ ady (oracle: evict the page whose next use is furthest in the future). B ́ el ́ ady upper-bounds what any on- line policy can achieve (Bel ́ ady, 1966). Recalled pages are returned to the active set for one probe and then re-evicted. We report recall accuracy: a probe is correct if either (a) the needed page is still active at probe time, or (b) the model calls recall() with the correct page id. Full-grid runs total 3,176 probes on synthetic data and 1,600 probes on LoCoMo. 5.2 Result 1: Page Granularity Dominates Figure 4 and Table 7 show accuracy by boundary strategy, averaged across all four eviction policies. topic_ shift fixed (5) fixed (10) exch. (5) fixed (20) 0 20 40 60 80 100 Accuracy (%) Synthetic LoCoMo Figure 4: Page boundary ablation. Coarse fixed-size pages (fixed20) dominate on both synthetic and Lo- CoMo data. The “smart” topicshift strategy is worst because it over-fragments the conversation into 30+ small pages. Table 7: Page boundary ablation. fixed 20 is the best strategy on both synthetic conversations (20 convs, 3,176 probes) and real LoCoMo data (10 convs, 1,600 probes). The content-aware topic shift strategy is worst on both because it over-fragments into 30+ pages. BoundarySynth.LoCoMoAvg Pages topicshift56.7%45.9%30 fixed561.3%53.3%24 fixed 1077.0%50.9%12 exchange577.3%59.2%12 fixed 2096.7%63.9%6 The effect is striking:a simple fixed-size boundary of 20 turns achieves 96.7% on synthetic, while content-aware topic shift reaches only 56.7%—a 40-point gap.The mechanism is not that topic detection is unhelpful but that it over-fragments: the threshold triggers too of- ten, producing 30 pages on average versus 6 for fixed 20. More pages means more bookmarks, making it harder for the model to identify the right one to recall. Implication. For this workload, the dominant cost is not boundary coherence but bookmark search. Simple, coarse-grained pages beat sophis- ticated semantic splits because they keep the book- mark inventory small. 5.3 Result 2: Eviction Policies Have Room to Improve Figure 5 and Table 8 compare eviction policies, averaged across boundaries. B ́ el ́ ady’s oracle—which knows future probes— outperforms the best online policy on both LFULRUFIFO Bélády (oracle) 0 20 40 60 80 100 Accuracy (%) Synthetic LoCoMo Figure 5: Eviction policy ablation. B ́ el ́ ady’s oracle (rightmost, highlighted) upper-bounds online policies by 8–14 points, revealing headroom for smarter practi- cal policies. Table 8: Eviction policy ablation, averaged across boundaries. B ́ el ́ ady’s oracle beats the best online pol- icy on both datasets. Striking divergence: FIFO is the best online policy on synthetic data but the worst on LoCoMo, while LFU flips from worst on synthetic to second-best on LoCoMo. Eviction policy choice is data-dependent. PolicySynth.LoCoMo FIFO73.4%41.7% LRU68.6%52.8% LFU65.6%58.1% B ́ el ́ ady (oracle)87.7%66.0% B ́ el ́ ady – best online gap+14.3 p+7.9 p datasets: by 14.3 points on synthetic and 7.9 points on LoCoMo. This gap is an upper bound on any practical eviction policy and represents the head- room for smarter online strategies. The most striking finding is the inversion be- tween datasets. On synthetic data, FIFO is the best online policy and LFU is the worst; on LoCoMo, FIFO is the worst and LFU is the second-best. We attribute this to conversation topology (Figure 6). In our synthetic conversations, topics shift forward monotonically—each later topic has little connec- tion to earlier ones—so FIFO’s “age out old con- tent” heuristic aligns well with information utility. LoCoMo conversations, by contrast, are human- generated multi-session dialogues in which peo- ple routinely revisit earlier topics (shared friends, plans, past events). On such data, FIFO discards pages that will be accessed again, while LRU and LFU correctly preserve frequently-accessed early pages. Implication.There is no single “best” on- line eviction policy for conversational paging: the Table 9: Per-type accuracy on synthetic data, averaged across all 20 boundary×eviction configurations. Facts with distinctive keywords (allergy, number) are far more reliable than facts sharing generic domain to- kens (medical, schedule). Fact typeAccuracyn allergy93.9%478 number88.5%200 deadline83.4%380 contact82.4%398 budget74.3%420 preference73.6%360 schedule54.2%380 medical51.8%560 right choice depends on whether the conversation topology is forward-moving or revisit-heavy. A practical paging system should either (i) measure this property and switch policies, or (i) use a re- cency+frequency hybrid that handles both cases. The 8–14 point B ́ el ́ ady gap shows substantial headroom for such adaptive policies. 5.4 Result 3: The Bookmark Bottleneck When we look at why the model fails on evicted pages, a clear pattern emerges.Of the 1,828 probes in which the needed page was evicted, the model correctly triggered recall() in 96.3% of cases—the mechanism itself is reliable. But among those triggered recalls, only 56.6% se- lected the correct page. In other words, the model now knows when to recall, but often recalls the wrong page. This reveals that the primary remaining bottle- neck is bookmark discrimination: when multiple bookmarks share generic keywords (e.g., several pages mention “budget” or “medical”), the model cannot distinguish them from the bookmark alone. The effect is sharply non-uniform across informa- tion types (Table 9): facts with distinctive surface features (allergy, numerical values) reach 88– 94% accuracy, while facts sharing generic domain tokens (medical, schedule) drop to 52–54%. Section 6 addresses this directly through book- mark format and keyword specificity analyses. Best configuration. On synthetic data, the best combination is fixed 20 + B ́ el ́ ady at 98.7%. The best practical (non-oracle) combination is fixed 20 + LRU at 97.5%. Both dramatically simplify the paging problem: with only 6 pages on average and a 5-slot context budget, the system rarely needs to evict, and when it does, the large- Figure 6: Two conversation topologies explain the eviction-policy inversion. In forward-moving conversations (left), each topic is visited once and queries reference only recent turns, so FIFO’s age-based eviction matches access patterns. In revisit-heavy conversations (right, typical of real LoCoMo dialogues), topics like “friends” or “work” recur, and queries reference both old and recent content—making FIFO discard pages that will be accessed again while LRU/LFU correctly preserve them. Table 10: Cross-model paging results on synthetic data. The three headline findings—fixed 20 wins, topic shift loses, and B ́ el ́ ady exceeds online policies—hold on both GPT-4o-mini and DeepSeek- v3.2. DeepSeek is more robust overall: all online evic- tion policies fall within 2 points of each other. GPT-4o-miniDeepSeek-v3.2 By boundary (avg. over evictions) topicshift56.7%71.0% fixed561.3%77.6% fixed 1077.0%89.1% exchange577.3%88.4% fixed2096.7%98.7% By eviction (avg. over boundaries) FIFO73.4%83.6% LRU68.6%82.6% LFU65.6%84.5% B ́ el ́ ady87.7%89.2% page bookmarks carry enough information for re- liable recall. 5.5 Cross-Model Validation To verify that the paging design findings are not idiosyncratic to one model, we replicate the syn- thetic experiment on DeepSeek-v3.2 (5 conversa- tions, 780 probes). Table 10 shows the head-to- head comparison. All three headline findings hold: fixed 20 isthebestboundaryonbothmodels, topicshift is the worst, and B ́ el ́ ady upper- bounds online policies.DeepSeek achieves uniformly higher accuracy and, notably, the spread among online policies collapses to under 2 points—on DeepSeek, the choice of FIFO vs. 020406080100 Tokens per Bookmark 0 20 40 60 Recall Accuracy (%) ID only [p1] Minimal [p1:kw] Medium [p1:kw|"..."] Structured [p1:t=;e=] Figure 7: Bookmark format Pareto analysis (n=22 controlled probes). Minimal keywords (∼24 tokens) achieve the highest accuracy at the lowest token cost. LRU vs. LFU matters much less than on GPT-4o- mini. This is consistent with DeepSeek being a stronger model: it can compensate for mediocre eviction by more aggressively calling recall() to bring evicted pages back.The B ́ el ́ ady gap shrinks from 14.3 to 4.7 points, but remains non- zero: even strong models benefit from smarter eviction. 6 What Makes a Bookmark Work? 6.1 Format Ablation We test four bookmark formats ranging from 4 to 91 tokens per bookmark, evaluating recall accu- racy on the same 22 controlled probes (§4). Figure 7 and Table 11 show the results. On these 22 controlled probes, the minimal format ([pN:keywords]) achieves the highest accu- racy (63.6%) at the lowest token cost (24 tokens). Table 11: Bookmark format comparison (n=22 con- trolled probes).Minimal keyword format achieves highest accuracy at lowest cost (bold). FormatExampleAcc.Tok. ID only [p1]9.1%4 Minimal [p1:kw1,kw2]63.6%24 Medium [p1:kw|"text..."]54.5%91 Structured [p1:t=..;e=..]59.1%78 Table 12: Impact of keyword specificity. Domain- specific keywords improve recall accuracy by 25.7 per- centage points. Keyword TypeExampleAcc. Generic“personal preferences”65.2% Domain-specific“dietary pref., vegetarian”90.9% Improvement+25.7 p Richer formats—which add quoted text snippets or structured entity fields—use 3–4× more tokens but achieve lower accuracy. We hypothesize this occurs because longer bookmarks give the model a false sense of hav- ing sufficient information, reducing the likeli- hood that it calls recall(). Short bookmarks clearly signal “details are missing”—prompting the model to retrieve when needed. We call this the information-gap hypothesis: Figure 8 illus- trates the conjectured dynamic. We note that this explanation is post-hoc and correlational—the ob- served accuracy difference could also reflect con- founds such as prompt length, keyword overlap between formats, or model-specific prompt sensi- tivity. Controlled ablations isolating each factor would be needed to establish causality. 6.2 Keyword Specificity The most impactful design choice is keyword specificity. In our initial experiments, generic cat- egory labels (e.g., “personal preferences”) yielded 65.2% overall recall accuracy, with preference- type probes at 0%—the model never connected “personal preferences” to food or programming recommendations. After replacing generic labels with domain- specific keywords (e.g., “dietary preference, veg- etarian” or “programming language, Rust only”), accuracy jumped to 90.9%—a 25.7 percentage point improvement from keyword quality alone (Table 12). This finding suggests that bookmark generation should prioritize extracting consequence-relevant keywords—terms that help the model anticipate when the information will be needed—rather than abstract topic labels. 7 Closing the Bookmark Bottleneck Section 5 identified a sharp asymmetry in coop- erative paging: the model triggers recall() on 96.3% of evicted-page probes (the mecha- nism works) but only selects the correct page on 56.6% of those triggered recalls. We call this gap the bookmark bottleneck. In this section we ask whether better bookmark generation can close it, and we report the first systematic study of book- mark generation strategies for cooperative paging. 7.1 Strategies We compare six strategies that span the natural de- sign space, from a sanity-check baseline to LLM- driven generation: • random — four random non-stopword tokens from the page. Lower-bound baseline. • heuristic — the surface extractor used in §4: capitalised multi-character tokens, numerics, dollar amounts, and dates, filtered against a 60- word stopword list. • tfidf — top tokens by TF·IDF computed over the pages within the conversation, providing statis- tical discrimination. • llm-contextual — one LLM call per page. The prompt includes a one-line digest of every other page so the model can pick distinguishing facts. • llm-batch — a single LLM call covering all pages at once, with an explicit non-overlap con- straint across pages. • hybrid — start from the heuristic keywords and use a single LLM call to add one cross-page- discriminative keyword to each page. We hold the boundary and eviction policy fixed at fixed 10 + LRU (k=5 active pages)—a mid- range configuration from §5 that ensures meaning- ful eviction pressure (12 pages on average,∼58% evicted) while remaining representative of practi- cal deployments. (The overall best practical con- figuration fixed20 + LRU rarely evicts, mak- ing bookmark generation differences hard to ob- serve.) The only varying factor is the bookmark Figure 8: The information gap principle. A minimal bookmark (left) creates just enough uncertainty for the model to call recall() before answering, retrieving the full page and responding correctly. A rich bookmark (right) gives the same model a false sense of sufficiency, suppresses the recall call, and leads to a hallucinated answer. Paradoxically, more information in the bookmark yields worse end-to-end accuracy. generator. We evaluate on both synthetic conver- sations and real LoCoMo dialogues for a total of 956 (synth) + 480 (LoCoMo) probes. 1 7.2 Result 1: Two strategies improve over the baseline; four fail. Table 13 summarises end-to-end accuracy and re- call precision (the fraction of recall() calls that retrieve the needed page). Two findings stand out. First, na ̈ ıve sophistica- tion hurts: tfidf and llm-contextual both fall below the heuristic on both datasets, despite using comparable or larger token budgets. Sec- ond, two strategies do beat the heuristic, and they win on different data: hybrid improves the syn- thetic configuration by +4.4 E2E points and +4.5 recall-precision points, while llm-batch improves LoCoMo by +8.7 E2E points and +6.7 recall- precision points. Both improvements come at less than 10% extra bookmark token cost. 1 We additionally replicate every result under maximum eviction pressure (fixed 5; 24 pages on average, ∼85% evicted) in Appendix B. Table 13: Bookmark generation ablation. We rule out tfidf and llm-contextual (which under-perform the heuristic) and identify two complementary winners: hybrid for synthetic data and llm-batch for real Lo- CoMo. “∆ vs. heuristic” shows the improvement of the best strategy on each dataset. SyntheticLoCoMo StrategyE2EPrec.E2EPrec. random50.932.538.818.5 heuristic72.364.352.637.7 tfidf62.345.147.529.1 llm-contextual66.056.638.826.3 llm-batch68.661.561.344.4 hybrid76.768.845.028.3 ∆ vs. heuristic+4.4+4.5+8.7+6.7 7.3 Result 2: The improvement is concentrated in open-domain questions. Breaking the LoCoMo result down by question category reveals where the gain comes from (Ta- ble 14). The largest gap is on open-domain ques- tions (cat4): llm-batch achieves 100% accuracy versus 50% for the heuristic baseline—a +50 point improvement on the question type that re- quires the most cross-page reasoning. Table 14: Bookmark strategy by LoCoMo question cat- egory. llm-batch dominates on every category, with the most dramatic improvement on open-domain questions (cat4: +50 points). Categoryheuristicllm-batch∆ cat1 (single-hop)61%67%+6 cat2 (temporal)55%64%+9 cat3 (multi-hop)25%33%+8 cat4 (open-domain)50%100%+50 7.4 Result 3: The information-gap hypothesis re-emerges. The two strategies that fail (tfidf and llm- contextual) share an unexpected property: their recall trigger rates are 96.2% and 93.0% respec- tively, lower than the heuristic baseline’s 97.5%. By packing more or longer tokens into bookmarks, they nudge the model into “I have enough” mode and suppress the recall call entirely. This is the exact failure mode predicted by the information- gap hypothesis of §6, now observed at the level of bookmark generation strategy rather than format. The two successful strategies (hybrid, llm-batch) are precisely those that preserve the trigger rate (97.3% and 93.2%) while improving discrimina- tion among already-triggered recalls. The implication is structural: better book- marks can only improve cooperative pag- ing if they preserve the model’s incentive to recall.Strategies that summarise pages too informatively—no matter how accurate the summary—defeat the mechanism. 7.5 Result 4: A principled, data-aware selection rule. Why does hybrid win on synthetic data while llm-batch wins on LoCoMo? The split is princi- pled, not noise. Synthetic conversations contain planted facts whose surface forms (allergens, dol- lar amounts, named contacts) are exactly the tar- gets of the heuristic’s domain extractor; the LLM’s role is therefore discrimination—adding one key- word that distinguishes near-duplicates while leav- ing the strong heuristic base intact. LoCoMo con- versations contain no such surface anchors: the heuristic base degenerates into generic chat tokens (“Dave”, “Wow”), so adding a single discrimi- native keyword cannot rescue it; the model must regenerate keywords from scratch under cross- page non-overlap, which is exactly what llm-batch does. This suggests a simple selection rule: use hy- brid when the heuristic base is informative; use llm-batch when it is not. A practical proxy is the average TF·IDF mass of heuristic-extracted tokens—when this is high, hybrid wins; when it is low, llm-batch wins. Applied to our two datasets, this rule yields the best per-dataset strat- egy and improves average E2E accuracy from 62.5% (heuristic) to 69.0% (+6.5 points). 7.6 Discussion What we closed and what remains. On real LoCoMo data, our best strategy raises end-to-end accuracy from 52.6% to 61.3% and recall preci- sion from 37.7% to 44.4%. This is a meaningful step but not a complete solution: the remaining gap to perfect bookmark precision is still large, and we conclude that bookmark-level changes alone cannot close it. Future directions. The structural lesson from Result 3—that any improvement must preserve the recall incentive—suggests two promising direc- tions that remain orthogonal to bookmark content. First, query-time reranking: when the model trig- gers recall, the system can rerank the bookmark inventory by query-bookmark embedding similar- ity before resolving the call, which decouples dis- crimination from generation. Second, multi-stage recall: the model first picks a coarse topic, the system narrows to candidate pages, and the model commits in a second pass. We leave both to future work. Take-away. The bookmark bottleneck is real, partially solvable, and fundamentally constrained by the information-gap hypothesis. Practitioners should choose their bookmark generator based on whether their data exposes distinctive surface to- kens; researchers should look beyond bookmark content for the next factor of improvement. 8 Discussion and Conclusion Why cooperative paging works. LLMs, un- like traditional OS applications, are cooperative agents: they follow instructions and use tools when appropriately prompted. Mason (2026) ob- served that LLMs have incentive to cooperate with memory management because cleaner context im- proves output quality. Our bookmark mechanism exploits this: by providing a lightweight table of contents for compressed memory, we enable the model to make informed retrieval decisions with- out requiring it to guess what information might be missing. The key insight is that a minimal cue (“food allergy, peanut”) is sufficient—the model can infer when this information is relevant to a food recommendation, even when the user does not explicitly mention allergies. Bookmark design matters more than mecha- nism complexity. Our format ablation reveals a counterintuitive finding: richer bookmarks de- crease recall accuracy despite using 3–4× more tokens. We hypothesize that verbose stubs give the model a false sense of having sufficient informa- tion, reducing tool usage. This suggests that mem- ory management systems should invest in key- word quality (specificity, domain relevance) rather than keyword quantity. The 25.7p improvement from switching generic to domain-specific key- words underscores this: the choice of which key- words to include matters far more than how many details to pack into each stub. Page granularity matters more than “smart” boundaries. The paging design study (§5) pro- duces a related lesson at the structural level. Content-aware topic shift boundaries—the natural first instinct—are the worst performing strategy (56.7%), while coarse fixed-size pages (fixed20) achieve 96.7%. The reason is not that topic detection is bad in principle, but that bookmarks carry a search cost for the model: ev- ery additional bookmark dilutes the model’s atten- tion over the compressed history. Under a tight context budget, the system should minimize the bookmark inventory, even at the cost of imperfect topical coherence within each page. The bottleneck has shifted to bookmark dis- crimination. A striking by-product of the pag- ing ablation is that recall() triggering is now essentially solved (96.3% triggered when needed), but page selection among triggered recalls is only 56.6% correct. The remaining errors are almost entirely cases where multiple bookmarks share generic tokens (“budget,” “medical,” a date) and the model picks one indistinguishable from an- other. This closes a loop with §6: making key- words more specific is not only a bookmark- format optimization, but the primary path to fur- ther improvement in end-to-end paging quality. Limitations. Answer quality on LoCoMo is scored by LLM judges without human evaluation; while our multi-judge validation (§4) confirms that the ranking holds across four independent mod- els with a statistically significant advantage over BM25 (p=0.017), human evaluation would fur- ther strengthen confidence. Our experiments pri- marily use GPT-4o-mini, with DeepSeek-v3.2 as a secondary model; broader cross-model valida- tion would strengthen generalizability claims. The paging design study uses 20 template-based long conversations plus 10 real LoCoMo conversations; a larger corpus of real long-horizon dialogues would further strengthen the boundary/eviction re- sults. Our topic shift boundary uses simple word-overlap; a stronger semantic detector (e.g., embedding similarity) might narrow but likely not close the gap to fixed 20 given that the under- lying cost is bookmark inventory size. Keyword extraction is currently heuristic-based; learned ex- traction is an important direction, especially given that bookmark discrimination is the new bottle- neck. Finally, the approach depends on the model cooperating by calling recall()—if the model ignores bookmarks and answers directly, evicted information remains inaccessible. Future work. Key directions include:(1) learned keyword extraction that produces dis- criminative keywords conditioned on likely fu- ture queries—directly targeting the 57% page- selection bottleneck; (2) online eviction policies that close the 14-point B ́ el ́ ady gap, possibly us- ing lightweight predictors of future access; (3) adaptive page sizing that grows or shrinks pages based on observed access locality; and (4) scaling to very long conversations (100+ sessions) where bookmark-level indexing may need hierarchical organization. Conclusion. We proposed cooperative paging with minimal keyword bookmarks for LLM con- versation memory. On the LoCoMo benchmark with real multi-session conversations, bookmark paging achieves the highest score among six meth- ods on both GPT-4o-mini and DeepSeek-v3.2, a ranking confirmed by four independent LLM judges with statistical significance (p=0.017 vs. BM25). A format ablation on controlled probes suggests that minimal bookmarks (∼8–24 tokens) achieve the best accuracy-to-cost ratio, with key- word specificity as the most impactful design fac- tor (+25.7p). These findings suggest that effec- tive conversation memory management requires not complex retrieval systems, but well-designed minimal cues that enable model-driven coopera- tive retrieval. References L ́ aszl ́ o A Bel ́ ady. 1966. A study of replacement algo- rithms for a virtual-storage computer. IBM Systems Journal, 5(2):78–101. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready AI agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Christopher Cruz. 2025. Adaptive focus memory for language models. arXiv preprint arXiv:2511.12712. Peter J Denning. 1968. The working set model for program behavior. Communications of the ACM, 11(5):323–333. Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lam- pouras, Haitham Bou-Ammar, and Jun Wang. 2025. Human-inspired episodic memory for infinite con- text LLMs. In International Conference on Learn- ing Representations. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dong- sheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. LongLLMLingua: Accelerating and enhanc- ing LLMs in long context scenarios via prompt com- pression. arXiv preprint arXiv:2310.06839. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E Gonzalez, Hao Zhang, and Ion Stoica. 2023. Effi- cient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles. Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. 2024. A human-inspired reading agent with gist memory of very long con- texts. In Proceedings of the 41st International Con- ference on Machine Learning. Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance infer- ence efficiency of large language models.arXiv preprint arXiv:2310.06201. Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024.Evaluating very long-term conversational memory of LLM agents.In Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics. Tony Mason. 2026. The missing memory hierarchy: Demand paging for LLM context windows. arXiv preprint arXiv:2603.09023. Tsendsuren Munkhdalai, Manaal Faruqui, and Sid- dharth Gopal. 2024.Leave no context behind: Efficient infinite context transformers with infini- attention. arXiv preprint arXiv:2404.07143. Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, Ion Stoica, and Joseph E Gon- zalez. 2023. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Xufang Luo, Hao Cheng, Dongsheng Li, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Jian- feng Gao. 2025. On memory construction and re- trieval for personalized conversational agents. arXiv preprint arXiv:2502.05589. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In Interna- tional Conference on Learning Representations. Wujiang Xu, Zujie Liang, Kai Mei, and 1 others. 2025. A-MEM: Agentic memory for LLM agents. arXiv preprint arXiv:2502.12110. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. MemoryBank: Enhancing large language models with long-term memory. In Pro- ceedings of the AAAI Conference on Artificial Intel- ligence, volume 38. A Reproducibility Details Model. The primary model is gpt-4o-mini via OpenAI-compatible API. Temperature = 0.0 for deterministic outputs. logprobs=true for the NLL pilot study.Cross-model val- idation experiments (§4, §5) additionally use deepseek-v3.2 with the same temperature and tool-calling setup. Segmentation. For the synthetic paging-design experiments (§5, §7), conversations are divided into pages of N turns each (where N varies by boundary strategy). For the LoCoMo benchmark (§4), each multi-session conversation is treated at session granularity: each session becomes one page, and eviction operates over sessions. No overlap between pages/sessions. Bookmark syntax. Each bookmark follows the format: [SN(date):kw1,kw2,...] for Lo- CoMo sessions or [pN:kw1,kw2,kw3] for synthetic pages. Keywords are extracted by se- lecting capitalized multi-character tokens from the first 3–4 turns, filtered against a 60-word stopword list, limited to 3–6 keywords. System prompt. The exact system prompt used for all bookmark experiments: “You are a helpful assistant. Some earlier con- versation has been compressed into bookmarks like [SN(date):keywords] or [pN:keywords]. If you need specific details from a bookmark, call recall(session ids=[N]) first. Do NOT guess de- tails you don’t have.” Recall tool definition. A single function recall(pageids) is registered via the Ope- nAI tool-calling API. It accepts a list of integer page IDs and returns their full content. Evaluation scoring. Answer quality is judged by GPT-4o-mini with the prompt: “Rate how well the answer addresses the question given the ground truth. Score 1–5.” Scores are parsed from JSON output. No human judges were used. LoCoMo setup. We use the LoCoMo-10 subset (Maharana et al., 2024). For each conversation, all sessions except the last 3 are compressed to book- marks. Recent sessions are included as full turns (last 20 turns if exceeding context). Up to 10 QA pairs are sampled per conversation (stratified by category), totaling 98 probes for the main evalua- tion and 100 for the multi-judge validation (slight variation due to stratification). Context budgets. Full context truncates at 60 turns. Truncation baseline keeps last 20 turns. Bookmark+recall stores evicted content externally and injects it on demand via recall(); all book- marks fit in∼200 tokens, so the in-context cost is comparable to the truncation baseline plus book- mark overhead. When recall is triggered, the re- trieved page temporarily expands the context; this asymmetry is a design feature of the mechanism (external storage enables reversibility) rather than an evaluation advantage, as all baselines have ac- cess to the same underlying content. Baseline configurations. All baselines use the same model, temperature (0.0), and max output to- kens (200). BM25 and word-overlap retrieval re- trieve the top-3 evicted sessions ranked by query- document similarity.The search-tool baseline uses the same BM25 ranking as backend but gives the model a free-form memory search tool in- stead of keyword bookmarks. Full context in- cludes as many recent turns as possible (up to 60). All baselines receive the same 3 most recent ses- sions as full context; only the treatment of older sessions differs. Table 15: Full boundary×eviction accuracy grid on synthetic data (GPT-4o-mini, 3,176 probes). Each cell averages over 159 probes. BoundaryFIFOLRULFUB ́ el ́ ady fixed559.152.853.579.9 fixed1079.972.362.393.1 fixed2096.997.593.798.7 topic shift50.947.852.275.5 exchange579.272.365.491.2 Bookmark generation pipeline. Keywords are extracted automatically: we scan the first 3–4 turns of each session, collect capitalized multi- character tokens not in a 60-word stopword list, and keep the first 5–6 unique matches. No man- ual curation, LLM calls, or domain-specific rules are used. The same pipeline runs identically for all conversation types. B Stress Test: Maximum Eviction Pressure Section 5 reported the paging design ablation av- eraged across all boundary×eviction configura- tions. Here we examine the extreme end of the de- sign space: fixed 5 page boundaries, which pro- duce 24 pages on average and an eviction rate of 79.4%, creating maximum pressure on the book- mark mechanism. Full grid. Table 15 shows the complete 5×4 boundary×eviction accuracy grid on synthetic data.Figure 9 visualises the same data as a heatmap. The gradient confirms that page granu- larity (vertical axis) is the dominant factor: the top row (fixed 20) is uniformly high regardless of eviction policy, while the bottom rows show large variation. Per-type breakdown under stress. Under fixed5 boundaries (636 probes, 79.4% evic- tion rate), accuracy varies sharply by fact type (Table 16). Facts with distinctive surface tokens (allergy: 92.7%, number: 82.5%) remain ro- bust even under extreme eviction pressure, while facts sharing generic domain tokens (medical: 33.0%, schedule: 28.9%) collapse. This con- firms the bookmark discrimination bottleneck identified in §5: under stress, only facts whose bookmarks are naturally distinctive survive. Bookmark strategies under stress. Table 17 replicates the bookmark generation ablation from §7 under fixed 5 boundaries (k=5 active pages, FIFOLRULFU Bélády fixed_5 fixed_10 fixed_20 topic_shift exchange_5 59535380 80726393 97979499 51485375 80726591 40 50 60 70 80 90 100 Accuracy (%) Figure 9: Heatmap of boundary×eviction accuracy. Page granularity (rows) dominates eviction policy (columns): fixed20 is uniformly high regardless of policy. 159 probes per strategy). All strategies degrade compared to the fixed 10 results in the main text, but the ranking is preserved: hybrid remains the best strategy (54.7%), and random remains worst (22.6%). Table 16: Per-type accuracy under maximum eviction pressure (fixed 5, 636 probes, 79.4% eviction rate). Fact typeAccuracyn allergy92.7%96 number82.5%40 deadline76.3%76 contact75.0%80 budget63.1%84 preference52.8%72 medical33.0%112 schedule28.9%76 Table 17: Bookmark strategies under maximum evic- tion pressure (fixed 5, k=5, 159 probes each). StrategyE2E Accuracy random22.6% tfidf32.1% llm contextual42.8% llm batch42.8% heuristic50.3% hybrid54.7%

Instant research discovery