Paper deep dive

MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents

Jiayi Chen, Yingcong Li, Guiling Wang

Year: 2026Venue: arXiv preprintArea: cs.MAType: PreprintEmbeddings: 84

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%

Last extracted: 7/8/2026, 9:32:11 AM

Summary

MemFlow is a training-free, intent-driven memory orchestration framework designed for Small Language Models (SLMs) to handle long-horizon, multi-turn interactions. It externalizes memory planning via a Router Agent that classifies queries and dispatches them to a Memory Agent, which executes one of three specialized tiers (Profile Lookup, Targeted Retrieval, or Deep Reasoning). An Answer Agent generates responses from compact, tier-aware contexts, while a Validator Agent checks grounding and triggers structured escalation if needed. Evaluated on LongMemEval, LoCoMo, and LongBench using a frozen Qwen3-1.7B backbone, MemFlow nearly doubles accuracy over full-context SLM baselines while maintaining a compact answer context (~2,223 tokens).

Entities (12)

MemFlow · framework · 99%Qwen3-1.7B · model · 97%Small Language Models · concept · 96%Memory Agent · component · 95%Router Agent · component · 95%Answer Agent · component · 94%Validator Agent · component · 94%LoCoMo · benchmark · 93%LongBench · benchmark · 93%LongMemEval · benchmark · 93%RAG · method · 92%ReAct · method · 92%

Relation Signals (16)

MemFlow → uses → Router Agent

confidence 98% · A Router Agent classifies each query by intent and dispatches it to the Memory Agent

MemFlow → uses → Memory Agent

confidence 98% · dispatches it to the Memory Agent, which executes one of three specialized tiers

MemFlow → backbone → Qwen3-1.7B

confidence 97% · Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks

MemFlow → uses → Answer Agent

confidence 97% · An Answer Agent then generates a response from this compact context

MemFlow → uses → Validator Agent

confidence 97% · and a Validator Agent optionally retries with a heavier memory tier when the response is not supported

MemFlow → evaluatedon → LoCoMo

confidence 96% · Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks - LongMemEval, LoCoMo, and LongBench

MemFlow → evaluatedon → LongMemEval

confidence 96% · Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks - LongMemEval, LoCoMo, and LongBench

MemFlow → evaluatedon → LongBench

confidence 96% · Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks - LongMemEval, LoCoMo, and LongBench

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Modern language agents must operate over long-horizon, multi-turn histories, yet deploying such agents with Small Language Models (SLMs) remains fundamentally difficult. Full-context prompting causes context overflow, flat retrieval exposes the model to noisy evidence, and open-ended agentic loops are unreliable under limited reasoning capacity. We argue that a substantial portion of SLM memory failure arises from mismatched memory operations: different query types demand categorically different retrieval strategies, evidence transformations, and context budgets that SLMs cannot reliably self-orchestrate through open-ended reasoning. We introduce MemFlow, a training-free memory orchestration framework that externalizes memory planning from the SLM. A Router Agent classifies each query by intent and dispatches it to the Memory Agent, which executes one of three specialized tiers (Profile Lookup, Targeted Retrieval, or Deep Reasoning) and assembles the resulting evidence under a dynamic, tier-aware token budget. An Answer Agent then generates a response from this compact context, and a Validator Agent optionally retries with a heavier memory tier when the response is not supported by the provided evidence. This route-then-compile design avoids tool-selection hallucination and reasoning loops while keeping the answer context compact. Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks - LongMemEval, LoCoMo, and LongBench - MemFlow improves accuracy by nearly 2x over full-context SLM baselines. These results suggest that structured intent routing and deterministic evidence preparation can make limited-capacity models substantially more effective in resource-constrained long-horizon agents.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

83,761 characters extracted from source content.

Expand or collapse full text

MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents Jiayi ChenYingcong LiGuiling Wang New Jersey Institute of Technology, Newark, USA jc2693, yingcong.li, guiling.wang@njit.edu Abstract Modern language agents must operate over long-horizon, multi-turn histories, yet deploying such agents with Small Language Models (SLMs) remains fundamen- tally difficult. Full-context prompting causes context overflow, flat retrieval exposes the model to noisy evidence, and open-ended agentic loops are unreliable under limited reasoning capacity. We argue that a substantial portion of SLM memory failure arises from mismatched memory operations: different query types demand categorically different retrieval strategies, evidence transformations, and context budgets that SLMs cannot reliably self-orchestrate through open-ended reason- ing. We introduce MemFlow, a training-free memory orchestration framework that externalizes memory planning from the SLM. A Router Agent classifies each query by intent and dispatches it to the Memory Agent, which executes one of three specialized tiers (Profile Lookup, Targeted Retrieval, or Deep Reasoning) and assembles the resulting evidence under a dynamic, tier-aware token budget. An Answer Agent then generates a response from this compact context, and a Validator Agent optionally retries with a heavier memory tier when the response is not supported by the provided evidence. This route-then-compile design avoids tool-selection hallucination and reasoning loops while keeping the answer con- text compact. Evaluated on a frozen Qwen3-1.7B backbone across long-horizon memory benchmarks—LongMemEval, LoCoMo, and LongBench—MemFlow improves accuracy by nearly2×over full-context SLM baselines. These results suggest that structured intent routing and deterministic evidence preparation can make limited-capacity models substantially more effective in resource-constrained long-horizon agents. 1 Introduction Language agents operating over long-horizon interactions must answer queries whose evidence spans many turns, sessions, and topic shifts [43,25,30,32]. The common strategy of appending the full history causes unbounded memory growth, high cost, and degraded reasoning once inputs exceed the model’s effective attention span [23]. Frontier LLMs partly absorb this cost with scale and extended context windows [29,2,14], but Small Language Models (SLMs) face hard context limits in resource-constrained deployments [1, 24, 35, 6, 52]. RAG [22,21] mitigates overflow by selecting evidence, but uniform retrieval cannot serve the structural diversity of long-horizon queries [43,4]: preferences, timelines, knowledge updates, and multi-session synthesis require different retrieval strategies, transformations, and budgets. ReAct- style reasoning [46] and learned tool use [40] provide flexibility, but sub-3B agents frequently produce hallucinated calls or broken reasoning traces [12]. We therefore frame a substantial portion of SLM memory failure as an intent-routing mismatch: for SLM agents, the central question is not only what to retrieve, but which memory operation the query requires. Preprint. arXiv:2605.03312v1 [cs.MA] 5 May 2026 Figure 1: Comparison of existing SLM memory approaches and MemFlow. Left: existing agents suffer from context explosion, hallucination, and lost-in-the-middle failures when full histories or uniform retrieval are used with open-ended reasoning loops. Right: MemFlow addresses these failure modes through intent-driven routing to specialized memory tiers, priority-aware context compilation under a dynamic tier-aware token budget, and grounding-validated escalation. Prior work addresses pieces of this problem. Prompt compressors such as LongLLMLingua [19] and LLMLingua-2 [31] reduce context but are task-agnostic; Self-RAG [3] and CRAG [45] add critique but still delegate orchestration to the model. Memory systems such as MemGPT [30], Mem0 [8], and Zep [36] provide external stores, while routing methods [27,37] allocate compute across models or subgoals. MEM1 [53] learns compact memory end-to-end, but requires task-specific reinforcement learning. A training-free framework that jointly handles intent classification, retrieval specialization, context budgeting, and grounding validation for frozen SLMs remains missing. We introduce MemFlow, a training-free, intent-driven memory orchestration framework for frozen SLM agents (Figure 1). A Router Agent classifies each query and dispatches it to a Memory Agent tier: direct profile lookup, targeted retrieval, or deep reasoning with deterministic preprocessing for temporal sorting, conflict resolution, and synthesis. The Memory Agent packs evidence under a dynamic, priority-aware token budget; an Answer Agent generates from this compact context; and a Validator Agent triggers heavier-tier recovery on grounding failure. This route-then-compile design reduces open-ended orchestration: after routing, execution follows a fixed memory path and the answer model sees intent-matched evidence rather than raw history. Across LongMemEval [43], LoCoMo [25], and LongBench [4], MemFlow nearly doubles full-context accuracy with a frozen Qwen3-1.7B [35] and improves over RAG/ReAct by+6.2p while keeping the final answer context compact. Because the full orchestration pipeline is costlier than the final context alone, we report both full-pipeline cost and answer-context size. MemFlow also improves additional sub-3B backbones— Qwen3-0.6B [35], SmolLM2-1.7B [6], LLaMA-3.2-1B [15], and Gemma-3-1B [13]—suggesting that structured memory routing can substitute for some missing model capacity without training. 2 Related Work Retrieval, compression, and evidence preparation. RAG [22,21,16] established the retrieved- memory paradigm; FiD [18], Contriever [17], and ColBERTv2 [38] improved passage fusion and retrieval precision. Later systems refine retrieval decisions through reasoning state or critique, including IRCoT [42], Active RAG [20], Self-RAG [3], CRAG [45], and RAFT [49]; hierarchical and ranking methods such as RAPTOR [39], GraphRAG [11], and RankRAG [47] improve synthesis 2 Validator Agent User Query Router Agent Tier 1: Profile Lookup Tier 2: Targeted Retrieval Tier 3: Deep Reasoning Packer Answer Agent Final Answer SLM SLM SLM PASS Retry with heavier tier Memory Agent FAIL Figure 2: Overview of the MemFlow pipeline. SLM chip icons denote SLM inference points. under limited budgets. Compression and long-context studies show why selective evidence matters: attention underuses middle-positioned evidence [23]; window-extension methods reduce attention cost [5,48,9,50]; and LongLLMLingua [19] and LLMLingua-2 [31] compress prompts. These systems improve retrieval or compression, but generally apply policies independent of the memory operation a query structurally requires. Memory-augmented language agents. Generative Agents [32] established persistent memory and reflection for long-horizon agents. MemGPT [30] uses OS-style virtual memory paging, Mem0 [8] uses a dual vector/graph store, and Zep [36] adds temporal graph structure, but these systems still rely on model-led memory access or similarity-driven retrieval. Reflexion [41] stores episodic failure traces for self-improvement, while MEM1 [53] learns a compact memory state through reinforcement learning. Recent memory-selection systems also use typed or intent-aware memory, including ENGRAM’s lightweight typed stores [33] and MemGuide’s intent-aligned multi-session dialogue retrieval [10]. MemFlow instead targets frozen SLMs with bounded route-then-compile execution that couples intent routing, deterministic evidence preparation, and grounding validation. Agentic orchestration, tool reliability, and routing. Open-ended orchestration remains brittle: AutoGen [44] scales multi-agent reasoning, WebArena [51] exposes stability failures, and tool-use studies report hallucinated calls and mismatched arguments [34,12]. Routing systems such as FrugalGPT [7], RouteLLM [27], System-1.x [37], and homogeneous-tool query routing for RAG [26] separate dispatch from solving, but mainly route across models, tools, or planning modes. MemFlow applies routing to the memory operations themselves, coupling intent classification with query-type- aware retrieval, context compilation, and grounding validation without additional model training. 3 MemFlow The central premise of MemFlow is that long-horizon memory is not one problem but a family of structurally distinct problems. A query asking for a user’s dietary preference needs no retrieval at all; a query about the chronological order of two events requires date arithmetic over retrieved evidence; a query about the most recent rule governing a constraint needs post-filtering for policy language. Applying a single retrieval strategy uniformly across these cases systematically fails on the cases it was not designed for. MemFlow addresses this by treating memory as an intent-conditioned orchestration problem and externalizing all memory decisions from the SLM into a structured multi- agent pipeline composed of four components: a Router Agent that classifies each query by intent, a Memory Agent that executes the corresponding retrieval and evidence compilation, an Answer Agent that generates a grounded response, and a Validator Agent that checks grounding quality and triggers structured recovery on failure (Figure 2). Formally, given a conversation historyHand a queryq, the goal is to produce an answer faithful to the evidence inH. For SLMs with effective context windows of 2–4K tokens,|H|vastly exceeds what can be reliably ingested at once. MemFlow decomposes the problem into three decisions: what memory operation doesqrequire? (intent routing), what evidence should the SLM see? (retrieval, 3 preprocessing, and packing), and is the answer grounded? (validation and escalation). This defines MemFlow as a bounded memory-control policy rather than an open-ended agent loop. LetObe the finite set of typed memory operations; each operation specifies a retrieval mode, deterministic evidence transformation, packing budget, answer prompt, and validation or escalation rule. The Router Agent estimatesπ(q) ∈ O, after which execution is fixed by the selected operation. Thus, unlike model or tool routing, MemFlow does not choose among arbitrary tools, models, or free-form plans, but selects a memory operation whose downstream computation is deterministic. Crucially, the SLM drives only the intent-classification, response-generation, and grounding-validation stages; the Memory Agent operates entirely outside the SLM, free from SLM-driven tool selection. The full pipeline executes with at most four SLM calls per query, with per-stage token statistics reported in Appendix A.1. 3.1 Router Agent and Memory Agent The separation between routing and execution is MemFlow’s core structural choice. Rather than asking the SLM to select tools during generation, the Router Agent determines what to do and the Memory Agent executes how. This separation avoids open-ended model-driven tool selection: once a tag is assigned, the execution path is fixed. Our ablation validates this design principle: disabling tag-specific retrieval and preprocessing produces the single largest accuracy drop in the system, costing 18.7 percentage points (Table 3). Router Agent. The Router Agent classifiesqinto one of seven typed memory operations(profile-injection,targeted-extraction,temporal-reasoning, conflict-resolution,broad-summarization,constraint-validation, state-tracking) via a three-layer cascade (full prompt and tag taxonomy in Appendix B.1): a rule layer fires first on unambiguous intents; if no rule matches, a single SLM call classifies the query; if the output fails to parse, keyword heuristics serve as a final fallback. The cascade achieves 87.7% routing accuracy and reduces hard routing failures caused by malformed SLM outputs. Critically, the router has no knowledge of downstream retrieval mechanisms or tools: its sole output is an intent tag, ensuring that all downstream dispatch remains deterministic and less exposed to the hallucinated tool calls that afflict open-ended tool-selection at sub-3B scale [12, 46]. Memory Agent.The deterministic Memory Agent executes retrieval and preprocessing appropriate to the action tag, then compiles the result into a token-budgeted context. It is organized into three tiers, each corresponding to a distinct retrieval regime: zero-retrieval for stable facts, standard retrieval for factual lookups, and retrieval with deterministic preprocessing for complex multi-step queries. Together, the three tiers partition all seven action tags into exhaustive, non-overlapping execution paths [27, 37, 7]. Tier 1 (Profile Lookup) handlesprofile-injectionqueries without any retrieval. Counterintu- itively, for preference and identity facts that are stable across sessions, retrieval is counterproductive: it risks surfacing conflicting or outdated phrasings of the same preference. Instead, MemFlow pre-compiles these facts into a structured user profile during ingestion and loads it directly. Tier 2 (Targeted Retrieval) handlestargeted-extractionqueries via multi-pass entity-aware retrieval. A primary hybrid BM25+dense pass [21,38] retrieves top-kchunks for the full query; secondary passes retrieve separately for each extracted entity, handling queries that reference multiple subjects (full hyperparameters in Appendix A.2). Tier 3 (Deep Reasoning) handles the five remaining tags, each requiring deterministic preprocessing before the SLM sees any evidence. The key insight is that SLMs at sub-3B scale cannot reliably perform operations like chronological sorting, date arithmetic, or rule filtering on raw retrieved text [23,12]. MemFlow therefore completes these computations prior to any SLM inference, as specified per-tag in Appendix A.2: temporal queries trigger a Chronological Sorter and Date Math Calculator; conflict queries apply a Recency Filter that marks stale facts; summarization queries enforce a session-diversity guarantee to prevent redundancy. After tier execution, the Memory Agent assembles outputs into a priority-ordered, token-budgeted context: pinned facts are never truncated, pre-computed summaries are reduced on overflow, and raw chunks are truncated last. The result averages 2,223 tokens overall — a fraction of the full history — with allocation tables broken down by tag and benchmark in Appendix A.3. 4 Table 1: Main results across three long-context benchmarks. Answer Ctx = tokens fed to the answer agent. All accuracies are judged by GPT-4o-mini. Best SLM result per column in bold. SystemLongMemEval (%)LoCoMo (%)LongBench (%)AVG (%)Answer Ctx SLM Baselines Direct QA (short)34.224.014.621.32,458 Direct QA (full)42.233.719.829.08,614 RAG46.045.946.746.22,266 ReAct50.249.641.246.28,732 Memory Systems Memobase13.47.710.79.6∼260 † MemGPT38.831.627.830.9∼1,260 † Ours MemFlow61.851.251.152.42,223 Frontier Reference GPT-4o (matched)67.856.664.761.39,800 ∗ GPT-4o (ceiling)68.476.373.774.3120,000 ∗ All SLM Baselines and Memory Systems use Qwen3-1.7B as the backbone. † Estimated from retrieval metadata. ∗ Token budget cap, not measured. N =4,236 questions across three benchmarks. 3.2 Answer Agent and Validator Agent Answer Agent. Evidence compilation and response generation are separated into distinct stages because the Memory Agent’s output is structured specifically for the query type, and exploiting that structure requires a matching prompt rather than a generic one. The Answer Agent generates a response from the packed context via a single SLM call, with a tag-specific system prompt (all seven templates in Appendix B.2): temporal queries receive a chronological framing that cues the model to narrate a timeline; profile queries receive a preference-injection framing; extraction queries receive a grounded-recall framing. For temporal and summarization queries, the agent may additionally emit a boundedTOOL:call (e.g.,days_between), which is executed deterministically and injected as a TOOL_RESULT — the model cannot select arbitrary tools or enter open-ended loops. Validator Agent. Even with compact, well-organized context, SLMs at sub-3B scale can halluci- nate [23]. Prior work shows that critic-guided verification can substantially reduce such errors [3,45]; the Validator Agent applies this principle without any additional training. It operates in three stages that are intentionally ordered from cheapest to most expensive: (1) hard-failure detection (de- terministic, zero cost) triggers immediate escalation on empty answers or explicit abstentions; (2) short-answer passthrough (deterministic) skips grounding checks for answers of six words or fewer; (3) a lightweight SLM grounding judge (conditional) outputs a binary yes/no on whether the answer is supported by the context (at most 8 new tokens, no chain-of-thought). The cascade ensures that expensive LLM inference is invoked only when necessary. When validation fails, an escalation loop re-routes the query to a heavier tier and regenerates according to a policy-driven retry schedule described in Appendix E.4. In practice, escalation is invoked on only 14.9% of queries, and only 3.4% ultimately adopt the escalated response, indicating that most queries are handled on the first pass while a smaller subset benefits from structured recovery. 4 Experiments & Results We evaluate MemFlow along two axes: accuracy on long-horizon memory QA, and efficiency, measured by the answer-context tokens fed to the SLM. We conduct four experiments targeting distinct questions: (1) Does MemFlow outperform SLM baselines and dedicated memory systems on established benchmarks? (2) Does the improvement generalize across SLM backbones? (3) Which pipeline components drive performance? (4) What does MemFlow’s internal behavior look like on representative queries? Together, these experiments validate both the system’s practical value and the design principles described in Section 3. 5 single-session- preference temporal- reasoning multi- session single- session-user single-session- assistant knowledge- update Question Type 0 2k 4k 6k 8k 10k 12k Avg Context Tokens (a) Per-question-type context token usage (bars, left axis) and accuracy (lines, right axis) across baselines on LongMemEval. MemFlow (left) RAG (left) ReAct (left) GPT-4o matched (left) GPT-4o ceiling (left) MemFlow RAG ReAct GPT-4o matched GPT-4o ceiling 1k10k100k Answer context tokens (log scale) 0 10 20 30 40 50 60 70 80 Accuracy (%) Direct QA (short ctx) Direct QA (full ctx) RAG ReAct Memobase MemGPT MemFlow GPT-4o (matched) GPT-4o (ceiling) (b) Accuracy vs. answer context tokens. SLM BaselinesMemory SystemsMemFlowGPT-4o 0 20 40 60 80 100 Accuracy (%) Figure 3: (a) Per-question-type accuracy (lines, right axis) and mean answer-context tokens (bars, left axis) on LongMemEval. (b) Accuracy vs. answer-context tokens across all systems (log scale). 4.1 Experimental Setup Benchmarks.We evaluate on three established long-context QA benchmarks: LongMemEval [43] (500 questions across 6 question types testing long-term chat memory), LoCoMo [25] (1,986 questions over 600-turn conversations spanning adversarial, multi-hop, temporal, preference, and single-hop queries), and LongBench [4] (1,750 questions across 9 document-level tasks). Together these yieldN =4,236evaluation items covering both session-based episodic memory and single- document comprehension. Model and baselines. All SLM systems use Qwen3-1.7B [35] as the frozen backbone unless otherwise noted. We compare against four SLM baselines: Direct QA (short/full), with raw conver- sation truncated to match MemFlow’s answer-context and full-pipeline token budgets respectively; RAG [22], hybrid BM25+dense retrieval with top-kchunks (2,266 avg tokens); and ReAct [46], multi-step reasoning with access to the same retrieval and deterministic tools (8,732 avg tokens). We further compare against two dedicated memory systems using the same backbone: Memobase [8] (user-profile memory with BM25 retrieval) and MemGPT [30] (working+archival memory with model-driven retrieval). GPT-4o [29] results are reported at matched (9.8k) and full (120k) token budgets as frontier references. All SLM baselines use the same benchmark inputs and GPT-4o-mini judge; RAG and ReAct use the same hybrid BM25+dense retrieval backend as MemFlow. For external memory systems, context sizes are estimated from retrieval metadata where direct token accounting is unavailable; these comparisons should therefore be interpreted as system-level references rather than exact efficiency-matched baselines. Full implementation details are in Appendix C.3. Evaluation.All systems are scored by a GPT-4o-mini judge [28] that compares predicted answers against gold references with a binary correct/incorrect judgment. We use deterministic decoding over the fullN =4,236question pool and report accuracy (%) as the primary metric; exact match and F1 are not reported as they systematically underestimate performance on open-ended memory QA where semantically correct answers differ lexically from gold references [43]. To check judge reliability, we compare GPT-4o-mini and GPT-4o on 300 held-out predictions, observing 85.7% agreement; disagreements are mostly cases where GPT-4o-mini is stricter (Appendix C.2). We also manually annotate a held-out sample of 300 predictions stratified across benchmarks and systems; GPT-4o-mini agrees with human labels on 89.6% of cases, with most disagreements involving borderline semantic-equivalence judgments (Appendix C.2). We therefore treat LLM-judge accuracy as a semantic correctness estimate rather than an exact lexical benchmark score. 4.2 Main Results Table 1 summarizes average accuracy across the three benchmarks. MemFlow achieves 52.4% overall, the highest among all SLM-based systems, representing a+23.4 p lift over Direct QA (full) and+6.2 p over both RAG and ReAct. RAG and ReAct happen to tie at 46.2% on average (a coincidence of aggregation, as their per-type profiles differ considerably), yet ReAct uses3.9×as 6 Table 2: MemFlow generalizes across SLM backbones. Each model is evaluated in Direct QA mode and with MemFlow scaffolding.∆= absolute improvement from MemFlow. All accuracies judged by GPT-4o-mini. BackboneModeLongMemEval (%)LoCoMo (%)LongBench (%)Overall (%)∆ Qwen3-1.7B Direct42.233.719.829.0 MemFlow61.851.251.152.4+23.4 SmolLM2-1.7B Direct25.018.619.319.6 MemFlow37.247.939.043.0+23.4 Qwen3-0.6B Direct26.623.834.028.3 MemFlow50.641.638.541.4+13.1 LLaMA-3.2-1B Direct20.621.416.919.4 MemFlow38.245.532.139.1+19.7 Gemma-3-1B Direct24.621.625.023.4 MemFlow28.627.931.429.4+6.0 Direct QA uses the available direct-prompt setting for each backbone. MemFlow answer context averages∼2,200 tokens across all backbones. Models ordered by Overall MemFlow accuracy. many tokens (8,732 vs. 2,266), indicating that open-ended reasoning loops provide no net benefit over simple retrieval for SLM-scale memory tasks. Among dedicated memory systems, MemGPT reaches 30.9% and Memobase only 9.6%—both sub- stantially below MemFlow despite sharing the same backbone. The performance gap is architectural: Memobase’s profile compression discards episodic recall (∼260 context tokens), while MemGPT’s unstructured retrieval (∼1,260 tokens) lacks the intent-aware routing needed to surface relevant evidence. Context and pipeline cost. Figure 3(b) plots accuracy against answer-context tokens, i.e., the evidence budget seen by the final answer model. By this measure, MemFlow reaches the highest SLM accuracy at an answer context comparable to RAG (2,223 vs. 2,266 tokens) and far smaller than ReAct (2,223 vs. 8,732 tokens). This is not the same as total inference cost: the full MemFlow pipeline averages 10,167 prompt tokens per query across the router, answer agent, validator, and conditional escalation (Appendix A.1), slightly above ReAct’s 8,732-token reasoning loop. Thus MemFlow should be interpreted as trading a modest amount of orchestration overhead for a more reliable and compact final reasoning context: it improves average accuracy by+6.2p over ReAct while reducing the context seen by the final answer stage by3.9×. We report both views because they measure different costs: full pipeline tokens capture system-level inference overhead, while answer-context length measures the reasoning burden placed on the small model. Figure 3(a) further shows that the answer-context reduction holds across all six LongMemEval question types. The gap to budget-matched GPT-4o is 8.9 p (52.4% vs. 61.3%), representing the residual cost of compressing a long conversation into a compact packed context rather than attending to the full history with a substantially larger model. Per-question-type breakdown.MemFlow matches or exceeds budget-matched GPT-4o on single-session-preference(+3.3 p) andtemporal-reasoning(+3.8 p) on LongMemEval (Figure 3(a)), and dominates adversarial queries on LoCoMo (+8.7 p over ReAct), where the Valida- tor Agent’s grounding check rejects hallucinated answers before they are returned. Remaining gaps to GPT-4o are concentrated onknowledge-update(−25.6 p), where cross-session conflict resolution exceeds the packer’s compression fidelity, and on queries routed to theconflict-resolutiontier (−48.9 p relative to GPT-4o on those items), driven partly by router over-prediction on that tag. Full per-type results are in Appendix E.2. 4.3 Generalization Across SLM Backbones Table 2 evaluates MemFlow with four additional SLM backbones: Qwen3-0.6B [35], SmolLM2- 1.7B [6], LLaMA-3.2-1B [15], and Gemma-3-1B [13]. MemFlow improves every backbone without exception, with lifts ranging from+6.0 p (Gemma-3-1B) to+23.4 p (Qwen3-1.7B and SmolLM2- 1.7B). The largest gains occur on models that produce near-degenerate outputs under direct prompting: SmolLM2-1.7B and LLaMA-3.2-1B both score under 20% on direct QA, with manual inspection 7 Table 3: Ablation study. Each row removes one component from the full MemFlow pipeline.∆= change in overall accuracy relative to full MemFlow. All accuracies judged by GPT-4o-mini. ConfigurationLongMemEval (%)LoCoMo (%)LongBench (%)Overall (%)∆ MemFlow (full)61.851.251.152.4— – Retrieval Strategy45.633.930.133.7−18.7 – Uniform RAG60.442.837.942.9−9.5 – Router56.045.339.244.0−8.4 – Tools60.845.137.944.0−8.4 – Escalation & Validator58.244.341.244.7−7.7 – Packer58.043.043.144.8−7.6 Rows ordered by impact (|∆|). All ablations use the same Qwen3-1.7B backbone, evaluation set, judge, and approximately matched answer-context budget. revealing near-empty or format-broken responses, yet reach 43.0% and 39.1% under MemFlow. This confirms that MemFlow’s structured context injection restores coherent output formatting regardless of backbone capacity. Gemma-3-1B receives the smallest lift (+6.0 p), suggesting it handles the structured prompt format less effectively than other architectures. Notably, all backbones benefit consistently across all three benchmarks, confirming that MemFlow’s gains are not specific to any one model or evaluation domain. All of these gains are obtained without model fine-tuning, reinforcement learning, or benchmark-specific training. The same orchestration pipeline is applied across LongMemEval, LoCoMo, and LongBench; we therefore interpret the backbone results as evidence that the improvements come from structured memory operations rather than adaptation to a single model family. 4.4 Ablation study Table 3 isolates the contribution of each component under the same Qwen3-1.7B backbone, evaluation set, judge, and approximately matched answer-context budget. – Retrieval Strategy disables tag-specific retrieval and preprocessing while retaining the rest of the pipeline; – Uniform RAG replaces the tiered Memory Agent with flat hybrid BM25+dense retrieval; – Router bypasses intent classification and uses a fixed fallback memory path; – Tools removes deterministic preprocessing and bounded tool calls such as date arithmetic and counting; – Escalation & Validator returns the first answer without grounding checks or heavier-tier retry; and – Packer replaces priority-aware context compilation with simple retrieved-chunk concatenation. Retrieval strategy selection is the single most critical component (−18.7 p): disabling tag-specific retrieval and preprocessing causes the largest drop. Router, Uniform RAG, and Tools form a cluster at−8.4 to−9.5 p, confirming that routing quality, retrieval specialization, and deterministic prepro- cessing are tightly coupled; the explicit – Uniform RAG row isolates the flat-retrieval replacement (−9.5 p). Escalation/Validator and Packer each contribute a meaningful margin (−7.6 to−7.7 p), validating that grounding verification and priority-ordered context compilation are both necessary. To- ken costs remain stable across ablations (within±5%), suggesting that MemFlow’s gains derive from the quality of retrieved context rather than the quantity of computation. Per-benchmark breakdowns are in Appendix E.5. 4.5 Qualitative analysis Figure 4 illustrates MemFlow’s internal behavior on six representative examples. Panel (a) shows the router correctly dispatching a temporal query to a Tier 3 handler with tool access, producing the correct answer from only 711 packed tokens. Panel (c) demonstrates45×context compression (9,935→220 tokens) with no information loss — the packed context contains exactly the evidence needed to answer the query. Panel (e) shows the validator–escalation loop recovering from an initially ungrounded answer: the first pass (1,571 prompt tokens) fails validation, triggering re-retrieval with a broader strategy (2,294 tokens) that produces the correct answer. Panel (f) confirms adversarial robustness: when queried about information absent from the conversation, MemFlow correctly abstains rather than hallucinating — a failure mode that afflicts both Direct QA and ReAct on the same questions. 8 Q: "What was the first issue I had with my new car after its first service?" Router: temporal-reasoning → Tier 3 Heavy Retrieved: 711 tokens packed (from ~4k total pipeline) Pred: "The first issue was the GPS system malfunction on March 22, 2023." Gold: "GPS system not functioning correctly" ✓ (a) Triage Routing - correct temporal classification + tool use Router identifies temporal nature and assigns to a high tier for detailed reasoning. Q: "Can you suggest some accessories that would complement my current photography setup?" Router: profile-injection → Tier 1 Fast Retrieved: 973 tokens (user profile loaded) Pred: "Based on your focus on Sony A7R IV and recent discussions about upgrades, I'd recommend complementary accessories like a lens hood, filter holder..." Gold: Sony-compatible accessories ✓ (b) Profile Injection - fast path, preference recall Fast routing triggered by specific user preference keywords. Q: "When did John plan to return from his trip to Toronto and Vancouver?" Total pipeline: 9,935 tokens → packed to 220 tokens (45× compression) Pred: "John planned to return on July 20, 2022." ✓ (c) Context Packing - 45x compression Model effectively compresses large volumes of retrieved text before reasoning. Q: "How many days had passed between the Sunday mass at St. Mary's Church and the Ash Wednesday service at the cathedral?" Router: temporal-reasoning → tool: date_math Pred: "30 days had passed." Gold: "30 days" ✓ (d) Deterministic Tool Dispatch - date_math Router identifies tool requirement and dispatches specific computation. Q: "Which trip did the narrator take first, the solo trip to Europe or the family road trip across the American Southwest?" 1st pass: answer_initial (1,571 prompt tok) → Validator: FAIL (not grounded) Escalation: answer_esc (2,294 prompt tok, re-retrieved) → Validator: PASS Pred: "The narrator took the family road trip across the American Southwest first, followed by their solo trip to Europe." Gold: "The family road trip across the American Southwest" ✓ (e) Validator Rejection & Escalation - first attempt failed, re-retrieval succeeded Model detects its own failure and escalates to a more capable pipeline. Q: "What country is Melanie's grandma from?" (adversarial — never mentioned) Router: targeted-extraction Packed: 404 tokens Pred: "The context does not provide information about the country where Melanie's grandma is from." Gold: (unanswerable) ✓ (f) Adversarial Robustness — rejects false premise Model identifies unanswerable questions and avoids hallucination. Figure 4: Snippets of MemFlow’s internal states and actions on representative queries. Each panel shows a distinct pipeline behavior: (a) triage routing, (b) profile injection, (c) context packing, (d) deterministic tool dispatch, (e) validator rejection and escalation, (f) adversarial robustness. All examples are drawn from actual predictions on the LongMemEval, LoCoMo, and LongBench benchmarks. 5 Conclusion, Limitations, and Future Work We presented MemFlow, a training-free memory orchestration framework for long-horizon SLM agents. Its core finding is that a substantial portion of SLM memory failure can be mitigated by matching each query to an appropriate memory operation before evidence is retrieved, transformed, and packed. Rather than introducing a new retriever, compressor, or verifier in isolation, MemFlow shows that bounded orchestration—structured intent routing, deterministic evidence compilation, and grounding validation—can help frozen SLMs use limited context more effectively and narrow the gap to frontier models in targeted long-horizon memory settings. Although motivated by personal and conversational memory, the same operation-routing view extends to document QA and assistant settings where small models answer from large, heterogeneous context. Several limitations remain: the full pipeline has nontrivial orchestration overhead dominated by the router prompt, single-intent routing can miss compound queries, evolving cross-session facts remain difficult, and LLM-judge accuracy is a semantic rather than exact lexical score. A promising future direction is to reduce router overhead and incorporate learned decision policies for validation, escalation, and tool dispatch, using answer correctness as a reward signal while preserving MemFlow’s bounded execution paths and compact final-answer contexts. 9 References [1]Marah Abdin, Jyoti Aneja, Hany Awadalla, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024. [2] Anthropic. Introducing the next generation of Claude. Anthropic Blog, 2024. [3]Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self-RAG: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2024. [4]Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. LongBench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3119–3137, 2024. [5]Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020. [6] Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, et al. SmolLM2: When smol goes big – data-centric training of a small language model. arXiv preprint arXiv:2502.02737, 2025. [7]Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023. [8]Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready AI agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025. [9]Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022. [10]Yiming Du, Bingbing Wang, Yang He, Bin Liang, Baojun Wang, Zhongyang Li, Lin Gui, Jeff Z. Pan, Ruifeng Xu, and Kam-Fai Wong. MemGuide: Intent-driven memory selection for goal-oriented multi-session LLM agents. arXiv preprint arXiv:2505.20231, 2025. [11]Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. From local to global: A graph RAG approach to query-focused summarization. arXiv preprint arXiv:2404.16130, 2024. [12]Kazem Faghih, Wenxiao Wang, Yize Cheng, Siddhant Bharti, Gaurang Sriramanan, Sriram Balasubramanian, Parsa Hosseini, and Soheil Feizi. Tool preferences in agentic LLMs are unreliable. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20954–20969, 2025. [13] Gemma Team. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. [14]Google DeepMind. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2024. [15]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The LLaMA 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [16]Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. REALM: Retrieval-augmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning, pages 3929–3938, 2020. [17]Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with contrastive learning. arXiv preprint arXiv:2112.09118, 2021. 10 [18]Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874–880, 2021. [19]Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1658–1677, 2024. [20] Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. Active retrieval augmented generation. In Proceedings of EMNLP, 2023. [21]Vladimir Karpukhin, Barlas O ̆ guz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, 2020. [22]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv preprint arXiv:2005.11401, 2020. [23]Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. [24]Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, and Vikas Chandra. MobileLLM: Optimizing sub-billion parameter language models for on-device use cases. arXiv preprint arXiv:2402.14905, 2024. [25]Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. Evaluating very long-term conversational memory of LLM agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13851–13870, 2024. [26]Feiteng Mu, Yong Jiang, Liwen Zhang, Chu Liu, Wenjie Li, Pengjun Xie, and Fei Huang. Query routing for homogeneous tools: An instantiation in the RAG scenario. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10225–10230, Miami, Florida, USA, 2024. Association for Computational Linguistics. [27]Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. RouteLLM: Learning to route LLMs with preference data. arXiv preprint arXiv:2406.18665, 2024. [28] OpenAI. GPT-4o mini: Advancing cost-efficient intelligence. OpenAI Blog, 2024. [29] OpenAI. GPT-4o system card. arXiv preprint arXiv:2410.21276, 2024. [30] Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, and Joseph E. Gonzalez. MemGPT: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023. [31] Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. LLMLingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. In Findings of the Association for Computational Linguistics: ACL 2024, pages 963–981, 2024. [32]Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 2023. 11 [33]Daivik Patel and Shrenik Patel. ENGRAM: Effective, lightweight memory orchestration for conversational agents. arXiv preprint arXiv:2511.12960, 2025. [34] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive APIs. arXiv preprint arXiv:2305.15334, 2023. [35] Qwen Team. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [36]Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. Zep: A temporal knowledge graph architecture for agent memory. arXiv preprint arXiv:2501.13956, 2025. [37] Swarnadeep Saha, Archiki Prasad, Justin Chih-Yao Chen, Peter Hase, Elias Stengel-Eskin, and Mohit Bansal. System-1.x: Learning to balance fast and slow planning with language models. arXiv preprint arXiv:2407.14414, 2024. [38]Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. ColBERTv2: Effective and efficient retrieval via lightweight late interaction. In Proceedings of NAACL, 2022. [39] Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. RAPTOR: Recursive abstractive processing for tree-organized retrieval. In Interna- tional Conference on Learning Representations (ICLR), 2024. [40]Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettle- moyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. [41]Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023. [42] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10014–10037, 2023. [43]Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. Long- MemEval: Benchmarking chat assistants on long-term interactive memory. arXiv preprint arXiv:2410.10813, 2024. [44]Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Awadallah, Ryen W. White, Doug Burger, and Chi Wang. AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arXiv preprint arXiv:2308.08155, 2024. [45]Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884, 2024. [46]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023. [47]Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. RankRAG: Unifying context ranking with retrieval-augmented generation in LLMs. arXiv preprint arXiv:2407.02485, 2024. [48]Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems, 2020. [49]Tianhao Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, and Joseph E. Gonzalez. RAFT: Adapting language model to domain specific RAG. arXiv preprint arXiv:2403.10131, 2024. 12 [50]Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, and Beidi Chen. H2O: Heavy-hitter oracle for efficient generative inference of large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023. [51] Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations (ICLR), 2024. [52]Zihan Zhou, Simon Kurz, and Zhixue Zhao. Revisiting pruning vs quantization for small language models. Findings of the Association for Computational Linguistics: EMNLP 2025, pages 12055–12070, 2025. [53]Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. Mem1: Learning to synergize memory and reasoning for efficient long-horizon agents. arXiv preprint arXiv:2506.15841, 2025. 13 A System Implementation Details A.1 Computational Setup All MemFlow experiments are conducted on Google Colab using NVIDIA L4, A100, and H100 GPUs (whichever was available at runtime; results are hardware-independent since no training is per- formed). The frozen Qwen3-1.7B [35] backbone is loaded via thetransformersinference API with bfloat16precision and served on GPU. The dense retrieval encoder (BAAI/bge-small-en-v1.5) runs on CPU only; GPU inference for the encoder was disabled because LoCoMo’s 35-session conversations exhaust GPU memory during FAISS index construction on smaller GPU tiers. No training is performed: all model weights are frozen and identical across all system variants and GPU configurations. Token cost breakdown. Table 4 decomposes the per-query token usage across pipeline stages, computed from the full N =4,236 evaluation set. Table 4: Per-stage token usage, MemFlow main experiment (N = 4,236, Qwen3 tokenizer).†2.0% of queries short-circuit before routing/answering/validation; the remaining router reduction comes from the rule layer.‡Of the 14.9% that trigger escalation, 3.4% receive the escalated response as the final answer.∗Weighted contribution to mean pipeline cost after multiplying conditional mean tokens by invocation rate. StageMean tokensMedianWeighted %Invocation rate Router Agent (prompt)6,9546,95358.0% ∗ 84.8% † Answer Agent (prompt)2,5501,46824.6% ∗ 98.0% † Validator (prompt)1,0871,07510.5% ∗ 98.0% † Escalation answer (prompt)2,2892,2603.4% ∗ 14.9% ‡ Fixed wrappers/tool results314—3.1%100% Total pipeline10,1678,793— Packed context (answer input)2,223874— The Router Agent accounts for the largest weighted share of token usage (58.0% of the pipeline mean), driven by its fixed system prompt that encodes the full seven-tag taxonomy. In deployments that serve repeated queries with the same router prompt, this static prefix can be reused through prefix/KV caching, reducing prefill overhead even though the token accounting in Table 4 reports uncached prompt tokens. The Answer Agent’s mean prompt of 2,550 tokens exceeds its median of 1,468 tokens, indicating that most queries are handled with relatively compact packed contexts while a subset of complex queries receives substantially larger inputs. The Validator is invoked on 98.0% of queries at an average of 1,087 tokens per call (10.5% weighted contribution). The Escalation answer stage fires on 14.9% of queries; its 3.4% weighted contribution to the mean pipeline cost is computed as2,289× 14.9%/10,167. The remaining 314-token average covers fixed chat-template wrappers, query fields, and tool-result prompts not assigned to a single agent row. Of those 14.9% escalated queries, 3.4% of all queries (146/4,236) ultimately receive a different escalated response as the final answer (is_escalated = True); the remaining 11.5% trigger validation-driven retry but retain the original answer or a fallback. Per-benchmark packed context. The 2,223-token average masks substantial variation across benchmarks, reflecting the adaptive nature of the packer: LongMemEval (session-based episodic queries) averages 1,543 tokens; LoCoMo (peer-conversation queries) averages 469 tokens; Long- Bench (document-level QA and summarization) averages 4,408 tokens, approaching the 6,000-token Tier-2 ceiling for broad-coverage document tasks. A.2 Retrieval Configuration The retrieval layer uses hybrid BM25+dense retrieval with Reciprocal Rank Fusion (RRF). Table 5 lists all hyperparameters. For Tier-2 targeted-extraction queries, the handler performs multi-pass entity-aware retrieval: a primary hybrid pass retrieves the top-kchunks for the full query; secondary passes retrieve separately 14 Table 5: Retrieval hyperparameters. ParameterValueNotes Strategy hybridBM25 + dense, fused by RRF Dense encoder BAAI/bge-small-en-v1.5384-dim, CPU inference BM25 k 1 1.5Okapi BM25 saturation BM25 b0.75Length normalization RRF k60Standard RRF constant Base top-k8Candidates per hybrid pass Tier-2 top-k20Targeted extraction Tier-2 doc top-k40LongBench document tasks Tier-3 top-k20Heavy reasoning passes Tier-3 broad top-k80Broad-summarization tag Map chunk size5Turns per map-reduce shard for each extracted entity or noun phrase; results are unioned, deduplicated by chunk ID, and ranked by score. For Tier-3 temporal-reasoning queries, dual-anchor retrieval extracts event references from the query and retrieves separately for each anchor before merging. Entity and noun-phrase anchors are extracted with a deterministic lightweight parser: capitalized name spans, quoted strings, numeric/date expressions, and noun phrases around possessive or relational markers are retained after stopword filtering. Event anchors for temporal queries are the two highest-scoring verb-phrase or date-bearing spans that overlap with question terms. No external trained parser is used. For the remaining Tier-3 tags, conflict-resolution sorts candidate evidence by timestamp and marks stale facts, broad-summarization applies session-diverse map-reduce aggregation, constraint-validation filters for modal and rule language (“always”, “never”, “must”, “allowed”), and state-tracking keeps chronological state-change candidates before packing. A.3 Context Packer: Detailed Budget Allocation The Context Packer enforces a strict global SLM ceiling and allocates it across three priority slots (ordered below by packer priority, not by retrieval tier number): (1) Tier-1 slots (pinned facts, user profile, behavioral constraints; never truncated), (2) Tier-3 slots (pre-computed summaries and aggregated facts; reduced on overflow), and (3) Tier-2 slots (raw episodic chunks from retrieval; truncated first on overflow). Unused budget from higher-priority tiers is dynamically reallocated to lower tiers. The resulting packed context averages∼2,200 tokens across all benchmarks (see Table 10 for per-benchmark breakdowns). Per-tag Tier-2 budgets. Differentaction_tagvalues impose different Tier-2 chunk budgets, reflecting the information density required for each query type: Table 6: Per-tag Tier-2 chunk budgets and per-chunk word caps. Dashes indicate no per-chunk word cap is applied. Action tagTier-2 budget (tokens)Per-chunk word cap profile-injection0N/A targeted-extraction6,000300 words temporal-reasoning4,400— conflict-resolution6,000— broad-summarization8,000— constraint-validation6,000200 words state-tracking6,000150 words Packer enhancements.Beyond budget enforcement, the packer applies four post-retrieval enhance- ments: (1) Relevance sort — chunks are ordered by reranker score before packing; (2) Sentence-level extraction — for precision-sensitive tags, the packer keeps only the top-2 most relevant sentences per chunk rather than the full chunk; (3) Jaccard deduplication — near-duplicate chunks (Jaccard similarity≥ 0.85) are dropped; and (4) Dynamic budget reallocation — unused budget from Tier-1 or Tier-3 sections is reallocated to the Tier-2 episodic section.targeted-extraction, 15 constraint-validation, andstate-trackingare treated as precision-sensitive because the answer usually depends on a specific fact, rule, or state transition rather than a broad summary. Sentence relevance is scored by normalizing the sum of dense similarity and token-overlap with the query. The top-2 sentence cap and Jaccard threshold of 0.85 are tuned implementation parameters selected on development runs to reduce duplicate evidence under the SLM budget; no test-set labels are used to choose them. B Complete Prompt Listings B.1 Router Agent System Prompt The Router Agent receives the user query and outputs a structured JSON routing decision. The full system prompt encodes the seven-tag taxonomy, disambiguation rules, and few-shot JSON examples. Below we reproduce the tag definitions and selected disambiguation rules; the complete prompt (with all examples) is released with the code. Tag taxonomy. • profile-injection — the user asks for a draft, recommendation, or action that should reflect their personal style, tone, or stated preferences. No retrieval required; the answer comes from a stored user profile. • targeted-extraction — the user wants to recall one specific fact, detail, item, or piece of content from a single past conversation. The fact is stable and not subject to change. • temporal-reasoning— the user asks about the order or timing of past events: which came first, how long between two events, when something happened relative to something else, or how much time elapsed. Requires chronological sorting and/or date arithmetic. • conflict-resolution— the user asks for a value that may have changed over time and wants the most recent value (e.g. current job, latest address, updated plan). Must retrieve across sessions and select the newest version. • broad-summarization — the user asks for a count, list, frequency, or synthesis of something mentioned across multiple past conversations. Requires scanning and combining facts from distinct sessions. • constraint-validation— the user asks about a behavioral rule, policy, or constraint: what is always/never allowed, what rule applies to a situation. • state-tracking — the user asks how a fact, preference, or situation evolved or changed over time: the full progression, not just the current state. Output schema. The router outputs a constrained JSON object: "requires_rag": boolean, "requires_reasoning": boolean, "action_tag": string // one of the seven tags above If the SLM output cannot be parsed as valid JSON, a regex rescue extracts the tag from free-form text; a keyword heuristic serves as the final fallback. This three-tier cascade (rules→SLM→heuristic) achieves 87.7% overall routing accuracy (§E.1). Selected disambiguation rules (abridged). The prompt encodes explicit disambiguation rules to resolve frequent boundary cases. A representative selection: • “How many days/weeks passed between X and Y?”→ temporal-reasoning(elapsed-time measurement, not event counting). •“How long was I in Japan?”→ targeted-extraction(duration of a single event = one stable recalled fact, not temporal comparison). 16 •“What is my current job?”→ conflict-resolution(one tracked thing whose value may have changed; need most recent). •“How many magazine subscriptions do I have?”→ broad-summarization(many distinct items each acquired in a different conversation; requires broad scan). •“What should I do when a customer complains?”→ constraint-validation(procedural rule for a conditional situation, not open-ended advice). •“Any tips for keeping my kitchen clean?”→ profile-injection(advice request without an explicit condition; answer depends on user preferences). Programmatic rule layer. Before the SLM router is called, deterministic rules handle unambiguous lexical patterns: elapsed-time questions with “how many days/weeks/months between” map totemporal-reasoning; “current/latest/most recent” value questions map toconflict-resolution; explicit “always/never/allowed/must” policy questions map to constraint-validation; “how has X changed/evolved” maps tostate-tracking; and person- alized drafting or recommendation requests with user-preference cues map toprofile-injection. If multiple rules match, the query is sent to the SLM router rather than resolved by the rule layer. B.2 Per-Tag Answer Agent System Prompts The Answer Agent receives a tag-specific system prompt that frames the reasoning task. We provide the full prompt text for each of the seven primary tags below. profile-injection. “You are a personalized assistant. First, identify the user’s stated preferences, past choices, or personal details from the conversation history in the context. Then give a tailored response that reflects those specific preferences. If the question asks what the user would prefer or what kind of response they want, describe their preferences explicitly (e.g. ‘Based on your past conversations, you prefer X’). If there is only partial information, still give your best tailored response and cite the specific preference or fact you are drawing on. Prefer a personalized answer whenever the context contains any relevant preference. Keep the response concise (1–3 sentences). [shared grounding instruction]” targeted-extraction. “You are answering a targeted memory-recall question. Use only the retrieved context to identify the specific fact requested by the user. If multiple context snippets mention the same entity, choose the snippet that most directly answers the question and quote or paraphrase only that fact. Do not add background details that are not needed for the answer. [shared grounding instruction]” temporal-reasoning (tool mode, elapsed-time queries). This prompt enforces a three-step protocol when date arithmetic is required: “STEP 1 — IDENTIFY BOTH DATES: Scan the context and write down the date for each event mentioned in the question. State them explicitly: ‘Event A date: Y- M-D (from [quote from context])’. STEP 2 — CALL THE TOOL: After iden- tifying both dates, call exactly ONE tool:TOOL: days_between | Y-M-D | Y-M-D(orweeks_between/months_between). STEP 3 — FINAL ANSWER: After the TOOL_RESULT line appears, give your answer using the computed number. [Additional date extraction rules and shared grounding instruction]” conflict-resolution. “You are answering with latest-only conflict-resolved memory facts. The context is sorted chronologically — the most recent entries appear last. Always use the most recent value when facts conflict. [shared grounding instruction]” broad-summarization (tool mode, counting queries). 17 “You are answering a question that may require counting or aggregating facts across multiple past conversations. The context contains map-reduced cross-session summaries. You have access to a counting tool. If the question asks how many times a specific thing appears, call:TOOL: count_occurrences | keyword. Call only one tool per response. After receiving TOOL_RESULT, give your final answer. [shared grounding instruction]” constraint-validation. “You are answering a question about behavioral rules, policies, or constraints. List all applicable Always/Never rules that answer the question and cite the exact rule text. If multiple rules apply, list each one. If no constraint is found in context, say so explicitly. [shared grounding instruction]” state-tracking. “You are answering a question about how something changed or evolved over time. The context is sorted chronologically — earlier sessions appear first. Present the full progression: describe how the fact or situation changed at each stage. Be specific about what changed and when (by session or date if available). Do not just give the final state — show the journey. [shared grounding instruction]” Shared grounding instruction (appended to all prompts). “You are a helpful assistant. Answer using ONLY the information in the provided context. If you have partial or relevant information, always attempt your best answer — do not default to ESCALATE_REQUIRED out of uncertainty. Output the exact literal string ESCA- LATE_REQUIRED ONLY when the context contains absolutely no information relevant to the question. Keep the answer concise.” LoCoMo peer-conversation override. For LoCoMo items, the context is a dialogue between two named speakers (e.g. Alice and Bob) discussing third-party people (e.g. Caroline, Melanie). A specialized prefix instructs the model to search every turn for mentions of the third party and extract facts embedded in narrative speech (e.g. “I talked to X and she mentioned Y”), and to avoid premature ESCALATE_REQUIRED output. B.3 Validator System Prompt and Cascade Logic The Validator uses the following system prompt for its LLM grounding check (Qwen3-1.7B, max_new_tokens=8, temperature=0.0): “You are a grounding verifier for a memory retrieval system. Given a question, retrieved context, and a candidate answer, decide whether the candidate answer is supported by — or can reasonably be inferred from — the retrieved context. Reply with exactly one word: ‘yes’ if supported, ‘no’ if not supported.” The user message template is: Question: question Retrieved context: ctx_snippet [truncated to 6,000 characters] Candidate answer: answer Is the candidate answer supported by the retrieved context? Reply yes or no. Three-stage cascade. The validator applies the following cascade before invoking the LLM: 1.Hard-failure detection. Empty answers, the exact stringESCALATE_REQUIRED(or variants matched by a regex), and “not found” patterns immediately trigger escalation. No LLM call is made. 18 2.Short-answer passthrough. Answers of≤6 words, purely numeric answers (e.g. “5”, “21 days”), and boolean answers (“yes”/“no”) bypass the grounding check entirely. Single-number or short factual extractions are almost always correct when they appear in the answer agent’s output. 3. LLM grounding check. For all remaining answers, the validator calls Qwen3-1.7B with the prompt above. If the response cannot be parsed as yes/no, the validator falls back to a token- overlap heuristic (τ ground = 0.07). C Benchmark and Evaluation Details C.1 Benchmark Composition Table 7 summarizes the three benchmarks used in our evaluation. Table 7: Benchmark statistics. “Session depth” refers to the number of conversation sessions per evaluation item. BenchmarkQuestionsSession depthQuery types LongMemEval [43]500Multi-sessionSingle-session-preference, single-session- user, single-session-assistant, multi-session, temporal-reasoning, knowledge-update LoCoMo [25]1,986600-turn peer conversationsAdversarial, multi-hop, temporal, prefer- ence; third-party named-entity queries LongBench [4]1,750Single document QA, summarization, multi-document reason- ing (9 subtasks) Total4,236— LongMemEval. 500 questions over long-term chat histories, testing six memory capabilities. We use the oracle split (longmemeval_oracle.json) which provides ground-truth session annotations, allowing us to report oracle retrieval recall alongside end-to-end accuracy. LoCoMo.1,986 questions over 600-turn conversations between two named speakers discussing a shared social network. Many questions ask about third-party people (e.g. “What does Caroline do for work?”) where the answer is embedded in narrative speech rather than direct statements. This makes the benchmark particularly challenging for retrieval systems that rely on surface-level keyword matching. LongBench.1,750 questions across nine document-level subtasks from LongBench v1. We include tasks spanning single-document QA (NarrativeQA, Qasper, MultiFieldQA-en), multi-document QA (HotpotQA, 2WikiMultiHopQA, MuSiQue), and summarization (GovReport, QMSum, MultiNews). LongBench is qualitatively different from the other two benchmarks: it tests document comprehension rather than episodic memory, and is included to evaluate MemFlow’s generalization beyond its primary session-based memory use case. C.2 GPT-4o-mini Judge Protocol All systems are scored by a GPT-4o-mini judge [28] that compares the predicted answer against the gold reference and outputs a binary correct/incorrect judgment. Judge prompts. The system and user prompts used for evaluation are: System: “You are an expert judge evaluating question-answering accuracy. Be lenient on phrasing but strict on factual correctness.” User template: 19 Question: question Gold Answer: gold Predicted Answer: predicted Is the predicted answer correct or semantically equivalent to the gold answer? Accept partial matches if the key information is present. Reply ONLY with JSON: "correct": true or "correct": false Why GPT-4o-mini judging over EM/F1.Exact match (EM) scores are near-zero across all systems (0.07%–5.1%), while GPT-4o-mini accuracy ranges 9.6%–74.3%. Token-level F1 (2.7%–24.4%) is also severely deflated. This reflects the open-ended nature of memory QA: a correct answer such as “GPS system malfunction on March 22, 2023” has low lexical overlap with the gold reference “GPS system not functioning correctly” yet is clearly correct. We report only GPT-4o-mini accuracy as the primary metric, consistent with the evaluation protocol of LongMemEval [43]. Choice of GPT-4o-mini as judge. We use GPT-4o-mini rather than GPT-4o to keep evaluation cost tractable across 4,236 items×five systems. On a held-out sample of 300 predictions, the two judges agree on 85.7% of items; in the 14.3% of disagreements, GPT-4o-mini is the stricter judge (GPT-4o accepts 35 answers that GPT-4o-mini rejects, vs. 8 in the reverse direction). Consequently, all reported accuracy figures are conservative estimates relative to the LongMemEval standard judge; replacing GPT-4o-mini with GPT-4o would be expected to increase all system scores uniformly, leaving relative rankings and p-gap claims unchanged. Human audit. To further validate the automatic judge, we manually annotated 300 randomly sampled predictions stratified across benchmarks and systems. GPT-4o-mini agreed with the human label on 89.6% of items, with disagreements concentrated in borderline semantic-equivalence cases. This audit supports using GPT-4o-mini accuracy as a scalable semantic correctness estimate rather than an exact lexical metric. Fallback. In 0.9% of cases where the OpenAI API is unavailable (network errors, rate limits), we fall back to a token-overlap correctness check with a 50% recall threshold. These fallbacks are flagged in the output JSONL as gpt4o_fallback: true. C.3 Baseline Implementation Details Direct QA (short, 2,333 tokens). The raw conversation history is truncated to 2,333 tokens and passed directly to Qwen3-1.7B. No retrieval or routing is performed. The reported “Answer Ctx” for this baseline (avg. 2,458 tokens) exceeds 2,333 because it includes the system prompt and question tokens appended to the truncated history; the 2,333-token figure refers strictly to the history truncation budget. Token budget matches MemFlow’s average packed context. Direct QA (full, 9,800 tokens). Same as Direct QA (short) but truncated to 9,800 tokens. Token budget matches MemFlow’s full pipeline budget. RAG. Hybrid BM25+dense retrieval (same retriever as MemFlow) with a fixed top-kof 8. No routing layer; all queries receive the same flat retrieval strategy. Average packed context: 2,266 tokens. ReAct [46]. Multi-step reasoning loop with access to the same hybrid BM25+dense retrieval backend and deterministic tools used by MemFlow. The SLM decides when to search, which tool call to issue, and when to answer. Maximum 5 reasoning turns per query. Average prompt tokens: 8,732. No validator or escalation. This baseline tests whether an SLM can use the same retrieval and deterministic tool interface through open-ended reasoning rather than MemFlow’s bounded route-then-compile execution policy. MemGPT [30]. Working memory + archival memory with model-driven page-in/page-out calls. The SLM issues memory management commands (recall_memory,archival_memory_search) as part of its response. Average packed context: ≈1,260 tokens (estimated from retrieval metadata). 20 Memobase [8] (open-source Mem0 implementation).User-profile dual-store with BM25 retrieval. Compresses all conversation history into a structured user profile; retrieval operates over the profile rather than raw turns. Average packed context:≈260 tokens. This extremely small context budget explains Memobase’s low accuracy (9.6%): profile compression destroys episodic recall needed for most benchmark queries. GPT-4o [29] (matched, 9,800 tokens). GPT-4o with the conversation history truncated to 9,800 tokens. No retrieval or routing. Serves as an upper-bound reference at the same token budget as MemFlow’s full pipeline. GPT-4o (ceiling, 120k tokens).GPT-4o with up to 120,000 tokens of conversation history. Serves as the unconstrained frontier reference. D Full Pipeline Algorithm Algorithm 1 provides pseudocode for the MemFlow read-path pipeline. Algorithm 1 MemFlow Read-Path Pipeline Require: Query q, history H , SLM π θ Ensure: Answer a 1: retries← 0 2:if token_overlap(q, last-8 turns of H)≥ 0.6then return highest-overlap contiguous span from the last-8 turns▷ Stage 1 3: end if 4: r ← ROUTERAGENT(q)▷ Stage 2: tag + routing flags 5: evidence←    TIER1(q)if r.tag = profile-injection TIER2(q,H)if r.tag = targeted-extraction TIER3(q,H,r) otherwise ▷ Stage 3 6: C ← PACK(evidence, r.tag)▷ Stage 4: ceiling 20,480 tok, avg 2,223 7: a← π θ (q, C, TAGPROMPT(r.tag))▷ Stage 5: answer agent 8: tool_rounds← 0 9: while tool call in a and tool_rounds < 3 do 10: a← π θ (EXECUTETOOL(a))▷ at most 3 tool rounds 11:tool_rounds← tool_rounds +1 12: end while 13: v ← VALIDATE(a,C,q)▷ Stage 6: grounding check 14: if¬v.grounded and retries = 0 then 15:retries← 1; r ← ESCALATE(r.tag); goto Stage 3 16: else if¬v.grounded then return “I could not find reliable information.” 17: end if 18: return a E Extended Experimental Results E.1 Router Accuracy: Per-Benchmark Breakdown Table 8 reports triage routing accuracy across all three evaluation benchmarks. The three-tier cascade (rules→SLM→heuristic) ensures routing never fails outright. LoCoMo (88.6%) and LongBench (90.9%) achieve higher end-to-end routing accuracy than the dedicated router eval set (87.7%), reflecting that their question structures map cleanly onto MemFlow’s intent-based action tags. LongMemEval’s lower accuracy (72.4%) reflects the difficulty of aligning its question- type ontology—which mixes topic and memory-type labels—with MemFlow’s action tags; the router_correct labels for LongMemEval are derived from benchmark question-type annotations rather than MemFlow-native ground truth. The current router emits one mutually exclusive tag per query. This simplifies execution and avoids open-ended tool plans, but it can under-serve compound 21 Table 8: Router Agentrouter_correctaccuracy per benchmark. 87.7% is from a dedicated router evaluation set (scripts/eval_router_accuracy.py). BenchmarkNRouting accuracy Router eval set (dedicated)—87.7% LoCoMo (end-to-end)1,98688.6% LongBench (end-to-end)1,75090.9% LongMemEval (end-to-end)50072.4% questions that require multiple simultaneous memory operations, such as temporal reasoning over facts that also need conflict resolution. E.2 Per-Question-Type Results Across All Benchmarks Table 9: Per-question-type accuracy (%) and mean packed context tokens fed to the answer agent across all three benchmarks. Bold = MemFlow≥GPT-4o (matched). All accuracies judged by GPT-4o-mini. Question typenMemFlowRAGReActGPT-4oGPT-4oAvg pack (matched)(ceiling)(tokens) LongMemEval single-session-preference3070.063.343.366.766.71,089 temporal-reasoning13345.925.633.142.141.41,159 multi-session13354.928.626.357.160.91,857 single-session-user7084.375.787.194.392.92,031 single-session-assistant5687.582.185.798.298.21,078 knowledge-update7859.051.364.184.684.61,729 Overall LongMemEval50061.846.050.267.868.41,543 LoCoMo adversarial44686.160.877.481.874.0482 multi-hop84156.757.054.357.690.8473 preference9630.228.136.557.378.1415 single-hop28222.330.129.439.762.8517 temporal-reasoning32119.615.320.633.652.6412 Overall LoCoMo1,98651.245.949.656.676.3469 LongBench multifieldqa_en15090.075.378.082.786.04,666 hotpotqa20064.051.558.575.582.05,321 multi_news20058.066.548.070.084.01,382 gov_report20050.543.016.081.588.54,036 qasper20049.548.543.062.563.04,120 2wikimqa20045.043.555.573.075.54,842 narrativeqa20042.030.537.551.067.05,177 qmsum20036.541.019.544.055.54,962 musique20034.527.524.046.565.05,232 Overall LongBench1,75051.146.741.264.773.74,408 Bold = MemFlow≥ GPT-4o (matched). See Section E.2 for analysis. Figure 5 and Table 9 provide a fine-grained breakdown of MemFlow’s performance across all question types. Three patterns emerge. Where MemFlow leads.On LongMemEval, MemFlow leads all SLM baselines on single-session- preference (70.0%,+6.7 p over RAG) and temporal-reasoning (45.9%,+20.3 p over RAG,+3.8 p over GPT-4o matched) — the two categories whose Tier-3 handlers (chronological sort, date math, profile injection) are most directly tailored to the query structure. On LoCoMo, MemFlow dominates adversarial queries (86.1%,+8.7 p over ReAct and+4.3 p over GPT-4o matched), where the 22 s-preference temporal reasoning multi session s-userss-assistant knowledge update 0 2k 5k 7k 10k 12k Avg Context Tokens LongMemEval adversarial multi-hop preference single-hop temporal reasoning 0 5k 10k 15k 20k 25k Avg Context Tokens LoCoMo multifieldqa _en hotpotqa multi_news gov_report qasper 2wikimqa narrativeqa qmsum musique 0 10k 20k 30k 40k Avg Context Tokens LongBench 0 20 40 60 80 100 Accuracy (%) 0 20 40 60 80 100 Accuracy (%) 0 20 40 60 80 100 Accuracy (%) MemFlow (left) RAG (left) ReAct (left) GPT-4o matched (left) GPT-4o ceiling (left) MemFlow RAG ReAct GPT-4o matched GPT-4o ceiling Figure 5: Per-question-type accuracy (%, lines, right axis) and Average context tokens (hatched bars, left axis), grouped by benchmark. MemFlow (blue line), RAG (orange), ReAct (green), and GPT-4o matched (red dashed) are shown for each question type. Bold question-type labels indicate categories where MemFlow meets or exceeds budget-matched GPT-4o. Packed context (hatched bars) reflects the adaptive allocation per question type: LongBench tasks receive 4–5k tokens on average due to the depth of evidence required; LoCoMo adversarial queries require only∼480 tokens. All accuracies judged by GPT-4o-mini; N =4,236 across all three benchmarks. Validator Agent’s grounding check rejects hallucinated answers before they are returned. On Long- Bench, MemFlow outperforms RAG on 7 of 9 task types, with the widest margins on multifieldqa_en (+14.7 p) and gov_report (+7.5 p), where long-document comprehension benefits from structured evidence extraction over raw truncation. Remaining gaps.The largest gaps to GPT-4o matched are on knowledge-update (−25.6 p on Long- MemEval) and LoCoMo preference (−27.1 p), both of which require cross-session reconciliation of evolving facts that the packer’s compression budget cannot fully preserve. On conflict-resolution, router over-prediction (527 queries routed vs. 78 ground-truth conflict queries) causes many non- conflict queries to pass through a more expensive retrieval path, degrading accuracy and representing the primary area for future routing improvement. 23 Context allocation reflects query complexity.The hatched bars in Figure 5 show that MemFlow’s adaptive allocation distributes context budget in proportion to query difficulty: LongBench tasks re- quiring multi-document evidence receive 4,000–5,300 tokens on average, while LoCoMo adversarial queries require only∼480 tokens. This confirms that the packer is not simply filling a fixed budget but is responding to the structural requirements of each query type. Token budget vs. accuracy. The hatched bars in Figure 5 should be read as evidence of adaptive allocation, not as a monotonic predictor of accuracy. On LongBench, all nine task types receive 1.3k– 5.3k tokens—substantially more than LoCoMo’s 412–517 token budget—yet both achieve similar overall accuracy (LoCoMo 51.2%, LongBench 51.1%), because LoCoMo’s queries are typically answerable from a single short span while LongBench requires aggregating evidence across long documents. Within a benchmark, high context use can also mark intrinsically harder tasks such as multi-hop QA. E.3 Token Pipeline Cost Breakdown by Benchmark Table 10: Per-benchmark token statistics. “Pack” = tokens fed to the answer agent (packed context only). “Pipeline” = total tokens across all stages (router + answer + validator + escalation). BenchmarkNPack meanPack medianPipeline mean LongMemEval5001,5431,4857,990 LoCoMo1,9864694498,269 LongBench1,7504,4085,05212,942 Overall4,2362,22387410,167 The large variance across benchmarks reflects MemFlow’s adaptive compression: LoCoMo’s peer- conversation queries tend to be answerable from a single short span (median pack<500 tokens), while LongBench’s document summarization tasks require packing large portions of the document, approaching the 6,000-token Tier-2 ceiling. E.4 Escalation and Validator Analysis Escalation routing chains. When the Validator flags a response as ungrounded, the query is re-routed to a heavier execution tier according to the per-tag escalation chains below. Escalation is capped at one retry; if the escalated response also fails the validator, the system returns a calibrated abstention. Table 11: Escalation policy: one-retry tag-to-tag re-routing. Original tagRetry tag profile-injection targeted-extraction targeted-extraction conflict-resolution temporal-reasoning targeted-extraction conflict-resolution targeted-extraction broad-summarization targeted-extraction constraint-validation targeted-extraction state-tracking conflict-resolution Escalation statistics. Of the 4,236 evaluation queries: •98.0% of queries invoke the validator (4,150/4,236). The remaining 2.0% are short-circuited by the Active Context Check before reaching the validator. • 14.9% of queries trigger escalation (validation failure + re-retrieval, 630/4,236). These represent cases where the initial answer agent response was flagged as ungrounded. •3.4% of queries are truly escalated (146/4,236): the escalation re-retrieval produces a different response that is returned as the final answer. The remaining 11.5% (484/4,236) are cases where 24 escalation was triggered but the original response was ultimately retained or the re-retrieval failed to improve on it. The 146 adopted escalations should not be read as the full contribution of the “Escalation & Validator” ablation in Table 12. That ablation removes both the grounding gate and the retry path, so its effect includes prevented ungrounded answers, abstention behavior, and retry opportunities, not only final-answer swaps. Escalated query accuracy. Escalated queries (is_escalated = True,n = 146) achieve 10.3% accuracy, compared to 53.9% for non-escalated queries. This gap suggests that escalated queries are disproportionately drawn from hard cases (knowledge-update, adversarial unanswerable) where the memory state does not contain sufficient evidence regardless of retrieval strategy. The escalation mechanism helps marginal cases through validation and retry, but cannot recover fundamentally information-deficient queries. E.5 Ablation Study: Per-Benchmark Breakdown Table 12: Per-benchmark ablation results. LME = LongMemEval, LCM = LoCoMo, LB = LongBench. ∆ = overall accuracy vs. full MemFlow (52.4%). ConfigurationLME (%)LCM (%)LB (%)Overall (%)∆ MemFlow (full)61.851.251.152.4— – Retrieval Strategy45.633.930.133.7−18.7 – Uniform RAG60.442.837.942.9−9.5 – Router56.045.339.244.0−8.4 – Tools60.845.137.944.0−8.4 – Escalation & Validator58.244.341.244.7−7.7 – Packer58.043.043.144.8−7.6 Table 12 extends the main-paper ablation (Table 3) with per-benchmark accuracy for each configura- tion. Removing the Retrieval Strategy causes the largest drop across all three benchmarks (−16.2 p on LME,−17.3 p on LCM,−21.0 p on LB), confirming that adaptive intent-aware retrieval is the primary driver of MemFlow’s advantage regardless of benchmark type. Removing Uniform RAG and the Router produce comparable overall drops (−9.5 and−8.4 p), but LongBench is dispro- portionately affected by both Retrieval Strategy and Tools, consistent with its document-level tasks requiring broader chunk coverage and arithmetic reasoning. The Packer and Escalation & Validator components show the smallest individual drops (−7.6 and−7.7 p overall), yet both contribute consistently across all three benchmarks. F Broader Impact MemFlow is a training-free memory scaffolding system designed to enable sub-3B-parameter SLMs to operate as capable long-horizon agents. MemFlow’s deterministic, modular design lowers the barrier to deploying capable AI agents in resource-constrained environments: on-device mobile assistants, edge-computing systems, and institutions with limited GPU infrastructure. By preserving a bounded context budget, MemFlow makes it practical to run persistent personal assistants locally rather than relying on cloud-hosted frontier models, with potential privacy benefits for users who prefer to keep their conversation history on-device. The architecture is model-agnostic: it improves all five tested SLMs, suggesting broad applicability across the open-source model ecosystem. Risks and limitations. Persistent memory systems maintain long-term user profiles derived from conversation history. If deployed without explicit user consent or adequate data governance, such systems could enable unauthorized behavioral profiling, preference tracking, or manipulation. Users should be clearly informed of what is stored, how it is used, and how to delete it. As with all current SLM-based memory systems, accuracy remains below the threshold required for high-stakes decisions (medical, legal, financial) without human oversight. The reliance on a GPT-4o-mini judge for evaluation introduces a potential evaluation bias: judge scores may not perfectly align with human 25 judgments, particularly for ambiguous or culturally specific answers. All experiments are conducted on English-language datasets; generalization to other languages has not been tested. 26

Instant research discovery

MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents

Intelligence

Summary

Entities (12)

Relation Signals (16)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations