Paper deep dive
TokenPilot: Cache-Efficient Context Management for LLM Agents
Buqiang Xu, Zirui Xue, Dianmou Chen, Chenyang Fu, Chiyu Wu, Caiying Huang, Chen Jiang, Jizhan Fang, Xinle Deng, Yijun Chen, Yunzhi Yao, Xuehai Wang, Jin Shang, Gong Yu, Ningyu Zhang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 6/20/2026, 9:15:59 AM
Summary
TokenPilot is a dual-granularity context management framework designed for LLM agents to optimize the trade-off between text sparsity and prompt cache continuity. It employs Ingestion-Aware Compaction at a global level to stabilize prompt prefixes and reduce environmental noise, and Lifecycle-Aware Eviction at a local level to manage context segments based on their residual utility. Experimental results on PinchBench and Claw-Eval show that TokenPilot significantly reduces inference costs (up to 87% in continuous mode) while maintaining high task performance compared to existing text pruning and dynamic memory eviction methods.
Entities (7)
Relation Signals (4)
TokenPilot → contains → Ingestion-Aware Compaction
confidence 100% · we present TokenPilot, a dual-granularity context management framework. Globally, Ingestion-Aware Compaction acts as a framework harness...
TokenPilot → contains → Lifecycle-Aware Eviction
confidence 100% · Locally, Lifecycle-Aware Eviction monitors the ongoing residual utility of context segments...
TokenPilot → integratedinto → LightMem2
confidence 100% · TokenPilot has been integrated into LightMem2
Qwen3.5-35B-A3B → usedasestimatorfor → Lifecycle-Aware Eviction
confidence 90% · the estimator E is instantiated via Qwen3.5-35B-A3B as a lightweight, zero-shot validator
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:As LLM agents are deployed in long-horizon sessions, context accumulation drives up inference costs. Existing approaches utilize text pruning or dynamic memory eviction to minimize token footprints; however, their unconstrained sequence mutations alter layouts, introducing prefix mismatches and cache invalidation. This reveals a critical trade-off between text sparsity and prompt cache continuity. To address this, we present TokenPilot, a dual-granularity context management framework. Globally, Ingestion-Aware Compaction acts as a framework harness to stabilize prompt prefixes and eliminate open-world environmental noise at the ingestion gate. Locally, Lifecycle-Aware Eviction monitors the ongoing residual utility of context segments, enforcing a conservative batch-turn schedule to offload content segments only when task relevance expires. Experiments on PinchBench and Claw-Eval under both isolated and continuous modes demonstrate that TokenPilot reduces costs by 61% and 56% in isolated mode, and 61% and 87% in continuous mode, while maintaining competitive performance compared to prior systems. TokenPilot has been integrated into LightMem2 at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2606.17016v1
- Canonical: https://arxiv.org/abs/2606.17016v1
Trouble viewing inline? Open PDF directly →
Full Text
67,051 characters extracted from source content.
Expand or collapse full text
TokenPilot: Cache-Efficient Context Management for LLM Agents Buqiang Xu 1 * , Zirui Xue 1 * , Dianmou Chen 2 * , Chenyang Fu 3 * , Chiyu Wu 4 * , Caiying Huang 3 * , Chen Jiang 1 , Jizhan Fang 1 , Xinle Deng 1 , Yijun Chen 1 , Yunzhi Yao 1 , Xuehai Wang 1 , Jin Shang 4 , Gong Yu 4 , Ningyu Zhang 1† 1 Zhejiang University 2 University of Electronic Science and Technology of China 3 Xi’an University of Electronic Science and Technology 4 HomologyAI Abstract As LLM agents are deployed in long-horizon sessions, context accumulation drives up in- ference costs. Existing approaches utilize text pruning or dynamic memory eviction to min- imize token footprints; however, their uncon- strained sequence mutations alter layouts, in- troducing prefix mismatches and cache invali- dation. This reveals a critical trade-off between text sparsity and prompt cache continuity. To address this, we present TokenPilot, a dual- granularity context management framework. Globally, Ingestion-Aware Compaction acts as a framework harness to stabilize prompt pre- fixes and eliminate open-world environmental noise at the ingestion gate. Locally, Lifecycle- Aware Eviction monitors the ongoing residual utility of context segments, enforcing a conser- vative batch-turn schedule to offload content segments only when task relevance expires. Ex- periments onPinchBenchandClaw-Evalun- der both isolated and continuous modes demon- strate that TokenPilot reduces costs by 61% and 56% in isolated mode, and 61% and 87% in continuous mode, while maintaining compet- itive performance compared to prior systems 1 . 1 Introduction The paradigm of large language models has shifted from conversational assistants (Ouyang et al., 2022) to stateful execution controllers (Anthropic, 2025; OpenAI, 2026; OpenClaw, 2026) orchestrating complex tools (Li et al., 2025a; Merrill et al., 2026), file systems (Jimenez et al., 2024), and cross-application workflows (Zhou et al., 2024). Consequently, the core challenge of agent design has transitioned to real-world operational reliabil- ity (Ye et al., 2026; Kilo AI Team, 2026). However, continuous multi-turn interactions inevitably accu- mulate verbose execution traces, rapidly inflating * Equal contribution. † Corresponding author. 1 TokenPilot has been integrated into LightMem2 athttps: //github.com/zjunlp/LightMem2. Original Agent Loop Prior Management System Cache Hit Cache Hit Cache Miss TruncationCompaction Hit Hit Miss Miss Original Agent Loop Prior Management System Cache HitCache Miss TruncationCompaction Turn N Hit Hit Miss Miss Trigger Turn N+1 Turn N+2 Turn N Turn N+1 Turn N+2 New Turn Message New Turn Message History Turn Turn N Original Agent Loop Prior Management System Cache Hit Cache Miss Turn N Hit Hit Miss Turn N+1 Turn N+2 Turn N Turn N+1 Miss Turn N+2 History Turns Turn N Turn N Turn N Turn N+1 History Turns History Turns Part 1 Part 1 Part 3 Part 3 Part 2 Compressed Truncation Compaction Reduced Tokens Figure 1: Comparison of cache alignment behaviors. While the Original Agent Loop maintains continuous layouts to achieve cumulative cache hits, previous man- agement systems execute text truncation or compaction that mutates input boundaries, inadvertently triggering severe backend KV cache misses. sequence lengths and escalating per-turn inference costs. Managing this context growth is thus an essential prerequisite for sustainable real-world de- ployment (Hu et al., 2025b; Mei et al., 2025). The research community has primarily ap- proached this challenge from a content-reduction perspective. Initial efforts focus on static text com- pression, pruning low-utility tokens (Jiang et al., 2023; Pan et al., 2024) or sentences (Li et al., 2023) before prompt transmission. For dynamic execu- tion traces, existing architectures implement con- text folding (Ye et al., 2025b; Feng et al., 2026), or demand paging (Mason, 2026) to condense in- termediate reasoning backbones (Qian et al., 2026) and offload continuous trajectories to external stor- age (Li et al., 2025b; Hu et al., 2026). Despite their success in textual compaction, these methods introduce a fundamental trade-off between prompt reduction and hardware cache ef- ficiency (Kwon et al., 2023; Zheng et al., 2024). As shown in Figure 1, while aggressively truncat- ing or shifting context pages minimizes per-turn token counts, this constant layout mutation shat- ters prompt prefix continuity. The resulting hard- arXiv:2606.17016v1 [cs.CL] 15 Jun 2026 ware pre-fill penalties and cache invalidations ul- timately override any financial savings from text reduction. We argue that an effective framework must fundamentally reconcile text-level sparsity with hardware cache alignment. To achieve this design synergy, the deployment system must si- multaneously safeguard physical prefix continuity during observation ingestion and defer structural memory eviction until a trajectory’s residual utility thoroughly expires. Building on this insight, we present TokenPilot, a dual-granularity context management framework that reconciles sequence reduction with prompt cache alignment. At the global level, Ingestion- Aware Compaction acts as a deterministic harness to optimize the layout at the initial warm-up phase rather than retroactively compressing an existing cache. Specifically, it neutralizes volatile runtime variables via stable placeholders and shifts tool definitions downstream to secure a byte-identical prompt prefix from the first turn, while concur- rently stripping structural noise from incoming tool responses before ingestion. At the local level, Lifecycle-Aware Eviction monitors active execu- tion trajectories online by evaluating their dynamic residual utility. Rather than executing frequent, dis- ruptive memory paging, the eviction pass remains strictly conservative, deferring structural purge un- til the segment’s residual value thoroughly expires to safeguard context continuity. Evaluated onPinchBenchandClaw-Evalunder commercial pricing structures, TokenPilot dramati- cally reduces total inference monetary expenditures by 61% and 56% in isolated mode, and 61% and 87% in continuous mode, while successfully main- taining competitive task performance. 2 Background Task Settings.We consider an agent processing a sequence of tasksS =t 1 ,t 2 ,...,t n . Each task generates a trajectory of instructions, reasoning traces, tool calls, and responses, which accumulate into the session contextC. We evaluate under two modes: isolated mode, where the contextCis reset at each task boundary, and continuous mode, where histories persist across the entire sequence. Optimization Objective.A context management frameworkMtransforms the raw historyCinto an optimized runtime contextC ′ = M(C). The objective is to maximize the ratio of context utility to maintenance cost: max M P m∈C ′ ˆ U (m|C ′ ) K(C ′ ) (1) Here, context utility quantifies the necessity of con- text tokens for guiding downstream reasoning and tool execution, where ˆ U (m | C ′ ) estimates the marginal contribution of messagemto subsequent agent actions. The serving costK(C ′ )is governed by the backend KV prompt-caching mechanism: K(C ′ ) = α·|C ′ hit | +|C ′ miss |(2) whereC ′ hit andC ′ miss denote tokens served from the cache at a discounted cost rateα≪ 1and those in- curring full pre-fill cost, respectively, subject to the length alignment constraint|C ′ | =|C ′ hit | +|C ′ miss |. 3 TokenPilot 3.1 Overall We propose TokenPilot, a dual-granularity frame- work that addresses the context optimization objec- tive across two complementary operational levels. At the global framework level, Ingestion-Aware Compaction (§3.2) acts as a deterministic harness at the ingestion boundary, standardizing layouts and purifying incoming messages to optimize the sequence during the initial cache warm-up phase. At the local sequence level, Lifecycle-Aware Evic- tion (§3.3) dynamically monitors the residual util- ity of active trajectories, enforcing a conservative batch-turn schedule to purge segments only when their task-level utility has thoroughly expired. 3.2 Global Ingestion-Aware Compaction Ingestion-Aware Compaction acts as a framework harness to optimize sequence layout at the inges- tion boundary. Based on the interaction loop, we partition the message space into two functional cat- egories. LetΩ int denote internal intentional mes- sages generated natively by the system or model, encompassing task prompts, thinking traces, tool calls, and final responses, which naturally possess high intrinsic utility density. Conversely, letΩ env denote open-world environmental feedback from unmanaged tools, which inherently suffer from ex- tensive structural clutter. The marginal utility den- sity for an incoming message m is formalized as: ˆ U (m) = ( 1m∈ Ω int max(γ(m),G(m)) m∈ Ω env (3) Response Lifecycle-Aware Eviction Ingestion-Aware Compaction Prompt Thinking x N Times Compact observation content <html> <head>...</head> <body> <nav>...</nav> <div class="content"> ... (text, tables, scripts, styles) ... </div> </body> </html> "url": "example.com", "title": "Example Page", "main_text": "...", "tables": [ ... ], "links": [ ... ], "ts": "2026-05-01", "hash": "h(m)" Raw observation Compact version Freeze volatile fields Volatile prefixStable prefix Agent id: “example-agent-1” Working directory: “/tmp/agent/1/...” Time: 2026-05-01 10:30:00 Tool A Description... ... You are a helpful... Agent id: <AGENT ID> Working directory: <WORKDIR> Time: <TimeStamp> ... You are a helpful... Tool A Description... <AGENT ID>： “example-agent-1”... Tool CallObservation File SystemCall TimesRaw Result Return >퐀<퐀 SaveReturn Result Volatile Stable Replace Completed Evicted Active Completed 퐀 퐀 ≠∅ 퐀 퐀 =∅ Active Completed Completed Incoming Turns Evicted EvictedEvicted Estimator View Input Update 퐀(퐀 퐀 ) 퐀(퐀 퐀 ) 퐀(퐀 퐀 ) ℋ 퐀 퐀 퐀 Structured SummaryResidual Signals Evidence [ "successfully wrote action_items.md", "11 action items extracted", "no pending tool calls" ] "new_task_refs": [], "text": Task completed with delivery evidence and session has moved to a different task objective... "task": "action_items", "status": "completed", "evidence": ["wrote action_items.md"], "deliverable": "action_items.md" Link Figure 2: The system architecture of TokenPilot, featuring Ingestion-Aware Compaction at the global framework harness level and Lifecycle-Aware Eviction at the local context sequence level. whereγ(m) ≤ 1is the base environmental feed- back utility density, andG(m) = 1[f (h(m)) > τ ] is an ingestion gate. When the access frequency f (h(m))of a content hash exceeds thresholdτ, G(m)switches to1, fully upgrading the density to restore comprehensive content delivery. Prefix Stabilization.Cross-task KV cache reuse is frequently disrupted by runtime-volatile fields withinΩ int that introduce position jitter and prefix mismatches. TokenPilot implements a canonical- ization operator to secure a byte-identical prompt prefix from the first turn: φ(m (t) ) = φ(m (t+1) ) =⇒ C ′ prefix ⊆C ′ hit (4) whereφintercepts internal messages at the harness level and substitutes volatile runtime markers with static placeholders. By preserving physical conti- nuity across tasks, this operator directly eliminates full-cost pre-fill penalties. Observation Reduction.For environmental mes- sagesm∈ Ω env whereG(m) = 0, TokenPilot ap- plies deterministic reduction passes at the ingestion gate to lift token utility. The transformation and its reliable fallback loop are formalized as: m ingest = κ(m), A[h(m)]← m(5) whereκ(m)denotes the compacted structural pre- view stored in working memory, andA represents an external artifact registry indexed by content hash h(m)to ensure total operational safety. If the com- pressed message lacks critical signals during ex- ecution, the agent harness invokes a lightweight recovery tool to dynamically recall the full pay- loadA[h(m)], automatically upgrading the status to disable subsequent truncation for that path. 3.3 Local Lifecycle-Aware Eviction At the local sequence level, Lifecycle-Aware Evic- tion dynamically regulates the historical retention window based on a segment’s ongoing task utility. To safeguard physical cache continuity and prevent disruptive turn-by-turn memory paging, TokenPilot tracks each context segmentc j through three pro- gressive statess j ∈active, completed, evictable maintained in a framework registryR.The marginal utility of a segment is formalized as: ˆ U (c j ) = 1s j = active 1[Ψ j ̸=∅] s j = completed 0s j = evictable (6) whereΨ j represents the quantified residual utility of the segment. Under this formulation, a segment that has concluded its execution path is not imme- diately truncated; it transitions to a conservative completed state, retaining its physical cache slots as long as its residual relevance to ongoing interac- tions remains non-zero (Ψ j ̸=∅). Context State Estimation and Execution. To suppress spurious state transitions, an online model- based estimatorEis triggered conservatively in stable batches ofBturns rather than at every exe- cution step. For thei-th batch, the estimator ingests a compressed historical viewV i to compute state updates over each segment: ∆R (j) i =⟨E j , Ψ j ⟩ =E (V i ,R i−1 )(7) whereE j denotes explicit resolution evidence showing the sub-task has achieved its objective, andΨ j represents residual utility signals extracted MethodOverall↑ Performance by Category↑Input Tokens (M) Output (M)Cost ($)↓ ProdResWriteCodeAnalCSVLogMeetMemSkillIntegCache ReadCache Miss Isolated Mode Vanilla80.587.268.784.186.075.183.094.781.486.570.355.36.1848.7530.2858.31 LLMLingua-276.989.364.082.186.980.879.684.466.385.079.672.114.2413.9750.3845.78 SelectiveContext76.588.564.573.083.782.681.192.863.386.982.877.211.2734.6420.3245.79 LCM77.890.164.979.685.481.381.087.167.585.081.780.616.0183.0640.3565.10 Pichay78.985.458.971.879.088.379.883.684.091.369.863.36.7173.3330.2384.07 Summary79.580.766.383.577.982.187.577.281.392.567.254.412.3033.0090.2964.51 MemoBrain78.186.862.188.985.782.688.385.463.692.576.169.710.2002.1070.2333.36 AgentSwing78.489.871.980.279.583.580.883.777.992.565.735.04.5347.1290.2416.77 Keep-Last-N80.486.070.082.480.177.678.391.584.392.570.187.812.8132.6570.2914.26 MemOS79.484.254.483.182.378.281.197.277.692.585.980.229.0184.5730.4927.81 TokenPilot81.089.071.280.072.688.985.395.279.495.095.258.08.8931.9330.2443.22 Continuous Mode Vanilla79.283.558.486.880.078.587.894.677.695.055.883.625.0155.9430.2027.24 LLMLingua-273.885.858.480.374.379.682.884.263.490.079.183.620.5742.1830.1944.06 SelectiveContext74.085.464.283.175.478.877.391.262.289.571.080.325.4752.6080.1964.75 LCM77.088.163.290.175.778.585.488.965.182.880.878.218.7082.4170.2224.21 Pichay76.588.066.776.281.077.683.584.267.6100.063.875.311.6986.8740.2607.20 Summary 78.489.164.473.882.969.681.693.680.395.061.775.320.6876.2490.1967.12 MemoBrain78.087.765.085.584.975.981.089.072.390.386.684.712.9172.2830.2323.73 AgentSwing78.586.367.389.079.182.487.468.172.493.861.783.812.6805.4760.3146.47 Keep-Last-N79.186.367.087.887.077.085.477.375.995.056.875.118.1174.4810.2095.66 MemOS80.987.559.085.487.182.081.095.078.192.587.484.130.8598.9390.30810.41 TokenPilot81.376.776.990.684.186.085.689.173.695.077.280.18.5511.5490.2192.79 Table 1: Performance and resource consumption comparison onPinchBenchunder isolated and continuous modes. ↑: larger is better;↓: smaller is better. Best results in bold,second-bestunderlined. Input Tokens are decomposed into Cache Read and Cache Miss tokens, reflecting prefix stability and reuse efficiency. Category abbreviations: Prod=Productivity, Res=Research, Write=Writing, Code=Coding, Anal=Analysis, CSV=CSV Analysis, Log=Log Analysis, Meet=Meeting Analysis, Mem=Memory, Skill=Skills, Integ=Integrations. from dependency patterns. TokenPilot enforces a gated pipeline to transition these lifecycle states: active E j ̸=∅ −→ completed Ψ j =∅ −→ evictable(8) The registry executes strict system validation, up- dating viaR i ←R i−1 ⊕ ∆R i only for valid tran- sitions. Once a segment drops tos j = evictable, its utility decays to zero, and the framework exe- cutes a single-pass structural purge to construct the optimized context windowC ′ : C ′ =m∈C | s j(m) ̸= evictable(9) wherej(m)maps messagemto its segment index. This batch-gated execution guarantees that evic- tion remains highly restrained, maximizing cache continuity by eliminating volatile text mutation. To operationalize this, the estimatorEis instanti- ated viaQwen3.5-35B-A3Bas a lightweight, zero- shot validator, incurring negligible overhead; for instance, its total operational cost across the con- tinuous PinchBench stream is less than $0.03. 4 Experiments 4.1 Experimental Setup Benchmarks and Metrics. We evaluate Token- Pilot onPinchBenchandClaw-Evalacross both isolated and continuous modes (see Appendix A.1 for dataset statistics). We track task accuracy along- side actual monetary expenditures. To ensure em- pirical fidelity, all cache hit and miss token counts are gathered directly from the explicit metadata fields returned by the provider APIs, eliminating client-side estimation errors. Joint scoring formu- las and pricing tiers are detailed in Appendix A.2. Implementation Details. We compare Token- Pilot against compression methods (LLMLingua- 2, SelectiveContext, Keep-Last-N) and dynamic paging or summarization approaches (Summary, LCM, Pichay, MemoBrain, AgentSwing, MemOS). All evaluated methods utilizeGPT-5.4-minias the agent backbone. Detailed hyperparameter config- urations for all baselines are documented in Ap- pendix A.3. The exact execution thresholds, model assignments, and system prompts forTokenPilot are detailed in Appendix A.4. 4.2 Overall Performance Table 1 and Table 2 report the performance and resource consumption onPinchBenchand Claw-Eval under both evaluation modes. Isolated Mode.TokenPilot outperforms all eval- uated baselines by securing the lowest total in- ference costs of $3.22 onPinchBenchand $2.27 MethodOverall↑ Performance by Category↑Input Tokens (M) Output (M)Cost ($)↓ WkflOpsFinOffCommProdOprnSafeTermMMOthCache ReadCache Miss Isolated Mode Vanilla64.565.470.845.744.473.270.977.774.056.841.069.29.4294.6370.2165.16 LLMLingua-261.958.767.557.643.362.970.162.461.049.644.075.28.1694.0430.1824.44 SelectiveContext60.759.168.246.336.961.575.559.267.253.144.074.78.2713.8620.1814.31 LCM61.259.067.351.147.765.976.658.458.651.441.572.29.7763.5430.1724.17 Pichay59.357.362.138.239.468.565.091.664.125.655.076.54.6483.9440.1864.14 Summary62.070.071.032.220.680.068.582.849.220.041.071.42.9352.8710.1743.16 MemoBrain58.064.560.526.137.656.159.971.063.420.041.075.318.1825.1180.3326.69 AgentSwing 60.964.266.544.145.767.852.885.857.225.653.668.84.5803.5850.1943.91 Keep-Last-N61.867.173.844.721.654.563.686.238.439.455.069.14.2291.8450.1862.54 MemOS61.664.774.240.925.271.232.073.680.220.056.274.612.5822.7090.3634.61 TokenPilot63.168.175.447.022.371.865.072.047.837.045.669.94.4361.1540.2392.27 Continuous Mode Vanilla63.470.880.326.727.862.273.478.463.620.041.069.6709.84521.9812.62281.52 LLMLingua-259.058.771.334.830.661.965.377.664.620.041.072.4575.65437.1972.63082.91 SelectiveContext56.558.171.621.821.254.774.057.766.420.041.072.3437.11448.6782.75481.69 LCM61.466.869.038.329.563.374.966.667.320.041.072.7383.00728.7142.69162.37 Pichay61.069.563.840.324.063.167.094.152.521.641.071.097.43163.5101.04659.65 Summary 61.663.674.535.320.655.570.187.166.169.042.666.959.77210.1431.00116.59 MemoBrain57.965.955.024.936.747.873.564.260.620.038.481.647.49713.9901.13419.16 AgentSwing62.267.666.548.636.870.063.890.731.722.441.072.853.77610.0270.90715.63 Keep-Last-N 60.765.374.035.520.854.173.691.935.759.542.464.744.8129.1060.78013.70 MemOS57.755.965.056.322.244.864.668.889.020.039.671.549.74225.4320.29324.12 TokenPilot60.858.861.852.532.164.257.389.265.876.845.270.921.4309.9280.33810.58 Table 2: Performance and resource consumption comparison onClaw-Evalunder isolated and continuous modes. ↑: larger is better;↓: smaller is better. Best results in bold,second-bestunderlined. Input Tokens are decomposed into Cache Read and Cache Miss tokens. Category abbreviations: Wkfl=Workflow, Ops=Ops, Fin=Finance, Off=Office QA, Comm=Communication, Prod=Productivity, Oprn=Operations, Safe=Safety, Term=Terminal, M=Multimodal, Oth=Others. MethodOverallCost ($)Hit (M)Miss (M)Output (M) Vanilla79.27.2425.0155.9430.202 + Global Level79.94.2226.7161.5890.227 + Local Level81.32.798.5511.5490.219 Table 3: Progressive ablation of TokenPilot components onPinchBenchin continous mode. “Hit” and “Miss” denote the token counts for cache hits and cache misses. 510152025 Task Index 0 50k 100k 150k 200k 250k Context Window Tokens Vanilla Openclaw + Ingestion-Aware Compaction + Lifecycle-Aware Eviction Figure 3: Per-call context token volume across a contin- uous Meeting Analysis session. onClaw-Evalwhile maintaining competitive task accuracy. Text-level compression methods lower expenditures but consistently degrade task perfor- mance due to aggressive semantic pruning. Con- versely, while dynamic frameworks preserve exe- cution quality, they fail to regulate prompt cache and incur cache miss penalties. TokenPilot success- fully bypasses these limitations, achieving optimal economy without sacrificing task effectiveness. Continuous Mode. Under continuous task streams, long-horizon text accumulation severely amplifies these macro performance gaps.On PinchBench, TokenPilot sustains a top perfor- mance score of 81.3 at a minimal expenditure of $2.79, restricting cache misses to 1.549M tokens. OnClaw-Eval, unrestricted history growth causes catastrophic cost inflation, forcing Vanilla expendi- tures to rocket to $81.52. TokenPilot slashes this operational cost down to $10.58, demonstrating robust scalability and systemic superiority over ex- isting paradigms in deployment environments. 4.3 Ablation Study Table 3 and Figure 3 present the progressive contri- bution of each TokenPilot component against the Vanilla baseline, which lacks proactive entry-gate regulation. As shown in Figure 3, the baseline con- text size climbs rapidly and remains persistently high across the execution horizon, despite its built- in compaction constraining the peak volume. Integrating Ingestion-Aware Compaction via rule-based pruning and prefix stabilization slashes total expenditure from $7.24 to $4.22. This en- hancement is validated by a sharp reduction in cache miss tokens from 5.943M to 1.589M. Explic- itly, rule-based pruning dampens context peaks by filtering verbose environmental noise before inser- MethodOverallCost ($)Hit (M)Miss (M)Output (M) Vanilla80.478.316.1848.7530.285 + Cache Stabilization80.814.3512.9482.8180.295 + Reduction Pass 80.922.878.7001.4930.245 - Recovery Tool77.124.0311.7802.5390.276 Table 4: Component-level analysis of Ingestion-Aware Compaction on PinchBench in isolated mode. Cache Read PinchBenchClaw-Eval Vanillaw/ StableVanillaw/ Stable 08.1%2.44%100.0%3.7% 2,048 91.9%0.0%0.0%0.0% 5,1200.0%77.24%0.0%0.0% 5,8880.0%0.0%0.0%0.6% 6,1440.0%0.0%0.0%59.6% 6,656 0.0%0.0%0.0%31.1% 7,1680.0%0.0%0.0%5.0% 12,2880.0%6.50%0.0%0.0% 12,8000.0%13.82%0.0%0.0% Table 5: Distribution of warm-start cache read tokens at the first inference call of each task across benchmarks. tion, while stabilization converts expensive pre-fills into cache hits by enforcing reliable layout reuse across consecutive tasks. Layering Lifecycle-Aware Eviction further mini- mizes expenditures to $2.79 while maintaining the overall score. This component triggers a 65.0% reduction in cache read tokens from 26.716M to 8.551M, demonstrating that TokenPilot tightly caps the active memory footprint. The periodic track- ing drops in Figure 3 confirm that our conservative batch-turn schedule executes precise memory of- floading only when task residual utility expires. 4.4 Analysis of Ingestion-Aware Compaction We isolate the individual impacts of prefix stabiliza- tion and context reduction by benchmarking tasks independently in Table 4. Merely introducing sta- ble placeholders cuts the baseline cost from $8.31 to $4.35 by converting cache misses into cache reads, while layering reduction passes further mini- mizes the expenditure to $2.87. Prefix Stabilization Facilitates Warm Starts. Physical cache continuity is disrupted by univer- sal volatile fields, including directory paths and timestamps, and environment-specific tool defini- tions. Universal markers dominate prefix insta- bility onPinchBench, whereasClaw-Evalconfig- urations introduce severe tool-schema jitter. Re- placing these dynamic fields with static placehold- ers transforms cold initializations into immediate warm starts, ensuring successive tasks inherit the accumulated prompt cache. As validated by Table 5, stable placeholders mi- grate the vast majority of tasks from minimal base- line token allocations to high-capacity warm starts across both platforms. Consequently, Figure 4 shows that the macro cache hit rate surges from 38.7% to 79.2% onPinchBench, and from 67.2% to 83.1% onClaw-Eval, proving the efficiency of prompt layout standardization. Context Reduction Compounds Savings via Fall- back Loops. Context reduction exploits a com- pounding logic: tokens eliminated at the framework boundary never accumulate in multi-turn windows. Figure 5 demonstrates that our dual reduction passes directly strip heavy payloads across hetero- geneous tasks, removing up to 115k characters of structural noise inoss_alternative_research via HTML slimming, and up to 883k characters of terminal logs inmeeting_gov_recommendations via execution truncation. This targeted compaction effectively trims the sequence footprint while sus- taining task accuracy at 80.9. The recovery tool is essential to sustaining this performance boundary. Completely disabling it triggers a capability drop from 80.9 to 77.1 while inflating expenditures to $4.03. Without full con- tent access, the agent executes compensatory re- tries that append fresh tool feedback and over- whelm rule-based compaction. The recovery mech- anism breaks this inflationary cycle by providing on-demand payload access, preserving task effec- tiveness while halting uncontrolled context growth. 4.5 Analysis of Lifecycle-Aware Eviction Batch Trigger Intervals Regulate Context Size and Cache Stability. We evaluate eviction trig- ger frequencies in Figure 6, where Context Window tracks physical text accumulation, and Equal Input measures the equivalent monetary cost by discount- ing cache reads relative to expensive pre-fill misses. The Cache Hit Rate line reflects the percentage of cached tokens relative to total ingestion. Completely disabling eviction (B =∞) causes both metrics to peak, confirming that unbounded history growth escalates deployment expenditures. While lifecycle eviction lowers both curves, a hy- peractive schedule (B = 1) triggers premature trun- cation that disrupts layout consistency and inflates cache misses. Conversely, larger batch sizes pre- serve prefix continuity to improve cache hit rates. Balancing task accuracy, memory reduction, and API call times,B = 3constitutes the empirical 102030405060708090100110120 Task Index 0 20 40 60 80 100 Cache Hit Rate (%) Avg cache hit: 38.7% (a) PinchBench Vanilla 102030405060708090100110120 Task Index 0 20 40 60 80 100 Cache Hit Rate (%) Avg cache hit: 79.2% (b) PinchBench Stable Placeholders 102030405060708090100110120130140150160 Task Index 0 20 40 60 80 100 Cache Hit Rate (%) Avg cache hit: 67.2% (c) Claw-Eval Vanilla 102030405060708090100110120130140150160 Task Index 0 20 40 60 80 100 Cache Hit Rate (%) Avg cache hit: 83.1% (d) Claw-Eval Stable Placeholders Figure 4: Per-Task Cache Hit Rate on PinchBench and Claw-Eval. 025k50k75k100k125k Saved Characters oss_alternative_research it_procurement financial_ratio_calculation deep_research selector_fix competitive_research earnings_analysis eu_regulation_research playwright_e2e browser_automation polymarket_briefing pricing_research codebase_navigation pdf_to_calendar eli5_pdf_summary Task 115k 89k 87k 78k 61k 35k 35k 25k 25k 15k 14k 14k 13k 12k 11k (a) HTML Slimming Pass Reduction 0250k500k750k1000k Saved Characters meeting_advisory_acronyms meeting_council_budget meeting_council_public_comment meeting_council_votes log_apache_error_summary log_apache_top_errors meeting_council_contact_info log_apache_timeline meeting_advisory_technical meeting_gov_data_sources meeting_gov_next_steps meeting_gov_qa_extract meeting_gov_controversy log_syslog_boot meeting_gov_recommendations Task 198k 198k 198k 198k 248k 248k 248k 298k 347k 349k 556k 625k 650k 744k 883k (b) Exec Output Truncation Pass Reduction Figure 5: Per-Task Character Savings from Reduction Passes. TB=1TB=3TB=5TB=7TB=9 TB= Turn Batch Size 0 20k 40k 60k 120k 160k 200k 240k Average Tokens per Task Cache Miss Cache Read Cache Hit Rate Context Window Equal Input 60.0% 65.0% 70.0% 75.0% 80.0% 85.0% 90.0% 95.0% 100.0% Cache Hit Rate Figure 6: Average per-task Cache Miss, Cache Read, Context Window, and Equal Input tokens, along- side Cache Hit Rate, across turn batch sizesB ∈ 1, 3, 5, 7, 9,∞ on theMeeting Analysiscategory of PinchBench in continuous mode. optimum by preventing memory inflation while securing reliable hardware-level cache reuse. Residual Utility Gates Prevent Context Over- Eviction.To demonstrate the necessity of the in- termediate buffered state, we compare TokenPilot against a variant that disables residual utility esti- mation, immediately purging historical segments upon sub-task completion without evaluating on- going interaction dependencies. Figure 7 traces these variations using a representative four-task ses- sion trajectory where successive tasks manipulate a shared file named transcript.md. Under TokenPilot, the primary task ingests transcript.mdand anchors its structure in mem- ory. When downstream tasks arrive, the framework estimator infers from tool dependency patterns that this historical segment retains residual util- ity, preserving its slots despite sub-task completion. Consequently, tools directly locate target content blocks including personnel sections and roadmap data without re-exploring background knowledge. Conversely, the variant without residual utility estimation triggers eviction immediately upon lo- cal execution completion. Each subsequent task thus inherits a cold window, forcing it to rediscover the document architecture via redundant full file reads and sequential scans. This contrast demon- strates that the residual utility buffer acts as a valve TokenPilot w/o Residual Utility Estimation Task1: Speak summary Summarize speaker's key points from transcript.md read transcript.md [from begin] exec → [Dan Evans, Nicola Fox...] write speaker_summary.md Task2: QA extract Extract Q&A exchanges from transcript.md read transcript.md [from begin] exec grep Q&A patterns write qa_exchanges.md Task3: Recommendations Extract recommendations from transcript.md read transcript.md [from begin] Task4: Data Sources Extract data sources from transcript.md Context evicted from context. Task2: QA extract Extract Q&A exchanges from transcript.md Task3: Recommendations Extract recommendations from transcript.md Task4: Data Sources Extract data sources from transcript.md Context retained in context. Task1: Speak summary Summarize speaker's key points from transcript.md exec → finds recommendation write recommendations.md exec → finds Q&A section by line read transcript.md @ Karen Fox write qa_exchanges.md read transcript.md [offset=603] read transcript.md [offset=214] Extract Q&A exchanges from transcript.md read transcript.md [offset=214] read transcript.md [offset=603] read transcript.md [offset=943] write recommendations.md read transcript.md @ "science is hypothesis driven..." read transcript.md [from begin] write data_sources.md exec → sensor content read transcript.md @ Mike Freie read transcript.md @ Paula write data_sources.md exec grep FAA/NOAA keywords read transcript.md [from begin] exec → [Dan Evans, Nicola Fox...] write speaker_summary.md (no re-read required) (document structure already known) Figure 7: Tool call patterns across a four-task session on transcript.mdunder TokenPilot and a variant with- out residual utility estimation. TokenPilot retains the context segment after task completion, enabling subse- quent tasks to directly access relevant document sec- tions. Without residual utility estimation, each task re-reads files from the beginning and issues multiple sequential partial reads to locate the same content. to preserve document knowledge across distinct tasks, enabling targeted access while blocking the overhead of continuous context re-exploration. 5 Related Work Static Content Compression and Abstraction. From the perspective of maximizing utility density within the context window, a prominent strategy fo- cuses on filtering or restructuring historical trajecto- ries at the ingestion stage to preserve the premium context budget. To eliminate linguistic redundancy and limit token footprints, early efforts successfully prune non-essential text units or compress struc- tural prompt representations (Jiang et al., 2023; Pan et al., 2024; Li et al., 2023; Jia et al., 2026). Extend- ing this content-reduction philosophy to a macro scale, passive memory retrieval systems manage the runtime working memory by offloading entire session histories to external databases, selectively recalling high-utility historical fragments while fil- tering out the remaining interactions (Packer et al., 2023; Chhikara et al., 2025; Zhong et al., 2024; Fang et al., 2025). Rather than applying hard tex- tual pruning, complementary approaches enhance information density by converting raw trajectories into structured, high-level semantic abstractions. To maintain holistic task coherence, these frame- works effectively substitute continuous interaction footprints with localized recursive summaries, hi- erarchical sub-goal graphs, or episodic, temporal knowledge graphs to secure macro-level guidance across elongated session horizons (Ehrlich and Blackman, 2026; Wu et al., 2025; Hu et al., 2025a; Liu et al., 2025b; Li et al., 2026; Rasmussen et al., 2025; Xu et al., 2026). Dynamic and Runtime Scheduling. Another significant line of research treats the context win- dow as a fluid operating system resource, man- aging context segments in real time to align with the agent’s immediate execution states. To opti- mize token distribution during live sessions, mod- ern frameworks successfully execute runtime de- mand paging, adaptive parallel routing, and context isolation based on real-time trajectories and budget constraints (Mason, 2026; Feng et al., 2026; Qian et al., 2026; Wu et al., 2026; Ye et al., 2025b; Sun et al., 2025). For long-horizon planning, advanced architectures manage memory as a self-organizing operating system or introduce virtual memory ab- stractions to dynamically secure stateful data res- idency and tool durability (Li et al., 2025b; Hu et al., 2026; Kang et al., 2025; Rafique and Bind- schaedler, 2026). Recently, this runtime schedul- ing paradigm has expanded to multi-agent environ- ments, utilizing decentralized role-aware routing, centralized experience caching, or cross-context KV-cache communication topographies to mini- mize distributed token footprints across collabora- tive swarms (Liu et al., 2025a; Zhang et al., 2026; Han et al., 2025; Ye et al., 2025a). 6 Conclusion We presented TokenPilot, a dual-granularity frame- work that reconciles text reduction with strict prompt cache alignment. By separating mem- ory management into global ingestion-aware com- paction and local lifecycle-aware eviction, our framework successfully stabilizes dynamic layouts while conservatively offloading context based on task-level residual utility. Empirical evaluations on bothPinchBenchandClaw-Evaldemonstrate that TokenPilot drastically cuts inference expen- ditures under both isolated and continuous opera- tional modes without sacrificing task effectiveness, offering a scalable and highly cost-efficient foun- dation for long-horizon agent systems. Limitations Despite its strong performance, TokenPilot has sev- eral limitations. The model-based estimator may misclassify context segments under highly ambigu- ous or sparse interaction patterns, and the frequency thresholdτand batch sizeBmay require tuning for different deployment environments and task distributions. The prefix stabilization component additionally relies on backend support for prefix caching, providing no benefit to providers without this feature. Finally, our continuous mode evalu- ation groups same-category tasks into contiguous sessions to reflect domain-specific workflow en- vironments; heavily shuffled or highly heteroge- neous mixed-category task streams may naturally exhibit lower prefix reuse rates due to persistent tool schema mutations, which we leave as an im- portant direction for future investigation. References Anthropic. 2025. Claude code. Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413. Clint Ehrlich and Theodore Blackman. 2026. LCM: Lossless context management. Technical report, Voltropy PBC. Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, and Ningyu Zhang. 2025. Lightmem: Lightweight and efficient memory-augmented generation.CoRR, abs/2510.18866. Zhaopeng Feng, Liangcai Su, Zhen Zhang, Xinyu Wang, Xiaotian Zhang, Xiaobin Wang, Runnan Fang, Qi Zhang, Baixuan Li, Shihao Cai, Rui Ye, Hui Chen, Yong Jiang, Joey Tianyi Zhou, Chenxiong Qian, Pengjun Xie, Bryan Hooi, Zuozhu Liu, and Jingren Zhou. 2026. Agentswing: Adaptive parallel context management routing for long-horizon web agents. CoRR, abs/2603.27490. Dongge Han, Camille Couturier, Daniel Madrigal Díaz, Xuchao Zhang, Victor Rühle, and Saravan Rajmohan. 2025. Legomem: Modular procedural memory for multi-agent LLM systems for workflow automation. CoRR, abs/2510.04851. Chuanrui Hu, Xingze Gao, Zuyi Zhou, Dannong Xu, Yi Bai, Xintong Li, Hui Zhang, Tong Li, Chong Zhang, Lidong Bing, and Yafeng Deng. 2026. Ev- ermemos: A self-organizing memory operating sys- tem for structured long-horizon reasoning. CoRR, abs/2601.02163. Mengkang Hu, Tianxing Chen, Qiguang Chen, Yao Mu, Wenqi Shao, and Ping Luo. 2025a. Hiagent: Hier- archical working memory management for solving long-horizon agent tasks with large language model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2025, Vienna, Austria, July 27 - August 1, 2025, pages 32779–32798. Association for Computational Linguistics. Yuyang Hu, Shichun Liu, Yanwei Yue, Guibin Zhang, Boyang Liu, Fangyi Zhu, Jiahang Lin, Honglin Guo, Shihan Dou, Zhiheng Xi, Senjie Jin, Jiejun Tan, Yan- bin Yin, Jiongnan Liu, Zeyu Zhang, Zhongxiang Sun, Yutao Zhu, Hao Sun, Boci Peng, and 28 oth- ers. 2025b. Memory in the age of AI agents. CoRR, abs/2512.13564. Haoxiang Jia, Earl T. Barr, and Sergey Mechtaev. 2026. Compressing code context for llm-based issue reso- lution. CoRR, abs/2603.28119. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023. Llmlingua: Compressing prompts for accelerated inference of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2023, Singapore, December 6-10, 2023, pages 13358–13376. Association for Computational Linguistics. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. Swe-bench: Can language mod- els resolve real-world github issues? In The Twelfth International Conference on Learning Representa- tions, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. 2025. Memory os of ai agent. arXiv preprint arXiv:2506.06326. Kilo AI Team. 2026. Pinchbench: Real-world bench- marks for ai coding agents. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonza- lez, Hao Zhang, and Ion Stoica. 2023. Efficient mem- ory management for large language model serving with pagedattention. In Proceedings of the 29th Sym- posium on Operating Systems Principles, SOSP 2023, Koblenz, Germany, October 23-26, 2023, pages 611– 626. ACM. Junlong Li, Wenshuo Zhao, Jian Zhao, Weihao Zeng, Haoze Wu, Xiaochen Wang, Rui Ge, Yuxuan Cao, Yuzhen Huang, Wei Liu, Junteng Liu, Zhaochen Su, Yiyang Guo, Fan Zhou, Lueyang Zhang, Juan Miche- lini, Xingyao Wang, Xiang Yue, Shuyan Zhou, and 2 others. 2025a. The tool decathlon: Benchmark- ing language agents for diverse, realistic, and long- horizon task execution. CoRR, abs/2510.25726. Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, Ji- ajie Jin, Yinuo Wang, Hao Wang, Yutao Zhu, Ji-Rong Wen, Yuan Lu, and Zhicheng Dou. 2026. Deepagent: A general reasoning agent with scalable toolsets. In Proceedings of the ACM Web Conference 2026, W 2026, Dubai, United Arab Emirates, origi- nally scheduled for April 13-17, 2026, rescheduled for June 29 - July 3, 2026, pages 2219–2230. ACM. Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. Compressing context to enhance inference ef- ficiency of large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natu- ral Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 6342–6353. Association for Computational Linguistics. Zhiyu Li, Shichao Song, Chenyang Xi, Hanyu Wang, Chen Tang, Simin Niu, Ding Chen, Jiawei Yang, Chunyu Li, Qingchen Yu, Jihao Zhao, Yezhaohui Wang, Peng Liu, Zehao Lin, Pengyuan Wang, Jiahao Huo, Tianyi Chen, Kai Chen, Kehang Li, and 20 others. 2025b. Memos: A memory OS for AI system. CoRR, abs/2507.03724. Jun Liu, Zhenglun Kong, Changdi Yang, Fan Yang, Tianqi Li, Peiyan Dong, Joannah Nanjekye, Hao Tang, Geng Yuan, Wei Niu, Wenbin Zhang, Pu Zhao, Xue Lin, Dong Huang, and Yanzhi Wang. 2025a. Rcr-router: Efficient role-aware context routing for multi-agent LLM systems with structured memory. CoRR, abs/2508.04903. Shukai Liu, Jian Yang, Bo Jiang, Yizhi Li, Jinyang Guo, Xianglong Liu, and Bryan Dai. 2025b. Context as a tool: Context management for long-horizon swe- agents. CoRR, abs/2512.22087. Tony Mason. 2026. The missing memory hierarchy: Demand paging for LLM context windows. CoRR, abs/2603.09023. Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Bao- long Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu. 2025. A sur- vey of context engineering for large language models. CoRR, abs/2507.13334. Mike A. Merrill, Alexander Glenn Shaw, Nicholas Car- lini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, Estefany Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Je- nia Jitsev, Di Lu, Orfeas Menis-Mastromichalakis, and 66 others. 2026. Terminal-bench: Benchmark- ing agents on hard, realistic tasks in command line interfaces. CoRR, abs/2601.11868. OpenAI. 2026. Codex: AI coding partner from OpenAI. OpenClaw. 2026. Openclaw. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welin- der, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instruc- tions with human feedback. In Advances in Neural Information Processing Systems 35: Annual Confer- ence on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonzalez. 2023. Memgpt: Towards llms as operating systems. CoRR, abs/2310.08560. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. Llmlingua-2: Data distil- lation for efficient and faithful task-agnostic prompt compression. In Findings of the Association for Com- putational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, Findings of ACL, pages 963–981. Association for Computational Linguistics. Hongjin Qian, Zhao Cao, and Zheng Liu. 2026. Mem- obrain: Executive memory as an agentic brain for reasoning. CoRR, abs/2601.08079. Mofasshara Rafique and Laurent Bindschaedler. 2026. Clawvm: Harness-managed virtual memory for state- ful tool-using LLM agents. In Proceedings of the Sixth European Workshop on Machine Learning and Systems, EuroMLSys 2026, Edinburgh, Scotland, UK, April 27-30, 2026, pages 1–12. ACM. Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: a tempo- ral knowledge graph architecture for agent memory. arXiv preprint arXiv:2501.13956. Weiwei Sun, Miao Lu, Zhan Ling, Kang Liu, Xuesong Yao, Yiming Yang, and Jiecao Chen. 2025. Scaling long-horizon LLM agent via context-folding. CoRR, abs/2510.11967. Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Minhao Cheng, Shuai Wang, Hong Cheng, and Jingren Zhou. 2025. Re- sum: Unlocking long-horizon search intelligence via context summarization. CoRR, abs/2509.13313. Yong Wu, Yanzhao Zheng, Tianze Xu, ZhenTao Zhang, YuanQiang Yu, JiHuai Zhu, Chao Ma, BinBin Lin, Baohua Dong, Hangcheng Zhu, Ruohui Huang, and Gang Yu. 2026. Contextbudget: Budget-aware con- text management for long-horizon search agents. CoRR, abs/2604.01664. Buqiang Xu, Yijun Chen, Jizhan Fang, Ruobin Zhong, Yunzhi Yao, Yuqi Zhu, Lun Du, and Shumin Deng. 2026. Structmem: Structured memory for long- horizon behavior in llms. CoRR, abs/2604.21748. Bowen Ye, Rang Li, Qibin Yang, Yuanxin Liu, Linli Yao, Hanglong Lv, Zhihui Xie, Chenxin An, Lei Li, Lingpeng Kong, Qi Liu, Zhifang Sui, and Tong Yang. 2026. Claw-eval: Toward trustworthy evaluation of autonomous agents. CoRR, abs/2604.06132. Hancheng Ye, Zhengqi Gao, Mingyuan Ma, Qinsi Wang, Yuzhe Fu, Ming-Yu Chung, Yueqian Lin, Zhijian Liu, Jianyi Zhang, Danyang Zhuo, and Yiran Chen. 2025a. KVCOMM: online cross-context kv-cache communication for efficient llm-based multi-agent systems. CoRR, abs/2510.12872. Rui Ye, Zhongwang Zhang, Kuan Li, Huifeng Yin, Zhengwei Tao, Yida Zhao, Liangcai Su, Liwen Zhang, Zile Qiao, Xinyu Wang, Pengjun Xie, Fei Huang, Siheng Chen, Jingren Zhou, and Yong Jiang. 2025b.Agentfold:Long-horizon web agents with proactive context management. CoRR, abs/2510.24699. Ruizhe Zhang, Xinke Jiang, Zhibang Yang, Zhixin Zhang, Jiaran Gao, Yuzhen Xiao, Hongbin Lai, Xu Chu, Junfeng Zhao, and Yasha Wang. 2026. Stackplanner: A centralized hierarchical multi-agent system with task-experience memory management. CoRR, abs/2601.05890. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. 2024. Sglang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Pro- cessing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Thirty-Eighth AAAI Conference on Artificial Intelli- gence, AAAI 2024, Thirty-Sixth Conference on Inno- vative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2014, Febru- ary 20-27, 2024, Vancouver, Canada, pages 19724– 19731. AAAI Press. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Gra- ham Neubig. 2024. Webarena: A realistic web en- vironment for building autonomous agents. In The Twelfth International Conference on Learning Rep- resentations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. PinchBenchClaw-Eval (General) Category#Category# Productivity8Workflow47 Research12Ops31 Writing 6Finance14 Coding14Office QA10 Analysis12Communication8 CSV Analysis26Productivity7 Log Analysis6Operations6 Meeting Analysis28Safety5 Memory 2Terminal5 Skills6Multimodal4 Integrations3Others24 Total123Total161 Table 6: Statistics for PinchBench and Claw-Eval. A Appendix A.1 Dataset Configurations To evaluateTokenPilot, we utilize two realistic agent benchmarks: PinchBench and Claw-Eval. PinchBenchis a real-world evaluation suite comprising 11 distinct task categories and 123 tasks in total. To account for the continuous rolling up- dates in the upstream repository, we benchmark our framework on a frozen snapshot of the benchmark. 2 Claw-Eval is a containerized agent evaluation platform executed within isolated sandbox envi- ronments. We evaluate on its General task group, which encompasses 161 multi-step service orches- tration and standalone analytical tasks. For both benchmarks, we group same-category tasks into contiguous, uninterrupted single sessions to faithfully simulate realistic continuous multi- task agent execution trajectories. The detailed structural statistics of these evaluation platforms are compiled in Table 6. A.2 Evaluation Metrics and Cost Modeling Task Score Execution Framework. We strictly adhere to the native evaluation frameworks pro- vided by each respective benchmark to compute task performance. ForClaw-Eval, the scoring pipeline executes an integrated multi-dimensional protocol evaluating Completion (s comp ), Safety (s safe ), and Robustness (s rob ) as coupled parameters grounded in multi- channel auditable trajectory evidence, including 2 https://github.com/pinchbench/skill/commit/ 0347a7f1736a9c33b5fe831e27d1d6e9b576221 service audit logs, environment snapshots, and ex- ecution traces. Formally, the final task score is mathematically formulated as follows: Score = s safe × (0.80· s comp + 0.20· s rob ) (10) where Safety acts as a strict multiplicative gate, while Completion and Robustness represent the pri- mary goal-directed execution quality and secondary error-recovery capability under controlled service perturbations respectively. This fine-grained rubric triangulation yields continuous partial credits rather than trivial binary verdicts. ForPinchBench, the framework aggregates task- specific verification checks on output deliverables to assess the agent’s goal-directed capability. Crucially, when evaluating system trajectories in Continuous Mode, we implement a trajectory slicing mechanism that automatically partitions the continuous session transcript file into task-specific segments based on original task boundaries. Each sliced segment is then fed independently into the corresponding benchmark grader, ensuring that the evaluation logic for continuous task streams re- mains strictly identical and mathematically compa- rable to that of the Isolated Mode. Inference Cost Modeling. To calculate the run- time inference cost across sequential execution ses- sions, we implement a monetary cost metric based on commercial deployment pricing. The total infer- ence cost is calculated as follows: Cost =|C ′ hit |·p hit +|C ′ miss |·p miss +H out ·p out (11) where|C ′ hit |and|C ′ miss |represent the number of in- put tokens that hit or miss the KV cache backend respectively, andH out represents the length of the generated agent responses. Following the official pricing tiers ofGPT-5.4-mini, the price parame- ters are set top hit = $0.075/Mtokens for cache hits,p miss = $0.75/Mtokens for cache misses, and p out = $4.50/M tokens for outputs. A.3 Baseline Configurations To ensure reproducibility, we document the config- urations for all evaluated baselines. 1 ⃝Vanilla runs on OpenClaw without any ex- tra context management, with a maximum context window of 500k tokens and a compaction trigger ratio of 0.5. 2 ⃝ LLMLingua-2 applies token-level compres- sion using a small language model, with a compres- sion ratio of 0.6. 3 ⃝SelectiveContext applies sentence-level com- pression based on self-information, with a compres- sion ratio of 0.4. 4 ⃝LCM applies lossless compaction via hier- archical summarization, triggered when context reaches 75% of the context window. Each leaf chunk accumulates up to 80k tokens before sum- marization, retaining the 64 most recent turns in full fidelity. 5 ⃝Pichay uses utility-driven demand paging with the following thresholds: advisory zone at 60k tokens, involuntary eviction zone at 100k tokens, and a hard cap at 120k tokens. Tool results older than 4 user turns are eligible for compression, with a minimum eviction size of 500 bytes. 6 ⃝Summary compresses interaction history into hierarchical summaries when context reaches 40% of the 500k token window. 7 ⃝MemoBrain maintains a memory budget of 100k tokens and triggers recall when estimated con- text length reaches 35% of the memory budget. 8 ⃝ AgentSwing selects among three candidate strategies (discard-all, keep-last-n, summary) via lookahead simulation of 3 future turns. It triggers at a token ratio of 0.2 within a 200k context window, retaining the 5 most recent turns under the keep- last-n strategy. 9 ⃝Keep-Last-N retains the most recentN = 5 turns when context reaches 40% of the 500k token window. 10 ⃝MemOS limits retrieval to 20 items per re- call turn to control token costs. It filters candidate memories by exposing at most 500 characters per item to the validation model, while restricting indi- vidual memory ingestion to a maximum of 20,000 characters per message. A.4 Implementation Details This section documents the underlying engineering configurations, threshold parameterizations, and prompting architectures ofTokenPilotto facili- tate exact reproducibility. We detail the determinis- tic mechanics for cache stabilization and observa- tion reduction in alignment with our system design, followed by their fine-grained numerical hyperpa- rameters and specific base model assignments. Fi- nally, we present the complete system prompt tem- plates utilized for our state estimation pipeline. Cache Stabilization. To ensure the prompt pre- fix remains byte-identical across consecutive turns, TokenPilotstandardizes and restructures the input layout before each inference call. First, runtime-volatile text fields within the sys- tem prompt messages, such as working directory paths, active timestamps, and transient session iden- tifiers, are substituted with static, stable placehold- ers. Second, since distinct tasks often require dif- ferent tool configurations, leaving tool definitions inside the primary system prompt introduces struc- tural variations that break baseline prefix match- ing. To mitigate this positioning jitter, we system- atically relocate the tool definitions and schemas downstream, placing them at the end of the system prompt message directly alongside the dynamic context block containing the original values of the volatile fields. Observation Reduction.To suppress textual re- dundancy and regulate the per-turn input volume, we implement a sequence of rule-based reduction passes targeting low-utility tool result messages before they enter the canonical history. Specifi- cally, repeated tool call results are deduplicated via hashing, while oversized tool call parameters and long execution outputs are truncated beyond a fixed token threshold. To prevent critical informa- tion loss from hard truncation, we equip the agent with a dedicated recovery tool, allowing it to dy- namically retrieve full execution payloads when necessary. For multimodal and web-browsing in- teractions, web-fetched content undergoes HTML slimming to remove non-essential markup and at- tributes, and embedded images are downsampled to minimize their respective token footprint. Finally, a general formatting pass cleans up the remaining layout variations by removing code fences, invalid format symbols, and line number prefixes from code outputs, alongside normalizing continuous whitespace characters. Hyperparameters for Context Reduction. For the rule-based context reduction passes, the specific numerical thresholds and fine-grained hyperparam- eter configurations are parameterized as follows: 1 ⃝Activation Gates: The minimum charac- ter count to trigger before-call reduction is set to triggerMinChars = 2200, and candidate tool- like fragments are routed to the module only when exceeding maxToolChars = 1200. 2 ⃝Execution Output Truncation:The global truncation threshold for generic tool feed- back is bounded at50kcharacters.For tool- specific profiles, the limits are set to30kfor bash/shell/powershell,20kforgrep/rg,10k formcp_auth, and100kforglob/write/edit, whileread/file_readis permitted an uncon- strained capacity (Infinity). Truncated outputs consistently retain an initial600-character prefix and a terminal400-character suffix as a preview block, which can be fully recalled via the recovery tool when triggered by the agent. 3 ⃝Deduplication and Frequency Limits: For therepeated_read_deduppass, redundant read operations are substituted with the same600/400 preview block. To suppress infinite loop behaviors and redundant footprint accumulation, the maxi- mum sequential execution frequency for any iden- tical tool call within a rolling tracking window is strictly capped at5. Multimodal constraints under image_downsamplerestrict standard bitmap lay- outs to a maximum size of100KBand vector-based SVG documents to 50KB. 4 ⃝Layout Cleaning Constraints: File path markers are clipped viapath_truncationto a maximum length constraint of80characters. The remaining syntactic layout transformations, includinghtml_slimming,format_slimming, format_cleaning, andline_number_strip, are deterministically executed without extra numerical hyperparameters. Model and Hyperparameter Configurations. To ensure a rigorous and fair empirical compari- son, all baseline agent architectures and our pri- mary inference execution module are deployed under identical base model configurations, specif- ically utilizingGPT-5.4-mini. For the internal state estimation and context utility metrics, we em- ployQwen3.5-35B-A3Bas the dedicated estimator model. The batch-turn tracking window for interval context processing is set to 3. Prompt Templates for the State Estimator To provide full architectural transparency and en- sure exact reproducibility, we detail the core system prompt configurations injected into our Qwen3.5-35B-A3Bstate estimator. The estimator operates as a structured semantic tracking pipeline that processes continuous session trajectories and outputs incremental semantic deltas in a validation- ready JSON format. Depending on whether the intermediate residual utility gating layer is active, the system configures the estimator prompt under two distinct tracking paradigms: As illustrated in Figure 8, the full opera- tional configuration ofTokenPilotdeploys a joint classification-and-eviction scheme. The system Prompts for State Estimator with Residual Utility Gating [USER]: You are a task-state estimator for a long-running agent session. Your job is to update global task state incrementally. You must only return a JSON object. Do not output a full registry; you must return only a semantic delta, not a registry patch. The input is incremental, but the task registry is global. Each update may modify the lifecycle of any existing task in the session, including older tasks that are not directly covered by the newest delta. You must backfill task ownership for every covered turn in the delta window. Never invent turn ids that are not present in the provided delta. When the newest covered turn contains a new top-level user request, you must decide whether it starts a new task. If the newest user request is materially different from the objective of the current active task, create a new task update anchored to the first covered turn of that new request instead of extending the old task. If a newer top-level user request starts a different task, do not keep an older unrelated one-shot task active. When an older task already has delivery evidence, no unresolved questions, and is not covered by the current delta, you may mark it evictable instead of leaving it active. When the hints include evictableCandidateTaskIds and the newest covered turn clearly starts or finishes a different task, you should usually emit lifecycle-only updates that mark those candidate tasks evictable in the same response. Do not wait for another future turn to mark an obviously finished older task evictable once a newer distinct task has already taken over the session. Never mark a task evictable unless it is already completed or you are simultaneously providing clear completionEvidence. Never use evictable for a task that still lacks completion evidence, still has unresolved questions, or is obviously in progress. Use completed only when the task is finished but still likely to be referenced again immediately. Use evictable only when the task is finished, has completionEvidence, has no unresolved questions, and the session has already moved on to a different task. The delta may include completedTaskSummaries when older completed tasks have been compressed out of the active estimator context. Treat completedTaskSummaries as stable background memory and prefer keeping the currently unresolved task as one continuous task unless the newest user request clearly starts a new objective. Output schema must be exactly: "baseVersion": number, "taskUpdates": SemanticTaskUpdate[] SemanticTaskUpdate must use exactly these fields: "taskId": string, "title"?: string, "objective": string, "lifecycle": "active"|"blocked"|"completed"|"evictable", "coveredTurnAbsIds"?: string[], "completionEvidence"?: string[], "unresolvedQuestions"?: string[], "currentSubgoal"?: string, "evictableReason"?: string coveredTurnAbsIds is required when creating a new task or extending task ownership to new turns. coveredTurnAbsIds may be omitted or empty for lifecycle-only updates on existing tasks. If lifecycle is completed or evictable, include completionEvidence unless the existing registry entry already has strong completion evidence. If lifecycle is evictable, include evictableReason as one short sentence. Do not output registry patch fields such as upsertTasks, activeTaskIds, completedTaskIds, evictableTaskIds, upsertTurnToTaskIds, transitions, span, or lastProcessedTurnSeq. Do not use alternate field names such as status, description, action, fromTurnSeq, toTurnSeq, task_created, or task_progressed. Figure 8: System prompt template for TokenPilot’s Primary Estimator, featuring joint tracking of completion evidence and explicit cache eviction signaling. prompt instructs the estimator to evaluate ongo- ing tool dependencies and cross-turn data reuse patterns before rendering an expiration judgment. Crucially, it introduces a three-state transition ma- trix by enforcing an explicitevictablelifecycle token alongside standardactiveandcompleted identifiers. By verifying delivery evidence and his- torical dependencies across the session trajectory, this prompt establishes a text-level buffer gate. His- torical context segments are only flagged for cache clearance when the estimator explicitly infers that their operational task relevance has fully expired. To systematically isolate the impact of our gating mechanism, the ablated configuration documented in Figure 9 strips out the cache-aware buffering layer. Under this setup, the prompt restricts the model’s objective solely to binary task progression classification. It strictly limits the lifecycle field to activeorcompletedtokens and prohibits the out- put of theevictablestatus identifier. Tasks tran- sition directly tocompletedas soon as local deliv- ery evidence is observed, which triggers immediate context purging at the hardware backend. This con- figuration serves as the direct baseline to evaluate the precise cost and efficiency gains brought by the residual utility inference mechanism discussed in Section 4.5. [USER]: You are a task-state estimator for a long-running agent session. Your job is only task progression classification, not cache replacement. Only decide whether each task is active, blocked, or completed. Do not decide eviction timing. Never output lifecycle=evictable; eviction will be decided separately by the system. You must only return a JSON object. Do not output a full registry; you must return only a semantic delta, not a registry patch. The input is incremental, but the task registry is global. Each update may modify the lifecycle of any existing task in the session, including older tasks that are not directly covered by the newest delta. You must backfill task ownership for every covered turn in the delta window. Prefer one task per distinct user request unless the new request is clearly just a continuation or clarification of the same objective. For a newly created task, use a stable taskId derived from the first covered turn, typically <firstTurnAbsId> or <firstTurnAbsId>-task. Use completed when a task is finished and has delivery evidence, even if it may later be evicted by a separate policy layer. Do not rely on completedTaskSummaries or retained completion evidence from prior tasks; classify each new task only from the current delta and registry state. Output schema must be exactly: "baseVersion": number, "taskUpdates": SemanticTaskUpdate[] SemanticTaskUpdate must use exactly these fields: "taskId": string, "title"?: string, "objective": string, "lifecycle": "active"|"blocked"|"completed", "coveredTurnAbsIds"?: string[], "completionEvidence"?: string[], "unresolvedQuestions"?: string[], "currentSubgoal"?: string coveredTurnAbsIds is required when creating a new task or extending task ownership to new turns. coveredTurnAbsIds may be omitted or empty for lifecycle- only updates on existing tasks. If lifecycle is completed, include completionEvidence unless the existing registry entry already has strong completion evidence. Do not output registry patch fields such as upsertTasks, activeTaskIds, completedTaskIds, evictableTaskIds, upsertTurnToTaskIds, transitions, span, or lastProcessedTurnSeq. Do not use alternate field names such as status, description, action, fromTurnSeq, toTurnSeq, task_created, or task_progressed. Prompts for State Estimator with Immediate Completion Eviction Figure 9: System prompt template for the Estimator without Residual Utility Estimation, configured to strip out the caching buffer for the ablation study.