Paper deep dive

SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning

Sanjay Kariyappa, G. Edward Suh

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 53

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 7/20/2026, 10:14:11 AM

Summary

SideQuest is a novel framework for managing Key-Value (KV) cache in Large Reasoning Models (LRMs) during long-horizon agentic tasks. It addresses the memory and performance bottlenecks caused by rapidly growing context in multi-hop reasoning by using the model itself to perform KV cache compression. SideQuest executes this compression as an auxiliary task in parallel to the main reasoning thread, preventing management tokens from polluting the primary context. The approach leverages a training methodology using hindsight analysis on reasoning traces to teach the model to identify and evict stale information, achieving up to 65% reduction in peak token usage with minimal accuracy degradation compared to heuristic-based methods.

Entities (12)

SideQuest · method · 98%G. Edward Suh · person · 95%NVIDIA · organization · 95%Sanjay Kariyappa · person · 95%KV Cache Compression · technique · 92%Large Reasoning Model · modeltype · 90%ReAct Framework · framework · 90%BrowseComp · dataset · 88%FRAMES · dataset · 88%H2O · method · 85%Scissorhands · method · 85%SnapKV · method · 85%

Relation Signals (10)

G. Edward Suh → affiliatedwith → NVIDIA

confidence 95% · Sanjay Kariyappa 1 G. Edward Suh 1 NVIDIA

Sanjay Kariyappa → affiliatedwith → NVIDIA

confidence 95% · Sanjay Kariyappa 1 G. Edward Suh 1 NVIDIA

SideQuest → reduces → peak token usage

confidence 95% · SideQuest reduces peak token usage by up to 65% on agentic tasks

SideQuest → uses → Large Reasoning Model

confidence 95% · SideQuest -- a novel approach that leverages the Large Reasoning Model (LRM) itself to perform KV cache compression

SideQuest → employs → ReAct Framework

confidence 90% · SideQuest integrates directly with the ReAct framework

SideQuest → evaluatedon → FRAMES

confidence 90% · We report our metrics on 424 samples from this dataset [FRAMES]

SideQuest → evaluatedon → BrowseComp

confidence 90% · BrowseComp (Wei et al., 2025). This dataset tests the ability...

SideQuest → outperforms → SnapKV

confidence 90% · outperforming heuristic-based KV cache compression techniques... SnapKV

SideQuest → →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Long-running agentic tasks, such as deep research, require multi-hop reasoning over information distributed across multiple webpages and documents. In such tasks, the LLM context is dominated by tokens from external retrieval, causing memory usage to grow rapidly and limiting decode performance. While several KV cache compression techniques exist for long-context inputs, we find that existing heuristics fail to support multi-step reasoning models effectively. We address this challenge with SideQuest -- a novel approach that leverages the Large Reasoning Model (LRM) itself to perform KV cache compression by reasoning about the usefulness of tokens in its context. To prevent the tokens associated with this management process from polluting the model's memory, we frame KV cache compression as an auxiliary task executed in parallel to the main reasoning task. Our evaluations, using a model trained with just 215 samples, show that SideQuest reduces peak token usage by up to 65% on agentic tasks with minimal degradation in accuracy, outperforming heuristic-based KV cache compression techniques.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

52,107 characters extracted from source content.

Expand or collapse full text

SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning Sanjay Kariyappa 1 G. Edward Suh 1 NVIDIA skariyappa, esuh@nvidia.com Abstract Long-running agentic tasks, such as deep research, require multi-hop reasoning over information dis- tributed across multiple webpages and documents. In such tasks, the LLM context is dominated by tokens from external retrieval, causing memory usage to grow rapidly and limiting decode per- formance. While several KV cache compression techniques exist for long-context inputs, we find that existing heuristics fail to support multi-step reasoning models effectively. We address this challenge with SideQuest – a novel approach that leverages the Large Reasoning Model (LRM) it- self to perform KV cache compression by rea- soning about the usefulness of tokens in its con- text. To prevent the tokens associated with this management process from polluting the model’s memory, we frame KV cache compression as an auxiliary task executed in parallel to the main reasoning task. Our evaluations, using a model trained with just215samples, show that Side- Quest reduces peak token usage by up to65% on agentic tasks with minimal degradation in ac- curacy, outperforming heuristic-based KV cache compression techniques. 1. Introduction The evolution of Large Language Models (LLMs) from static conversational interfaces to autonomous agents has catalyzed a shift toward long-running, multi-hop reasoning tasks. In applications such as deep research, automated software engineering, and complex workflow orchestration, models must synthesize information distributed across nu- merous retrieved documents and intermediate thought traces. This shift has fundamentally changed the memory profile of LLM inference; rather than processing a single prompt, agents maintain a growing context that can span hundreds of thousands of tokens over the course of a single task. What’s the first long weekend after GTC 2026 ? <think> We need the dates for GTC 2026. Let’s search <think> <tool> browser.search(“GTC 2026 dates”)</tool> [Cursor 0] Search results: “GTC 2026 dates” [ID 0†] AI Conference https://nvidia.com/gtc [ID 1†] Conference Schedule.. https://w.nvidia.com/gtc/conference.. <think> open second link<think> <tool> browser.open(cursor:0, id:1)</tool> [Cursor 1] GTC 2026 Schedule at a Glance ..Join us in San Jose, CA, on March 16-19.. <think> okay, GTC is on March 16-19. Next, we need to find the list of long weekends in 2026. Let’s search.. <think> **Memory management mode** Open cursors: [0,1]. The GTC dates are mentioned in [1]. We don’t need the search results page. <think> del_cursors: [0] [Cursor 2] Search results: “Long weekends 2026”... Main Thread Auxiliary Thread a b c d 粒 . . NVIDIA GTC runs from Mar 16-19 [1]. The first federal holiday long weekend after the conference is Memorial Day weekend, which falls on May 23 - 25, 2026 [8] . 푡 ! 푡 " 푡 # 푡 $ 푡 % 푡 & 푡 ' 푡 ( 푡 푢푡푖푙푖푡푦 푡 푢푡푖푙푖푡푦 Figure 1. Walkthrough example of SideQuest. (a) The main thread processes the user request by performing multi-turn reasoning and tool calling. (b) At regular intervals we spawn an auxiliary thread that runs in parallel with the shared context (c) The auxiliary thread reflects on the context and lists the cursors that can be deleted. (d) We clear the messages in the context by invoking a tool, reducing the context size for future turns. The primary bottleneck for scaling these agentic workloads is the Key-Value (KV) cache. The KV cache grows linearly with sequence length, posing two critical challenges for effi- cient inference. First, it consumes a significant and increas- ing fraction of GPU memory, which reduces the effective batch size and limits the number of concurrent requests a system can service. Second, as the cache grows, the atten- tion mechanism becomes increasingly memory-bandwidth bound, as the GPU must load a massive volume of KV tensors to generate a single new token (Fu, 2024). 1 arXiv:2602.22603v2 [cs.AI] 27 Feb 2026 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning To alleviate these pressures, the community has introduced KV cache compression, which aims to bound the memory footprint by retaining only a subset of the most important tokens in the context. Existing approaches largely rely on fixed heuristics to decide which entries to evict. For in- stance, techniques likeH 2 O(Zhang et al., 2023) and Scis- sorhands (Liu et al., 2023) identify Heavy Hitters—tokens that consistently receive high attention scores—and pre- serve them while pruning others. Other methods, such as SnapKV (Li et al., 2024), automatically identify and retain clusters of key information within local attention windows to preserve essential context. These methods operate on the premise that a token’s past importance is a reliable proxy for its future relevance. However, we observe that these existing techniques are ill- suited for the dynamic nature of agentic reasoning. Current KV cache compression methods are largely designed for settings where a fixed long-context corpus (e.g., a book or a legal document) is queried by a user. In contrast, agentic workloads involve a context that evolves over time as the model performs multiple rounds of tool-calling and internal reflection. In these scenarios, heuristic-based pruning is often too blunt a tool; a low-importance token in an early reasoning step may suddenly become critical for a synthesis step ten turns later. The imperfect nature of these heuristics can lead to the premature pruning of important information, resulting in reasoning failures that are difficult to debug. In this paper, we propose SideQuest, a novel approach that shifts KV cache management from a fixed heuristic to an in- telligent, model-driven process. SideQuest equips the model with a mechanism to selectively clear segments of its own KV cache, effectively allowing the model to perform its own memory garbage collection. To ensure that the over- head of this self-management does not interfere with the primary task, we frame KV cache compression as an auxil- iary task executed in parallel to the main reasoning process (Fig. 1c). This architecture prevents the management tokens from polluting the model’s primary attention window while maintaining a lean, high-fidelity KV cache. We make the following key contributions in this paper. • Evaluation of Heuristic-Based Methods: We measure the efficacy of heuristics based KV cache compression techniques on multi-step agentic workloads to show that static importance metrics fail to capture the dynamic, non- monotonic utility of tokens in multi-step reasoning. • Model-Driven Memory Management: We introduce SideQuest, a novel framework that empowers Large Rea- soning Models (LRMs) to actively manage their own con- text. By leveraging the model’s semantic understanding of the task state, SideQuest performs self-referential KV cache eviction to remove obsolete information. •Low-Overhead Parallel Reasoning:We propose a shared-context parallel reasoning architecture that executes auxiliary tasks—such as memory manage- ment—concurrently with the main reasoning thread. This design allows for intelligent intervention without polluting the primary context window with management tokens. • Training Methodology: We develop a scalable data syn- thesis pipeline that uses hindsight analysis on reasoning traces to generate high-quality supervision. This allows us to train the model to recognize stale information without requiring expensive human annotation. We demonstrate that SideQuest achieves up to 60% reduc- tion in KV cache usage on complex agentic benchmarks with negligible degradation in accuracy, establishing a supe- rior efficiency-utility trade-off compared to existing heuris- tic based techniques. 2. Related Work To mitigate the computational and memory overhead of long-context inference, the community has proposed various architectural and algorithmic optimizations. We provide a brief overview of these techniques in this section. Additional related work can be found in Appendix C 2.1. Architectural Optimizations Architectural innovations aim to reduce the inherent size of the KV state at the model-design level. Grouped-Query Attention (GQA) (Ainslie et al., 2023) and Multi-Head La- tent Attention (MLA) (Liu et al., 2024) reduce memory footprint by sharing key and value heads across multiple query heads, significantly lowering the per-token storage cost. Furthermore, alternatives to the standard attention layer, such as Mamba (Gu & Dao, 2024) and Gated Delta Layers (Yang et al., 2024), have been proposed to achieve sub-linear memory growth. These are often interleaved with standard attention layers to create hybrid models (e.g. Qwen3-Next (Qwen-Team, 2025), Nemotron-3 (NVIDIA, 2025)) that balance long-range dependency modeling with inference efficiency. While these methods reduce the rate of memory consumption, they do not address the problem of managing the historical context that accumulates during multi-turn agentic reasoning. 2.2. Sparse Attention Techniques Sparse attention aims to reduce the memory bandwidth bot- tleneck by attending to only a subset of tokens in the context. For instance, DeepSeek Sparse Attention (DSA) (Liu et al., 2025) utilizes a lightning indexer to dynamically identify and attend to the most relevant tokens for a given query. While such techniques effectively reduce the pressure on memory bandwidth during the attention computation, they typically do not reduce the physical storage requirements of 2 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning the KV cache; the full context remains resident in memory, even if only a fraction is accessed during decode. 2.3. KV Cache Management and Eviction Closest to our work are KV cache eviction techniques that prune unimportant entries to bound the memory footprint. StreamingLLM andH 2 O(Zhang et al., 2023) utilize static heuristics, such as keeping attention sinks or heavy hitter tokens based on cumulative attention scores. SnapKV (Li et al., 2024) further optimizes this by identifying key infor- mation clusters within an attention window. Recent work has extended these efforts to specifically target reasoning models. R-KV (Cai et al., 2025) leverages the notion of redundancy to determine which reasoning tokens can be safely discarded. RaaS (Hu et al., 2025) introduces the concept of milestone tokens—intermediate steps critical for logical progression—and prioritizes their retention. Sim- ilarly, LazyEviction (Zhang et al., 2025b) selectively retains thought tokens that demonstrate recurrent importance dur- ing the reasoning process, while ThinKV (Ramachandran et al., 2025) employs a hierarchical approach, categorizing tokens to guide selective quantization and pruning based on their importance levels. Crucially, however, these reasoning- focused methods predominantly target single-step Chain- of-Thought (CoT) tasks, where the context is static and the goal is a single final answer. These methods face a fundamental limitation in agentic set- tings: heuristic rigidity. In multi-step agentic workflows, a token’s importance is highly dynamic. A piece of in- formation that appears irrelevant in turnt(resulting in low attention scores) may become the critical pivot for turnt +n. Because existing heuristics lack semantic awareness of the agent’s high-level goals and evolving state, they risk irre- versibly pruning information that is vital for downstream reasoning steps. 3. SideQuest SideQuest is designed for agentic workloads that require multi-step reasoning to resolve complex user queries. These tasks typically employ the ReAct framework, where the model interleaves reasoning traces (thought) with external actions (tool calls) to progressively gather information. To illustrate the memory profile of such tasks, consider the example in Fig. 1. Given a user query about “long weekend after GTC 2026”, the model must perform multiple distinct steps: determining the submission date, finding a calendar of holidays, and synthesizing the final answer. Each step generates tool outputs—such as search results or webpage content—that are appended to the context. Goal. Our primary objective is to identify stale tool calls and associated responses that have lost their utility and evict associated tokens from context as early as possible during the execution of the ReAct agent 1 . Why is this challenging? The utility of a tool response is highly dynamic and non-monotonic, making it difficult to track with simple heuristics. For example, in Fig. 1, the initial search results (Cursor 0) become obsolete the mo- ment the model successfully retrieves the specific webpage containing the conference dates (Cursor 1). It can be safely deleted immediately. In contrast, Cursor 1 undergoes a more complex transition. It is critical to identify the dates of the conference (Mar 16-19), but it becomes temporarily irrel- evant during the subsequent search for “long weekends”. However, unlike the search results, Cursor 1 regains utility at the end of the task when the model must cite its sources in the final response. Standard attention-based heuristics often fail to capture these semantic shifts, leading to either the retention of useless noise (wasteful) or the premature eviction of useful information (harmful). How SideQuest solves this challenge. Instead of relying on proxy metrics like attention scores, SideQuest leverages the inherent reasoning capabilities of the LRM itself. By analyzing the current state of the ReAct loop and the specific problem definitions, the model explicitly determines which tool responses are no longer needed. 3.1. Overview SideQuest integrates directly with the ReAct framework to provide a mechanism for self-referential context manage- ment. As shown in Figure 1, the architecture operates by periodically spawning an auxiliary thread that executes in parallel with the main thread on the same shared context. The workflow proceeds as follows: 1.Parallel Execution: At regular intervals, SideQuest forks the generation process (1b). The auxiliary thread analyzes the history of open tool outputs (e.g., Cursors [0, 1]) relative to the current reasoning step. 2.Staleness Reasoning: The model produces a dedicated reasoning trace to determine which inputs are redundant. In the example, it recognizes that because the conference dates were successfully found in Cursor 1, the search results page (Cursor 0) is no longer required. 3. Eviction: The auxiliary thread outputs a structured com- mand, such asdelcursors: [0], flagging spe- cific KV cache entries for removal. 4.Synchronization: To prevent disrupting the main thread’s generation, the system waits for the current turn of the main thread to complete before evicting the tool 1 We only focus on tool calls and associated responses (referred collectively as “tool response”) as they occupy a significant fraction of the context in the benchmarks that we consider. 3 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning responses that are marked for deletion. A more formal description of SideQuest’s operation is pro- vided in Algorithm 1. Algorithm 1 Operation of SideQuest Require:User QueryQ, ModelM, Trigger IntervalK, Trigger Phrase p 1: Initialize contextC ← [Q] 2: Initialize turn counter t← 0 3: Initialize auxiliary threadS thread ← None 4: while True do 5: // Phase 1: Eviction 6:ifS thread ̸= None andS thread .is finished() then 7:∆ ids ←S thread .getoutput() 8: C.clearkv(∆ ids )// Prune stale tool outputs from cache 9: S thread ← None 10:end if 11: // Phase 2: Auxiliary Thread Spawning 12:if t (mod K) = 0 andS thread = None then 13: C aux ←C + p // Append trigger phrase 14: S thread ← ASYNCGENERATE(M,C aux ) 15:end if 16: // Phase 3: Main ReAct Execution 17: R←M(C) 18: C ←C + R 19:if R contains FINAL ANSWER then 20:return R 21:end if 22:if R contains Tool Call T then 23: O ← EXECUTETOOL(T ) 24: C ←C + O 25:else 26:break //End of turn if no tool called 27:end if 28: t← t + 1 29: end while Advantages of Parallel Execution. We chose a parallel thread architecture over a sequential approach (performing the user and memory management task in the main thread) for two reasons. First, interleaving memory management steps into the main thread would significantly increase la- tency, delaying the primary task. Second, explicit manage- ment tokens would pollute the context window, partially negating the benefits of compression. By isolating this logic in a transient parallel thread, SideQuest maintains a pristine context for the main agent. 3.2. Steering the Model to Perform Auxiliary Task In standard multi-turn reasoning, models are conditioned to solve the user’s initial prompt. SideQuest requires a mechanism to override this behavior in the auxiliary thread, shifting the model’s focus from answering the user’s query to managing the context. We achieve this through a two- pronged steering strategy: 1.Contextual Trigger: We append a distinct trigger phrase, ** Memory management mode ** , to the start of the auxiliary thread. This serves as a strong indicator to the model that it has entered a maintenance subroutine. 2.Task-Specific Fine-Tuning: We train the model to rec- ognize this trigger and output the expected deletion com- mands. This ensures that even when the main context contains complex reasoning about the user’s query, the presence of the trigger successfully switches the model’s output distribution to focus on analyzing the utility of tool responses. 3.3. Generating Training Data To equip the model with memory management capabilities without degrading its general reasoning performance, we construct a hybrid training dataset composed of two types of data: main traces (for preserving capability) and auxiliary traces (for learning eviction). The complete pipeline is formalized in Algorithm 2. Hindsight Annotation. The core of our data generation relies on hindsight analysis. We start by running inference on a dataset of web-browsing tasks using a base policy (the original model). We filter for traces that lead to correct answers to ensure high-quality supervision. For every tool output (cursor) in a correct trace, we compute its last-use index (Line 9). A cursor is considered “expired” at any given turn if it is never referenced again in the future either by tool calls or the final answer 2 . Main Traces (Distillation). To prevent catastrophic forget- ting of the model’s primary reasoning skills, we include the original correct traces in the training set. For these samples, we extract the logits from the base policy (Line 10). Dur- ing training, we apply a logit distillation loss (Hinton et al., 2015) to these entries. This forces the SideQuest model to match the probability distribution of the original model, en- suring that the introduction of memory management tokens does not alter the model’s behavior on standard tasks. Auxiliary Traces (Cross-Entropy). We synthesize aux- iliary traces to teach the model how to identify stale tool responses. At fixed intervals along the trace (Line 15), we identify the set of expired cursors. To simulate realistic in- ference conditions where the cache is partially compressed, we randomly partition these expired cursors into two sets. •Simulated Eviction: We mask out the tokens correspond- 2 This is a simplifying assumption to help with annotation. In reality it is possible that the model uses the information in a tool response without referencing the cursor in some cases. 4 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning ing to evicted cursors (Line 16), presenting the model with a context where some eviction has already occurred. •Target Generation: We prompt an annotation model (Line 19) to generate reasoning that explains why the remaining cursors are stale, followed by the requisite deletion command. The prompt used for annotation is provided in Appendix A. We prepend the trigger phrase to these sequences and use them to train the model using standard Cross-Entropy loss. Joint Optimization. The final model is trained on the union of these datasets using a weighted average of the two objectivesL = L CE (D aux ) + λL distill (D main ). This joint optimization ensures the model learns to enter the auxiliary mode strictly when triggered, while retaining the robust reasoning capabilities of the base model for the main task. Algorithm 2 Training Data Pipeline Require:DatasetD, base policyπ, annotation modelM, interval k, trigger phrase p 1: D train ←∅ 2: for each task x∈D do 3: τ ← RUNINFERENCE(π,x) 4:if ISCORRECT(τ ) = false then 5:continue 6:end if 7: // Annotate each cursor with its last-use turn index 8:for each cursor c in τ do 9: ℓ c ← LASTUSEINDEX(τ,c) 10:end for 11: // Add main trace with full attention and logits 12:z← π(τ ) // Extract logits from base policy 13: D train ←D train ∪(τ, I, z,main) 14: // Generate auxiliary traces at intervals 15:for each reasoning turn t where t mod k = 0 do 16: C expired ←c : ℓ c < t // Cursors past last-use 17: C open ,C closed ← RANDOMPARTITION(C expired ) 18:M ← BUILDMASK(τ 1:t ,C closed ) // Mask out closed cursors 19: r ←M(τ 1:t ,C open ,C closed ) // Generate reasoning 20: r ← p⊕ r // Prepend trigger phrase 21: τ aux ← τ 1:t ⊕ [r, CLOSEACTION(C open )] 22: D train ←D train ∪(τ aux , M,∅,aux) 23:end for 24: end for 25: returnD train 3.4. Overheads of SideQuest Storage Costs. The auxiliary thread operates on the shared context of the main thread, meaning it does not duplicate the large KV cache of the conversation history. The only additional storage cost arises from the transient tokens gen- erated during the auxiliary reasoning phase. Once the thread concludes and the deletion targets are identified, these to- kens are discarded, ensuring that SideQuest introduces zero permanent token overhead to the main context. Compute and Memory Movement Cost. Executing an auxiliary thread naturally incurs additional computational and memory bandwidth overhead. However, long-context agentic workloads are predominantly bottlenecked by mem- ory bandwidth rather than compute capacity. Although the auxiliary thread imposes a transient cost, we demon- strate empirically that this investment yields a net reduc- tion in resource consumption. By proactively pruning vo- luminous tool responses—often spanning hundreds of to- kens—SideQuest significantly reduces the cumulative mem- ory movement required for all subsequent reasoning turns. Furthermore, this overhead can be minimized by leveraging optimizations for shared-context inference. Techniques such as Cascade Inference (Ye et al., 2024) and FastTree (Pan et al., 2025) introduce specialized kernels that decouple attention computation for shared prefixes from unique suf- fixes. Since the Main and Auxiliary threads share the vast majority of their context history, these methods can elimi- nate redundant memory loads, allowing the auxiliary task to be executed with negligible marginal cost. 4. Experiments 4.1. Datasets We focus our evaluation on two long-context, multi-turn web browsing tasks that require agents to synthesize information across multiple retrieval steps. As illustrated in Figure 2, these tasks are characterized by long reasoning chains and extensive context windows often exceeding 100k tokens. FRAMES (Krishna et al., 2024). FRAMES evaluates a model’s ability to perform retrieval and multi-hop reasoning over Wikipedia articles. To simulate a realistic scale, we utilize a corpus of 6.4 million Wikipedia articles (Wikime- dia). To ensure solvability, we augment this corpus with the specific articles containing the ground truth for each FRAMES query. We report our metrics on 424 samples from this dataset. BrowseComp (Wei et al., 2025). This dataset tests the ability to navigate the web to find specific, hard-to-locate 020406080100 ReAct Iterations 0 20 40 60 80 100 120 Frequency BrowseComp Frames 020406080100120 Token Count (K) 0 20 40 60 80 100 120 Frequency BrowseComp Frames Figure 2. Distribution of ReAct Iterations and token count for FRAMES and BrowseComp with gpt-oss-20b (medium effort). 5 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning 40%50%60%70%80%90%100% 10K15K20K25K Peak Token Usage 40 45 50 55 60 65 70 Accuracy (%) FRAMES (Medium) 40%60%80%100% 10K15K20K25K30K35K40K Peak Token Usage 40 45 50 55 60 65 70 FRAMES (High) 20%40%60%80%100% 20K30K40K50K Peak Token Usage 10 15 20 25 30 35 40 45 BrowseComp (Medium) 20%40%60%80%100% 10K20K30K40K50K60K70K80K Peak Token Usage 10 15 20 25 30 35 40 45 BrowseComp (High) 20%40%60%80%100% 40M60M80M100M120M140M160M KV Cache Memory Reads 40 45 50 55 60 65 70 Accuracy (%) 20%40%60%80%100% 100M200M300M400M KV Cache Memory Reads 40 45 50 55 60 65 70 20%40%60%80%100% 50M100M150M200M250M300M KV Cache Memory Reads 10 15 20 25 30 35 40 45 20%40%60%80%100% 200M400M600M800M KV Cache Memory Reads 10 15 20 25 30 35 40 45 BaselineSidequestH2OSnapKVRKV Figure 3. Efficiency vs. Utility Trade-off. We evaluate Accuracy against Peak Token Usage and KV cache memory reads for gpt-oss-20b with Medium and High reasoning effort, on the FRAMES and BrowseComp benchmarks. The Uncompressed Baseline establishes the upper bound for accuracy but incurs the highest memory cost. SideQuest achieves substantial memory savings—reducing peak token usage by 56-65% compared to the baseline—while providing a better accuracy compared to heuristic based methods. information. It features difficult user queries with short, ver- ifiable answers. To ensure a fair and reproducible compar- ison, we utilize the corpus from BrowseComp-Plus (Chen et al., 2025). This provides a fixed set of100k documents containing both the necessary supporting evidence and chal- lenging negative distractors, simulating a realistic retrieval environment. We report results on a subset of 500 samples from this dataset. For both benchmarks, we implement a local server that exposes search and retrieval APIs (e.g.,search(), open()) backed by the respective corpora. This setup serves as the backend for the browser tool, ensuring deter- ministic evaluation by removing the variability of live web results. To facilitate effective retrieval within this environ- ment, we employ the Qwen3-Embedding-8B (Zhang et al., 2025c) model to generate dense vector embeddings for all webpages in the corpus. 4.2. Models We conduct experiments using the gpt-oss-20b (OpenAI, 2025) model. This model is natively trained with a browser tool that indexes all tool outputs with unique cursor identi- fiers (e.g.,[Cursor 0]). This cursor-based indexing is central to our method, as it allows SideQuest to perform granular, object-level eviction of search results and web/doc- ument content. It also facilitates the automated collection of training data for the auxiliary task, as detailed in Section 3.3. We report results for this model under both Medium and High reasoning effort configurations. 4.3. Training To construct our training corpus, we sampled 400 tasks from the FRAMES (Krishna et al., 2024) dataset. After filtering for traces that resulted in correct answers, we obtained 215 high-quality samples. We then applied the data generation pipeline described in Section 3.3 (Algorithm 2) using gpt- oss-120b as the annotation modelMand an interval of k = 4. This process yielded a dataset comprising 215 main traces and 1274 auxiliary traces. To balance the dataset and preserve the model’s core reasoning capabilities, we upsampled the main traces by a factor of3×. We fine-tuned the gpt-oss-20b model using LoRA (Hu et al., 2021) for 3 epochs with a learning rate of2× 10 −4 and a distillation loss weight ofλ = 500. The LoRA configuration used rank r = 8andα = 16. To minimize training overhead while re- taining capacity, we applied adapters exclusively to a subset of the projection layers (gateupproj, downproj) in the Mixture-of-Experts (MoE) layers at depths 7, 15, and 23, keeping all other parameters frozen. Finally, we employed Quantization Aware Training (QAT) during this fine-tuning stage to ensure the final model could be robustly quantized to the mxfp4 format. 4.4. Baselines We compare SideQuest against an uncompressed baseline (full attention) and three representative KV cache compres- sion techniques. For all heuristic baselines, we evaluate performance at two token budgets: 16k and 24k tokens. • H 2 O(Zhang et al., 2023). A “Heavy Hitter” oracle 6 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning Baseline Sidequest H2O (16K)H2O (24K) SnapKV (16K)SnapKV (24K) RKV (16K)RKV (24K) 0 20 40 60 Non-Completion Rate (%) FRAMES (Medium) Baseline Sidequest H2O (16K)H2O (24K) SnapKV (16K)SnapKV (24K) RKV (16K)RKV (24K) 0 20 40 60 FRAMES (High) Baseline Sidequest H2O (16K)H2O (24K) SnapKV (16K)SnapKV (24K) RKV (16K)RKV (24K) 0 20 40 60 BrowseComp (Medium) Baseline Sidequest H2O (16K)H2O (24K) SnapKV (16K)SnapKV (24K) RKV (16K)RKV (24K) 0 20 40 60 BrowseComp (High) Unparsable ResponseContext LimitTurn Limit Figure 4. Non-Completion Rate across benchmarks, categorized by failure type: Unparsable Responses (orange), Context Limits (green), and Turn Limits (purple). SideQuest demonstrates superior reliability, matching the near-zero failure rate of the uncompressed baseline, while other methods suffer from high rates of model collapse. that retains tokens with the highest cumulative attention scores while evicting others. This represents the standard for frequency-based pruning. •SnapKV (Li et al., 2024). This method identifies and retains clusters of key information within the attention window, aiming to preserve local semantic structure better than individual token pruning. •R-KV (Cai et al., 2025). A reasoning-focused compres- sion technique that scores tokens based on redundancy, pruning those deemed repetitive or non-essential for the current generation step. We implement our compression techniques within the SGLang framework (Zheng et al., 2024), ensuring full com- patibility with prefix caching to enable efficient context reuse across multiple ReAct iterations. 4.5. Metrics We evaluate the trade-off between system efficiency and model utility using three primary metrics: •Peak Token Utilization: We report the maximum size of the KV cache reached during the execution of a task. This metric serves as a proxy for the worst-case memory capacity requirement, determining the maximum batch size that can be supported on a given GPU. •KV Cache Memory Reads: We quantify the total volume of data transfer required by the attention mechanism dur- ing the decode phase. As agentic workloads are typically memory-bandwidth bound, this metric directly correlates with end-to-end inference latency and system throughput. •Accuracy: We measure the success rate on the FRAMES and BrowseComp benchmarks to assess model utility. To visualize the cost-benefit trade-off, we plot Utility vs. Efficiency curves across both benchmarks under medium and high reasoning efforts, highlighting the Pareto frontier of memory savings versus reasoning performance. Serving Metrics. To evaluate the real-world impact on pro- duction systems, we additionally report System Through- put (tokens/second), Normalized KV Cache Usage, and Total Benchmark Runtime by implementing SideQuest in SGLang. These metrics capture the practical efficiency gains in a high-concurrency production-grade environment. 4.6. Results We present our main experimental results in Figure 3, plot- ting the trade-off between model utility (Accuracy) and sys- tem efficiency (Peak Token Usage and KV Cache Reads). Efficiency vs. Utility. SideQuest fundamentally shifts the Pareto frontier for agentic memory management. As shown in Figure 3, SideQuest reduces Peak Token Utilization by 56− 65%and KV cache memory reads by53− 71%, com- pared to the uncompressed baseline. This massive reduction in memory load comes with minimal cost to reasoning per- formance: we observe only a marginal degradation in accu- racy of up to2%on the in-distribution FRAMES benchmark and a5%degradation on the out-of-distribution BrowseC- omp benchmark. In contrast, heuristic baselines likeH 2 O, SnapKV, and R-KV suffer precipitous drops in accuracy at comparable compression levels, failing to maintain the context fidelity required for complex multi-hop reasoning. Failure of Fixed Token Budgets. A key finding from our analysis is the inadequacy of fixed-budget compression for agentic tasks. As illustrated in the token distribution his- tograms in Figure 2, there is significant variance in task difficulty, with token counts ranging from a few thousand to over 120k. Fixed-budget methods (e.g., forcing a 16k window) inevitably fail on the long tail of complex queries, while wasting memory on simple ones. Unlike these rigid approaches, SideQuest adaptively adjusts its context size based on the problem’s instantaneous difficulty. By dynam- ically evicting only the specific cursors that are no longer semantically relevant 3 , SideQuest discovers the optimal to- ken budget for each specific query without a priori tuning. 3 See Appendix B for examples of SideQuest’s reasoning. 7 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning 21242730333639 Batch Size 0 500 1000 1500 Throughput (tok/s) 83.9% Gen. Throughput Baseline Sidequest Baseline best: 828 tok/s Sidequest best: 1523 tok/s 21242730333639 Batch Size 0.00 0.25 0.50 0.75 1.00 KV Usage (Norm) 53.9% Peak KV Cache Usage Baseline Sidequest Baseline (BS=24): 0.977 Sidequest (BS=36): 0.450 21242730333639 Batch Size 0 1000 2000 3000 Benchmark Runtime (s) 36.8% Benchmark Runtime Baseline Sidequest Baseline best: 2356s Sidequest best: 1489s Figure 5. Serving Performance in SGLang. We compare Sidequest against the uncompressed baseline for gpt-oss-20b (Medium Effort) on the FRAMES benchmark using a single NVIDIA H100 GPU. (Left) Sidequest increases peak throughput by83.9%by enabling larger batch sizes. (Center) Peak KV cache usage is reduced by53.9%, freeing up significant memory headroom. (Right) The combination of higher concurrency and reduced memory movement lowers total benchmark runtime by 36.8%. Robustness and Coherence. Beyond accuracy, we analyze the reliability of the model’s generation process. Figure 4 reports the Non-Completion Rate, which aggregates failures caused by unparsable/non-terminating responses, context length limits, or non-terminating loops. We find that heuris- tic baselines exhibit a dangerously high rate of unparsable responses (orange bars). This suggests that heuristic pruning often removes tokens critical for syntactic coherence or logic flow, causing the model to produce meaningless responses. SideQuest, by contrast, maintains a non-completion rate comparable to the uncompressed baseline, ensuring that the aggressive memory savings do not compromise the struc- tural integrity of the agent’s reasoning loop. Note that the lower peak memory usage observed for some baselines in Figure 3 (e.g., BrowseComp-High) is partly an artifact of these early crashes; SideQuest achieves its efficiency gains while successfully running tasks to completion. 4.7. Serving Efficiency Analysis To validate the real-world impact of our method on produc- tion systems, we report various performance metrics using an SGLang-based implementation of SideQuest. We report system throughput, peak KV Cache usage and total runtime by running 424 samples of the FRAMES benchmark with gpt-oss-20b under medium-effort setting. We report these metrics under different levels of concurrency (batch size). As shown in Figure 5, Sidequest increases peak system throughput by83.9%(1523tok/s vs.828tok/s) compared to the uncompressed baseline, enabling the engine to scale to larger batch sizes (up to 36) without saturating mem- ory. This performance gain is driven by a massive53.9% reduction in peak KV cache usage (dropping normalized oc- cupancy from0.977to0.450), which significantly alleviates the memory bandwidth bottlenecks inherent to long-context agentic workloads. Consequently, Sidequest reduces the total end-to-end benchmark runtime by36.8%(1489s vs. 2356s), demonstrating that our proactive memory manage- ment translates directly to faster, more efficient production serving for LRMs. 5. Limitations While SideQuest demonstrates significant memory savings, we acknowledge two primary limitations in our current im- plementation. First, although our method closely matches the uncompressed baseline, we observe minor performance degradation, especially for the out-of-distribution BrowseC- omp dataset. We hypothesize that this is a data scale issue rather than an architectural flaw. Our current model was fine-tuned on a relatively small dataset constructed from only 215 traces. We believe that scaling the training data to include a larger, more diverse distribution would close this remaining gap. Second, our current eviction strategy is scoped exclusively to tool-responses. Unlike heuristic techniques which can prune any token in the sequence, Sid- eQuest does not yet attempt to compress the agent’s own intermediate reasoning steps. Extending our approach to perform thought pruning is a promising future direction of research. Additionally, our method operates at a higher level of abstraction compared to prior works on KV compression that rely on attention weights. Exploring a combination of the two is an interesting avenue for future research. 6. Future Work New Domains for Memory Management. Our evaluations in this paper are focused on multi-turn web browsing, which serves as an ideal testbed for dynamic context management. However, the principles of SideQuest are domain-agnostic. A promising direction for future research is applying this framework to coding agents, where tasks involve traversing massive codebases and reasoning over long-context depen- dency graphs. The ability to selectively forget irrelevant file contents while retaining critical function definitions could unlock significant efficiency gains. SideQuest for Other Auxiliary Tasks. While this work explores memory management, the SideQuest architec- ture—running parallel auxiliary threads on a shared con- text—has far broader applications. As illustrated in Figure 6, the same mechanism can be used to steer LRMs to perform 8 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning [Cursor 1] GTC 2026 Schedule at a Glance ..Join us in San Jose, CA, on March 16-19..  <think> okay, GTC is on March 16-19. Next, we need to find the list of long weekends in 2026. Let’s search.. 烙 <think> **Memory management mode** Open cursors: [0,1]. The GTC dates are mentioned in [1].. . . . <think> **Prompt-Injection Check** Web page mentions details of GTC.. <think> **Auxiliary mode** Dev. Prompt: Is the user trying to jailbreak the model? Respond with “yes” or “no” Assistant:.. What’s the first long weekend after GTC 2026 ?  <think> **Platform Policy Check** User’s question does not violate platform policies.. . . [context] Explored in this paper Other Applications (Future Work) Figure 6. The SideQuest framework can be extended to various auxiliary tasks beyond the memory management task. various governance and safety tasks in parallel to the main user interaction. It can also be trained to follow a custom in- struction that is inserted in the context after a trigger phrase. Currently, these auxiliary tasks (e.g. safety, security, gover- nance) are typically handled by separate, smaller guardrail models. By leveraging SideQuest, these checks can be exe- cuted as auxiliary tasks on the shared context. This would allow the primary LRM to apply its full multi-turn context awareness to safety enforcement without incurring the cost of independent context reprocessing. 7. Conclusion In this work, we demonstrate that static heuristic compres- sion fails to capture the dynamic utility of tokens in long- running agentic tasks. We propose SideQuest, a novel frame- work that empowers Large Reasoning Models to actively manage their own memory via a parallel auxiliary thread. This architecture allows for precise, semantic-aware eviction of stale tool outputs without interfering with the primary reasoning process. Empirically, SideQuest achieves a reduc- tion of up to65%in peak memory usage with only a minor drop in accuracy, strictly outperforming heuristic baselines. Crucially, our method eliminates the need for manual to- ken budgeting, adaptively scaling context size to match the instantaneous complexity of the query. By transforming memory management from a fixed constraint into a learn- able reasoning skill, SideQuest establishes a new paradigm for efficient, long-context agentic inference. 8. Acknowledgements We thank Shizhe Diao and Yaosheng Fu for their feedback, which has helped shape this work. References Ainslie, J., Lee-Thorp, J., De Jong, M., Zemlyanskiy, Y., Lebr ́ on, F., and Sanghai, S. Gqa: Training generalized multi-query transformer models from multi-head check- points. arXiv preprint arXiv:2305.13245, 2023. Cai, Z., Xiao, W., Sun, H., Luo, C., Zhang, Y., Wan, K., Li, Y., Zhou, Y., Chang, L.-W., Gu, J., et al. R-kv: Redundancy-aware kv cache compression for training- free reasoning models acceleration. arXiv preprint arXiv:2505.24133, 2025. Chen, Z., Ma, X., Zhuang, S., Nie, P., Zou, K., Liu, A., Green, J., Patel, K., Meng, R., Su, M., et al. Browsecomp- plus: A more fair and transparent evaluation benchmark of deep-research agent. arXiv preprint arXiv:2508.06600, 2025. Fu, Y. Challenges in deploying long-context transformers: A theoretical peak performance analysis. arXiv preprint arXiv:2405.08944, 2024. Gu, A. and Dao, T. Mamba: Linear-time sequence mod- eling with selective state spaces. In First conference on language modeling, 2024. Hinton, G., Vinyals, O., and Dean, J.Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arxiv 2021. arXiv preprint arXiv:2106.09685, 10, 2021. Hu, J., Huang, W., Wang, W., Li, Z., Hu, T., Liu, Z., Chen, X., Xie, T., and Shan, Y. Raas: Reasoning-aware atten- tion sparsity for efficient llm reasoning. arXiv preprint arXiv:2502.11147, 2025. Katz,J.Dynamiccontextdiscovery,jan 2026.URLhttps://cursor.com/blog/ dynamic-context-discovery. Cursor Blog. Krishna, S., Krishna, K., Mohananey, A., Schwarcz, S., Stambler, A., Upadhyay, S., and Faruqui, M. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation, 2024. URLhttps://arxiv.org/abs/ 2409.12941. 9 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning Li, Y., Huang, Y., Yang, B., Venkitesh, B., Locatelli, A., Ye, H., Cai, T., Lewis, P., and Chen, D. Snapkv: Llm knows what you are looking for before generation. Advances in Neural Information Processing Systems, 37:22947– 22970, 2024. Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., Dai, D., Guo, D., et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434, 2024. Liu, A., Mei, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., Dong, C., et al. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556, 2025. Liu, Z., Desai, A., Liao, F., Wang, W., Xie, V., Xu, Z., Kyril- lidis, A., and Shrivastava, A. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. Advances in Neural Information Processing Systems, 36:52342–52364, 2023. NVIDIA. Nemotron 3 Nano: Open, efficient mixture-of- experts hybrid Mamba-Transformer model for Agentic reasoning, 2025. URLhttps://arxiv.org/abs/ 2512.20848. Technical report. OpenAI. gpt-oss-120b, 2025. URLhttps://arxiv. org/abs/2508.10925. Packer, C., Fang, V., Patil, S., Lin, K., Wooders, S., and Gonzalez, J. Memgpt: Towards llms as operating systems. 2023. Pan, Z., Ding, Y., Guan, Y., Wang, Z., Yu, Z., Tang, X., Wang, Y., and Ding, Y. Fasttree: Optimizing attention kernel and runtime for tree-structured llm inference. Pro- ceedings of Machine Learning and Systems, 7, 2025. Qwen-Team. Qwen3 technical report, 2025. URLhttps: //arxiv.org/abs/2505.09388. Ramachandran, A., Neseem, M., Sakr, C., Venkatesan, R., Khailany, B., and Krishna, T. Thinkv: Thought-adaptive kv cache compression for efficient reasoning models. arXiv preprint arXiv:2510.01290, 2025. Sun, W., Lu, M., Ling, Z., Liu, K., Yao, X., Yang, Y., and Chen, J. Scaling long-horizon llm agent via context- folding. arXiv preprint arXiv:2510.11967, 2025. Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., and Glaese, A. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516, 2025. Wikimedia. Wikimedia downloads. URLhttps:// dumps.wikimedia.org. Yang, S., Kautz, J., and Hatamizadeh, A. Gated delta net- works: Improving mamba2 with delta rule. arXiv preprint arXiv:2412.06464, 2024. Ye, Z., Lai, R., Lu, B.-R., Lin, C.-Y., Zheng, S., Chen, L., Chen, T., and Ceze, L. Cascade inference: Memory bandwidth efficient shared prefix batch decoding, Febru- ary 2024. URLhttps://flashinfer.ai/2024/ 02/02/cascade-inference.html. Zhang, A. L., Kraska, T., and Khattab, O. Recursive lan- guage models. arXiv preprint arXiv:2512.24601, 2025a. Zhang, H., Zhang, H., Ma, X., Zhang, J., and Guo, S. Lazye- viction: Lagged kv eviction with attention pattern ob- servation for efficient long reasoning. arXiv preprint arXiv:2506.15969, 2025b. Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., and Zhou, J. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025c. Zhang, Z., Sheng, Y., Zhou, T., Chen, T., Zheng, L., Cai, R., Song, Z., Tian, Y., R ́ e, C., Barrett, C., et al. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36:34661–34710, 2023. Zheng, L., Yin, L., Xie, Z., Sun, C. L., Huang, J., Yu, C. H., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J. E., et al. Sglang: Efficient execution of structured language model programs. Advances in neural information processing systems, 37:62557–62583, 2024. 10 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning A. Prompt Used for Generating Auxiliary Traces We use the following prompt for the annotation model to generate reasoning that’s used to construct auxiliary traces as described in Section 3.3. I will provide you with a ** Task ** , a ** Partial Conversation ** (the history), and the ** Solution ** . I want you to create a synthetic reasoning content for the task leads to the provided solution. ** Your Objective: ** Generate a concise reasoning trace that an agent would think * in the moment * to distinguish between cursors that are still valuable versus those that are now just noise. ** Input Data: ** <Task> Given a partial conversation, identify the cursors (corresponding to browser tool calls ) that are no longer useful for the assistant and can be removed. </Task> <Partial Conversation> formatted_conv </Partial Conversation> <Current State> open_cursors: open_cursor_ids </Current State> <Target Solution> removable_cursors: del_cursor_ids </Target Solution> <FinalResponseCitations> cursors_final: cursors_final (These are the cursors cited in the agent’s final answer. They are by definition " useful" and must not be removed.) </FinalResponseCitations> ** Guidelines for Reasoning: ** 1. Do not mention that you have the <Target Solution> OR <FinalResponseCitations> in the reasoning. Reason based * only * on the conversation history up to this point. It ’s okay to say that a cursor is needed for citation in the final answer. 2. The reasoning should first reflect on all the open cursors and only then determine which cursors are no longer useful. 3. Reasoning should be ** concise ** . 4. If ‘<Target Solution>‘ is an empty list, all currently open cursors are useful and no cursors can be removed. ** Response Format: ** Respond with the following JSON structure: "reasoning": "...", "useful_cursors": [<id>, <id>], "removable_cursors": [<id>, <id>] B. Examples of Memory Management Reasoning Due to the long nature of the agentic tasks we cannot fit an entire trace within a page. However, we provide some examples of the auxiliary trace produced by SideQuest in Fig. 7 to demonstrate the sophisticated reasoning used in KV cache management. 11 SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoning Figure 7. Examples of memory-management reasoning produced by SideQuest. C. Related Works on Agentic Memory Management C.1. Operating System and Retrieval-Based Approaches Recent works have proposed moving beyond static context windows by drawing inspiration from operating systems. The most prominent example, MemGPT (Packer et al., 2023), creates a hierarchical memory architecture where the model explicitly manages its own context by swapping text between its active prompt (Main Context) and external storage (External Context) via function calls. Similarly, in the domain of automated software engineering, Cursor has introduced Dynamic Context Discovery (Katz, 2026), which applies virtualization principles to code generation. Rather than stuffing the context window with static tool definitions and history, Cursor abstracts these elements as files that the agent can lazily load only when necessary. While both MemGPT and Cursor solve the information retrieval problem (deciding what text to show the model), they do not address the inference efficiency problem of linear context growth. Even with dynamic discovery, once information is loaded, it occupies GPU memory linearly. Sidequest complements these methods by operating on the internal state: it allows the model to “garbage collect” the heavy tensors of intermediate reasoning steps without breaking the continuity of the task. C.2. Hierarchical and Decomposition-Based Context Management Another class of approaches addresses the long-context challenge by restructuring agentic reasoning into hierarchical or recursive formats. Recursive Language Models (RLMs) (Zhang et al., 2025a) treat the context as an external environment, allowing the model to write code that inspects, slices, and recursively calls itself on specific data chunks. Similarly, Context Fold (Sun et al., 2025) introduces a dynamic branch-and-fold mechanism where agents can spawn temporary sub-trajectories for specific sub-tasks; upon completion, the entire sub-trajectory is folded into a concise summary, and the intermediate tokens are discarded. Complementarity with Sidequest. These approaches mitigate context growth by isolating intermediate reasoning steps into separate, transient contexts (child processes or branches) and propagating only the final result to the main thread. However, they do not solve the fundamental problem of linear context growth within a specific sub-task. A complex sub-problem (e.g., “fix a failing test”) will still accumulate a massive local context of tool outputs and retrieval artifacts before a result can be returned. Sidequest addresses this orthogonal challenge. By operating as a garbage collector for the active linear stream, Sidequest can be deployed inside the sub-tasks of an RLM or Context Fold architecture. It ensures that the local context of each branch remains lean by surgically evicting stale tool outputs (observations) while preserving the necessary tool outputs. Consequently, Sidequest is highly compatible with structural decomposition methods, offering a mechanism to maximize the efficiency of the individual workers within a hierarchical system. 12

Instant research discovery