Paper deep dive
LightThinker++: From Reasoning Compression to Memory Management
Yuqi Zhu, Jintian Zhang, Zhenjie Wan, Yujie Luo, Shuofei Qiao, Zhengke Gui, Da Zheng, Lei Liang, Huajun Chen, Ningyu Zhang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/10/2026, 2:16:41 AM
Summary
LightThinker++ is a framework for Large Language Models (LLMs) that introduces Explicit Adaptive Memory Management to optimize reasoning efficiency. It evolves from LightThinker, which uses implicit representation-level compression, by incorporating explicit memory primitives (commit, expand, fold) to manage reasoning trajectories, significantly reducing token usage and memory overhead while improving performance in complex and long-horizon agentic tasks.
Entities (4)
Relation Signals (3)
LightThinker → utilizes → Explicit Adaptive Memory Management
confidence 99% · LightThinker++... introducing Explicit Adaptive Memory Management.
LightThinker → evolvedfrom → LightThinker
confidence 98% · we evolve the framework into LightThinker++, introducing Explicit Adaptive Memory Management.
LightThinker → compresses → Intermediate Thoughts
confidence 95% · LightThinker, a method that enables LLMs to dynamically compress intermediate thoughts into compact semantic representations.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) excel at complex reasoning, yet their efficiency is limited by the surging cognitive overhead of long thought traces. In this paper, we propose LightThinker, a method that enables LLMs to dynamically compress intermediate thoughts into compact semantic representations. However, static compression often struggles with complex reasoning where the irreversible loss of intermediate details can lead to logical bottlenecks. To address this, we evolve the framework into LightThinker++, introducing Explicit Adaptive Memory Management. This paradigm shifts to behavioral-level management by incorporating explicit memory primitives, supported by a specialized trajectory synthesis pipeline to train purposeful memory scheduling. Extensive experiments demonstrate the framework's versatility across three dimensions. (1) LightThinker reduces peak token usage by 70% and inference time by 26% with minimal accuracy loss. (2) In standard reasoning, LightThinker++ slashes peak token usage by 69.9% while yielding a +2.42% accuracy gain under the same context budget for maximum performance. (3) Most notably, in long-horizon agentic tasks, it maintains a stable footprint beyond 80 rounds (a 60%-70% reduction), achieving an average performance gain of 14.8% across different complex scenarios. Overall, our work provides a scalable direction for sustaining deep LLM reasoning over extended horizons with minimal overhead.
Tags
Links
- Source: https://arxiv.org/abs/2604.03679v1
- Canonical: https://arxiv.org/abs/2604.03679v1
Trouble viewing inline? Open PDF directly →
Full Text
161,308 characters extracted from source content.
Expand or collapse full text
LightThinker++: From Reasoning Compression to Memory Management Yuqi Zhu 1* , Jintian Zhang 1* , Zhenjie Wan 1 , Yujie Luo 1 , Shuofei Qiao 1 , Zhengke Gui 2 , Da Zheng 2 , Lei Liang 2 , Huajun Chen 1 , Ningyu Zhang 1 † 1 Zhejiang University, 2 Ant Group Abstract Large language models (LLMs) excel at complex reasoning, yet their efficiency is limited by the surging cognitive overhead of long thought traces. In this paper, we propose LightThinker, a method that enables LLMs to dynamically compress intermediate thoughts into compact semantic representations. However, static compression often struggles with complex reasoning where the irreversible loss of intermediate details can lead to logical bottlenecks. To address this, we evolve the framework into LightThinker++, introducing Explicit Adaptive Memory Management. This paradigm shifts to behavioral-level management by incorporating explicit memory primitives, supported by a specialized trajectory synthesis pipeline to train purposeful memory scheduling. Extensive experiments demonstrate the framework’s versatility across three dimensions. (1) LightThinker reduces peak token usage by 70% and inference time by 26% with minimal accuracy loss. (2) In standard reasoning, LightThinker++ slashes peak token usage by 69.9% while yielding a +2.42% accuracy gain under the same context budget for maximum performance. (3) Most notably, in long-horizon agentic tasks, it maintains a stable footprint beyond 80 rounds (a 60%–70% reduction), achieving an average performance gain of 14.8% across different complex scenarios. Overall, our work provides a scalable direction for sustaining deep LLM reasoning over extended horizons with minimal overhead. Correspondence: zhangningyu@zju.edu.cn Code: https://github.com/zjunlp/LightThinker 1 Introduction Recent advancements in Large Language Models (LLMs) have demonstrated their remarkable capabilities in complex reasoning tasks [1,2]. As research in this domain progresses, the reasoning patterns of these models have gradually evolved from “fast thinking” to “slow thinking”. This transition is exemplified by methods such as Chain-of-Thought (CoT) [3] prompting, which enhances reasoning by breaking down complex problems into sequential sub-steps. Building on this, the o1-like thinking mode [4,5,6] introduces multiple reasoning abilities such as trial-and-error, backtracking, correction, and iteration, further improving the success rate of models in solving complex problems. However, this performance improvement comes at the cost of generating a large number of tokens [7]. Given that current LLMs are predominantly based on the Transformer architecture [8], the computational complexity of the attention mechanism grows quadratically with the context length, while the storage overhead of the KV Cache increases linearly with the context length. For example, in the case of Qwen32B [9], when the context length reaches10 4 , the KV Cache occupies a space comparable to the model itself. Consequently, the increase in token generation leads to a sharp rise in memory overhead and computational costs, severely limiting the practical efficiency of LLMs in long-text ∗ Equal Contribution. † Corresponding author. arXiv:2604.03679v1 [cs.CL] 4 Apr 2026 [Thought1] Frist, Mike starts with 12 apples and gives away half of them, which is 12 ÷ 2 = 6 apples, leav- ing him with 12 − 6 = 6 apples. [Thought2] Then, He buys 5 more apples from the market, bringing him total to 6 + 5 = 11 apples. [Answer] Therefore, Mike now has 11 apples. generate Mike gave 6 apples to his friends and had 6 left. After buying 5, Mike had 11. Thought 2 Answer Thought 1 Question C T2 C T1 Answer Question Input Output (b) Vanilla (a) A CoT example requiring two-step reasoning. (c) LightThinker(d) LightThinker++ Thought 2 Answer Thought 1 Memory Manager Question Commit (Summary) Compress CompressCommit (Summary) Mike gave 6 apples to his friends and had 6 left. After buying 5, Mike had 11. Thought 1 Thought 2 Mike has 12 apples. He gives away half of his apples to his friend and then buys 5 more apples from the market. How many apples does Mike have now? similar similar Figure 1 An illustration of the compressed reasoning paradigms. (a) A CoT example. Tokens highlighted in yellow represent critical reasoning tokens, while the remaining tokens primarily ensure fluency. (b) The Vanilla approach generates full thought tokens. (c) LightThinker compresses each thought into a concise representation (퐶 푇 푖 ). (d) LightThinker++ further incorporates explicit memory management to handle summaries of thoughts, enhancing reasoning efficiency and coherence. generation and complex reasoning tasks. To mitigate this issue, two main approaches have been proposed, primarily differentiated by their intervention requirements during inference. The first category requires no additional intervention during inference, achieving efficiency through prompt engineering [10,11,12] or specialized training [13,14,15,16, 17,18] to guide LLMs in generating fewer or even zero [19,20] intermediate tokens during reasoning. The second category operates through real-time token-by-token intervention during inference [21,22], reducing memory usage by selectively retaining important parts of the KV Cache while discarding less critical ones. However, both approaches face distinct challenges: the first typically requires careful data construction and iterative refinement, while the second introduces substantial inference latency due to the computational overhead of token-wise importance assessment. In this work, we propose a new approach by training LLMs to dynamically compress and manage historical content during reasoning. Our motivation stems from the concept of cognitive economy: 1) Tokens generated by the LLM serve dual purposes of ensuring linguistic fluency and facilitating actual reasoning, which makes it possible to distill the “gist” of thoughts. 2) When humans solve complex problems, they do not maintain every intermediate word in active working memory; instead, they store key conclusions mentally and only “unpack” or revisit details when encountering logical bottlenecks. Based on these insights, we first introduce LightThinker, which achieves efficiency through representation- level thought compression. As illustrated in Fig. 1(c), after generating a lengthy thought step (e.g.,Thought i), it is compressed into a compact representation (e.g.,C Ti), and the original thought chain is discarded, with reasoning continuing based solely on the compressed content. Specifically, we train the LLM to condense lengthy thoughts into a set of hidden states corresponding to a small number of special tokens (i.e., gist tokens [23]). Through carefully designed attention masks, the LLM learns when and how to compress and continue generating based on the compressed content. While representation-level compression works well for many reasoning tasks, we find that purely implicit compression can lead to irreversible information loss in more complex scenarios. Then, to address this limitation, we evolve the framework into LightThinker++, introducing Explicit Adaptive Management. As shown in Fig. 1(d), this paradigm shifts to behavioral-level memory management by introducing explicit memory primitives (e.g., commit, expand, fold). It enables the model to autonomously archive thoughts into semantic summaries or retrieve raw details upon logical necessity, ensuring robustness in both complex reasoning and long-horizon agentic tasks like DeepResearch. To support this evolution, we develop a collaborative synthesis pipeline, which generates expert trajectories that interleave reasoning with purposeful memory operations, training the model to master complex memory scheduling. We conduct extensive experiments across four datasets using two representative model families: Llama and Qwen. First, with the Qwen model, LightThinker reduces the peak token usage by 70% and decreases inference time by 26% compared to the Vanilla model, while maintaining comparable accuracy (with only a 1% drop). Furthermore, LightThinker++ achieves a superior accuracy–efficiency balance under two distinct scenarios: (1) in the Throughput setting (prioritizing inference speed), it slashes peak memory by 69.9% while maintaining baseline accuracy; (2) in the Budget setting (prioritizing reasoning quality), it not only reduces peak memory by 45.0% but also yields a +2.42% average accuracy gain. This confirms that a condensed, high-signal context is more effective for complex reasoning than a verbose, unmanaged one. 2 Beyond standard benchmarks, a more critical challenge is whether the compressed-context reasoning paradigm can also benefit long-horizon agentic tasks, where the context continuously grows over many interaction rounds. To validate the scalability of our approach in complex scenarios, we further evaluate LightThinker++ on long-horizon agentic tasks using the Qwen3-30B-A3B model. On challenging benchmarks including xBench-DeepSearch, BrowseComp-ZH, and BrowseComp-EN, LightThinker++ demonstrates remarkable efficiency. While the Vanilla agent’s context inflates to 100k tokens within 60 rounds, Light- Thinker++ maintains a lean 30k–40k footprint (a 60%–70% reduction) even beyond 80 rounds. By decoupling reasoning depth from memory consumption, our method delivers an average +4.4% Pass@1 boost across three benchmarks, with a remarkable 2.51× performance leap on the challenging hard subset. Our contributions are as follows: 1) We propose LightThinker, a method that dynamically compresses thought chains during reasoning, significantly reducing memory overhead and inference time. 2) We also present LightThinker++, a framework for explicit adaptive memory management using memory primitives, and show that it can work effectively for both standard reasoning benchmarks and agentic tasks. 3) We demonstrate that the LightThinker family achieves a good balance between reasoning efficiency and accuracy, offering new insights for future LLM inference acceleration. 2 Background Slow Thinking. The ability of LLMs to perform reasoning is fundamental [24], particularly when addressing complex tasks that require moving beyond the rapid, intuitive mode of System 1 toward the deliberative mode of System 2 [25,26,27]. A representative approach is Chain-of-Thought (CoT) [3], which tackles difficult problems by breaking them down into smaller steps and solving them sequentially. Building on this idea, the o1-like thinking mode [4,5,6] introduces additional behaviors such as trial-and-error, reflection, backtracking, and self-correction. Existing empirical results [4,6] indicate that this thinking mode yields markedly better performance than CoT on challenging reasoning tasks. Such slow-thinking behavior can be learned by models through Supervised Fine-Tuning (SFT) with carefully designed training data. From the perspective of generation length, the token usage of System 1, CoT, and o1-like thinking mode increases in an orderly manner. Inference Challenges. Prior studies on the o1-like thinking mode [7] have pointed out that solving complex problems often requires producing a large number of tokens. However, this long-generation behavior poses two major challenges for the attention mechanism, which lies at the core of Transformers [8]. First, the memory burden grows continuously during inference. To improve decoding efficiency, the Key and Value of each token are stored in the KV cache at every layer. For Qwen-32B [9], once the context length reaches10 4 tokens, the memory footprint of the KV cache becomes comparable to the size of the model itself. Second, the computational cost of autoregressive generation also increases substantially. Because of the attention operation in Transformers [8], the computation required for inference scales quadratically with the number of tokens. 3 Methodology We present the LightThinker family to accelerate the reasoning process of LLMs, as illustrated in Figure 1. The core idea is to train LLMs to dynamically compress the current thought during reasoning, enabling subsequent generation to be based on the compressed content rather than the original long thought. Here we introduce two complementary paradigms. LightThinker (§3.2) targets maximal efficiency via implicithidden-statecompression, where completed thought spans in the attention context are replaced with a small number of compressed tokens. LightThinker++ (§3.3) improves robustness forgeneralreasoningby introducingexplicitadaptivememorymanagementand a set of controllable actions that support on-demand archival and expansion of information. While our experiments focus on the reasoning setting, this explicit management mechanism also naturally extends to long-horizon agentic scenarios (see §5). 3.1 Overview The efficiency of complex reasoning is fundamentally constrained by the cognitive overhead of processing long contexts. Inspired by human Working Memory (WM) and the Information Bottleneck (IB) principle, intelligence emerges not from retaining all information, but from strategically compressing and maintaining what is predictive for future reasoning. Specifically, we propose a hierarchical framework that progresses from implicit, representation-level distillation to explicit, behavioral-level memory management, analogous 3 to the shift from automatic intuition to deliberate control in human cognition. From this perspective, the LightThinker family can be understood as a progressive design that balances efficiency, interpretability, and controllability through different levels of memory abstraction. From Implicit Representation to Explicit Management. Based on the principle of thought compression, we construct two progressive paradigms to balance reasoning efficiency and task complexity: • LightThinker: Focuses on representation-level optimization via an information bottleneck. It implicitly encodes thoughts into hidden state Cache Tokens, offering extreme efficiency without altering the output format. However, its lossy nature may limit performance in high-precision tasks. •LightThinker++: Advances toward behavior-level memory management. To mitigate the irreversibility of implicit compression, we introduce explicit memory mechanisms that allow the model to regulate information retention and reactivation. The model learns to actively manage its memory by archiving details or retrieving them upon logical necessity, ensuring robustness in complex, long-horizon reasoning. Design. To realize these paradigms, we systematically address two key design dimensions: • When to compress? The timing of compression dictates the balance between reasoning efficiency and semantic integrity. In LightThinker, we employ rule-driven strategies such as token-level [28] or thought-level [29] triggers. In LightThinker++, this evolves into a model-driven decision, where the LLM autonomously invokes different memory operations based on reasoning complexity. •How to compress? The objective is to transform lengthy thoughts into compact, navigable repre- sentations. LightThinker utilizes hidden state compression via gist tokens [23], implemented through a thought-based attention mask. In contrast, LightThinker++ upgrades this to explicit summarization and bidirectional management, dynamically reorganizing contextual information to approximate human working memory regulation. What content has been compressed? We do not aim to compress lengthy thought information into a compact representation without loss. Instead, our focus is on preserving only the information that is essential for subsequent reasoning. As highlighted by the gray dashed box in Figure 1, the lengthy thought is retained solely for the elements that contribute to further inference. 3.2 LightThinker: Implicit Thought Compression Notation. We first clarify the notation used in this section. A lowercase symbol such as푥 푖 refers to a single token, while an uppercase symbol such as푋denotes a token sequence. We use ‘[·]’ to represent a special token, e.g., ‘[c]’, and ‘<·>’ to denote an optional special token, such as ‘<w>’. Our training corpus is the o1-like thinking mode dataset풟=(푋,푌) 푖 |풟| 푖=1 , where푋=푥 푖 |푋| 푖=1 is a question and푌=푦 푖 |푌| 푖=1 is the corresponding reasoning trace together with the final answer. Prior work [6, 30] has demonstrated that SFT on풟 can substantially improve the reasoning performance of LLMs. Data Reconstruction. Our goal is to expose the model to an intermediate compression process while it is generating reasoning traces. To this end, we transform each sample in풟into a compressed form ˆ 풟. Given an input-output pair(푋,푌), we first apply a segmentation functionSeg()to split푌into푘segments, written as푌=푆 푖 푘 푖=1 . The segmentation may operate either at the token level or at the thought level, depending on the desired granularity. After segmentation, we augment the sequence by inserting three types of special tokens between neighboring segments:<w>,퐶, and[o]. Here,<w>acts as an optional signal that indicates the preceding segment should be compressed; it can be removed when token-level segmentation is used or when the trigger already appears in푆 푖 . The component퐶=[c i ] |퐶| 푖=1 is a fixed set of gist tokens, which serve as a compact carrier of the distilled information from the preceding segment. We refer to this set as the 4 Compress S1 into [c 1][c2] Compress S 2 into [c1][c2] (b) Our Attention Mask during training. (a) Original Attention Mask. (c) Inference Process and Corresponding Attention Mask during Inference. t=1 Generate t=1,w=30 t=3 Generate t=5 Generate t=2 Compress t=2,w=32 w=10+20+ 15+17=62 t=3,w=28t=4,w=30t=5,w=33 t=4 Compress Don’t MaskMask Number of tokens Figure 2 An overview of LightThinker, illustrated with a three-step reasoning example. Fig. (a) shows the attention mask of Vanilla during both training and inference. Fig. (b) depicts the attention mask of LightThinker during the training. Fig. (c) presents the complete inference process of LightThinker along with the attention mask corresponding to each step. Here, ‘w’ denotes the size of the matrix. Peak Tokens Prompt Length a) Vanilla c) AnLLM & LightThinker b) H2O Context Length Context Length Generated Tokens LengthGenerated Tokens Length Generated Tokens Length Context Length Peak Tokens DependencyDependency Dependency Prompt Length Prompt Length Peak Tokens d) LightThinker++ Generated Tokens Length Context Length Dependency Prompt Length Peak Tokens Expand Action Figure 3 Relationship between context length and the number of generated tokens across different methods. The Dependency metric corresponds to the area under the curve, while Peak Token indicates the curve’s maximum value. See Appx. A for details. cache tokens, and use|퐶|to denote the cache size. The token[o]is required to resume generation from the compressed representation, inspired by Zhang et al.. With these insertions, the reconstructed output becomes ˆ 푌=푆 1 , <w>, 퐶, [o] ,푆 2 , <w>, 퐶, [o], ... ,푆 푘 . Accordingly, the transformed dataset is ˆ 풟=(푋, ˆ 푌) 푖 | ˆ 풟| 푖=1 . For simplicity, we assume<w>∈ 푆 푖 and omit it in the rest of the section. We also use superscripts to distinguish instances of identical special tokens at different locations, e.g., 퐶 (1) and [o] (1) denote the tokens following 푆 1 . Thought-based Attention Mask Construction. To ensure that the model learns both how to compress and how to reason from compressed content, we introduce the Thought-based Mask Construction illustrated in Fig. 2(b). Let 푆 <푖 =푆 1 , ... ,푆 푖−1 denote the sequence of thoughts before the 푖-th thought 푆 푖 . In the compression stage, tokens in퐶 (푖) are permitted to attend only to the question푋, the previously generated compressed content퐶, [o] (<푖) , and the current thought 푆 푖 . This process can be expressed as 퐶 (푖) ← Cmp(푋,퐶 (1) , [o] (1) , ... , 퐶 (푖−1) , [o] (푖−1) ,푆 푖 ), whereCmp()denotes the compression operation. Under this constraint, the model is encouraged to distill the essential information in 푆 푖 into 퐶 (푖) . A detailed mathematical description of Cmp() is given in Appx. B. In the generation stage,[o] (푖) can only attend to the question푋and the compressed history퐶, [o] (≤푖) . Accordingly, the next segment is generated by 푆 푖+1 ← Gen(푋,퐶 (1) , [o] (1) , ... , 퐶 (푖) , [o] (푖) ), whereGen()denotes the generation operation. This design allows the model to continue reasoning based on the question and the compact summary of previous thoughts. Training and Inference. The training objective is to maximize the likelihood of the reconstructed sequence under the model parameterized by 휃: 푃 휃 (푆 1 |푋)· 퐾 Ö 푘=2 푃 휃 (푆 푘 |푋,퐶 (푖) , [o] (푖) 푘−1 푖=1 ) 5 ( , ) a) Memory Action Spaceb) Inference overview of LightThinker++ Content Content can be Thought / Observation / Action (..., Folded ) ( , ... ) ...... Folded Content Expanded Content ... Model End Content ...... ... .................. ...... ...... ...... ... ... ... ( , ... ) Figure 4 Overview of LightThinker++. a) Memory Action Space: Reasoning steps are instantiated as dual-form entities ℐ 푖 =(푅 푖 , 푍 푖 ). The visibility state of each step is explicitly managed via commit, expand, and fold primitives. b) Inference Overview: An illustration of the step-wise inference process. ̃ ℋ 푡 denotes the stateful managed context where historical steps are dynamically projected as either summaries (푍 푖 , marked with<) or raw derivations (푅 푖 , marked with∨) based on the model’s self-directed memory policy. where휃represents the LLM parameters. During training, the model is not required to predict the input푋 or the inserted special tokens퐶and[o]. Instead, these tokens serve as structural placeholders that guide the model to learn compression-aware reasoning. The optimization still follows the standard next-token prediction paradigm, and the samples are drawn from ˆ 풟with an attention mask that enforces the desired dependency structure. The detailed inference procedure is shown in Fig. 1(c) and Fig. 2(c). 3.3 LightThinker++: Explicit Adaptive Memory Management While the implicit compression in Sec. 3.2 provides significant efficiency gains, its predefined information bot- tleneck may struggle with highly complex reasoning tasks which require precise backtracking to intermediate logical steps. To address this, we propose LightThinker++, a paradigm that evolves thought compression from the representation-level to the behavioral-level by empowering the model to actively manage its own context memory through a dynamic substitution mechanism. The Explicit Memory Management Framework. As illustrated in Fig. 4(b), We formalize the model’s reasoning history as an ordered sequence of reasoning entitiesℳ= ℐ 1 , ... ,ℐ 퐾 , where each entityℐ 푘 represents a discrete, instantiated reasoning step. Each entityℐ 푘 = (푅 푘 , 푍 푘 )is a dual-form container:푅 푘 captures the Raw Reasoning (the full, verbose derivation), while푍 푘 provides a Semantic Summary of its logical core. As the reasoning progresses to round푡, the model operates within a managed context ̃ ℋ 푡 = 푚 (푡) 1 , ... , 푚 (푡) 퐾 , where푚 (푡) 푘 is a dynamic projection of the푘-th reasoning step. This projection is governed by an explicit visibility state휎 푘 ∈ archive, active, updated by the model’s own memory primitives 푎 푡−1 : 푚 (푡) 푘 = ( 푍 푘 , if 휎 (푡) 푘 = archive (Compressed Thought) 푅 푘 , if 휎 (푡) 푘 = active (Restored Thought) (1) This architecture enables granularity-aware control over the thought stream through two core mechanisms: •Step Instantiation: A new entityℐ 퐾+1 is instantiated only when the model emits푎= commit(푅, 푍). This signals the completion of a reasoning unit, allowing the verbose푅to be offloaded from the active workspace while preserving its distilled gist 푍 in the archive state. • Step Manipulation: By outputtingexpand(푘)orfold(푘), the model actively “toggles” the resolution of the푘-th historical step. This allows the model to re-examine evidentiary details through expansion when encountering logical bottlenecks, and subsequently fold them back to maintain a high-signal, noise-free context. 6 •Termination: The primitiveanswer( ˆ 푎 ans )concludes the entire reasoning chain by submitting the final solution. By decoupling reasoning depth from sequence length growth, LightThinker++ ensures that the model can sustain long-range coherence in complex Chain-of-Thought tasks, effectively mitigating performance degradation caused by contextual redundancy. Environment-Aware Trajectory Synthesis. To train the model in mastering these behaviors, we develop an Online Thought Synthesis framework designed to simulate a memory-constrained environment. Concretely, we use a strong teacher model to generate high-quality reasoning trajectories that are interleaved with explicit memory actions, forming demonstrations of how to reason under a managed context. Unlike traditional static supervised fine-tuning, our framework is environment-aware: when the teacher model issues a structural commitcall, the environment dynamically modifies the prompt for the next iteration by setting휎 푘 = archive and providing the summary푍 푘 , thereby hiding the raw reasoning푅 푘 . This closed-loop synthesis forces the teacher model to continue its deduction under a state of true memory compression, creating high-fidelity trajectories that interleave reasoning, archiving, and on-demand retrieval. Behavioral Pruning and Quality Control. To extract the most effective reasoning patterns from the synthesized trajectories, we implement a Behavioral Pruning mechanism centered on a strict Memory Lifecycle constraint. This ensures the fine-tuning data reflects purposeful context management rather than stochastic tool usage. A trajectory is deemed admissible only if it satisfies the following criteria: (1) Lifecycle Completeness: The trajectory must exercise the full풜 mem suite and yield a correct answer through consistent tool invocations, ensuring the reasoning process is both functional and verifiable. (2) Symmetry Constraint: We enforce structural integrity where a fold operation must be strictly preceded by an expand on the same step. This ensures retrieved details are purposefully reverted once their immediate utility within the reasoning chain is exhausted. (3) Anti-Jitter Heuristics: To ensure management is purposeful rather than stochastic, we enforce operational density limits where푁 푒푥푝 ≤ 푁 푐표푚 and푁 푒푥푝 + 푁 푓 푙푑 ≤ 2푁 푐표푚 , with푁 action denoting the total occurrences of each respective primitive within a trajectory. In addition, we disallow consecutive identical memory operations (e.g., back-to-back expand or fold) and discard trajectories in which consecutive commit steps have a lexical similarity measured by longest common subsequence greater than 0.9, to encourage progressive reasoning. Such filtering encourages monotonic logical progression and prevents the model from falling into repetitive or redundant reasoning loops. Training and Implementation. Through this rigorous filtering, we obtain a final expert dataset풟of high-quality trajectories. To ensure the model internalizes the causal relationship between its cognitive operations and context states, we treat each trajectory풯as a sequence of context-action transitions. Specifically, instead of training on a static full-length chain, we decompose each trajectory into a series of training instances (푋, ̃ ℋ 푘 )→ℐ 푘 퐾 푘=1 . We fine-tune the model by maximizing the predictive likelihood over the pruned expert set풟: ℒ SFT (휃)=−피 풯 ∗ ∼풟 " 퐾 Õ 푘=1 log푃 휃 (ℐ 푘 | 푋, ̃ ℋ 푘 ) # (2) where ̃ ℋ 푘 is the managed history reconstructed at each step푘. Following the environmental state휎푗 < 푘 determined during synthesis, ̃ ℋ 푘 only retains the raw reasoning푅 푗 if it remainsactive, while replacing it with the summary푍 푗 once it isarchived. This step-wise state-aware training forces the model to learn that its current reasoningℐ 푘 must remain logically consistent even when conditioned on a highly compressed historical context. 3.4 Discussions: Implicit vs. Explicit The transition here in our work reflects a shift from structural optimization to behavioral management, addressing the inherent limitations of rigid information bottlenecks. We summarize their key differences as follows: Information Abstraction Paradigm. LightThinker operates at the representation-level via attention masking to enforce a fixed bottleneck. In contrast, LightThinker++ shifts to a behavioral-level approach, delegating 7 Method GSM8KMMLUGPQABBHAVG. Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓ Qwen2.5-7B Series CoT86.121.66 5130.1M66.501.77 6490.2M26.760.60 9680.5M65.450.68 5700.1M61.211.18 6750.2M Distill-R181.885.60 8441.1M51.70 14.31 2483 7.5M24.758.016718 31M57.785.531967 6.0M54.038.363003 11.3M Vanilla90.90 11.83 2086 3.9M59.98 20.61 3417 10M30.81 10.76 8055 39M69.90 11.50 3786 13M62.90 13.68 4336 16.6M + H2O89.9222.196401.2M59.6929.0210243.2M24.7515.6112009.8M70.1015.6110243.5M61.1220.619724.4M + SepLLM30.4053.5210246.9M10.8153.4510249.0M0.0011.65102410M8.0826.6410249.4M12.3236.3210248.9M AnLLM78.3915.267891.6M54.6314.138752.0M19.709.14340111M54.9510.0413033.8M51.9212.1415924.6M LThinker tho 90.1411.466761.0M60.4713.099441.9M30.308.4123859.3M70.307.7111512.7M62.8010.1712893.7M LThinker tok 87.1111.4810381.5M57.3513.804893.5M28.288.26394018M62.838.9518845.6M58.8910.6218387.2M Llama3.1-8B Series CoT85.142.15 5500.2M65.822.39 7360.3M24.750.961231 0.9M66.460.93 6420.2M60.541.61 7900.4M Distill-R173.622.58 3950.1M53.462.97 5820.8M20.205.243972 16M61.210.83 3800.2M52.122.911332 4.4M Vanilla91.43 12.06 1986 3.0M69.62 14.82 2883 6.9M40.917.986622 26M83.036.802793 5.9M71.25 10.42 3571 10.5M + H2O90.4520.236401.0M65.9227.117361.8M31.8112.5515367.9M78.9911.4310242.1M66.7917.839843.2M + SepLLM26.2550.0510245.8M25.1250.1110247.5M2.5312.62102410M14.5527.1410248.5M17.1134.9810248.0M AnLLM77.3317.925891.1M58.6216.535891.2M31.317.198383.7M68.899.796211.6M59.0412.866591.9M LThinker tho 88.2512.656290.9M63.3914.888821.8M36.366.3817966.4M79.397.469111.9M66.8510.3410552.7M LThinker tok 85.5213.8711041.7M61.0515.8515383.3M31.826.94315012M74.147.4315122.9M63.1311.0218264.8M Table 1 Main results of LThinker. The CoT is based on the instruction model, while Vanilla, AnLLM, and LightThinker are based on Distill-R1. The light blue background indicates acceleration methods, with bold representing the best and underline the second best among them. The Acc of Vanilla serves as the upper bound for Acc of acceleration methods. Dep is measured in million, Time in hours, and Peak in counts. The compression ratio can be roughly estimated by the ratio of Dep between acceleration methods and Vanilla. See Appendix A for more details. Note that the results here are based on the same batch size. The results under the same memory budget are shown in Table 2. context control to explicit memory primitives. This alignment between reasoning actions and working memory states enables interpretable context orchestration that adapts to the model’s logical needs. Static vs. Adaptive Rhythm. While LightThinker follows predefined token or thought-level intervals, LightThinker++ is inherently adaptive. It empowers the model to autonomously decide when to archive or retrieve information, demonstrating a more flexible cognitive economy for varying task complexities. Efficiency vs. Fidelity. LightThinker is optimized for maximal inference speedup, making it ideal for standard tasks where a general gist of thoughts suffices. However, the lossy nature of hidden-state compression can lead to the irreversible loss of critical details. LightThinker++ prioritizes reasoning fidelity by ensuring that fine-grained information remains restorable through explicit backtracking via expand. This bidirectional capability mitigates the “information evaporation” common in implicit bottlenecks, providing the precision necessary for long-horizon tasks. 4 Experiments: General Reasoning 4.1 Experimental Settings Baselines. 1) LightThinker. We evaluate our method on two backbone LLMs: Qwen2.5-7B [9] and Llama3.1-8B [32]. To provide an upper-bound reference, we apply full-parameter instruction tuning on the Bespoke-Stratos-17k dataset (BS17K; an example is shown in Fig. 27), and refer to the resulting model as Vanilla. Following our preliminary experiments, training is initialized from the R1-Distill [6] (e.g., DeepSeek-R1-Distill-Qwen-7B), since fine-tuning instruction-tuned models such asQwen2.5-7B-instruct leads to only modest gains. For comparison, we consider five baselines: two training-free acceleration methods applied to Vanilla, namely H2O [21] and SepLLM [22], both of which preserve important KV cache entries using different strategies; one training-based approach, AnLLM [29]; and two CoT [3] baselines, obtained by prompting the instruction-tuned model and the R1-Distill model, respectively. 2) LightThinker++. We follow the same model setup, conducting experiments on the R1-Distill models and fine-tuning all methods from the corresponding R1-Distill checkpoints. For this setup, we curated a high-quality distillation dataset by sampling from BS17k and DeepScaleR [33], employing DeepSeek-V3.2- Thinking as the teacher model for data synthesis. To ensure a comprehensive comparison, we introduce four additional baselines: the Vanilla model, which is fine-tuned on the uncompressed, full-length original outputs synthesized by our framework; TokenSkip [34], which fine-tunes models on pruned CoT paths to enable selective token-level skipping; and the Base Prompting results, which provide the performance of the 8 original models without fine-tuning. For simplicity, we use LThinker and LThinker++ to denote our models. Evaluation Metrics and Datasets. We conduct experiments on four benchmark datasets: GSM8K [35], MMLU [36], GPQA [37], and BBH [38]. For MMLU and BBH, we evaluate on randomly sampled subsets rather than the full datasets. To assess model performance, we consider two aspects: effectiveness and efficiency. Effectiveness is measured by accuracy (Acc), while efficiency is evaluated using three indicators: inference time (Time), the maximum number of tokens appearing in the context during decoding (Peak), and the cumulative dependency of generated tokens on earlier tokens (Dep). As illustrated in Fig. 3, Peak reflects the highest contextual load at a specific moment, whereas Dep is represented by the area enclosed by the curves and captures the overall information usage throughout inference. A smaller Dep value indicates that the model relies on less information, suggesting stronger compression. Since Peak measures a transient state and Dep summarizes the entire generation process, the two quantities are not directly linked. Further details on Dep are provided in Appx. A. Implementation 1) Variants of LThinker. To study implicit compression at different granularities, we consider two variants of LThinker. LThinker 푡표푘 operates at the token-level and compresses every 6 original tokens into 2 gist tokens, i.e.,|퐶|= 2. LThinker 푡ℎ표 is the thought-level variant, where “ ” serves as the boundary between thoughts; each thought is then compressed into|퐶|= 9tokens for Qwen backbones and|퐶|= 7tokens for Llama backbones. 2) Variants of LThinker++. For the explicit management, we compare: LThinker++, the full model supporting all memory primitives (commit, expand, and fold) for reversible and dynamic context management; and LThinker ∗ , a commit-only ablation variant that performs irreversible summary compression, used to verify the necessity of active retrieval (expand and fold). We evaluate LThinker++ under two inference configurations, Throughput and Budget, to assess its performance under varied serving constraints(see Sec. 4.3.1). Please refer to Appx. C.1 and Appx. C.2 for further technical details on LThinker and LThinker++, respectively. 4.2 Evaluation of LightThinker. We assess LThinker from three perspectives: overall performance, inference efficiency, and ablation-based component analysis. We also include a qualitative case study to better understand its behavior. 4.2.1 Main Results Table 1 reports the results on four metrics, two backbone models, and four datasets. The main findings are summarized as follows. 1) Distill-R1 consistently performs worse than CoT on all datasets. A likely reason is its weaker instruction-following ability [39], which makes rule-based answer extraction unreliable, even when an LLM is used as the evaluator. Since this issue is orthogonal to our study, we do not investigate it further. 2) H2O reduces memory consumption effectively while preserving the accuracy of Vanilla, suggesting that its greedy eviction strategy works well for long-form generation. That said, it comes with a noticeable latency cost: compared with Vanilla, inference time increases by 51% on Qwen ((20.61− 13.68)/13.68≈ 0.51) and by 72% on Llama. This overhead is mainly caused by its token-level eviction mechanism, which adds extra computation at every decoding step. 3) SepLLM yields the weakest performance overall. During generation, it gradually loses language capability, often failing to produce termination tokens, which in turn leads to much longer inference time. 4) Compared with H2O, LThinker (tho.) maintains similar performance at lower Dep values, indicating a comparable compression ratio, while cutting inference time by 52% on Qwen and 41% on Llama on average. It also achieves higher accuracy and faster decoding than AnLLM. These results lead to the following conclusions. 1) BS17K is an effective instruction-tuning dataset for improving reasoning quality. Vanilla outperforms both CoT and Distill-R1 on most datasets, suggesting that BS17K helps SFT mitigate the repetition issue observed in Distill-R1. 2) LThinker achieves a favorable trade-off between reasoning quality and inference cost. On Qwen, it sacrifices only 1% accuracy while saving 26% inference time, reducing Peak by 70% and Dep by 78%, corresponding to a 4.5×compression ratio (16.6/3.7). On Llama, it sacrifices 6% accuracy but saves 1% inference time, reduces Peak by 70%, and lowers Dep by 74%, giving a 3.9×compression ratio (10.5/2.7). 3) The segmentation strategy plays a critical role in LThinker. Thought-level segmentation consistently outperforms token-level segmentation, improving accuracy by 6.2% on Qwen and 5.6% on Llama. This suggests that token-level segmentation may blur semantic boundaries and thus weaken the quality of compression. 9 12% 25% 72% 44% 1% 1% 4% 85% 85% 78% 82% 84% Figure 5 Efficiency Analysis and Ablation Results. (a) shows the average number of generated tokens for each model on each dataset. (b) shows the distribution of token lengths across ranges, while the cumulative curve indicates the overall proportion up to each range. (c) illustrates the relationship between output length and inference time, with each subplot reporting inference time and peak token count. (d) reports the average compression ratios, with error bars showing 95% confidence intervals. (e–f) examine how cache size|퐶|affects accuracy, Dep, inference time, peak tokens, generated tokens, and compression frequency. 4.2.2 Efficiency For readability, we use “LThinker” in the following to refer to LThinker (tho.). This section examines the efficiency of LThinker from four perspectives. GSM8K MMLU GPQA BBH AVG Vanilla11.8320.6110.76 11.50 13.68 LightThinker6.737.443.863.97 5.50 Table 2 Inference time comparison in hours between Vanilla and LThinker on the Qwen model across four datasets under the same memory budget. GSM8K MMLU GPQA BBH Qwen203711548 Llama264713955 Table 3 Average compression count per question for LThinker across datasets. How does LightThinker accelerate under same memory budget? We measure efficiency in terms of both memory usage and inference speed. As shown in Tab. 1, LThinker can substantially reduce memory consumption at the same batch size. In turn, this allows larger batches to be processed under the same memory budget, which can improve throughput in practice. Under identical memory constraints, experiments on four datasets with the Qwen model show that LThinker reduces inference time by 2.5×on average relative to Vanilla, as reported in Tab. 2. These results indicate that LThinker not only lowers memory and time costs at a fixed batch size (Tab. 1), but also brings additional speed gains when the memory budget is held constant. Does LightThinker generate more tokens compared to Vanilla? Figure 5(a) compares the average output tokens of H2O, AnLLM, LightThinker, and Vanilla across four datasets, with additional results provided in Appx. C.1.6. Two observations are worth noting: 1) LThinker is the only method that consistently generates fewer tokens than Vanilla, reducing the output length by 15% on Qwen and 13% on Llama on average. This shorter generation length is one of the main reasons behind its faster decoding speed. 2) H2O shows an inconsistent trend: it increases the number of generated tokens by 10% on Qwen but decreases it by 7% on Llama. However, even when fewer tokens are generated on Llama, inference time still increases, as shown in Tab. 1, suggesting that the overhead introduced by its eviction policy accumulates as generation proceeds. 10 What is the compression ratio of LightThinker? The compression behavior of LightThinker is summarized in three views: Fig. 5(d) presents the compression ratio across four datasets, Tab. 3 reports the average number of compressions, and Fig. 5(b) shows the distribution of compressed token counts on GPQA with Qwen (additional datasets are included in Appx. C.1.6). From these results, we observe that: 1) Compression frequency and compression ratio are influenced more by the task than by the backbone model. For example, easier tasks such as GSM8K tend to require fewer compressions and achieve higher ratios, whereas harder tasks such as GPQA involve more frequent compressions and lower ratios. 2) The compressed token counts exhibit a clear long-tail distribution. How efficient is LightThinker in memory usage and inference for long-text generation? Figure 5(c) compares the inference time and peak tokens of LThinker and Vanilla as the output length increases. Unless otherwise specified, we use a prompt length of 125 and compress 56 tokens into 8 tokens, corresponding to|퐶|= 7. The results show two clear trends. First, our method yields substantial speedups for long generations. When the output length reaches 32K tokens, inference time is reduced by 44%. For shorter outputs between 1K and 4K tokens, the reduction is smaller, but still ranges from 1% to 4%. Second, LThinker consistently lowers peak token usage, even for short generations. For example, peak tokens decrease by 72% at 1K tokens and by 85% at 32K tokens. 4.2.3 Ablation We study two factors that may contribute to LThinker’s performance: 1) the decoupled token design with its corresponding attention-mask strategy, and 2) the cache size|퐶|. GSM8K MMLU GPQA BBH AVG AnLLM78.3954.6319.70 54.95 51.92 Ours (|C|=1, T) 78.3258.2320.7155.3553.15 Ours (|C|=1, F) 80.2158.2322.22 62.02 55.67 Table 4 Ablation results on Qwen for accuracy across four datasets. “T” refers to AnLLM’s attention-mask mechanism, whereas “F” refers to LThinker’s attention-mask mechanism. Decoupled Token and Attention Mask Mode. Compared with AnLLM, LThinker introduces two differences: a decoupled token design and a different attention mask, as illustrated in Fig. 21. To examine their roles, we conduct controlled ablations. As reported in Table 4, when the cache size is fixed and LThinker uses AnLLM’s attention-mask pattern (“AnLLM” vs. “Ours (|퐶|= 1, T)”), the decoupled token design alone improves accuracy by 2%. If we further switch to LThinker’s attention-mask mode, accuracy increases by another 7%. These results confirm that both components contribute meaningfully to LThinker’s performance. Cache Size. We sweep|퐶|over1, 3, 5, 7, 9to study its effect on accuracy, inference time, dependency (i.e., Dep), peak tokens, generated token count, and compression frequency. The corresponding trends on the Qwen model are shown in Fig. 5(e-g). The results suggest the following: 1) As Fig. 5(e) shows, a larger cache generally improves accuracy while lowering inference time. This suggests that increasing cache capacity helps preserve more information after compression. 2) Fig. 5(g) shows that a larger cache size reduces both the compression frequency and the number of generated tokens. 3) Taken together, Fig. 5(e) and Fig. 5(g) indicate a clear trade-off: smaller caches trigger more frequent generation and compression to preserve information, whereas larger caches reduce this need. 4.2.4 Case Study Fig. 6 presents a failure case from GSM8K. Although the model reaches the correct answer during intermediate reasoning (see theModel’s Thoughtsfield in Fig. 6), the final response is incorrect (see theModel’s Solutionfield). In particular, the first “4000” in the third sentence ofModel’s Solutionis wrong. This suggests that the second compression step discarded part of the necessary numerical information: ideally, “8000”, “4000”, and “24000” should all have been preserved, but the model retained only “4000” and “24000”. As a result, the subsequent reasoning became inconsistent. Such failures are common on GSM8K, indicating that the current compression mechanism is still not sufficiently sensitive to numerical details. 11 Boris owns a chocolate factory. He produc- es 50,000 bars of chocolate each month. Boris produces 8,000 bars of chocolate the first week. The second week, Boris only produces half as much as the first week. But, the third week, Boris produces three times as much as the first week. How much does he produce the fourth week? Okay, ...let me break this down step ... Boris's weekly chocolate production is calculated as follows: First week: 8,000 bars. Second Week: 8,000/2=4000 bars. Third week: 8,000*3=24,000 bars. Total for the first three weeks: 4,000 + 4,000 + 24,000 = 32,000. Subtracting this from the monthly total: 50,000 - 32,000 = 18,000 bars. Thus, Boris produces 18,000 bars in the fourth week. Model’s Thoughts(Compress six times) (Compress five times) Question:The answer is 14000. Model’s Solution First, the total ... ... So third week is 24,000 bars. Now, to find the fourth... The total monthly production is 50,000 bars ... so the answer should be 14,000 bars in the fourth week. Compression Error! Figure 6 Case study. The figure shows a partial inference trace for one GSM8K example. The full example is provided in App. C.1.6. Pink and light blue backgrounds indicate alternating compression steps, with each color corresponding to one compression. 4.3 Evaluation of LightThinker++. Following the evaluation of LThinker, we evaluate LThinker++ on the same benchmarks and analyze how explicit action-based memory management improves the accuracy–efficiency trade-off. Method GSM8KMMLUGPQABBHAVG. Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓Acc↑Time↓Peak↓Dep↓ Qwen2.5-7B Series CoT86.1299.6 5130.1M66.50 106.26490.2M26.7636.0 9680.5M65.4540.8 5700.1M61.2170.8 6750.2M Distill-R181.88 336.08441.1M51.70 858.6 2483 7.5M24.75 480.6 6718 31M57.78 331.8 1967 6.0M54.03 501.6 3003 11.3M Vanilla87.3412.7 8121.1M55.2733.126827.7M34.3415.35781 21.9M61.5524.73205 10.2M59.6221.53120 10.3M TokenSkip 87.9247.8 775 1.0M54.4072.32831 9.1M38.2217.55611 21.2M59.8733.22850 9.3M60.1042.73017 10.2M LThinker*84.9413.53760.3M52.8741.77181.6M24.7520.612987.3M51.8536.58512.9M53.6028.08113.0M LThinker++88.3212.74080.3M55.0531.87551.6M35.6917.416698.1M61.0128.59272.5M60.0222.69403.1M Llama3.1-8B Series CoT85.14 129.05500.2M65.82 143.47360.3M24.7557.61231 0.9M66.4655.8 6420.2M60.5496.9 7900.4M Distill-R173.62 154.83950.1M53.46 178.25820.8M20.20 314.4 3972 16M61.2149.8 3800.2M52.12 174.6 1332 4.4M Vanilla82.7916.1 8111.3M61.1545.82570 7.1M30.3022.06364 25.7M67.6828.52826 8.8M60.4828.13143 10.7M TokenSkip79.4054.1 8381.2M57.0677.92499 6.9M26.6022.76016 23.8M69.1634.22581 7.6M58.0647.22984 9.9M LThinker*75.5412.53570.2M56.8035.27821.3M21.0415.612754.6M58.7921.27341.1M53.0421.17871.8M LThinker++82.2313.34240.3M61.7731.48831.4M33.1618.617937.2M69.0921.98961.7M61.5621.39992.7M Table 5 Main results of LThinker++ (Throughput setting). Time reports the end-to-end wall-clock time in minutes (see Appx. C.2.5 for more). The CoT is based on the instruction model, while the other methods are compared within each corresponding model series. In this table, LThinker++ denotes our full method, and LThinker* denotes our variant with only the commit action enabled. 4.3.1 Main Results We report results under two serving configurations with the same global context budget (MaxContext) but different token allocation policies: i) Throughput (Tab. 5): a fixed, small max_new_tokens per reasoning round to mimic latency-constrained serving; i) Budget (Tab. 6): max_new_tokens is set to the remaining budget, i.e., MaxContext minus tokens kept after memory actions (e.g., commit, fold). As shown in Tab. 5 and Tab. 6, our method achieves a better trade-off between reasoning accuracy and memory efficiency. We summarize the key observations: 1) Overall Performance and Accuracy-Cost Trade-off. As shown in Tab. 5 and Tab. 6, LThinker++ achieves a superior balance between reasoning accuracy and memory efficiency. In the Throughput setting, LThinker++ demonstrates extreme resource efficiency by slashing both average Peak and Dep by 69.9% (e.g., 3120→ 940 tokens on Qwen2.5-7B) while maintaining comparable accuracy to the Vanilla baseline. In the Budget setting, LThinker++ prioritizes reasoning depth, yielding a +2.42% average accuracy gain while still reducing Peak and Dep by 45.0% and 33.7%, respectively. We attribute this to a semantic denoising effect: by explicitly pruning logical redundancies, LThinker++ maintains a cleaner reasoning context, allowing the model to focus on critical logical anchors rather than being distracted by verbose intermediate steps. 2) Efficiency and the Latency-Throughput Balance. In the Throughput setting (Tab. 5), LThinker++ demonstrates a more stable efficiency ceiling for deployment. For Qwen2.5-7B, it slashes Peak memory (3120 →940) and Dep (10.3M→3.1M) with comparable accuracy to Vanilla (60.02 vs. 59.62). Notably, while methods like TokenSkip also aim for compression, they often incur a “latency paradox”—a significantly higher time cost (e.g., 42.7 vs. 21.5 on Qwen2.5-7B). This is likely because pruning-style training forces the model to 12 Method GSM8KMMLUGPQABBHAVG. Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓ Qwen2.5-7B Series CoT86.125130.1M66.506490.2M26.769680.5M65.455700.1M61.216750.2M Distill-R1 81.888441.1M51.702483 7.5M24.756718 31M57.781967 6.0M54.03300311.3M Vanilla87.627821.1M57.582432 6.5M32.495493 20.0M62.762714 8.2M60.112855 8.9M TokenSkip87.927751.0M54.402831 9.1M38.22561121.2M59.872850 9.3M60.10301710.2M LThinker*84.614440.7M57.6811414.1M28.96349623.2M56.6315807.0M56.9716658.7M LThinker++87.874710.4M60.2711333.0M38.22326516.1M63.7714154.2M62.5315715.9M Llama3.1-8B Series CoT85.145500.2M65.827360.3M24.751231 0.9M66.466420.2M60.547900.4M Distill-R1 73.623950.1M53.465820.8M20.203972 16M61.213800.2M52.121332 4.4M Vanilla79.387400.9M59.822111 4.9M31.82577321.7M67.952606 7.5M59.742808 8.8M TokenSkip79.408381.2M57.062499 6.9M26.60601623.8M69.1625817.6M58.062984 9.9M LThinker*76.725220.9M59.5913946.5M28.28427134.5M63.1016648.1M56.92196312.5M LThinker++77.695280.7M59.7213704.5M33.67410122.9M73.2014814.6M61.0718708.2M Table 6 Main results of LThinker++ (Budget setting). Note that the results here are based on the same batch size. Other settings and metric definitions are the same as Table 5. generate more exhaustive token sequences to restore the probabilistic coherence of its Chain-of-Thought, a trend especially pronounced in long-CoT distilled models. In contrast, while LThinker++ introduces minor prefill overhead due to multi-round generation, its drastic reduction in KV cache footprint enables a much higher system-level throughput via increased batch sizes. 3) Maximizing Reasoning Potential under Strict Budgets. When operating under fixed global context constraints (Tab. 6), LThinker++’s dynamic management proves most effective. By actively compressing redundant history, the model “saves” budget for subsequent critical reasoning steps. This leads to substantial gains in reasoning-heavy benchmarks like GPQA, where LThinker++ achieves a +5.73 accuracy boost on Qwen2.5-7B while utilizing 40.5% less Peak memory. These results suggest that for complex, multi-step problems, a condensed and high-signal context is fundamentally more effective than a verbose, unmanaged one. 4.3.2 Efficiency To analyze the underlying mechanisms of LThinker++ and how it manages the trade-off between reasoning accuracy and memory efficiency, we conduct an in-depth diagnostic study. Our analysis primarily focuses on the Throughput configuration (Fig. 7), with corresponding statistics for the Budget setting provided in App. Fig.20. We focus on the following four questions: How much context memory does LightThinker++ save compared to Vanilla? Fig. 7(a) illustrates the average number of visible generated tokens per reasoning step, reflecting the actual context window the model attends to. We observe that LThinker++ constrains this window significantly: on Qwen-2.5-7B, visible tokens are reduced by 82.9% (from 2982 to 511) compared to Vanilla; on Llama-3.1-8B, the reduction is 80.4% (from 3007 to 590). These results demonstrate that LThinker++ distills redundant reasoning traces into compact semantic representations, reducing the memory footprint while maintaining logical continuity. How does LightThinker++ adapt its memory-management strategy to task complexity? Fig. 7(b) presents the distribution of actions across benchmarks, revealing a pattern of cognitive economy: 1)Compression-driven efficiency:commit dominates across all tasks, serving as the key operation for context compression and faster inference. 2)StrategicadaptivityviaContextRefinement:While commit frequency remains high, the model modulates its context refinement actions (expand and fold) based on task difficulty. On simpler tasks such as MMLU, the model prioritizes straightforward archiving with minimal refinement (5.8%). Conversely, on challenging tasks like GPQA, these refinement actions increase substantially to 21.5%. This suggests that our method learns to proactively reorganize or retrieve historical details to compensate for potential information loss during complex reasoning, rather than compressing indiscriminately. 13 Peak Peak Accuracy Accuracy Figure 7 Efficiency Analysis and Ablation Results under the Throughput setting. (a) illustrates the average number of generated tokens retained in the current context. (b) presents the distribution ratio of memory actions (Commit vs. Expand+Fold) for Qwen-2.5-7B. (c) shows the percentage of pre-commit token lengths falling within specified ranges on GPQA(Qwen-2.5-7B). (d) examines the scaling of peak memory tokens as a function of the generation budget. (e) summarizes the average compression factors and saved token ratios across different model series. (f) provides an ablation study comparing accuracy and average peak tokens between the full LThinker++ and its degraded variants. What are the characteristics of LightThinker++’s compression granularity and semantic density? Fig. 7(c) depicts the distribution of raw reasoning segment lengths immediately preceding eachcommiton GPQA, while Fig. 7(e) quantifies the resulting compression ratios. We observe that: 1)High-fidelitycompression:LThinker++ achieves a compression ratio of15.0×on GPQA, and still maintains8.6×on the simpler GSM8K. 2)Task-aware cadence:The frequency of compression actions scales naturally with task difficulty. Specifically, LThinker++ executes fewercommitoperations on simpler tasks like GSM8K (average 3 times per question) compared to more challenging benchmarks such as GPQA (average 7–8 times) and BBH (average 5–7 times). The distribution in Fig. 7(c) indicates that mostcommitactions occur after long logical blocks. This adaptive cadence aligns compression with logical boundaries, allowing LThinker++ to encode deeper semantics in fewer tokens. Does LightThinker++ decouple reasoning depth from memory limits? Fig. 7(d) examines peak memory usage as the generation budget scales. While Vanilla’s peak tokens climb linearly, LThinker++ maintains a remarkably flat ceiling, peaking at only 1,830 (a 71.3% reduction). This confirms that LThinker++ effectively decouples reasoning depth from physical memory constraints, facilitating long-horizon complex reasoning with a substantially smaller memory footprint. Consistent scaling trends under the Budget setting are further detailed in Fig. 20. 4.3.3 Ablation As shown in Fig. 7(f), restricting LThinker++ to a one-way compression mode (No-Ex&Fold) causes accuracy to plummet from 60.1% to 53.6%, proving that irreversible compression leads to critical information loss. Restoring the full action set recovers accuracy to 60.0% while maintaining a 69.9% reduction in peak memory (940 vs. 3120). Interestingly, LThinker++ achieves higher accuracy with only a marginal increase in peak tokens (940 vs. 811) compared to the degraded variant. We attribute this to a reasoning compensation effect: without explicit retrieval (e.g., expand), the model may generate redundant, circular explanations to bridge memory gaps, whereas a full action suite enables semantic denoising. By distilling noisy trajectories into curated logical entities, our method allows the model to attend more effectively to core logical nodes, matching the performance of a full-context baseline at a fraction of the resource cost. 14 Method GSM8KMMLUGPQABBHAVG. Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓Acc↑Peak↓Dep↓ Qwen2.5-7B Series CoT86.125130.1M66.506490.2M26.769680.5M65.455700.1M61.216750.2M Distill-R181.888441.1M51.70 2483 7.5M24.75 6718 31M57.78 1967 6.0M54.03 3003 11.3M LThinker tho1 84.6110432.1M52.9627928.4M18.69678129.7M53.54368713.8M52.45357613.5M LThinker tho2 85.7510031.8M52.0028228.23M24.24704230.8M51.31413016.3M53.33374914.3M LThinker++87.874710.4M60.2711333.0M38.22326516.1M63.7714154.2M62.5315715.9M LThinker*84.614440.7M57.6811414.1M28.96349623.2M56.6315807.0M56.9716658.7M Llama3.1-8B Series CoT85.145500.2M65.827360.3M24.75 1231 0.9M66.466420.2M60.547900.4M Distill-R173.623950.1M53.465820.8M20.20 3972 16M61.213800.2M52.12 1332 4.4M LThinker tho1 76.729171.5M53.2626827.6M20.70669728.8M57.58367413.3M52.01349312.8M LThinker tho2 78.928421.2M52.6827837.7M18.18642027.1M56.57412315.7M51.59354212.9M LThinker++77.695280.7M59.7213704.5M33.67410122.9M73.2014814.6M61.0718708.2M LThinker*76.725220.9M59.5913946.5M28.28427134.5M63.1016648.1M56.92196312.5M Table 7 Comparison between LThinker and LThinker++ on four benchmarks. For a fair comparison,LThinker tho1 and LThinker tho2 are trained on the same training data as in Table 5. 4.3.4 Case Study [Step 1] John initially has... Thus total money ... [Step 2] John buys 8 video games, ... Set up equation based on revenue, expenses, and leftover to solve for unsold Lego sets. Extract the equation setup to solve for unsold Lego sets [Step 3] [Step 4] From Step 3, we have... Verification confirms that John has 2 unsold lego sets. Therefore, John still has ( 2) lego sets. Total revenue from selling 13 Logo sets at $15 each is $195 Commit Expand Spending $160 on games plus $5 leftover equals $165 spent; revenue is $195, leaving $30 unaccounted. [Step 3] John sold 13 Lego sets at $15 each, ... (some unknown and set up an equation) ..., Thus John still has 2 lego sets. John plans to sell all his toys and use the money to buy video games. He has 13 lego sets and he sells them for $15 each. He ends up buying 8 video games for $20 each and has $5 left. How many lego sets does he still have? Question: The answer is 2. Model’s Thoughts Compression Error! Lookback Commit Commit Figure 8 Case Study. The figure illustrates partial inference results of a case of LThinker++ from GSM8K. In our case analysis, we observe two typical uses of memory: (i) verifying previous reasoning states, and (i) recovering information that is lost during compression. We take the second case as an example. As shown in Fig. 8, the model has already inferred the key fact that John has 2 Lego sets, but this information is not reliably retained after compression due to missing intermediate details. The model then performs an expand action to retrieve the missing context from memory and continue reasoning from a consistent state, demonstrating the effectiveness of our recovery mechanism. 4.4 Discussions: Implicit vs. Explicit 4.4.1 Reasoning Performance Comparison We first evaluate implicit and explicit reasoning under the same experimental protocol. In particular, to ensure a fair comparison, we keep the training and evaluation setup identical to LThinker and only replace the training traces with the synthetic traces generated by LThinker++. Within this setting,tho1follows the same preprocessing/segmentation procedure astho(in Tab. 1), whereastho2additionally introduces a finer-grained thought segmentation strategy using . The corresponding results are reported in Table 7. The table reveals a clear performance divergence: whileLThinker tho remains competitive on the original Distilled-R1 traces, the implicit variants degrade more noticeably on our newly synthesized traces generated byLThinker++(Section 3.3). We hypothesize this is due to differences in information density. The synthesized structured traces often align each segment with a self-contained logical step, concentrating critical anchors within a compact span. In contrast, R1-Distill traces contain more discourse connectives and redundant phrasing, which “dilutes” information across a softer linguistic flow. 15 4.4.2 Quantitative Data Analysis: Length Distribution and Logical Density To investigate whether the data characteristics contribute to the above performance degradation, we analyze the character-length distribution of thought segments for the datasets corresponding to the strong implicit baseline in Table 1 (denoted astho) and our two segmented variants (tho1andtho2). Figure 9 reports kernel density estimates (KDE) of per-thought segment lengths. Figure 9 Thought segment length distribution. Kernel den- sity estimates of per-thought segment lengths (in characters) forLThinker_tho,LThinker_tho1, andLThinker_tho2. The x-axis is the segment length (characters), and the y-axis is probability density. Dashed vertical lines indicate the peak locations. The original-flow data (tho) peaks at≈ 38 characters and is characterized by substantial linguistic redundancy such as connectives and hedging tokens. This redundancy acts as a seman- tic buffer. Even if implicit compression discards certain local details, the model can often recon- struct the global intent via remaining semantic cues and pretrained language priors. In contrast, LThinker++ synthesized traces (tho1) exhibit a peak shift to≈ 51characters. This distributional shift reflects our synthesis strategy which explicitly generates segments to encapsu- late semantically complete reasoning units. By design, these structured traces prioritize the in- clusion of critical logical anchors including sub- scripts, constants, and variable bindings within each segment. Consequently,tho1possesses a significantly higher information density than the redundant natural language flow observed in tho. We attribute the degradation of implicit variants to a representation bottleneck when processing such high-density information. Within a constrained latent space, the loss of a single pivotal anchor like a mathematical sign becomes irrecoverable due to the lack of surrounding redundancy. This causes errors to propagate and eventually break the reasoning chain. Furthermore, while tho2 reduces per-segment length with a peak at≈ 22characters, the excessive number of boundaries leads to semantic fragmentation. Many segments contain only isolated punctuation or vestigial characters that lack independent logical value. This effectively injects structural noise into the context, forcing the model into frequent and error-prone state switching which leads to cumulative reconstruction failures. 4.4.3 Takeaways and Future Directions Overall, fixed-capacity implicit compression appears well-suited to “soft,” redundant natural-language reasoning flows, but it is more likely to expose capacity limits when confronted with dense, atomic “hard- logic” steps. In contrast,LThinker++mitigates this issue by explicitly managing key anchors (e.g., via textual summaries or structured records), providing a robust mechanism for state transfer in high-density settings. These findings also suggest a clear optimization direction for implicit models: rather than merely increasing a static number of Gist tokens, future work should explore adaptive latent capacity allocation (e.g., dynamically adjusting the number or budget of Gist tokens based on estimated segment information density/entropy), thereby improving fidelity and robustness on dense reasoning steps. 5 LightThinker++: Long-Horizon Agentic Reasoning To demonstrate the robustness of explicit memory management under extreme context demands, we extend LightThinker++ to the DeepResearch domain (Fig. 10). While our evaluation focuses on this scenario, which is characterized by high-entropy web interactions and multi-step information synthesis, the LightThinker++ framework is fundamentally domain-agnostic. It serves as a general-purpose paradigm for any long-horizon task where an agent must reconcile iterative planning with voluminous external feedback. Formalizing LightThinker++ within the TAO Paradigm. Standard agentic interaction typically follows the Thought-Action-Observation (TAO) cycle. To bridge this with the framework in Sec. 3.3, we instantiate the reasoning entityℐ 푘 = (푅 푘 , 푍 푘 )as the푘-th TAO turn. Here,푅 푘 = (푇 푘 , 퐴 푘 , 푂 푘 )captures the full interaction, while 푍 푘 represents its distilled semantic core. 16 Who was the shortest-serving US president, and from which college did his vice president graduate? William Henry Harrison is the short- est presidency in US. I will first search the shortest-serving US president. Then I find his vice President ... Wait, the provided content mentioned the vice president of Harrison being John Tyler. I’l search the John Tyler I’l expand the folded wiki content . The shortest-serving U.S. president was William Henry Harri- son. His vice presi- dent was John Tyler, who graduated from the College of William & Mary. William ... ... was the 9th president ... (..., ...) ( , ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . Action Commit Expand Fold Answer Google Search Observation Observation from memory action Observation from google search Think The observation#1 has been commited. I will first search the shortest-serving US president. Then I find his vice President ... t=1~2 (b) LightThinker++ (a) Vanilla t=1~3t=1~4t=1~6t=1~7 t=1t=1t=2t=2t=3 I will first search the shortest-serving US president. Then I find his vice President ... I will first search the shortest-serving US president. Then I find his vice President ... I will first search the shortest-serving ... William Henry Harrison Search “Shortest serving US President” ... was the 9th president of the US., serving from March 4 to April 4, 1841, the shortest presidency in U.S. history. ... assumed to be for Vice President John Tyler ... William Henry Harrison Search “Shortest serving US President” ... was the 9th president of the US., serving from March 4 to April 4, 1841, the shortest presidency in U.S. history. ... assumed to be for Vice President John Tyler ... John Tyler Search “US Vice President John Tyler” ... At age 12, he ... entered the preparatory branch of the College of William and Mary. Tyler graduated from the ... ( , ...) I’l fold this lengthy content . Wait, the provided content mentioned the vice president. The expanded content mentioned the vice president of Harrison being John Tyler. I’l search the John Tyler From the wiki, William Henry Harrison is the shortest president. ..., William Henry Harrison is the ... The observation#1 has been expanded. Graduated from College of William & Mary. Then I’l commit. The observation#1 has been folded. I will first search the shortest-serving ... ..., William Henry Harrison is the ... From the wiki, John Tyler was graduated from the College of William & Mary. The observation#2 has been commited. John Tyler Search “US Vice President John Tyler” ... At age 12, he ... entered the preparatory branch of the College of William and Mary. Tyler graduated from the ... (..., ...) The shortest-serving U.S. president was William Henry Harri- son. His vice presi- dent was John Tyler, who graduated from the College of William & Mary. Figure 10 Overview of LightThinker++ for long-horizon agentic reasoning. LightThinker++ follows the Thought– Action–Observation loop while explicitly managing a stateful memory. Each interaction step can be committed into an archived summary, expanded back into the active context to recover raw evidence when needed, and folded again to maintain context hygiene. To counteract the context rot inherent in linear histories, LightThinker++ redefines the research log as a stateful, managed memory ̃ ℋ 푡 =푚 (푡) 1 , ... , 푚 (푡) 퐾 . The visibility of each historical turn푘remains governed by the state 휎 (푡) 푘 , which triggers the same lossy-to-lossless toggling mechanism: 푚 (푡) 푘 = ( 푍 푘 , if 휎 (푡) 푘 = archive 푅 푘 , if 휎 (푡) 푘 = active (3) By treating memory management as an explicit decision, LightThinker++ ensures the context window is prioritized for high-signal content. High-entropy evidentiary details are only restored via expand when required for synthesis and are promptly folded to maintain context hygiene. Adapted Memory Actions for Deep Research. To manage the high informational density inherent in web-based research, we instantiate the memory framework introduced in Sec. 3.3 through a specialized toolset tailored for the agentic interaction loop: •Environment Actions (풜 env ): High-throughput tools such as search and visit, optimized for rapid information discovery across the open web. • Memory Actions (풜 mem ): Operators governing the contextual lifecycle. Specifically, commit distills the holistic interactionℐ 푘 into a summary푍 푘 to preserve long-term coherence; expand re-activates a past 17 step to retrieve raw evidence푅 푘 for precise synthesis; and fold purges these details once their utility is exhausted. Trajectory Synthesis via Multi-Agent Orchestration. To generate expert-level trajectories for research tasks, we extend the Environment-Aware Trajectory Synthesis framework (Sec. 3.3) into a Multi-Agent Orchestration paradigm. While the single-model synthesis described in Sec. 3.3 suffices for standard reasoning, the high informational density of web-based research imposes a heavy cognitive load, requiring a model to reconcile long-term strategic planning with high-entropy external data. To maintain synthesis quality, we partition this labor between two specialized roles within the closed-loop environment: (1) Interaction Agent: Executes풜 env tools while performing granular memory retrieval via expand and fold to navigate raw information. (2) Contextual Governor: Acts as the curator of the research log, adaptively triggering commit to distill holistic interaction steps into actionable summaries푍 푘 . By decoupling the generation of execution-level details from high-level context management, this collaborative synthesis ensures that the resulting trajectories maintain both rigorous logical depth and optimized context density. Behavioral Pruning and Memory Lifecycle. To extract the most effective reasoning patterns from the multi-agent orchestration, we implement a Behavioral Pruning mechanism tailored for the high-entropy research domain. We filter the synthesized trajectories against the Memory Lifecycle constraint established in Sec. 3.3, ensuring the data reflects active context governance rather than passive logging of web interactions. A trajectory is deemed admissible only if it satisfies the following criteria: 1)Lifecycle Completeness: It demonstrates the full cycle, including archiving via commit and evidence retrieval via expand/fold; 2) Symmetry Constraint: Reflecting the need for context hygiene in long-horizon interactions, a fold operation must strictly revert a previously expanded step, ensuring the context window remains purged of raw snippets once synthesis is complete. 3)Anti-Jitter Heuristics: We prohibit redundant memory operations including consecutive actions on the same step and operations targeting non-existent step IDs to ensure management is purposeful. This is particularly crucial in the agentic loop to prevent the model from falling into stochastic retrieval patterns when faced with complex external observations. Through this rigorous filtering, the collaborative expertise of the multi-agent system is distilled into a single, cohesive policy. We fine-tune the model by minimizing the negative log-likelihood over the pruned expert trajectories풯 ∗ : ℒ SFT (휃)=−피 풯 ∗ ∼풟 " 푛 Õ 푘=1 log 휋 휃 (푇 푘 , 퐴 푘 | 푋, ̃ ℋ 푘 ) # (4) By optimizing the joint predictive likelihood of reasoning traces and memory operations, the agent internalizes explicit context engineering as a core component of its decision-making process. This high-density learning signal allows the model to maintain context hygiene and reasoning fidelity across extended interaction horizons. 6 Experiments: Long-Horizon Agentic Reasoning 6.1 Experimental Settings Dataset Construction and Filtering. The base query pool is curated from a diversified ensemble of sources, including HotpotQA [40], MuSiQue [41], WebDancer [42], WebShaper [43], and WebWalkerQA-Silver [44]. To ensure the necessity of multi-hop reasoning and high-order planning, we perform heuristic filtering on HotpotQA and MuSiQue by selecting only those instances where Qwen3-30B-A3B-Instruct-2507 fails to yield direct solutions. Regarding the WebWalkerQA-Silver corpus, we adopted a language-specific selection policy: the English subset was fully incorporated to maintain linguistic diversity, while the Chinese subset was filtered to include only those instances explicitly categorized as “hard” according to the dataset’s intrinsic difficulty metadata. This collection is further augmented with diversified web-navigation tasks to form the final query pool, with a detailed categorical breakdown provided in Appx. C.2.1. From this pool, we employ a hierarchical filtering pipeline to construct two distinct training sets: 1) Vanilla Baseline: We synthesize reasoning trajectories using DeepSeek-V3.2 in non-thinking mode, utilizing only the Aenv tool. After filtering for correctness, we retain 6,625 high-quality standard trajectories. 2) 18 MethodxBench-DeepSearchBrowseComp-ZHBrowseComp-EN Proprietary Agents GPT-566.061.361.5 Claude-4-Sonnet35.029.112.2 DeepSeek-V3.251.053.635.0 DeepSeek-V3.1 44.049.523.6 GLM-4.647.042.234.9 Kimi-K2-Instruct 30.028.814.1 Qwen3-235B-A22B-Instruct27.021.8- Our Agents Qwen3-30B-A3B-Thinking8.7 (16.0)10.0 (17.3)2.1 (4.0) + SFT (vanilla)38.3 (53.0) ↑29.6 31.5 (47.8) ↑21.5 16.0 (27.3) ↑13.9 + LThinker++ (ours) 44.0 (60.0) ↑35.3 36.9 (57.1) ↑26.9 18.1 (31.5) ↑16.0 Table 8 Overall results on BrowseComp benchmarks across three datasets. We report Pass@1 for all methods, and additionally report Pass@3 (in gray parentheses) for our agents to reflect multi-attempt robustness. LThinker++: We augment the correctness filter with the Behavioral Pruning constraints described in Sec. 5. In this configuration, the agent utilizes both the Aenv and Amem tools. Due to the stringent requirements for logical memory transitions, only 3,677 expert trajectories were retained. Despite this 44.5% reduction in base trajectories compared to the Vanilla, these trajectories were decomposed into 42,633 fine-grained training instances. This yields a more potent and logically dense learning signal, providing the model with the necessary supervision to maintain context hygiene and reasoning fidelity in context-heavy tasks. Baselines and Training. We evaluate our framework against several state-of-the-art LLMs, including GLM-4.6 [45], Claude-4-Sonnet [46], GPT-5 [47], Kimi-K2 [48] and Qwen3-235B-A22B-Instruct [49] and the DeepSeek-V3 series (V3.1 and V3.2). To assess the specific impact of explicit memory management, we develop and evaluate two internal variants initialized from Qwen3-30B-A3B-Thinking-2507 [49]. The first, Vanilla-Agent, is fine-tuned on the Vanilla Baseline dataset to equip the model with environment-level capabilities via풜 env , serving as a robust standard agentic baseline. The second, LThinker++, is trained on our pruned expert trajectories to internalize the joint policy of reasoning and memory orchestration defined in Eq. 5. Implementation details are provided in Appx. C.2.4. Evaluation Metrics and Datasets. We benchmark our models across three representative agentic datasets: xbench-DeepSearch-2510 [50], BrowseComp-EN [51], and BrowseComp-ZH [52], which we refer to as xbench, BC-EN, and BC-ZH for brevity. Performance is quantified by the average Pass@1 score across all test samples and the Pass@3 score over three independent rollouts to assess reasoning stability. Specifically, we employ gpt-5-2025-08-07 as the primary automated judge to evaluate the semantic alignment between model predictions and ground-truth answers. To facilitate autonomous web interaction, we implement two core functional tools: 1) Search: An interface with the Google Search API that supports concurrent queries and retrieves the top-10 results per query. 2)Visit: A navigation module that utilizes Jina [53] for HTML parsing and Qwen-Flash to distill task-relevant evidence from the extracted content. 6.2 Main Results Table 8 summarizes the results across three benchmarks, illustrating the incremental gains from our data synthesis and memory orchestration: Effectiveness of Standard Synthesis. The Vanilla-Agent, trained on standard trajectories, exhibits a substantial performance leap over the base Qwen3-Thinking model. For instance, Pass@1 scores rise from 8.7% to 38.3% on xbench and from 10.0% to 31.5% on BrowseComp-ZH. This improvement confirms that our base data pipeline effectively equips the model with fundamental environment-level execution (풜 env ) and basic research planning capabilities. Superiority of Orchestrated Memory Actions. Building upon this baseline, LThinker++ achieves further performance leaps across all benchmarks by internalizing memory management actions (풜 mem ) via multi- agent orchestration. Compared to the Vanilla-Agent, LThinker++ delivers a significant Pass@1 improvement of 5.7% on xbench. More importantly, our method demonstrates consistent gains in reasoning stability, with Pass@3 scores increasing across all three benchmarks by 7.0%, 9.3%, and 4.2% on xbench, BrowseComp-ZH, 19 and BrowseComp-EN, respectively. These gains demonstrate that orchestrated memory primitives allow the agent to maintain a high signal-to-noise ratio, preventing reasoning collapse in long-horizon tasks where standard agents typically struggle with context clutter. Performance Gain on Hard Instances. To better isolate the effect of our approach on robustness, we bucket examples by the Vanilla-Agent’s success count over three runs,푘 ∈ 0, 1, 2, 3. We define the hard subset (ℎ푎푟푑 01 ) as instances where the baseline succeeds at most once (푘 ≤ 1). Importantly, this subset captures not only intrinsically difficult tasks, but also a characteristic failure mode of standard SFT agents in long-horizon web research: performance becomes highly unstable across runs due to goal drift, accumulation of irrelevant context, and critical evidence being overwritten or buried by noise. As shown in Table 9, LThinker++ yields substantially larger improvements onℎ푎푟푑 01 across all three benchmarks. Specifically, Pass@1 increases from 6.8% to 20.9% on xbench (∼3.08×), from 8.6% to 20.6% on BrowseComp-ZH (∼2.38×), and from 5.1% to 10.5% on BrowseComp-EN (∼2.06×), with consistent and substantial gains in Pass@3 as well. These results demonstrate that memory-management primitives are vital for maintaining reasoning fidelity in long-horizon tasks. By dynamically refining the context to emphasize critical evidence, our method successfully handles complex scenarios where the Vanilla-Agent typically fails due to information overload. Method xBench-DeepSearch (ℎ푎푟푑 01 ) BrowseComp-ZH (ℎ푎푟푑 01 )BrowseComp-EN (ℎ푎푟푑 01 ) Pass@1 (%)↑ Pass@3 (%)↑Pass@1 (%)↑ Pass@3 (%)↑Pass@1 (%)↑ Pass@3 (%)↑ Our Agents Vanilla SFT6.820.38.625.95.115.4 LightThinker++20.933.920.640.310.522.4 Table 9 Performance comparison on the ℎ푎푟푑 01 subset across three benchmarks (in %). 6.3 Efficiency and Scalability Analysis Figure 11 Quantitative Analysis of Context Management Efficiency. Fig.(a) illustrates the active context trajectories (퐶 푡 ) across interaction rounds, highlighting the stable footprint of LThinker++ compared to the rapid inflation of the Vanilla baseline. Fig.(b) provides a statistical comparison of Mean and P95 (peak) token usage across tasks, demonstrating significant reductions in memory overhead. 6.3.1 Action Budget Efficiency. We evaluate model performance under varying action budget constraints in Figure 11(a). The budget푏 represents the maximum permitted invocations of search and visit tools. By plotting the Acc@Budget≤ 푏 curve, we characterize the trade-off between task success and interaction cost. Action Efficiency. Across all benchmarks, LThinker++ exhibits a significantly steeper performance trajectory compared to the Vanilla baseline. Our model reaches the peak performance levels of the Vanilla model (achieved at 60 actions) with substantially fewer interactions. Specifically, on xbench, LThinker++ attains the Vanilla peak of 38.3% in only 24 actions, representing a 2.5×efficiency gain. Similar trends are observed on BrowseComp-ZH and BrowseComp-EN, which achieve 2.1×and 1.6×efficiency improvements, respectively. This efficiency stems primarily from our explicit memory management, which effectively mitigates context 20 rot. By folding redundant observations and expanding only task-relevant details, LThinker++ maintains a high-signal context window throughout extended interactions. This prevents the accumulation of irrelevant noise that typically degrades reasoning in the Vanilla baseline, allowing the agent to reach high-precision decisions with significantly fewer environment probes. Scaling with Action Budgets. The performance gap in Figure 11(a) reveals a critical advantage of our approach: superior information utility per environment interaction. Since the action budget only constrains 풜 env , LThinker++ is able to leverage its internalized memory-management actions (풜 mem ) to distill gathered data without increasing the external “search cost.” Notably, this performance margin widens as the budget expands. While the Vanilla baseline’s gains quickly plateau because the model becomes overwhelmed by information redundancy, LThinker++ maintains a more sustained growth trajectory through its internal thinking process. This allows the model to not only find answers faster but also achieve a higher performance ceiling, effectively turning the same amount of raw environment feedback into more accurate and robust reasoning outcomes. 6.3.2 Scaling with Horizon and Token Budgets. We evaluate the scaling characteristics of the model across two dimensions: the number of interaction rounds (Horizon) and total input consumption (Token Budget). Horizon Scaling. Figure 11(b) illustrates accuracy trends relative to maximum interaction rounds. Performance scales consistently with exploration depth, with the most significant gains occurring between 10 and 60 rounds. For instance, xbench accuracy surges to 44.0% as the horizon extends. The convergence observed after 80 rounds suggests that LThinker++ effectively balances exploration breadth and reasoning depth. Rather than being limited by information overload or “lost-in-the-middle” effects, the model successfully resolves complex queries within a strategic window, beyond which additional rounds yield diminishing utility as the solution space has been sufficiently exhausted. Token Budget Scaling. Figure 11(c) illustrates the performance trends as a function of the token budget. Accuracy improves rapidly in the low-budget regime and gradually saturates around 32k–48k tokens, indicating that most task-relevant information can be effectively utilized within a moderate context size. As the token budget increases further, performance remains stable up to 110k tokens, suggesting that the method can maintain consistent reasoning behavior under large-context settings. However, the observed plateau also highlights that once sufficient information is captured, the ultimate performance upper bound is governed by the model’s inherent reasoning capacity rather than further context expansion. Figure 12 Quantitative Analysis of Context Management Efficiency. Fig.(a) illustrates the active context trajectories (퐶 푡 ) across interaction rounds, highlighting the stable footprint of LThinker++ compared to the rapid inflation of the Vanilla baseline. Fig.(b) provides a statistical comparison of Mean and P95 (peak) token usage across tasks, demonstrating significant reductions in memory overhead. 21 6.3.3 Active Context Analysis. To quantify the efficiency of context management, we analyze the trajectory of the active context size per round퐶 푡 , defined as퐶 푡 = tokens(input 푡 )+ tokens(output 푡 ). Figure 12(a) illustrates the average퐶 푡 across the interaction horizon, while Figure 12(b) provides the corresponding Mean and P95 Peak token. Suppression of Contextual Growth. The Vanilla model suffers from rapid contextual inflation: its active context window swells to approximately 100k tokens within merely 50–60 rounds. This extreme redundancy not only consumes excessive resources but also triggers performance degradation due to noise. In sharp contrast, LThinker++ maintains a remarkably lean and stable footprint, staying between 30k and 40k tokens even as the interaction extends to 80 rounds. This suggests that LThinker++ effectively distills environmental feedback into high-density insights, allowing the agent to sustain long-horizon reasoning without the cognitive and computational strain of an unmanaged context. Dynamic Changes and Peak Control. Unlike the continuous growth observed in the baseline, the LThinker++ trajectory shows periodic changes driven by our expand-and-fold mechanism. The model temporarily expands context to process new evidence before condensing it into essential insights. Statistical results in Figure 12(b) show that LThinker++ consistently outperforms the Vanilla model in both Mean and P95 metrics. Notably, in BC_EN, our P95 peak of 46,104 tokens is not only much lower than the Vanilla peak of 110,634, but also approximately 24.3% lower than the Vanilla model’s mean usage of 60,928 tokens. This proves that LThinker++ effectively prevents “contextual explosion” even during intensive search phases. Extended Reasoning Lifespan. The termination patterns reveal a fundamental difference in exploration depth. Vanilla models often stop prematurely, which is typically triggered by degraded information processing capabilities rather than simple sequence length limits. As the context becomes increasingly noisy, these models suffer from lost-in-the-middle effects or hallucinations, causing them to provide incomplete answers before fully exploring the solution space. In contrast, by maintaining a high-density and manageable context, LThinker++ avoids such cognitive failures and supports sustained investigation beyond 80 rounds. This ensures the model’s reasoning lifespan is governed by task complexity rather than the internal strain of redundant information. 6.4 Ablation We conducted incremental ablation experiments to verify the effectiveness of our memory orchestration com- ponents. As illustrated in Figure 13, we compare three configurations: (1) Base, representing the vanilla model with standard SFT; (2) + commit, which introduces the summary-based memory commit mechanism; and (3) + all memory actions, our model incorporating the complete orchestration suite including Commit, Fold, and Ex- pand. Figure 13 Ablation results of memory orchestration. Impact of Information Distillation. Comparing the Base and + commit variants reveals that introducing thecommit mechanism improves Pass@1 performance from 38.3% to 41.7% on xbench and from 31.5% to 34.8% on BC_ZH. These gains indicate that distilling raw interaction data into structured evidence effectively filters environmental noise, which helps the model maintain logical consistency throughout extended reasoning trajectories. Benefits of Structural Context Control. The integra- tion offoldandexpandmechanisms in the + all memory actions model leads to the highest performance, reaching 44.0% on xbench and 36.9% on BC_ZH. These improve- ments show that dynamic context control is as important as information distillation: an example in Fig. 29 shows that when archiving omits crucial intermediate evidence, the model can expand the relevant past step to recover it from memory and fold it back after use. Consistency Across Benchmarks. The performance trends are highly consistent across both xbench and BC_ZH, with steady improvements observed as more orchestration components are added. This evolution confirms the generalizable value of our memory orchestration mechanism across different languages and task domains, ensuring that the model avoids early stopping during difficult investigations. 22 7 Related Work Current research on accelerating the inference process of large language models (LLMs) primarily focuses on three categories of methods: Quantizing Model, Generating Fewer Tokens, and Reducing KV Cache. Quantizing Model includes both parameter quantization [54,55] and KV Cache quantization [56,57], while this section will concentrate on the latter two categories. It is important to note that generating long texts and understanding long texts represent distinct application scenarios; therefore, acceleration methods specifically targeting the long-text generation phase (e.g., pre-filling stage acceleration techniques such as AutoCompressor [58], ICAE [59], LLMLingua [60], Activation Beacon [28], SnapKV [61], and PyramidKV [62]) are not discussed here. This section provides a detailed overview of the latter two categories while introducing a systematic analysis of Context Management. Reducing KV Cache. This category can be divided into two types of strategies: pruning-based KV Cache selection in discrete space and merging-based KV Cache compression in continuous space. 1) Pruning- Based Strategies. Specific eviction policies [21,22,63,64] are designed to retain important tokens during inference. For example, StreamingLLM [63] considers the initial sink tokens and the most recent tokens as important. H2O [21] focuses on tokens with high historical attention scores. SepLLM [22] emphasizes tokens corresponding to punctuation marks. 2) Merging-Based Strategies. Anchor tokens are introduced, and LLMs are trained to compress historically important information into these tokens, thereby achieving KV Cache merging [29]. Both strategies require intervention during inference. The key difference is that the first strategy is training-free but applies the eviction policy for every generated token, while the second strategy is a training-based method and allows the LLM to decide when to apply the eviction policy. Generating Fewer Tokens. This category can be further divided into three strategies based on the number and type of tokens used during inference. 1) Discrete Token Reduction. Techniques such as prompt engineering [10,11,12], instruction fine-tuning [13,14], or reinforcement learning [15,16] are used to guide LLMs to use fewer discrete tokens during inference. For example, TALE [10] prompts LLMs to complete tasks under a predefined token budget. Arora and Zanette construct specific datasets and employ reinforcement learning reward mechanisms to encourage models to generate concise and accurate outputs, thereby reducing token usage. TokenSkip [34] introduces a controllable framework that fine-tunes models on pruned CoT paths, enabling the selective skipping of redundant tokens at adjustable compression ratios. 2) Continuous Token Replacement. These methods [17,18] explore using continuous-space tokens instead of traditional discrete vocabulary tokens. A representative example is CoConut [18], which leverages Curriculum Learning to train LLMs to perform inference with continuous tokens. 3)No Token Usage. By internalizing the inference process between model layers, the final answer is generated directly during inference without intermediate tokens [19,20]. These three strategies are implemented after model training and do not require additional intervention during inference. Technically, the acceleration effect of these methods increases sequentially, but at the cost of a gradual decline in the generalization performance of LLMs. Additionally, the first strategy does not significantly reduce GPU memory usage. Context Management. Unlike methods that focus on hardware-level optimization or simply shortening the generation length, context management dynamically reorganizes the information within the context window throughout the reasoning process [65,66,67,68]. This approach is particularly critical for agents performing complex, long-horizon interactions. Specifically, MEM1 [69] and MemAgent [70] utilize rein- forcement learning to maintain a fixed-size internal memory, allowing agents to handle long-term tasks by retaining essential information and discarding redundant data. ReSum [71] addresses context constraints by periodically summarizing interaction histories, enabling agents to resume exploration from compact, state-based representations. Further advancing this paradigm, AgentFold [72] and Context-Folding [73], introduce a “folding” mechanism that compresses detailed interaction histories into compact reasoning states. Compared to token-level KV-cache pruning, these semantic-level methods better preserve the task-critical logic required for complex reasoning scenarios. 8 Conclusion In this paper, we present LightThinker, a new approach to enhance the efficiency of LLMs in complex reasoning tasks by dynamically compressing intermediate thoughts during generation. By training the LLM to learn when and how to compress verbose thought steps into compact representations, LightThinker significantly reduces memory overhead and computational costs while maintaining competitive accuracy. We introduce 23 the Dependency (abbr., Dep) metric to quantify the degree of compression across different accelerating methods. Extensive experiments demonstrate that LightThinker is an effective approach to balancing efficiency and performance. References [1]Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models.arXivpreprintarXiv:2303.18223, 1(2), 2023. [2]Amos Azaria, Rina Azoulay, and Shulamit Reches. Chatgpt is a remarkable tool—for experts.DataIntelligence, 6(1): 240–296, 2024. doi: 10.1162/dint_a_00235. [3] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,AdvancesinNeuralInformationProcessingSystems 35:AnnualConferenceonNeuralInformationProcessingSystems2022,NeurIPS2022,NewOrleans,LA,USA, November28-December9,2022, 2022. URLhttp://papers.nips.c/paper_files/paper/2022/hash/ 9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html. [4] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre Saraiva, Andrea Vallone, Andrew Duberstein, Andrew Kondrich, Andrey Mishchenko, Andy Applebaum, Angela Jiang, Ashvin Nair, Barret Zoph, Behrooz Ghorbani, Ben Rossen, Benjamin Sokolowsky, Boaz Barak, Bob McGrew, Borys Minaiev, Botao Hao, Bowen Baker, Brandon Houghton, Brandon McKinzie, Brydon Eastman, Camillo Lugaresi, Cary Bassin, Cary Hudson, Chak Ming Li, Charles de Bourcy, Chelsea Voss, Chen Shen, Chong Zhang, Chris Koch, Chris Orsinger, Christopher Hesse, Claudia Fischer, Clive Chan, Dan Roberts, Daniel Kappler, Daniel Levy, Daniel Selsam, David Dohan, David Farhi, David Mely, David Robinson, Dimitris Tsipras, Doug Li, Dragos Oprica, Eben Freeman, Eddie Zhang, Edmund Wong, Elizabeth Proehl, Enoch Cheung, Eric Mitchell, Eric Wallace, Erik Ritter, Evan Mays, Fan Wang, Felipe Petroski Such, Filippo Raso, Florencia Leoni, Foivos Tsimpourlas, Francis Song, Fred von Lohmann, Freddie Sulit, Geoff Salmon, Giambattista Parascandolo, Gildas Chabot, Grace Zhao, Greg Brockman, Guillaume Leclerc, Hadi Salman, Haiming Bao, Hao Sheng, Hart Andrin, Hessam Bagherinezhad, Hongyu Ren, Hunter Lightman, Hyung Won Chung, Ian Kivlichan, Ian O’Connell, Ian Osband, Ignasi Clavera Gilaberte, and Ilge Akkaya. Openai o1 system card.CoRR, abs/2412.16720, 2024. doi: 10.48550/ARXIV.2412.16720. URL https://doi.org/10.48550/arXiv.2412.16720. [5]Team Qwen. Qwq: Reflect deeply on the boundaries of the unknown, 2024. URLhttps://qwenlm.github.io/ blog/qwq-32b-preview/. [6]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng 24 Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. [7]Siwei Wu, Zhongyuan Peng, Xinrun Du, Tuney Zheng, Minghao Liu, Jialong Wu, Jiachen Ma, Yizhi Li, Jian Yang, Wangchunshu Zhou, Qunshu Lin, Junbo Zhao, Zhaoxiang Zhang, Wenhao Huang, Ge Zhang, Chenghua Lin, and Jiaheng Liu. A comparative study on reasoning patterns of openai’s o1 model.CoRR, abs/2410.13639, 2024. doi: 10.48550/ARXIV.2410.13639. URL https://doi.org/10.48550/arXiv.2410.13639. [8] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors,AdvancesinNeuralInformation ProcessingSystems30:AnnualConferenceonNeuralInformationProcessingSystems2017,December4-9,2017, LongBeach,CA,USA, pages 5998–6008, 2017. URLhttps://proceedings.neurips.c/paper/2017/hash/ 3f5e243547dee91fbd053c1c4a845a-Abstract.html. [9] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report.CoRR, abs/2412.15115, 2024. doi: 10.48550/ARXIV.2412.15115. URL https://doi.org/10.48550/arXiv.2412.15115. [10]Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware LLM reasoning.CoRR, abs/2412.18547, 2024. doi: 10.48550/ARXIV.2412.18547. URLhttps://doi.org/10. 48550/arXiv.2412.18547. [11]Mengru Ding, Hanmeng Liu, Zhizhang Fu, Jian Song, Wenbo Xie, and Yue Zhang. Break the chain: Large language models can be shortcut reasoners.CoRR, abs/2406.06580, 2024. doi: 10.48550/ARXIV.2406.06580. URL https://doi.org/10.48550/arXiv.2406.06580. [12]Sania Nayab, Giulio Rossolini, Giorgio C. Buttazzo, Nicolamaria Manes, and Fabrizio Giacomelli. Concise thoughts: Impact of output length on LLM reasoning and cost.CoRR, abs/2407.19825, 2024. doi: 10.48550/ARXIV.2407.19825. URL https://doi.org/10.48550/arXiv.2407.19825. [13]Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Jiayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. Can language models learn to skip steps? In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors,AdvancesinNeuralInformationProcessing Systems38:AnnualConferenceonNeuralInformationProcessingSystems2024,NeurIPS2024,Vancouver,BC, Canada,December10-15,2024, 2024. URLhttp://papers.nips.c/paper_files/paper/2024/hash/ 504fa7e518da9d1b53a233ed20a38b46-Abstract-Conference.html. [14]Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. C3ot: Generating shorter chain-of-thought without compromising effectiveness.CoRR, abs/2412.11664, 2024. doi: 10.48550/ARXIV.2412.11664. URLhttps://doi. org/10.48550/arXiv.2412.11664. [15]Daman Arora and Andrea Zanette. Training language models to reason efficiently.arXivpreprintarXiv:2502.04463, 2025. [16]Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning, 2025. [17]Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations.CoRR, abs/2412.13171, 2024. doi: 10.48550/ARXIV.2412.13171. URLhttps://doi.org/10. 48550/arXiv.2412.13171. [18]Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space.CoRR, abs/2412.06769, 2024. doi: 10.48550/ARXIV.2412.06769. URL https://doi.org/10.48550/arXiv.2412.06769. [19] Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart M. Shieber. Implicit chain of thought reasoning via knowledge distillation.CoRR, abs/2311.01460, 2023. doi: 10.48550/ARXIV.2311.01460. URL https://doi.org/10.48550/arXiv.2311.01460. 25 [20]Yuntian Deng, Yejin Choi, and Stuart M. Shieber. From explicit cot to implicit cot: Learning to internalize cot step by step.CoRR, abs/2405.14838, 2024. doi: 10.48550/ARXIV.2405.14838. URLhttps://doi.org/10.48550/arXiv. 2405.14838. [21]Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuan- dong Tian, Christopher Ré, Clark W. Barrett, Zhangyang Wang, and Beidi Chen. H2O: heavy-hitter oracle for efficient generative inference of large language models. In Alice Oh, Tristan Naumann, Amir Glober- son, Kate Saenko, Moritz Hardt, and Sergey Levine, editors,AdvancesinNeuralInformationProcessing Systems36:AnnualConferenceonNeuralInformationProcessingSystems2023,NeurIPS2023,NewOrleans, LA,USA,December10-16,2023, 2023. URLhttp://papers.nips.c/paper_files/paper/2023/hash/ 6ceefa7b15572587b78ecfcebb2827f8-Abstract-Conference.html. [22]Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, and Chao Huang. Sepllm: Accelerate large language models by compressing one segment into one separator.CoRR, abs/2412.12094, 2024. doi: 10.48550/ARXIV.2412.12094. URLhttps://doi.org/10.48550/arXiv.2412.12094. [23]Jesse Mu, Xiang Li, and Noah D. Goodman. Learning to compress prompts with gist tokens. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors,AdvancesinNeuralInformation ProcessingSystems36:AnnualConferenceonNeuralInformationProcessingSystems2023,NeurIPS2023,New Orleans,LA,USA,December10-16,2023, 2023. URLhttp://papers.nips.c/paper_files/paper/2023/ hash/3d77c6dcc7f143a2154e7f4d5e22d68-Abstract-Conference.html. [24]Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. Reasoning with language model prompting: A survey. InProceedingsofthe61stAnnualMeetingof theAssociationforComputationalLinguistics(Volume1:LongPapers), pages 5368–5393, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https://aclanthology.org/2023.acl-long.294. [25] Steven A Sloman. The empirical case for two systems of reasoning.Psychologicalbulletin, 119(1):3, 1996. [26] Daniel Kahneman. Thinking, fast and slow.Farrar,StrausandGiroux, 2011. [27]Grady Booch, Francesco Fabiano, Lior Horesh, Kiran Kate, Jonathan Lenchner, Nick Linck, Andreas Loreggia, Keerthiram Murgesan, Nicholas Mattei, Francesca Rossi, et al. Thinking fast and slow in ai. InProceedingsofthe AAAIConferenceonArtificialIntelligence, volume 35, pages 15042–15046, 2021. [28]Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. Long context compression with activation beacon, 2024. [29]Jianhui Pang, Fanghua Ye, Derek F. Wong, Xin He, Wanshun Chen, and Longyue Wang. Anchor-based large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors,FindingsoftheAssociation forComputationalLinguistics,ACL2024,Bangkok,Thailandandvirtualmeeting,August11-16,2024, pages 4958–4976. Association for Computational Linguistics, 2024. doi: 10.18653/V1/2024.FINDINGS-ACL.295. URL https://doi.org/10.18653/v1/2024.findings-acl.295. [30] OpenThoughts Team. Open Thoughts. https://open-thoughts.ai, January 2025. [31]Jintian Zhang, Cheng Peng, Mengshu Sun, Xiang Chen, Lei Liang, Zhiqiang Zhang, Jun Zhou, Huajun Chen, and Ningyu Zhang. OneGen: Efficient one-pass unified generation and retrieval for LLMs. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors,FindingsoftheAssociationforComputationalLinguistics:EMNLP 2024, pages 4088–4119, Miami, Florida, USA, November 2024. Association for Computational Linguistics. URL https://aclanthology.org/2024.findings-emnlp.237. [32]Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurélien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozière, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Graeme Nail, Grégoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel M. Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, 26 Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Alwala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, and et al. The llama 3 herd of models.CoRR, abs/2407.21783, 2024. doi: 10.48550/ARXIV.2407.21783. URL https://doi.org/10.48550/arXiv.2407.21783. [33]Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica.Deepscaler: Surpassing o1-preview with a 1.5b model by scaling rl.https://pretty-radio-b75.notion.site/ DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2 , 2025. Notion Blog. [34]Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of- thought compression in llms.CoRR, abs/2502.12067, 2025. doi: 10.48550/ARXIV.2502.12067. URLhttps: //doi.org/10.48550/arXiv.2502.12067. [35] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.CoRR, abs/2110.14168, 2021. URL https://arxiv.org/abs/2110.14168. [36]Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In9thInternationalConferenceonLearningRepresentations, ICLR2021,VirtualEvent,Austria,May3-7,2021. OpenReview.net, 2021. URLhttps://openreview.net/forum? id=d7KBjmI3GmQ. [37] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-proof q&a benchmark. InFirstConferenceonLanguage Modeling, 2024. URL https://openreview.net/forum?id=Ti67584b98. [38]Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki, editors,Findings oftheAssociationforComputationalLinguistics:ACL2023,Toronto,Canada,July9-14,2023, pages 13003– 13051. Association for Computational Linguistics, 2023. doi: 10.18653/V1/2023.FINDINGS-ACL.824. URL https://doi.org/10.18653/v1/2023.findings-acl.824. [39]Xiaomin Li, Zhou Yu, Zhiwei Zhang, Xupeng Chen, Ziji Zhang, Yingying Zhuang, Narayanan Sadagopan, and Anurag Beniwal. When thinking fails: The pitfalls of reasoning for instruction-following in llms, 2025. URL https://arxiv.org/abs/2505.11423. [40]Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. InProceedingsofthe2018 ConferenceonEmpiricalMethodsinNaturalLanguageProcessing, pages 2369–2380, 2018. [41] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition.TransactionsoftheAssociationforComputationalLinguistics, 10:539–554, 2022. [42]Jialong Wu, Baixuan Li, Runnan Fang, Wenbiao Yin, Liwen Zhang, Zhengwei Tao, Dingchu Zhang, Zekun Xi, Gang Fu, Yong Jiang, et al. Webdancer: Towards autonomous information seeking agency.arXivpreprintarXiv:2505.22648, 2025. [43]Zhengwei Tao, Jialong Wu, Wenbiao Yin, Junkai Zhang, Baixuan Li, Haiyang Shen, Kuan Li, Liwen Zhang, Xinyu Wang, Yong Jiang, et al. Webshaper: Agentically data synthesizing via information-seeking formalization.arXiv preprintarXiv:2507.15061, 2025. [44]Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, et al. Webwalker: Benchmarking llms in web traversal.arXivpreprintarXiv:2501.07572, 2025. [45]Z.ai. Glm-4.6: Advanced agentic, reasoning and coding capabilities, 2025. URLhttps://z.ai/blog/glm-4.6/. [46]Anthropic. System card: Claude opus 4 & claude sonnet 4, 2025. URLhttps://w-cdn.anthropic.com/ 6d8a8055020700718b0c49369f60816ba2a7c285.pdf. [47] OpenAI. Introducing gpt-5, 2025. URL https://openai.com/index/introducing-gpt-5/. [48] Kimi, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence.arXivpreprintarXiv:2507.20534, 2025. 27 [49] Qwen Team. Qwen3 technical report, 2025. URL https://arxiv.org/abs/2505.09388. [50] Xbench Team. Xbench-deepsearch, 2025. URL https://xbench.org/agi/aisearch. [51]Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents. arXivpreprintarXiv:2504.12516, 2025. [52] Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese.arXiv preprintarXiv:2504.19314, 2025. [53] Jina.ai. Jina, 2025. URL https://jina.ai/. [54] Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: activation-aware weight quantization for on-device LLM compression and acceleration. In Phillip B. Gibbons, Gennady Pekhimenko, and Christopher De Sa, editors,Proceedingsof theSeventhAnnualConferenceonMachineLearningandSystems,MLSys2024,SantaClara,CA,USA,May 13-16,2024. mlsys.org, 2024. URLhttps://proceedings.mlsys.org/paper_files/paper/2024/hash/ 42a452cbafa9d64e9ba4a95c1ef21-Abstract-Conference.html. [55]Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. Gpt3.int8(): 8-bit matrix multiplication for trans- formers at scale. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors,Advances inNeuralInformationProcessingSystems35:AnnualConferenceonNeuralInformationProcessingSystems2022, NeurIPS2022,NewOrleans,LA,USA,November28-December9,2022, 2022. URLhttp://papers.nips.c/ paper_files/paper/2022/hash/c3ba4962c05c49636d4c6206a97e9c8a-Abstract-Conference.html. [56]Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. KIVI: A tuning-free asymmetric 2bit quantization for KV cache. InForty-firstInternationalConferenceonMachine Learning,ICML2024,Vienna,Austria,July21-27,2024. OpenReview.net, 2024. URLhttps://openreview.net/ forum?id=L057s2Rq8O. [57]Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. Kvquant: Towards 10 million context length LLM inference with KV cache quantization. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Pa- quet, Jakub M. Tomczak, and Cheng Zhang, editors,AdvancesinNeuralInformationProcessingSystems 38:AnnualConferenceonNeuralInformationProcessingSystems2024,NeurIPS2024,Vancouver,BC, Canada,December10-15,2024, 2024. URLhttp://papers.nips.c/paper_files/paper/2024/hash/ 028fcbcf85435d39a40c4d61b42c99a4-Abstract-Conference.html. [58]Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. Adapting language models to compress contexts. In Houda Bouamor, Juan Pino, and Kalika Bali, editors,Proceedingsofthe2023ConferenceonEmpiricalMethods inNaturalLanguageProcessing,EMNLP2023,Singapore,December6-10,2023, pages 3829–3846. Association for Computational Linguistics, 2023. doi: 10.18653/V1/2023.EMNLP-MAIN.232. URLhttps://doi.org/10.18653/ v1/2023.emnlp-main.232. [59] Tao Ge, Jing Hu, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. InTheTwelfthInternationalConferenceonLearningRepresentations,ICLR2024,Vienna, Austria,May7-11,2024. OpenReview.net, 2024. URL https://openreview.net/forum?id=uREj4ZuGJE. [60]Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Llmlingua: Compressing prompts for accelerated inference of large language models. In Houda Bouamor, Juan Pino, and Kalika Bali, edi- tors,Proceedingsofthe2023ConferenceonEmpiricalMethodsinNaturalLanguageProcessing,EMNLP2023, Singapore,December6-10,2023, pages 13358–13376. Association for Computational Linguistics, 2023. doi: 10.18653/V1/2023.EMNLP-MAIN.825. URL https://doi.org/10.18653/v1/2023.emnlp-main.825. [61] Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. Snapkv: LLM knows what you are looking for before generation. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors,Advances inNeuralInformationProcessingSystems38:AnnualConferenceonNeuralInformationProcessingSystems2024, NeurIPS2024,Vancouver,BC,Canada,December10-15,2024, 2024. URLhttp://papers.nips.c/paper_ files/paper/2024/hash/28ab418242603e0f7323e54185d19bde-Abstract-Conference.html. 28 [62]Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Wen Xiao. Pyramidkv: Dynamic KV cache compression based on pyramidal information funneling.CoRR, abs/2406.02069, 2024. doi: 10.48550/ARXIV.2406.02069. URLhttps://doi.org/10.48550/arXiv.2406.02069. [63]Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. InTheTwelfthInternationalConferenceonLearningRepresentations,ICLR2024,Vienna,Austria, May7-11,2024. OpenReview.net, 2024. URL https://openreview.net/forum?id=NG7sS51zVF. [64] Jialong Wu, Zhenglin Wang, Linhai Zhang, Yilong Lai, Yulan He, and Deyu Zhou. SCOPE: optimizing key-value cache compression in long-context generation.CoRR, abs/2412.13649, 2024. doi: 10.48550/ARXIV.2412.13649. URL https://doi.org/10.48550/arXiv.2412.13649. [65]Lingrui Mei, Jiayu Yao, Yuyao Ge, Yiwei Wang, Baolong Bi, Yujun Cai, Jiazhi Liu, Mingyu Li, Zhong-Zhi Li, Duzhen Zhang, Chenlin Zhou, Jiayi Mao, Tianze Xia, Jiafeng Guo, and Shenghua Liu. A survey of context engineering for large language models.CoRR, abs/2507.13334, 2025. doi: 10.48550/ARXIV.2507.13334. URL https://doi.org/10.48550/arXiv.2507.13334. [66]Zhuoen Chen, Dongfang Li, Meishan Zhang, Baotian Hu, and Min Zhang. Dynamic long context reasoning over compressed memory via end-to-end reinforcement learning, 2026. URLhttps://arxiv.org/abs/2602.08382. [67]Yilun Zheng, Dongyang Ma, Tian Liang, Jiahao Xu, Xinting Huang, Lijie Chen, Haitao Mi, and Yan Wang. Free(): Learning to forget in malloc-only reasoning models, 2026. URL https://arxiv.org/abs/2602.08030. [68]Xiaoyuan Liu, Tian Liang, Dongyang Ma, Deyu Zhou, Haitao Mi, Pinjia He, and Yan Wang. The pensieve paradigm: Stateful language models mastering their own context, 2026. URL https://arxiv.org/abs/2602.12108. [69]Zijian Zhou, Ao Qu, Zhaoxuan Wu, Sunghwan Kim, Alok Prakash, Daniela Rus, Jinhua Zhao, Bryan Kian Hsiang Low, and Paul Pu Liang. MEM1: learning to synergize memory and reasoning for efficient long-horizon agents.CoRR, abs/2506.15841, 2025. doi: 10.48550/ARXIV.2506.15841. URLhttps://doi.org/10.48550/arXiv.2506.15841. [70]Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, and Hao Zhou. Memagent: Reshaping long-context LLM with multi-conv rl-based memory agent.CoRR, abs/2507.02259, 2025. doi: 10.48550/ARXIV.2507.02259. URLhttps://doi.org/10. 48550/arXiv.2507.02259. [71]Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Yong Jiang, Pengjun Xie, Fei Huang, Minhao Cheng, Shuai Wang, Hong Cheng, and Jingren Zhou. Resum: Unlocking long-horizon search intelligence via context summarization.CoRR, abs/2509.13313, 2025. doi: 10.48550/ARXIV.2509.13313. URL https://doi.org/10.48550/arXiv.2509.13313. [72] Rui Ye, Zhongwang Zhang, Kuan Li, Huifeng Yin, Zhengwei Tao, Yida Zhao, Liangcai Su, Liwen Zhang, Zile Qiao, Xinyu Wang, Pengjun Xie, Fei Huang, Siheng Chen, Jingren Zhou, and Yong Jiang. Agentfold: Long-horizon web agents with proactive context management.CoRR, abs/2510.24699, 2025. doi: 10.48550/ARXIV.2510.24699. URL https://doi.org/10.48550/arXiv.2510.24699. [73]Weiwei Sun, Miao Lu, Zhan Ling, Kang Liu, Xuesong Yao, Yiming Yang, and Jiecao Chen. Scaling long-horizon LLM agent via context-folding.CoRR, abs/2510.11967, 2025. doi: 10.48550/ARXIV.2510.11967. URLhttps: //doi.org/10.48550/arXiv.2510.11967. [74] Bespoke Labs.Bespoke-stratos:The unreasonable effectiveness of reasoning distillation. https://w.bespokelabs.ai/blog/bespoke-stratos-the-unreasonable-effectiveness-of-reasoning-distillation, 2025. Accessed: 2025-01-22. [75] Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, Wenmeng Zhou, and Yingda Chen. SWIFT: A scalable lightweight infrastructure for fine-tuning. In Toby Walsh, Julie Shah, and Zico Kolter, editors,AAAI-25,SponsoredbytheAssociationfortheAdvancementof ArtificialIntelligence,February25-March4,2025,Philadelphia,PA,USA, pages 29733–29735. AAAI Press, 2025. doi: 10.1609/AAAI.V39I28.35383. URL https://doi.org/10.1609/aaai.v39i28.35383. [76] Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, and Bang Liu. System-1.5 reasoning: Traversal in language and latent spaces with dynamic shortcuts.CoRR, abs/2505.18962, 2025. doi: 10.48550/ARXIV.2505.18962. URL https://doi.org/10.48550/arXiv.2505.18962. [77]Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of llms in continuous concept space.CoRR, abs/2505.15778, 2025. doi: 10.48550/ARXIV.2505.15778. URL https://doi.org/10.48550/arXiv.2505.15778. 29 Appendix A Metric: Dependency Peak Tokens Prompt Length (a) Vanilla (c) AnLLM & Ours(b) H2O Context LengthContext Length Generated Tokens Length Generated Tokens Length Generated Tokens Length Context Length Peak Tokens DependencyDependencyDependency Prompt Length Prompt Length Peak Tokens Figure 14 An illustration of the Dependency metric. A.1 Motivation LightThinker and AnLLM [29] belong to the class of dynamic compression methods, where the number of compressions and the compression ratio are determined automatically by the LLM rather than specified as fixed hyperparameters. By contrast, H2O [21] and SepLLM [22] rely on user-defined hyperparameters to constrain the maximum number of tokens retained during inference. Because of this difference, directly comparing dynamic methods such as LightThinker and AnLLM with KV cache compression methods like H2O and SepLLM is not straightforward. A common practice for KV cache compression is to compare methods under the same maximum peak token count. However, this criterion is not fully suitable here. As shown in Fig. 14, which plots generated tokens against context length for Vanilla, H2O, and LightThinker, LightThinker may occasionally have a larger peak token count than H2O. Even so, this does not necessarily mean higher memory cost, since LightThinker reaches its peak only briefly, whereas H2O keeps a high token count throughout the generation process. Another limitation is that prior KV cache compression methods often compress only the prompt and assume a fixed prompt length, which makes it possible to predefine the compression ratio. In our setting, however, the generated output must also be compressed. Since the output length is unknown in advance, it is impossible to specify a single global compression ratio beforehand. Therefore, peak token count alone is insufficient for a fair comparison. To address these issues, we introduce a metric called Dependency, which measures the total degree of information dependence throughout generation. It provides a common basis for comparing dynamic compression methods with traditional KV cache compression approaches under comparable effective compression ratios. 30 A.2 Definition We define Dependency (or Dep) as the cumulative dependency of each generated token on preceding tokens during decoding. Geometrically, this quantity corresponds to the area under the curve in Fig. 14. It can be computed either directly from the definition or by using this geometric interpretation; here we focus on the latter. Let the prompt length be퐿 푃 , the output length be퐿 푂 , and the cache limit imposed by KV cache compression methods be 퐿 퐶 . Dependency for Vanilla. For Vanilla, the curve forms a right trapezoid, and the dependency is: Dependency= (퐿 푃 + 퐿 푃 + 퐿 푂 )× 퐿 푂 2 = 퐿 푂 2 2 + 퐿 푃 × 퐿 푂 Dependency for H2O. For H2O, the area consists of a trapezoid on the left side of Fig. 14(b) and a rectangle on the right side: 푆 Trapezoid = (퐿 푃 + 퐿 퐶 )×(퐿 퐶 − 퐿 푃 ) 2 푆 rectangle = 퐿 퐶 ×(퐿 푂 − 퐿 퐶 + 퐿 푃 ) Dependency= 푆 Trapezoid + 푆 rectangle = 2퐿 푃 퐿 퐶 + 2퐿 푂 퐿 퐶 − 퐿 푃 2 − 퐿 퐶 2 2 Dependency for LightThinker and AnLLM. For LightThinker and AnLLM, Dependency does not admit a closed-form expression and must instead be computed step by step according to the definition above. A.3 Application Interpreting Dependency. A larger Dependency value means that more contextual information is involved during generation, which reflects heavier information usage. Conversely, a smaller value indicates stronger effective compression. Dependency Ratio. By taking the ratio between the Dependency of an accelerated method and that of Vanilla, we obtain a relative compression ratio. For instance, in the “Avg.” column of Table 1, Vanilla has a Dependency of 16.6M, H2O has 4.4M, and LightThinker has 3.7M. This corresponds to compression ratios of 16.6 4.4 ≈ 3.8 for H2O and 16.6 3.7 ≈ 4.5 for LightThinker. Overall, this metric offers a unified way to evaluate both dynamic and static compression methods, enabling fair and meaningful comparisons. (a) AnLLM’s Attention Mask. (b) LightThinker’s Attention Mask. (|C|=1, T) (c) LightThinker’s Attention Mask. (|C|=1, F) Figure 15 An illustration of the attention mask used in Table 4. B Mathematical Description of Compression This section provides a formal description of the compression operation introduced in Section 3.2. 31 Notation. During compression, the context can be decomposed into three parts: 1. the portion that remains uncompressed, denoted by푃푟푒 :=푋,퐶 (1) , [o] (1) ... , 퐶 (푖−1) , [o] 푖−1 , with token count푁; 2. the thought sequence to be compressed, denoted by푇ℎ표 := 푆 푖 , with length푇; 3. the compressed segment, denoted by 퐶 := 퐶 (푖) , with length|퐶|. Compression Operation. We now describe the compression process at a given layer, focusing on how information is transferred into the sequence퐶. Following the definition of self-attention [8], the attention matrix for 퐶 over the other segments is computed as: 퐴= Softmax(mask( 푄 퐶 [퐾 푃푟푒 : 퐾 푇ℎ표 : 퐾 퐶 ] ⊤ √ 푑 )) where[:]denotes concatenation,mask(·)is the attention mask defined by the “Thought-based Attention Mask Construction”,퐾 푃푟푒 ,푉 푃푟푒 ∈ ℝ 푁×푑 ,퐾 푇ℎ표 ,푉 푇ℎ표 ∈ ℝ 푇×푑 ,퐾 퐶 ,푉 퐶 ∈ ℝ |퐶|×푑 ,푄 퐶 ∈ ℝ |퐶|×푑 , and푑is the hidden dimension. Thus,퐴∈ ℝ |퐶|×(푁+푇+|퐶|) specifies how each element in퐶attends to the rest of the context. The corresponding values are then aggregated through: 퐻= 퐴×[푉 푃푟푒 : 푉 푇ℎ표 : 푉 퐶 ] where[푉 푃푟푒 : 푉 푇ℎ표 : 푉 퐶 ]∈ ℝ (푁+푇+|퐶|)×푑 , and therefore퐻 ∈ ℝ |퐶|×푑 . In this way, information from the current 푇ℎ표is retained in퐻. During training, the model learns to preserve the most useful content from푇ℎ표in this representation. Finally, after being passed through an MLP and the next layer’s projection,퐻is stored in the KV cache. C Experiment C.1 LightThinker C.1.1 Training Data Examples of training samples are provided in Fig. 27. C.1.2 Baseline Details H2O [21] is a training-free acceleration method that greedily retains the tokens with the largest cumulative attention scores from historical context. It has two hyperparameters: the maximum number of retained tokens and the current window size (i.e.,local_size). The maximum token limit for each task is listed in the “Peak” column of Table 1, andlocal_sizeis set to half of this value. Our implementation is based on https://github.com/meta-llama/llama-cookbook. SepLLM [22] is another training-free acceleration method that treats tokens at punctuation positions as more important. It uses four hyperparameters: the maximum number of tokens is set to 1024,local_size to 256,sep_cache_sizeto 64, andinit_cache_sizeto 384. We also tried an alternative configuration (init_cache_size=4,sep_cache_size=64,local_size=720, maximum number of tokens=1024), but the first setting performed slightly better. AnLLM [29] is a training-based method that follows a workflow similar to LightThinker but accelerates inference by storing historical content in anchor tokens. The key differences between the two methods are described in Section E. C.1.3 Training Details Both Vanilla and AnLLM are trained on the BS17K [74] dataset using the R1-Distill [6] model for 5 epochs, while LightThinker is trained for 6 epochs. We set the maximum sequence length to 4096 and adopt a cosine warmup schedule withwarmup_ratio=0.05. All experiments are conducted on 4 A800 GPUs with DeepSpeed ZeRO-3 offload enabled. The batch size per GPU is 5, and we use a gradient accumulation step of 4, which results in a global batch size of 80. The learning rate is set to 1e-5 for Vanilla and 2e-5 for both AnLLM and LightThinker. 32 C.1.4 Evaluation Details For the CoT results in Table 1, we use the prompts shown in Fig. 23 and Fig. 26. For the R1-Distill model, no system prompt is used; the task-specific prompts are shown in Fig. 25. Vanilla, H2O, SepLLM, AnLLM, and LightThinker all use the same prompt templates, with the system prompt shown in Fig. 24 and the task prompts shown in Fig. 25. For the multiple-choice tasks in MMLU [36] and GPQA [37], the answer options are randomized. C.1.5 Implementation In our implementation, we use two segmentation functionsSeg()depending on the compression level. For the token-level setting, we compress every 6 tokens into 2 tokens, i.e.,|퐶|= 2, and refer to this variant as “ours (token)”. For the thought-level setting, we take “ ” as the delimiter to split BS17K samples into thoughts, which we denote as “ours (tho.)”. Under this setting, each thought is compressed into 9 tokens for Qwen and 7 tokens for Llama, corresponding to|퐶|= 9and|퐶|= 7, respectively. In all experiments, we perform greedy decoding with a maximum output length of 10240 tokens. C.1.6 Additional Results Additional figures are provided to complement the main results. Fig. 16 reports how many tokens are generated on average by the two models across datasets. Fig. 22 summarizes the distribution of compressed lengths produced by LightThinker on both models over four datasets. Fig. 15 visualizes the attention masks adopted by the baseline methods in Table 4. Fig. 28 presents the full example corresponding to the case study in Fig. 6. Figure 16 Average number of generated tokens. C.2 LightThinker++ C.2.1 Training Data. General Reasoning Data. The training set for general reasoning is curated from a diversified collection of high-difficulty mathematical and logical problems. We primarily utilize two high-quality reasoning corpora: 1) a refined subset of BS17K, where code-related tasks were excluded to maintain focus on linguistic and mathematical logic, resulting in 11,315 samples; and 2) a sampled subset of 6,000 instances from 33 the DeepScaleR dataset. In total, 17,315 unique problems form the foundation of our general reasoning trajectories. We employDeepSeek-V3.2as the teacher model to synthesize long-thought reasoning paths. The synthesis prompt used to guide the teacher model is shown in Figure 17. These trajectories are subsequently reconstructed into the LightThinker and LightThinker++ formats by inserting memory management labels (e.g.,commit,expand), ensuring the model internalizes the ability to compress thoughts during complex deduction. Then through the rigorous filtering and transformation process, we obtain a final expert dataset comprising 13,855 high-quality trajectories. Agentic Research Data. For the interactive research tasks (DeepResearch), the base query pool is curated from a diversified ensemble of sources, including HotpotQA, MuSiQue, WebDancer, WebShaper, and WebWalkerQA- Silver. Table 10 provides the detailed statistics of the 8,954 unique queries and the resulting trajectories after our hierarchical filtering pipeline. The prompt used for synthesizing these agentic research trajectories is illustrated in Figure 18. Table 10 Breakdown of the Base Query Pool for Agentic Research Synthesis. Query SourceCount HotpotQA + MuSiQue (Filtered by Qwen3) 5,322 WebDancer200 WebShaper500 WebWalkerQA-Silver (EN + Hard ZH)2,932 Total Base Queries8,954 C.2.2 Training and Inference Details for General Reasoning. For Vanilla, TokenSkip, and LightThinker++, we follow the same training setup as above, except that we increase the maximum sequence length to 16,384 and adopt a cosine learning-rate schedule with a warmup ratio of 0.05. All methods are trained for three epochs. All experiments are conducted on 8 A800 GPUs with DeepSpeed ZeRO-3 offload enabled. We use a batch size of 32, and set the learning rate to1× 10 −5 for Vanilla and2× 10 −5 for TokenSkip and LightThinker++. During inference, we set the temperature to 0.7 and perform three independent sampling runs per example, reporting the average performance over the three runs. C.2.3 Evaluation Details for General Reasoning The evaluation of the instruct model on general reasoning tasks follows the distribution shown in Figure 16. In our implementation, to ensure consistency between LightThinker and LightThinker++, we remove system prompts and instead rely on specialized task prompts to guide the reasoning process. The task prompts used for evaluation are shown in Figure 19, which includes both the Vanilla and the LightThinker++ models. C.2.4 Agentic Implementation of LightThinker++ We develop our training pipeline using the ms-swift framework [75] with Qwen3-30B-A3B-Thinking as the foundational backbone. All models are fine-tuned for 3 epochs on a cluster of 8 GPUs, employing a cosine learning rate scheduler with a푤푎푟푚푢푝_푟푎푡푖표of0.05. For the Vanilla baseline, the learning rate is set to1× 10 −5 with a global batch size of 32 and a maximum sequence length of 32,768. In contrast, LightThinkerutilizes a learning rate of 2× 10 −5 , a global batch size of 64, and a sequence length of 16,384. C.2.5 Timing Protocol and Throughput Setting for Agentic Tasks To align the effective concurrency across methods, we use a total concurrency of 32 in all timing experiments. For Vanilla and all LThinker-based methods, we measure time with 32-way question-level parallel inference. For TokenSkip, we measure time with a batch size of 32, since its throughput drops sharply under question- level concurrency in our serving implementation; thus we time TokenSkip in its intended batched decoding mode to avoid an implementation-induced penalty. 34 System Prompt: You are a highly intelligent mathematical reasoning agent that solves problems by committing a sequence of calculation steps to a persistent scratchpad. Every reasoning step you take is automatically recorded. Core Concept: Committing Steps to Memory - When you callcommit, the system archives your detailed work and automatically creates a summary of the result, assigning it a [Step ID]. - You do not need to call a separate function to save your work; committing your steps (commit) is the memory-saving action itself. Problem-Solving Workflow: 1. Calculate & Commit: Usecommitto perform calculations and logical reasoning. Each call creates a new summary step on your scratchpad. 2. Manage Context: The summaries of your past steps form your working memory. - To access the full details of a past step, use expand. - Conditional Expand Rule: Before using a past step as a dependency, you MUSTexpandit only if its summary does not contain the exact values, formulas, or assumptions you need, or if you have any uncertainty about correctness, domain constraints, or edge cases. If the summary is sufficient, explicit, and unambiguous, you may proceed without expand. - Usage:expand(step_id=N)reveals the full, detailed calculations and formulas from a previous committed step. - Efficiency: To maintain a clear workspace, do not keep more than 2 items expanded simultaneously. - Use the information: After expanding, you MUST use the revealed details in your nextcommit before folding. - Cleanup: Only after you have used the expanded details in a new reasoning step, you MUST fold the summary to keep your context clean. 3. Completion: Once all calculations are done and you have the final answer, you MUST callfinal_- answer to submit the result. Key Rules: - Your output must ONLY be function calls (commit,expand,fold,final_answer). No conversational text. - Critical Workflow (when expanding):expand→ commit(using expanded information)→ fold. This sequence is mandatory and must not be broken. - Always fold an expanded step once you are done with its details. - Fold Validity: Afold(step_id=N)call is invalid unless the samestep_id=Nwas expanded earlier and is currently expanded. - Yourfinal_answermust contain the complete solution with the key result enclosed in . Figure 17 Synthesis prompt used for generating reasoning data. 35 System Prompt You are a deep research assistant. Your core function is to conduct thorough, multi-source investigations into any topic. You must handle both broad, open-domain inquiries and queries within specialized academic fields. For every request, synthesize information from credible, diverse sources to deliver a comprehensive, accurate, and objective response. When you have gathered sufficient information and are ready to provide the definitive response, you must enclose the entire final answer within <answer></answer> tags. Available tools for managing information during investigation: 1. OPERATIONAL LOGIC: TOOL CHOICE Every step in your history is assigned anid(e.g.,[Thought ID],[Observation ID]). Use tools based on these logic states: • Information Acquisition: Usesearch(query)orvisit(url)to find new data or explore primary sources. • Deepening or Re-visiting (expand): –Discrepancy Resolution: Useexpand(id)to compare conflicting data points across sources by evaluating the raw context, source authority, and publication dates. –Granular Extraction / Raw Data Inspection: Useexpand(id)when a summary is too vague or blurred to provide required specific evidence (e.g., exact quotes, full names, specific numbers, statistics), or when you need to inspect the full raw content to verify what was actually stated. – Re-verification / Latent Clue Retrieval: Useexpand(id)to re-examine a previously folded (or secondary) record when it becomes newly relevant due to a shift in reasoning, when you identify a new key entity/clue, or when a previously used detail must be re-verified before concluding. • Post-Expansion Cleanup (fold): –Strict Precondition (Must Have Been Expanded): You may callfold(id)ONLY for anidthat you have previously called expand(id) on, and that is currently in an Expanded state. ̳ Do NOT call fold on Raw or Folded records. ̳ Do NOT call fold on an id that was never expanded. ̳After folding, thatidis no longer expanded; do not immediately fold it again unless it is expanded again later. –Timing: Once you have utilized the information from an expanded record—either to inform your reasoning or to decide your next action (such as a newsearchorvisit)—you MUST immediately call fold(id) in the next turn to return it to its folded state. –Efficiency: To maintain a clear workspace, do not keep more than 2 items expanded simultane- ously. 2. TOOL PROTOCOLS • search(query): Find new, credible information on the web. • visit(url, goal): Extract detailed, specific information from a webpage. • expand(id, reason): Reveals the full content of a folded record for deep inspection or re-visiting. • fold(id, reason): Collapses a previously expanded record back into its folded state after use. 3. CRITICAL RULES •One Action Per Turn: Output either a single tool call (preceded by your thought) OR the final answer within <answer> tags. •Input Reality: After callingexpand, your first priority in the next turn is to re-read the now-visible full content for that id before taking further steps. •Evidence-Based: Base your synthesis only on details currently visible. Never guess information that is hidden behind a folded record. •Fold Validity: Afold(id)call is invalid unless the sameidwas expanded earlier and is currently expanded. • Definitive Answer: Only provide the final answer once all aspects of the query are thoroughly verified and synthesized. Current date: Figure 18 Synthesis prompt used for agentic research trajectory generation and filtering. 36 GSM8K/MMLU/GPQA/BBH: Vanilla & Initial Round of LightThinker++ Return your final response within . ### Question: Question Subsequent Rounds of LightThinker++ Return your final response within . ### Question: Question ### Current Scratchpad: scratchpad Figure 19 Task prompt forVanilla, LightThinker++ in general reasoning tasks for both Qwen-based model and Llama-based model. D Efficiency Analysis of LightThinker++ (Budget setting) Figure 20 Efficiency Analysis and Ablation Results under the Budget setting. Fig.(a) illustrates the average number of generated tokens retained in the current context. Fig.(b) presents the distribution ratio of memory actions (Commit vs. Expand+Fold) for Qwen-2.5-7B. Fig.(c) shows the percentage of pre-commit token lengths falling within specified ranges on GPQA(Qwen-2.5-7B). Fig.(d) examines the scaling of peak memory tokens as a function of the generation budget. Fig.(e) summarizes the average compression factors and saved token ratios across different model series. Fig.(f) provides an ablation study comparing accuracy and average peak tokens between the full LThinker++ and its degraded variants. E Discussions E.1 Difference between LightThinker and AnLLM Although AnLLM [29] and LightThinker both aim to accelerate LLM inference, they differ substantially in motivation and design. AnLLM was proposed before the emergence of long-CoT methods [4,6], and its primary focus is prompt compression rather than output compression. By contrast, LightThinker decouples compression from generation, which makes it possible to scale the number of cache tokens in a way that 37 AnLLM does not support. As a result, the two methods only share the use of sparse attention [21,61] for efficiency, but are otherwise quite different. To make this distinction clearer, Fig. 21 highlights two key differences in their attention-mask design: 1) Separation of compression and generation. In AnLLM, the[c i ]token is responsible for both summarizing historical information and producing the next content, as indicated by the blue and pink arrows in Fig. 21. This means that compression and generation are tightly coupled. LightThinker instead assigns these two roles to different tokens:[c i ]is used only for compression, while the[o]token carries out reasoning based on the compressed context. 2) Visibility of context during compression. When AnLLM performs compression, it can only attend to the current thought. In contrast, LightThinker allows the compression step to condition on the original prompt푋, previously compressed content, and the current thought, which provides richer contextual information. These design differences are validated by the ablation results in Section 4.2.3, which show clear performance gains. only S2 is not masked 1 Compression 2 Generation Coupled Disoupled (a) AnLLM’s Attention Mask. X, previous compressed content, and S2 are not masked. (b) Ours’ Attention Mask. Figure 21 Comparison between AnLLM and LightThinker. The two main differences are highlighted with a red box and with blue/pink arrows. E.2 Viewing LightThinker from Other Perspectives Beyond the compression view adopted in earlier sections, LightThinker can also be interpreted from the perspectives of memory and KV cache compression, since KV cache itself can be regarded as a form of working memory for LLMs. From the memory perspective, the workflow of LightThinker can be viewed as a loop of reasoning, storing important information, and reasoning again based on the stored content. In this sense, the cache tokens serve as a compact memory for the current model, although such memory is model-specific and does not transfer across LLMs. From the KV cache compression perspective, LightThinker differs from methods such as H2O [21], which use manually designed eviction policies to keep important tokens. Instead, LightThinker combines previous tokens in a continuous space and creates new representations through learning. In other words, the model itself determines what to merge and how to merge it, rather than relying on a discrete token-selection rule. E.3 Why LightThinker generates more tokens with smaller cache size? As shown in Fig. 5(e-f), LightThinker tends to generate more tokens when the cache size is smaller. By examining the generated outputs under different cache sizes, we found that smaller caches often cause the model to repeat earlier content more frequently. A possible explanation is that a smaller cache leads to greater information loss during compression, which in turn forces the model to regenerate previously mentioned content more often in order to preserve as much information as possible. E.4 Comparison with Implicit CoT Works Although both our method and implicit CoT approaches operate in continuous spaces, they differ in how reasoning is carried out. Implicit CoT performs the entire reasoning process in continuous space, whereas 38 Figure 22 Token compression frequency distribution for LightThinker. 39 System Prompt: Below is a question. Please think through it step by step, and then provide the final answer. If options are provided, please select the correct one. ## Output format: Use “<THOUGHT>...</THOUGHT>” to outline your reasoning process, and enclose the final answer in ‘ ‘. ## Example 1: Question: What is 2 + 3? Output: <THOUGHT>First, I recognize that this is a simple addition problem. Adding 2 and 3 together gives 5.</THOUGHT> Therefore, the final answer is 5. ## Example 2: Question: What is 2 + 3? A. 4 B. 5 C. 10 Output: <THOUGHT>First, I recognize that this is a simple addition problem. Adding 2 and 3 together gives 5.</THOUGHT> Therefore, the final answer is B. Figure 23 System prompt for Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct. System Prompt: Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. Please structure your response into two main sections: Thought and Solution. In the Thought section, detail your reasoning process using the specified format:<|begin_of_thought|>thought with steps separated with ‘ ’ <|end_of_thought|>Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps. In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows:<|begin_of_solution|> final formatted, precise, and clear solution<|end_of_thought|>Now, try to solve the following question through the above guidelines: Figure 24 System prompt forVanilla,H2O,SepLLM,AnLLM, andLightThinkerfor both Qwen-based model and Llama-based model. GSM8K/MMLU/GPQA/BBH: Return your final response within . Question Figure 25 Task prompt forDistill-R1,Vanilla,H2O,SepLLM,AnLLM, andLightThinkerfor both Qwen-based model and Llama-based model. 40 GSM8K: Return your final response within . Question MMLU: Please select the option that best answers the question. Return your final response within . Here are the Question: Question GPQA: Given a question, please select the option that best answers it. Return your final response within . Question BBH: Return your final response within . If options are provided, please select the correct one. Question Figure 26 Task prompt for Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct. System Prompt: See Figure 24 Input: Return your final response within . The average of the numbers 23 and푥is 27. What is the positive difference between 23 and푥? (Recall that the average of two numbers is their sum divided by 2). Output: <|begin_of_thought|>Okay, let’s see. The problem says that the average of the numbers 23 and x is 27. We need to find the positive difference between 23 and x. Hmm, right. Since the average of two numbers is their sum divided by 2, I can start by setting up the equation for the average.<w>So, the average of 23 and x is (23 + x)/2. According to the problem, this average equals 27. So I can write the equation:<w>(23 + x)/2 = 27<w>To find x, I should solve this equation. Let me multiply both sides by 2 to get rid of the denominator. That gives:<w>23 + x = 54<w>Then, subtract 23 from both sides to isolate x:<w>x = 54 - 23<w>Calculating that, 54 minus 23 is 31. So x is 31. Now, the question asks for the positive difference between 23 and x. The positive difference would be the absolute value of 23 - x. Since x is 31, that’s |23 - 31|, which is | -8 |, so 8. Alternatively, it could also be |31 - 23|, which is the same result, 8. Either way, the positive difference is 8.<w>Wait, let me check my steps again to make sure I didn’t make a mistake. Starting with the average: (23 + x)/2 = 27. Multiply both sides by 2: 23 + x = 54. Subtract 23: x = 31. Difference between 23 and 31 is 8. That seems right. Yeah, I think that’s correct. No issues here.<w> <|end_of_thought|> <|begin_of_solution|> <w>The average of 23 and푥is given by 23+푥 2 = 27. Multiplying both sides by 2: 23+ 푥= 54 Subtracting 23 from both sides: 푥= 54− 23= 31 The positive difference between 23 and 푥 is: |31− 23|= 8 <w> 8 <|end_of_solution|> Figure 27 A training case. <w> is a compression trigger. 41 Question: Boris owns a chocolate factory. He produces 50,000 bars of chocolate each month. Boris produces 8,000 bars of chocolate the first week. The second week, Boris only produces half as much as the first week. But, the third week, Boris produces three times as much as the first week. How much does he produce the fourth week? Model Output: <|begin_of_thought|> Okay, let’s see. Boris has a chocolate factory that produces 50,000 bars each month. The question is ask- ing how much he produces in the fourth week. Hmm, let me break this down step by step. <|splitter|> First, the total monthly production is 50,000 bars. The month is divided into four weeks, right? So the production for each week adds up to 50,000. The problem gives the production for the first three weeks and asks for the fourth week. Let me note down the given information. <|splitter|> First week: 8,000 bars. That’s straightforward. Then the second week, he produces half as much as the first week. Half of 8,000 is 4,000. So second week is 4,000 bars. Then the third week, he produces three times as much as the first week. Three times 8,000 is 24,000. So third week is 24,000 bars. <|splitter|> Now, to find the fourth week’s production, I need to subtract the sum of the first three weeks from the total monthly production. Let me add up the first three weeks: 8,000 + 4,000 + 24,000. Let’s do the math step by step. 8,000 plus 4,000 is 12,000. Then 12,000 plus 24,000 is 36,000. So the total for the first three weeks is 36,000 bars. <|splitter|> The total monthly production is 50,000 bars. Subtracting the first three weeks from the to- tal gives the fourth week’s production. So 50,000 minus 36,000 equals 14,000 bars. Therefore, Boris produces 14,000 bars in the fourth week. <|splitter|> Wait a second, let me double-check my calculations to make sure I didn’t make a mistake. First week: 8,000. Second week: half of 8,000 is indeed 4,000. Third week: three times 8,000 is 24,000. Adding those: 8,000 + 4,000 = 12,000; 12,000 + 24,000 = 36,000. Total monthly is 50,000. So 50,000 - 36,000 = 14,000. Yep, that seems right. I don’t think I missed anything here. The problem states the production for each week as fractions or multiples of the first week, so the calculations are straightforward. No complicated formulas needed. Just basic arithmetic. So the answer should be 14,000 bars in the fourth week. <|splitter|> <|end_of_thought|> <|begin_of_solution|> Boris’s weekly chocolate production is calculated as follows: <|splitter|> - First week: 8,000 bars - Second week: 8,000 2 = 4, 000 bars - Third week: 8, 000× 3= 24, 000 bars <|splitter|> Total for the first three weeks: 4, 000+ 4, 000+ 24, 000= 32, 000 bars <|splitter|> Subtracting this from the monthly total: 50, 000− 32, 000= 18, 000 bars <|splitter|> Thus, Boris produces 18000 bars in the fourth week. <|splitter|> <|begin_of_solution|> Figure 28 Bad Case. <|splitter|> is equal to <w> in Figure 27. LightThinker adopts a hybrid scheme that combines continuous and discrete reasoning. The main differences are summarized in Table 11, and we highlight them below. •Reasoning Acceleration Mechanism. Implicit CoT methods, such as System-1.5 [76] and SoftThink- ing [77], speed up reasoning by reducing the number of generation steps through continuous token 42 MethodMemory Optimization Training Cost Interpretability Generalization System-1.5 [76]SignificantHighWeakWeak SoftThinking [77]LimitedNoneWeakGood LightThinkerSignificantLowGoodGood Table 11 Comparison of different methods. representations. In these methods, the reasoning process remains fully dependent on the entire context. In contrast, LightThinker improves efficiency by reducing the number of historical tokens required for generation, without relying on full-context dependence. •Training Approach. Existing implicit CoT methods often require multi-stage training procedures with substantial overhead, such as the two-phase training used in System-1.5 [76] or the curriculum learning strategy adopted by Coconut [18]. Some methods [17,77] further introduce architectural modifications. By comparison, LightThinker is trained using standard SFT with modified attention masks, and does not require specialized training data or changes to the model architecture. •Interpretability and Generalization. Because they reason in continuous space, current implicit CoT methods are often less interpretable and may generalize poorly out of domain, with SoftThinking [77] being a notable exception. LightThinker, on the other hand, retains discrete tokens, which makes the reasoning process more interpretable and yields promising out-of-domain generalization in our experiments, as shown in Table 11. F Agent Case Study In this section, we present a representative agent trajectory to illustrate how memory orchestration (commit, fold, and expand) supports long-horizon reasoning by distilling evidence and dynamically controlling context (see Fig. 29). 43 Who was the shortest-serving US president, and from which college did his vice president graduate? William ... ... was the 9th president ... t=1~2t=1~3t=1~4t=1~6t=1~7 From the wiki, William Henry Harrison is the shortest president. From the wiki, William Henry Harrison is the shortest president. From the wiki, John Tyler was graduated from the College of William & Mary. (..., ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . I will first search the shortest-serving US president. Then I find his vice President ... William Henry Harrison Search “Shortest serving US President” ... was the 9th president of the US., serving from March 4 to April 4, 1841, the shortest presidency in U.S. history. ... assumed to be for Vice President John Tyler ... (..., ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . I will first search the shortest-serving US president. Then I find his vice President ... John Tyler Search “US Vice President John Tyler” ... At age 12, he ... entered the preparatory branch of the College of William and Mary. Tyler graduated from the ... I’l search the John Tyler Graduated from College of William & Mary. Then I’l commit. The observation#1 has been folded. (..., ...) Graduated from College of William & Mary. Then I’l commit. (..., ...) ( , ...) I’l fold this lengthy content . The expanded content mentioned the vice president of Harrison being John Tyler. The observation#1 has been expanded. I’l expand the folded wiki content . ( , ...) The observation#1 has been commited. Wait, the provided content mentioned the vice president. (..., ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . I will first search the shortest-serving US president. Then I find his vice President ... I’l expand the folded wiki content . ( , ...) The observation#1 has been commited. Wait, the provided content mentioned the vice president. ( , ...) I’l fold this lengthy content . The expanded content mentioned the vice president of Harrison being John Tyler. The observation#1 has been expanded. (..., ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . I will first search the shortest-serving US president. Then I find his vice President ... I’l expand the folded wiki content . ( , ...) The observation#1 has been commited. Wait, the provided content mentioned the vice president. From the wiki, William Henry Harrison is the shortest president. I’l search the John Tyler The observation#1 has been folded. ( , ...) I’l fold this lengthy content . The expanded content mentioned the vice president of Harrison being John Tyler. The observation#1 has been expanded. (..., ...) William Henry Harrison is the shortest. I’l commit the lengthy wiki content . I will first search the shortest-serving US president. Then I find his vice President ... I’l expand the folded wiki content . ( , ...) The observation#1 has been commited. Wait, the provided content mentioned the vice president. Action Commit Expand Fold Answer Google Search Observation Observation from memory action Observation from google search Think The shortest-serving U.S. president was William Henry Harrison. His vice president was John Tyler, who graduated from the College of William & Mary. The observation#2 has been commited. Figure 29 Case Study: Memory Orchestration for Long-Horizon Reasoning. 44