Paper deep dive
Mergeable Model-Side Aggregation States for Long-Context Language Models
Dachuan Song, Junyu Yin, Zechen Hu, Xuan Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/4/2026, 10:36:22 AM
Summary
The paper introduces a model-side aggregation interface for long-context language models that maintains compact HyperLogLog (HLL) sketch states alongside a frozen language model. This approach addresses the limitation of LLMs in performing non-additive, set-based aggregation (e.g., cardinality estimation, set relationships) by mapping records to canonical identities, hashing them, and updating fixed-size HLL states. These states are mergeable across context segments and provide accurate estimates (e.g., 1.6% mean relative error for 1M records) without increasing memory usage with context length, significantly outperforming direct full-context reasoning and chain-of-thought methods.
Entities (8)
Relation Signals (7)
Model-Side Aggregation Interface → uses → HyperLogLog
confidence 95% · we introduce a model-side aggregation interface that maintains compact Hash-based HyperLogLog (HLL) sketch states
Model-Side Aggregation Interface → outperforms → Chain-of-Thought
confidence 92% · The corresponding gains over chain-of-thought (CoT) reasoning were 60.9 and 63.2 points, respectively.
Model-Side Aggregation Interface → improvesaccuracyon → Gemma 4
confidence 90% · the fixed-budget interface reached 99.2% accuracy on Gemma 4 (31B, BF16)
Model-Side Aggregation Interface → supports → distinct counting
confidence 90% · The interface supports distinct counting, set combination, overlap estimation, and grouped aggregation
HyperLogLog → supports → distinct counting
confidence 90% · distinct count estimates the number of identities from a single HyperLogLog (HLL) sketch state.
HyperLogLog → supports → Jaccard similarity
confidence 85% · paired HLL states can estimate Jaccard similarity and containment
Count-Min Sketch → supports → frequency estimation
confidence 85% · Count-Min Sketch can provide a complementary state for frequency estimation
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped statistics, which widely exist in logs, program outputs, tables, and multi-turn conversations. To provide the aggregation state required by these tasks, we introduce a model-side aggregation interface that maintains compact Hash-based HyperLogLog (HLL) sketch states alongside a frozen language model. While the model processes the context, an extractor maps each relevant record to a canonical identity. The identity is then hashed and updates the HLL state. These states can be merged across context segments and/or read out directly for downstream reasoning, avoiding an additional generate-execute-return cycle. We validate the proposed approach by setting the HLL state size as 2 KiB (2,048 registers), which does not increase with context length or set cardinality. In a distinct-count experiment involving one million records, the mean relative error was 1.6%. In a separate merge test, states built from as many as 256 segments produced exactly the same readout as a single pass over the same stream. On 3,969 aggregate-then-reason tasks from 174 source windows, the fixed-budget interface reached 99.2% accuracy on Gemma 4 (31B, BF16), compared with 100.0% under exact aggregation; the paired gap was 0.8 percentage points (95% window-cluster CI: 0.5-1.3 points). On a matched set of 174 items, our method improved over direct full-context reasoning by 63.2 points on Qwen and 56.3 points on Gemma. The corresponding gains over chain-of-thought (CoT) reasoning were 60.9 and 63.2 points, respectively. On a fixed 1,200-task Oolong-Synth subset, our method reached 91.1% on Qwen and 99.3% on Gemma. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2607.26448v1
- Canonical: https://arxiv.org/abs/2607.26448v1
Trouble viewing inline? Open PDF directly →
Full Text
73,216 characters extracted from source content.
Expand or collapse full text
Mergeable Model-Side Aggregation States for Long-Context Language Models Dachuan Song1, Junyu Yin2, Zechen Hu1, Xuan Wang1 Abstract A known limitation of long-context language models is their increasingly unreliable performance in non-additive, set-based aggregation as context length grows. Examples include cardinality estimation, set relationships, and grouped statistics, which widely exist in logs, program outputs, tables, and multi-turn conversations. To provide the aggregation state required by these tasks, we introduce a model-side aggregation interface that maintains compact Hash-based HyperLogLog (HLL) sketch states alongside a frozen language model. While the model processes the context, an extractor maps each relevant record to a canonical identity. The identity is then hashed and updates the HLL state. These states can be merged across context segments and/or read out directly for downstream reasoning, avoiding an additional generate–execute–return cycle. We validate the proposed approach by setting the HLL state size as 2 KiB (2,048 registers), which does not increase with context length or set cardinality. In a distinct-count experiment involving one million records, the mean relative error was 1.6%. In a separate merge test, states built from as many as 256 segments produced exactly the same readout as a single pass over the same stream. On 3,969 aggregate-then-reason tasks from 174 source windows, the fixed-budget interface reached 99.2% accuracy on Gemma 4 (31B, BF16), compared with 100.0% under exact aggregation; the paired gap was 0.8 percentage points (95% window-cluster CI: 0.5–1.3 points). On a matched set of 174 items, our method improved over direct full-context reasoning by 63.2 points on Qwen and 56.3 points on Gemma. The corresponding gains over chain-of-thought (CoT) reasoning were 60.9 and 63.2 points, respectively. On a fixed 1,200-task Oolong-Synth subset, our method reached 91.1% on Qwen and 99.3% on Gemma. Code is available at attr/Border [0 0 0] user/Subtype /Link /A << /S /URI /URI (https://github.com/songdc98/sketchops) >>https://github.com/songdc98/sketchops. 1 Introduction Long-context language models are increasingly used to analyze logs, program outputs, tables, and multi-turn conversations (Bertsch et al. 2025; Kate et al. 2026; Xie et al. 2026; Cao et al. 2026). Although these models can effectively identify relevant records, they tend to become increasingly unreliable at non-additive, set-based aggregation as context length grows (Hsieh et al. 2024; Bertsch et al. 2025). Examples of such aggregation include cardinality estimation, set relationships, and grouped statistics, all of which may provide essential evidence for downstream reasoning and decision-making. This limitation arises because the attention and pooling operations used in language models rely extensively on averaging and normalization. While these mechanisms are well suited for summarizing semantic content, they are not necessarily effective for numerical state aggregation, which requires explicitly tracking item uniqueness, duplication, or set union and overlaps. Although learned readouts may approximate these behaviors within short sequence lengths, they may fail when the context length and/or set cardinality extend beyond the training distribution. Existing approaches to address this problem involve two representative lines of work. The first one relies on external execution, such as generating code or invoking tools to perform exact aggregation (Chen et al. 2022; Kate et al. 2026; Cao et al. 2026; Xie et al. 2026). Although these methods can achieve exact aggregates, they require a separate generate–execute–return cycle, where pausing and restarting the model’s normal reasoning flow comes with additional resource consumption and time delay. The second one learns neural representations of sets, such as Deep Sets, Set Transformer, and Universal Mini-Batch Consistency (UMBC) (Zaheer et al. 2017; Lee et al. 2019; Willette et al. 2023; Wagstaff et al. 2019), that can be used to enhance long context aggregation. However, these general-purpose representations do not necessarily guarantee operator-specific behaviors, such as reliably recognizing repeated identities, nor do they provide predictable accuracy under a fixed resource budget. As will be demonstrated in our experiments, their aggregation error can increase as context length grows. Contributions: Motivated by these gaps, this paper aims to enhance the performance of long-context language models on non-additive, set-based aggregation tasks. Accordingly, we introduce a model-side aggregation interface for long-context reasoning that maintains compact aggregation states alongside a frozen language model through HyperLogLog (HLL). The interface supports distinct counting, set combination, overlap estimation, and grouped aggregation with controlled error and bounded resource usage. Different from external execution methods, it does not require an additional generate–execute–return cycle. Different from general-purpose learned set representations, it provides explicit and predictable update, merge, and readout operations. More importantly, our method requires only lightweight state updates. For each input stream or active group, it maintains one fixed-budget sketch state, whose memory does not grow with context length or set cardinality. Conceptually, this mechanism provides a form of approximate number sense to language models: it produces fast and sufficiently reliable estimates of quantities and set relationships and makes them directly available for downstream reasoning. Our technical contributions include: 1. We integrate lightweight, mergeable sketch states that are updated in parallel with the forward computation of a frozen language model, enabling efficient model-side aggregation without invoking an external executor. 2. We develop a unified mergeable interface for set-based aggregation. It maintains one fixed-budget mergeable HLL state per input stream, from which distinct count, union, Jaccard similarity, and containment are read out. 3. We validate the proposed framework through both task-specific aggregation experiments and end-to-end reasoning evaluations, covering downstream reasoning performance, length extrapolation, mergeability, resource–error tradeoffs, and robustness to item-selection noise. It is worth noting that the HLL sketch algorithm used in the paper is not new. Our contribution lies in organizing aggregation states into a model-side interface for long-context reasoning, allowing them to be updated throughout context processing and their readouts to be made directly available for subsequent reasoning, and the validation of this design. 2 Related Work 2.1 Code Execution and Tool-Augmented Aggregation A common way to obtain exact aggregates is to introduce an external executor that computes these quantities for the language model. Program-Aided Language Models (PAL), Program of Thoughts (PoT), database-query systems, and coding agents could deploy aggregation computation to external Python runtimes or Structured Query Language (SQL) engines (Gao et al. 2023; Chen et al. 2022; Cao et al. 2026). LogCopilot applies the same idea to log analysis by translating natural-language requests into executable log queries (Xie et al. 2026). If the code is correctly generated and receives the relevant records, the executor can return an exact result to the language model as tool output (Kate et al. 2026). However, the result becomes available only after query generation, execution, and tool return. Although some executors can retain state across queries, an exact set still requires memory that grows with the number of distinct identities. In contrast, our approach relies on a fixed-budget model-side state that is incorporated into the frozen model during context processing. Thus, the readouts are always immediately available for subsequent reasoning without invoking an external executor. 2.2 Long-Context Aggregation and Learning-Based Set Representations Set representation and processing methods offer an alternative route for long-context set aggregation. Deep Sets (Zaheer et al. 2017) maps items into permutation-invariant set functions, which can summarize variable-size sets without requiring the model to retain every item or depend on their input order. Set Transformer (Lee et al. 2019) uses attention to model interactions among set elements, allowing the aggregate to capture relationships that simple pooling may miss. It also reduces the cost of processing large sets. Universal Mini-Batch Consistency (UMBC) (Willette et al. 2023) enables a large set to be processed in smaller partitions while guaranteeing that the results are invariant to different ways of set partitioning. This allows robust processing of extra-long contexts in a sequential manner. While these methods make set aggregation more scalable, the specific aggregation behaviors are still learned from data. They do not explicitly guarantee deterministic merging of independently computed states, nor offer a predictable error under a fixed resource budget. Our method, on the other hand, provides these properties through the design of an explicit aggregation state and update rules. 2.3 Learned Data Structures and Streaming Sketches Beyond general set encoders, existing works also learn compact state representations for approximate set queries. Learned Bloom filters (Rae et al. 2019; Song and Wang 2026; Vaidya et al. 2020) and neural sketches (Cao et al. 2023) adapt their internal representations to the corresponding data distribution. Among them, MaxSketch is the closest method to our setting because it also maintains a compact, mergeable state for approximate cardinality estimation. However, MaxSketch is based on random Gaussian projections, which makes it suitable for noisy, high-dimensional observations (Tsikouras et al. 2026). Our setting instead assumes that discrete item identities are directly available and focuses on integrating explicit streaming aggregation states with language-model reasoning. From a technical perspective, our model-side interface leverages existing sketch operator HyperLogLog (HLL) to facilitate the update, merge, and value readout. Specifically, HLL estimates distinct cardinality with a fixed register array and merges two states by taking the maximum in each register (Flajolet et al. 2007). In addition, paired HLL states can estimate Jaccard similarity and containment through joint maximum-likelihood estimation (Ertl 2017). This allows a single compact state representation to support distinct counting, set combination, and overlap-related queries. There are alternative sketch operators that can provide similar capabilities. For example, k-Minimum Values (KMV) retains the k smallest hash values and estimates both cardinality and overlap, whereas MinHash directly estimates Jaccard similarity (Bar-Yossef et al. 2002; Broder 1997). However, compared with HLL, their readouts are more specialized. For queries involving multiplicity, Count-Min Sketch can provide a complementary state for frequency estimation (Cormode and Muthukrishnan 2005). We note that these sketch operators are established in the literature, our contribution lies in the interface that selects and maintains task-appropriate aggregation states alongside a frozen language model, and returns the resulting statistics to the model for downstream reasoning. 3 Problem Setup Consider a log-analysis request such as “How many distinct users reported an error in each service?” Identifying all error records is not sufficient to answer this question, because the same user may report multiple errors in the same service and should still be counted only once. Similarly, if the request instead asks how much the affected-user populations of two services overlap, knowing the number of users in each service is also insufficient: the computation must retain which users occur in both. Thus, after relevant records are identified, their identities may still need to be aggregated. Identity-Aware Aggregation. Many non-additive aggregation tasks depend on how item identities occur across relevant records, including their uniqueness, repetition, grouping, and relationships across collections. We refer to this broad and fundamental family of tasks as identity-aware aggregation and focus the scope of this paper on this setting. Specifically, we consider support-based tasks, which depend on which distinct identities occur and include distinct counting, union and intersection cardinalities, overlap, Jaccard similarity, and containment, as well as grouped requests that apply these operations separately within request-defined groups. These tasks capture a wide range of practical aggregation needs and are closely related to standard set operations in relational processing and established measures of set comparison (Codd 1970; Broder 1997; Ertl 2021). Problem Formulation. To formalize our problem, let x1:L=(x1,…,xL)x_1:L=(x_1,…,x_L) denote a context of length L, and let q denote a request over this context. We assume that an extractor associated with a frozen language model identifies the relevant records and organizes them into τ operand-specific streams (referred to as streams hereafter for simplicity): E(x1:L,q) E(x_1:L,q) =(ℛ(1),…,ℛ(τ)), =(R^(1),…,R^(τ)), whereℛ(i) ~R^(i) =((zt(i),gt(i)))t=1ni. = ((z_t^(i),g_t^(i)) )_t=1^n_i. (1) Here, τ is the number of streams required by q and nin_i is the number of records in stream ℛ(i)R^(i). For each record, zt(i)∈z_t^(i) denotes its canonical identity in the item universe U, and gt(i)∈∪∅g_t^(i) ∪\ \ denotes an optional (i.e., ∅ if not used) group key. For each stream ℛ(i)R^(i), we maintain a fixed-budget sketch state si∈s_i , where S denotes the state space. Let yq=fq(ℛ(1),…,ℛ(τ))y_q=f_q\! (R^(1),…,R^(τ) ) denote the exact answer computed from the complete record streams. A query-specific readout ρq _q uses only the sketch states to compute an estimate: y^q=ρq(s1,…,sτ)≈yq. y_q= _q(s_1,…,s_τ)≈ y_q. (2) The problem is therefore to construct and update the sketch states sis_i and to design the readout function ρq _q that satisfies the following requirements: • Model-side operation: Each sketch state sis_i is updated as the model extracts records into the corresponding stream ℛ(i)R^(i), with low computational overhead. • Fixed budget: The sizes of the sketch states do not grow with the stream length or the number of distinct identities. • Aggregation consistency: Sketch states constructed from different segments of the same stream must be mergeable into a state for the complete stream. Under these constraints, the objective is to allow y^q y_q to accurately approximate the exact answer yqy_q. Remark 1. Normalized attention combines record representations through weighted averaging, which generally does not match the operations required by identity-aware aggregation. If an identity zt(i)z_t^(i) appears twice, the second record enters the average and usually changes the readout, whereas a support-based aggregation should remain unchanged. The mismatch also persists if a stream is processed in segments, where the same items in different segments are usually counted multiple times. Related issues have been identified in attention-based graph aggregation (Zhang and Xie 2020). Appendix A gives more detailed discussion and counterexamples. 4 Method Figure 1: Model-side aggregation workflow. As the frozen language model processes the context, the extractor E assigns canonical identities to relevant records and places them in the streams required by request q. For grouped requests, a group key routes each record to the corresponding group state. Each stream maintains a fixed-budget state, and states built from separate segments can be merged without reading the records again. The readout ρq _q then obtains the requested aggregate from one or more states and returns y^q y_q to the model for downstream reasoning. HLL is used for the set-based readouts, while Count–Min supports point-frequency queries. Our method augments a frozen language model with explicit aggregation states implemented using sketch operators. Each state is assigned a user-defined budget that controls the tradeoff between estimation accuracy and storage overhead. As the language model processes the context, the extractor E identifies the relevant records, assigns each record a canonical identity zt(i)z_t^(i), and routes it to the corresponding stream ℛ(i)R^(i). The associated sketch states run on model-side, i.e., they are updated online in parallel with the model’s forward computation and may be merged across compatible segments of the same stream. Once all relevant records have been processed, a query-specific readout converts one or more states into an estimate. For example, distinct count estimates the number of identities from a single HyperLogLog (HLL) sketch state. Union size first merges two HLL states register by register and then reads the cardinality of the merged state. Jaccard similarity and containment keep the two sketch states separate; Joint maximum-likelihood estimation (JMLE) jointly estimates the distinct numbers of identities appearing only in either stream and in both streams, from which the two ratios are computed. Group-by applies the same readouts to the HLL state of the requested group. These queries reuse the maintained HLL states rather than constructing a separate sketch for each readout. Finally, the readout ρq _q produces y^q y_q, which is appended to the model input as explicit evidence, and decoding continues from the retained KV cache. Figure 1 illustrates this data flow. 4.1 State construction from streams We first construct a fixed-size state that records the distinct identities in each stream. For this purpose, we use HyperLogLog (HLL) (Flajolet et al. 2007). Stream i maintains the state si=(Mi,1,…,Mi,m)s_i=(M_i,1,…,M_i,m), where m is the number of registers (size) in the state and all registers are initialized to zero. A fixed hash function maps each identity z to a register index j(z)j(z) and a rank r(z)r(z): Hash(z)=(j(z),r(z))Hash(z)=(j(z),r(z)). Only the register selected by j(z)j(z) is updated: Mi,j(z)←maxMi,j(z),r(z).M_i,j(z)← \! \M_i,j(z),r(z) \. (3) Because the hash function is fixed, every occurrence of the same identity produces the same pair (j(z),r(z))(j(z),r(z)). Once the selected register has reached r(z)r(z), other occurrences of z have no further effect. Processing all identities in ℛ(i)R^(i) in this way produces the final state sis_i. For grouped requests, we first use the group key gt(i)g_t^(i) to select the state associated with that group, then, zt(i)z_t^(i) updates one of its registers according to (3). A repeated identity therefore has no additional effect within the same group, but an occurrence in another group updates that group’s state independently. A stream can also be processed in PiP_i segments. Let si[a]=(Mi,1[a],…,Mi,m[a])s_i^[a]=(M_i,1^[a],…,M_i,m^[a]) be the state produced from segment a. When all segment states use the same register count, hash function, and seed, the state of the full stream is obtained by taking the maximum in each register: si=⨁a=1Pisi[a],Mi,j=max1≤a≤PiMi,j[a].s_i= _a=1^P_is_i^[a], M_i,j= _1≤ a≤ P_iM_i,j^[a]. (4) The merge produces the same HLL state compared with processing the complete stream as a whole. States associated with different streams or groups remain separate because later readouts may need to distinguish where each identity occurred. The HLL construction above supports uniqueness-based queries, which are the primary focus of this paper. However, we note that hash-based methods can also handle frequency (multiplicity) queries that accumulate repeated occurrences. For this purpose, we use Count–Min Sketch (Cormode and Muthukrishnan 2005), which maintains a fixed-size hash table and estimates an identity’s frequency from its associated counters. Compatible Count–Min states can be merged by elementwise addition. The detailed update and readout rules are provided in Appendix D. 4.2 State reuse and query readout Once the HLL state of a stream has been constructed, it can be reused for multiple set queries without rebuilding the state. Let F^0(s) F_0(s) denote the specific standard HLL cardinality estimator corresponding to the ρq _q in (2). A distinct-count query reads the cardinality directly from one state, whereas a union query first forms a temporary registerwise merge and then reads its cardinality: y^q=ρq(si) y_q= _q(s_i) =F^0(si), = F_0(s_i), q asks for distinct count, q asks for distinct count, (5) y^q=ρq(s1,s2) y_q= _q(s_1,s_2) =F^0(s1⊕s2), = F_0(s_1 s_2), q asks for union. q asks for union. The union merge is temporary and does not overwrite s1s_1 or s2s_2. Their separate states remain available for other readouts. For Jaccard similarity and containment, we use the HLL joint maximum-likelihood estimator (JMLE) (Ertl 2017). Let s1s_1 and s2s_2 represent sets A and B, respectively, and define n10=|A∖B|n_10=|A B|, n01=|B∖A|n_01=|B A|, and n11=|A∩B|n_11=|A∩ B|. JMLE estimates these three disjoint regions. The corresponding readouts are y^q y_q =ρq(s1,s2) = _q(s_1,s_2) (6) =n^11n^10+n^01+n^11,Jaccard similarity,n^11n^10+n^11,containment of A in B. = Union continues to use the registerwise merge in Equation (5). If the JMLE optimizer does not return finite, feasible estimates that satisfy its convergence checks, the relation readout is marked invalid rather than replaced with an inclusion–exclusion estimate based on |A∩B|=|A|+|B|−|A∪B||A∩ B|=|A|+|B|-|A∪ B|. Appendix C.4 provides details of the likelihood and optimization procedure. Because the sketch states are preserved after construction, distinct count, union size, Jaccard similarity, and containment can all be obtained from the same HLL registers by changing only the readout. Group-by applies the same reuse within each selected group. An existing state may also be reused for a later request, provided that its definition, identity mapping, register count, hash function, and seed remain unchanged. Complexity. Because our approach performs aggregation on the model side, we briefly summarize the storage and computational complexity of the HLL states. For storage, our implementation represents each HLL register using one byte. Thus, an HLL state with m=2048m=2048, (i.e., the number of registers in each state) occupies 22 KiB. For a fixed number of streams and active groups, processing additional records only updates the existing states and does not increase their size. Each new stream or active group requires one additional HLL state, while group-by queries also require an index over the active group keys. For computation, each HLL update modifies a single register and therefore takes O(1)O(1) time. Registerwise state merging and cardinality estimation each require O(m)O(m) (m is fixed by the state budget). A JMLE readout with I optimization steps takes O(mI)O(mI) time. 5 Experiments Figure 2: Main results. Colour is the model (blue Qwen 3.6 (35B), vermillion Gemma 4 (31B)); solid fill is SketchOps, pale fill the comparison method. (a) 174 matched tasks against the strongest full-context baseline. (b) All 3,969 tasks: the black tick is exact aggregation and the whisker the 95% window-cluster CI of the paired gap. (c) Mean output tokens per task against external code execution. (d, e) State level, no language model: relative error stays flat in stream length under a fixed 2 KiB budget and follows 1.04/m1.04/ m. This section evaluates whether fixed-budget aggregation states can preserve statistics from long record streams and subsequently support downstream reasoning. We validate our method on scenario-based aggregate-then-reason tasks, which require the model to obtain the relevant aggregate from the records and then use it as intermediate evidence to make a final decision. We compare our method with exact aggregation, full-context reasoning, and code execution. Additionally, to examine the effectiveness of our method at the component level, we isolate the behavior of the aggregation states themselves through experiments that measure estimation error, merge consistency, the resource–error trade-off, and computational overhead. Figure 2 summarizes the main results from these evaluations. 5.1 Evaluation setup The reasoning experiments use the publicly available Oolong-Synth test set (Bertsch et al. 2025). This dataset organizes news articles, user comments, question answering, natural-language inference, and various linguistic data into long record sequences, with user identities and category labels provided for each record. We selected 174 record sequences from this dataset, referring to each sequence as a source window. These windows have lengths ranging from 1K to 4M tokens. Based on the labels provided by the dataset, we converted each source window into a structured identity stream, then constructed multiple aggregate-then-reason tasks from the same records. The required aggregates include distinct counts, grouped distinct counts, set relationships, and a small number of point-frequency queries. Each task also specifies a decision condition that maps the aggregate to one of two candidate answers. Under this setup, the aggregate serves only as intermediate evidence, and the model must still make a final selection based on it. For example, the model selects between “Handle Immediately” and “Continue Monitoring”, depending on whether more than 100 users report an error. Because multiple tasks were constructed from each source window, the 174 windows yielded 3,969 tasks in total. These tasks were used to compare fixed-budget aggregation states with exact aggregation and to conduct a counterfactual evidence-use test. We evaluated Qwen 3.6 (35B) and Gemma 4 (31B) using the same tasks, aggregation states, prompts, and scoring rules. Accuracy for all reasoning tasks was calculated based on whether the model’s final selection matched the reference answer. Model-call failures, context overflows, and unparseable answers were counted as errors. Because tasks generated from the same source window share the underlying records, they cannot be considered independent of one another. Thus, we treated a source window and its derived tasks as a single resampling unit when computing confidence intervals. We repeated this resampling procedure 10,000 times and calculated 95% confidence intervals from the resulting distribution. When presenting results, we refer to our method as SketchOps. 5.2 Explicit aggregation states improve downstream reasoning The full-context reasoning comparison used 174 matched tasks. Since a single source window can generate multiple related questions, we selected one question from each window according to a predetermined, answer-independent rule, ensuring that each underlying record sequence appeared only once in the comparison. The full-context baselines directly read the original records and used direct answering (Kojima et al. 2022), chain-of-thought (CoT) reasoning (Wei et al. 2022), or guided-choice prompting (Geng et al. 2023). Our method first processes the same records with fixed-budget aggregation states, then passes the resulting aggregate to the model. All methods receive the same questions and candidate answers. Tasks exceeding the 150K-token context limit remain in the evaluation and are scored as incorrect. Table 1 reports these results. On Qwen, our method achieves 92.0%, which is 60.9 percentage points higher than the strongest full-context baseline, CoT. On Gemma, our method achieves 99.4%, exceeding the best full-context result by 56.3 points. Part of this difference comes from context length. Of the 174 tasks, 80 exceed the 150K-token limit and are therefore scored as incorrect for the full-context baselines, while our method still achieves 86.2% on Qwen and 98.8% on Gemma for these tasks. More importantly, the advantage remains among the 94 tasks that fit within the context limit. Under this subset, our method reaches 96.8% and 100%, whereas the strongest full-context baseline reaches 57.4% and 79.8%, respectively. These results suggest two benefits of the proposed aggregation state. First, it preserves the required aggregate when the records exceed the model’s context limit. Second, it gives the model explicit results for deduplication, grouping, and set relations when the records do fit. Table 1: Accuracy on 174 matched aggregate-then-reason tasks. One task is selected from each source window, and all methods use the same question and answer choices. Method Qwen (%) Gemma (%) SketchOps (ours) 92.0 99.4 Direct full-context reasoning 28.7 43.1 Full context ++ CoT 31.0 36.2 Full context ++ guided choice 27.6 43.1 We next evaluate our method on all 3,969 tasks and measure the accuracy degradation introduced by the fixed-budget state. The exact-aggregation condition in Table 2 follows the same pipeline as our method but replaces the fixed-budget readout with an exact aggregate computed from the complete streams. On Qwen, our method achieves 89.0%, while exact aggregation achieves 93.0%, yielding a gap of 4.0 percentage points (95% window-cluster CI: 3.2–4.9 points). On Gemma 4 (31B, BF16), the corresponding accuracies are 99.2% and 100.0%, with a gap of 0.8 percentage points (95% window-cluster CI: 0.5–1.3 percentage points). This comparison measures the downstream accuracy lost through fixed-budget approximation. The preceding matched experiment separately measures the value of supplying an explicit aggregation state rather than asking the model to recover the same information from the full records. Overall, the fixed-budget state preserves most of the downstream performance of exact aggregation while retaining bounded memory usage and model-side operation. Finally, we test whether the model’s decision output responds to the supplied aggregate. For each task, we replace the aggregate with a counterfactual aggregate that changes the correct answer while leaving the decision condition unchanged. The model selects the outcome implied by the modified aggregate on 92.8% of the Qwen tasks and 100.0% of the Gemma tasks. In most cases, the downstream decision changes, which confirms that the model generally uses the supplied aggregate as evidence when making its final decision. Table 2: Results on 3,969 aggregate-then-reason tasks. The exact-aggregate row replaces the fixed-budget readout with the exact value. The counterfactual row replaces the aggregate with a value that changes the correct answer and is scored against the outcome implied by the modified evidence. Condition Qwen (%) Gemma (%) Approximation comparison SketchOps (ours) 89.0 99.2 with exact aggregate 93.0 100.0 Evidence-use test with counterfactual 92.8 100.0 5.3 Comparison with external code execution We also compared SketchOps (ours) with exact code execution (Gao et al. 2023; Chen et al. 2022) using 1,200 tasks preselected from the set of 3,969 tasks, which covered all 174 source windows and different aggregation types. Qwen and Gemma were evaluated using the same tasks and experimental protocol. In the code-execution path, the model first generates a complete Python program, which is then run by a controlled executor that returns the result. With our method, the same model directly receives the fixed-budget state readout and generates only the final selection. Table 3 reports accuracy and generated output tokens for the two approaches. An independent CPU verifier re-derives every code-execution answer from the original records. Table 3: Comparison of SketchOps (ours) and external code execution on the same 1,200 tasks for each model. Output tokens are reported as the mean per task. Model Method Acc. (%) Tokens / task Qwen SketchOps 91.1 2.2 Qwen Code execution 100.0 1,260.2 Gemma SketchOps 99.3 2.2 Gemma Code execution 100.0 208.6 On Qwen, code execution exceeded SketchOps (ours) by 8.9 percentage points (95% source-window-cluster bootstrap CI: 7.1–10.7 percentage points); on Gemma, the difference was less than one percentage point. This accuracy advantage came with a much longer code generation path. Code execution produced an average of 1,260.2 output tokens on Qwen and 208.6 on Gemma, compared with 2.2 for SketchOps on both models. SketchOps generated only the final decision and did not generate or execute a program. While external code execution is reliable when exactness of the result matters most, our SketchOps is intended for a different setting, where an aggregation state must be updated and merged within a fixed memory budget and its readout must be available without generating a program. 5.4 Length extrapolation and resource–error trade-off The preceding downstream results rely on the aggregation state providing reliable statistics. To test this in an isolated environment, we fix the identity streams and exact targets and compare only the state-readout errors. In the length-extrapolation experiment, both methods use a 22 KiB carried state for each record stream. UMBC (Willette et al. 2023) is trained on record streams of length 88K. HLL requires no training. During testing, UMBC and HLL process the same identity streams, with the longest stream containing 200200K records. The number of distinct identities in each stream is sampled independently of stream length from 1,0001,000 to 4,0004,000. Therefore, neither method can infer the exact distinct count from the number of records alone. Table 4: Length extrapolation under a fixed 22 KiB carried-state budget. Each length contains 100 test streams; entries are median relative errors for distinct count. UMBC’s static model parameters are excluded from the per-stream state budget. Stream length L HLL Official UMBC UMBC/HLL 88K 1.29% 27.60% 21.4×21.4× 5050K 1.49% 26.78% 18.0×18.0× 200200K 1.49% 26.95% 18.1×18.1× Table 4 shows that, as the number of records increased from 88K to 200200K, HLL’s median relative error rose only from 1.29% to 1.49%, while its state remained fixed at 22 KiB. Under the same carried-state budget, UMBC’s error remained close to 27%. This comparison isolates the aggregation state used for distinct count. It shows that an update rule matched to set semantics can track distinct cardinality without learning this behavior from training data. A fixed state size also comes at a cost in accuracy. We therefore varied the number of HLL registers m and measured the root-mean-square (RMS) relative error over a grid in which the number of distinct identities ranged from 100100 to 1,000,0001,000,000. The current implementation allocates one byte per register, so the state size in bytes corresponds directly to m. We also tested whether segmenting a stream changes the HLL result. After dividing each identity stream into as many as 256 segments and merging the resulting states in different orders, we found no register mismatches or readout differences from a single pass over the full stream. Table 5: Resource–error trade-off for HLL. The measured column reports RMS relative error; the final column gives the reference value 1.04/m1.04/ m. Registers m State size Measured error Reference 256256 0.250.25 KiB 6.47% 6.50% 512512 0.500.50 KiB 4.60% 4.60% 1,0241,024 11 KiB 3.33% 3.25% 2,0482,048 22 KiB 2.34% 2.30% 4,0964,096 44 KiB 1.72% 1.63% 8,1928,192 88 KiB 1.10% 1.15% Table 5 shows that the measured error generally follows the reference value 1.04/m1.04/ m, the standard asymptotic relative error for HLL. This error is caused by the approximation using a fixed register budget that does not expand with additional stream length. The storage crossover between a 2-KiB HLL sketch state and an exact identity set is reported in Appendix F.3. 6 Conclusion We introduce a model-side aggregation interface for long-context reasoning that maintains compact aggregation states alongside a frozen language model through HyperLogLog (HLL). The interface supports distinct counting, set combination, overlap estimation, and grouped aggregation with controlled error and bounded resource usage. Different from external execution methods, it does not require an additional generate–execute–return cycle. For each stream or active group, it maintains one fixed-budget sketch state, whose memory does not grow with context length or set cardinality. Through experiment validation using Qwen and Gemma, we show that our method substantially outperforms direct full-context reasoning and chain-of-thought (CoT) reasoning. The counterfactual test further confirms that the models change their final choices in response to the supplied aggregate. State-level experiments show that state size remains fixed as the record stream grows, while states constructed from separate segments merge into exactly the same state produced by a single pass over the full stream. Limitations: Although each sketch operator uses a fixed-size state, it introduces a trade-off between estimation error for bounded state size. Dedicated storage or external execution might still be preferred for small-scale tasks that require exact numerical outputs. The fixed state size applies to a single stream or active group, so total aggregation memory still grows with the number of states maintained. Also, the current interface supports only the aggregation targets considered here: distinct count, union size, Jaccard similarity, containment, grouped distinct count, and point frequency. Additional targets require corresponding state implementations and readout functions. Our future work involves dynamically selecting between exact sets and sketches with different budgets based on set cardinality, error requirements, and available resources. We will also extend the state and readout interfaces to additional aggregation operators. References Z. Bar-Yossef, T. Jayram, R. Kumar, D. Sivakumar, and L. Trevisan (2002) Counting distinct elements in a data stream. In International Workshop on Randomization and Approximation Techniques in Computer Science, p. 1–10. Cited by: §2.3. A. Bertsch, A. Pratapa, T. Mitamura, G. Neubig, and M. R. Gormley (2025) Oolong: evaluating long context reasoning and aggregation capabilities. arXiv preprint arXiv:2511.02817. Cited by: §E.2, §1, §5.1. A. Z. Broder (1997) On the resemblance and containment of documents. In Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171), p. 21–29. Cited by: §2.3, §3. W. Cao, X. Yin, B. Dhingra, and S. Zhou (2026) Coding agents are effective long-context processors. arXiv preprint arXiv:2603.20432. Cited by: §1, §1, §2.1. Y. Cao, Y. Feng, and X. Xie (2023) Meta-sketch: a neural data structure for estimating item frequencies of data streams. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37, p. 6916–6924. Cited by: §2.3. W. Chen, X. Ma, X. Wang, and W. W. Cohen (2022) Program of thoughts prompting: disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588. Cited by: §1, §2.1, §5.3. E. F. Codd (1970) A relational model of data for large shared data banks. Communications of the ACM 13 (6), p. 377–387. Cited by: §3. G. Cormode and S. Muthukrishnan (2005) An improved data stream summary: the count-min sketch and its applications. Journal of Algorithms 55 (1), p. 58–75. Cited by: Appendix D, §2.3, §4.1. O. Ertl (2017) New cardinality estimation algorithms for hyperloglog sketches. arXiv preprint arXiv:1702.01284. Cited by: §C.4, §2.3, §4.2. O. Ertl (2021) SetSketch: filling the gap between minhash and hyperloglog. arXiv preprint arXiv:2101.00314. Cited by: §3. P. Flajolet, É. Fusy, O. Gandouet, and F. Meunier (2007) HyperLogLog: the analysis of a near-optimal cardinality estimation algorithm. In Conference on Analysis of Algorithms (AofA), p. 137–156. External Links: Document, Link Cited by: §C.3, §2.3, §4.1. L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig (2023) Pal: program-aided language models. In International conference on machine learning, p. 10764–10799. Cited by: §2.1, §5.3. S. Geng, M. Josifoski, M. Peyrard, and R. West (2023) Grammar-constrained decoding for structured nlp tasks without finetuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 10932–10952. Cited by: §5.2. C. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y. Zhang, and B. Ginsburg (2024) RULER: what’s the real context size of your long-context language models?. arXiv preprint arXiv:2404.06654. Cited by: §1. K. Kate, Y. Rizk, P. Ghosh, A. Gulati, T. Chakraborti, Z. Wright, and M. Agarwal (2026) How good are LLMs at processing tool outputs?. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 2918–2941. External Links: Document, Link Cited by: §1, §1, §2.1. T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa (2022) Large language models are zero-shot reasoners. Advances in neural information processing systems 35, p. 22199–22213. Cited by: §5.2. J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh (2019) Set transformer: a framework for attention-based permutation-invariant neural networks. In International conference on machine learning, p. 3744–3753. Cited by: §1, §2.2. J. Rae, S. Bartunov, and T. Lillicrap (2019) Meta-learning neural bloom filters. In International conference on machine learning, p. 5271–5280. Cited by: §2.3. D. Song and X. Wang (2026) Elastic spectral state space models for budgeted inference. arXiv preprint arXiv:2601.22488. Cited by: §2.3. N. Tsikouras, C. Caramanis, and C. Tzamos (2026) MaxSketch: robust distinct counting in streams via random projections. arXiv preprint arXiv:2605.15571. Cited by: §2.3. K. Vaidya, E. Knorr, T. Kraska, and M. Mitzenmacher (2020) Partitioned learned bloom filter. arXiv preprint arXiv:2006.03176. Cited by: §2.3. E. Wagstaff, F. Fuchs, M. Engelcke, I. Posner, and M. A. Osborne (2019) On the limitations of representing functions on sets. In International conference on machine learning, p. 6487–6494. Cited by: §1. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, p. 24824–24837. Cited by: §5.2. J. Willette, S. Lee, B. Andreis, K. Kawaguchi, J. Lee, and S. J. Hwang (2023) Scalable set encoding with universal mini-batch consistency and unbiased full set gradient approximation. In International Conference on Machine Learning, p. 37008–37041. Cited by: §F.2, §1, §2.2, §5.4. S. Xie, C. Zhang, T. Zhou, J. Liu, X. Hong, Q. Li, and X. Peng (2026) LogCopilot: automating log aggregation analysis through large language models. arXiv preprint arXiv:2606.17094. Cited by: §1, §1, §2.1. M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola (2017) Deep sets. Advances in neural information processing systems 30. Cited by: §1, §2.2. S. Zhang and L. Xie (2020) Improving attention mechanism in graph neural networks via cardinality preservation. In IJCAI: proceedings of the conference, Vol. 2020, p. 1395. Cited by: Appendix A, Remark 1. Appendix A Normalized Attention and the Aggregation-Operator Mismatch In the paper, we stated that “a normalized readout does not itself expose the update and merge rules needed for set-based aggregation.” This statement concerns the state returned by the readout, not the computational power of a Transformer, an external executor, or a model supplied with an additional aggregation state. To further justify this, consider the records in one stream ℛ(i)R^(i). Let vtv_t denote the model representation associated with record t, and let ata_t be its attention score at the readout position. A normalized attention readout can be written as o=ND,N=∑t=1nieatvt,D=∑t=1nieat.o= ND, N= _t=1^n_ie^a_tv_t, D= _t=1^n_ie^a_t. (7) The notations in this section are local to the analysis; they are not additional components of our method. Observation 2 (Effect of a repeated record). Suppose another record for an identity already present in ℛ(i)R^(i) enters the readout with value v and positive weight w=eaw=e^a. If o′o is the updated output, then o′−o=wD+w(v−o).o -o= wD+w(v-o). (8) The output is unchanged if and only if v=ov=o. Proof. The new output is o′=(N+wv)/(D+w)o =(N+wv)/(D+w). Subtracting N/DN/D and using N=DoN=Do gives Equation (8). Since w>0w>0, o′=o =o holds exactly when v=ov=o. ∎ Set-based aggregation requires a different rule. Once an identity has entered the set, seeing it again must leave the represented set unchanged. Normalized attention has no such guarantee: the second record remains another term in the average. This loss of cardinality information in attention-based aggregation has also been studied in graph neural networks (Zhang and Xie 2020). Observation 3 (Normalized outputs do not determine a merged output). No binary function can recover the normalized output of every concatenated pair of record segments from their two normalized outputs alone. Proof. Equal attention scores reduce Equation (7) to an arithmetic mean. Take the scalar segments ℛA=(0,2),ℛB=(1),=(0).R_A=(0,2), _B=(1), =(0). Both ℛAR_A and ℛBR_B have mean 11, while T has mean 0. Their concatenations do not agree: o(ℛA∥)=23,o(ℛB∥)=12.o(R_A\|T)= 23, o(R_B\|T)= 12. A merge function receives the same input pair (1,0)(1,0) in both cases and therefore cannot return both answers. ∎ Based on the above observations, keeping (N,D)(N,D) would make the average mergeable because the numerators and denominators can be added. It would not make the update duplicate-idempotent: an identity present in two segments would still contribute twice. The result would be a mergeable weighted average, not set union. We use aggregation-operator mismatch for this difference between the update-and-merge algebra of normalized averaging and that required by the target aggregation. Scope of the theoretical contribution. The two observations above and Proposition 4 are the complete set of new theoretical statements made in this paper. Their assumptions, restrictions, and proofs are stated in this appendix. The cardinality estimator, Count–Min bound, and JMLE likelihood used later are established tools and are cited rather than presented as new theorems. Appendix B Notation and Exact Targets B.1 Notation shared with the main paper Table 6 repeats the notation needed to read the appendix. Notations introduced only inside a proof or estimator are not included and are defined where they first appear. Table 6: Notation used in the main paper and appendix. Notation Meaning x1:Lx_1:L, q context of length L and the request over that context E extractor that organizes relevant records into streams τ number of streams required by request q ℛ(i)R^(i) stream i, containing nin_i records zt(i)z_t^(i) canonical identity of record t in stream i gt(i)g_t^(i) optional group key of that record U, G identity universe and group-key universe si∈s_i maintained state for stream i and its state space fqf_q, yqy_q exact aggregation function and exact answer ρq _q, y^q y_q request-specific readout and its estimate m number of one-byte registers in one HLL state PiP_i number of separately processed segments of stream i dcmd_cm, wcmw_cm Count–Min rows and counters per row B.2 Target aggregations and empty-set conventions For each stream, define the set of observed identities and the identity set within group g as Ai=zt(i):1≤t≤ni,Ai,g=zt(i):gt(i)=g, 1≤t≤ni.A_i=\z_t^(i):1≤ t≤ n_i\, A_i,g=\z_t^(i):g_t^(i)=g,\ 1≤ t≤ n_i\. (9) The number of occurrences of identity z in the same stream is μi(z)=|t:zt(i)=z, 1≤t≤ni|. _i(z)= |\t:z_t^(i)=z,\ 1≤ t≤ n_i\ |. (10) The exact targets used in this paper follow directly: fdistinct(ℛ(i)) f_distinct(R^(i)) =|Ai|, =|A_i|, funion(ℛ(1),ℛ(2)) f_union(R^(1),R^(2)) =|A1∪A2|, =|A_1∪ A_2|, fJaccard(ℛ(1),ℛ(2)) f_Jaccard(R^(1),R^(2)) =|A1∩A2||A1∪A2|, = |A_1∩ A_2||A_1∪ A_2|, fcontain(ℛ(1),ℛ(2)) f_contain(R^(1),R^(2)) =|A1∩A2||A1|, = |A_1∩ A_2||A_1|, fgroup(i,g)(ℛ(i)) f_group(i,g)(R^(i)) =|Ai,g|, =|A_i,g|, ffreq(i,z⋆)(ℛ(i)) f_freq(i,z )(R^(i)) =μi(z⋆). = _i(z ). (11) Distinct count and grouped distinct count are zero for an empty input or group. We set Jaccard similarity to zero when A1∪A2A_1∪ A_2 is empty and containment to zero when A1A_1 is empty. The exact targets and approximate readouts use the same conventions. Appendix C HLL State Construction and Readout C.1 Hashing and register updates One HLL state for stream i is si=(Mi,1,…,Mi,m),s_i=(M_i,1,…,M_i,m), with every register initialized to zero. The implementation hashes each canonical identity to 64 bits using a fixed seed. For p=log2mp= _2m, p hash bits select a register j(z)j(z); the remaining bits determine the rank r(z)r(z), defined as one plus their number of leading zeros. The finite 64-bit hash caps the rank at 64−p+164-p+1. Processing identity z changes one register: Mi,j(z)←maxMi,j(z),r(z).M_i,j(z)← \M_i,j(z),r(z)\. (12) The original identity is not retained. Because a fixed hash and seed map every occurrence of z to the same pair (j(z),r(z))(j(z),r(z)), applying Equation (12) again has no effect once the register has reached that rank. For a group-by request, gt(i)g_t^(i) first selects the state si,gt(i)s_i,g_t^(i). The identity zt(i)z_t^(i) then performs the same one-register update. An identity may therefore appear in several groups, while repeated occurrences within one group remain idempotent. C.2 Segment composition Let si[a]=(Mi,1[a],…,Mi,m[a])s_i^[a]=(M_i,1^[a],…,M_i,m^[a]) be the state built from segment a of stream i. States are compatible when they use the same register count, hash function, and seed. Their merge is the registerwise maximum: si=⨁a=1Pisi[a],Mi,j=max1≤a≤PiMi,j[a].s_i= _a=1^P_is_i^[a], M_i,j= _1≤ a≤ P_iM_i,j^[a]. (13) Proposition 4 (Segment composition). Equation (13) produces exactly the same register array as processing the complete stream once. Proof. For register j, each segment stores the largest rank among identities in that segment that map to j, or zero if no such identity occurs. Taking the maximum over segments is therefore the largest rank among all identities in the complete stream that map to j. This is precisely the value produced by a single pass. The argument holds independently for every register. ∎ The equality is at the state level; the cardinality readout remains an estimate. States from different streams are not combined during construction, because relation readouts must still know which stream contributed each register array. C.3 Cardinality and union For the register counts evaluated in this paper (m≥256m≥ 256), let F~0(si)=αmm2(∑j=1m2−Mi,j)−1,αm=0.72131+1.079/m. F_0(s_i)= _mm^2 ( _j=1^m2^-M_i,j )^-1, _m= 0.72131+1.079/m. (14) If F~0(si)≤2.5m F_0(s_i)≤ 2.5m and Vi>0V_i>0, where ViV_i is the number of zero registers, the implementation uses the small-range correction F^0(si)=mlogmVi. F_0(s_i)=m mV_i. (15) Otherwise, F^0(si)=F~0(si) F_0(s_i)= F_0(s_i). These are the standard HLL readouts (Flajolet et al. 2007). A distinct-count request returns F^0(si) F_0(s_i). A union request forms s1⊕s2s_1 s_2 temporarily and returns y^q=ρq(s1,s2)=F^0(s1⊕s2). y_q= _q(s_1,s_2)= F_0(s_1 s_2). (16) The temporary merge does not overwrite either state. C.4 Jaccard similarity and containment Jaccard similarity and containment read the two compatible HLL arrays jointly. Let A=A1A=A_1 and B=A2B=A_2, and define n10=|A∖B|,n01=|B∖A|,n11=|A∩B|.n_10=|A B|, n_01=|B A|, n_11=|A∩ B|. The joint maximum-likelihood estimator (JMLE) estimates these three regions directly from the paired registers (Ertl 2017). The following likelihood records the implementation used in our experiments. Let Q=64−log2mQ=64- _2m be the number of rank bits and define w(k)=2−k,0≤k≤Q,0,k≥Q+1.w(k)= cases2^-k,&0≤ k≤ Q,\\ 0,&k≥ Q+1. cases (17) The second branch handles the saturated register value Q+1Q+1 exactly. Under the standard Poissonized HLL model, the joint cumulative probability of a paired register is F(u,v;)=exp[−n10w(u)+n01w(v)+n11w(minu,v)m],F(u,v; n)= \! [- n_10w(u)+n_01w(v)+n_11w( \u,v\)m ], (18) where =(n10,n01,n11) n=(n_10,n_01,n_11) and F(u,v;)=0F(u,v; n)=0 if u<0u<0 or v<0v<0. The probability of observing the register pair (u,v)(u,v) is the two-dimensional finite difference p(u,v;)= p(u,v; n)= F(u,v;)−F(u−1,v;) F(u,v; n)-F(u-1,v; n) −F(u,v−1;)+F(u−1,v−1;). -F(u,v-1; n)+F(u-1,v-1; n). (19) If HuvH_uv counts how many of the m register pairs equal (u,v)(u,v), JMLE maximizes ℓ()=∑u,vHuvlogp(u,v;)subject ton10,n01,n11≥0. ( n)= _u,vH_uv p(u,v; n) to n_10,n_01,n_11≥ 0. (20) The implementation performs deterministic coordinate ascent in log space and tests zero explicitly for every coordinate. A candidate step is accepted only when it does not decrease Equation (20). Convergence requires both parameter stability and the boundary-aware first-order condition: interior coordinates must have negligible gradient, while a coordinate at zero must have no improving feasible direction. The optimizer returns its termination reason, iteration count, boundary coordinates, and optimality residual. If these checks fail, the relation readout is marked invalid; it is not replaced by inclusion–exclusion. With the resulting estimates (n^10,n^01,n^11)( n_10, n_01, n_11), the readouts are J^=n^11n^10+n^01+n^11,C^A→B=n^11n^10+n^11. J= n_11 n_10+ n_01+ n_11, C_A→ B= n_11 n_10+ n_11. (21) The zero-denominator cases follow the conventions in Section B. Both ratios use the same two states; no additional persistent set state is allocated. Appendix D Frequency-State Extension HLL deliberately ignores repeated identities. If request q asks for the frequency of one identity z⋆z known before processing, an exact scalar counter is enough: c←c+[z=z⋆].c← c+1[z=z ]. (22) Scalar states built from separate segments merge by addition. When the queried identity is selected after processing, or when several identities may be queried, Count–Min Sketch maintains C∈ℕ0dcm×wcmC _0^d_cm× w_cm (Cormode and Muthukrishnan 2005). Let ηℓ _ be the hash function for row ℓ . Each occurrence updates one counter per row, and a point query returns the smallest corresponding counter: Cℓ,ηℓ(z) C_ , _ (z) ←Cℓ,ηℓ(z)+1, ← C_ , _ (z)+1, ℓ=1,…,dcm, =1,…,d_cm, (23) μ^i(z⋆) μ_i(z ) =min1≤ℓ≤dcmCℓ,ηℓ(z⋆). = _1≤ ≤ d_cmC_ , _ (z ). Compatible tables use the same dimensions and row hashes and merge by elementwise addition. If NiN_i is the number of updates to stream i, the standard hashing assumptions give μi(z)≤μ^i(z)≤μi(z)+eNiwcm _i(z)≤ μ_i(z)≤ _i(z)+ eN_iw_cm (24) with probability at least 1−e−dcm1-e^-d_cm. The table size is fixed, but the collision term grows with stream mass. Count–Min is included as a frequency extension; the primary set-based experiments use HLL. Appendix E Experimental Protocol E.1 Cohorts and comparison arms The aggregate–then–reason evaluation fixes the structured records and precompiled operator specification before model inference. This isolates the question studied here: whether a fixed-budget aggregation state supplies the evidence needed for a later decision. It does not evaluate extraction from raw prose. The large-scale cohort contains 3,969 unique tasks from 174 source windows and is evaluated in full by both Qwen and Gemma. A fixed 1,200-task subset spanning the same 174 windows is used for the same-cohort code-execution comparison. The full-context comparison selects one task per window without looking at its answer, giving 174 matched tasks per model. All full-context methods use the same served 150,000-token limit. Four state-handoff arms separate approximation error from missing decision information. The exact arm supplies the exact aggregate and the decision condition. The budgeted arm replaces the exact aggregate with the fixed-budget readout. The aggregate-only and policy-only controls withhold the decision condition and the aggregate, respectively. The full-context comparison uses direct prompting, chain-of-thought prompting, and guided-choice prompting on the matched raw context. E.2 Data provenance and deterministic task construction The experiments introduce no new raw corpus. They use the public Oolong-Synth dataset (Bertsch et al. 2025), available from https://huggingface.co/datasets/oolongbench/oolong-synth. The large-scale campaign loads the dataset at revision f0d59eaf0febf130664cfceb710436c8e3216b2b. The released structured records are consumed directly: there is no imputation, learned preprocessing, token filtering, or data augmentation. Canonical identities are converted to UTF-8 strings before fixed-seed hashing. Generator settings were fixed using 30 validation windows and then applied once to the test split. The final values were a maximum of 30 candidate items per source window, a 16-item minimum, a 0.60 non-additive fraction, a target of 26 tasks per window, and 16 generation attempts per operator. The generator seed was 20,260,720. Test answers were not used to select these settings or the 1,200-task replication subset. E.3 Final settings and development ranges Table 7 lists the final settings used by the paper-facing runs. The main HLL size was fixed before test evaluation. The controlled budget study additionally evaluated m∈256,512,1024,2048,4096,8192m∈\256,512,1024,2048,4096,8192\ without changing the main-run setting. UMBC architecture selection used validation streams only; test streams were opened after the winning architecture was fixed. Table 7: Final experimental settings. “TP” denotes tensor parallelism. Component Final setting HLL m=2048m=2048 one-byte registers; 64-bit keyed hash; seed 0; register-max merge HLL JMLE 24 coordinate sweeps; parameter tolerance 10−310^-3; KKT tolerance 5×10−45× 10^-4; no inclusion–exclusion fallback Count–Min 4 rows × 64 columns of int64 cells (2,048 bytes per table); label seed 0 and user seed 1 Qwen reasoner Qwen3.6-35B-A3B revision 995ad96eacd98c81ed38be0c5b274b04031597b0; BF16; temperature 0; test seed 20,260,718; 96 output tokens for state handoff; 8,192 direct and 16,384 CoT output-token caps; 150,000 served tokens; TP2 Gemma reasoner Gemma-4-31B-IT revision 3548789868c5356dbf307c98e6f609007b82b3eb; BF16; temperature 0; test seed 20,260,721; 96 output tokens for state handoff and a 150,000-token served limit; direct MLX serving with mlx-lm 0.31.3; no quantization Code execution The Qwen and Gemma revisions above; temperature 0; at most 4,096 generated tokens per round, three rounds, and 60 seconds of isolated Python execution per round; Apptainer isolation for Qwen and macOS Seatbelt isolation for Gemma Official UMBC 2,048-byte carried state; deterministic 16-slot state with d^=28 d=28, four heads, and slot-sigmoid attention; AdamW, learning rate 3×10−33× 10^-3, 4,000 final steps; training seeds 0,1,2\0,1,2\ Uncertainty 10,000 source-window-cluster bootstrap resamples; analysis seeds 20,260,720 and 20,260,721; paired randomization seed 20,260,728 E.4 Run counts Every reported model–task–arm cell is one deterministic inference run at temperature zero; rows from repeated arms are not counted as additional independent tasks. The Qwen and Gemma scale experiments each evaluate the same 3,969 tasks from 174 windows once in each of six prespecified conditions (23,814 terminal rows per model). The balanced full-context experiment uses 174 tasks per prompt condition and model. The code-execution reference uses the fixed 1,200-task subset once per model (2,400 terminal task runs in total). Failed calls or parses remain in their corresponding denominator with score zero. E.5 Counterfactual mechanism check The counterfactual arm changes the supplied aggregate and recomputes the answer that follows from that change. It therefore checks whether the reasoner uses the aggregate evidence rather than reproducing the original answer. Across 3969 tasks from 174 source windows, counterfactual accuracy is 92.82% [91.94%, 93.67%]. This is a mechanism diagnostic, not accuracy on the original task. E.6 Common-packing-eligible sensitivity The primary full-context comparison retains all 174 tasks, including failures. As a secondary check, we intersect the tasks that fit under direct, chain-of-thought, and guided prompting for both models. This answer-blind rule leaves 94 tasks. Parsing failures would remain in the denominator, although none occur in the three values shown in Table 8. Table 8: Common-packing-eligible sensitivity analysis. Values are task accuracy or paired percentage-point differences with 95% source-window-cluster bootstrap intervals. This selected analysis does not replace the all-window primary comparison. Model n Budgeted state Guided context Difference Qwen3.6-35B-A3B 94 96.81% [92.55%, 100%] 51.06% [41.49%, 61.7%] 45.74% [36.17%, 55.32%] Gemma-4-31B-IT 94 100% [100%, 100%] 79.79% [71.28%, 87.23%] 20.21% [12.77%, 28.72%] Appendix F Controlled State and Resource Results F.1 Full-scale state-handoff results Table 9 reports the complete 3,969-task state-handoff evaluation for both models. Every condition uses the same task set and source windows. Table 9: Accuracy on all 3,969 tasks from 174 source windows. Brackets give 95% source-window-cluster bootstrap intervals. All failures remain in the denominator with score zero. Condition Qwen Gemma Fixed-budget state 88.99% [87.87%, 90.09%] 99.17% [98.72%, 99.52%] Exact aggregate 93.02% [92.23%, 93.83%] 100% [100%, 100%] Counterfactual aggregate 92.82% [91.94%, 93.67%] 100% [100%, 100%] Aggregate-only 52.18% [51.13%, 53.2%] 51.37% [50.1%, 52.58%] Policy-only 52.58% [51.59%, 53.56%] 27.29% [24.97%, 29.72%] Gemma reaches 99.17% with the fixed-budget state and 100.00% with the exact aggregate. The paired fixed-budget-minus-exact difference is −0.83-0.83 percentage points (95% cluster interval, −1.28-1.28 to −0.48-0.48). The Gemma policy-only arm has 1,772 parsing failures; they remain in the denominator. The other five Gemma arms have no parsing, transport, or model failures. F.2 Official UMBC comparison at equal carried-state size We compare HLL with the official Universal Mini-Batch Consistency (UMBC) implementation (Willette et al. 2023). Both methods carry 2,048 bytes from one partition to the next and receive the same identity stream. Equal carried state does not mean equal total memory: UMBC also has learned parameters, and both methods use the same frozen feature trunk. Table 10 therefore reports these resource cells separately. Table 10: Median relative error at equal 2,048-byte carried state, using 100 streams per length. Brackets give 95% bootstrap intervals. The final column is the paired UMBC-minus-HLL difference. Length HLL Official UMBC Paired difference 8K 1.29% [1.08%, 1.54%] 27.6% [24.36%, 33.8%] 26.69% [22.35%, 32.41%] 50K 1.49% [1.23%, 1.6%] 26.78% [21.32%, 33.46%] 25.05% [19.43%, 30.93%] 200K 1.49% [1.2%, 1.62%] 26.95% [22.76%, 33.46%] 25.49% [20.86%, 32.37%] Table 11: Resource accounting for the equal-carried-state comparison. Static parameters and the shared trunk are not folded into the carried-state cells. Resource cell Bytes HLL resident register state 2048 UMBC carried state 2048 UMBC static parameters 66868 Shared frozen feature trunk 8394880 HLL has lower median error at all three lengths under the equal carried-state budget. The comparison supports a claim about the state passed across partitions, not a claim that the two end-to-end systems have identical memory. F.3 Register budget and distinct-count error Table 12 gives the complete controlled budget sweep. The state uses one byte per register, so the storage column is also the serialized register payload. Increasing m lowers error without changing the update rule or introducing storage that grows with the stream. Table 12: Distinct-count RMSE across HLL register budgets. Brackets give 95% bootstrap intervals; n is the number of controlled streams at that budget. Registers m State bytes n Relative-error RMSE 256 256 240 6.47% [5.92%, 7.02%] 512 512 200 4.6% [4.24%, 4.98%] 1,024 1,024 160 3.33% [2.98%, 3.65%] 2,048 2,048 160 2.34% [2.09%, 2.6%] 4,096 4,096 120 1.72% [1.52%, 1.89%] 8,192 8,192 120 1.1% [0.95%, 1.24%] The 1.6% value reported in the abstract uses a different, explicitly fixed summary. At m=2048m=2048 and one million records per stream, the fixed-data hash-replicate axis evaluates two streams under ten hash seeds (20 estimates). Its mean absolute relative error, mean(|y^−y|/y)mean(| y-y|/y), is 1.568%, reported as 1.6%. In contrast, Table 12 reports RMSE over the combined data- and hash-replicate axes. The two values differ because both the replicate pool and the error functional differ. Fixed size does not imply that a sketch is always smaller than an exact representation. For one m=2048m=2048 HLL state, the measured crossover against the serialized UTF-8 identity payload is 139.26 distinct identities (95% CI, 138.69–139.86). Exact payload is smaller below this point; HLL is smaller above it. The measurement excludes container, allocator, and whole-system overhead and must not be read as a Python-set memory comparison. F.4 Same-cohort code-execution reference The code-execution arm uses the same 1,200 tasks, structured records, and operator specifications as the state arms for each model. It generated a Python program for every task and executed that program in a sandbox. Table 13 reports failure-as-zero accuracy and average generation resources on these common cohorts. Table 13: Same-cohort comparison on 1,200 tasks and 174 windows per model. Calls and output tokens are averages per task. Model Method Accuracy Calls Output tokens Qwen Fixed-budget state 91.08% 1.00 2.2 Qwen Code execution 100.00% 1.01 1260.2 Gemma Fixed-budget state 99.25% 1.00 2.2 Gemma Code execution 100.00% 1.0 208.6 All 1,200 runs completed correctly for each model. Qwen used 1.01 rounds and 1,260.2 output tokens per task on average; twelve initially unparseable attempts were repaired by the fixed retry rule. Gemma used one round and 208.6 output tokens per task, with no generation, execution, parsing, or transport failures. Code execution exceeds the fixed-budget state by 8.92 points on Qwen (95% cluster interval, 7.14–10.71) and 0.75 points on Gemma (0.25–1.33). It is an exact-execution reference with access to the full records and a trusted sandbox, not an equal-state or equal-compute baseline. Appendix G Statistical Analysis and Evaluation Scope G.1 Evaluation and statistical analysis Task accuracy is the fraction of final choices matching the independently derived gold answer. Context-packing, model-call, and parsing failures stay in the prespecified denominator and receive score zero. Because tasks from the same source window are related, all reported confidence intervals for task accuracy and paired differences use 10,000 source-window-cluster bootstrap resamples. The estimand is task-weighted accuracy; the 174 windows, not the number of arm rows, are the independent clusters. For numerical aggregation, relative error is relerr(y^,y)=|y^−y|max(1,|y|).relerr( y,y)= | y-y| (1,|y|). (25) The budget study reports the root mean square of this quantity. The learned state comparison reports its median over 100 streams at each length. Those two summaries answer different questions and are not compared across tables. For the main paired contrasts, we additionally use a two-sided source-window sign-flip randomization test. The statistic is the task-weighted mean paired score difference, but the sign is randomized once per source window, preserving all within-window dependence. We use 100,000 Monte Carlo draws, a plus-one correction, and Holm correction over the eight contrasts in Table 14. The analysis rejects non-terminal rows, duplicate item–arm keys, mismatched item sets, and mismatched source windows; all input SHA-256 values are recorded with the result. Table 14: Paired source-window randomization tests. Differences are left minus right in percentage points. pHolmp_Holm corrects the eight displayed tests. Contrast Difference pHolmp_Holm Qwen budgeted −- exact (3,969 tasks) −4.03-4.03 8.0×10−58.0× 10^-5 Gemma budgeted −- exact (3,969 tasks) −0.83-0.83 8.0×10−58.0× 10^-5 Qwen budgeted −- direct context (174 tasks) +63.22+63.22 8.0×10−58.0× 10^-5 Qwen budgeted −- CoT context (174 tasks) +60.92+60.92 8.0×10−58.0× 10^-5 Gemma budgeted −- direct context (174 tasks) +56.32+56.32 8.0×10−58.0× 10^-5 Gemma budgeted −- CoT context (174 tasks) +63.22+63.22 8.0×10−58.0× 10^-5 Qwen code execution −- budgeted (1,200 tasks) +8.92+8.92 8.0×10−58.0× 10^-5 Gemma code execution −- budgeted (1,200 tasks) +0.75+0.75 0.007920.00792 G.2 Scope of the evaluation The following boundaries apply to every result: • The evaluated aggregate-to-decision path starts from released, structured records with stable canonical identities and a precompiled operator specification. It does not establish the accuracy of extracting those identities from unrestricted prose. • The fixed budget is per operand state or active group. Total aggregation memory grows with the number of simultaneously maintained operands and groups, even though each state does not grow with stream length or set cardinality. • The state is updated alongside the frozen model’s forward computation. Its readout is appended as explicit evidence, and decoding continues from the retained KV cache; no fused Transformer kernel or hidden-state modification is claimed. • JMLE relation results are consumed only when the optimizer returns a finite feasible estimate satisfying its convergence checks. Failed readouts remain failures; inclusion–exclusion is not used silently. • The learned-state comparison matches carried-state bytes. The crossover study compares one HLL state with serialized UTF-8 identity payload. Neither result is a total-system memory comparison.