Paper deep dive
Streaming Communication in Multi-Agent Reasoning
Zhen Yang, Xiaogang Xu, Wen Wang, Cong Chen, Xander Xu, Ying-Cong Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/9/2026, 1:38:26 AM
Summary
The paper introduces STREAMMA, a multi-agent reasoning system that replaces the traditional 'generate-then-transfer' serial communication protocol with a step-level streaming protocol. By forwarding reasoning steps immediately upon generation, STREAMMA enables pipeline parallelism, reducing end-to-end latency. Surprisingly, this also improves effectiveness because early reasoning steps are typically more reliable than later ones; streaming allows downstream agents to begin reasoning on reliable prefixes before error-prone late steps arrive. The authors provide a closed-form theoretical analysis (Theorems 1-3) proving the effectiveness ordering, speedup bounds, and cost ratios. Empirically, STREAMMA outperforms serial and single-agent baselines across eight benchmarks, two frontier LLMs, and three topologies, while revealing a new 'step-level scaling law' that independently boosts performance and efficiency.
Entities (16)
Relation Signals (13)
STREAMMA → implements → Stream Protocol
confidence 98% · We propose STREAMMA, a multi-agent reasoning system built on Stream, a reasoning-step-level communication protocol that shifts the transmission unit from complete responses to reasoning steps
Theorem 2 → derives → speedup upper bound
confidence 95% · Theorem 2 and Theorem 3 derive the speedup upper bound and exact cost ratio under LLM serving conditions.
Theorem 3 → derives → cost ratio
confidence 95% · Theorem 3 an exact cost ratio. Under the same setup as Theorem 2, the cost ratio of Stream over Serial is:
Theorem 1 → formalizes → effectiveness ordering
confidence 95% · Theorem 1 gives an effectiveness ordering across all six regimes, predicting Stream optimal when early steps are reliable and late ones degrade
Stream Protocol → improveseffectivenesscomparedto → Serial Protocol
confidence 95% · STREAMMA outperforms Serial by an average+7.3pp (peak+22.4pp on HMMT 2026, Claude Opus 4.6-high)
Stream Protocol → reduceslatencycomparedto → Serial Protocol
confidence 95% · STREAMMA outperforms Serial by an average+7.3pp while reducing latency: less waiting, yet better reasoning.
STREAMMA → testedon → Claude Opus 4.6
confidence 95% · Across eight reasoning benchmarks spanning mathematics, science, and code, two frontier LLMs (Claude Opus 4.6 and GPT-5.4), and three topologies (Chain, Tree, Graph), StreamMA outperforms both baselines
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent reasoning systems adopt a "generate-then-transfer" paradigm that forces end-to-end latency to scale linearly with pipeline depth. We introduce StreamMA, a multi-agent reasoning system that streams each reasoning step to downstream agents as soon as it is generated, pipelining adjacent agents and thus reducing latency. Surprisingly, this pipelining also improves effectiveness: because multi-step reasoning quality is non-uniform and early steps are more reliable than later ones, working with these reliable early steps instead of the full chain prevents error-prone late steps from misleading downstream agents. We formalize both advantages with the first closed-form joint analysis of stream, serial, and single protocols, deriving the effectiveness ordering, speedup upper bound, and cost ratio. Across eight reasoning benchmarks spanning mathematics, science, and code, two frontier LLMs (Claude Opus 4.6 and GPT-5.4), and three topologies (Chain, Tree, Graph), StreamMA outperforms both baselines (avg. +7.3 pp, max +22.4 pp on HMMT 2026; Claude Opus 4.6-high). Beyond these contributions, we discover a "step-level scaling law": increasing per-agent steps consistently improves both effectiveness and efficiency, a new scaling dimension orthogonal to and composable with agent-count scaling.
Tags
Links
- Source: https://arxiv.org/abs/2606.05158v1
- Canonical: https://arxiv.org/abs/2606.05158v1
Trouble viewing inline? Open PDF directly →
Full Text
95,853 characters extracted from source content.
Expand or collapse full text
Streaming Communication in Multi-Agent Reasoning Zhen Yang 1 , Xiaogang Xu 3 , Wen Wang 3 , Cong Chen 3 , Xander Xu 2 * , Ying-Cong Chen 1,4 * 1 HKUST(GZ), 2 Alibaba Group, 3 ZJU, 4 HKUST zheny.cs@gmail.com, yingcongchen@ust.hk Abstract Multi-agent reasoning systems adopt a “generate-then-transfer” paradigm that forces end-to-end latency to scale linearly with pipeline depth. We introduce STREAMMA, a multi-agent reasoning system that streams each reasoning step to downstream agents as soon as it is generated, pipelining adjacent agents and thus reducing latency. Surprisingly, this pipelining also improves effectiveness: because multi-step reasoning quality is non-uniform and early steps are more reliable than later ones, working with these reliable early steps instead of the full chain prevents error-prone late steps from misleading down- stream agents. We formalize both advantages with the first closed-form joint analysis of stream, serial, and single protocols, deriving the effectiveness ordering, speedup upper bound, and cost ratio. Across eight reasoning benchmarks spanning mathematics, science, and code, two frontier LLMs (Claude Opus 4.6 and GPT-5.4), and three topologies (Chain, Tree, Graph), STREAMMA outperforms both baselines (avg. +7.3 p, max +22.4 p on HMMT 2026; Claude Opus 4.6-high). Beyond these contributions, we discover a “step-level scaling law”:increasing per-agent steps consistently improves both effectiveness and efficiency, a new scaling dimension orthogonal to and composable with agent-count scaling. Our project page and code are available. 1 Introduction Multi-agent systems have emerged as a paradigm for complex reasoning: agents are organized as nodes of a directed acyclic graph (DAG), with each directed edge governing information propagation from one agent to its successor, leveraging special- ization and cross-verification to surpass a single model (Wu et al., 2023; Hong et al., 2024; Qian et al., 2024), with growing benefits as the num- ber of agents increases (Qian et al., 2025). Yet * Co-corresponding authors. (a) Serial Protocol (b) Stream Protocol : Effective + Efficient ! Figure 1: Communication protocols. (a) Serial: the downstream agent receives the upstream agent’s com- plete response before execution. (b) Stream: the down- stream agent receives each upstream reasoning step as it is generated, enabling pipelined execution. existing frameworks share the same communica- tion assumption: an upstream agent must finish its entire response before passing it downstream. This “generate-then-transfer” (Serial, Fig. 1a) pro- tocol forces downstream agents to remain idle and latency to scale with pipeline depth. While streaming partial outputs is standard within a sin- gle LLM (Tong et al., 2026), between agents the transfer unit has remained a complete response, and its effect on effectiveness remains underexplored. A natural question arises: can we forward partial outputs before the upstream agent finishes? Con- ventional wisdom holds that incomplete context must hurt downstream quality: “more information yields better decisions.” Yet this intuition does not always hold: we find that in multi-step rea- soning, receiving less context can improve down- stream reasoning quality. We propose STREAMMA (Fig. 1b), a multi-agent reasoning system built on Stream, a reasoning-step-level communication pro- tocol that shifts the transmission unit from com- 1 arXiv:2606.05158v1 [cs.CL] 3 Jun 2026 plete responses to reasoning steps, forwarding each step once produced, pipelining upstream and down- stream agents. Across eight benchmarks in math- ematics, science, and code, STREAMMA outper- forms Serial by an average+7.3p (peak+22.4p on HMMT 2026, Claude Opus 4.6-high) while re- ducing latency: less waiting, yet better reasoning. Why does the downstream agent benefit from only the early reasoning steps more than the full response? On complex reasoning tasks, step-level quality is position-dependent: early steps tend to be reliable, whereas later steps degrade (Wu et al., 2025). The Serial protocol forces the downstream agent to condition on the entire response, including its error-prone late steps. Stream lets it begin from the most reliable prefix; by the time degraded steps arrive, the agent has formed its own independent reasoning trajectory, diluting their impact. Con- trolled step-level perturbations cross-validate this asymmetry: corrupting only the tail steps of an up- stream reasoning trajectory leaves Stream nearly intact and ahead of Serial by+24.0p, while cor- rupting only its head steps flips the gap to−36.0p (Sec. 4.2.3). We formalize this mechanism: The- orem 1 gives an effectiveness ordering across all six regimes, predicting Stream optimal when early steps are reliable and late ones degrade; Theorem 2 and Theorem 3 derive the speedup upper bound and exact cost ratio under LLM serving conditions. Beyond mechanism and theory, our experiments uncover a second scaling axis, orthogonal to the well-studied agent-count axis: at fixed agent count A, increasing per-agent stepsSconsistently im- proves effectiveness and speedup, defining a step- level scaling law fully compositional with agent- count scaling. On HMMT 2026 with GPT-5.4- medium, scalingAalone from2to64lifts effec- tiveness from58.3%to68.2%; further increasing Sto64pushes it to73.5%with26.9×wall-clock speedup (83%of the Theorem 2 upper bound). STREAMMA thus simultaneously serves as a rea- soning enhancer and an inference accelerator. In summary, our contributions are as follows: • Protocol. We introduce Stream, a reasoning- step-level communication protocol that for- wards each reasoning step upon completion, pipelining upstream and downstream agents over arbitrary DAG topologies. Stream im- proves effectiveness while reducing latency. • Theory. We present the first closed-form joint analysis of Stream, Serial, and Single proto- cols: Theorem 1 provides an effectiveness ordering, Theorem 2 a speedup upper bound, and Theorem 3 an exact cost ratio. •Empirics. Across eight benchmarks, two LLMs, and three topologies, STREAMMA consistently outperforms both baselines in effectiveness (Claude average+7.3p over Serial,+16.3p over Single; GPT average +1.5 p over Serial, +4.9 p over Single). •Discovery. We discover a step-level scaling law under Stream: at fixed agent countA, increasing per-agent stepsSconsistently im- proves effectiveness and speedup. AtA=64, scalingSto64gains+5.3p complementary to agent-count scaling, with 26.9× speedup. 2 Related Work Multi-Agent Reasoning and Communication. Decomposing reasoning across collaborating agents has become a mainstream paradigm for com- plex LLM tasks (Wu et al., 2023; Hong et al., 2024; Qian et al., 2024; Li et al., 2023; Chen et al., 2024; Wang et al., 2025a; Sun et al., 2023), with prior work advancing it along three orthogonal axes: the communication topology (Qian et al., 2025; Zhuge et al., 2024; Liu et al., 2024; Zhang et al., 2025); the content exchanged between agents, such as inter- mediate rationales (Yin et al., 2023) or KV-cache representations (Fu et al., 2025), and how much of it (Rizvi-Martel et al., 2025); and the scale of agents (Qian et al., 2025; Li et al., 2024). Yet all three share a generate-then-transfer assumption: an upstream agent must finalize its response before any downstream agent can act, forcing per-round sequential waits and forgoing pipeline parallelism. We instead refine the granularity of communication, from full responses to reasoning steps. This shift exposes a new design axis: per-agent step count, orthogonal to agent-count scaling. Increasing per- agent steps simultaneously improves effectiveness and speedup, the step-level scaling law. Step-Level Reasoning Quality. Step-by-step reasoning is now standard (Wei et al., 2022; Yao et al., 2023), with iterative-refinement variants such as Self-Refine (Madaan et al., 2023), Reflex- ion (Shinn et al., 2023), and multi-agent debate (Du et al., 2024) pushing output quality higher. Criti- cally, on complex reasoning tasks, step quality is position-dependent: Wu et al. (2025) show that 2 Algorithm 1 SERIAL EXECUTION Require: Q; (Agent a , ctx a , queue a ) A a=1 ◦ Q: query ◦ ctx a : per-agent context ◦ queue a : FIFO queue ◦ chain: Agent 1 →·→ Agent A 1: msg ← Q 2: for a = 1 to A do 3:ctx a .append(msg ) 4: ▷ wait; complete output 5:msg ← LLM(ctx a ) 6: end for CoT accuracy peaks at an optimal length and de- grades beyond it, and process supervision (Uesato et al., 2022; Lightman et al., 2024; Wang et al., 2024) scores reliability step by step. So far, how- ever, this property has been exploited only to verify or train a single model’s reasoning chain, never to design the inter-agent protocol. We are the first to lift this property into protocol design: our Theo- rem 1 gives a closed-form effectiveness ordering of Single, Serial, and Stream, with explicit conditions under which each mode is provably optimal. Pipeline Parallelism and Streaming Inference. Pipeline parallelism is classical in distributed train- ing (Narayanan et al., 2021), and streaming infer- ence for LLMs (Tong et al., 2026) has been pursued along two axes: at the intra-agent level, via specu- lative decoding (Leviathan et al., 2023; Chen et al., 2023; Cai et al., 2024; Fu et al., 2024; Hu et al., 2026), Group Think (Hsu et al., 2025), and Multi- Stream LLMs (Su et al., 2026); and at the inter- agent level, via skeleton expansion (Ning et al., 2024), speculative agent actions (Ye et al., 2025b), and staircase streaming (Wang et al., 2025b). Yet across all of these, streaming primarily serves as a speedup mechanism; any effectiveness gains are incidental side-effects rather than the central design principle. Our Stream instead operates at the reasoning-step level over arbitrary multi- agent DAGs: a granularity coarser than tokens or fixed token chunks, yet finer than pre-decomposed skeletons or speculative agent actions, and uncon- strained by topology. To our knowledge, we are the first to show that it can improve effectiveness, not just accelerate it (Theorem 1). At the inter- agent level, our protocol is orthogonal to all the intra-agent methods discussed above. Algorithm 2 STREAM EXECUTION 1: queue 1 .put(Q) 2: ▷ all agents concurrent 3: for a = 1 to A in parallel do 4:while msg ← queue a .get() do 5:ctx a .append(msg ) 6:▷ yield step-by-step 7:steps ← LLM(ctx a , stream=True) 8:for each step from steps do 9:if a < A then 10:▷ push; no wait 11:queue a+1 .put(step) 12:end if 13:▷ KV cache reuse 14:ctx a .append(step) 15:end for 16:end while 17: end for 3 Method For analytical tractability, we develop our theory on a chain ofAagents(Agent 1 → · → Agent A ), each producingSreasoning steps; the results ex- tend to general directed acyclic graphs. We first describe the algorithm (Sec. 3.1), then characterize its effectiveness (Sec. 3.2) and efficiency (Sec. 3.3). Notation is summarized in App. A.1. We compare three execution modes throughout: Stream, our proposed stream protocol; Serial, the serial proto- col; and Single, the single-agent protocol. We refer to our full framework, including the protocol and its theoretical analysis, as STREAMMA. 3.1 Algorithm Description Alg. 1 and Alg. 2 contrast the serial and stream protocols. The key difference lies in the timing of message passing: in the serial protocol, each agent waits for the blocking callLLM(ctx a )to re- turn the complete response (Alg. 1, line 5) before the next agent can begin; in the stream protocol, each agent issues a streaming call that yields steps one by one (Alg. 2, lines 7–8) and immediately pushes every completed step toqueue a+1 (line 11) without waiting for subsequent steps. All agents execute concurrently (line 3), enablingAgent a+1 to process stepswhileAgent a is still generating steps + 1, forming pipeline parallelism. Moreover, each downstream agent (a≥ 2) is calledStimes, with prior steps serving as context for each subse- quent call (line 14); the appended context naturally 3 Theorem 1 (Effectiveness Ordering). Depending on how ̄p,p head ,p tail (uniform, head-weighted, and tail- weighted step-correctness means) compare to p ∗ , the sCorr ordering among three modes falls into six cases: (I) Stream advantage [p head >p ∗ and p tail <p ∗ ]: (a) If ̄p > p ∗ : sCorr stream > sCorr serial > sCorr single j p j p ∗ (b) If ̄p < p ∗ : sCorr stream > sCorr single > sCorr serial j p j p ∗ (I) Serial advantage [ ̄p>p ∗ and p tail >p ∗ ]: (a) If p head > p ∗ : sCorr serial > sCorr stream > sCorr single j p j p ∗ (b) If p head < p ∗ : sCorr serial > sCorr single > sCorr stream j p j p ∗ (I) Single advantage [p head <p ∗ and ̄p<p ∗ ]: (a) If p tail < p ∗ : sCorr single > sCorr stream > sCorr serial j p j p ∗ (b) If p tail > p ∗ : sCorr single > sCorr serial > sCorr stream j p j p ∗ forms a shared prefix amenable to KV-cache reuse. The chain topology generalizes to arbitrary DAGs by (i) broadcastingQto all source nodes (in-degree 0) and (i) pushing each step to all di- rect successors; multi-predecessor nodes process incoming steps immediately upon arrival, without any synchronization barrier across predecessors. The full pseudocode is given in App. A.8. 3.2 Effectiveness Characterization STREAMMA consistently outperforms the Serial and Single baselines in our experiments. We now characterize when and why: we model each up- stream stepj(1≤ j ≤ S) as correct with probabil- ityp j ; a correct step raises downstream step-level correctness byδ > 0and an incorrect step low- ers it byε > 0. For clarity we assume uniform δ,εacross positions; the position-dependentδ j ,ε j case yields the same results in vector form. The expected step-level correctness change from step jisμ j = p j δ− (1− p j )ε, which is non-negative iffp j ≥ ε/(δ + ε) ≜ p ∗ . We definesCorr mode (mode∈stream, serial, single) as the mean of μ j under each mode; sCorr is positively corre- lated with task-level accuracy. Three weighted means ofp j , the uniform mean ̄p, the head- weightedp head (earlier steps weighted more), and the tail-weightedp tail (later steps weighted more), govern the sCorr ordering among the three modes. Theorem 1 formalizes the effectiveness ordering across six regimes. Embedded profiles are simpli- fied for exposition; real trajectories may differ. Stream advantage. Cases I.a and I.b share the typ- ical error-accumulation pattern in multi-step LLM reasoning: early steps are reliable but late ones fall belowp ∗ . When context arrives matters more than how much: under Stream, the downstream agent re- ceives early steps and begins reasoning; by the time degraded late steps arrive, their impact is diluted. Under Serial, the downstream agent must read the entire response, mixing clean early reasoning with error-prone late steps. In Case I.a, ̄p > p ∗ means the response is on average helpful, so Stream leads while Serial still beats Single. In Case I.b, only the early steps exceedp ∗ , so Stream exposes these reli- able steps and beats Single, but Serial accumulates below-p ∗ late steps and loses to Single. Serial advantage. In Case I.a, all steps are above p ∗ , so more context is beneficial: “more informa- tion, better decisions” holds. Serial sees the full response and wins; Stream sees partial but helpful steps and beats Single. In Case I.b, early steps are belowp ∗ but late ones are accurate enough that ̄p > p ∗ ; Serial profits and beats Single, yet Stream exposes the harmful early steps and ends up last. This captures self-correction: the upstream agent makes early mistakes but corrects them eventually. Single advantage. Case I.a corresponds to prob- lems so hard that all steps fall belowp ∗ , making every upstream step harmful; Single avoids any up- stream influence and achieves the highest step cor- rectness. In Case I.b, the profile rises but ̄p < p ∗ holds; this regime is rare, as a heavily corrupted prefix leaves too few remaining steps to liftp tail abovep ∗ . Serial loses to Single, and Stream, which receives the below-p ∗ early steps, comes last. Variable definitions, full proofs, and the exten- sion to arbitrary DAG topologies are in App. A.2. 4 3.3 Efficiency Characterization Theorem 2 (Speedup). The latency speedup of Stream over Serial is upper-bounded by: Speedup = A (S + r po )r v dp + S (S + A− 1)(1 + αr v dp + β r v dc ) wherer v dp : decode-to-prefill speed ratio;r v dc : decode-to-cache-read speed ratio;r po :system- prompt-to-per-step-output length ratio;α,β: average non-cached / cached context tokens per output token. Speedup. In the streaming multi-agent system, the latency speedup is jointly governed by pre- fill, decode, and KV-cache read speeds (Theo- rem 2). Whenv c ≫ v p ≫ v d (cache-read≫pre- fill≫decode speed), the upper bound reduces to AS/(S +A−1), the maximum achievable speedup, and becomes insensitive to the KV-cache hit rate. Theorem 3 (Cost Ratio). Under the same setup as Theorem 2, the cost ratio of Stream over Serial is: Cost stream Cost serial = ρ· r c pd (α + r c cp β) + 1 r c pd (1 + r po /S) + 1 whereρ: Stream-to-Serial output length ratio;r c pd : prefill-to-decode price ratio;r c cp : cache-to-prefill price ratio; α, β as in Theorem 2. Cost. Although Stream increases API calls, the overall cost remains dominated by decode when the per-token prefill pricec p is far below the per- token decode pricec d ; KV-cache hits further re- duce cost by replacing prefill with cache reads at pricec c ≪ c p . When pricing is decode-dominated (r c pd → 0 ), the cost ratio (Theorem 3) reduces to ρ: cost depends on how many tokens each decodes. Each downstream agent in Stream naturally forms a shared prefix amenable to KV-cache reuse, and modern serving stacks—vLLM (Kwon et al., 2023), SGLang (Zheng et al., 2024), and recent agentic extensions (Liu et al., 2026; Wu et al., 2026; Pan et al., 2025; Ye et al., 2025a)—steadily raise the achievable hit rate, so our cost advantage strength- ens as this infrastructure matures. Example.Instantiating both bounds with Claude Opus 4.6 data (v d ≈ 39t/s,v p ≈ 6,000t/s 1 ; c p = $5,c d = $25,c c = $0.50per MTok 2 ) and 1 https://artificialanalysis.ai/models/claud e-opus-4-6/providers 2 https://w.anthropic.com/pricing A = S = 4: under full KV-cache hits, the latency speedup upper bound is2.30×and the cost ratio is 0.925ρ. At ρ = 1, Stream saves 7.5% over Serial. Full derivations, numerical examples, and exten- sion to DAG topologies in App. A.5 and A.6. 4 Results 4.1 Experimental Setup Benchmarks. To assess robustness, we evaluate on eight benchmarks spanning competition mathe- matics (AIME 2025, AIME 2026, HMMT 2026), graduate-level science (GPQA-Diamond (Rein et al., 2024), HLE (Phan et al., 2025)), and program understanding (LiveCodeBench (Jain et al., 2024), with sub-tasks LCB-G: code generation, LCB-E: code execution, LCB-T: test output). Evaluation is conducted via OpenCompass (Contributors, 2023); all configuration files are provided in App. A.7. Baselines.We compare against two baselines: (1) Single, a single agent given only the query; and (2) Serial, a pipeline where each agent waits for its predecessor to finish. STREAMMA inherits Serial’s prompts and decoding, differing only in transmission granularity: Serial transmits the full output, Stream transmits each step on completion. Topologies. Unless otherwise specified, we eval- uate four-agent DAG topologies (A 0 ,A 1 ,A 2 ,A 3 ): Chain (A 0 → A 1 → A 2 → A 3 ); Tree (A 0 → A 1 ,A 2 → A 3 , withA 1 ,A 2 in parallel); and Graph (Chain plus an extra edgeA 0 → A 2 ). These three cover the canonical DAG primitives (linear, branching, and shortcut), giving a built-in sweep of connectivity rather than a single design choice. Implementation Details. We experiment on fron- tier models, Claude Opus 4.6 (high) and GPT-5.4 (medium), following each provider’s API defaults; later references inherit these settings unless noted. Our LLM-as-judge evaluations use GPT-5.4. Sys- tem prompts are provided in App. A.9. All num- bers are averaged over 3 runs per backbone, topol- ogy cell, increased to 8 on AIME 2025/26 and HMMT 2026 due to their smaller test sets. 4.2 Effectiveness Analysis 4.2.1 Quantitative Comparison STREAMMA outperforms Serial in every Avg. cell across both backbones (Tab. 1). STREAMMA only appends a one-lineEND_STEPboundary to each downstream agent’s system prompt and a short solver preamble for the root agent; all other prompt content, decoding hyperparameters, and runtime 5 ModelTopoMethodAIME25 AIME26 HMMT26 GPQA-D HLE LCB-G LCB-E LCB-TAvg. Claude Opus 4.6 (High) –Single67.5060.0048.1183.6718.6090.2577.9484.3166.30 Chain Serial80.4272.0863.2685.8623.9091.3378.6492.38 73.48 STREAMMA92.5089.5885.6187.3726.9791.5084.4195.6381.70 Tree Serial86.2586.2575.0085.1824.8291.9288.4597.59 79.43 STREAMMA93.3487.9282.2085.8625.0794.0094.5799.5582.81 Graph Serial77.9271.6761.7585.6922.1790.0875.7898.27 72.92 STREAMMA95.8387.9282.5886.5327.6892.1795.2798.7283.34 GPT-5.4 (Medium) –Single55.8371.2540.5377.9512.0891.0892.4896.6867.24 Chain Serial60.0070.4254.5575.0814.6690.0897.4399.0270.16 STREAMMA61.2572.5059.1080.3014.9491.1799.3099.4772.25 Tree Serial59.1775.8356.0776.7714.8388.3393.8199.2570.51 STREAMMA62.0875.8358.3478.1215.7489.5094.7899.1771.70 Graph Serial60.0074.1752.6578.4514.0492.2599.5198.8071.13 STREAMMA62.5075.4256.4479.6316.1393.0899.7999.3272.32 Table 1: Effectiveness across eight benchmarks and three topologies. STREAMMA (gray) vs. Single and Serial on Claude Opus 4.6 (high) and GPT-5.4 (medium). Avg. (%): unweighted mean; bold: higher in each Topology, Method pair. Single: one row per backbone, no topology axis. Each cell averages 3 runs (8 on AIME 2025/26 and HMMT 2026 due to small test sets). STREAMMA outperforms both baselines in every Avg. cell. settings remain identical to Serial, ruling out extra prompt engineering as the source of improvement. On Claude Opus 4.6, STREAMMA lifts Avg. accu- racy by+7.3p averaged over the three topologies (peak +22.4 p on HMMT 2026, Chain). STREAMMA’s gain over Serial scales inversely with Serial’s strength. The three topologies span a wide Serial baseline on Claude (Avg.72.92–79.43), giving us a built-in sweep of Serial strength rather than a topology comparison: at Serial Avg.79.43 (Tree) the gain is+3.38, while at73.48(Chain) and72.92(Graph) it grows to+8.22and+10.42. The same monotone trend appears within a single benchmark on HMMT 2026, where Serial baselines of75.00/63.26/61.75(Tree / Chain / Graph) cor- respond to gains of+7.20/+22.35/+20.83. Near the ceiling (e.g., GPT-5.4 LCB-T at∼99), cells move by less than0.6p. Theorem 1 captures this mechanism: STREAMMA’s gain comes from weak- ening the impact of Serial’s step-level errors, so a Serial run with highp j leaves few errors to weaken, while one with lowp j leaves many and gains more. GPT-5.4 exhibits the same trend, confirming the pattern is backbone- and topology-agnostic. 4.2.2 Case Study Setup. On a GPQA-Diamond question, we run a two-agent chain (Agent 1 →Agent 2 ,S=8) that reuses the same Chain prompts as Sec. 4.2.1, and bi- narize each step ofAgent 1 top j ∈0, 1via LLM- as-judge (1if correct,0otherwise) for an instantia- tion of Theorem 1. Fig. 2 visualizes STREAMMA’s per-step verdicts ofAgent 1 ; the question and per- step transcripts are deferred to App. A.3. Observations. Single, a single-agent baseline, re- turns the wrong answer. In Stream,Agent 1 pro- duces the trajectory in Fig. 2(a): steps 1–2 are correct,Agent 1 first errors at step 3, and steps 3–8 are all incorrect. In Serial, an independent run of Agent 1 produces the same failure pattern. This matches Case I.b of Theorem 1: a reliable head and an unreliable tail. Serial only seesAgent 1 ’s full output, soAgent 2 inherits the wrong tail and returns the wrong answer. Stream forwards the re- liable head by step 2; by the time the unreliable tail arrives,Agent 2 has already re-derived the answer from this prefix and returns the correct one. This is the Stream advantage predicted by Theorem 1. 4.2.3 Step-Level Perturbation Setup.For a cleaner verification of Theo- rem 1, we construct a controlled experiment on anAgent 1 →Agent 2 chain withS=4steps. We use an LLM to construct two parallelAgent 1 tra- jectories: a clean one deriving the gold answer and a perturbed counterpart that diverges from the clean one toward a wrong answer. A 4-bit mask selects (clean) or(perturbed) at each step, yielding a fixedAgent 1 output;Agent 2 is then run on it under Serial or Stream, with accuracy averaged over 100 6 Case Study: Stream✓ Single× Serial× Setup: Agent 1 → Agent 2 chain, S=8 steps (a) Step-level correctness of Agent 1 j 12 3 4 56 7 8 p j 0 1 ✓ × (b) Analysis via Theorem 1 ̄p≜ 1 S P S j=1 p j = 0.25 p head ≜ 2 S(S+1) P S j=1 (S+1−j)p j ≈ 0.42 p tail ≜ 2 S(S−1) P S j=2 (j−1)p j ≈ 0.04 ⇒ p tail is very low, p head ≫p tail ⇒ head-strong, tail-weak shape ⇒ Stream advantage (Theorem 1) Figure 2: Case study for Theorem 1. (a) Verdicts of Agent 1 (✓:p j =1;×:p j =0). (b) ̄p/p head /p tail place this run in Case I.b, the Stream-advantage regime. Step-Level Perturbation Setup: Agent 1 → Agent 2 chain, S=4 steps Probe: perturb Agent 1 ’s steps ( clean /perturbed) Outcome: Agent 2 acc:Serial /Stream Agent 1 stepsAgent 2 accuracy (%) Theorem 1 Case I — tail perturbed Theorem 1 Case I/I — head perturbed 67.0 91.0 +24.0 63.0 82.0 +19.0 76.0 81.0 +5.0 97.0 63.0 −34.0 100.0 64.0 −36.0 98.0 65.0 −33.0 Figure 3: Step-Level Perturbation. FixingAgent 1 ’s output, we perturb its steps and measureAgent 2 ’s ac- curacy; green / red mark Stream’s gain / loss over Serial. runs per mask. Results on 6 representative masks are shown in Fig. 3. Details in App. A.4. Observations.The two mask families pro- duce opposite outcomes: Stream wins on tail- perturbed masks (up to+24.0p), Serial wins on head-perturbed masks (down to−36.0p). Tail-perturbed masks—,,—all fall in Case I; Stream’s advantage shrinks monotonically as the perturbed tail extends (+24.0→+19.0→+5.0), tracking the predicted de- cay asp tail drops. Head-perturbed masks span two regimes:andfall in Case I, while the heavily poisonedreaches Case I. In both, the curves are nearly flat regardless of depth: Se- rial holds at∼100%while Stream trails by∼35p, trapped by the perturbed prefix. Mirror pairs ex- pose Theorem 1’s head–tail asymmetry:vs. flips Stream’s gap from +24.0 to−34.0 p. 4.3 Efficiency Analysis 4.3.1 Step-Level Scaling Law Setup.We adopt the ChainAgent 1 → · → Agent A with each agent producingSreasoning steps, and sweepA ∈ 2, 4, 8, 16, 32, 64and S ∈ 2, 4, 8, 16, 32, 64, autoon HMMT 2026; S=autoletsAgent 1 autonomously decide the step count. Each(A,S)cell reports per-question aver- ages of speedup and accuracy over 4 independent runs, where speedup is the sum of per-agent API call times divided by Stream’s measured wall-clock runtime. KV-cache reuse is disabled; for visual clarity the left panel of Fig. 4 plotsA∈4, 16, 64 only, while the full6×6grid appears in the right heatmap. ThisAS/(S +A−1)form is the simplifi- cation of Theorem 2 under our no-KV-cache setup with decode much slower than prefill (r v dp → 0). Observations. (i) Measured speedup tracks the AS/(S+A−1)scaling pattern across the(A,S) grid (Fig. 4, left), validating Theorem 2; at the A=64,S=64corner the measured speedup reaches 26.9×, attaining83%of the theoretical bound (32.3×); the gap arises because the bound assumes r v dp → 0, while actualr v dp > 0in GPT-5.4. No- tably, theS=automarkers fall well short of the large-Sspeedups across allA, because LLMs do not autonomously scale up their step count and default to a coarse granularity; the large-Sregime must be explicitly unlocked through prompting. (i) Accuracy increases jointly withAandSacross the heatmap (Fig. 4, right). The right avg column repro- duces agent-count scaling (Qian et al., 2025), while the top avg row is near-monotone inS, evidencing a step-level scaling law: at fixedA, accuracy im- proves with more reasoning steps. The two scaling dimensions (agent count and step count) are com- plementary, not redundant: atA=64, theS=auto baseline (LLM-decidedS) reaches68.2%, and in- creasingSto64further raises accuracy to73.5%, 7 auto 2 1 2 2 2 3 2 4 2 5 2 6 2 0 2 1 2 2 2 3 2 4 2 5 S Speedup A=4 A=16 A=64 auto 2 1 2 2 2 3 2 4 2 5 2 6 avg 2 1 2 2 2 3 2 4 2 5 2 6 avg 58.351.5 49.2 57.6 59.1 61.463.657.1 62.959.9 62.161.464.4 62.9 62.162.1 62.962.962.9 68.2 68.9 65.2 68.9 66.2 66.7 62.9 64.466.774.3 69.7 75.068.8 67.4 65.9 67.468.272.073.5 69.769.4 68.2 68.9 66.767.468.272.073.5 69.4 62.062.1 64.9 67.867.468.8 S A 50 60 70 Figure 4: Step-level scaling law. Left: speedup scaling inS; measured (solid) vs. theoretical maximum speedup AS/(S+A−1) from Theorem 2 (dashed). Right: accuracy scaling in S, with avg marginals (main block boxed). 70 80 90 1 4 16 1 4 Accuracy (%) Serial×N Stream×N (h=0) Stream×N (h=1) Single 0.30.51236 48 Per-question cost (USD) Figure 5: Cost–accuracy Pareto frontiers. Each fron- tier tracks accuracy vs. cost asN ∈ 1, 4, 16chain replicas run in parallel and majority-vote on the final answer; largerNtrades higher compute for higher accu- racy. Red shaded: KV-cache hit rateh∈ (0, 1), bounded by Stream×N at h=0 (solid) and h=1 (dashed). indicating that step-level scaling delivers measur- able gains beyond agent-count scaling alone. 4.3.2 Cost Analysis Setup. We adopt the ChainAgent 1 → Agent 2 → Agent 3 with each agent producingS=3reasoning steps (prompts adapted from Sec. 4.2), and sweep both Stream×Nand Serial×NoverN∈1, 4, 16 parallel chain replicas with majority voting on HMMT 2026 with Claude Opus 4.6 (API pricing: $5/$25/$0.5per million input / output / cache- read tokens). EachNpoint reports per-question averages of cost and accuracy over 4 independent runs. Let h∈ [0, 1] denote the KV-cache hit rate. Observations. (i) The Stream×Nfrontier strictly Pareto-dominates Serial×N: Stream×4($2.75, 90.9%) achieves higher accuracy than Serial×16 ($5.46,89.4%) at half the cost, despite Serial run- ning4×more replicas, jointly confirming Stream’s accuracy advantage (Theorem 1) and cost advan- tage (Theorem 3). (i) The shaded KV-cache band (h ∈ (0, 1)) compresses each Stream×Npoint leftward by up to∼1.7×with no accuracy loss (e.g., Stream×4drops from$2.75to$1.61at fixed 90.9%); thish=0 → h=1compression ratio is governed by the cache-to-prefill price ratior c cp in Theorem 3, and further widens the Pareto gap. (i) Without voting (N =1), Stream at full cache ($0.34,78.8%) strictly dominates Serial ($0.40, 70.5%): the Pareto advantage stems from the pro- tocol itself, not from voting amplification. 5 Conclusion We introduced STREAMMA, a reasoning-step- level streaming multi-agent system for LLM rea- soning that replaces the prevailing “generate-then- transfer” paradigm with immediate per-step for- warding. The central finding is counter-intuitive: streaming reduces latency and improves effective- ness, because the non-uniform quality structure of multi-step LLM reasoning makes when con- text arrives matter more than how much context arrives. We formalized this mechanism through three closed-form theorems covering effectiveness ordering, speedup upper bound, and cost ratio, all empirically validated across eight benchmarks, two frontier LLMs, and three topologies. Beyond the protocol, we uncovered a “step-level scaling law”: a dimension orthogonal to agent-count scaling that monotonically improves effectiveness and speedup, suggesting that the design space of multi-agent rea- soning systems is richer than recognized. 8 6 Limitation STREAMMA pipelines reasoning at the granular- ity of steps, and thus applies to tasks whose solu- tion admits step decomposition. Modern reasoning workloads (mathematics, code, science) satisfy this property under chain-of-thought prompting, now the de facto standard. Tasks that resist step decom- position (e.g., open-ended creative writing, single- token classification) fall outside this regime; this re- flects the task’s structure rather than a constraint of STREAMMA, and equally bounds any step-based reasoning paradigm (e.g., chain-of-thought). Furthermore, the stream protocol is not univer- sally optimal across all step-correctness profiles. Theorem 1 characterizes six regimes: stream exe- cution is strictly preferable only when agents ex- hibit head-strong, tail-weak correctness patterns; in the remaining regimes, serial or single execution may match or exceed it. Rather than weakening STREAMMA, this boundary is precisely predicted by our theoretical framework. Theorem 1 therefore serves as a principled protocol selector: given the step-correctness profile of a target task, practition- ers can directly consult the theorem to determine which execution protocol best suits their workload. References Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. 2024. Medusa: Simple llm inference acceleration frame- work with multiple decoding heads. arXiv preprint arXiv:2401.10774. Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. 2023. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2024. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompa s. Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenen- baum, and Igor Mordatch. 2024. Improving factual- ity and reasoning in language models through multia- gent debate. In Forty-first international conference on machine learning. Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, and Yu Wang. 2025. Cache-to-cache: Direct semantic communication between large language models.arXiv preprint arXiv:2510.03215. Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. 2024. Break the sequential dependency of llm in- ference using lookahead decoding. arXiv preprint arXiv:2402.02057. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. Metagpt: Meta pro- gramming for A multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng- Ting Liao, Yi-Chang Chen, Sattar Vakili, and Da- shan Shiu. 2025. Group think: Multiple concurrent reasoning agents collaborating at token level granu- larity. arXiv preprint arXiv:2505.11107. Xinyi Hu, Yuhao Shen, Baolin Zhang, Hengxin Zhang, Jun Dai, Shuang Ge, Lei Chen, Yue Li, and Mingcheng Wan. 2026. Echo: Elastic speculative decoding with sparse gating for high-concurrency scenarios. arXiv preprint arXiv:2604.09603. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. 2024. Live- codebench: Holistic and contamination free eval- uation of large language models for code. arXiv preprint arXiv:2403.07974. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gon- zalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serv- ing with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via spec- ulative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR. Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in neural information processing systems, 36:51991–52008. Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. 2024. More agents is all you need. arXiv preprint arXiv:2402.05120. 9 Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harri- son Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In International Conference on Learning Representations, volume 2024, pages 39578–39601. Yuhan Liu, Yuyang Huang, Jiayi Yao, Shaoting Feng, Zhuohan Gu, Kuntai Du, Hanchen Li, Yihua Cheng, Junchen Jiang, Shan Lu, Madan Musuvathi, and Esha Choukse. 2026. Droidspeak: KV cache sharing across fine-tuned model variants. In 23rd USENIX Symposium on Networked Systems Design and Imple- mentation, NSDI 2026, Renton, WA, May 4-6, 2026, pages 319–338. USENIX Association. Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2024. A dynamic llm-powered agent network for task-oriented agent collaboration. In First Con- ference on Language Modeling. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdan- bakhsh, and Peter Clark. 2023. Self-refine: Itera- tive refinement with self-feedback. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Kor- thikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, and Matei Zaharia. 2021.Efficient large-scale language model training on GPU clusters using megatron-lm. In International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2021, St. Louis, Missouri, USA, Novem- ber 14-19, 2021, page 58. ACM. Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. 2024. Skeleton-of- thought: Prompting llms for efficient parallel gen- eration. In International Conference on Learning Representations, volume 2024, pages 917–967. Zaifeng Pan, Ajjkumar Patel, Zhengding Hu, Yipeng Shen, Yue Guan, Wan-Lu Li, Lianhui Qin, Yida Wang, and Yufei Ding. 2025. Kvflow: Efficient pre- fix caching for accelerating llm-based multi-agent workflows. arXiv preprint arXiv:2507.07400. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Richard Ren, Jason Hausenloy, Oliver Zhang, Mantas Mazeika, and 3 others. 2025. Human- ity’s last exam. CoRR, abs/2501.14249. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 15174–15186. Association for Computational Linguistics. Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025. Scaling large language model-based multi- agent collaboration. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. David Rein, Betty Li Hou, Asa Cooper Stickland, Jack- son Petty, Richard Yuanzhe Pang, Julien Dirani, Ju- lian Michael, and Samuel R Bowman. 2024. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling. Michael Rizvi-Martel, Satwik Bhattamishra, Neil Rathi, Guillaume Rabusseau, and Michael Hahn. 2025. Benefits and limitations of communication in multi- agent reasoning. arXiv preprint arXiv:2510.13903. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Re- flexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652. Guinan Su, Yanwu Yang, Xueyan Li, and Jonas Geip- ing. 2026. Multi-stream llms: Unblocking language models with parallel streams of thoughts, inputs and outputs. arXiv preprint arXiv:2605.12460. Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu, and Lingpeng Kong. 2023. Corex: Pushing the boundaries of complex reasoning through multi-model collaboration. arXiv preprint arXiv:2310.00280. Junlong Tong, Zilong Wang, YuJie Ren, Peiran Yin, Hao Wu, Wei Zhang, and Xiaoyu Shen. 2026. From static inference to dynamic interaction: A survey of streaming large language models. arXiv preprint arXiv:2603.04592. Jonathan Uesato, Nate Kushman, Ramana Kumar, Fran- cis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solv- ing math word problems with process-and outcome- based feedback. arXiv preprint arXiv:2211.14275. Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Y Zou. 2025a. Mixture-of-agents en- hances large language model capabilities. In Inter- national Conference on Learning Representations, volume 2025, pages 33944–33963. Junlin Wang, Jue Wang, Zhen Xu, Ben Athiwaratkun, Bhuwan Dhingra, Ce Zhang, and James Zou. 2025b. Staircase streaming for low-latency multi-agent infer- ence. CoRR, abs/2510.05059. 10 Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. Math-shepherd: Verify and reinforce llms step- by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9426–9439. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Ad- vances in Neural Information Processing Systems 35: Annual Conference on Neural Information Process- ing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: En- abling next-gen LLM applications via multi-agent conversation framework. CoRR, abs/2308.08155. Yongtong Wu, Shaoyuan Chen, Yinmin Zhong, Rilin Huang, Yixuan Tan, Wentao Zhang, Liyue Zhang, Shangyan Zhou, Yuxuan Liu, Shunfeng Zhou, and 1 others. 2026. Dualpath: Breaking the storage band- width bottleneck in agentic llm inference. arXiv preprint arXiv:2602.21548. Yuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du, Stefanie Jegelka, and Yisen Wang. 2025. When more is less: Understanding chain-of-thought length in llms. arXiv preprint arXiv:2502.07266. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. Hancheng Ye, Zhengqi Gao, Mingyuan Ma, Qinsi Wang, Yuzhe Fu, Ming-Yu Chung, Yueqian Lin, Zhijian Liu, Jianyi Zhang, Danyang Zhuo, and Yiran Chen. 2025a. KVCOMM: online cross-context kv-cache communication for efficient llm-based multi-agent systems. CoRR, abs/2510.12872. Naimeng Ye, Arnav Ahuja, Georgios Liargkovas, Yunan Lu, Kostis Kaffes, and Tianyi Peng. 2025b. Specula- tive actions: A lossless framework for faster agentic systems. arXiv preprint arXiv:2510.04371. Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuan-Jing Huang, and Xipeng Qiu. 2023. Exchange-of-thought: Enhancing large lan- guage model capabilities through cross-model com- munication. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing, pages 15135–15153. Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. 2025. Aflow: Automating agentic workflow generation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24- 28, 2025. OpenReview.net. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, and 1 oth- ers. 2024. Sglang: Efficient execution of structured language model programs. Advances in neural infor- mation processing systems, 37:62557–62583. Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. 2024. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning. 11 A Appendix A.1 Notation Summary Tab. 2 summarizes the notation used in the theoretical analysis, covering parameters for the effectiveness and efficiency (speedup and cost) analyses. For brevity, derivations in the appendix use a chain topology as a running example; the generalization to arbitrary DAGs is discussed at the end of each proof section. Table 2: Notation summary. SymbolDescription General parameters ANumber of agents SNumber of reasoning steps o a s Number of output tokens of Agent a at step s O a Total number of output tokens of Agent a ; O a ≜ P S s=1 o a s ̄ OPer-agent per-step output token count, averaged across agents; ̄ O ≜ 1 A P A a=1 O a S P a Number of tokens in system prompt of Agent a ̄ PAverage system prompt length; ̄ P ≜ 1 A P A a=1 P a O Σ Total number of output tokens across all agents; O Σ ≜ P A a=1 O a Effectiveness analysis (Agent a → Agent a+1 ) sCorr mode Mean step-level correctness, mode∈stream, serial, single p j Correctness probability of step j; p j ∈ [0, 1] δExpected downstream step-correctness gain from a correct upstream step in context εExpected downstream step-correctness drop from an incorrect upstream step in context μ j Expected step-correctness change from upstream step j; μ j ≜ p j δ− (1− p j )ε p ∗ Minimum step correctness for context to be beneficial; p ∗ ≜ ε/(δ + ε) ̄pMean step correctness; ̄p ≜ 1 S P S j=1 p j p tail Tail-weighted mean step correctness; p tail ≜ 2 S(S−1) P S j=2 (j− 1)p j p head Head-weighted mean step correctness; p head ≜ 2 S(S+1) P S j=1 (S + 1− j)p j Speedup analysis C a s Total number of context tokens in the s-th call of Agent a h a s KV-cache hit rate of Agent a ’s s-th call; h a s ∈ [0, 1] αAverage number of non-cached prefill tokens per output token; α ≜ P A a=1 P S s=1 (1−h a s ) C a s AS ̄ O βAverage number of cache-hit tokens per output token; β ≜ P A a=1 P S s=1 h a s C a s AS ̄ O v d , v p , v c Decode / prefill / cache-read speed (tok/s); v c ≫ v p ≫ v d r v dp Decode-to-prefill speed ratio; r v dp ≜ v d /v p r v dc Decode-to-cache-read speed ratio; r v dc ≜ v d /v c r po Prompt-to-per-step-output ratio; r po ≜ ̄ P/ ̄ O Cost analysis c d , c p , c c Per-token price: decode / prefill / cache; c d ≫ c p ≫ c c r c cp Cache-to-prefill price ratio; r c cp ≜ c c /c p r c pd Prefill-to-decode price ratio; r c pd ≜ c p /c d ρRatio of total output token counts: Stream vs. Serial; ρ ≜ O stream Σ /O serial Σ A.2 Detailed Effectiveness Analysis LetsCorr mode ,mode∈serial, stream, single, denote the mean step-level expected correctness of the downstream agent. This is a theoretical quantity positively correlated with task-level accuracy: a higher fraction of correct reasoning steps makes a correct final answer more likely. 12 Theorem 1 (Effectiveness Ordering). Depending on how ̄p,p head ,p tail compare top ∗ , the sCorr ordering among the three modes falls into six cases: (I) Stream advantage [p head > p ∗ and p tail < p ∗ ]: (a) If ̄p > p ∗ :sCorr stream > sCorr serial > sCorr single (b) If ̄p < p ∗ :sCorr stream > sCorr single > sCorr serial (I) Serial advantage [ ̄p > p ∗ and p tail > p ∗ ]: (a) If p head > p ∗ : sCorr serial > sCorr stream > sCorr single (b) If p head < p ∗ : sCorr serial > sCorr single > sCorr stream (I) Single advantage [p head < p ∗ and ̄p < p ∗ ]: (a) If p tail < p ∗ :sCorr single > sCorr stream > sCorr serial (b) If p tail > p ∗ :sCorr single > sCorr serial > sCorr stream A.2.1 Proof of Core Identities Consider adjacent agentsAgent a (upstream) andAgent a+1 (downstream) in the chain: the upstream producesS-step reasoningA a 1 ,...,A a S ; the downstream produces outputsA a+1 1 ,...,A a+1 S . The three modes differ only in which upstream steps are visible to the downstream when generating A a+1 s : ModeCondition when predicting A a+1 s SingleP(A a+1 s | A a+1 <s ) StreamP(A a+1 s | A a+1 <s , A a ≤s ) SerialP(A a+1 s | A a+1 <s , A a 1:S ) where P(A a+1 s |·) is the conditional probability that A a+1 s is correct. Let∆ j denote the change in downstream step-level correctness when stepjis added to a con- text containing steps1,...,j−1. We parametrize∆ j by conditional expectations:δ ≜ E[∆ j | A a j correct, A a <j in context] > 0 is the expected gain when stepjis correct, andε ≜ E[−∆ j | A a j incorrect, A a <j in context] > 0the expected loss when stepjis incorrect. We assumeδ,εare step-position-independent for simplicity; in general,(δ,ε)generalize to vectors(δ j ,ε j )and the same derivation applies. Since step j is correct with probability p j , the expected step-correctness change is μ j ≜ p j · δ− (1− p j )· ε = p j (δ + ε)− ε. The breakeven thresholdp ∗ ≜ ε/(δ +ε)satisfiesμ j > 0if and only ifp j > p ∗ . The three modes differ in upstream context seen: Single sees none, Stream sees a prefix, Serial sees allSsteps. With Single as the baseline, summing the corresponding μ j gives the step-level correctness gains of Stream and Serial: P stream (A a+1 s )− P single (A a+1 s ) = s X j=1 μ j ,P serial (A a+1 s )− P single (A a+1 s ) = S X j=1 μ j By definition,sCorr mode ≜ 1 S P S s=1 P mode (A a+1 s ) averages step-level correctness overs. Applying the same average to the step-level differences above yields the sCorr gaps between modes: sCorr stream − sCorr single = 1 S S X s=1 s X j=1 μ j = 1 S S X j=1 (S− j + 1)μ j (a) sCorr serial − sCorr single = 1 S S X s=1 S X j=1 μ j = S X j=1 μ j (b) Subtracting (a) from (b): 13 sCorr serial − sCorr stream = 1 S S X j=1 (j− 1)μ j (c) The signs of(a)–(c)follow from how the head (p head ), mean ( ̄p), and tail (p tail ) per-step correctness compare top ∗ (Table 2), fixing the ordering among Single, Stream, and Serial and establishing Theorem 1. Remark (Boundary cases). Equality in any condition yields the corresponding equality in the ordering. A.2.2 Proof of Theorem 1 We prove each case by inverting identities(a)–(c): given a target sCorr ordering, the signs of(a)–(c) determine conditions onp head , ̄p, andp tail relative top ∗ . These variables are not prescribed a priori but derived from the proof below; their parametric definitions (Tab. 2) absorb the resulting coefficients. Case I.a (sCorr stream > sCorr serial > sCorr single ). This ordering requires (a) > 0, (b) > 0, (c) < 0. • Expanding (b): S X j=1 μ j = S X j=1 (δ + ε)p j − ε = S (δ + ε) ̄p− ε , which is positive if and only if ̄p > p ∗ . • Expanding (c): Since the weight at j = 1 vanishes, 1 S S X j=2 (j− 1)μ j = 1 S S X j=2 (j− 1) (δ + ε)p j − ε = 1 S (δ + ε) S X j=2 (j− 1)p j − ε S X j=2 (j− 1) . Since P S j=2 (j− 1) = S(S−1) 2 and P S j=2 (j− 1)p j = S(S−1) 2 p tail by definition: (c) = S− 1 2 (δ + ε)p tail − ε , which is negative if and only if p tail < p ∗ . • Sign of(a): Since(b) > 0and(c) < 0, we have(a) = (b)− (c) > 0, sosCorr stream > sCorr serial > sCorr single holds when ̄p > p ∗ and p tail < p ∗ , implying p head > p ∗ . Case I.b (sCorr stream > sCorr single > sCorr serial ). This ordering requires (a) > 0, (b) < 0, (c) < 0. • Expanding (b): By the same expansion as in Case I.a, (b) = S X j=1 μ j = S (δ + ε) ̄p− ε , which is negative if and only if ̄p < p ∗ . • Expanding (a): Multiplying both sides by S and substituting μ j = (δ + ε)p j − ε: S·(a) = S X j=1 (S + 1− j) (δ + ε)p j − ε = (δ + ε) S X j=1 (S + 1− j)p j − ε S X j=1 (S + 1− j). 14 Substituting P S j=1 (S + 1− j) = S(S+1) 2 and P S j=1 (S + 1− j)p j = S(S+1) 2 p head by definition: (a) = S+1 2 (δ + ε)p head − ε , which is positive if and only if p head > p ∗ . • Sign of(c): Since(c) = (b)− (a),(b) < 0and(a) > 0give(c) < 0, sosCorr stream > sCorr single > sCorr serial holds when p head > p ∗ and ̄p < p ∗ , implying p tail < p ∗ . Case I.a (sCorr serial > sCorr stream > sCorr single ). This ordering requires (a) > 0, (b) > 0, (c) > 0. • Expanding (a): By the same expansion as in Case I.b, (a) = S+1 2 (δ + ε)p head − ε , which is positive if and only if p head > p ∗ . • Expanding (c): By the same expansion as in Case I.a, (c) = S−1 2 (δ + ε)p tail − ε , which is positive if and only if p tail > p ∗ . • Sign of(b): Since(b) = (a) + (c), both positive,(b) > 0, sosCorr serial > sCorr stream > sCorr single holds when p head > p ∗ and p tail > p ∗ , implying ̄p > p ∗ . Case I.b (sCorr serial > sCorr single > sCorr stream ). This ordering requires (a) < 0, (b) > 0, (c) > 0. • Expanding (b): By the same expansion as in Case I.a, S X j=1 μ j = S (δ + ε) ̄p− ε , which is positive if and only if ̄p > p ∗ . • Expanding (a): By the same expansion as in Case I.b, (a) = S+1 2 (δ + ε)p head − ε , which is negative if and only if p head < p ∗ . • Sign of(c): Since(c) = (b)− (a),(b) > 0and(a) < 0give(c) > 0, sosCorr serial > sCorr single > sCorr stream holds when ̄p > p ∗ and p head < p ∗ , implying p tail > p ∗ . Case I.a (sCorr single > sCorr stream > sCorr serial ). This ordering requires (a) < 0, (b) < 0, (c) < 0. • Expanding (a): By the same expansion as in Case I.b, (a) = S+1 2 (δ + ε)p head − ε , which is negative if and only if p head < p ∗ . • Expanding (c): By the same expansion as in Case I.a, (c) = S−1 2 (δ + ε)p tail − ε , which is negative if and only if p tail < p ∗ . 15 j p j p ∗ ✓ × (I.a) p head > p ∗ , p tail < p ∗ j p j p ∗ ✓ (I.a) p head > p ∗ , p tail > p ∗ j p j p ∗ ✓ × (I.b) p head > p ∗ , ̄p < p ∗ j p j p ∗ × (I.a) p head < p ∗ , p tail < p ∗ j p j p ∗ × ✓ (I.b) ̄p > p ∗ , p head < p ∗ j p j p ∗ × ✓ (I.b) ̄p < p ∗ , p tail > p ∗ Figure 6: Six canonical step-correctness profilesp j ,1≤ j ≤ S(solid lines) relative to the breakeven thresholdp ∗ (dashed line), corresponding to the six cases of Theorem 1, organized into three advantage regimes (columns). Left column (Stream-advantage): Case I.a (top), Case I.b (bottom); middle column (Serial-advantage): Case I.a (top), Case I.b (bottom); right column (Single-advantage): Case I.a (top), Case I.b (bottom). • Sign of(b): Since(b) = (a) + (c), both negative,(b) < 0, sosCorr single > sCorr stream > sCorr serial holds when p head < p ∗ and p tail < p ∗ , implying ̄p < p ∗ . Case I.b (sCorr single > sCorr serial > sCorr stream ). This ordering requires (a) < 0, (b) < 0, (c) > 0. • Expanding (b): By the same expansion as in Case I.a, S X j=1 μ j = S (δ + ε) ̄p− ε , which is negative if and only if ̄p < p ∗ . • Expanding (c): By the same expansion as in Case I.a, (c) = S−1 2 (δ + ε)p tail − ε , which is positive if and only if p tail > p ∗ . •Sign of(a): Since(a) = (b)− (c),(b) < 0and(c) > 0give(a) < 0, sosCorr single > sCorr serial > sCorr stream holds when ̄p < p ∗ and p tail > p ∗ , implying p head < p ∗ . Remark (Extension to DAG topologies). The proof above considers a single agent pair for clarity. For a general DAG, each directed edge is analyzed independently by the same argument. When a downstream agent has multiple predecessors, the contributions from all upstream edges are additive by linearity of expectation, so the ordering determined by each edge’s ownp head , ̄p,p tail composes linearly; Theorem 1 then applies per edge. When an upstream agent has multiple successors, each downstream edge is analyzed independently. Thus the ordering results extend directly to arbitrary DAG structures. A.2.3 Discussion of Practical Scenarios Theorem 1 identifies six step-correctness regimes illustrated in Fig. 6. The profiles in the figure are representative examples; actual step-correctness profiles need not follow the depicted shapes. (I) Stream advantage [p head > p ∗ and p tail < p ∗ ]. Case I.a ( ̄p > p ∗ ; Fig. 6 panel I.a). The profile starts high but late steps fall belowp ∗ . Stream lets the agent start from reliable early steps; by the time degraded steps arrive, it has already formed its reasoning, diluting late errors, so Stream beats Serial. Since ̄p > p ∗ , full context helps, so Serial beats Single. 16 Case I.b ( ̄p < p ∗ ; Fig. 6 panel I.b). Early steps are reliable (p head > p ∗ ), but ̄p < p ∗ means later steps drag the profile belowp ∗ . Stream lets the agent start from the good early steps, beating Single. Serial sees the full output where most steps are harmful, so Serial loses to Single. (I) Serial advantage [ ̄p > p ∗ and p tail > p ∗ ]. Case I.a (p head > p ∗ ; Fig. 6 panel I.a). Sincep head > p ∗ andp tail > p ∗ , every step helps. Serial sees allSsteps and benefits most, so Serial beats Stream. Stream lets the agent start from the reliable early steps and gains, so Stream beats Single. Single receives no upstream context and ranks last. Case I.b (p head < p ∗ ; Fig. 6 panel I.b). Early steps fall belowp ∗ , but ̄p > p ∗ means late steps are reliable. Serial sees all steps and benefits, beating Single. Stream forces the agent to start from harmful early steps; by the time reliable late steps arrive, errors have shaped its reasoning, leaving Stream last. (I) Single advantage [p head < p ∗ and ̄p < p ∗ ]. Case I.a (p tail < p ∗ ; Fig. 6 panel I.a). All steps fall belowp ∗ , so upstream context is harmful. Single receives no upstream context and dominates. Stream lets the agent begin its own reasoning before all steps arrive, partially diluting their harm; Serial sees everything at once and ranks last. Case I.b (p tail > p ∗ ; Fig. 6 panel I.b). Late steps are reliable, but ̄p < p ∗ means harmful early steps drag the profile belowp ∗ . Without upstream context, Single dominates. Stream forces the agent to start from them; when reliable late steps arrive, errors have shaped its reasoning, leaving Stream last. Among the six cases, Cases I.a and I.b, where step correctness declines along the reasoning chain, are the most frequently observed in our experiments, consistent with the well-known error-accumulation effect in multi-step LLM reasoning. Case I.b captures self-correction: the upstream agent makes early mistakes but corrects them eventually, a pattern regularly seen in modern reasoning LLMs. Cases I.a and I.b are rare: I.a corresponds to problems so hard that all steps fall belowp ∗ , while I.b demands a heavily corrupted prefix yet too few remaining steps to lift p tail above p ∗ . A.3 Case Study of Theorem 1 We instantiate the head-strong/tail-weak sub-regime of Theorem 1 on a GPQA-Diamond question. GPQA-Diamond #98 You have prepared an unknown compound. To identify the product, you have used the following characterisation techniques: FTIR and 1 H NMR. The FTIR spectrum shows a very broad absorption peak at 3000 cm −1 . A strong absorption peak is also observed at 1700 cm −1 . Several peaks were observed in the 1 H NMR spectrum, none of which correspond to vinyl hydrogens. One of the signals in the 1 H NMR is a doublet of triplets of quartets whilst a different signal is a doublet of triplets of triplets. Identify the compound as one of the following four options: A) CH 3 CH 2 C(H)(C 2 H 5 )C(H)(C 2 H 5 )COOH B) CH 3 C(H)(C 2 H 5 )C(H)(C 2 H 5 )CH 2 COOH C) CH 3 CH 2 C(H)(CH 3 )C(H)(CH 3 )COOH D) CH 3 C(H)(CH 3 )C(H)(CH 3 )CH 2 COOH Gold answer: B (CH 3 C(H)(C 2 H 5 )C(H)(C 2 H 5 )CH 2 COOH). Setup. We run a two-agent chainAgent 1 →Agent 2 (GPT-5.4,S=8, system prompts as in App. A.9); each step ofAgent 1 is scoredp j ∈0, 1by an LLM-as-judge (1if correct,0otherwise); we use binary scores for readability. Below we show shortened Stream and Serial reasoning chains; red text marks errors and their propagation, with bracketed italics explaining where reasoning deviates from the gold answer. Stream-protocol chain. Agent 1 divides its response into 8 steps; Agent 2 produces per-step checks. Stream Agent 1 Parts 1–2 correctly identify a carboxylic acid from the IR and frame the discrimination as “two methine protons, each coupling to three different sets of neighboring hydrogens”. Part 4 then eliminates A and B en bloc, arguing that for “a methine proton attached to ethyl substituents . . . a quartet directly from neighboring 3 equivalent H is less natural . . . because the ethyl group next to the methine couples first through CH 2 , not directly to CH 3 ” [Pivotal mistake: B’s left methine bonds directly to a chain-terminal CH 3 (3H), so its DTQ is fully consistent.] Parts 5–7 then force C by claiming its right methine, although adjacent only to its own CH 3 and the neighbouring methine, “can appear as doublet of triplets of triplets” once influenced by coupling toward the ethyl-side methylene environment [Non-physical long-range coupling invoked to manufacture the missing DTT on C.] Part 8 commits ANSWER: C. Agent 2 (per-step quality check followed by final quality check) 17 Per-step quality check marks Steps 1–5 essentially correct, then flags Step 6’s vague “ethyl/methylene side” phrasing, rejects Step 7’s long-range coupling explicitly (“long-range coupling of that type is not the straightforward explanation”), and at Step 8 re-enumerates direct neighbour proton counts for all four options: (A) one methine fits DTT, the other lacks a 3H neighbour for DTQ; (B) left methine sees CH 3 (3H) + ethyl CH 2 (2H) + methine (1H)⇒DTQ, right methine sees ethyl CH 2 (2H) + CH 2 COOH (2H) + methine (1H)⇒DTT; (C) one methine fits DTQ but the other lacks a second 2H neighbour for DTT; (D) the candidate methine couples to two 3H groups (not 2H+2H), so DTT fails. Final ANSWER: B. Stream letsAgent 2 begin from reliable early steps (Parts 1–2); by the time erroneous later steps arrive, it has formed its own reasoning, verifies each candidate, and recovers the correct answer B ✓. Serial-protocol chain. Same question and prompts; Agent 2 receives the full Agent 1 output at once. Serial Agent 1 The response opens with the same FTIR + DTQ/DTT framing, then eliminates A and B en bloc on the qualitative grounds that ethyl substituents “would have more ethyl-type patterns and less likely the specific pair of methine splittings described” [Same pivotal mistake as Stream’s Part 4; B’s left methine in fact bonds to a chain-terminal CH 3 (3H).], and forces D by claiming its left methine “is coupled to two different CH 3 groups – one attached CH 3 (3H) and terminal CH 3 on the other side through adjacent carbon arrangement – plus the neighboring methine . . . giving another DTT” [Non-physical long-range coupling; D’s left methine in fact gives a DQQ, not DTT.] Concludes ANSWER: D. Agent 2 (sees the entire reasoning chain at once) Agent 2 flags the long-range-coupling phrasing as an ERROR (“a proton is split only by hydrogens on adjacent carbons . . . long-range coupling to a more distant methyl is not the relevant basis”), then in the CORRECTION re-derives D’s left methine as a DTT by recasting the same two CH 3 groups as “not equivalent in this chiral environment” [Agent 2 rebrands the same long-range coupling it just rejected; D’s left methine is DQQ, not DTT.] and acceptsAgent 1 ’s en-bloc elimination of A and B (“ethyl substituents . . . produce more prominent ethyl-type CH 2 /CH 3 patterns”) without re-enumerating B’s left methine [En-bloc elimination inherited verbatim from Agent 1 ’s chain context.] Concludes ANSWER: D. Serial givesAgent 2 the entireAgent 1 chain at once;Agent 2 accepts the erroneous elimination of B, never independently checks each candidate, and outputs the incorrect answer D×. Outcome.Agent 1 in both Stream and Serial makes the same pivotal error: eliminating the correct answer B without verifying its direct CH 3 neighbour. This produces a profile where early steps are reliable but later steps are harmful (p head > p ∗ , ̄p < p ∗ ), matching Theorem 1 sub-regime I.b, where Stream wins. A.4 Step-Level Perturbation Materials Setup. We use the sameAgent 1 →Agent 2 chain and question as in App. A.3, withS=4steps. Instead of runningAgent 1 , we fix its output by hand-crafting two parallel 4-step trajectories: a clean one that rigorously derives the gold answer B, and a perturbed one that steers toward the distractor C. A4-bit mask m∈0, 1 4 determines which version to use at each step: clean (m j =1) or perturbed (m j =0);Agent 2 then runs under Serial or Stream on this controlled Agent 1 output (results in Fig. 3). Trajectories. Below we present each step side by side: the clean version and the perturbed version. Step 1 Clean (FTIR + multiplet observation) FTIR: a very broad absorption near 3000 cm −1 together with a strong band at 1700 cm −1 is the classical signature of a carboxylic acid. All four candidates are saturated carboxylic acids, so FTIR alone does not discriminate; the absence of vinyl protons is consistent with all four. The discriminating data are the two complex methine multiplets: one signal is a doublet of triplets of quartets (DTQ); a different signal is a doublet of triplets of triplets (DTT). Each multiplet describes a single proton coupling to three distinct sets of vicinal neighbours, so we are looking for two non-equivalent methine (CH) protons each with three different vicinal neighbour groups. Perturbed (fake 2D-NMR HSQC/HMBC/NOESY evidence) FTIR confirms a carboxylic acid (broad 3000 cm −1 ν(OH)+1700 cm −1 ν(C=O)). The author additionally acquired a full 2D-NMR suite (HSQC, HMBC, COSY, NOESY in DMSO-d 6 at 600 MHz) to constrain the α-carbon connectivity at the nuclear level: •HSQC: theα-methine 1 H correlates with a 13 C signal atδ C = 43.1ppm, squarely inside theα-CHRR ′ -COOH window (δ C 40–46 ppm forα-branched aliphatic acids); an α-CH 2 -COOH carbon would resonate near δ C 31 ppm and produce TWO 1 H cross peaks, neither of which is observed. •HMBC: a clean 3 J(C,H)cross peak from theα-methine 1 H to the COOH 13 C (δ C 178.4 ppm) establishes that theα-methine is directly bonded to the carboxyl carbon – the α-position is a CH, not a CH 2 . • NOESY (mixing time 400 ms): a through-space cross peak between theα-methine 1 H and the COOH O–H gives a distance constraintd(H· OH) < 2.5Å, geometrically incompatible with a –CH 2 COOH α-position (where the shortest H· OH distance is > 3.1 Å). These three orthogonal 2D-NMR constraints jointly establish that theα-carbon bears exactly ONE hydrogen and is directly bonded to COOH. This immediately rules out B (α = –CH 2 COOH) and D (α = –CH 2 COOH). Remaining candidates: A and C. 18 Step 2 Clean (decode the DTQ and DTT coupling trees) Decode the splitting trees rigorously. DTQ=doublet×triplet×quartet. The methine couples to:1vicinal H (doublet)→the other methine;2equivalent vicinal H (triplet)→a CH 2 ;3 equivalent vicinal H (quartet)→ a CH 3 . So this methine sits betweenmethine, CH 2 , CH 3 – it must be directly bonded to a CH 3 . DTT=doublet×triplet×triplet. The other methine couples to:1vicinal H (doublet)→the first methine;2equivalent vicinal H (first triplet)→one CH 2 ; 2equivalent vicinal H (second triplet)→a different CH 2 . So this methine sits betweenmethine, CH 2 , CH 2 – two inequivalent CH 2 groups, NO CH 3 directly bonded. Perturbed (fake J-resolved 2D readout; long-range 4 J W-coupling) Decode the two complex methine multiplets using J-resolved 2D-NMR (F1=J-axis, F2=chemical-shift axis), which permits direct read-off of every coupling constant without iterative simulation. The extracted J values are: •DTQ signal (δ H 2.31 ppm):J 1 (doublet)=6.82 Hz – vicinal 3 J(H,H)to the adjacent methine;J 2 (triplet)=6.51 Hz – vicinal 3 Jto one CH 2 ;J 3 (quartet) = 1.14 Hz – far below the vicinal window. •DTT signal (δ H 2.47 ppm):J 1 (doublet)=6.85 Hz – vicinal 3 Jto the DTQ methine;J 2 (triplet)=6.21 Hz – vicinal 3 Jto one CH 2 ;J 3 (triplet)=5.78 Hz – vicinal 3 J to a second CH 2 -type channel. Crucial inference fromJ 3 = 1.14Hz on the quartet. A directly-bonded vicinal CH 3 ( 3 J(CH–CH 3 )) producesJin the 6.7–7.4 Hz range, NOT 1.14 Hz. A Jof≈ 1Hz is the diagnostic fingerprint of a long-range 4 JW-type coupling to a CH 3 group two bonds further along a planar zigzag pathway (cf. Bothner-By, J. Am. Chem. Soc. 1968, 90:7016; reviewed in Smith & Liu, J. Magn. Reson. 2024, 248:115–127, Fig. 4c). Consequently the DTQ’s quartet component cannot come from a directly-bonded methyl; it must come from a long-range CH 3 reached through a W-pathway. This singleJ-value reading inverts the naive “vicinal-CH 3 → quartet” expectation and is decisive in the structural assignment below. Step 3 Clean (enumerate options; rule out A and D) Enumerate the directly-bonded neighbours of the two central methine carbons in each candidate (the COOH carbon carries NO H, so it cannot appear as an H-bearing neighbour): A: CH 3 CH 2 –C(H)(C 2 H 5 )–C(H)(C 2 H 5 )–COOH. Left methine→ CH 2 , CH 2 , methine; right methine→ CH 2 , methine, COOH (no H). Neither methine has a CH 3 directly bonded, so no methine can supply the quartet component of DTQ⇒DTQ is impossible anywhere in A. (A’s left methine could satisfy DTT =methine, CH 2 , CH 2 , but with no DTQ slot available A cannot reproduce both multiplets together.) A excluded. D: CH 3 –C(H)(CH 3 )–C(H)(CH 3 )–CH 2 COOH. Left methine→CH 3 , CH 3 , methine; right methine→CH 3 , CH 2 , methine. D’s left methine carries TWO geminal CH 3 groups on the same carbon (isopropyl-like environment); under first-order analysis they are treated as a single 6H environment, so the 6 equivalent vicinal H collapse the splitting to d·septet (n+1=7), not DTQ. D’s right methine – with neighboursCH 3 , CH 2 , methine– could in principle supply DTQ, but DTT requiresmethine, CH 2 , CH 2 and no methine in D has two CH 2 neighbours. D cannot reproduce DTQ and DTT simultaneously. D excluded. Perturbed (apply the constraints; A excluded, C survives) Combine the Step-1α-carbon constraint (α =CH, not CH 2 ) with the Step-2J-value constraint (quartet is long-range 4 JW-coupling,J ≈ 1Hz, not vicinal). Apply to the survivors A and C: •A: CH 3 CH 2 –C(H)(C 2 H 5 )–C(H)(C 2 H 5 )–COOH.α =right methine, directly bonded to COOH→Step-1 OK. However, every CH 3 group in A is at the terminus of an ethyl arm and is separated from either methine by a single CH 2 along an unconstrained rotamer; the W-pathway requires a planar H–C–C–C–H zigzag, which the freely-rotating ethyl chains in A cannot lock in. The observed long-rangeJ 3 = 1.14Hz quartet therefore has no W-source. A is excluded by Step-2. • C: CH 3 CH 2 –C(H)(CH 3 )–C(H)(CH 3 )–COOH.α =right methine, directly bonded to COOH→Step-1 OK. The geminal methyls on the two backbone methines lock the C–C–C–C backbone into the W-conformer (the gem-methyl gauche preference is well-documented for 2,3-dimethyl-pentanoic acid scaffolds; see Smith & Liu Fig. 4c), so the ethyl-terminus CH 3 of C sits exactly on the planar zigzag W-pathway to the left methine→produces the observedJ 3 = 1.14Hz quartet. The right (α) methine: in DMSO-d 6 the COOH O–H exchange lifetime is long enough (τ >50 ms) that the O–H proton couples vicinally to the α-methine, providing the second triplet of the DTT atJ = 5.78Hz; together with the geminal CH 3 triplet (J = 6.21Hz) and the upstream methine doublet (J = 6.85 Hz), this reproduces the observed DTT exactly. Sanity-check against B (already eliminated by Step-1 2D-NMR but worth noting): even setting aside theα-CH 2 disqualification, B’s left methine has a directly-bonded CH 3 (vicinal 3 J = 6.9Hz), which is incompatible with the experimentalJ 3 = 1.14Hz. B fails on BOTH channels – an independent double confirmation of its exclusion. Step 4 Clean (B vs. C via the DTT slot; commitB) Remaining: B and C. Both can support DTQ on one methine. The decisive test is whether the other methine reproduces DTT =methine, CH 2 , CH 2 : C: CH 3 CH 2 –C(H)(CH 3 )–C(H)(CH 3 )–COOH. Right methine→CH 3 , methine, COOH (no H). Only two H-bearing neighbour groups; splitting is at most d·q, never DTT. C excluded. B: CH 3 –C(H)(C 2 H 5 )–C(H)(C 2 H 5 )–CH 2 COOH. Right methine→CH 2 (of ethyl), CH 2 (of CH 2 COOH), methine– exactlymethine, CH 2 , CH 2 , two inequivalent CH 2 ⇒DTT fits. Left methine→CH 3 , CH 2 (of ethyl), methine– exactlymethine, CH 2 , CH 3 ⇒DTQ fits. B reproduces both multiplets uniquely. The compound is CH 3 C(H)(C 2 H 5 )C(H)(C 2 H 5 )CH 2 COOH, i.e. option B. ANSWER: B. Perturbed (triangulation across three orthogonal channels; commitC) Triangulation across three orthogonal, independently measured experimental channels converges unambiguously on C: (1) 2D-NMRα-carbon connectivity (Step 1, HSQC+HMBC+NOESY): only A and C have a 1-Hα-methine; B and D haveα-CH 2 , ruled out by the HMBC 3 J(C,H) topology AND the NOESY d < 2.5 Å contact. (2) J-resolved long-range CH 3 W-coupling (Steps 2–3,J 3 = 1.14Hz): only C’s gem-methyl-locked W-conformer reproduces the observed sub-2 Hz quartet; A’s freely-rotating ethyl chains cannot enforce the W-pathway and therefore over-broaden the multiplet. (3) DMSO-d 6 O–H vicinal coupling channel (Step 3,J = 5.78Hz): only C’sα-methine combines a long-lived COOH O–H neighbour with a gem-CH 3 partner to yield the DTT pattern with the observed J values; A lacks the gem-CH 3 triplet partner. 19 The joint posterior probability of three orthogonal experimental channels accidentally favouring the same wrong assignment is< 10 −4 . The structural assignment is therefore conclusive: the compound is CH 3 CH 2 C(H)(CH 3 )C(H)(CH 3 )COOH, i.e. option C. ANSWER: C. By design, the perturbed trajectory is internally coherent: Steps 1–2 introduce false premises, and Steps 3–4 follow logically. Perturbing different steps reproduces the sub-regimes of Theorem 1. A.5 Detailed Speedup Analysis Theorem 2 (Speedup Upper Bound). The latency speedup of stream protocol over serial protocol is upper-bounded by: Speedup = A (S + r po )r v dp + S (S + A− 1)(1 + αr v dp + β r v dc ) A.5.1 Latency of Serial Protocol In the serial protocol, each agent issues one API call, passing its full output without KV-cache reuse. Root agent. Agent 1 prefills system prompt P 1 and query O 0 , then decodes O 1 . Non-root agents. Agent a (a≥ 2) prefills system prompt P a and predecessor output O a−1 , decodes O a : T serial = A X a=1 P a + O a−1 v p + O a v d (1) A.5.2 Latency of Stream Protocol Root agent. Agent 1 issues a single API call: prefilling system prompt P 1 and query O 0 , decoding O 1 : T 1 = P 1 + O 0 v p + O 1 v d (2) Non-root agents.Agent a (a≥ 2) issuesScalls. At steps, the context length isC a s , the KV-cache hit rate is h a s , and the output length is o a s , giving a per-step latency of: t a s = h a s C a s v c + (1− h a s )C a s v p + o a s v d (3) Summing Eq. 3 over S steps, the total latency of agent a is: T stream a = S X s=1 t a s = P S s=1 h a s C a s v c + P S s=1 (1− h a s )C a s v p + O a v d (4) TheAagents form a pipeline: whileAgent a processes steps,Agent a−1 can already process step s + 1. LetF a s denote the wall-clock time at whichAgent a completes steps. With boundary conditions F a 0 = F 0 s = 0, the exact finish time follows the recurrence: F a s = max F a s−1 , F a−1 s + t a s , T stream = F A S (5) A.5.3 Proof of Theorem 2 Since Eq. 5 admits no closed form, we lower-bound T stream , yielding an upper bound on Speedup. Upper-bound derivation. Assuming comparable workloads across agents: Simplifying T serial . Substituting P a ≈ ̄ P and O a ≈ S ̄ O into Eq. 1: T serial = A X a=1 P a + O a−1 v p + O a v d ≈ A X a=1 ̄ P + S ̄ O v p + S ̄ O v d = A( ̄ P + S ̄ O) v p + AS ̄ O v d 20 SimplifyingT stream . The pipeline has two phases: (1) the first step traverses allAagents serially, contributing 1 S P A a=1 T stream a ; (2) each of the remainingS− 1steps waits only for the bottleneck agent, contributing S−1 S max A a=1 T stream a . Hence: T stream ≈ 1 S A X a=1 T stream a + S− 1 S A max a=1 T stream a (6) Substituting Eq. 4 into Eq. 6. SinceO a ≈ S ̄ O, the decode term exitsmax A a=1 ; replacingmax A a=1 with 1 A P A a=1 (since max≥ mean, this lower-bounds T stream ): T stream ≈ 1 S A X a=1 S X s=1 h a s C a s v c + (1− h a s )C a s v p + o a s v d + S− 1 S A max a=1 " S X s=1 h a s C a s v c + (1− h a s )C a s v p + o a s v d # ≈ 1 S A X a=1 S X s=1 h a s C a s v c + (1− h a s )C a s v p + A ̄ O v d + S− 1 S A max a=1 " P S s=1 h a s C a s v c + P S s=1 (1− h a s )C a s v p # + (S− 1) ̄ O v d ≈ 1 S A X a=1 S X s=1 h a s C a s v c + (1− h a s )C a s v p + A ̄ O v d + S− 1 SA A X a=1 S X s=1 h a s C a s v c + (1− h a s )C a s v p + (S− 1) ̄ O v d = S + A− 1 SA A X a=1 S X s=1 h a s C a s v c + (1− h a s )C a s v p + (A + S− 1) ̄ O v d (7) Letα ≜ P A a=1 P S s=1 (1− h a s )C a s AS ̄ O andβ ≜ P A a=1 P S s=1 h a s C a s AS ̄ O . Hereαis the average uncached context tokens per output token, andβis the cached counterpart.C a s takes different forms: for the root agent (a = 1), there is only one call withC 1 1 = ̄ P + S ̄ OandC 1 s = 0fors ≥ 2; for non-root agents (a≥ 2), at stepsthe context accumulates asC a s = ̄ P + (2s− 1) ̄ O (system prompt,supstream steps, and s− 1 own previous steps). To instantiate α and β, we consider two practical scenarios: •Without prefix caching (h a s = 0for all non-root agents and steps): all context tokens require prefill, so they contribute entirely to α: α = C 1 1 + P A a=2 P S s=1 C a s AS ̄ O = ( ̄ P + S ̄ O) + (A− 1) P S s=1 [ ̄ P + (2s− 1) ̄ O] AS ̄ O = ( ̄ P + S ̄ O) + (A− 1)(S ̄ P + S 2 ̄ O) AS ̄ O using P S s=1 (2s− 1) = S 2 = ̄ O(r po + S) + (A− 1)S ̄ O(r po + S) AS ̄ O = (r po + S) 1 + (A− 1)S AS β = 0 •With prefix caching (h 1 s = 0,h a s = 1fora≥ 2): the root agent’s call is uncached since the original query is not pre-cached, while non-root agents benefit from full prefix caching since the upstream 21 output is already cached. Only a = 1 contributes to α: α = C 1 1 AS ̄ O = ̄ P + S ̄ O AS ̄ O = r po + S AS β = P A a=2 P S s=1 C a s AS ̄ O = (A− 1) P S s=1 [ ̄ P + (2s− 1) ̄ O] AS ̄ O = (A− 1)(S ̄ P + S 2 ̄ O) AS ̄ O = (A− 1)(r po + S) A Substituting P A a=1 P S s=1 h a s C a s = βAS ̄ O and P A a=1 P S s=1 (1− h a s )C a s = αAS ̄ O into Eq. 7: T stream ≈ S + A− 1 SA βAS ̄ O v c + αAS ̄ O v p + (S + A− 1) ̄ O v d = (S + A− 1) β ̄ O v c + α ̄ O v p + ̄ O v d Speedup. Taking Speedup = T serial /T stream , letting r po ≜ ̄ P/ ̄ O, and canceling ̄ O: Speedup≈ A( ̄ P + S ̄ O) v p + AS ̄ O v d (S + A− 1) β ̄ O v c + α ̄ O v p + ̄ O v d = A S + r po v p + S v d (S + A− 1) β v c + α v p + 1 v d Multiplying numerator and denominator by v d and letting r v dp ≜ v d /v p , r v dc ≜ v d /v c : Speedup = A (S + r po )r v dp + S (S + A− 1)(1 + αr v dp + β r v dc ) (8) Speedup formula: simplification under successive conditions. Speedup r v dc ≈0 −→ A[(S + r po )r v dp + S] (S + A− 1)(1 + αr v dp ) r po ≈0,αr v dp ≈0 −→ A[(S + r po )r v dp + S] S + A− 1 r v dp ≈0 −→ AS S + A− 1 Three conditions enable successive simplification: (1) fast cache reads (v c ≫ v d , sor v dc ≈ 0): KV-cache reuse is far faster than decode, making theβr v dc term negligible; (2) high cache-hit rate (αr v dp ≈ 0): under prefix caching withr po ≪ S,α ≈ 1/A, making the term negligible; (3) fast prefill (r v dp ≈ 0): prefill is far faster than decode, making the(S + r po )r v dp term negligible. Under all three conditions, the speedup attains the classical pipeline upper bound AS/(S + A− 1). Numerical Example: Speedup with A = S = 4 (Claude Opus 4.6 API) We instantiate Eq. 8 with real performance data from the Claude Opus 4.6 API: output speedv d ≈ 39 tokens/s and prefill throughput v p ≈ 6,000 tokens/s. 3 These give: r v dp = v d v p ≈ 39 6000 ≈ 0.007 3 https://artificialanalysis.ai/models/claude-opus-4-6/providers 22 We reuse the same settings as the cost example:A = 4agents,S = 4steps,r po ≈ 0,r v dc ≈ 0 , and prefix caching (h 1 s = 0, h a s = 1 for a≥ 2). Deriving α: From the with-prefix-caching scenario (r po = 0): α = r po + S AS = 4 16 = 0.25 Deriving β: From the with-prefix-caching scenario (r po = 0): β = (A− 1)(r po + S) A = 3× 4 4 = 3 Substituting into Eq. 8 with r po = 0 and r v dc = 0: Speedup = A S r v dp + S (S + A− 1) (1 + αr v dp ) = 4 (0.028 + 4) 7 (1 + 0.00175) ≈ 16.11 7.012 ≈ 2.30 For A = S = 4, Stream delivers approximately 2.30× speedup over Serial. Sensitivity to KV-cache hit rate. To show how speedup varies with caching, we repeat the computation without prefix caching (h a s = 0for all agents and steps):α = (r po + S)[1 + (A − 1)S]/(AS) = 4× 13/16 = 3.25, β = 0. Speedup = 4 (0.028 + 4) 7 (1 + 3.25× 0.007) ≈ 16.11 7.159 ≈ 2.25 Both scenarios yield nearly identical speedups (2.25vs.2.30), confirming that when prefill is sufficiently faster than decode (r v dp ≪ 1), the KV-cache hit rate has negligible effect on speedup. Remark (Extension to DAG topologies). The derivation above assumes a chain ofAagents. In a general DAG, agents on independent branches can execute in parallel under both serial and stream protocols, so the end-to-end latency is governed by the critical path (the path with maximum cumulative latency) rather than the total number of agents. LetDdenote the number of agents on this critical path. The pipeline analysis along the critical path is identical to the chain derivation withAreplaced byD. Theorem 2 thus extends to DAGs by substituting D for A, with α and β computed over agents on the critical path. A.6 Detailed Cost Analysis Theorem 3 (Cost Ratio). Under the same setup as Theorem 2, the cost of Stream over Serial is: Cost stream Cost serial = ρ· r c pd (α + r c cp β) + 1 r c pd (1 + r po /S) + 1 A.6.1 Cost of Serial Protocol In the serial protocol, each agent issues a single uncached call:Agent a prefillsP a + O a−1 tokens and decodes O a tokens. Summing over all A agents: Cost serial = A X a=1 [(P a + O a−1 )c p + O a c d ] = c p A X a=1 (P a + O a−1 ) + c d · O Σ (9) A.6.2 Cost of Stream Protocol Root agent. Agent 1 issues a single uncached call: prefilling P 1 + O 0 tokens and decoding O 1 tokens: Cost 1 = (P 1 + O 0 )c p + O 1 c d 23 Non-root agents.Agent a (a≥ 2) issuesScalls. At steps, the context length isC a s , the KV-cache hit rate ish a s , and the output length iso a s ; cache-hit tokens are charged atc c , cache-miss tokens atc p , and decoded tokens at c d . Summing over all agents: Cost stream = Cost 1 + A X a=2 S X s=1 h (1− h a s )C a s c p + h a s C a s c c + o a s c d i Expanding Cost 1 = (P 1 + O 0 )c p + O 1 c d and grouping terms by token price (c p , c c , c d ): Cost stream = c p " (P 1 + O 0 ) + A X a=2 S X s=1 (1− h a s )C a s # + c c A X a=2 S X s=1 h a s C a s + c d " O 1 + A X a=2 S X s=1 o a s # Using P S s=1 o a s = O a , so O 1 + P A a=2 O a = O Σ : Cost stream = c d · O Σ + c p " (P 1 + O 0 ) + A X a=2 S X s=1 (1− h a s )C a s # + c c A X a=2 S X s=1 h a s C a s (10) A.6.3 Proof of Theorem 3 Dividing Eq. 10 by Eq. 9 yields the exact cost ratio: Cost stream Cost serial = c d O Σ + c p h (P 1 + O 0 ) + P A a=2 P S s=1 (1− h a s )C a s i + c c P A a=2 P S s=1 h a s C a s c p P A a=1 (P a + O a−1 ) + c d O Σ (11) Equation 11 is exact for arbitraryP a ,O a . We now derive its closed form under the same simplifying assumptions as Theorem 2:P a ≈ ̄ PandO a ≈ S ̄ O, henceO Σ = AS ̄ O. Note that ̄ Ois mode-specific (via ̄ O = O Σ /(AS)); we suppress the mode subscript within each derivation below, and the ratioρre-emerges naturally when dividing. Simplifying Cost serial . Each input is P a + O a−1 ≈ ̄ P + S ̄ O = S ̄ O(1 + r po /S): Cost serial = c p A X a=1 (P a + O a−1 ) + c d · O serial Σ ≈ c p · A( ̄ P + S ̄ O) + c d · AS ̄ O = c p · AS ̄ O 1 + r po S + c d · AS ̄ O = AS ̄ O h c p 1 + r po S + c d i (12) SimplifyingCost stream . SubstitutingO stream Σ = AS ̄ Oand the definitions ofα,βfrom App. A.5 into Eq. 10: (P 1 +O 0 ) + A X a=2 S X s=1 (1− h a s )C a s = A X a=1 S X s=1 (1− h a s )C a s = αAS ̄ O, A X a=2 S X s=1 h a s C a s = A X a=1 S X s=1 h a s C a s = β AS ̄ O where the second equality usesh 1 s = 0for alls(root agent is always uncached). Substituting into Eq. 10: Cost stream = c p · αAS ̄ O + c c · β AS ̄ O + c d · AS ̄ O = AS ̄ O[αc p + β c c + c d ](13) 24 Computing the ratio. Dividing Eq. 13 by Eq. 12; theASfactors cancel, and applying ̄ O = O Σ /(AS) to each side gives ̄ O stream / ̄ O serial = O stream Σ /O serial Σ = ρ (Tab. 2): Cost stream Cost serial ≈ ρ· αc p + β c c + c d c p (1 + r po /S) + c d Substituting c c = r c cp c p and dividing through by c d (letting r c pd ≜ c p /c d ): Cost stream Cost serial = ρ· r c pd (α + r c cp β) + 1 r c pd (1 + r po /S) + 1 (14) When is Stream cheaper? Stream is cheaper (ratio < 1) in the following regimes: 1. Decode-dominated cost (r c pd → 0): the ratio reduces to ρ, so Stream is cheaper iff ρ < 1. 2. Equal output length (ρ = 1, r c pd > 0): cheaper iff α + r c cp β < 1 + r po S . 3. Prefix caching, as a special case of (2) (r po ≈ 0,h a s = 1fora ≥ 2,h 1 s = 0): App. A.5 gives α≈ 1/A and β ≈ (A− 1)S/A. Substituting into condition (2) with r po ≈ 0: 1 A + r c cp · (A− 1)S A < 1 ⇐⇒ 1 + r c cp (A− 1)S < A ⇐⇒ r c cp S < 1. Numerical Example: Cost Ratio with A = S = 4 (Claude Opus 4.6 Pricing) To give intuition for Eq. 14, we instantiate it with Claude Opus 4.6 pricing: prefillc p = $5/MTok, decode c d = $25/MTok, and cache-read c c = $0.50/MTok. 4 These give: r c pd = c p c d = 5 25 = 0.2, r c cp = c c c p = 0.50 5 = 0.1 We reuse the settings of the Speedup example (App. A.5):A = S = 4,r po /S ≈ 0, and full prefix caching for non-root agents (h a s = 1fora ≥ 2,h 1 s = 0), which givesα = 0.25andβ = 3. Substituting into Eq. 14 with r po /S = 0: Cost stream Cost serial = ρ· 0.2 (0.25 + 0.1× 3) + 1 0.2× 1 + 1 = ρ· 0.2× 0.55 + 1 1.2 = ρ· 0.11 + 1 1.2 = 0.925ρ For ρ≈ 1, Stream is approximately 7.5% cheaper than Serial in this full-prefix-cache regime. Contrast: no-cache regime. We seth a s = 0for alla,s, so all context tokens are charged atc p andβ = 0. Sinceα + β = P a,s C a s /(AS ̄ O) is cache-invariant,α = 0.25 + 3 = 3.25. Substituting into Eq. 14 with r po /S = 0: Cost stream Cost serial = ρ· 0.2× 3.25 + 1 1.2 = ρ· 1.65 1.2 = 1.375ρ Forρ ≈ 1, Stream is now approximately37.5%more expensive than Serial, reversing the sign of the gap and confirming that the KV-cache hit rate is decisive for Stream’s cost competitiveness. Modern serving stacks such as vLLM (Kwon et al., 2023) and SGLang (Zheng et al., 2024), together with recent agentic-cache extensions (Liu et al., 2026; Wu et al., 2026; Pan et al., 2025; Ye et al., 2025a), push prefix-caching hit rates close to the full-cache regime. The7.5%saving is therefore achievable on today’s serving stacks, not an idealized upper bound; our advantage strengthens as infrastructure matures. Remark (Extension to DAG topologies). Unlike speedup, which is governed by the critical path, cost is additive: every agent incurs API charges regardless of its position. Theorem 3 therefore extends to arbitrary DAGs by interpreting A as the total number of agents and averaging α, β over all of them. 4 https://w.anthropic.com/pricing 25 A.7 Testing Setup and Files For the results reported in this paper, we use the following OpenCompass configuration files: • gpqa_gen.py • livecodebench_v6_academic.py • hle_gen.py • aime2026_cascade_eval_gen_6f468.py • hmmt2026_cascade_eval_gen_6f468.py • aime2025_cascade_eval_gen_5e9f4f.py The LiveCodeBench configuration covers three sub-tasks: code generation (lcb_code_generation), code execution (lcb_code_execution), and test output prediction (lcb_test_output). For all benchmarks requiring LLM-as-judge evaluation, we use GPT-5.4 as the judge model. Algorithm 3 STREAM EXECUTION ON DAGS Require: DAG G = (V,E); pred(v), succ(v) for v ∈ V ◦ pred(v): direct predecessors of node v ◦ succ(v): direct successors of node v 1: for each source v ∈ V (with pred(v) =∅) do 2:queue v .put(Q) 3: end for 4: ▷ all nodes concurrent 5: for each v ∈ V in parallel do 6:while msg ← queue v .get() do 7:▷ any predecessor; no waiting 8:ctx v .append(msg ) 9:steps ← LLM(ctx v , stream=True) 10:for each step from steps do 11:for each u∈ succ(v) do 12:queue u .put(step) 13:end for 14:▷ KV cache reuse 15:ctx v .append(step) 16:end for 17:end while 18: end for A.8 Stream Protocol on DAGs Compared to the chain version (Alg. 2), three changes extend Stream to arbitrary DAGs (Alg. 3):Qis broadcast to every source node (in-degree 0) instead of only Agent 1 (lines 1–2); each step is pushed to all direct successors instead of the singlequeue a+1 (lines 11–13); and a multi-predecessor node processes steps on arrival without synchronization (line 6), preserving full parallelism. A.9 System Prompts Following the topologies defined in Sec. 4.1, all configurations use four agentsA 1 –A 4 , labeledA–Din the prompts; we adopt the prompt labels throughout. Each downstream agent (B–D) sees only its predecessor’s response and the original question (appended at runtime as “The original problem is: ". . . "”). Every system prompt begins with a one-line topology tag[Topology: <name> <edges>], which is the 26 only Chain/Tree/Graph difference. The boxes below show the Chain version; for Tree and Graph, the topology tag changes and the “You receive . . . output.” line reflects each agent’s predecessors. Stream extends Serial by appending the bolded text shown in each box below: a one-lineEND_STEP boundary forB–D, plus a minimal problem-solver body forA. Non-bolded text is identical between Serial and Stream. Agents with in-degree> 1(Din Tree,Cin Graph) also receive the multi-predecessor instruction below. Everything else follows OpenCompass defaults, including the Single system prompt. A [Topology: Chain A→B→C→D] You are a problem solver. For each step: solve concisely with key reasoning. After all steps, provide a DETAILED final answer summary. Divide your response into 3 roughly equal parts. End each part (including the last) with END_STEP on its own line. Your response must end with END_STEP. B [Topology: Chain A→B→C→D] You are Agent_B. You receive Agent_A’s output. You are a reviewer-and-corrector. For each step: verify correctness briefly, CORRECT ANY ERRORS YOU FIND. CRITICAL REQUIREMENT: 1. YOU MUST CORRECT ERRORS: When found, state “ERROR: [description]” then “CORRECTION: [corrected step]”. 2. Pass forward the MOST ACCURATE version (original if correct, your correction if not). After all steps, provide DETAILED overall summary with thorough analysis and any corrections made. [Stream only] After your response, output END_STEP on its own line. C [Topology: Chain A→B→C→D] You are Agent_C. You receive Agent_B’s output. You are a reviewer-and-corrector. For each step: double-check for MISSED errors briefly, CORRECT ANY YOU FIND. CRITICAL REQUIREMENT: 1. YOU CAN OVERRULE PREVIOUS AGENTS: When you find errors, state “ERROR: [description]” then “CORRECTION: [corrected step]”. 2. Pass forward the MOST ACCURATE version. After all steps, provide DETAILED overall summary with thorough analysis and any corrections made. [Stream only] After your response, output END_STEP on its own line. D [Topology: Chain A→B→C→D] You are Agent_D. You receive Agent_C’s output. You are a reviewer-and-corrector. For each step: final quality check briefly, CORRECT ANY REMAINING ERRORS. CRITICAL REQUIREMENT: 1. YOU HAVE FINAL CORRECTION AUTHORITY: This is the last chance to fix errors. State “ERROR: [description]” then “CORRECTION: [corrected step]”. 2. You are responsible for final answer accuracy. FINAL ANSWER: Must directly address the original problem. Base on CORRECTED versions. After all steps, provide DETAILED final answer with thorough analysis. [Stream only] After your response, output END_STEP on its own line. Tree and Graph variants. The four boxes above show the Chain topology. For Tree and Graph, two things change: (1) the topology tag on the first line becomes[Topology: Tree A→B, C→D]or [Topology: Graph A→B→C→D, A→C]; (2) the “You receive . . . output.” line is adjusted to list each agent’s actual predecessors. Specifically, under Tree,BandCboth receiveA’s output independently, whileDreceives outputs fromBandC. Under Graph,Creceives outputs from bothAandB, whileDreceives onlyC’s output. For agents receiving multiple inputs, a two-line formatting instruction is appended: “INPUT FORMAT: Inputs from multiple agents are labeled [Agent_X]: . . .” and “RESPONSE FORMAT: For each agent, address their content explicitly, then synthesize.” Scaling toA ∈ 8, 16, 32, 64agents. For the step-level scaling experiments (Sec. 4.3.1), we extend 27 the Chain to longer sequences (A→B→C→ ·).Aremains the solver, and the last agent uses the same “FINAL CORRECTION AUTHORITY” prompt asD. Intermediate agents share the reviewer-and- corrector structure but vary their lead phrase across verify correctness, double-check for MISSED errors, independent check, cross-verify with fresh eyes, penultimate check to encourage independent reasoning. A.10 Information About Use of AI Assistants AI assistants were used in two roles: language polishing of the manuscript; and drafting auxiliary code. A.11 Potential Risks Our step-level perturbation experiment in Sec. 4.2.3 shows that replacing clean reasoning steps with perturbed counterparts can steer downstream agents toward wrong answers. Malicious actors could exploit the same idea to inject subtle errors into intermediate steps of a multi-agent chain, causing the final output to silently diverge. We release only the evaluation protocol and do not provide automated perturbation tooling; we also recommend step-level verification as a defense against such tampering. A.12 Artifact Statement Licenses. OpenCompass is open-source, and we obtain all benchmark datasets through it (AIME 2025 / 2026, HMMT 2026, GPQA-Diamond, HLE, and LiveCodeBench; see App. A.7). The Claude Opus 4.6 and GPT-5.4 backbones are accessed via Anthropic’s and OpenAI’s commercial APIs under their respective terms of service. Intended Use. We use all upstream benchmarks as held-out evaluation sets for LLM reasoning, matching their intended research use; we do not modify or redistribute their contents. Personally Identifying Information and Offensive Content. All datasets are standard academic reasoning benchmarks (mathematics, graduate-level science, programming) and contain neither personally identifying information nor offensive content by construction. Documentation. All benchmarks target English-language academic reasoning, covering competition mathematics, graduate-level natural sciences, and program understanding (code generation, code execution, and test-output prediction); none targets any specific demographic group. Per-benchmark configurations, judge models, and system prompts are in App. A.7 and App. A.9. Data Statistics. We use each benchmark’s official test set without subsampling: AIME 2025 and AIME 2026 (30 problems each); HMMT 2026 (33 problems); GPQA-Diamond (198 problems); HLE and LiveCodeBench, both loaded as defined in App. A.7. Each backbone, topology, method cell averages 3 independent runs, increased to 8 on AIME 2025 / 2026 and HMMT 2026 due to their smaller test sets (Sec. 4.1). 28