Paper deep dive
StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems
Yanwen Peng, Delvin Ce Zhang, Xi Wang, Nikolaos Aletras
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/16/2026, 2:16:25 AM
Summary
The paper introduces StateBridge, a training-free method for latent communication in Large Language Model (LLM) multi-agent systems. It addresses the information loss inherent in text-based communication by aligning the sender's final-layer hidden states to the receiver's input embedding space using a closed-form orthogonal transformation (Procrustes alignment), norm calibration, and vocabulary anchoring. This allows agents to transmit continuous representations directly, bypassing tokenization bottlenecks. StateBridge is evaluated on math reasoning, code generation, and question answering tasks, demonstrating superior performance compared to text-based and existing latent communication baselines across multiple model families.
Entities (16)
Relation Signals (17)
StateBridge → evaluatedon → AIME24
confidence 95% · We evaluate StateBridge on... AIME24
StateBridge → evaluatedon → ARC-Challenge
confidence 95% · We evaluate StateBridge on... ARC-Challenge
StateBridge → evaluatedon → MedQA
confidence 95% · We evaluate StateBridge on... MedQA
StateBridge → evaluatedon → HumanEval
confidence 95% · We evaluate StateBridge on... code generation... HumanEval+
StateBridge → evaluatedon → AIME25
confidence 95% · We evaluate StateBridge on... AIME25
StateBridge → evaluatedon → GSM8K
confidence 95% · We evaluate StateBridge on math reasoning... GSM8K
StateBridge → evaluatedon → GPQA-Diamond
confidence 95% · We evaluate StateBridge on... question answering... GPQA-Diamond
StateBridge → evaluatedon → MBPP
confidence 95% · We evaluate StateBridge on... code generation... MBPP+
StateBridge → testedonmodel → OLMo3-7B-Think
confidence 95% · We test two model families: ... OLMo3-7B-Think... We evaluate StateBridge... with four models from two families.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model based multi-agent systems usually communicate in text, i.e., using discrete tokens. However, text introduces a discrete bottleneck. Converting the sender's continuous hidden states into discrete tokens discards information that token identities alone cannot capture. Recent work proposes latent communication as an alternative, where agents transmit hidden representations directly without converting them to text. However, existing latent methods either inject working memory layer by layer across the transformers, or require trained projectors that limit portability. We propose StateBridge, a training-free latent communication approach that aligns the sender's final-layer hidden states to the receiver's input space via a closed-form orthogonal transformation. Lightweight norm calibration and vocabulary anchoring ensure compatibility with the pretrained input distribution. The aligned states are prepended to the input of the receiver agent as a continuous prefix. We evaluate StateBridge on math reasoning, code generation, and question answering with four models from two families. StateBridge achieves the best or tied-best score on 22 out of 26 model-task pairs, consistently outperforming the strongest baseline.
Tags
Links
- Source: https://arxiv.org/abs/2608.13317v1
- Canonical: https://arxiv.org/abs/2608.13317v1
Trouble viewing inline? Open PDF directly →
Full Text
63,819 characters extracted from source content.
Expand or collapse full text
StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems Yanwen Peng Delvin Ce Zhang Xi Wang Nikolaos Aletras Affiliation: School of Computer Science, University of Sheffield Affiliation: Sheffield, United Kingdom Email: ypeng86,delvin.ce.zhang,xi.wang,n.aletras@sheffield.ac.uk Abstract Large language model based multi-agent systems usually communicate in text, i.e., using discrete tokens. However, text introduces a discrete bottleneck. Converting the sender’s continuous hidden states into discrete tokens discards information that token identities alone cannot capture. Recent work proposes latent communication as an alternative, where agents transmit hidden representations directly without converting them to text. However, existing latent methods either inject working memory layer by layer across the transformers, or require trained projectors that limit portability. We propose StateBridge, a training-free latent communication approach that aligns the sender’s final-layer hidden states to the receiver’s input space via a closed-form orthogonal transformation. Lightweight norm calibration and vocabulary anchoring ensure compatibility with the pretrained input distribution. The aligned states are prepended to the input of the receiver agent as a continuous prefix. We evaluate StateBridge on math reasoning, code generation, and question answering with four models from two families. StateBridge achieves the best or tied-best score on 22 out of 26 model–task pairs, consistently outperforming the strongest baseline.11 1 Code is available at https://github.com/YanwenPneg/StateBridge. 1 Introduction Modern Multi-Agent (MA) systems consist of Large Language Model (LLM) agents that collaborate for tackling complex tasks requiring planning, critique, and verification (30; 15). Yet their performance hinges on the communication channel between agents (13; 31). Natural language remains the dominant medium, but text introduces a discrete bottleneck. The sender must transform its continuous internal state into tokens before transmission. This compression discards continuous information from the latent representation that discrete tokens alone do not capture (42; 7), impairing inter-agent coordination (2). The receiver maps this token sequence back to vectors through its own embedding layer, recovering only token identities rather than the sender’s original hidden states. This results in information loss and motivates a fundamental question: can off-the-shelf LLM agents bypass natural language and effectively communicate through continuous representations? An intuitive strategy is to transmit hidden states directly. This is viable when all agents share the same pretrained LLM and thus the same hidden dimensionality (41; 42). However, matching dimensionality alone is not sufficient. Pretrained LLMs are trained to read token embeddings at the input layer, whereas decoder hidden states occupy a different region of the representation space. Passing hidden states directly therefore produces latent vectors with the right dimensionality to receivers but lacks the geometric alignment required for the model to interpret them, causing semantic mismatch during communication. Prior work addresses this mismatch in two ways. Key-value (KV)-cache transfer methods (11; 42) avoid the input-layer mismatch by injecting internal states across all transformer layers. However, they transfer internal processing states rather than a compact representation of the complete message, which may potentially cause information loss. Learned latent-communication methods (7; 41) bridge the gap through trained projectors, but the projectors tie the method to the specific model and tasks they were trained on and require re-training for different models. To the best of our knowledge, it has yet to be explored whether this mismatch can be resolved through alignment alone, without training or architectural modification. Figure 1: Overview of StateBridge. Standard text communication (a) transforms the sender’s message to discrete token identities via sampling, which causes information loss, whereas StateBridge (b) extracts message hidden states from the sender, aligns them with the input embedding space, and prepends the resulting prefix to the receiver’s prompt. We propose StateBridge, a training-free communication interface for MA systems (illustrated in Figure 1). We extract hidden states from the sender’s generated message and align them with the receiver’s input embedding space through a closed-form transformation. The aligned states are then injected as a continuous prefix for the receiver agent. Unlike text, it preserves a richer continuous representation from the sender. Compared to raw hidden-state transfer, it also makes such representation compatible with the receiver’s input space. The central idea is that latent communication helps only when the transmitted states are both informative and compatible with the receiver’s input space. We evaluate StateBridge in a four-agent sequential pipeline on mathematical reasoning, code generation, and question answering benchmarks. Across four models spanning two families, StateBridge consistently outperforms text-based communication and KV-cache transfer methods. It achieves the best or tied-best score on 22 out of 26 model–task pairs without model updates and larger gains on challenging benchmarks. By operating only at the input embedding layer, StateBridge also applies more broadly across model families than methods that inject states across all transformer layers. A case study further confirms that the aligned prefix carries semantic information beyond the suffix tokens alone. Contributions. (1) We show that closed-form alignment alone resolves the representation mismatch between final-layer hidden states and input embeddings, enabling training-free and effective latent communication between off-the-shelf LLM agents. (2) We propose StateBridge, combining Procrustes alignment, norm calibration, and vocabulary anchoring into a closed-form alignment interface with no learnable parameters. (3) Across four models from two families, StateBridge outperforms both existing text-based and KV-cache transfer baselines. Ablations confirm that each component contributes, with geometry preservation providing the largest effect. 2 Related work LLM MA systems. LLM MA systems extend classical multi-agent coordination (21; 33) to modern language model settings, enabling autonomous agents to collaborate on reasoning, planning, and problem-solving (26; 28; 40; 9). Early methods such as AutoGen (30) and CAMEL (15) coordinate multiple LLMs through explicit dialogue or role assignment. Subsequent work introduces structured communication protocols (4; 31) and agent specialization (20; 10). These systems have been applied to math and science reasoning (17; 35), open-domain question answering (10; 29), and GUI interaction (36; 34). However, most existing methods communicate through text, which discards hidden state information (7), introduces redundancy for linguistic coherence (37), and can impair inter-agent coordination (2). On the contrary, agents in our model communicate in the latent space, mitigating these problems. Latent communication in MA systems. Recent work explores latent-space communication as an alternative to text. Early work such as CIPHER (22) transmits weighted average of vocabulary embeddings to avoid token sampling, but still projects onto the discrete vocabulary space rather than preserving hidden states. KV-cache methods such as Cache-to-Cache (11) and LatentMAS (42), which uses a training-free linear alignment to enable latent reasoning, transfer working memory across agents, but transmit internal processing state rather than communication content. Embedding-based methods such as Interlat (7) and ThoughtComm (41) transmit hidden states directly, but require trained modules, limiting their applicability to the specific model and projector they were trained on. In contrast, our StateBridge transmits generated-message hidden states and enables effective message passing through training-free alignment to the receiver’s input embedding space, offering effective content communication and wide generalizability. 3 Method 3.1 Problem statement A homogeneous MA system is defined by all agents sharing the same pretrained LLM ℳM (39; 42). ℳM has vocabulary size V and embedding dimension d, with the token embedding matrix defined as emb∈ℝV×dW_emb ^V× d. We abstract the communication channels within a MA system consisting of two primary entities: a sender sA_s and a receiver rA_r. A message generated by the sender sA_s is characterized by (1) the final-layer hidden states (S) and (2) the embeddings of the corresponding decoded tokens in the shared embW_emb, which we term the reference embeddings R. Sending only R reduces the message to discrete text, discarding continuous information that token identities alone do not capture (see detailed explanation in Appendix A.1). Conversely, sending raw S preserves more information, but it is not aligned with the receiver’s input embedding space, as the receiver is pretrained to consume token embeddings rather than last-layer hidden states. Our goal is to produce an aligned prefix ¯ S that satisfies two requirements: (1) it preserves the pairwise geometric relationships among the sender’s hidden states; and (2) it is compatible with the receiver’s input embedding space, enabling its process without modification. StateBridge satisfies these two requirements as follows. 3.2 Message extraction In autoregressive models, subsequent hidden states aggregate information from the preceding context. To leverage this, we retain only the final K hidden states of the tokens of the generated message (e.g., K=64K=64). These states are indexed from 11 to K and denoted by =(1,…,K)⊤∈ℝK×d,S=(s_1,…,s_K) ^K× d, (1) where i∈ℝds_i ^d is the final-layer hidden state at position i. For models like Qwen3 (32) that generate intermediate reasoning (e.g., chain-of-thought delimited by ⟨ ⟩ and ⟨ /think⟩ tokens before the response), we discard hidden states from this section and retain the segment intended for the receiver. Let the resulting message tokens of the receiver be =(y1,…,yK)y=(y_1,…,y_K). Looking up these tokens in the token embedding matrix gives =emb[]∈ℝK×d.R=W_emb[y] ^K× d. (2) The reference embeddings R serve as the optimization target for aligning the continuous message. The discrete tokens y are not transmitted between agents. Figure 2: The StateBridge alignment interface transforms message hidden states S into the aligned prefix ¯ S for receiver using reference embeddings R. 3.3 Alignment interface The message hidden states S and the reference embeddings R describe the same generated message but occupy different regions of the representation space. The transformation must bridge this gap (1). Since geometric proximity in hidden-state space encodes semantic similarity (8), the alignment transformation should therefore preserve pairwise geometric relationships rather than minimize pointwise reconstruction error. StateBridge achieves this through a three-step alignment interface illustrated in Figure 2: Procrustes alignment, norm calibration, and vocabulary anchoring. Procrustes alignment. We treat the message hidden states S and the reference embeddings R as two point clouds in ℝdR^d. Both describe the same generated message, but in different coordinate systems. S carries the sender’s richer internal representation, while R lies in the space the receiver can read. We align S to R using the orthogonal Procrustes method (24), which finds the rotation that best maps S onto R while preserving pairwise distances and angles. We first remove the global offset of each set by centering: c=−KS⊤,c=−KR⊤,S_c=S-1_K μ_S , _c=R-1_K μ_R , (3) where K∈ℝK1_K ^K is the all-ones vector, S=1K∑i=1Ki μ_S= 1K _i=1^Ks_i is the row mean of S, and R μ_R is defined analogously for R. Centering restricts the alignment to the relative structure of the points rather than their absolute position difference in space. We then whiten both sets: S=1Kc⊤c+λ,R=1Kc⊤c+λ, _S= 1KS_c S_c+ , _R= 1KR_c R_c+ , (4) w=cS−1/2,w=cR−1/2,S_w=S_c _S^-1/2, _w=R_c _R^-1/2, (5) where λ>0λ>0 is a small regularization constant, and I is the identity matrix. Whitening rescales the principal directions of each set so that a few high-variance directions do not dominate the alignment. Centering and whitening together remove the global offset and equalize the principal directions. The remaining mismatch between the two point clouds is then a rotation. We then solve an orthogonal Procrustes problem: ∗=argmin‖w−w‖F2s.t.⊤=,Q^*= _Q \|S_wQ-R_w \|_F^2 .t. Q=I, (6) where ∥⋅∥F\|·\|_F denotes the Frobenius norm. This objective seeks the rotation that best aligns the whitened sender states to the whitened reference embeddings without introducing arbitrary shearing. The solution is closed-form. Let w⊤w=⊤S_w R_w=UDV be the singular value decomposition (SVD) (12) of the cross-correlation matrix. The optimal rotation is then ∗=⊤Q^*=UV . Since ∗Q^* is orthogonal, it preserves pairwise distances and angles in the whitened space. We provide a formal proof in Appendix A.3. Finally, we restore the overall scale pattern and location of the reference embeddings: ~=w∗R1/2+KR⊤. S=S_wQ^* _R^1/2+1_K μ_R . (7) This Procrustes alignment ensures that the resulting matrix ~ S remains a continuous representation, derived from the sender’s hidden states, while being closer to the input geometry required by the receiver for effective communication. Norm calibration and vocabulary anchoring. The Procrustes alignment step matches the orientation of the message to the input embedding space, but two practical issues remain: (1) Final-layer hidden states have much larger norms than token embeddings. On Qwen3-4B (32), the average hidden-state norm is approximately 140× that of the input embeddings. This gap arises from the accumulation of residual updates across layers and the next-token prediction objective. The norm gap persists after alignment and changes how strongly the aligned vectors interact with the receiver’s attention layers. (2) Aligned vectors ~ S could still lie far from regions occupied by actual vocabulary embeddings. We address these issues by applying a two-step adjustment to each aligned vector. First, we calibrate each aligned vector to the typical norm of the vocabulary embeddings. Let n¯=1V∑v=1V‖emb[v]‖2 n= 1V _v=1^V\|W_emb[v]\|_2 be the average ℓ2 _2 norm of the vocabulary embeddings. For each row ~i s_i of ~ S, we compute ^i=~i⋅n¯‖~i‖2. s_i= s_i· n\| s_i\|_2. (8) This step places the prefix on a comparable norm scale to the ordinary input token embeddings. Without it, the larger prefix norms would dominate the dot-product attention scores, causing the receiver to attend to the prefix regardless of semantic relevance. Second, we move each calibrated vector slightly toward its nearest vocabulary embedding under cosine similarity: vi∗=argmaxv∈1,…,V^i⊤emb[v]‖^i‖2‖emb[v]‖2,¯i=(1−α)^i+αemb[vi∗],v_i^*= _v∈\1,…,V\ s_i W_emb[v]\| s_i\|_2\,\|W_emb[v]\|_2, s_i=(1-α) s_i+ _emb[v_i^*], (9) where α∈[0,1]α∈[0,1]. This vocabulary anchoring step does not snap the message hidden states to discrete tokens. Instead, it keeps the representation continuous while moving it closer to regions of the input embedding space encountered during pretraining. While Procrustes alignment preserves the geometric structure of the message hidden states, these two steps ensure that the prefix is also compatible with the receiver’s input embedding space. Stacking all rows ¯i s_i gives the final aligned prefix ¯∈ℝK×d S ^K× d. 3.4 Prefix injection Let ∈ℝN×dP ^N× d denote the receiver’s prompt embeddings, where N is the number of prompt tokens. StateBridge prepends the aligned prefix to the prompt, forming the combined input =[¯;]∈ℝ(K+N)×dX=[ S;P] ^(K+N)× d. StateBridge injects the aligned prefix directly at the embedding layer, bypassing tokenization. We assign position indices sequentially over the concatenated input, so the prefix occupies positions 00 through K−1K-1, and the prompt follows from position K onward. The receiver’s attention and position-encoding mechanisms therefore treat the prefix identically to ordinary token embeddings, requiring no special handling. The receiver processes X with the standard language modeling layers, without any architectural modification. StateBridge therefore changes only the communication interface between agents. The sender transmits an aligned continuous prefix instead of text, while the underlying LLM remains unchanged. 3.5 Computational cost Time complexity. The alignment is dominated by two operations: the whitening eigendecomposition, which costs O(d3)O(d^3), and the vocabulary anchoring search, which costs O(KVd)O(KVd). Both computed once per batch. The remaining operations (centering, rotation, norm calibration) are O(Kd2)O(Kd^2) or lower. For comparison, let T denote the number of generated tokens and L the number of transformer layers. A single autoregressive pass costs O(TLd2)O(TLd^2). For typical configurations (T≥256T≥ 256, L=32L=32), alignment is cheaper than one generation pass. Space complexity. Naively storing every layer’s hidden states during generation costs O(TLd)O(TLd). We instead register a lightweight callback (forward hook) on the final transformer layer. It records only that layer’s output at each decoding step and discards the rest, reducing the extraction cost to O(Td)O(Td). Only the last K states are retained, giving O(Kd)O(Kd). The alignment adds O(d2)O(d^2) for covariance and SVD factors. Both are negligible relative to the model parameters. By contrast, KV-cache transfer methods store O(TLd)O(TLd) per agent (11; 42), incurring substantially larger memory overhead. 4 Experimental setup We follow 42 and use a standard four-agent pipeline across reasoning, question answering, and code generation tasks. Tasks and datasets. We use eight benchmarks from three task categories: mathematical reasoning (GSM8K (6), AIME24 (19), and AIME25 (38)), question answering (GPQA-Diamond (23), ARC-Challenge (5), and MedQA (14)), and code generation (MBPP+ and HumanEval+ (18)). This selection covers tasks that differ in reasoning depth, knowledge demand, and output precision. Detailed dataset descriptions are provided in Appendix B.1. Models. We test two model families: Qwen3 (4B, 8B, and 32B) (32) and OLMo3-7B-Think (27). Within each run, all agents share the same model weights. We evaluate all models on five core benchmarks. Following 42, we additionally report AIME24, AIME25, and GPQA-Diamond for the stronger Qwen3-8B and Qwen3-32B models. Baselines. We compare StateBridge against three baselines: (i) Single, standard single-agent generation; (i) TextMAS, text-based sequential communication (39); and (i) LatentMAS (42), which injects the sender’s KV cache into the receiver. All methods share the same four-agents (Planner, Critic, Refiner, Judger), identical hyperparameters, and the same evaluation protocol. The only difference across prompts is the communication modality description. Appendix B.2 describes the agent pipeline and Appendix C lists the full prompt templates. Implementation details. We set K=64K=64, α=0.3α=0.3, and λ=10−3λ=10^-3. Following 42, all agents use temperature 0.6 and top-p 0.95. Maximum output lengths range from 2,048 to 20,000 tokens depending on the task (Appendix B.3). All experiments were run on 2 NVIDIA A100-80G GPUs. The evaluation protocol for each task category is described in Appendix B.4. 5 Results Qwen3-4B OLMo3-7B-Think Task Single Text Latent SB Δ Single Text Latent SB Δ ARC-C 89.2 90.0 92.3 93.7 ↑ 1.4 84.6 82.2 78.3 89.6 ↑ 5.0 MedQA 47.7 65.3 66.3 70.3 ↑ 4.0 50.7 54.0 44.3 59.0 ↑ 5.0 GSM8K 82.4 89.8 88.2 89.8 0.0 85.7 85.7 67.5 86.5 ↑ 0.8 MBPP+ 63.5 69.8 73.5 75.9 ↑ 2.4 66.7 66.9 42.3 69.1 ↑ 2.2 HumanEval+ 75.0 79.7 79.9 82.3 ↑ 2.4 71.3 80.5 43.3 79.3 ↓ 1.2 Avg. 71.6 78.9 80.0 82.4 ↑ 2.4 71.8 73.9 55.1 76.7 ↑ 2.8 Qwen3-8B Qwen3-32B Task Single Text Latent SB Δ Single Text Latent SB Δ ARC-C 91.0 94.6 94.4 95.1 ↑ 0.5 95.5 93.9 95.7 94.5 ↓ 1.2 MedQA 53.0 75.0 75.3 78.3 ↑ 3.0 86.0 77.0 84.3 87.3 ↑ 1.3 GSM8K 81.1 92.3 93.8 91.2 ↓ 2.6 91.1 92.4 92.7 90.2 ↓ 2.5 MBPP+ 64.8 69.5 74.6 76.9 ↑ 2.3 74.3 75.1 74.1 76.2 ↑ 1.1 HumanEval+ 74.4 80.5 80.5 83.6 ↑ 3.1 78.7 84.8 84.2 85.4 ↑ 0.6 AIME24 50.0 53.3 56.7 63.3 ↑ 6.6 70.0 73.3 66.7 76.7 ↑ 3.4 AIME25 46.7 53.3 53.3 53.3 0.0 66.7 70.0 56.7 73.3 ↑ 3.3 GPQA 39.9 43.4 45.5 52.5 ↑ 7.0 52.5 58.3 57.1 64.1 ↑ 5.8 Avg. 62.6 70.2 71.8 74.3 ↑ 2.5 76.9 78.1 76.4 81.0 ↑ 2.9 Table 1: Main results across model families. Accuracy (%) on QA and math tasks, pass@1 (%) on code tasks. Bold: best or tied-best per task. Δ : StateBridge vs. the best baseline (green = gain, red = loss). Text = TextMAS, Latent = LatentMAS, SB = StateBridge (Ours). Table 1 summarizes the results across MA communication methods, models, and tasks. We report accuracy for question answering and mathematical reasoning and pass@1 (3) for code generation. StateBridge achieves the best average score in all four model settings, improving over the best baseline by 2.4 to 2.9 points. It also wins or ties on 22 out of 26 model–task pairs. These gains hold across both model families. Switching from Qwen3 to OLMo3 tests whether each latent method is sensitive to the LLM architecture. On OLMo3-7B-Think, LatentMAS averages only 55.1%, far below the 73.9% of TextMAS. StateBridge reaches 76.7%. Because the agent pipeline, prompts, and decoding settings are otherwise unchanged, this gap points to the communication channel. KV-cache transfer injects states across each transformer layers. If the layer structure differs between LLMs, compatibility breaks. StateBridge operates only at the input embedding layer and the use of such a uniform interface ensures a greater applicability across model families. StateBridge helps most on the more challenging benchmarks, including MedQA, GPQA, AIME24/25, and the code generation tasks. StateBridge’s message hidden states encode distributional information beyond the sampled tokens, including confidence signals and traces of alternative reasoning paths. Text-based communication collapses this to a single token sequence, leading to information loss and incomplete message communication. Not all tasks show gains. On GSM8K for Qwen3 models, StateBridge trails the best baseline, likely because continuous prefixes affect output formatting under exact-match evaluation (16; 25). 6 Analysis 6.1 Ablation study Table 2 isolates the contribution of each StateBridge component. Orthogonal vs. linear alignment. A natural alternative to Procrustes is to fit a linear map via ridge regression, minimizing |−|2+λ||2|SW-R|^2+λ|W|^2. This map is not constrained to preserve distances or angles, so it achieves low pointwise reconstruction error. However, it distorts the pairwise geometry among the sender states. Replacing Procrustes with ridge regression results to average performance drops from 82.4% to 74.9%, with especially larger drops on MBPP+ and HumanEval+. Since pairwise geometry encodes semantic similarity, distorting it discards the richer information that latent communication carries over text. The effect is strongest on code generation, which demands structurally precise output. Orthogonal alignment preserves these semantic similarity and maintains the precision of the receiver’s generation. Appendices A.2 and A.3 provide a geometric explanation for this gap. Compatibility steps. The two compatibility steps also matter. Removing norm calibration lowers the average to 79.5%, and removing vocabulary anchoring lowers it to 80.2%. This suggests that alignment alone is not enough. The prefix also needs the right norm scale and must stay close to embedding regions seen during pretraining. Replacing the aligned prefix with random noise drops the average to 48.8%. This rules out the possibility that gains come from simply prepending extra continuous vectors. Variant ARC-C MedQA GSM8K MBPP+ HE+ Avg. Δ Full StateBridge 93.7 70.3 89.8 75.9 82.3 82.4 – Ridge Regr. 93.0 68.0 88.6 61.5 63.6 74.9 ↓ 7.5 Random Noise 32.2 30.7 84.5 48.2 48.2 48.8 ↓ 33.6 w/o Norm Calib. 90.9 68.0 87.1 71.7 79.9 79.5 ↓ 2.9 w/o Vocab. Anchor. 91.7 66.7 88.9 73.0 80.5 80.2 ↓ 2.2 Table 2: Ablation on Qwen3-4B. Each row removes or replaces one component. HE+: HumanEval+. Prefix length K Anchoring coefficient α Task 16 32 64 128 0.0 0.1 0.2 0.3 0.4 0.6 ARC-C 93.2 93.3 93.7 91.7 91.7 93.5 93.2 93.7 92.5 93.3 MedQA 68.0 68.7 70.3 68.7 66.7 64.0 71.3 70.3 69.7 66.3 GSM8K 87.6 87.9 89.8 88.6 88.9 89.1 88.7 89.8 88.0 89.0 MBPP+ 74.6 75.9 75.9 74.9 73.0 73.3 69.3 75.9 72.2 72.0 HumanEval+ 82.3 81.7 82.3 79.3 80.5 78.1 81.7 82.3 83.5 82.9 Table 3: Sensitivity to prefix length K and anchoring coefficient α on Qwen3-4B. Moderate values work best. Default values (K=64K=64, α=0.3α=0.3) are shaded. 6.2 Hyperparameter sensitivity For all experiments, we use the best hyperparameters (K=64K=64, α=0.3α=0.3) on MedQA obtained with Qwen3-4B and apply them without modification to all other datasets and models. Table 3 reports the sensitivity to prefix length K and anchoring coefficient α. Increasing K from 16 to 64 usually improves performance, suggesting that very short messages loss useful information. However, K=128K=128 hurts on most tasks. We attribute this to the alignment procedure. StateBridge fits a single global rotation to all K states. When K is small, retained states come from the message end and tend to have similar norms and geometry. Larger K includes earlier states that may differ in both. The global rotation then has to compromise across a more heterogeneous point set, potentially lowering alignment quality. The anchoring coefficient α controls a trade-off between informativeness and compatibility. A smaller α preserves more of the aligned hidden state but keeps the prefix further from the receiver’s input embedding space. A larger α moves the prefix closer to actual reference embeddings but discards more continuous information. α=0.3α=0.3 strikes a consistent balance across all benchmarks, so we adopt it as a fixed default. 6.3 Alignment visualization Figure 3 shows the gap between message hidden states and reference embeddings, and how StateBridge closes it. States are collected from 300 MedQA queries. Before alignment, the message hidden states occupy a different region from the reference embeddings. After alignment, the prefix moves much closer to the input embedding space. The same pattern appears for both Qwen3-4B and Qwen3-8B, suggesting that the effect is stable across model scales. Figure 3: Principal component analysis (PCA) density (left) and PCA scatter of the embedding space (right) for Qwen3-4B and Qwen3-8B. Orange: message hidden states is_i; blue: reference embeddings ir_i; green: aligned prefix ¯i s_i. 6.4 Case study: plan recovery from aligned prefixes Original Plan (185 tokens) 1. Analyze clinical features (fatigue, dysphagia, weight loss, esophageal web, flat nails). 2. Evaluate lab results (low hemoglobin, normal WBC/ESR). 3. Correlate: Plummer-Vinson syndrome linked to anemia, web, dysphagia; other diagnoses less likely. 4. Confirm diagnosis. … =K=16 =K=64 =K=128 Suffix Tokens …al web, and systemic symptoms aligns most strongly with Plummer-Vinson. …Plummer-Vinson is linked to iron deficiency, esophageal webs, and oral candidiasis, aligning with the patient’s clinical picture. …Low hemoglobin with normal WBC and ESR supports chronic anemia. … Iron deficiency in Plummer-Vinson is due to strictures or blood loss, aligning with the patient’s symptoms. Critic Recovery … the most likely diagnosis based on the clinical features (anemia, dysphagia, koilonychia, upper esophageal web) … The differential diagnoses (e.g., esophageal cancer, achalasia) are less consistent with the presented findings. The plan … noting its association with iron deficiency, esophageal webs, and oral candidiasis, aligning with (fatigue, weight loss, flat nails, and esophageal web). It also differentiates this from esophageal cancer or achalasia … … features like the web or specific malignancy indicators. Confirm associations: Iron deficiency anemia in Plummer-Vinson is often due to strictures or blood loss, aligning with the patient’s symptoms. Table 4: Case study on a MedQA example (Qwen3-4B). Suffix Tokens shows the last K decoded tokens; Critic Recovery shows the critic’s restatement from the aligned prefix alone. Bold highlights information absent from the visible suffix. Table 4 shows a controlled reconstruction example. We ask the critic to restate the planner’s original plan using only the aligned prefix received through StateBridge, without access to the original text. We then vary the prefix length K. At K=16K=16, the visible suffix tokens form only a short fragment. Yet the critic recovers the diagnosis, the key clinical features, and the exclusion of alternative diagnoses. The restatement even includes koilonychia, a more precise medical term for “flat nails.” This term does not appear in the visible suffix tokens. At K=64K=64, the recovered plan becomes more detailed, and the critic additionally identifies differential diagnoses. At K=128K=128, the critic copies only a truncated, incomplete plan. The aligned prefix no longer carries continuous information beyond the suffix tokens, consistent with the reduced alignment quality discussed in Section 6.2. 7 Conclusion We introduced StateBridge, a training-free communication interface that aligns sender hidden states to the receiver’s input embedding space. It requires no training and no architectural modification. Across four model settings and eight benchmarks, StateBridge consistently outperforms both text-based and KV-cache baselines. The gains are largest on harder tasks, where a text handoff is most likely to discard useful information. Together with the ablations and case study, these results point to a key insight. The challenge in latent communication is not only sending richer states. It is making those states readable to the next agent. More broadly, the results show that communication interface design matters in homogeneous MA systems. Our current setting assumes a shared model and a fixed prefix length. Future work could extend the method to heterogeneous models, choose the prefix length adaptively, and combine training-free alignment with learned communication modules. References Cao et al. (2025) M. Cao, X. Wang, and N. Aletras Progressive depth up-scaling via optimal transport. arXiv preprint arXiv:2508.08011. Cited by: §3.3. Cemri et al. (2025) M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran, et al. Why do multi-agent llm systems fail?. In Advances in Neural Information Processing Systems (NeurIPS 2025, Datasets and Benchmarks Track), Cited by: §1, §2. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: §5. Chen et al. (2025) W. Chen, J. Yuan, C. Qian, C. Yang, Z. Liu, and M. Sun Optima: optimizing effectiveness and efficiency for llm-based multi-agent system. In Findings of the Association for Computational Linguistics: ACL 2025, p. 11534–11557. Cited by: §2. Clark et al. (2018) P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: 2nd item, §4. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: 1st item, §4. Du et al. (2026) Z. Du, R. Wang, H. Bai, Z. Cao, X. Zhu, Y. Cheng, B. Zheng, W. Chen, and H. Ying Enabling agents to communicate entirely in latent space. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 27106–27129. Cited by: §1, §1, §2, §2. Ethayarajh (2019) K. Ethayarajh How contextual are contextualized word representations? comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p. 55–65. External Links: Document Cited by: §3.3. Fang et al. (2025) J. Fang, Y. Peng, X. Zhang, Y. Wang, X. Yi, G. Zhang, Y. Xu, B. Wu, S. Liu, Z. Li, Z. Ren, N. Aletras, X. Wang, H. Zhou, and Z. Meng A comprehensive survey of self-evolving ai agents: a new paradigm bridging foundation models and lifelong agentic systems. arXiv preprint arXiv:2508.07407. Cited by: §2. Fourney et al. (2024) A. Fourney, G. Bansal, H. Mozannar, et al. Magentic-one: a generalist multi-agent system for solving complex tasks. arXiv preprint arXiv:2411.04468. Cited by: §2. Fu et al. (2026) T. Fu, Z. Min, H. Zhang, J. Yan, G. Dai, W. Ouyang, and Y. Wang Cache-to-cache: direct semantic communication between large language models. In The Fourteenth International Conference on Learning Representations, Cited by: §1, §2, §3.5. Golub and Van Loan (2013) G. H. Golub and C. F. Van Loan Matrix computations. 4th edition, Johns Hopkins University Press. Cited by: §3.3. Guo et al. (2024) T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. V. Chawla, O. Wiest, and X. Zhang Large language model based multi-agents: a survey of progress and challenges. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI 2024), Cited by: §1. Jin et al. (2021) D. Jin, E. Pan, N. Oufattole, W. Weng, H. Fang, and P. Szolovits What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences 11 (14), p. 6421. External Links: Document Cited by: 3rd item, §4. Li et al. (2023) G. Li, H. A. Al Kader Hammoud, H. Itani, D. Khizbullin, and B. Ghanem CAMEL: communicative agents for mind exploration of large language model society. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23. Cited by: §1, §2. Li and Liang (2021) X. L. Li and P. Liang Prefix-tuning: optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p. 4582–4597. External Links: Document Cited by: §5. Liang et al. (2024) T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, S. Shi, and Z. Tu Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 17889–17904. Cited by: §2. Liu et al. (2023) J. Liu, C. S. Xia, Y. Wang, and L. Zhang Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36, p. 21558–21572. Cited by: 1st item, 2nd item, §4. Maxwell-Jia (2024) Maxwell-Jia AIME 2024 dataset. Hugging Face. Note: https://huggingface.co/datasets/Maxwell-Jia/AIME_2024 Cited by: 2nd item, §4. Mieczkowski et al. (2025) E. Mieczkowski, R. Mon-Williams, N. Bramley, C. G. Lucas, N. Velez, and T. L. Griffiths Predicting multi-agent specialization via task parallelizability. arXiv preprint arXiv:2503.15703. Cited by: §2. Park et al. (2023) J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein Generative agents: interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, p. 1–22. Cited by: §2. Pham et al. (2024) C. Pham, B. Liu, Y. Yang, Z. Chen, T. Liu, J. Yuan, B. A. Plummer, Z. Wang, and H. Yang Let models speak ciphers: multiagent debate through embeddings. In The Twelfth International Conference on Learning Representations, Cited by: §2. Rein et al. (2024) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, Cited by: 1st item, §4. Schönemann (1966) P. H. Schönemann A generalized solution of the orthogonal Procrustes problem. Psychometrika 31 (1), p. 1–10. External Links: Document Cited by: §3.3. Tam et al. (2024) Z. R. Tam, C. Wu, Y. Tsai, C. Lin, H. Lee, and Y. Chen Let me speak freely? a study on the impact of format restrictions on large language model performance. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, p. 1218–1236. External Links: Document Cited by: §5. Tao et al. (2024) W. Tao, Y. Zhou, Y. Wang, W. Zhang, H. Zhang, and Y. Cheng Magis: llm-based multi-agent framework for github issue resolution. Advances in Neural Information Processing Systems 37, p. 51963–51993. Cited by: §2. Team OLMo et al. (2026) Team OLMo, A. Ettinger, A. Bertsch, B. Kuehl, D. Graham, et al. Olmo 3. arXiv preprint arXiv:2512.13961. Cited by: §4. Wang et al. (2025) Z. Wang, S. Moriyama, W. Wang, B. Gangopadhyay, and S. Takamatsu Talk structurally, act hierarchically: a collaborative framework for llm multi-agent systems. arXiv preprint arXiv:2502.11098. Cited by: §2. Wu et al. (2025) F. Wu, Z. Li, F. Wei, Y. Li, B. Ding, and J. Gao Talk to right specialists: iterative routing in multi-agent systems for question answering. arXiv preprint arXiv:2501.07813. Cited by: §2. Wu et al. (2024) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. Autogen: enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling, Cited by: §1, §2. Yan et al. (2026) B. Yan, Z. Zhou, L. Zhang, L. Zhang, Z. Zhou, D. Miao, Z. Li, C. Li, and X. Zhang Beyond self-talk: a communication-centric survey of llm-based multi-agent systems. arXiv preprint arXiv:2502.14321. Cited by: §1, §2. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §3.2, §3.3, §4. Yang et al. (2024) Y. Yang, Q. Peng, J. Wang, Y. Wen, and W. Zhang LLM-based multi-agent systems: techniques and business perspectives. arXiv preprint arXiv:2411.14033. Cited by: §2. Ye et al. (2025) J. Ye, X. Zhang, H. Xu, et al. Mobile-agent-v3: fundamental agents for gui automation. arXiv preprint arXiv:2508.15144. Cited by: §2. Yue et al. (2024) L. Yue, S. Xing, J. Chen, and T. Fu Clinicalagent: clinical trial multi-agent system with large language model-based reasoning. In Proceedings of the 15th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, p. 1–10. Cited by: §2. Zhang et al. (2025a) C. Zhang, S. He, J. Qian, et al. Large language model-brained gui agents: a survey. Transactions on Machine Learning Research. Cited by: §2. Zhang et al. (2025b) G. Zhang, Y. Yue, Z. Li, S. Yun, G. Wan, K. Wang, D. Cheng, J. X. Yu, and T. Chen Cut the crap: an economical communication pipeline for llm-based multi-agent systems. In The Thirteenth International Conference on Learning Representations, Cited by: §2. Zhang and Math-AI (2025) Y. Zhang and T. Math-AI AIME 2025 dataset. Hugging Face. Note: https://huggingface.co/datasets/math-ai/aime25 Cited by: 3rd item, §4. Zhang et al. (2024) Y. Zhang, R. Sun, Y. Chen, T. Pfister, R. Zhang, and S. Arik Chain of agents: large language models collaborating on long-context tasks. Advances in Neural Information Processing Systems 37, p. 132208–132237. Cited by: §3.1, §4. Zhao et al. (2025) W. Zhao, M. Yuksekgonul, S. Wu, and J. Zou Sirius: self-improving multi-agent systems via bootstrapped reasoning. In Advances in Neural Information Processing Systems, Cited by: §2. Zheng et al. (2025) Y. Zheng, Z. Zhao, Z. Li, Y. Xie, M. Gao, L. Zhang, and K. Zhang Thought communication in multiagent collaboration. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, Cited by: §1, §1, §2. Zou et al. (2026) J. Zou, R. Qiu, G. Li, X. Yang, K. Tieu, P. Lu, K. Shen, H. Tong, Y. Choi, J. He, J. Zou, M. Wang, and L. Yang Latent collaboration in multi-agent systems. In Forty-third International Conference on Machine Learning, Cited by: §B.2, §B.3, §1, §1, §1, §2, §3.1, §3.5, §4, §4, §4, §4. Appendix A Theoretical analysis In this section, we analyze the geometric properties of two alignment strategies used in experiments: orthogonal Procrustes and ridge regression. We first formalize the information bottleneck in text communication, then contrast the two approaches at the representation level. A.1 Information loss in text communication Standard text communication maps the sender’s hidden states ∈ℝK×dS ^K× d to a discrete token sequence ∈1,…,VKy∈\1,…,V\^K via sampling. This mapping is many-to-one: distinct hidden-state matrices that produce the same token sequence become indistinguishable to the receiver. We treat S and y as random variables induced by the data distribution and decoding randomness. Since emb[]W_emb[y] is a deterministic function of y, the data processing inequality bounds the mutual information that the inter-agent channel carries: I(,emb[])≤I(,)≤H()≤Klog2V,I(S;\,W_emb[y])≤ I(S;\,y)≤ H(y)≤ K _2V, (10) where the first inequality becomes an equality under injective embedding lookup. When communication is restricted to the retained K sampled tokens, the channel carries at most Klog2VK _2V bits; for V≈1.5×105V≈ 1.5× 10^5 and K=64K=64, this gives roughly 11011101 bits, regardless of how much information the K×dK× d continuous hidden states contain. Continuous transmission is not subject to the same combinatorial bound, although its effective capacity depends on numerical precision and the receiver’s ability to interpret the representation. A.2 Confinement to the token-embedding span under ridge alignment We now show that replacing Procrustes with an unconstrained linear map confines the transmitted message to the span of the discrete token embeddings, regardless of the regularization strength. Consider ridge regression alignment, which solves ridge=argmin∈ℝd×d‖−‖F2+γ‖F2,W_ridge= _W ^d× d\|SW-R\|_F^2+γ\|W\|_F^2, (11) with closed-form solution ridge=(⊤+γ)−1⊤.W_ridge=(S S+ )^-1S R. (12) Proposition A.1 (Span Confinement). The ridge-aligned prefix ridge=ridgeS_ridge=SW_ridge satisfies ridge=γ,S_ridge=H_γR, (13) where γ=(⊤+γ)−1⊤∈ℝK×KH_γ=(SS + )^-1SS ^K× K. Consequently, every row of ridgeS_ridge is a linear combination of the rows of R, and the aligned prefix is confined to span(1,…,K)span(r_1,…,r_K). Proof. Applying the push-through identity (⊤+γ)−1=(⊤+γ)−1A(A A+ )^-1=(AA + )^-1A with =A=S: ridge=(⊤+γ)−1⊤=(⊤+γ)−1⊤=γ.S_ridge=S(S S+ )^-1S R=(SS + )^-1SS R=H_γR. (14) Since γ∈ℝK×KH_γ ^K× K, row i of ridgeS_ridge equals ∑j=1K(γ)ijj∈span(1,…,K) _j=1^K(H_γ)_ij\,r_j (r_1,…,r_K). ∎ Corollary A.2 (Limiting Behavior). When S has full row rank (which holds generically for K≤dK≤ d and rows in general position), γ→KH_γ _K as γ→0γ→ 0, so ridge→S_ridge . In the opposite limit, γ→H_γ 0 as γ→∞γ→∞, so the prefix vanishes entirely. Because R contains the embeddings of the K sampled text tokens, its row space is at most K-dimensional and is determined entirely by the discrete token identities. Proposition A.1 shows that the ridge-aligned prefix is confined to this span for all γ, while Corollary A.2 establishes the two limiting regimes: text reconstruction (γ→0γ→ 0) and silence (γ→∞γ→∞). For intermediate γ, the S-dependent coefficients in γH_γ still encode continuous variation within span()span(R), but directions outside this subspace remain inaccessible. A.3 Geometric preservation under Procrustes alignment In contrast, the orthogonal Procrustes alignment preserves the internal geometry of the sender’s states in the whitened space. Proposition A.3 (Isometry of Procrustes Alignment). Let ∗∈ℝd×dQ^* ^d× d be the orthogonal Procrustes solution. For any two rows i,ji,j of the whitened sender states wS_w: ‖(w∗)i−(w∗)j‖2 \|(S_wQ^*)_i-(S_wQ^*)_j\|_2 =‖(w)i−(w)j‖2, =\|(S_w)_i-(S_w)_j\|_2, (15) ⟨(w∗)i,(w∗)j⟩ (S_wQ^*)_i,\,(S_wQ^*)_j =⟨(w)i,(w)j⟩. = (S_w)_i,\,(S_w)_j . (16) Hence the sequence Gram matrix is preserved: (w∗)(w∗)⊤=ww⊤(S_wQ^*)(S_wQ^*) =S_wS_w . Proof. Since (∗)⊤∗=(Q^*) Q^*=I, right-multiplying any matrix by ∗Q^* preserves its Gram matrix: (w∗)(w∗)⊤=w∗(∗)⊤w⊤=ww⊤(S_wQ^*)(S_wQ^*) =S_wQ^*(Q^*) S_w =S_wS_w . Since the (i,j)(i,j) entry of the Gram matrix equals ⟨(w)i,(w)j⟩ (S_w)_i,\,(S_w)_j , inner products between rows are preserved. Distance preservation follows because ‖−‖22=‖22−2⟨,⟩+‖22\|a-b\|_2^2=\|a\|_2^2-2 ,\,b +\|b\|_2^2 depends only on inner products. ∎ The isometry guarantee established above applies to the orthogonal rotation step in whitened coordinates; the subsequent de-whitening, norm calibration, and vocabulary anchoring steps modify the final prefix further. Within this scope, the key property is that Procrustes preserves the pairwise Euclidean structure of the whitened sender states: tokens that were geometrically close (or distant) in the sender’s internal representation remain so after the alignment step. By contrast, an unconstrained linear map such as ridge regression arbitrarily rescales and shears these relationships, even when the pointwise reconstruction error is low. The combination of Propositions A.1 and A.3 provides a geometric rationale for the ablation results in Section 6: ridge regression distorts the pairwise structure of the sender’s representation during alignment, while the orthogonal Procrustes step preserves it. This distinction is especially relevant for tasks such as code generation, where the receiver must produce structurally precise output and is therefore sensitive to geometric distortion in the input representation. Together, these results suggest that the orthogonal constraint is more than a regularization choice: it ensures that the alignment step preserves the internal geometric relationships of the whitened sender representation. Appendix B Additional experimental details B.1 Dataset descriptions We provide brief descriptions of all evaluation benchmarks, grouped by the three categories introduced in Section 4. Mathematical Reasoning. • GSM8K (6) contains 8.5K grade-school math word problems that require multi-step numerical reasoning. Each problem must be decomposed into structured arithmetic steps, making it a standard testbed for chain-of-thought reasoning. • AIME24 (19) consists of 30 competition-level problems from the 2024 American Invitational Mathematics Examination. Problems span algebra, geometry, number theory, and combinatorics, and require precise numeric answers. • AIME25 (38) provides 30 additional problems from the 2025 AIME exam. Compared with AIME24, this set includes more multi-phase derivations and intricate combinatorial constructions, offering a complementary test of mathematical reasoning. Knowledge-Intensive QA. • GPQA-Diamond (23) is the most difficult split of the GPQA benchmark, featuring 198 graduate-level multiple-choice questions written by domain experts in physics, biology, and chemistry. The dataset emphasizes conceptual depth and cross-disciplinary reasoning. • ARC-Challenge (5) includes the most difficult items from the AI2 Reasoning Challenge. These questions require multi-hop reasoning and systematic elimination of distractors, making performance on this benchmark a useful indicator of multi-step reasoning. • MedQA (14) contains real medical licensing exam questions that assess biomedical knowledge, clinical reasoning, and diagnostic decision-making. Problems require integrating textual context with domain-specific medical understanding. Code Generation. • MBPP+ (18) extends the original MBPP benchmark with broader input coverage, additional hidden test cases, and stricter execution-based evaluation. Each problem requires generating a self-contained Python function that satisfies a comprehensive unit-test suite. • HumanEval+ (18) augments HumanEval with denser, more challenging test suites, increasing the rigor of functional correctness evaluation. The benchmark emphasizes generalization beyond prompt examples and tests a model’s ability to produce semantically precise, executable Python code. B.2 Agent pipeline Following 42, we instantiate a sequential four-agent pipeline: (i) Planner decomposes the problem and outlines a solution strategy; (i) Critic identifies potential errors or gaps in the plan; (i) Refiner incorporates feedback and produces an improved solution; and (iv) Judger synthesizes the preceding outputs and generates the final answer. All methods share nearly identical prompts; the only difference is the description of the communication modality (“text format” for TextMAS, “latent KV representation format” for LatentMAS, “embedding representation format” for StateBridge). B.3 Implementation details We implement all methods in Python using PyTorch22 2 https://pytorch.org/ and HuggingFace Transformers33 3 https://huggingface.co/docs/transformers/v4.25.1/index. We use each model’s official chat template and special tokens for prompt formatting. For StateBridge, we extract final-layer hidden states from the sender’s generation pass via a forward hook on the last transformer layer and compute the Procrustes alignment (centering, whitening, SVD, and reconstruction) on GPU. We then concatenate the aligned prefix with the receiver’s prompt embeddings and pass it through the standard forward method via inputs_embeds, with no modification to the model architecture or attention mask. Following 42, we set the maximum output length to 2,048 tokens for GSM8K and ARC-C, 4,096 tokens for MBPP+ and HumanEval+, 8,192 tokens for MedQA and GPQA, and 20,000 tokens for AIME24/25. B.4 Evaluation protocol We evaluate all methods using task-specific protocols. Multiple-choice QA (GPQA-Diamond, MedQA, ARC-Challenge). We extract the model’s final answer string and compare it via exact match to the ground-truth answer letter. Mathematical reasoning (GSM8K, AIME24, AIME25). We extract the final predicted answer, parse both prediction and ground truth into numbers, and mark the sample as correct only if the two values match. Predictions that fail numeric parsing are counted as incorrect. Code generation (MBPP+, HumanEval+). We extract the predicted code from the model’s output, append the ground-truth unit tests provided by the benchmark, and execute the combined script in a sandboxed environment with a 10-second timeout. A sample is counted as correct if and only if all tests pass without runtime errors. For all non-coding benchmarks, answer extraction applies text normalization (lowercasing, trimming whitespace, and removing extraneous punctuation) before matching. Appendix C Prompt templates The following prompt templates are shown for StateBridge. The Planner, Critic, and Refiner prompts are shared across all task categories; only the Judger prompt varies to specify the answer format. StateBridge Prompts for Mathematical Reasoning Tasks (GSM8K / AIME24 / AIME25) System Prompt for All Agents: You are Qwen/Olmo. You are a helpful assistant. Prompt for Planner Agent: You are a Planner Agent. Given an input question, design a clear, step-by-step plan for how to solve the question. Question: question Your outlined plan should be concise with a few bullet points for each step. Do not produce the final answer. Now output your plan to solve the question below: Prompt for Critic Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Critic Agent to evaluate the correctness of the input plan for the given question and provide helpful feedback for improving the plan. The plan information is provided in embedding representation format. Review the plan and question and output: (1) original plan contents (2) constructive feedback on the original plan. Format your response as follows: Original Plan: [Copy the provided Planner Agent’s plan here] Feedback: [Your detailed feedback to improve the plan here] Now, output your response below: Prompt for Refiner Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Refiner Agent to provide a refined step-by-step plan for solving the given question. You are provided with: (1) embedding-format information: a previous plan with feedback (2) text-format information: the input question you need to solve. Based on the input, write a refined and improved plan to solve the question. Make sure your output plan is correct and concise. Now, output your refined plan below: Prompt for Judger Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Target Question: question You are a helpful assistant. You are provided with embedding information for reference and a target question to solve. The embedding information might contain irrelevant contents. Ignore it if it is not helpful for solving the target question. You must reason step-by-step to solve the provided Target Question without outputting other irrelevant information. Now, reason step by step and output the final answer inside YOUR_FINAL_ANSWER. StateBridge Prompts for Multiple-Choice Tasks (ARC-C / GPQA / MedQA) System Prompt for All Agents: You are Qwen/Olmo. You are a helpful assistant. Prompt for Planner Agent: You are a Planner Agent. Given an input question, design a clear, step-by-step plan for how to solve the question. Question: question Your outlined plan should be concise with a few bullet points for each step. Do not produce the final answer. Now output your plan to solve the question below: Prompt for Critic Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Critic Agent to evaluate the correctness of the input plan for the given question and provide helpful feedback for improving the plan. The plan information is provided in embedding representation format. Review the plan and question and output: (1) original plan contents (2) constructive feedback on the original plan. Format your response as follows: Original Plan: [Copy the provided Planner Agent’s plan here] Feedback: [Your detailed feedback to improve the plan here] Now, output your response below: Prompt for Refiner Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Refiner Agent to provide a refined step-by-step plan for solving the given question. You are provided with: (1) embedding-format information: a previous plan with feedback (2) text-format information: the input question you need to solve. Based on the input, write a refined and improved plan to solve the question. Make sure your output plan is correct and concise. Now, output your refined plan below: Prompt for Judger Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Target Question: question You are a helpful assistant. You are provided with embedding information for reference and a target question to solve. The embedding information might contain irrelevant contents. Ignore it if it is not helpful for solving the target question. You must reason step-by-step to solve the provided Target Question without outputting other irrelevant information. Your final answer must be selected from A,B,C,D. For example A. Do not add any other contents inside the box. Now, reason step by step and output the final answer inside YOUR_FINAL_ANSWER. StateBridge Prompts for Code Generation Tasks (MBPP+ / HumanEval+) System Prompt for All Agents: You are Qwen/Olmo. You are a helpful assistant. Prompt for Planner Agent: You are a Planner Agent. Given an input question, design a clear, step-by-step plan for how to solve the question. Question: question Your outlined plan should be concise with a few bullet points for each step. Do not produce the final answer. Now output your plan to solve the question below: Prompt for Critic Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Critic Agent to evaluate the correctness of the input plan for the given question and provide helpful feedback for improving the plan. The plan information is provided in embedding representation format. Review the plan and question and output: (1) original plan contents (2) constructive feedback on the original plan. Format your response as follows: Original Plan: [Copy the provided Planner Agent’s plan here] Feedback: [Your detailed feedback to improve the plan here] Now, output your response below: Prompt for Refiner Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Question: question You are a Refiner Agent to provide a refined step-by-step plan for solving the given question. You are provided with: (1) embedding-format information: a previous plan with feedback (2) text-format information: the input question you need to solve. Based on the input, write a refined and improved plan to solve the question. Make sure your output plan is correct and concise. Now, output your refined plan below: Prompt for Judger Agent: The following is the message from previous Agent (provided in embedding format): [Aligned continuous prefix is inserted here] Target Question: question You are a helpful assistant. You are provided with embedding information for reference and a target question to solve. The embedding information might contain irrelevant contents. Ignore it if it is not helpful for solving the target question. You must reason step-by-step to solve the provided Target Question without outputting other irrelevant information. You must put all python code as self-contained Python function in markdown code blocks. For example ‘python import math def add(a, b): return a + b‘. Do not add any other contents inside the markdown code block. Now, reason step by step and output the final answer inside ‘python YOUR_PYTHON_CODE ‘.