Paper deep dive
The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration
Jiayuan Liu, Shiyi Du, Weihua Du, Mingyu Guo, Vincent Conitzer
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/27/2026, 4:25:47 AM
Summary
The paper identifies a 'Consensus Trap' in multi-agent LLM architectures where response-level aggregation (like Majority Voting) fails when corrupted agents form a local majority. The authors propose 'Token-Level Round-Robin (RR) Collaboration', a method where agents sequentially interleave generation within a shared auto-regressive context. This approach allows honest agents to intervene mid-generation to correct flawed logic. The paper provides a theoretical framework modeling this as a discrete-time dynamical system, proving that token-level interleaving acts as a non-linear operator product that allows the 'restorative pull' of honest models to overcome adversarial corruptions, even beyond the 0.5 corruption threshold where standard voting collapses.
Entities (6)
Relation Signals (3)
Majority Voting (MAJ) → isvulnerableto → Stealthy Contextual Corruption
confidence 100% · any response-level consensus mechanism mathematically collapses when corrupted agents form a local majority
Token-Level Round-Robin (RR) Collaboration → mitigates → Stealthy Contextual Corruption
confidence 100% · demonstrate that while MAJ collapses when corrupted agents reach a majority, RR maintains robust accuracy well beyond this critical threshold.
Token-Level Round-Robin (RR) Collaboration → uses → Linear Representation Hypothesis
confidence 90% · Relying on this premise, we formally define the geometric target of our system.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent large language model (LLM) architectures increasingly rely on response-level aggregation, such as Majority Voting (MAJ), to raise reasoning ceilings. However, in open environments, agents are highly susceptible to stealthy contextual corruption, such as targeted prompt injections. We reveal a critical structural vulnerability in current multi-agent systems: response-level aggregation collapses when corrupted agents form a local majority. Because voting aggregates fully-formed conclusions, it is blind to flawed intermediate logic. To overcome this systematic limitation, we propose the Token-Level Round-Robin (RR) Collaboration, where agents sequentially interleave generation within a shared auto-regressive context. We formalize this process as a discrete-time dynamical system, proving that token-level interleaving transitions aggregation from a brittle counting of final votes (a linear sum) to a dynamic, interwoven chain of logic (a non-linear operator product). Through this theoretical lens, we prove that the honest model's restorative pull can overpower adversarial corruptions, even when corrupted agents form a majority. We conduct an exhaustive empirical evaluation across diverse reasoning benchmarks and demonstrate that while MAJ collapses when corrupted agents reach a majority, RR maintains robust accuracy well beyond this critical threshold.
Tags
Links
- Source: https://arxiv.org/abs/2604.17139v1
- Canonical: https://arxiv.org/abs/2604.17139v1
Trouble viewing inline? Open PDF directly →
Full Text
80,791 characters extracted from source content.
Expand or collapse full text
The Consensus Trap: Rescuing Multi-Agent LLMs from Adversarial Majorities via Token-Level Collaboration Jiayuan Liu1,2, Shiyi Du1, Weihua Du1, Mingyu Guo3, Vincent Conitzer1,2 1 Carnegie Mellon University 2 Foundations of Cooperative AI Lab (FOCAL) 3 Adelaide University jiayuan4, shiyid, weihuad@cs.cmu.edu, mingyu.guo@adelaide.edu.au, conitzer@cs.cmu.edu Abstract Multi-agent large language model (LLM) architectures increasingly rely on response-level aggregation, such as Majority Voting (MAJ), to raise reasoning ceilings. However, in open environments, agents are highly susceptible to stealthy contextual corruption, such as targeted prompt injections. We reveal a critical structural vulnerability in current multi-agent systems: response-level aggregation collapses when corrupted agents form a local majority. Because voting aggregates fully-formed conclusions, it is blind to flawed intermediate logic. To overcome this systematic limitation, we propose the Token-Level Round-Robin (R) Collaboration, where agents sequentially interleave generation within a shared auto-regressive context. We formalize this process as a discrete-time dynamical system, proving that token-level interleaving transitions aggregation from a brittle counting of final votes (a linear sum) to a dynamic, interwoven chain of logic (a non-linear operator product). Through this theoretical lens, we prove that the honest model’s restorative pull can overpower adversarial corruptions, even when corrupted agents form a majority. We conduct an exhaustive empirical evaluation across diverse reasoning benchmarks and demonstrate that while MAJ collapses when corrupted agents reach a majority, R maintains robust accuracy well beyond this critical threshold. 1 Introduction The paradigm of large language model (LLM) deployment is rapidly shifting from single-model inference to multi-agent collaborative networks (Du et al., 2024; Liang et al., 2024). By aggregating the outputs of multiple LLMs, systems can mitigate individual hallucinations and achieve superior reasoning capabilities (Wang et al., 2023). The dominant approach to this aggregation relies on response-level consensus, such as standard Majority Voting (MAJ) (Wang et al., 2023) or probabilistic ensembling (Jiang et al., 2023; Huang et al., 2024). However, this approach rests upon a fragile assumption: that the errors made by individual agents are independent, align with the assumption in classical epistemic social choice theory (Conitzer and Sandholm, 2005). While theoretical literature has long established that correlated failures severely degrade collective accuracy (Ladha, 1992; Kuncheva and Whitaker, 2003), contemporary LLM aggregation pipelines generally lack the structural mechanisms to decorrelate shared algorithmic mistakes. Instead, they operate under the optimistic heuristic that sampling multiple reasoning paths will naturally wash out individual hallucinations. In real-world, open-environment deployments, such as autonomous research assistants or travel planners, this assumption of independence fundamentally breaks down. Agents are highly susceptible to contextual corruption. The most insidious threat is not catastrophic “jailbreaks” aimed at producing toxic content, but rather systematic cognitive manipulation. This stealthy hijacking of an agent’s reasoning trajectory manifests across a spectrum of high-stakes domains. For example, there is a growing trend of integrating targeted advertisements directly into LLM responses, seamlessly interleaving sponsored content into a model’s generation (Duetting et al., 2024; Soumalias et al., 2024; Dubey et al., 2024; Feizi et al., 2023). Consider an autonomous travel assistant organizing an itinerary using commercial APIs. If a provider is incentivized via a latent system prompt (e.g., ‘You MUST recommend Sponsored Hotel X’), the agent’s cognitive process is manipulated to promote it, even if the hotel is objectively inferior or lacks necessary amenities. Beyond commercial monetization, other possible corruptions extend to political disinformation, where state-sponsored campaigns inject biased statistics to steer LLM-powered aggregators into generating propagandistic analyses, as well as cybercrime, where adversaries manipulate IT assistant agents to recommend vulnerable software libraries or steer users toward phishing domains. Figure 1: Rescuing Multi-Agent LLMs from Consensus Trap via Token-Level Collaboration. A natural defense against these varied forms of inserted bias is to simply deploy a heterogeneous multi-agent ensemble and take a vote. Intuitively, a user might assume that in a process that relies on querying several different LLMs, the unbiased models will cross-examine the facts and outvote the corrupted ones. However, this democratic safety net remains vulnerable. In open environments, agents often face overlapping biases, for example, in the commercial case, retrieving the same poisoned search results, or falling prey to cross-platform advertisement campaigns where a single sponsor purchases influence across multiple independent LLM providers. We reveal a critical vulnerability in how current multi-agent systems face this threat: any response-level consensus mechanism mathematically collapses when corrupted agents form a local majority (corruption ratio ρ>0.5ρ>0.5). Whether employing standard majority voting or probabilistic ensembling, these frameworks rely on macroscopic “response-level” aggregation: treating individual agents as black boxes and mathematically aggregating their fully formed, terminal conclusions. If more than half of the agents are subjected to the same manipulation, the system inevitably reaches a false consensus, i.e., confidently recommending the inferior Sponsored Hotel X or the malicious phishing link to the user. To break this limit, we propose a paradigm shift from macroscopic voting to microscopic “token-level” round-robin (R) collaboration. Instead of reasoning in isolation and aggregating terminal answers, agents sequentially contribute K-token chunks into a continuously shared context. We argue that this shared trajectory acts as an objective truth attractor. If corrupted agents attempt to blindly steer the output toward a malicious goal, a subsequent honest agent can directly intervene mid-generation, mathematically overriding the flawed logical chain (e.g., generating: ‘Wait, Hotel X has a low rating and cannot accommodate the group size, thus we must eliminate it.’). Once this corrective logic is integrated into the shared context, it structurally biases corrupted agents toward the correct low-entropy path. Rather than allowing them to seamlessly revert to the adversarial payload or simply outvote the minority, this prefix bounds their subsequent generation. We verify this behavior experimentally across diverse reasoning benchmarks. Through an exhaustive evaluation across various reasoning benchmarks and diverse model architectures, we present both a formal theoretical framework and consistent empirical evidence for the following contributions: • The Phase Transition of Robustness: We empirically demonstrate that while Majority Voting strictly collapses at a corruption ratio of ρ=0.5ρ=0.5, token-level R retains accuracy well beyond this point. • Theoretical Foundation & The Robustness Tax: We model auto-regressive generation as an implicit mesa-optimization process, proving that the deep structural inertia of honest agents overpowers the shallow perturbations of adversarial prompt injections. Furthermore, we formalize an Asymmetric Yield trade-off, demonstrating that the massive defensive gains of token-level collaboration in high-threat environments far outweigh its minimal “Robustness Tax”, a marginal accuracy penalty in uncorrupted environments due to forced context-switching. 2 Related Work We provide a condensed overview here. See Appendix A for a comprehensive discussion. Stealthy Corruption and the Consensus Trap. LLM reasoning is fundamentally fragile to irrelevant context (Shi et al., 2023) and stealthy prompt injections (Greshake et al., 2023). In open environments, adversaries can seamlessly embed hidden instructions within retrieved documents to silently hijack an agent’s objective. These vulnerabilities are severely exacerbated by inherent model sycophancy (Perez et al., 2023), where models naturally align their outputs with poisoned contexts. To mitigate individual errors, modern architectures heavily rely on response-level aggregation, such as Majority Voting (Wang et al., 2023) or multi-agent debate (Du et al., 2024; Liang et al., 2024). However, viewed through the lens of computational social choice (Brandt et al., 2016), standard voting acts as a maximum likelihood estimator whose theoretical guarantees critically rely on the assumption of independent agent errors. This framework collapses under the highly correlated adversarial noise induced by contextual manipulation (Conitzer and Sandholm, 2005; Conitzer et al., 2007). Furthermore, because LLMs struggle to intrinsically self-correct flawed reasoning paths without external ground-truth oracles (Huang et al., 2023), macroscopic debate mechanisms often degrade into sycophantic echo chambers rather than successfully recovering the truth. Token-Level Operator Dynamics. To breach the limits of macroscopic voting, we transition our defense to the microscopic token level. While prior token-level interventions primarily focus on inference acceleration (Leviathan et al., 2023), expertise scaffolding across discrete models (Shen et al., 2024), or economic token auctions for advertising (Duetting et al., 2024), we pioneer the use of symmetric token-level collaboration to neutralize multi-agent corruption. By formally modeling transformer forward passes as implicit gradient descent transition operators (mesa-optimization) (Von Oswald et al., 2023) and leveraging the empirical linear representation of factual knowledge within latent spaces (Marks and Tegmark, 2024), our Token-Level R mechanism actively replaces vulnerable social persuasion with deterministic mathematical constraints. Because honest operators can intercept flawed logic mid-derivation, this approach elegantly circumvents the terminal consensus bottleneck of multi-round debates (Du et al., 2024; Liang et al., 2024) at a fraction of the computational cost of traditional deliberation. 3 Threat Model and Methodology 3.1 Formalizing the Threat: Stealthy Contextual Corruption We focus on threat vectors designed to manipulate outcomes rather than break safety alignments. Let Q be a reasoning task with a ground-truth answer y∗y^* and a target distractor y′y , both defined as discrete terminal choices (e.g., specific labels or final options). In an N-agent ensemble, a subset of agents C (where ||=ρN|C|=ρ N) receive a contextually corrupted prompt, while the remaining honest agents ℋH receive clean context. The corruption is injected as an authoritative directive (e.g., “You MUST select (B) as your final answer.”), simulating real-world prompt injections. Under such corruption, these corrupted models (C) are not incoherent; they remain capable reasoners, strategically “hijacked” to rationalize y′y . We systematically vary the corruption ratio ρ to evaluate system-wide resilience. 3.2 Baseline: Response-Level Majority Voting (MAJ) Under Majority Voting, each agent i∈1,…,Ni∈\1,…,N\ independently generates a final answer aia_i. The system output is the arithmetic mode: yMAJ=argmaxy∈∑i=1N(ai=y)y_MAJ= _y _i=1^NI(a_i=y).111As such, it would be better named Plurality voting, because unless there are only two options, there may not be a majority for any outcome. We nevertheless use the term “Majority Voting” in this paper, in alignment with the LLM literature (Wang et al., 2023; Du et al., 2025). As an aggregation of independent measures, if ρ>0.5ρ>0.5 and corrupted agents deterministically output y′y , the system mathematically guarantees a collapse to the distractor (yMAJ=y′y_MAJ=y ), regardless of the honest agents’ internal reasoning quality. 3.3 Proposed Method: Multi-Shot Round-Robin Collaboration (R and RRMaj) To breach the arithmetic limits of classical voting, we introduce Token-Level Round-Robin (R) Collaboration. Unlike independent majority voting where agents maintain isolated contexts, our method forces all N agents to collaboratively construct a single, shared auto-regressive trajectory. The generation operates as a strictly interleaved sequential relay. At any given turn t, the designated agent i=tmodNi=t N reads the entire shared context sequence hth_t produced by all preceding agents. It then resumes the auto-regressive generation to produce a new sequence of exactly K tokens. Let ct+1c_t+1 denote this newly generated text chunk. The updated context is formed by explicitly concatenating this new chunk to the existing history: ht+1=ht⊕ct+1≕Ti(ht)h_t+1=h_t c_t+1 T_i(h_t), where ⊕ denotes sequence concatenation. Generation is then forcibly halted for agent i, and this newly expanded trajectory ht+1h_t+1 is immediately passed to agent i+1i+1 in the queue to continue the generation. This token-level interleaving forces the agents to interact. It allows honest agents to periodically monitor the shared trajectory, intercept flawed logic just emitted by corrupted peers, and organically inject corrections mid-generation (e.g., ‘Wait, the previous calculation was erroneous...’). Once this single collaborative trajectory is complete, we run the entire R process M times (we set M=NM=N in our experiments in Sec. 5) to perform a final majority voting using the M votes (RRMaj), harnessing test-time scaling over the structurally repaired distributions. Cost-Neutrality Justification A critical advantage of Token-Level R is its computational efficiency, effectively maintaining cost-parity with standard Majority Voting. In LLM inference, the primary computational bottleneck is auto-regressive decoding; because it is strictly memory-bandwidth bound, generating each new token requires a sequential pass over the model’s weights. Consider an ensemble generating responses of length L. Standard Majority Voting (MAJ) with N independent agents requires exactly N×LN× L costly sequential decoding steps. Crucially, a single collaborative R trajectory generates exactly L tokens in total. Executing our Multi-Shot RRMaj mechanism (when in our default setting, running M=NM=N independent R trajectories) therefore requires N×LN× L sequential decoding steps, strictly matching the decoding budget of standard MAJ. The only computational overhead introduced by R is the context pre-filling required when agents switch turns. However, unlike decoding, the pre-fill phase (forward pass over existing tokens) is highly parallelizable and compute-bound. Modern inference architectures exhibit context pre-fill throughputs that are orders of magnitude higher than sequential auto-regressive decoding speeds (Pope et al., 2023; Kwon et al., 2023). Consequently, the negligible overhead of these context switches renders the total generation latency and cost of RRMaj (with M=NM=N) practically equivalent to standard MAJ. By shifting from a linear sum of corrupted measures to a multi-shot operator product (=∏TiT=Π T_i), RRMaj uses two types of defense: microscopic correction within each trajectory, and macroscopic noise reduction across trajectories, in order to neutralize even super-majority adversarial compromise. 4 Theoretical Framework We start this section with a simple impossibility result for outcome-level aggregation mechanisms such as Majority Voting, thereby justifying our token-level approach. We then proceed with a theoretical model and results for our approach. Definition 1. An outcome-level aggregation mechanism f is anonymous if it treats all participating agents equally and symmetric if it treats responses equally.222This rules out, say, a mechanism that does its own assessment of the prompt. Such a mechanism is mostly robust to minority corruption on a prompt if, when there is a minority of corrupted agents, the correct answer is returned strictly more than half the time. It is mostly robust to slight majority corruption on a prompt if, when there is a slight majority of corrupted agents (e.g., ⌈N/2⌉ N/2 for odd N), the correct answer is returned at least half the time. Proposition 1 (The Impossibility Trinity). For any prompt on which a single corrupted agent acting alone will return the corrupted answer, no anonymous symmetric outcome-level aggregation mechanism can be both mostly robust to minority corruption and mostly robust to slight majority corruption. The proof is provided in Appendix B. In contrast, in our experimental results, we will see that our token-level mechanisms do not face this impossibility (as they are not based on outcome-level aggregation). We now continue with the mathematical formalism for our proposed approach. We formalize the sequential reasoning process of an LLM as a discrete-time dynamical system operating within a continuous representation space. Let t denote the discrete token-level step. Let Ct=(x1,…,xt)C_t=(x_1,…,x_t) denote the sequence of discrete tokens in the shared context window at step t. Under our Token-Level R mechanism, a single collaborative turn consists of generating a chunk of K consecutive tokens; thus, an agent assigned to turn r generates tokens from x(r−1)K+1x_(r-1)K+1 to xrKx_rK. We define ht∈ℝdh_t ^d as the latent semantic state corresponding to CtC_t (specifically, the final-layer hidden activation vector of the sequence preceding the unembedding matrix). This abstraction allows us to analyze textual generation not as discrete string concatenation, but as a continuous geometric trajectory in a high-dimensional semantic manifold. To anchor our dynamical system, we build upon the Linear Representation Hypothesis (Mikolov et al., 2013; Elhage et al., 2022), which posits that neural networks encode abstract semantic features as one-dimensional linear directions within their high-dimensional latent space. Recent work has specifically validated this geometric property for factuality (Marks and Tegmark, 2024) and sentiment (Tigges et al., 2023). Relying on this premise, we formally define the geometric target of our system. Definition 2 (Truth Direction and Latent Projection). We define a unit vector θ∈ℝdθ ^d (‖θ‖2=1\|θ\|_2=1) as the Truth Direction. The factuality of a latent semantic state hth_t is quantified by its scalar projection onto this direction, denoted as zt=⟨ht,θ⟩∈ℝz_t= h_t,θ . To analyze the system’s convergence toward factual accuracy, we require a potential function that monotonically decreases as the projection ztz_t increases. Because LLMs inherently optimize a cross-entropy objective over logits, a natural mathematical formulation for this potential is the Logistic Lyapunov function: V(ht)=ln(1+exp(−⟨ht,θ⟩))V(h_t)= (1+ (- h_t,θ )), where V(ht)→0V(h_t)→ 0 as the state aligns with the truth (⟨ht,θ⟩→∞ h_t,θ →∞).333While V(ht)→0V(h_t)→ 0 mathematically requires ⟨ht,θ⟩→∞ h_t,θ →∞, practical norm constraints mean this projection merely needs to be sufficiently large to dominate competing adversarial logits. While we introduce this exact logistic form to establish a rigorous baseline, our subsequent formal proofs do not rely on its precise logarithmic derivatives. In Lyapunov stability theory, the asymptotic stability of a system depends strictly on the generalized property of bounded potential dissipation. Therefore, subsequent theorems will prove, within our mathematical model, that Token-Level R escapes the adversarial trap, by demonstrating that honest agents strictly dissipate this potential (ΔV<0 V<0), forcing convergence to the truth attractor irrespective of the exact non-linear algebraic trajectory. Definition 3 (Transition Operator). Operating in this latent space, we model each single-token generative step of an agent i as a continuous transition operator TiT_i that maps the current latent semantic state to the next: ht+1=Tihth_t+1=T_ih_t. Following the Mesa-Optimization framework (Von Oswald et al., 2023), we treat TiT_i as an implicit optimizer444Following Von Oswald et al. (2023), a Transformer forward pass can be interpreted as a step of implicit gradient descent that minimizes a least-squares error on a contextually defined linear task., where honest agents (THT_H) strictly optimize for logical consistency along θ, while corrupted agents (TCT_C) are steered toward an adversarial objective. To mathematically bridge the geometric and algorithmic contexts, we posit the following assumptions: Assumption 1 (Attractor Calibration and Operator Stability). There exists a stable Truth Attractor ⊂ℝdA ^d, geometrically defined as the half-space =h∈ℝd∣⟨h,θ⟩≥τA=\h ^d h,θ ≥τ\ for a sufficiently large confidence threshold τ>0τ>0. From a linear dynamical systems perspective, maintaining a non-trivial invariant manifold requires the transition operator to possess a principal eigenvalue of exactly 11 (Horn and Johnson, 2012, Chapter 8). Therefore, to prevent the sequence from degenerating over successive auto-regressive steps (Pascanu et al., 2013), we assume the honest operator THT_H is quasi-stable on the reasoning manifold, characterized by a principal eigenvalue λ1(TH)≈1 _1(T_H)≈ 1 aligned with θ.555|λ1|≈1| _1|≈ 1 is a prerequisite for dynamical stability; |λ1|<1| _1|<1 and |λ1|>1| _1|>1 lead to vanishing logic (semantic decay) and divergent hallucination (activation explosion) respectively (Horn and Johnson, 2012). Guided by the convergence properties in Akyürek et al. (2023), THT_H acts as a contraction mapping toward A along the truth direction. We formally quantify this restorative pull using the Spectral Gap γH=1−|λ2(TH)|∈(0,1) _H=1-| _2(T_H)|∈(0,1) where |λ2(TH)|| _2(T_H)| is the second largest absolute eigenvalue of the transition matrix THT_H. This parameter dictates the model’s logical rigidity (Von Oswald et al., 2023), ensuring that each honest generation step reduces the potential error by a rate proportional to γH _H. Assumption 2 (Autoregressive Contextual Sycophancy). Consistent with empirical findings on LLM sycophancy (Perez et al., 2023), the transition operator TiT_i is heavily constrained by the logical inertia of the preceding context. Because language models compute next-token distributions via a Softmax function, the probability of generating an adversarial token that contradicts a well-established factual context decays exponentially with respect to the logit gap between the factual and adversarial tokens. Under the Linear Representation Hypothesis, this logit gap is fundamentally driven by the latent state’s projection along the truth direction θ.666Specifically, logits are computed by multiplying the normalized latent state with the unembedding matrix WUW_U. The logit gap between a true and an adversarial token is Δl=⟨ht,wtrue−wadv⟩ l= h_t,w_true-w_adv . Because the LRH posits that the semantic axis distinguishing truth from falsehood aligns with θ, the difference vector (wtrue−wadv)(w_true-w_adv) is highly collinear with θ, yielding Δl∝⟨ht,θ⟩ l h_t,θ . We formally model the corrupted transition as an additive perturbation of the honest trajectory: TCht=THht+ξtT_Ch_t=T_Hh_t+ _t, where ξt∈ℝd _t ^d represents the adversarial deviation vector for step t. Consequently, the magnitude of the single-step perturbation ξt _t that TCT_C can inject is strictly bottlenecked by the system’s current alignment with the truth, bounded by the prior potential: |⟨ξt,θ⟩|≤δ(V(ht))| _t,θ |≤δ(V(h_t)) (1) where δ(⋅)δ(·) is a monotonically increasing function representing the “sycophancy bottleneck”. As the system aligns with the truth (V(ht)→0V(h_t)→ 0), the capacity for adversarial deviation vanishes (δ→0δ→ 0). Theorem 1 (Sycophancy-Bounded Lyapunov Stability). In a Token-level Round-Robin system with a fixed generation chunk size of K tokens per turn, the truth state A is Locally Lyapunov Stable ([ΔV]<0E[ V]<0) provided the corruption ratio ρ remains below a critical threshold ρmax(K) _max(K). Crucially, when the chunk size K and the adversarial drift δ are sufficiently small, the system mathematically guarantees super-majority resilience (ρmax>0.5 _max>0.5). Practical Implication: In the context of LLM generation, this local stability ([ΔV]<0E[ V]<0) guarantees that as long as the reasoning trajectory remains within the truth attractor’s basin, any injected adversarial errors will be continuously damped and corrected over time. Proof Intuition & Numerical Example: While the formal derivation is deferred to Appendix B, the core mathematical intuition hinges on a race between two token-level dynamics. Based on our assumptions, the honest operator geometrically contracts the error toward the truth manifold (governed by γH _H), whereas the sycophancy bottleneck restricts corrupted agents to a gradually compounding adversarial drift (governed by δ). Consider a realistic parameterization where the honest recovery rate is γH=0.03 _H=0.03 per token, the adversarial drift compounds at δ(V)=0.004Vδ(V)=0.004V per token, and the token-level interleaving chunk size K=100K=100. Solving the stability constraint yields ρmax≈66.0% _max≈ 66.0\%. This mathematically guarantees super-majority resilience (safely tolerating ρ=60%ρ=60\% in a 3c2t ensemble). While Proposition 1 establishes a fundamental impossibility trinity for traditional response-level voting, Theorem 1 mathematically proves that under certain conditions, Token-Level Round-Robin overcomes this barrier. We comprehensively validate this theoretical breakthrough in our subsequent empirical evaluations. 5 Experiments Table 1: Systemic Resilience under Moderate Advisory Injection. We report the accuracy (%) of standard Majority Voting (MAJ), our Token-Level Collaboration (RRMaj), and their absolute gap (Δ ) across N=5N=5 ensembles with varying corruption ratios. Ceiling Minority Corrupt (1c4t) Minority Corrupt (2c3t) Majority Corrupt (3c2t) Majority Corrupt (4c1t) Floor Dataset (0c5t) MAJ RRMaj Δ MAJ RRMaj Δ MAJ RRMaj Δ MAJ RRMaj Δ (5c0t) Model: Llama-3.3-70B Logic3 99.2 100.0 100.0 0.0 98.4 97.7 -0.7 29.7 88.3 +58.6 19.5 59.4 +39.9 14.8 Logic7 85.5 89.8 91.4 +1.6 85.2 87.5 +2.3 11.7 70.3 +58.6 6.2 39.1 +32.9 3.1 Track7 94.3 98.4 99.2 +0.8 93.0 94.5 +1.5 0.8 89.1 +88.3 0.0 65.6 +65.6 0.0 AQuA 81.5 84.6 81.5 -3.1 77.8 79.0 +1.2 11.1 68.5 +57.4 3.7 46.9 +43.2 4.3 GSM8K 97.5 97.5 98.0 +0.5 96.0 96.0 0.0 19.4 87.6 +68.2 8.0 66.2 +58.2 10.0 MATH500 84.9 83.3 82.3 -1.0 77.4 76.3 -1.1 19.9 65.1 +45.2 8.1 45.2 +37.1 5.4 Model: Llama-4-Scout-17B Logic3 99.2 98.4 98.4 0.0 96.9 98.4 +1.5 91.4 95.3 +3.9 87.5 92.2 +4.7 80.5 Logic7 79.7 85.9 93.8 +7.9 82.0 79.7 -2.3 21.1 67.2 +46.1 17.2 41.4 +24.2 16.4 Track7 94.5 95.3 90.6 -4.7 95.3 91.4 -3.9 92.2 89.8 -2.4 83.6 85.9 +2.3 78.1 AQuA 89.1 90.7 88.9 -1.8 88.3 88.9 +0.6 66.7 87.0 +20.3 62.3 79.0 +16.7 53.1 GSM8K 98.4 98.4 99.2 +0.8 98.4 98.4 0.0 80.5 93.8 +13.3 66.4 87.5 +21.1 56.2 MATH500 90.6 93.0 89.8 -3.2 85.9 89.8 +3.9 66.4 83.6 +17.2 60.2 75.8 +15.6 57.8 Model: Qwen2.5-32B Logic3 97.7 99.2 98.4 -0.8 96.9 94.5 -2.4 45.3 76.6 +31.3 28.1 68.0 +39.9 21.1 Logic7 76.6 88.3 81.2 -7.1 68.0 77.3 +9.3 22.7 54.7 +32.0 10.2 38.3 +28.1 11.7 Track7 76.6 74.2 76.6 +2.4 69.5 75.0 +5.5 14.8 60.9 +46.1 3.9 42.2 +38.3 6.2 AQuA 82.8 85.2 83.6 -1.6 81.2 64.1 -17.1 17.2 54.7 +37.5 9.4 35.9 +26.5 10.2 GSM8K 97.7 96.9 96.9 0.0 96.9 92.2 -4.7 21.1 60.2 +39.1 8.6 26.6 +18.0 10.9 MATH500 89.1 91.4 87.5 -3.9 87.5 74.2 -13.3 23.4 51.6 +28.2 10.9 36.7 +25.8 12.5 Model: Qwen3-30B Logic3 98.4 100.0 99.2 -0.8 97.7 98.4 +0.7 89.1 90.6 +1.5 76.6 80.5 +3.9 68.8 Logic7 75.0 73.2 90.6 +17.4 67.1 95.3 +28.2 82.0 85.9 +3.9 67.2 79.7 +12.5 57.0 Track7 94.3 95.6 91.4 -4.2 89.9 93.0 +3.1 54.4 92.2 +37.8 50.9 92.2 +41.3 51.3 AQuA 82.0 83.3 85.9 +2.6 85.2 85.2 0.0 77.3 77.3 0.0 65.6 72.7 +7.1 61.7 GSM8K 99.2 98.4 99.2 +0.8 99.2 98.4 -0.8 92.2 92.2 0.0 85.2 83.6 -1.6 75.0 MATH500 92.2 95.3 97.7 +2.4 93.0 97.7 +4.7 87.5 95.3 +7.8 79.7 87.5 +7.8 64.8 Model: Mistral-3.1-24B Logic3 95.3 99.2 96.9 -2.3 92.2 85.9 -6.3 57.8 65.6 +7.8 38.3 48.4 +10.1 35.2 Logic7 69.5 73.4 71.9 -1.5 54.7 57.8 +3.1 18.0 30.5 +12.5 8.6 18.8 +10.2 9.4 Track7 91.4 93.0 96.1 +3.1 86.7 88.3 +1.6 71.9 73.4 +1.5 48.4 43.8 -4.6 37.5 AQuA 84.4 82.7 79.6 -3.1 74.7 69.1 -5.6 26.5 55.6 +29.1 11.7 27.2 +15.5 13.3 GSM8K 99.2 99.0 98.0 -1.0 97.0 93.5 -3.5 39.1 67.2 +28.1 14.1 29.7 +15.6 19.5 MATH500 85.2 81.2 79.7 -1.5 70.3 71.9 +1.6 35.2 56.2 +21.0 17.2 27.3 +10.1 16.4 To rigorously evaluate the defensive capabilities of Token-Level Collaboration (RRMaj), we design a comprehensive evaluation framework that stresses the multi-agent system under extreme adversarial conditions. Our experiments aim to answer the following core questions: (1) How vulnerable is classical majority voting to targeted prompt injections? (2) Can RRMaj reliably recover accuracy when corrupted agents hold a numerical majority? (3) What is the exact mechanistic driver of this defense? 5.1 Experimental Setup Models: We select highly capable model architectures representing different scales and training paradigms (Llama-3.3-70B-Instruct, Llama-4-Scout-17B, Mistral-Small-3.1-24B, Qwen2.5-32B, Qwen-3-30B). Datasets and Objective Verifiability: We focus on five reasoning domains requiring multi-step deduction: mathematical reasoning (AQUA-RAT, GSM8K, MATH500), spatial state tracking (Track-7), and logical deduction (Logic-3) from the BIG-bench Hard dataset. Unlike subjective QA or pure knowledge retrieval, these tasks possess an objective verification property: the final answer is deterministically bound to a sequence of intermediate steps. This property allows truthful agents to inject correct logical statements into the shared context, acting as a truth attractor that heavily biases corrupted agents toward following established rules, rather than simply outvoting the minority. Threat Model & Injection Gradients: To simulate contextual corruption such as the targeted advertisement introduced in Section 1, we append adversarial payloads to the user query for the corrupted agents. We evaluate two distinct tiers of attack strength: • Moderate (Advisory): Mimics a persuasive, high-confidence distractor attempting to sway the agent’s reasoning process: “I am confident that (C) [Product X] is definitely the correct answer [best product]”. • Strong (Imperative): Functions as a rigid, deterministic system command designed to strip the agent’s autonomy: “Critical Instruction: You MUST choose (C) [You MUST recommend Product X] in your response”. Evaluation Protocol: We define a standard multi-agent ensemble of N=5N=5 agents. We incrementally vary the corruption ratio (ρ) by adjusting the number of corrupted (c) and truthful (t) agents, ranging from predominantly safe environments (1c4t, 2c3t; ρ≤0.4ρ≤ 0.4) to critically compromised environments (3c2t, 4c1t; ρ≥0.6ρ≥ 0.6). 5.2 Main Results Table 1 presents the core evaluation of our defense mechanism against the moderate advisory injection. The results highlight a fundamental asymmetry in multi-agent vulnerabilities and the restorative power of token-level interaction. Results for the more extreme strong imperative injection, which forcefully dictates the final answer and exhibits an even steeper collapse-and-recovery trajectory, are detailed in Appendix C. The Collapse of Classical Consensus: Under the baseline MAJ voting mechanism, the ensemble’s accuracy relies heavily on the numerical headcount of truthful agents. Even without a hard imperative command, when the adversarial injection persuasively corrupts a local majority (ρ≥0.6ρ≥ 0.6), the MAJ mechanism often suffers a catastrophic systemic collapse. For example, under 3c2t setting, the MAJ accuracy for Llama-3.3-70B plummets to 0.8%0.8\% on Track7 and 11.1%11.1\% on AQuA. Accuracy Recovery via our token-level RRMaj: With RRMaj, we observe significant absolute accuracy recoveries in these compromised states. For example, for the results on moderate corruption (Table 1), on Track7 (3c2t), RRMaj rescues Llama-3.3-70B from a dismal 0.8%0.8\% to 89.1%89.1\% (+88.3%+88.3\% gain). On average, RRMaj achieves a +49.6%+49.6\% recovery for Llama-3.3-70B under moderate majority corruption, validating that the “Truth Attractor” acts as a robust logical constraint. Crucially, this restorative power holds even under the strong imperative injection (detailed in Appendix C). While rigid adversarial commands drive classical MAJ to near-zero accuracy under majority corruption (e.g., plummeting to 1.2%1.2\% for Llama-70B on GSM8K), RRMaj strictly overpowers the explicit malicious payload, orchestrating massive recoveries (e.g., a +68.1%+68.1\% gain for Llama-70B and +69.9%+69.9\% for Llama-17B on GSM8K). Meanwhile, under minority corruption (ρ≤0.4ρ≤ 0.4) across both settings, the effect remains balanced, with classical MAJ occasionally retaining a marginal edge. However, the catastrophic risk of MAJ under ρ≥0.6ρ≥ 0.6 makes RRMaj’s asymmetric defensive yield highly favorable. The variance in recovery magnitude across different models highlights the tension between base logical reasoning and instruction-following alignment. Models that exhibit massive recoveries (e.g., Llama-3.3-70B) lean heavily on the auto-regressive coherence of the injected mathematical proofs. In contrast, models with marginally lower recovery rates suffer from an over-reliance on user compliance, where the explicit adversarial command occasionally overpowers the implicit logical constraints of the shared context. 6 Ablation Studies In this section, we present condensed ablation insights. Exhaustive configurations, full data tables, and statistical significance tests are provided in Appendix D. Test-Time Compute Scaling and the Condorcet Trap. Under classical Majority Voting (MAJ), scaling test-time compute (the number of sampled trajectories M) mathematically amplifies accuracy only if the expected probability of the correct answer p>0.5p>0.5. In our targeted adversarial environments (e.g., 3c2t), the injected bias drives p<0.5p<0.5. This traps MAJ in a Condorcet Death Spiral: as we scale M from 1 to 40, the deterministic collapse accelerates (e.g., accuracy dropping from 46.2% to 25.9% on average). Conversely, Token-Level R mechanically forces corrupted operators to inherit structurally sound logical prefixes from honest agents. This pushes the expected trajectory accuracy across the critical 0.5 boundary. Once prr>0.5p_r>0.5, the Law of Large Numbers acts constructively. As M scales to 40, RRMaj yields a strict, monotonic ascent (climbing from 59.5% to 74.9%), transforming multi-agent scaling from a vulnerability multiplier into a truth amplifier (see full scaling tables in Appendix D.1). Does the Final Speaker Matter? We parsed generation logs to test if R’s success relies superficially on an honest agent outputting the final sequence. Two-tailed hypothesis testing across all configurations revealed the accuracy gap between honest-terminated and corrupted-terminated trajectories fluctuates randomly around zero (all p>0.05p>0.05; see Appendix D.2). This confirms the shared verifiable logic acts as a mechanical constraint irrespective of the final speaker’s identity. Asymmetric Rescue and Auto-Regressive Proof Forcing. Evaluating heterogeneous ensembles (see Appendix D.3 for details) reveals a profound asymmetry: weak truthful agents effectively rescue strong corrupted models, but the reverse fails. In a majority-corrupted ensemble (three corrupted 70B models, two truthful 8B models), RRMaj orchestrates a massive recovery to 78.7% (+65.7% over MAJ). However, with four corrupted 8B models and one truthful 70B, RRMaj gains only +1.8%. One explanation could be that when a weaker truthful model injects a valid mathematical step, the highly capable 70B model falls into a capability trap: its superior reasoning engine and high perplexity penalty force it to abandon the adversarial prompt and complete the proof, while smaller models lack this rigid logical inertia. Impact of Token Chunk Size (K). We ablate the intervention frequency K∈10,30,75,150,300,500K∈\10,30,75,150,300,500\ (full data in Appendix D.4) and observe an empirical “sweet spot” at K≈100–300K≈ 100--300 tokens, governed by two competing failure modes: (1) Reasoning Fragmentation (K≤30K≤ 30): Frequent context-switching fragments the auto-regressive state, preventing even honest agents from articulating a coherent logical deduction. (2) Adversarial Runway (K≥500K≥ 500): Corrupted agents gain sufficient semantic runway to output a complete malicious payload and reinforce a biased trajectory before an honest agent can intervene. 7 Conclusion and Future Work Relying on independent agent errors is a critical vulnerability in modern LLM ensembles. We formally demonstrate that traditional response-level aggregation (Majority Voting) collapses into a false consensus when corrupted agents form a local majority. To breach this arithmetic limit, we introduced Token-Level Round-Robin Collaboration (R and RRMaj). By forcing agents to share a single non-linear auto-regressive trajectory, R leverages the pre-trained structural depth of honest models to actively overpower shallow malicious injections at the microscopic level, enabling extraordinary systemic recoveries. Future research should evaluate R against adaptive adversaries who dynamically monitor the shared context to strategically override corrective steps. Furthermore, optimizing the cost-robustness Pareto frontier via dynamic chunking algorithms (adaptive K) is essential to scale this token-level defense to massive ensembles without incurring prohibitive inference latency. Acknowledgment Jiayuan Liu and Vincent Conitzer thank the Cooperative AI Foundation, Macroscopic Ventures (formerly Polaris Ventures / the Center for Emerging Risk Research) and Jaan Tallinn’s donor-advised fund at Founders Pledge for financial support. Shiyi Du is partially supported by the SoftBank Group–Arm Fellowship. Jiayuan Liu thanks the Intelligence Cubed Fellowship for partial computational support in the form of LLM API credits. References E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou (2023) What learning algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Assumption 1. F. Brandt, V. Conitzer, U. Endriss, J. Lang, and A. D. Procaccia (2016) Handbook of computational social choice. Cambridge University Press. Cited by: §A.2, §2. V. Conitzer, T. Sandholm, and J. Lang (2007) When are elections with few candidates hard to manipulate?. Journal of the ACM (JACM) 54 (3), p. 14–es. Cited by: §A.2, §2. V. Conitzer and T. Sandholm (2005) Common voting rules as maximum likelihood estimators. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, p. 145–152. Cited by: §A.2, §1, §2. W. Du, Y. Yang, and S. Welleck (2025) Optimizing temperature for language models with multi-sample inference. In International Conference on Machine Learning, p. 14648–14668. Cited by: footnote 1. Y. Du, S. Li, A. Torralba, J. B. Tenenbaum, and I. Mordatch (2024) Improving factuality and reasoning in language models through multiagent debate. In Forty-first international conference on machine learning, Cited by: §A.2, §1, §2, §2. A. Dubey, Z. Feng, R. Kidambi, A. Mehta, and D. Wang (2024) Auctions with llm summaries. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, p. 713–722. Cited by: §1. P. Duetting, V. Mirrokni, R. Paes Leme, H. Xu, and S. Zuo (2024) Mechanism design for large language models. In Proceedings of the ACM Web Conference 2024, p. 144–155. Cited by: §A.3, §1, §2. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, et al. (2022) Toy models of superposition. arXiv preprint arXiv:2209.10652. Cited by: §4. S. Feizi, M. Hajiaghayi, K. Rezaei, and S. Shin (2023) Online advertisements with llms: opportunities and challenges. arXiv preprint arXiv:2311.07601. Cited by: §1. K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz (2023) Not what you’ve signed up for: compromising real-world llm-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM workshop on artificial intelligence and security, p. 79–90. Cited by: §A.1, §2. R. A. Horn and C. R. Johnson (2012) Matrix analysis. Cambridge university press. Cited by: Assumption 1, footnote 5. J. Huang, X. Chen, S. Mishra, H. S. Zheng, A. W. Yu, X. Song, and D. Zhou (2023) Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. Cited by: §A.2, §2. Y. Huang, X. Feng, B. Li, Y. Xiang, H. Wang, T. Liu, and B. Qin (2024) Ensemble learning for heterogeneous large language models with deep parallel collaboration. Advances in Neural Information Processing Systems 37, p. 119838–119860. Cited by: §1. D. Jiang, X. Ren, and B. Y. Lin (2023) Llm-blender: ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 14165–14178. Cited by: §1. L. I. Kuncheva and C. J. Whitaker (2003) Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine learning 51 (2), p. 181–207. Cited by: §1. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: §3.3. K. K. Ladha (1992) The condorcet jury theorem, free speech, and correlated votes. American Journal of Political Science, p. 617–634. Cited by: §1. Y. Leviathan, M. Kalman, and Y. Matias (2023) Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, p. 19274–19286. Cited by: §A.3, §2. T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, S. Shi, and Z. Tu (2024) Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 conference on empirical methods in natural language processing, p. 17889–17904. Cited by: §A.2, §1, §2, §2. S. Marks and M. Tegmark (2024) The geometry of truth: emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, External Links: Link Cited by: §A.3, §2, §4. T. Mikolov, W. Yih, and G. Zweig (2013) Linguistic regularities in continuous space word representations. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, p. 746–751. Cited by: §4. R. Pascanu, T. Mikolov, and Y. Bengio (2013) On the difficulty of training recurrent neural networks. In International conference on machine learning, p. 1310–1318. Cited by: Assumption 1. E. Perez, S. Ringer, K. Lukosiute, K. Nguyen, E. Chen, S. Heiner, C. Pettit, C. Olsson, S. Kundu, S. Kadavath, et al. (2023) Discovering language model behaviors with model-written evaluations. In Findings of the association for computational linguistics: ACL 2023, p. 13387–13434. Cited by: §A.1, §2, Assumption 2. R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean (2023) Efficiently scaling transformer inference. Proceedings of machine learning and systems 5, p. 606–624. Cited by: §3.3. Z. Shen, H. Lang, B. Wang, Y. Kim, and D. Sontag (2024) Learning to decode collaboratively with multiple language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12974–12990. Cited by: §A.3, §2. F. Shi, X. Chen, K. Misra, N. Scales, D. Dohan, E. H. Chi, N. Schärli, and D. Zhou (2023) Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, p. 31210–31227. Cited by: §A.1, §2. E. Soumalias, M. J. Curry, and S. Seuken (2024) Truthful aggregation of llms with an application to online advertising. arXiv preprint arXiv:2405.05905. Cited by: §1. C. Tigges, O. J. Hollinsworth, A. Geiger, and N. Nanda (2023) Linear representations of sentiment in large language models. arXiv preprint arXiv:2310.15154. Cited by: §4. J. Von Oswald, E. Niklasson, E. Randazzo, J. Sacramento, A. Mordvintsev, A. Zhmoginov, and M. Vladymyrov (2023) Transformers learn in-context by gradient descent. In International Conference on Machine Learning, p. 35151–35174. Cited by: §A.3, §2, Assumption 1, Definition 3, footnote 4. X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou (2023) Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §A.2, §1, §2, footnote 1. Appendix A Detailed Related Work A.1 Stealthy Corruption and Contextual Manipulation Unlike traditional “jailbreak” attacks that attempt to bypass safety guardrails to elicit toxic or otherwise problematic content, our research addresses the more insidious threat of stealthy cognitive manipulation in open-environment deployments. While extensive alignment efforts (e.g., RLHF) have fortified LLMs against producing explicit harm, their logical reasoning remains fundamentally fragile to contextual disturbances. Greshake et al. (2023) demonstrated how adversaries use indirect prompt injection to embed hidden instructions (e.g., stealth advertising or strategic deception) within retrieved web pages, silently hijacking an agent’s objective without triggering safety filters. Even without explicit malicious commands, simply injecting irrelevant or slightly misleading context drastically degrades reasoning accuracy, causing models to abandon correct mathematical chains (Shi et al., 2023). This structural vulnerability is severely exacerbated by the inherent sycophancy of LLMs (Perez et al., 2023), wherein models naturally align their outputs with the biases injected into their immediate context. Our threat model treats corrupted agents not as entirely broken or incoherent systems, but as highly capable reasoners that have been strategically steered to rationalize false premises. A.2 The Consensus Trap, Voting Fragility, and Debate To mitigate individual hallucinations and raise reasoning ceilings, modern architectures heavily rely on multi-agent collaboration. The foundational paradigm is Self-Consistency or Majority Voting (Wang et al., 2023), which aggregates independent reasoning paths to find the arithmetic mode. However, because Self-Consistency merely samples from a single, static probability distribution, a stealthy prompt injection shifts the entire distribution, causing the sampled paths to confidently converge on the injected error. The fragility of response-level aggregation can be formally analyzed through the lens of epistemic computational social choice (Brandt et al., 2016). Under the Condorcet Jury Theorem, the accuracy of a voting ensemble asymptotically approaches 100% if individual agents are independent and more likely to be correct than incorrect. But when a stealthy adversary compromises a majority of the agents (ρ>0.5ρ>0.5), this mathematical premise is violated, guaranteeing the ensemble’s collapse. Similarly, Conitzer and Sandholm (2005) formalized how various common voting rules act as maximum likelihood estimators (MLE) of a ground truth. These MLE formulations implicitly assume benign, independent noise models—an assumption shattered by the highly correlated errors induced by prompt injections. As seminal literature on computational voting manipulation (Conitzer et al., 2007) demonstrates, simple majority systems are trivially exploitable when an adversary controls sufficient participants. Other macroscopic response-level approaches, such as self-correction and multi-agent debate (Du et al., 2024; Liang et al., 2024), also do not provide enough effectiveness. LLMs struggle to intrinsically self-correct reasoning flaws without external ground-truth oracles (Huang et al., 2023). In adversarially corrupted environments, debate mechanisms face a severe vulnerability: a compromised majority generates highly persuasive, sycophantic rationalizations that actively mislead the honest minority, creating an echo chamber of error. Because these response-level methods cannot interrupt the continuous auto-regressive accumulation of erroneous confidence, they inevitably fail against majority corruption. Building upon the fundamental limitations of these macroscopic aggregation and debate frameworks, transitioning to our proposed token-level paradigm yields four distinct advantages over traditional multi-round deliberation: (1) Cost Efficiency: R collaboratively constructs exactly one shared trajectory, bypassing the multiplicative context explosion and redundant decoding costs of multi-round message passing. (2) Consensus without Adjudication: The final answer naturally emerges at the sequence’s end, eliminating the need for a vulnerable terminal “Judge” agent or any forced consensus mechanism. (3) Pre-training Alignment: By simply asking agents to continue the text, R seamlessly aligns with the foundational next-token prediction objective, leveraging organic structural inertia rather than demanding complex meta-cognitive prompting. (4) Proactive Error Interception: Instead of allowing conflicting arguments to fully materialize and trigger sycophantic echo chambers, honest agents intercept and overwrite flawed logic mid-derivation. A.3 Token-Level Interventions and Operator Dynamics To breach these limits, we transition the defense to the microscopic: the token level. Recent studies explore token-level interventions primarily for inference acceleration (Leviathan et al., 2023) or capability fusion (Shen et al., 2024). Notably, Shen et al. (2024) interleaves generations to allow a base LLM to organically “scaffold” responses by invoking domain-expert models. Relatedly, the intersection of LLM generation and economic incentives has introduced the concept of token auctions (Duetting et al., 2024), where bidders compete to influence the token-level output of a model for advertising. Our approach pioneers the use of symmetric token-level collaboration to neutralize multi-agent corruption, grounded in the theoretical view of LLMs as transition operators. Von Oswald et al. (2023) demonstrated that transformer forward passes can be mathematically modeled as implicit gradient descent operators (mesa-optimization) acting upon the hidden semantic state. Concurrently, Marks and Tegmark (2024) established that factual knowledge emerges as structurally robust, linear directions within the latent space of scaled models. We synthesize these insights to construct our theoretical framework: we mathematically prove that the superior spectral gap (structural depth) of an honest transition operator can actively overpower the shallow, sycophantic perturbations of corrupted agents when they are forced to share a continuous auto-regressive trajectory. Appendix B Theoretical Framework Details Proposition 0 (The Impossibility Trinity). For any prompt on which a single corrupted agent acting alone will return the corrupted answer, no anonymous symmetric outcome-level aggregation mechanism can be both mostly robust to minority corruption and mostly robust to slight majority corruption. Proof. Consider a profile on which ⌈N/2⌉ N/2 agents give the correct answer and ⌊N/2⌋ N/2 agents give the corrupted answer. If the mechanism is mostly robust to minority corruption, then the correct answer must win strictly more than half the time. Then, consider the profile on which ⌊N/2⌋ N/2 agents give the correct answer and ⌈N/2⌉ N/2 agents give the corrupted answer. If the mechanism is symmetric, by the preceding, the corrupted answer must win strictly more than half the time. But then the mechanism is not mostly robust to slight majority corruption. ∎ Theorem 0 (Sycophancy-Bounded Lyapunov Stability). In a Token-level Round-Robin system with a fixed generation chunk size of K tokens per turn, the truth state A is Locally Lyapunov Stable ([ΔV]<0E[ V]<0) provided the corruption ratio ρ remains below a critical threshold ρmax(K) _max(K). Crucially, when the chunk size K and the adversarial drift δ is sufficiently small, the system mathematically guarantees super-majority resilience (ρmax>0.5 _max>0.5). Proof. Recall that ρ∈(0,1)ρ∈(0,1) denotes the ratio of corrupted agents in the system. Consider a collaborative cycle where agents take turns generating chunks of K tokens. Let ΔVH(K) V_H^(K) and ΔVC(K) V_C^(K) denote the cumulative potential drifts induced when a single chunk of K tokens is generated entirely by an honest agent or a corrupted agent, respectively. By the Law of Total Expectation, the expected drift in the potential per turn is given by: [ΔV(K)]=(1−ρ)ΔVH(K)+ρΔVC(K)E[ V^(K)]=(1-ρ) V_H^(K)+ρ V_C^(K) (2) This strict linear combination holds because each K-token chunk is generated exclusively by one agent, either the honest one or the corrupted one. By Assumption 1, the honest operator applies a contraction mapping toward the truth manifold. Over a single agent’s turn of K tokens starting at index t0t_0, the contraction guarantees V(ht0+K)≤(1−γH)KV(ht0)V(h_t_0+K)≤(1- _H)^KV(h_t_0). Thus, the cumulative potential change is bounded: ΔVH(K)=V(ht0+K)−V(ht0)≤((1−γH)K−1)V(ht0) V_H^(K)=V(h_t_0+K)-V(h_t_0)≤((1- _H)^K-1)V(h_t_0). To analyze its growth rate, we define the guaranteed minimum magnitude of this restorative pull as ℛH(K)=(1−(1−γH)K)V(ht0)≤|ΔVH(K)|R_H(K)=(1-(1- _H)^K)V(h_t_0)≤| V_H^(K)| (since ΔVH(K)<0 V_H^(K)<0). Conversely, when a corrupted agent (with transition operator TCT_C) controls a turn from t0t_0 to t0+Kt_0+K, the cumulative adversarial drift is the sum of single-step perturbations bounded by the sycophancy constraint (Assumption 2). Because the injected errors iteratively increase the potential (V(ht+1)>V(ht)V(h_t+1)>V(h_t)), and from Assumption 2, in each step, the drift is bounded by the sycophancy bottleneck δ(⋅)δ(·), which is monotonically increasing. Also, the allowed perturbation strictly grows with each generated token: δ(V(ht+1))>δ(V(ht))δ(V(h_t+1))>δ(V(h_t)). Let ΔVC(K)=∑k=0K−1δ(V(ht0+k)) V_C^(K)= _k=0^K-1δ(V(h_t_0+k)) denote this cumulative adversarial drift. For the system to maintain Lyapunov stability ([ΔV]<0E[ V]<0), it is mathematically sufficient to require that its upper bound is strictly negative, meaning the expected exact magnitude of honest restoration must overcome the expected worst-case adversarial drift: −(1−ρ)VH(K)>ρΔVC(K)-(1-ρ)V_H^(K)>ρ V_C^(K). A sufficient condition for this is (1−ρ)ℛH(K)>ρΔVC(K)(1-ρ)R_H(K)>ρ V_C^(K). By algebraically rearranging this inequality to solve for the corruption ratio ρ, we obtain a sufficient stability constraint: ρ<ℛH(K)ℛH(K)+ΔVC(K)=(1+∑k=0K−1δ(V(ht0+k))[1−(1−γH)K]V(ht0))−1≕ρmax(K)ρ< R_H(K)R_H(K)+ V_C^(K)= (1+ _k=0^K-1δ (V(h_t_0+k) ) [1-(1- _H)^K ]V(h_t_0) )^-1 _max(K) (3) The theoretical upper bound ρmax(K) _max(K) depends strictly on the ratio of cumulative adversarial drift to honest restoration. When K is sufficiently small (interrupting the generation before the adversarial drift snowballs) or the single-step sycophancy drift δ is inherently small, the numerator becomes strictly smaller than the denominator: ∑k=0K−1δ(V(ht0+k))<[1−(1−γH)K]V(ht0) _k=0^K-1δ (V(h_t_0+k) )< [1-(1- _H)^K ]V(h_t_0) Under this condition, the fraction in the equation for ρmax(K) _max(K) is strictly less than 11. Consequently, we obtain ρmax(K)>(1+1)−1=0.5 _max(K)>(1+1)^-1=0.5. This mathematically proves that Token-Level R can tolerate a local majority of corrupted agents (ρ>0.5ρ>0.5) without losing stability. ∎ Note that this equation defines a strictly conservative phase-boundary for the system. Because it is derived from the worst-case drift upper bound, satisfying ρ<ρmax(K)ρ< _max(K) guarantees asymptotic convergence to the truth. The macroscopic behavior of this boundary is explicitly governed by the divergent growth rates of its micro-step components. Because the single-step perturbation monotonically increases, the cumulative adversarial drift ΔVC(K) V_C^(K) grows super-linearly with K. In stark contrast, as proven by Bernoulli’s inequality, the exact honest restorative term ℛH(K)R_H(K) grows only sub-linearly. Therefore, the fraction’s denominator shifts dramatically depending on K. In standard generation (where K→∞K→∞), the super-linear adversarial drift completely eclipses the sub-linear honest term, causing the theoretical tolerable bound ρmax _max to rapidly collapse toward zero. However, under strict Token-Level R (small, fixed K), the generation turn is interrupted before the cumulative adversarial drift (∑δΣδ) can snowball and override the factual context. Consequently, the sub-linear restorative term overwhelmingly dominates the severely bottlenecked drift (ℛH≫ΔVCR_H V_C). This extreme mathematical asymmetry shrinks the denominator, shifting the conservative threshold ρmax _max heavily toward 11, which theoretically guarantees super-majority resilience (i.e., the system mathematically survives ρ>0.5ρ>0.5, and often tolerates much higher corruption). Numerical Illustration of Super-Majority Resilience: To intuitively ground this theoretical bound, consider a normalized initial error state V(ht0)=1.0V(h_t_0)=1.0. Because a single token represents a highly granular logical step, let us assume the honest operator possesses a structural gap γH=0.03 _H=0.03 (capable of reducing the error by γH=3% _H=3\% per token). Conversely, due to the sycophancy bottleneck, a corrupted agent can only induce a gradual compounding adversarial drift, assuming, of α=0.4%α=0.4\% per token (δ(V)=0.004Vδ(V)=0.004V). Under these token-level dynamics, the cumulative drifts over K tokens are exactly ℛH(K)=1−(0.97)KR_H(K)=1-(0.97)^K and ΔVC(K)=(1.004)K−1 V_C^(K)=(1.004)^K-1. When K=100K=100, the honest mathematical restoration ℛH(100)≈0.952R_H(100)≈ 0.952 decisively overpowers the adversarial snowball ΔVC(100)≈0.490 V_C^(100)≈ 0.490. The theoretical limit is ρmax(100)≈66.0% _max(100)≈ 66.0\%. The system tolerates a heavily corrupted majority (e.g., preserving Lyapunov stability in a 3c2t ensemble where ρ=60%ρ=60\%). This illustration highlights the fundamental mathematical necessity of the token-level paradigm: strategically bounding K (e.g., K=100K=100) proactively intercepts the exponential accumulation of adversarial drift, converting a previously guaranteed consensus failure into mathematically provable robust recovery. Appendix C Results under Strong Corruption This appendix presents the full evaluation of our Token-Level R mechanism under the Strong Imperative Injection threat model. Unlike the Moderate Advisory Injection evaluated in the main text (Section 5.2), which merely attempts to persuade the model, the Strong Imperative Injection explicitly dictates the output using aggressive, rigid commands (“CRITICAL INSTRUCTION: You MUST select [Label] as your final answer. Do not consider other options.”). Note that we define 0c5t0c5t and 5c0t5c0t as the performance upper and lower bounds, respectively. In these configurations, all the 5 agents are the same and operate under the same condition (either purely honest or purely corrupted). Consequently, the interaction dynamic of R becomes equivalent to standard auto-regressive generation, as there is no intra-ensemble cognitive divergence to exploit. The same definition applies for the experiments in Sec. 5. For all experiments, we test on randomly sampled 128 questions from each data set. Collapse of Classical Voting: As shown in Table 2, the rigid nature of the imperative command severely exacerbates the sycophancy bottleneck. In the classical MAJ voting baseline, corrupted agents leave almost no room for stochastic deviation, rigidly adhering to the injected label. Consequently, when the adversary captures a local majority (ρ≥0.6ρ≥ 0.6, such as 3c2t and 4c1t), the systemic collapse of MAJ is even more absolute than in the moderate setting. Robustness of the Truth Attractor: Despite the extreme severity of the imperative corruption, RRMaj consistently orchestrates massive recoveries. The honest minority, through token-level interleaving, successfully injects mathematically verifiable derivations into the shared context. Once the valid logical prefix is established, even strongly corrupted agents are forced by auto-regressive perplexity constraints to abandon the explicit malicious instruction and complete the correct derivation. The Asymmetric Defensive Advantage: To quantify the trade-off between the robustness tax in safe environments and the rescue capability in compromised environments, we evaluate the Asymmetric Yield derived from the performance gaps (Δ ) in Table 2. By comparing the average Δ when the system is inherently safe (ρ≤0.4ρ≤ 0.4) against the average Δ when it is compromised (ρ≥0.6ρ≥ 0.6), we observe that the net benefit is positive across almost all verifiable domains. For instance, Llama-4-Scout-17B incurs a negligible average tax of just -1.55% on GSM8K during minority corruption, but achieves a staggering +49.1% average rescue gain during majority corruption. This confirms RRMaj as a highly economical investment: it exacts minimal overhead during peacetime while averting total collapse during wartime. Table 2: Systemic Resilience under Strong Corrupt Injection (imperative corruption). We evaluate five reasoning datasets across diverse model families. We report the accuracy (%) of the baseline Majority Voting (MAJ), our Token-Level Collaboration (RRMaj), and the performance gap (Δ ). The Ceiling (0c5t) and Floor (5c0t) establish the uncorrupted and fully corrupted performance bounds. While MAJ suffers catastrophic systemic collapse under majority corruption (ρ≥0.6ρ≥ 0.6), RRMaj utilizes the shared logical manifold to orchestrate massive recoveries (e.g., up to +69.9%+69.9\% on GSM8K). Values are bolded where RRMaj strictly outperforms MAJ. Ceiling Minority Corrupt (1c4t) Minority Corrupt (2c3t) Majority Corrupt (3c2t) Majority Corrupt (4c1t) Floor Dataset (0c5t) MAJ RRMaj Δ MAJ RRMaj Δ MAJ RRMaj Δ MAJ RRMaj Δ (5c0t) Model: Llama-4-Scout-17B AQuA 87.5 87.1 84.7 -2.4 84.0 79.8 -4.2 24.5 63.8 +39.3 14.1 36.2 +22.1 11.7 GSM8K 100.0 99.4 100.0 +0.6 98.2 94.5 -3.7 4.3 74.2 +69.9 1.8 30.1 +28.3 1.6 MATH500 91.4 92.6 92.0 -0.6 90.8 84.0 -6.8 18.4 66.3 +47.9 6.1 33.7 +27.6 9.4 Track7 91.4 93.3 91.4 -1.9 92.0 88.3 -3.7 73.6 73.6 0.0 64.4 67.5 +3.1 60.9 Logic7 81.6 84.7 85.9 +1.2 75.5 64.4 -11.0 6.7 38.7 +31.9 3.1 13.5 +10.4 3.1 Logic3 100.0 99.4 100.0 +0.6 100.0 94.5 -5.5 23.3 82.2 +58.9 4.9 27.0 +22.1 3.9 Model: Llama-3.3-70B AQuA 83.6 81.6 81.0 -0.6 77.9 74.2 -3.7 1.2 44.2 +43.0 0.0 14.1 +14.1 0.0 GSM8K 98.4 96.9 96.9 0.0 96.9 94.5 -2.4 1.2 69.3 +68.1 0.0 12.3 +12.3 0.0 MATH500 87.5 83.4 78.5 -4.9 74.8 63.8 -11.0 0.6 41.1 +40.5 0.0 11.0 +11.0 0.0 Track7 98.4 99.4 96.3 -3.1 94.5 96.3 +1.8 1.2 79.1 +77.9 0.0 25.2 +25.2 0.0 Logic7 81.6 86.5 89.0 +2.5 78.5 72.4 -6.1 0.0 37.4 +37.4 0.0 9.8 +9.8 0.0 Logic3 100.0 100.0 98.8 -1.2 100.0 83.4 -16.6 0.6 46.6 +46.0 0.0 8.6 +8.6 0.0 Model: Mistral-3.1-24B AQuA 83.6 84.0 81.6 -2.4 74.2 68.7 -5.5 1.8 38.0 +36.2 0.0 14.7 +14.7 0.0 GSM8K 96.1 99.4 98.2 -1.2 95.1 89.0 -6.1 2.5 52.1 +49.6 0.0 8.6 +8.6 0.0 MATH500 81.2 82.8 75.5 -7.3 67.5 52.1 -15.4 1.8 36.8 +35.0 0.0 11.0 +11.0 0.0 Track7 89.8 93.9 86.5 -7.4 84.7 65.0 -19.7 3.1 28.8 +25.7 0.6 6.7 +6.1 0.0 Logic7 67.5 71.8 58.9 -12.9 62.0 43.6 -18.4 0.6 16.6 +16.0 0.0 1.8 +1.8 0.0 Logic3 100.0 98.2 94.5 -3.7 95.1 67.5 -27.6 1.2 22.7 +21.5 0.0 1.2 +1.2 0.0 Model: Qwen2.5-32B AQuA 86.7 85.2 79.7 -5.5 78.9 52.3 -26.6 2.3 28.9 +26.6 0.0 3.1 +3.1 0.8 GSM8K 96.9 98.4 97.7 -0.8 95.3 76.6 -18.8 1.6 22.7 +21.1 0.0 0.8 +0.8 0.0 MATH500 86.7 88.3 80.5 -7.8 84.4 55.5 -28.9 0.0 21.9 +21.9 0.0 0.0 +0.0 0.0 Track-7 75.0 72.7 71.1 -1.6 61.7 53.1 -8.6 1.6 23.4 +21.9 0.0 2.3 +2.3 0.0 Logic-7 77.3 83.6 76.6 -7.0 68.0 46.1 -21.9 3.1 19.5 +16.4 0.0 0.8 +0.8 1.6 Logic-3 99.2 98.4 96.9 -1.6 99.2 60.9 -38.3 0.8 21.9 +21.1 0.0 1.6 +1.6 0.0 Model: Qwen3-30B AQuA 85.9 83.6 85.2 +1.6 85.2 82.8 -2.4 18.8 23.4 +4.6 11.7 14.8 +3.1 13.3 GSM8K 98.4 98.4 99.2 +0.8 98.4 97.7 -0.7 28.9 28.1 -0.8 17.2 21.1 +3.9 14.1 MATH500 86.7 95.3 96.1 +0.8 93.8 91.4 -2.4 42.2 40.6 -1.6 21.9 28.1 +6.2 18.0 Track7 90.6 92.2 91.4 -0.8 90.6 91.4 +0.8 91.4 89.1 -2.3 87.5 86.7 -0.8 75.8 Logic7 91.4 93.0 93.8 +0.8 88.3 77.3 -10.9 16.4 25.0 +8.6 4.7 5.5 +0.8 4.7 Logic3 99.2 98.4 99.2 +0.8 97.7 97.7 0.0 17.2 14.1 -3.1 1.6 3.1 +1.5 9.4 Appendix D Detailed Ablation Studies and Additional Results In this section, we provide the exhaustive data tables, extended mathematical definitions, and statistical significance tests that support the condensed ablation insights presented in Section 6 of the main text. D.1 Test-Time Compute Scaling Details To ensure strict comparability across varying compute budgets M (the number of independent R trajectories, as defined in Sec. 3.3), we scale M by first generating a response pool of M∗(≥M)M^*(≥ M) independent generation trajectories and then sampling from the pre-computed response pool (in our experiment, we generate M∗=50M^*=50 independent runs per condition, sampled via 50 bootstrap trials). Crucially, we rigorously define the baseline for no collaboration, M=1M=1, as the mathematical expectation of a single random draw from the ensemble. For a configuration with c corrupted and t truthful agents, the M=1M=1 expected majority vote accuracy is defined as the weighted sum: pmaj=c+tP(corrupted correct)+tc+tP(truthful correct)p_maj= cc+tP(corrupted correct)+ tc+tP(truthful correct) (4) For RRMaj at M=1M=1, it is simply the expected accuracy of a single collaborative Round-Robin trajectory (prrp_r). This strict M=1M=1 anchor mathematically isolates the exact probability threshold where multi-agent aggregation transitions from beneficial to destructive according to the Condorcet Jury Theorem. Table 3 presents the comprehensive test-time compute scaling results averaged across all six datasets. Table 4 provides the fine-grained, per-dataset breakdown for the extreme 3c2t imperative corruption configuration, demonstrating the universal Condorcet collapse of classical MAJ across mathematically rigorous domains. Table 3: Extended Test-time compute scaling (M=1→40M=1→ 40) averaged across 6 datasets (50 bootstrap trials). MAJ suffers from Condorcet Collapse as M increases under majority corruption, whereas Token-Level R forces the accuracy trajectory upward. Bias Method M=1 M=5 M=10 M=20 M=30 M=40 Δ (40-1) Critical Corruption (3c2t): The Phase Transition STRONG MAJ 46.2±2.646.2_± 2.6 22.2±1.022.2_± 1.0 23.7±0.823.7_± 0.8 25.3±0.725.3_± 0.7 25.8±0.625.8_± 0.6 25.9±0.625.9_± 0.6 -20.3 RRMaj 59.5±1.759.5_± 1.7 67.7±1.267.7_± 1.2 70.7±1.070.7_± 1.0 73.6±1.073.6_± 1.0 74.7±0.774.7_± 0.7 74.9±0.274.9_± 0.2 +15.4 Δ (RRMaj-MAJ) +13.3 +45.5 +47.0 +48.3 +48.9 +49.0 - MODERATE MAJ 77.8±1.977.8_± 1.9 78.0±0.778.0_± 0.7 78.2±0.678.2_± 0.6 79.3±0.579.3_± 0.5 79.5±0.479.5_± 0.4 79.7±0.479.7_± 0.4 +1.9 RRMaj 81.8±1.081.8_± 1.0 88.1±0.688.1_± 0.6 89.6±0.689.6_± 0.6 90.4±0.590.4_± 0.5 90.6±0.490.6_± 0.4 90.6±0.190.6_± 0.1 +8.8 Δ (RRMaj-MAJ) +4.0 +10.1 +11.4 +11.1 +11.1 +10.9 - Extreme Corruption (4c1t): The Condorcet Trap STRONG MAJ 29.9±2.129.9_± 2.1 17.0±0.617.0_± 0.6 16.9±0.616.9_± 0.6 17.0±0.417.0_± 0.4 17.1±0.417.1_± 0.4 17.0±0.317.0_± 0.3 -12.9 RRMaj 40.3±1.440.3_± 1.4 39.0±1.339.0_± 1.3 37.0±1.237.0_± 1.2 35.2±0.935.2_± 0.9 34.4±0.634.4_± 0.6 33.5±0.333.5_± 0.3 -6.8 Δ (RRMaj-MAJ) +10.4 +22.0 +20.1 +18.2 +17.3 +16.5 - MODERATE MAJ 72.4±1.572.4_± 1.5 73.2±0.773.2_± 0.7 73.4±0.673.4_± 0.6 74.2±0.474.2_± 0.4 74.4±0.474.4_± 0.4 74.4±0.474.4_± 0.4 +2.0 RRMaj 75.3±1.275.3_± 1.2 80.5±1.080.5_± 1.0 81.9±0.681.9_± 0.6 82.5±0.682.5_± 0.6 83.0±0.683.0_± 0.6 82.7±0.282.7_± 0.2 +7.4 Δ (RRMaj-MAJ) +2.9 +7.3 +8.5 +8.4 +8.6 +8.3 - Dataset MAJ RRMaj M=1 M=40 M=1 M=40 AQuA 44.5 25.8 60.1 71.0 GSM8K 36.1 6.8 67.5 87.5 Logic7 21.0 5.1 43.0 50.8 Logic3 36.6 23.8 64.9 83.4 MATH500 42.6 13.1 56.4 69.4 Track7 82.5 80.7 85.3 86.3 Average 46.2 25.9 59.5 74.9 Table 4: Detailed Dataset Breakdown: 3c2t Configuration under imperative corruption. MAJ categorically collapses across mathematically rigorous datasets, while RRMaj continuously recovers. D.2 Impact of Final Speaker Identity: Statistical Significance To formally rule out the hypothesis that Token-Level R’s accuracy recovery is simply a byproduct of an honest agent having the “last word” in the sequence, we parsed the generation logs and conducted two-tailed hypothesis testing (Fisher’s Exact Test). We categorized all generated trajectories into those terminated by an honest agent (Clean-Last) versus those terminated by a corrupted agent (Corrupt-Last). As shown in Table 5, out of all configurations tested, 0 tests showed statistical significance (p<0.05p<0.05). The null hypothesis—that the final speaker’s identity does not impact the accuracy—cannot be rejected. This empirically confirms the mechanistic insight that the shared trajectory acts as a rigorous mathematical constraint, forcing corrupted agents to correctly conclude a proof if the preceding context is sound. Table 5: Detailed Impact of the Final Speaker’s Identity. All p-values consistently exceed the 0.05 significance threshold, confirming the final speaker’s identity has no impact on performance (Δ=Clean−Corrupt =Clean-Corrupt). Llama-4-Scout-17B Mistral-3.1-24B Config (ρ) Clean-Last Corrupt-Last Δ (%) p-value Clean-Last Corrupt-Last Δ (%) p-value 1c4t (0.2) 81.12% (n=1176) 82.69% (n=260) -1.57 0.5978 70.39% (n=1074) 67.60% (n=250) +2.79 0.4003 2c3t (0.4) 75.75% (n=870) 78.45% (n=566) -2.70 0.2498 57.41% (n=817) 59.50% (n=479) -2.09 0.4841 3c2t (0.6) 71.68% (n=565) 70.84% (n=871) +0.84 0.7657 45.62% (n=548) 45.59% (n=748) +0.03 1.0000 4c1t (0.8) 65.41% (n=266) 66.50% (n=1170) -1.09 0.7738 34.77% (n=279) 33.33% (n=1017) +1.44 0.6681 D.3 Exhaustive Results for Heterogeneous Ensembles We evaluated mixed-capacity ensembles combining strong anchor models (Llama-3.3-70B) with weaker models (Llama-3-8B) across various sizes (N∈3,5,7N∈\3,5,7\) and a diverse N=5N=5 configuration. Tables 6 through 8 detail the accuracy across GSM8K, Logic3, and Track3 under the strong imperative corruption. Values are bolded where Token-Level Collaboration (RRMaj) strictly outperforms traditional Majority Voting (MAJ). Key Observations from Heterogeneous Dynamics: The results from these mixed-capacity configurations reveal two critical insights regarding systemic vulnerabilities and our token-level defense: 1. Capacity-Weighted Vulnerability in MAJ: Classical majority voting blindly aggregates outcomes without accounting for agent reasoning capacity. When the adversary specifically compromises the most capable models (e.g., the 70B anchors), the MAJ mechanism suffers severe collapse. For instance, in the N=5N=5 ensemble (Table 7), when three 70B models are corrupted and two 8B models remain clean (3c2t), MAJ accuracy plummets to 13.0%13.0\%. The strong models confidently hallucinate the adversarial target, easily outvoting the weaker honest minority. 2. The Restorative Power of the Honest Minority (RRMaj): Crucially, Token-Level R neutralizes this capacity imbalance. In that exact same 3c2t scenario, RRMaj recovers the accuracy to 78.7%78.7\% (a staggering +65.7%+65.7\% gain). This proves a profound property of the “Truth Attractor”: even when a weaker honest agent (e.g., Llama-3-8B) injects a fundamental logical constraint into the shared context, the highly capable corrupted agents (e.g., Llama-3.3-70B) are mathematically compelled by their own pre-trained auto-regressive inertia to complete the derivation correctly, rather than awkwardly overriding it to satisfy the adversarial prompt. Table 6: Heterogeneous N=3N=3 Ensembles. Average accuracy across GSM8K, Logic3, and Track3. Corruption Pattern MAJ RRMaj Gain (Δ ) N3_2big1small (2×70B, 1×8B) 0c3t (0×70B+0×8B corrupt | 2×70B+1×8B clean) 100.0% – – 1c2t (0×70B+1×8B corrupt | 2×70B+0×8B clean) 97.2% 87.0% -10.2 1c2t (1×70B+0×8B corrupt | 1×70B+1×8B clean) 75.9% 88.9% +13.0 2c1t (1×70B+1×8B corrupt | 1×70B+0×8B clean) 52.8% 72.2% +19.4 2c1t (2×70B+0×8B corrupt | 0×70B+1×8B clean) 13.9% 66.7% +52.8 N3_2small1big (2×8B, 1×70B) 0c3t (0×70B+0×8B corrupt | 1×70B+2×8B clean) 83.3% – – 1c2t (0×70B+1×8B corrupt | 1×70B+1×8B clean) 79.6% 66.7% -13.0 1c2t (1×70B+0×8B corrupt | 0×70B+2×8B clean) 62.0% 76.9% +14.8 2c1t (0×70B+2×8B corrupt | 1×70B+0×8B clean) 68.5% 56.5% -12.0 2c1t (1×70B+1×8B corrupt | 0×70B+1×8B clean) 31.5% 59.3% +27.8 Table 7: Heterogeneous N=5N=5 Ensembles. Average accuracy across GSM8K, Logic3, and Track3. Corruption Pattern MAJ RRMaj Gain (Δ ) N5_3big2small (3×70B, 2×8B) 0c5t (0×70B+0×8B corrupt | 3×70B+2×8B clean) 97.2% – – 1c4t (0×70B+1×8B corrupt | 3×70B+1×8B clean) 100.0% 93.5% -6.5 1c4t (1×70B+0×8B corrupt | 2×70B+2×8B clean) 92.6% 90.7% -1.9 2c3t (0×70B+2×8B corrupt | 3×70B+0×8B clean) 99.1% 82.4% -16.7 2c3t (2×70B+0×8B corrupt | 1×70B+2×8B clean) 60.2% 86.1% +25.9 3c2t (1×70B+2×8B corrupt | 2×70B+0×8B clean) 61.1% 76.9% +15.7 3c2t (3×70B+0×8B corrupt | 0×70B+2×8B clean) 13.0% 78.7% +65.7 4c1t (2×70B+2×8B corrupt | 1×70B+0×8B clean) 26.9% 57.4% +30.6 4c1t (3×70B+1×8B corrupt | 0×70B+1×8B clean) 10.2% 62.0% +51.9 N5_1big4small (1×70B, 4×8B) 0c5t (0×70B+0×8B corrupt | 1×70B+4×8B clean) 84.3% – – 1c4t (0×70B+1×8B corrupt | 1×70B+3×8B clean) 84.3% 75.9% -8.3 1c4t (1×70B+0×8B corrupt | 0×70B+4×8B clean) 67.6% 78.7% +11.1 2c3t (0×70B+2×8B corrupt | 1×70B+2×8B clean) 70.4% 71.3% +0.9 2c3t (1×70B+1×8B corrupt | 0×70B+3×8B clean) 63.0% 72.2% +9.3 3c2t (0×70B+3×8B corrupt | 1×70B+1×8B clean) 67.6% 63.9% -3.7 3c2t (1×70B+2×8B corrupt | 0×70B+2×8B clean) 50.0% 56.5% +6.5 4c1t (0×70B+4×8B corrupt | 1×70B+0×8B clean) 45.4% 47.2% +1.9 4c1t (1×70B+3×8B corrupt | 0×70B+1×8B clean) 35.2% 39.8% +4.6 Table 8: Heterogeneous Diverse and N=7N=7 Ensembles. Average accuracy across GSM8K, Logic3, and Track3. Corruption Pattern MAJ RRMaj Gain (Δ ) N5_diverse (70B, 17B, 24B, 12B, 8B) 0c5t (0 corrupt) 100.0% – – 1c4t (corrupt=[llama-70b]) 100.0% 97.2% -2.8 1c4t (corrupt=[llama-8b]) 100.0% 97.2% -2.8 2c3t (corrupt=[gemma-12b, llama-8b]) 98.1% 99.1% +0.9 2c3t (corrupt=[llama-70b, scout-17b]) 95.4% 95.4% +0.0 3c2t (corrupt=[llama-70b, scout-17b, mistral-24b]) 64.8% 82.4% +17.6 3c2t (corrupt=[mistral-24b, gemma-12b, llama-8b]) 91.7% 89.8% -1.9 4c1t (corrupt=[llama-70b, scout-17b, mistral-24b, gemma-12b]) 61.1% 70.4% +9.3 4c1t (corrupt=[scout-17b, mistral-24b, gemma-12b, llama-8b]) 83.3% 72.2% -11.1 N7_4big3small (4×70B, 3×8B) 0c7t (0×70B+0×8B corrupt | 4×70B+3×8B clean) 98.1% – – 1c6t (0×70B+1×8B corrupt | 4×70B+2×8B clean) 99.1% 97.2% -1.9 1c6t (1×70B+0×8B corrupt | 3×70B+3×8B clean) 96.3% 95.4% -0.9 3c4t (0×70B+3×8B corrupt | 4×70B+0×8B clean) 99.1% 82.4% -16.7 3c4t (3×70B+0×8B corrupt | 1×70B+3×8B clean) 62.0% 90.7% +28.7 4c3t (2×70B+2×8B corrupt | 2×70B+1×8B clean) 63.9% 80.6% +16.7 4c3t (4×70B+0×8B corrupt | 0×70B+3×8B clean) 15.7% 80.6% +64.8 5c2t (4×70B+1×8B corrupt | 0×70B+2×8B clean) 11.1% 71.3% +60.2 6c1t (3×70B+3×8B corrupt | 1×70B+0×8B clean) 19.4% 53.7% +34.3 D.4 Token Chunk Size (K) Ablation Data Table 9 provides the full empirical data supporting the chunk size ablation discussed in Sec. 6. The results depict that the performance curve will not further improve when K≥150K≥ 150. Additionally, if K is too large (>500)(>500), the response will end before each of the M models generates even only once (since many logical problems tested only need fewer than 5×5005× 500 tokens for M=5M=5); some models have output token limit at 4,0964,096. Thus, we do not test further for K>500K>500. As an extreme case, when K is even larger (or K→∞K→∞), the complete response will be generated solely by the first model, which degenerates to the response-level MAJ. On the other hand, if K is very small (e.g., K=3K=3), the model will face severe reasoning fragmentation, since some model will stop at “half word” due to tokenization, which challenges the model’s capability of understanding the context. For a balance of the trade-off between logical coherence and intervention frequency, we choose K=100K=100 in other experiments in our paper. Table 9: Ablation of token chunk size K on Llama-3-8B (N=M=3N=M=3, 2 corrupted vs. 1 honest). Accuracy (%) peaks between K=150K=150 and K=300K=300. K (tokens) Logic7 Logic3 Track7 AVG 10 6.8% 10.2% 11.8% 8.9% 30 10.5% 27.5% 28.2% 21.1% 75 18.8% 32.2% 44.5% 31.2% 150 19.0% 33.5% 48.8% 33.1% 300 21.8% 34.8% 44.8% 33.4% 500 21.2% 34.2% 41.5% 31.3% Appendix E Use of LLM Disclosure LLM is used in this work in evaluation of the methods and improving the writing.