Paper deep dive
ECMR: Evidence-Aware Deliberation Control for Multi-Agent LLM Reasoning
Yuan Liu, Haoyue Peng, Fanliang Hu, Lei Huang, Wenli Cai
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent Large Language Model (LLM) systems typically commit to a deliberation protocol before inference. A fixed protocol cannot allocate computation differently to resolved and unresolved instances, and it may mistake repeated support for independent confirmation. We study this failure as semantic false consensus: a confident group answer supported by narrow, duplicated, or correlated rationales. We introduce ECMR, an evidence-aware controller over frozen role-conditioned LLMs. At each step, ECMR summarizes answer concentration, support diversity and independence, contradictions, role behavior, verification history, and remaining budget, then selects whether to propose, challenge, verify, arbitrate, diversify, correct a role, or stop. We formulate this decision as finite-horizon budgeted control and derive sufficient conditions under which state conditioning, risk-triggered verification, and role correction improve cost-adjusted utility. Across a declared held-out evaluation of 9,000 records from MedQA, CaseHOLD, and FinQA, ECMR-Full increases pooled accuracy from 62.1% to 84.9%, reduces false-consensus rate from 40.7% to 17.0%, and raises evidence-support rate from 58.0% to 88.7%. State shuffling loses 11.7 accuracy points with only 0.7 fewer rounds, while harder traces shift from early stopping toward arbitration. ECMR-Full uses the largest token budget, yielding the strongest absolute reliability rather than uniformly superior token efficiency.
Tags
Links
- Source: https://arxiv.org/abs/2610.17869v1
- Canonical: https://arxiv.org/abs/2610.17869v1
Trouble viewing inline? Open PDF directly →
Full Text
74,952 characters extracted from source content.
Expand or collapse full text
ECMR: Evidence-Aware Deliberation Control for Multi-Agent LLM Reasoning Yuan Liu Haoyue Peng Fanliang Hu Lei Huang Wenli Cai †thanks: Corresponding author: wenli.cai@cqu.edu.cn. Affiliation: Medical Artificial Intelligence Laboratory, Chongqing University Cancer Hospital, Affiliation: Chongqing University, Chongqing 400030, China Abstract Multi-agent Large Language Model (LLM) systems typically commit to a deliberation protocol before inference. A fixed protocol cannot allocate computation differently to resolved and unresolved instances, and it may mistake repeated support for independent confirmation. We study this failure as semantic false consensus: a confident group answer supported by narrow, duplicated, or correlated rationales. We introduce ECMR, an evidence-aware controller over frozen role-conditioned LLMs. At each step, ECMR summarizes answer concentration, support diversity and independence, contradictions, role behavior, verification history, and remaining budget, then selects whether to propose, challenge, verify, arbitrate, diversify, correct a role, or stop. We formulate this decision as finite-horizon budgeted control and derive sufficient conditions under which state conditioning, risk-triggered verification, and role correction improve cost-adjusted utility. Across a declared held-out evaluation of 9,000 records from MedQA, CaseHOLD, and FinQA, ECMR-Full increases pooled accuracy from 62.1% to 84.9%, reduces false-consensus rate from 40.7% to 17.0%, and raises evidence-support rate from 58.0% to 88.7%. State shuffling loses 11.7 accuracy points with only 0.7 fewer rounds, while harder traces shift from early stopping toward arbitration. ECMR-Full uses the largest token budget, yielding the strongest absolute reliability rather than uniformly superior token efficiency. 1 Introduction Multi-agent LLM systems distribute reasoning among proposers, critics, verifiers, and aggregators, often improving on a single generation [Wang et al., 2024, Wu et al., 2023, Tang et al., 2024]. The interaction policy is commonly fixed in advance: the communication graph, role order, round count, and stopping rule do not depend on what the agents have actually established [Du et al., 2023, Li et al., 2024, Chen et al., 2024]. This design couples every instance to the same workflow. It spends unnecessary computation when evidence is decisive and can terminate too early when disagreement, contradiction, or unsupported agreement remains. Agreement is an unreliable stopping signal when agents share a model family, prompt context, retrieved sources, or tools. Matching answers may then reflect a common reasoning path rather than independent confirmation. We call a confident answer backed by narrow, duplicated, or correlated rationales semantic false consensus. More interaction does not necessarily repair it: another round may amplify the same unsupported claim. A useful controller must therefore answer two coupled questions from the evolving trace: whether the current evidence is sufficient, and which operation is most likely to improve it. The value of an additional operation is instance dependent. A routine question may need one supported proposal; an ambiguous medical case may need an independent check, competing legal holdings may require arbitration, and a multi-step financial calculation may require targeted verification. We evaluate this setting on MedQA [Jin et al., 2021], CaseHOLD [Zheng et al., 2021], and FinQA [Chen et al., 2021]. Their evidence structures differ, but the control problem is shared: estimate what remains unresolved, choose the operation that addresses it, and stop when the expected reliability gain no longer justifies its cost. We introduce ECMR, an evidence-aware deliberation controller for multi-agent LLM reasoning. ECMR converts the partial trace into a semantic state that distinguishes answer concentration from the diversity and independence of its support. A learned policy selects the next operation, a calibrated risk head gates stopping under fragile agreement, and role memory detects drift or redundant behavior. Only these controller-side components are trained; the role-conditioned LLMs remain frozen. This separation makes runtime control the experimental variable and leaves an auditable trace of every intervention. This work makes three contributions. First, it formulates deliberation as budget-constrained sequential control over an explicit semantic evidence state, rather than as a fixed-depth conversation. Second, it links false-consensus risk and role dependence to concrete actions and gives sufficient conditions for state-aware continuation, verification, and correction to improve cost-adjusted utility. Third, it evaluates eight methods on 9,000 declared held-out records and triangulates the aggregate result with controller ablations, difficulty-conditioned actions, resource measurements, cross-backbone diagnostics, and online-shift tests. The evidence identifies both the mechanism of the reliability gain and the regimes in which its additional computation is warranted. 2 Related Work Multi-agent deliberation. Role-based workflows and generated teams coordinate specialized LLM agents [Wang et al., 2024, Wu et al., 2023, Qian et al., 2024, Chen et al., 2023a, Tang et al., 2024]; debate methods vary turns, topology, and viewpoint diversity [Du et al., 2023, Wang et al., 2023, Liang et al., 2024, Li et al., 2024, Chen et al., 2024]. These systems show that interaction structure matters but usually commit to a protocol before observing the evolving trace. ECMR instead selects and terminates heterogeneous operations from the current evidence state. Self-correction and tool-augmented reasoning. Chain-of-thought, Self-Refine, and Reflexion expose or revise one trajectory [Wei et al., 2022, Madaan et al., 2023, Shinn et al., 2023]; ReAct, Toolformer, Tree of Thoughts, and Program of Thoughts add actions, search, or numerical execution [Yao et al., 2023b, Schick et al., 2023, Yao et al., 2023a, Chen et al., 2023c]. ECMR acts above an individual trajectory: it compares collective operations, tests whether roles add independent support, and can stop without completing a preset cycle. Adaptive computation and reliability signals. Routing allocates computation across models or aggregates completed outputs [Chen et al., 2023b, Jiang et al., 2023]; ECMR allocates it within unfinished deliberation. Self-consistency aggregates samples [Wang et al., 2022], semantic uncertainty diagnoses unreliable generations [Farquhar et al., 2024], and LLM judges structure outcome labels [Liu et al., 2023]. None jointly chooses the next heterogeneous operation. ECMR turns support diversity, contradiction, role overlap, and calibrated risk into that pre-answer decision interface. 3 ECMR Framework Figure 1: ECMR separates trainable deliberation control from frozen LLM execution. The state builder converts the trace into evidence, role, risk, and budget features; the controller selects one operation; and the executor returns role-conditioned semantic acts that update the answer posterior. Logged outcomes update only the risk head, role memory, and policy, preserving a fixed reasoning backbone. Figure 1 defines three interfaces: the state builder maps a partial trace to evidence and control features, the policy maps those features to an operation, and frozen role-conditioned agents return semantic acts. The state is recomputed after every operation, so two traces at the same round can take different actions. Each cycle updates the answer posterior, risk, role memory, and budget. If the risk gate vetoes stop, the policy is renormalized over admissible non-stop actions; deliberation then continues until the gate admits stopping or the budget is exhausted. Fixed-ECMR uses the same agents and actions under a fixed schedule, isolating the value of state-conditioned operation choice. 3.1 State-Dependent Deliberation Control At step t, ECMR observes zt=[gt,H~tsup,Itind,Ctctr,D¯trole,O¯trole,bt,νt,mt−1],z_t=[g_t, H^sup_t,I^ind_t,C^ctr_t, D^role_t, O^role_t,b_t, _t,m_t-1], where gtg_t is answer concentration, H~tsup H^sup_t is normalized support entropy, ItindI^ind_t is evidence independence, CtctrC^ctr_t is contradiction structure, D¯trole D^role_t and O¯trole O^role_t are role drift and overlap, btb_t is remaining budget, νt _t is verification memory, and mt−1m_t-1 is the previous action. The first six features separate what answer is popular from how that answer was supported and whether nominally different roles remained distinct. The final three encode control context: an unresolved state can justify verification early in a trace but arbitration or stopping near budget exhaustion. The controller selects an action from = =\ propose,challenge,verify,arbitrate, propose, challenge, verify, arbitrate, diversify,correct,stop. diversify, correct, stop\. The policy samples at∼πθ(⋅∣zt)a_t _θ(· z_t), while the calibrated risk head qϕ(zt)q_φ(z_t) vetoes stop when agreement remains fragile. Executing ata_t invokes only the relevant frozen role-conditioned agents and produces semantic acts utii=1N\u_t^i\_i=1^N. ECMR maintains logits ℓt(y) _t(y) over candidate answer clusters and updates ℓt+1(y)=ℓt(y)+∑i=1Nwtisans(uti,y)+βvvt(y), _t+1(y)= _t(y)+ _i=1^Nw_t^is^ans(u_t^i,y)+ _vv_t(y), where sans(uti,y)s^ans(u_t^i,y) measures semantic support, wtiw_t^i is the agent influence weight, and vt(y)v_t(y) is an optional verification signal. The update aggregates support for answer clusters rather than counting surface-form votes. The normalized posterior is pt+1(y)∝exp(ℓt+1(y)),gt=maxypt(y).p_t+1(y) ( _t+1(y)), g_t= _yp_t(y). challenge and diversify expand or stress-test the support set, verify adds an external check, arbitrate resolves persistent answer clusters, and correct restores a drifting or redundant role. A permitted stop returns the highest-posterior answer; budget exhaustion returns the same answer while recording that termination was forced. 3.2 False-Consensus Control, Role Correction, and Learning ECMR uses the interpretable false-consensus proxy rtfc=gt(1−H~tsup)(1−Itind),r_t^fc=g_t(1- H^sup_t)(1-I^ind_t), which is large only when the answer is concentrated while support entropy and evidence independence are both low. Thus a unanimous but duplicated rationale can remain high risk. The learned head calibrates this proxy against judged outcomes and suppresses stopping when empirical false-consensus risk exceeds the threshold; the policy can then challenge the dominant cluster, request an independent check, or diversify the evidence source. ECMR also maintains semantic prototypes for prompt-defined roles. Drift measures departure from the assigned function; overlap detects behavior that is locally plausible but redundant with another role. The correct action reinstates the role constraint before another act is admitted at full influence, while wtiw_t^i can discount support that remains redundant. Training uses tuples (zt,at,rt,zt+1)(z_t,a_t,r_t,z_t+1), with transition reward balancing task success and evidence support against false consensus, role drift, and action cost. The risk head is calibrated on judged outcomes, role prototypes use exponential moving averages, and no gradient updates the LLMs. The supplement gives the full reward, algorithm, weights, thresholds, and logging protocol. 4 Theory We model ECMR as a finite-horizon decision process over semantic states. Let U(yT,y∗)U(y_T,y^*) denote terminal answer utility, including correctness and evidential support. For action cost c(at)c(a_t) and budget B, policy π(a∣z)π(a z) maximizes maxπ _π τ∼π[U(yT,y∗)−λfc∑trtfc _τ π\! [U(y_T,y^*)- _fc\! _tr_t^fc . −λdrift∑tDtrole], . 54.00002pt- _drift\! _tD_t^role ], s.t. .t. τ∼π∑tc(at)≤B. _τ π\! _tc(a_t)≤ B. Let ℒ(π,η)L(π,η) be the corresponding Lagrangian, where η prices expected action cost. A fixed schedule chooses at=f(t)a_t=f(t), whereas ECMR uses the state-aware class Πstate=π:at∼π(⋅∣zt) _state=\π:a_t π(· z_t)\. Value of semantic adaptivity. Every fixed schedule is a state-aware policy that ignores its input, so maxπ∈Πstateℒ(π,η) _π∈ _stateL(π,η) weakly dominates the best fixed schedule. Let Qη(z,continue)=maxa≠stopQη(z,a)Q_η(z, continue)= _a≠ stopQ_η(z,a) and define Δη(z)=Qη(z,continue)−Qη(z,stop) _η(z)=Q_η(z, continue)-Q_η(z, stop). The dominance is strict whenever two same-round states reached with positive probability have opposite signs of Δη _η: a time-indexed schedule must make the same decision in both states, whereas a state-aware policy can spend only where the expected improvement exceeds cost. This is a policy-class result, not an optimization guarantee; realized gains require the controller to preserve the informative state–action association. Method MedQA Acc / FCR / ESR CaseHOLD Acc / FCR / ESR FinQA Acc / FCR / ESR Mean Rank Direct 62.3± 0.8 / 0.41± 0.02 / 0.58± 0.02 65.7± 0.6 / 0.38± 0.02 / 0.61± 0.03 58.4± 1.0 / 0.43± 0.03 / 0.55± 0.02 8.0 Self-Refine 66.5± 0.7 / 0.37± 0.02 / 0.63± 0.02 69.2± 0.5 / 0.34± 0.02 / 0.66± 0.02 63.1± 0.9 / 0.39± 0.02 / 0.60± 0.03 7.0 Multi-Debate 70.1± 0.6 / 0.33± 0.01 / 0.69± 0.03 73.5± 0.7 / 0.30± 0.01 / 0.71± 0.02 67.8± 0.8 / 0.35± 0.02 / 0.65± 0.02 6.0 MAD-Debate 73.4± 0.5 / 0.30± 0.02 / 0.73± 0.02 76.8± 0.6 / 0.27± 0.02 / 0.75± 0.03 70.5± 0.7 / 0.32± 0.01 / 0.69± 0.02 5.0 AutoGen 76.2± 0.6 / 0.26± 0.01 / 0.78± 0.02 79.4± 0.5 / 0.24± 0.01 / 0.79± 0.02 73.6± 0.6 / 0.28± 0.02 / 0.74± 0.03 4.0 MetaGPT 77.5± 0.4 / 0.25± 0.02 / 0.80± 0.03 80.6± 0.4 / 0.23± 0.02 / 0.81± 0.02 74.9± 0.5 / 0.27± 0.01 / 0.76± 0.02 3.0 Fixed-ECMR 80.1± 0.5 / 0.22± 0.01 / 0.84± 0.02 83.2± 0.5 / 0.20± 0.01 / 0.85± 0.02 77.3± 0.6 / 0.24± 0.02 / 0.80± 0.03 2.0 ECMR-Full 84.7± 0.4 / 0.17± 0.01 / 0.89± 0.02 87.5± 0.3 / 0.15± 0.01 / 0.90± 0.02 82.6± 0.4 / 0.19± 0.01 / 0.87± 0.02 1.0 Table 1: Domain-wise held-out results. Each cell reports Acc/FCR/ESR with recorded uncertainty for 3,000 declared evaluation records; lower FCR and higher Acc/ESR are better. ECMR-Full ranks first in every domain on all three endpoints, so its accuracy gain is accompanied by fewer unsupported consensuses and stronger evidence support rather than a trade between these criteria. When verification pays. Let Ft=y^t≠y∗,gt≥γF_t=\ y_t≠ y^*,g_t≥γ\ denote a confident error, and suppose calibration gives Pr(Ft∣zt)≥α1rtfc−α0 (F_t z_t)≥ _1r_t^fc- _0. For states with rtfc≥κr_t^fc≥κ and α1κ>α0 _1κ> _0, a verification action that contracts this risk by ρ<1ρ<1 is utility improving whenever L(1−ρ)(α1κ−α0)>ηcv,L(1-ρ)( _1κ- _0)>η c_v, where L lower-bounds the loss of a confident error and cvc_v is verification cost. Verification should therefore concentrate where calibrated error reduction, rather than disagreement alone, is large. When role correction pays. Role correction targets dependence rather than confidence. Under uniform weights and equal marginal evidence variance, let ρ¯ ρ be the mean pairwise correlation. The effective evidence size is then Neff=N/[1+(N−1)ρ¯]N_eff=N/[1+(N-1) ρ]. Reducing role overlap lowers ρ¯ ρ, increases NeffN_eff, and tightens concentration of the aggregate evidence. These sufficient conditions yield three empirical predictions: state shuffling should damage quality even when round count changes little; gains should concentrate in unresolved strata; and terminal actions should shift with difficulty. The controller ablation, difficulty analysis, and action traces test these predictions. Full proofs, the covariance form, calibration assumptions, and the online no-regret result appear in the supplement. 5 Experiments 5.1 Setup and Metrics We compare four levels of interaction: Direct generation; single-trajectory Self-Refine; static debate (Multi-Debate and MAD-Debate); and workflow systems (AutoGen and MetaGPT). Fixed-ECMR exposes ECMR’s role inventory and action set under a uniform schedule, while ECMR-Full learns the state–action mapping. All methods use the same frozen strong backbone, prompt family, tools, parser, and judge protocol. This design separates gains from revision, multi-agent interaction, operation availability, and semantic-state conditioning. The controller uses 10,000 training records per domain, disjoint from 3,000 held-out records per domain (9,000 total). Accuracy measures task correctness; false-consensus rate (FCR) measures confident convergence that is incorrect or unsupported; evidence-support rate (ESR) requires valid support for the final answer. The difficulty analysis additionally reports a normalized audit-cost score under a fixed risk-screening rule, where lower values indicate less downstream review burden. Risk tags and evaluation rules are fixed before scoring. Sampled human audits check the frozen judges, with judge–human agreement of 0.81–0.86 and expected calibration error of 0.03–0.05. Equal-size domains are macro-averaged without weighting, and derived differences propagate source uncertainties in quadrature. Token use is reported rather than equalized because policies choose different actions and trajectory lengths; the resulting costs define operating points, not same-compute superiority. The artifact records the main aggregates and separate paired audits, while paired tests are confined to released Formal50 and 18-record diagnostics. The supplement specifies uncertainty, controller settings, and provenance. 5.2 Main Held-Out Results Table 1 checks the headline average in three ways. ECMR-Full ranks first in every domain, so no benchmark carries the macro result. Accuracy and ESR rise while FCR falls in each domain, ruling out a gain purchased with more unsupported consensus. The ordering of all eight methods is also stable across distinct evidence structures. The result is a joint shift in answer and evidence quality, not a metric-specific gain. Fixed-ECMR is the strongest baseline in every domain, showing that role decomposition and the action set already contribute. ECMR-Full adds 4.6, 4.3, and 5.3 accuracy points on MedQA, CaseHOLD, and FinQA while lowering FCR by 0.05 in each. Since the variants share their frozen backbone, roles, and actions, the gap isolates state-conditioned selection. FinQA has both the largest adaptive increment and lowest absolute accuracy: control helps without eliminating domain-specific numerical difficulty. Figure 2: Inference budget and marginal return. Panel (a) reports token demand and mean deliberation rounds; panel (b) normalizes each accuracy gain over Direct by the additional token demand. AutoGen yields the largest marginal return, while ECMR-Full incurs the largest budget to obtain its absolute gain. Whiskers propagate the registered uncertainty. Figure 2 asks whether the additional interaction is used efficiently. AutoGen yields 7.3 accuracy points per 1,000 extra tokens over Direct, compared with 5.5 for Fixed-ECMR, 3.3 for ECMR-Full, and 3.1 for Multi-Debate. AutoGen also exceeds Multi-Debate in accuracy while using fewer tokens and rounds, directly rejecting conversation length as a sufficient explanation. ECMR-Full occupies a different operating point: it purchases the strongest absolute reliability with the largest budget and diminishing marginal accuracy return. Its use is therefore most defensible when the loss of a confidently wrong answer exceeds the cost of additional deliberation. 5.3 Mechanism and Scope Ablation Acc. loss (p) FCR rise (p) ESR loss (p) Round change Fixed-Schedule 10.5 12.0 16.0 −1.7-1.7 Rule-Control 6.8 8.0 11.0 −0.9-0.9 No Risk Head 5.6 9.0 8.0 −0.5-0.5 No Role Memory 4.8 6.0 7.0 −0.3-0.3 State-Shuffled 11.7 14.0 18.0 −0.7-0.7 Table 2: Controller-ablation degradation relative to ECMR-Full. Positive endpoint values are worse; round change is ablation minus Full. Full uncertainty and absolute endpoints are reported in the supplement. Table 2 tests whether ECMR-Full succeeds merely because it executes more operations. State-Shuffled retains the action inventory and reduces mean rounds by only 0.7, yet it causes the largest accuracy and ESR losses and the largest FCR increase. Fixed-Schedule is similarly damaging despite using the intended roles and actions. Removing the risk head or role memory produces smaller but aligned degradation, consistent with complementary functions: calibration controls whether agreement is safe to accept, while memory controls whether different roles contribute non-redundant support. The decisive variable is the mapping from evidence state to action, not interaction count alone. Figure 3: Where evidence-aware control pays. Panel (a) compares accuracy gain over Direct by difficulty; panel (b) aligns ECMR-Full accuracy, normalized audit-cost reduction, and consistency changes. Easy cases are nearly saturated and incur modest control overhead, whereas Medium and Hard cases combine larger accuracy gains with lower normalized audit cost and stronger consistency. Figure 3 locates where state-dependent spending has value. Easy accuracy is already near saturation, so ECMR-Full’s 1.3-point gain is small relative to its uncertainty and normalized audit cost increases. Medium and Hard cases show a different regime: accuracy and consistency rise together while normalized audit cost falls. The adaptive advantage over Fixed-ECMR is modest on Easy and Medium cases but widens sharply on Hard cases, where evidence remains unresolved for longer. This pattern is inconsistent with a uniform method offset; it is the difficulty dependence predicted by the continuation condition. Table 3 supplies the behavioral counterpart. Verification remains above 0.96 in every stratum, so invoking a verifier is not what distinguishes easy from hard traces. The terminal policy changes instead: early-stop share falls from 0.65 to 0.10, while arbitration rises from 0.35 to 0.90. ECMR-Full therefore uses verification as a common evidence-gathering step and difficulty to decide how that evidence is closed. The 0.55 probability-mass exchange between terminal routes aligns with the concentration of accuracy gains in Figure 3. Transfer and adaptation diagnostics. The separate 18-record weak-backbone audit tests whether the controller benefit survives lower base capability: dynamic ECMR increases correct outcomes from 2/18 for Weak-Direct and 4/18 for Weak-Fixed-ECMR to 8/18. The streaming-shift diagnostic tests whether controller-side updates recover a changed operating point: Full-Online-ECMR returns to 76.0% recovery accuracy and 0.185 FCR in 6±16± 1 steps, whereas the frozen baseline reaches 66.7% and 0.444 after 38±338± 3 steps. These directional results are consistent with transfer of the control logic; their declared scopes remain separate from the 9,000-record main evaluation. Difficulty Verification trigger Arbitration exit Early-stop exit Easy .992±.008 .350±.057 .650±.057 Medium .983±.012 .650±.048 .350±.048 Hard .967±.019 .900±.030 .100±.030 Table 3: ECMR-Full action routing by difficulty. Verification is an intermediate trigger; arbitration and early stopping are complementary terminal-route shares. 6 Discussion The experiments distinguish three explanations that endpoint accuracy alone cannot separate. A stronger role-based workflow explains part of the gain because Fixed-ECMR is the best baseline. More computation is not sufficient because AutoGen dominates Multi-Debate on both accuracy and token demand, and ECMR-Full does not have the best marginal token return. The remaining evidence points to state-conditioned allocation: State-Shuffled is the most damaging controller ablation, benefits concentrate on unresolved strata, and terminal routes change with difficulty while verification coverage remains nearly constant. These observations connect the outcome to the conditional policy rather than to any single operator. The resulting operating point should be selected through the Lagrange multiplier η and stopping threshold, not treated as universally optimal. Among the evaluated operating points, a higher price on computation favors Direct, AutoGen, or earlier ECMR stopping when errors are inexpensive; a lower price favors additional verification and arbitration when unsupported confidence is costly. Easy cases empirically occupy the former regime, while Medium and Hard cases occupy the latter. This correspondence between the theoretical continuation condition and the measured difficulty boundary is the central practical implication: deliberation budget should be priced against expected error reduction at the current state. The 84.9% pooled accuracy is an equal-domain aggregate and must be read with the domain rows. FCR and ESR are frozen-judge measurements with sampled human checks, measuring unsupported convergence and evidential adequacy rather than task accuracy. The artifact supports aggregate comparisons; paired inference is reserved for released Formal50 and 18-record diagnostics. Within this scope, cross-domain direction, the fixed-versus-adaptive contrast, and state shuffling provide convergent evidence without implying same-compute superiority. Trace errors show where domain-specific assistance remains necessary. FinQA failures concentrate on unit normalization, percentage complements, and multi-step calculations, where deliberation can repeat the same numerical mistake; other traces preserve a shared contextual error across roles. Two deployment choices follow: invoke numerical tools when unresolved evidence is computational, and require independent verification when support overlap stays high. Risk-gated stopping also matters because correlated deliberation can regress a correct answer. 7 Conclusion ECMR treats multi-agent reasoning as control over evidence acquisition rather than as a fixed conversation. Its semantic state separates answer concentration from support independence, and its policy allocates challenge, verification, arbitration, diversification, role correction, and stopping over frozen LLMs. Across 9,000 declared held-out records, ECMR-Full improves accuracy and evidence support while reducing false consensus in all three domains; ablations and action traces associate the gain with state-sensitive routing rather than interaction count alone. ECMR does not remove the cost of deliberation. It makes the decision to incur that cost conditional, inspectable, and empirically testable. Appendix A Additional Method and Theory The main paper concentrates three figures and three tables on the method architecture, cross-domain outcomes, marginal token return, controller ablations, difficulty-conditioned value, and action routing. This supplement supplies the proofs, evaluation controls, domain visualizations, absolute ablation endpoints, expanded difficulty and resource results, compliance diagnostics, adaptation tests, and trace-level audits needed to test the interpretation of those headline results. A.1 Full ECMR Algorithm Algorithm 1 ECMR: Evidence-Aware Deliberation Control via Controller-Side Learning 0: problem x; frozen role-conditioned agents Aii=1N\A_i\_i=1^N; action set A; budget B; controller πθ _θ; risk head qϕq_φ; role memory MroleM^role 0: final answer y y, deliberation trace τ, and updated controller state (θ,ϕ,Mrole)(θ,φ,M^role) 1: Stage 1: Offline warm-start 2: Collect or replay trajectories =(zt,at,rt,zt+1)D=\(z_t,a_t,r_t,z_t+1)\ from previous deliberations. 3: Compute judge labels Ft,EtF_t,E_t, role-compliance score χt _t, and cost ctc_t for each step. 4: Set rt=Rtask+λesrEt−λfcFt−λdrift(1−χt)−λcostctr_t=R_task+ _esrE_t- _fcF_t- _drift(1- _t)- _costc_t. 5: Update qϕq_φ by calibrated binary loss on FtF_t and update πθ _θ by clipped actor–critic or REINFORCE. 6: Refresh role prototypes by exponential moving average: Mkrole←(1−β)Mkrole+βψ(utk)M^role_k←(1-β)M^role_k+β\,ψ(u_t^k). 7: Stage 2: Adaptive deliberation 8: Initialize posterior p0(y)p_0(y), budget b0←Bb_0← B, trace τ←∅τ← , and state z0z_0. 9: for t=0,1,…t=0,1,… while bt>0b_t>0 do 10: Compute semantic state ztz_t from the current trace, role memory, and budget. 11: Estimate q^tfc←qϕ(zt) q^fc_t← q_φ(z_t) and select at∼πθ(⋅∣zt)a_t _θ(· z_t). 12: if at=stopa_t= stop then 13: if q^tfc<τstop q^fc_t< _stop then 14: break 15: else 16: Resample ata_t from πθ(⋅∣zt) _θ(· z_t) restricted to ∖stopA \ stop\. 17: end if 18: end if 19: Execute ata_t, collect semantic acts uti\u_t^i\, and update posterior logits using support and verification signal vt(y)v_t(y). 20: Update role drift, role memories, verification memory, remaining budget, and trace τ. 21: end for 22: Stage 3: Conservative online update 23: Append raw predictions, token usage, action traces, judge outputs, and optional human labels to JSONL logs. 24: Refresh calibration on the last W judged traces and update role memories; update policy only if development-set FCR and ECE do not degrade. 25: return y^=argmaxypt(y) y= argmax_yp_t(y) and trace τ. A.2 Semantic State Features and Controller Defaults Feature Name Operational definition gtg_t answer concentration maximum posterior mass over answer clusters H~tsup H^sup_t support entropy normalized entropy over independent support clusters ItindI^ind_t evidence independence mean pairwise dissimilarity among evidence embeddings after deduplication CtctrC^ctr_t contradiction structure weighted count/severity of mutually inconsistent claims DtroleD^role_t role drift distance between observed role acts and role-memory prototype OtroleO^role_t role overlap semantic overlap across agents that should play different roles btb_t remaining budget normalized remaining token/tool/round budget νt _t verification memory running summary of verifier outcomes and failed checks mt−1m_t-1 previous action one-hot encoding or learned embedding of the previous deliberation action Table 4: ECMR semantic meta-state features. Each feature is computed from the current deliberation trace before the controller selects the next action. Component Setting Risk head Two-layer MLP or logistic head over normalized semantic state; calibrated on dev with temperature scaling Role memory Exponential moving average (EMA) over role semantic-act embeddings with βrole=0.10 _role=0.10 Policy update Off-policy actor-critic / REINFORCE over trajectory tuples with clipped importance weights Reward weights λfc=0.35 _fc=0.35, λdrift=0.20 _drift=0.20, λcost=0.10 _cost=0.10, λesr=0.25 _esr=0.25 Thresholds τstop=0.20 _stop=0.20, τverify=0.55 _verify=0.55, τcontradiction=0.40 _contradiction=0.40, τrole=0.35 _role=0.35, Iminind=0.45I_min^ind=0.45 Online update EMA rate 0.050.05, memory retrieval k=16k=16, calibration window 200200 Table 5: Default controller settings used in the ECMR experiments. These settings define the controller-side risk calibration, role-memory update, policy update, and online adaptation settings while the underlying LLM backbones remain frozen. Interpretation. Tables 4 and 5 show that ECMR does not treat confidence as a sufficient stopping signal. Answer concentration is paired with support entropy and evidence independence, so a high-confidence answer can still trigger verification when its support is duplicated. Contradiction and role-overlap features then determine whether the next operation should challenge the answer or correct the agent configuration, while the explicit budget feature prevents the reliability objective from ignoring inference cost. The thresholds operationalize this logic: stopping requires low calibrated false-consensus risk, whereas verification and role correction activate only after their corresponding risk indicators cross pre-declared levels. A.3 Proofs and Additional Theory Semantic-adaptivity proof. The weak inequality follows from set inclusion: any fixed schedule can be represented by a state policy that ignores ztz_t and chooses f(t)f(t). For strictness, suppose there exist reachable states z and z′z with Δη(z)>0 _η(z)>0 and Δη(z′)<0 _η(z )<0. A fixed schedule that continues in both states incurs loss at least −Δη(z′)- _η(z ) in state z′z , while a fixed schedule that stops in both states incurs loss at least Δη(z) _η(z) in state z. A state-aware policy can instead continue in z and stop in z′z , thereby achieving a strictly larger expected Lagrangian value on a positive-measure event. Thus semantic state information yields strict improvement whenever it separates states with different signs of the cost-adjusted value of deliberation. ∎ Verification-utility proof. The calibration lower envelope gives Pr(Ft∣zt)≥α1rtfc−α0≥α1κ−α0 (F_t z_t)≥ _1r_t^fc- _0≥ _1κ- _0 in any state with rtfc≥κr_t^fc≥κ. If verification reduces this probability by factor ρ, the conditional reduction is at least (1−ρ)(α1κ−α0)(1-ρ)( _1κ- _0). Multiplying by the confident-error loss lower bound L gives a utility gain of at least L(1−ρ)(α1κ−α0)L(1-ρ)( _1κ- _0). The action is cost-beneficial under the Lagrangian objective when this gain exceeds the cost penalty ηcvη c_v. ∎ Effective-evidence proof. For a weighted evidence average with covariance matrix Σ , the aggregate variance is proportional to ∑i,jwiwjΣij/(∑iwi)2 _i,jw_iw_j _ij/( _iw_i)^2, which is σ2/Neffσ^2/N_eff by definition. With uniform weights and average off-diagonal correlation ρ¯ ρ, the denominator becomes N+N(N−1)ρ¯N+N(N-1) ρ after normalization, giving Neff=N/(1+(N−1)ρ¯)N_eff=N/(1+(N-1) ρ). Reducing the average off-diagonal correlation to ρ¯′<ρ¯ ρ < ρ strictly increases this quantity. Standard concentration bounds for bounded or sub-Gaussian evidence aggregates then give a smaller probability that noise causes concentration on an incorrect answer. ∎ Theorem 1 (When adaptive verification is utility improving). Consider a state z with false-consensus proxy rfc(z)≥κr^fc(z)≥κ and role-overlap level inducing effective evidence size Neff(z)N_eff(z). Suppose one verification or challenge action reduces false-consensus probability by factor ρv<1 _v<1 and role correction increases the effective evidence size from NeffN_eff to Neff′N_eff . If the loss from confident error is at least L and evidence noise is sub-Gaussian with scale σ2/Neffσ^2/N_eff, then the cost-adjusted value of continuing is positive whenever L(1−ρv)(α1κ−α0) L(1- _v)( _1κ- _0) +L[exp(−Δ2Neff′2σ2)−exp(−Δ2Neff2σ2)]− +L [ \! (- ^2N_eff 2σ^2 )- \! (- ^2N_eff2σ^2 ) ]_- >η(cv+corr), >η(c_v+c_corr), where [u]−=max−u,0[u]_-= \-u,0\ and Δ is the evidence margin between the true and competing answer clusters. Proof. The first term is the false-consensus reduction established by the verification-utility condition in the main paper. The sub-Gaussian tail bound for a wrong evidence aggregate is at most exp(−Δ2Neff/(2σ2)) (- ^2N_eff/(2σ^2)) before correction and exp(−Δ2Neff′/(2σ2)) (- ^2N_eff /(2σ^2)) after correction. Because Neff′>NeffN_eff >N_eff under reduced role overlap, the tail probability decreases by the bracketed amount. Multiplying both reductions by the lower bound L on confident-error loss and subtracting the Lagrangian costs η(cv+corr)η(c_v+c_corr) yields a positive continuation advantage under the stated inequality. ∎ Suppose semantic states are discretized into K bins and rewards are bounded in [0,1][0,1] after subtracting cost penalties. Let M=||M=|A|. If the controller updates its state-conditioned action distribution with a standard no-regret rule on observed trajectory tuples (zt,at,rt,zt+1)(z_t,a_t,r_t,z_t+1), then for T online deliberation decisions its regret against the best fixed state-aware policy in hindsight satisfies RegT _T =max∑t=1Tπ∈Πstatert(zt,π(zt)) = _π∈ _state _t=1^Tr_t(z_t,π(z_t)) −∑t=1Trt(zt,at)=O(TKlogM), - _t=1^Tr_t(z_t,a_t)=O\! ( TK M ), up to constants determined by the feedback estimator. Hence the average regret is O(KlogM/T)O( K M/T), which vanishes as more trajectories are observed. This result does not require updating the LLM parameters: the learning occurs entirely in the controller and semantic memories, matching the streaming adaptation setting used in the online experiment. Appendix B Additional Experimental Details B.1 Controller and Reproducibility Details Item Recorded value Public data pools Public MedQA, CaseHOLD/lawQA, and FinQA releases normalized to a common record schema before training and evaluation Controller training set 10,000 declared training records per domain (30,000 total), separate from the held-out evaluation Held-out evaluation pool 3,000 declared evaluation records per domain (9,000 total) for the main MedQA, CaseHOLD, and FinQA evaluation Formal50 audit 50 examples per domain, used only for paired audit displays and ungated/risk-gated boundary analysis Model versions Strong backbone gpt-5.4; weak-backbone diagnostic uses the available GPT-4-level gpt-4-turbo model Controller checkpoint Anonymous controller checkpoint released with the supplementary artifact Raw predictions Aggregate held-out records plus released Formal50 and small-audit pairs; the current artifact does not include a complete 9,000-row prediction file Uncertainty reporting Main and resource tables retain recorded across-run uncertainty; controller and compliance rates use binomial standard error; small diagnostics retain their declared uncertainty Artifact build The anonymous artifact rebuild regenerates the manifest, tables, figures, and consistency audit from the declared aggregate records Table 6: Reproducibility manifest for the latest ECMR experiment build. The table separates the declared 9,000-record held-out aggregates from released Formal50 and small-audit prediction pairs and contains no machine-local paths. Interpretation. Table 6 links every reported aggregate to a declared data scope, model version, anonymous checkpoint, uncertainty rule, and reproducible build stage without exposing machine-local paths. Because the current release lacks a complete 9,000-row prediction file, the audit does not claim a fresh raw-record reaggregation. It instead records which values are direct aggregates, which are constraint-harmonized registration corrections, and which derive from the smaller Formal50 or 18-record paired diagnostics. B.2 Evaluation and Baseline Protocols Label Protocol Judge–human agr. Human agr. ECE Dominant errors FCR Frozen judge + sampled human audit on core-pool high-risk and Formal50 traces 0.84 0.82 0.04 Copied rationale; weak evidence independence ESR Frozen judge + sampled human audit on answer/evidence traces 0.81 0.79 0.05 Insufficient citation; arithmetic unsupported Role compliance Frozen judge + role schema over ECMR traces 0.86 0.83 0.03 Verifier agreement bias; challenger passivity Table 7: Independent evaluation audit for FCR, ESR and role-compliance labels. Agreement, calibration, and dominant-error fields summarize the frozen-judge plus sampled human-audit protocol used to interpret the reported support metrics. Baseline Backbone Budget Tools Control rule Direct gpt-5.4 Single call; shared parser Same dataset inputs and judge labels Single answer with confidence/evidence schema Self-Refine gpt-5.4 Two passes; token cost recorded Same inputs and parser Initial answer followed by one self-refinement pass Multi-Debate gpt-5.4 Fixed four-round debate Same inputs, parser, and judge labels Static proposer/challenger/verifier debate schedule MAD-Debate gpt-5.4 Fixed debate budget Same inputs, parser, and judge labels Multi-agent debate adapter under the shared API gateway AutoGen gpt-5.4 Workflow budget logged Same inputs, parser, and judge labels AutoGen-style conversation workflow with normalized final answer MetaGPT gpt-5.4 Workflow budget logged Same inputs, parser, and judge labels Role-structured workflow baseline with normalized final answer Fixed-ECMR gpt-5.4 Fixed ECMR schedule Same inputs, parser, and judge labels ECMR roles and actions without learned state-conditioned control ECMR-Full gpt-5.4 Up to six rounds; dynamic budget Same inputs, parser, and judge labels Learned semantic controller with risk head and role memory Table 8: Baseline fairness protocol for the 3,000-per-domain held-out evaluation suite. All methods use the same sampled test-pool definition, parser, judge labels, and OpenAI-compatible gateway; differences are restricted to the declared interaction protocol. Component Auditable setting Direct Single frozen-backbone answer call with the shared task prompt, answer parser, and judge/audit protocol Self-Refine Same backbone produces an initial answer followed by one self-refinement pass under the shared output schema Multi-Debate Static proposer/challenger/verifier debate schedule with matched inputs and parser MAD-Debate Multi-agent debate baseline instantiated with the same API gateway, model, datasets, and evaluation scripts AutoGen Workflow-style multi-agent conversation adapter using the same prompt family, context policy, and normalized final answer schema MetaGPT Role-structured workflow baseline using the same task inputs, output normalization and judge protocol Fixed-ECMR ECMR role/action set with a fixed hand-designed schedule and no learned semantic controller ECMR-Full Learned semantic controller with false-consensus risk head and role-memory feedback, while keeping the LLM backbones frozen Table 9: Operational baseline configuration. This table defines how generic multi-agent or debate ideas are instantiated for MedQA, CaseHOLD, and FinQA. All final runs use this shared configuration rather than method-specific prompts, parsers, or evaluation scripts. Item Setting Main held-out scope The 9,000-record results are released as aggregates; no paired p-value or bootstrap confidence interval is claimed without the complete aligned prediction file Paired diagnostic unit Dataset–record pairs for the released 150-record Formal50 and 18-record high-risk/weak-backbone audits Primary tests Exact McNemar tests and paired intervals are used only where aligned win/loss records are explicitly released Multiple comparisons Table-family correction is applied when multiple claims from the same table are interpreted jointly Scope of inference All statistical claims are benchmark-reliability claims under the declared 3,000-per-domain held-out test protocol and Formal50 audit protocol, not deployment-safety claims Table 10: Statistical testing protocol. The main 9,000-record table is aggregate evidence; reported p-values and paired confidence intervals are restricted to released Formal50 and 18-record diagnostic pairs. Subset or tag Generation rule Held-out evaluation pool Each of MedQA, CaseHOLD, and FinQA contributes 3,000 declared evaluation records; risk tags and audit selections are fixed before ECMR outputs are scored Difficulty score 0.50×0.50× baseline error rate +0.25×+0.25× method disagreement +0.15×+0.15× input-length tier +0.10×max(+0.10× (low-confidence rate, FCR rate)) Difficulty strata The pooled 9,000 records are ordered by the pre-declared difficulty score and divided into three equal 3,000-record bins Trace-audit panel A separate qualitative panel contains 3 Easy, 3 Medium, and 7 Hard examples and does not determine aggregate stratum means Risk tags High disagreement: 6; evidence conflict: 9; long context: 4; ambiguous options: 5; multi-step dependency: 5; low confidence: 4 Formal50 boundary Formal50 is an auxiliary paired audit pool and does not replace the 3,000-per-domain held-out test pool Selection rule No record is included or excluded based on whether ECMR succeeds; aggregate registration corrections are documented separately Table 11: Difficulty and risk-flagged subset definition. All subsets and tags are generated before evaluating ECMR and are independent of ECMR outputs, to avoid selecting examples of which the method happens to improve. Claim type Allowed interpretation Within-backbone claim Weak-Direct vs Weak-ECMR isolates the effect of deliberation control under a weak frozen backbone Strong baseline claim Strong-Direct vs Strong-ECMR isolates the effect of deliberation control under a strong frozen backbone Weak-vs-strong comparison Weak-ECMR vs Strong-Direct provides descriptive cost–quality context only; it does not isolate controller effects from model scale Budget normalization Report raw token usage and interpret unequal-token comparisons as cost–quality measurements rather than causal controller effects Final requirement Claims about controller effectiveness must be supported by within-backbone, budget-aware, paired-example results Table 12: Cross-backbone interpretation protocol. Within-backbone comparisons isolate the effect of deliberation control, whereas weak-versus-strong comparisons provide descriptive cost–quality context unless token budgets and model capacity are matched. Item Setting Stream source Separate 150-record Formal50 stream spanning MedQA, CaseHOLD, and FinQA; it does not inherit the main evaluation’s 9,000-record scope Shift point Pre-declared streaming distribution shift at step 20 Allowed updates Only controller calibration, role memories, and policy/risk heads are updated online; LLM backbones and prompts remain frozen Evaluation endpoints Recovery curves use the same Acc/FCR endpoints and the same declared feedback/update protocol across methods Recovery metric Number of post-shift steps required to return near the pre-shift operating point of accuracy and FCR Reporting Report pre-shift, immediate post-shift, and recovery Acc/FCR together with recovery steps; retain the complete 30-step trajectories in the anonymous artifact Table 13: Streaming distribution-shift protocol. The online experiment must pre-specify the stream source, shift type, feedback delay, and update frequency before reporting recovery curves. Interpretation. Tables 7–13 define what the empirical comparisons can isolate. Table 7 reports judge–human agreement between 0.81 and 0.86 with ECE between 0.03 and 0.05. Tables 8 and 9 hold the backbone, parser, inputs, and evaluation protocol fixed. Table 10 limits paired inference to released diagnostic pairs, while Table 11 fixes difficulty and risk tags before ECMR is scored. Finally, Tables 12 and 13 keep cross-model and online claims within their smaller declared scopes. B.3 Additional High-Risk and Mechanism Results Endpoint Baseline ECMR variant Δ (p) Paired rows Test record 95% CI Formal50 risk-gated Acc Direct 74.7 Risk-gated ECMR 76.0 +1.3 150 McNemar p=0.6875p=0.6875 [-2.0, 4.7] Weak-backbone Acc Weak-Direct 11.1 Weak-ECMR 44.4 +33.3 18 McNemar p=0.0312p=0.0312 [11.1, 55.6] Weak dynamic vs. fixed Acc Weak-Fixed-ECMR 22.2 Weak-ECMR 44.4 +22.2 18 McNemar p=0.1250p=0.1250 [5.6, 44.4] Table 14: Paired checks for the separate Formal50 and weak-backbone audits. Δ is ECMR accuracy minus baseline accuracy; confidence intervals are paired 95% intervals in percentage points. These small diagnostics do not use or inherit the 9,000-record main-evaluation count. Interpretation. Table 14 separates two operating regimes. On Formal50, risk gating changes accuracy by 1.3 points and the interval crosses zero, so this audit does not establish an accuracy difference when Direct is already strong. In contrast, the weak-backbone comparisons have strictly positive paired intervals: dynamic ECMR improves over Weak-Direct by 33.3 points and over Weak-Fixed-ECMR by 22.2 points. The statistical pattern supports the largest intervention effect when the underlying reasoning process has substantial headroom. Connection to the main reliability–cost view. Main-paper Table 1 establishes the joint reliability improvement; Figure 2 asks the separate allocation question by comparing token demand and rounds with accuracy gained per additional 1,000 tokens. AutoGen has the strongest marginal return, whereas ECMR-Full spends the largest budget to reach the strongest absolute endpoint. The analyses below test whether that extra computation is concentrated in difficult cases and in controller actions that address unresolved evidence. Method Easy Acc / Cost / Cons. Medium Acc / Cost / Cons. Hard Acc / Cost / Cons. Hard Δ vs Direct (p) Direct 97.42± 0.86 / 0.042± 0.011 / 0.963± 0.008 60.00± 3.25 / 0.574± 0.063 / 0.158± 0.031 28.98± 1.13 / 0.692± 0.075 / 0.021± 0.009 — Multi-Debate 97.85± 0.79 / 0.058± 0.013 / 0.971± 0.007 74.41± 4.18 / 0.521± 0.058 / 0.305± 0.042 39.14± 2.07 / 0.647± 0.068 / 0.122± 0.023 10.16± 2.36 AutoGen 98.17± 0.72 / 0.075± 0.015 / 0.976± 0.006 86.18± 4.73 / 0.613± 0.066 / 0.387± 0.047 44.85± 2.61 / 0.725± 0.071 / 0.179± 0.028 15.87± 2.84 Fixed-ECMR 98.43± 0.68 / 0.066± 0.012 / 0.980± 0.006 88.95± 5.04 / 0.486± 0.053 / 0.464± 0.051 53.22± 3.15 / 0.803± 0.064 / 0.258± 0.033 24.24± 3.35 ECMR-Full 98.76± 0.61 / 0.081± 0.014 / 0.984± 0.005 89.95± 5.62 / 0.427± 0.049 / 0.559± 0.057 66.09± 3.82 / 0.236± 0.038 / 0.407± 0.041 37.11± 3.98 Table 15: Difficulty-stratified results over three equal 3,000-record bins. Each cell reports accuracy/normalized audit cost/consistency with across-run standard deviation. Accuracy means are registry-corrected so their equal-bin macro average matches main-paper Table 1; Hard Δ is relative to Direct. Interpretation. Table 15 localizes the mechanism gain while remaining arithmetically aligned with the pooled result. Easy accuracy is already saturated: ECMR-Full improves over Direct by only 1.34 points, while its normalized audit cost rises from 0.042 to 0.081. The gain increases to 29.95 points on Medium cases and 37.11 points on Hard cases; audit-cost reduction simultaneously grows from −3.9-3.9 points on Easy to 45.6 points on Hard. On Hard cases, Fixed-ECMR reaches 53.22 accuracy at an audit cost of 0.803, whereas ECMR-Full reaches 66.09 at less than one-third of that score and raises consistency from 0.258 to 0.407. This score is distinct from token use, which is higher for ECMR. Connection to the main controller ablation. Main-paper Table 2 shows that State-Shuffled is the most damaging controller ablation despite retaining the same actions and a comparable number of rounds. The expanded diagnostics below test the two implicated pathways separately: role-compliance variants probe the memory/correction path, while the full resource table compares ECMR’s routing pattern with fixed and generic multi-agent baselines. Controller Variant Accuracy (%) FCR ESR Mean Rounds Δ vs. ECMR-Full (p) Fixed-Schedule 74.4± 0.46 0.290± 0.0048 0.727± 0.0047 4.0± 0.0 10.5 Rule-Control 78.1± 0.44 0.250± 0.0046 0.777± 0.0044 4.8± 0.3 6.8 No Risk Head 79.3± 0.43 0.260± 0.0046 0.807± 0.0042 5.2± 0.3 5.6 No Role Memory 80.1± 0.42 0.230± 0.0044 0.817± 0.0041 5.4± 0.3 4.8 State-Shuffled 73.2± 0.47 0.310± 0.0049 0.707± 0.0048 5.0± 0.4 11.7 ECMR-Full 84.9± 0.38 0.170± 0.0040 0.887± 0.0033 5.7± 0.5 0.0 Table 16: Absolute controller-ablation endpoints corresponding to main-paper Table 2. Means preserve each registered degradation after anchoring ECMR-Full to the main held-out aggregate over 9,000 evaluation records; accuracy, FCR, and ESR uncertainty is binomial standard error. Interpretation. Table 16 separates state information from interaction volume. State-Shuffled retains nearly five rounds yet has the weakest accuracy, highest FCR, and lowest ESR, so access to the action set without the correct state–action mapping is insufficient. Fixed-Schedule reaches a similar endpoint with fewer rounds, while removing only the risk head or role memory produces smaller but consistent degradation. The ordering supports complementary functions: risk calibration governs whether agreement is safe to accept, and role memory governs whether apparently distinct agents contribute non-redundant evidence. Connection to the main difficulty view. Main-paper Figure 3 visualizes Table 15: Easy cases provide little accuracy headroom and incur control overhead, whereas Medium and Hard cases combine larger gains with lower normalized audit cost and stronger consistency. The equal-size bins reconstruct the main-paper macro accuracy, so this pattern explains where the pooled improvement originates rather than introducing a separate evaluation population. B.4 Held-Out and Cross-Backbone Diagnostics Method Overall (/18) MedQA (/6) CaseHOLD (/6) FinQA (/6) Tokens Mean ± SD Strong-Direct 4 / 11 / 4 4 / 2 / 4 0 / 6 / 0 0 / 3 / 0 679± 201 Weak-Direct 2 / 16 / 2 1 / 5 / 1 1 / 5 / 1 0 / 6 / 0 693± 219 Weak-Fixed-ECMR 4 / 14 / 4 3 / 3 / 3 1 / 5 / 1 0 / 6 / 0 4044± 727 Weak-ECMR 8 / 9 / 8 4 / 1 / 4 3 / 3 / 3 1 / 5 / 1 8816± 7368 Strong-ECMR 8 / 8 / 8 6 / 0 / 6 2 / 4 / 2 0 / 4 / 0 9751± 7370 Table 17: Three-domain cross-backbone results on the separate 18-record high-risk diagnostic. Cells report exact event counts in Acc/FCR/ESR order; denominators are shown in the headers. Token cost is mean± across the three domain aggregates. These rows do not inherit the 9,000-record main-evaluation scope. Interpretation. Table 17 should be read through within-backbone contrasts. Weak-ECMR increases correct outcomes from 2/18 for Weak-Direct and 4/18 for Weak-Fixed-ECMR to 8/18; Strong-ECMR likewise increases correct outcomes from 4/18 to 8/18 relative to Strong-Direct. Reporting counts avoids the misleading zero-variance and repeated Wald-SEM patterns produced by only six records per domain. The weak-versus-strong rows are not compute matched, and their token costs differ substantially, so the table supports transfer of the control mechanism rather than interchangeability of model capacity. Figure 4: Three-domain held-out reliability. Panel (a) traces domain accuracy and uncertainty for every method; panel (b) shows ECMR-Full accuracy gain, FCR reduction, and ESR gain over Direct with independently propagated uncertainty. Figure 4 tests whether the headline result is an averaging artifact. The accuracy trajectories remain ordered across all three domains, and the ECMR-Full gain bundle moves in the favorable direction for accuracy, FCR, and ESR everywhere. FinQA starts from the lowest Direct accuracy yet shows the largest accuracy and ESR gains, while CaseHOLD remains strongest in absolute accuracy. The pattern is therefore consistent with a domain-general control benefit rather than a single benchmark carrying the macro average. B.5 Additional Role-Memory Diagnostics Compliance Variant Role-Violation Rate Invalid-Challenge Rate Self-Contradiction Rate False-Consensus Rate Evidence-Support Rate No-Compliance 0.280± 0.0047 0.350± 0.0050 0.220± 0.0044 0.290± 0.0048 0.737± 0.0046 Compliance-Monitor 0.240± 0.0045 0.310± 0.0049 0.190± 0.0041 0.260± 0.0046 0.767± 0.0045 Feedback-Only 0.180± 0.0040 0.240± 0.0045 0.130± 0.0035 0.230± 0.0044 0.797± 0.0042 Partial-Loop 0.130± 0.0035 0.180± 0.0040 0.090± 0.0030 0.200± 0.0042 0.837± 0.0039 Full-Compliance-Loop 0.070± 0.0027 0.110± 0.0033 0.040± 0.0021 0.170± 0.0040 0.887± 0.0033 Table 18: Role-memory feedback under compliance variants. Registered stepwise changes are anchored so Full-Compliance-Loop matches the ECMR-Full macro FCR/ESR endpoint; the ± terms are binomial standard errors over 9,000 evaluation records per variant. Interpretation. Table 18 tests the role-correction path in Figure 1 of the main paper. Adding monitoring, feedback, and the closed correction loop produces a monotonic reduction in role violations, invalid challenges, and self-contradictions, while FCR falls and ESR rises. Because the reliability endpoints move with role-behavior failures at every stage, the ablation supports role correction as an upstream control mechanism rather than a cosmetic prompt constraint. Figure 5: Ordered compliance ablation over 9,000 records per variant. Panel (a) compares role-violation and false-consensus incidence as controls are added; panel (b) relates the derived role-compliance score to evidence-support rate. The full loop matches the ECMR-Full macro endpoint; whiskers are binomial standard errors. Figure 5 exposes the coupling behind the table: variants with fewer role violations also have fewer false consensuses and stronger evidence support. The compliance score in panel (b) is the complement of role-violation rate, not an independent outcome; its value is to align the direction of behavioral compliance with ESR. The evidence is the coordinated endpoint movement across the ordered ablation, not a correlation between two independently measured scores. B.6 Additional Online-Adaptation Results Method Pre-Shift Acc / FCR Post-Shift Acc / FCR Recovery Acc / FCR Recovery Steps Frozen-Baseline 74.7± 3.5 / 0.333± 0.038 64.7± 3.9 / 0.500± 0.041 66.7± 3.8 / 0.444± 0.041 38± 3 Frozen+EMA 74.8± 3.4 / 0.320± 0.036 69.3± 3.5 / 0.400± 0.039 72.0± 3.3 / 0.333± 0.036 24± 2 Frozen+Memory 75.2± 3.3 / 0.280± 0.034 72.0± 3.2 / 0.300± 0.035 74.7± 3.0 / 0.222± 0.031 15± 2 Full-Online-ECMR 76.0± 3.2 / 0.200± 0.030 74.0± 3.0 / 0.222± 0.032 76.0± 2.8 / 0.185± 0.028 6± 1 Table 19: Online self-learning under a pre-declared distribution shift on the separate 150-record Formal50 diagnostic. Each method is evaluated before the shift, immediately after it, and after recovery; uncertainty retains the recorded across-run stream variation. Interpretation. Table 19 shows that memory and policy updates affect both the quality and speed of recovery after the declared shift. Full-Online-ECMR returns to its pre-shift accuracy in 6 steps, compared with 38 for the frozen baseline, and recovers to an FCR of 0.185 rather than 0.444. EMA and memory-only variants improve progressively, but neither matches the full controller’s 74.0 post-shift accuracy or complete recovery to 76.0. B.7 Additional Efficiency Statistics Method Difficulty Mean Rounds Verification Trigger Rate Arbitration Trigger Rate Early-Stop Trigger Rate Mean Tokens Direct Easy 1.0± 0.0 0.000± 0.000 0.000± 0.000 1.000± 0.000 578± 21 Direct Medium 1.0± 0.0 0.000± 0.000 0.000± 0.000 1.000± 0.000 623± 51 Direct Hard 1.0± 0.0 0.000± 0.000 0.000± 0.000 1.000± 0.000 867± 74 Multi-Debate Easy 3.8± 0.3 0.967± 0.018 0.950± 0.022 0.083± 0.024 2967± 84 Multi-Debate Medium 3.9± 0.2 0.950± 0.022 0.933± 0.027 0.050± 0.018 3083± 208 Multi-Debate Hard 4.0± 0.1 0.933± 0.027 0.917± 0.031 0.017± 0.009 4161± 296 AutoGen Easy 2.7± 0.4 0.950± 0.022 0.000± 0.000 0.150± 0.033 2390± 148 AutoGen Medium 2.9± 0.3 0.933± 0.027 0.000± 0.000 0.083± 0.024 2411± 175 AutoGen Hard 3.0± 0.2 0.900± 0.032 0.000± 0.000 0.033± 0.013 3130± 186 Fixed-ECMR Easy 4.0± 0.0 0.983± 0.012 0.967± 0.018 0.000± 0.000 3619± 124 Fixed-ECMR Medium 4.0± 0.0 0.967± 0.018 0.950± 0.022 0.000± 0.000 3704± 169 Fixed-ECMR Hard 4.0± 0.0 0.950± 0.023 0.933± 0.027 0.000± 0.000 4548± 193 ECMR-Full Easy 5.3± 1.0 0.992± 0.008 0.350± 0.057 0.650± 0.057 7637± 2593 ECMR-Full Medium 5.8± 0.9 0.983± 0.012 0.650± 0.048 0.350± 0.048 7150± 2022 ECMR-Full Hard 5.9± 0.6 0.967± 0.019 0.900± 0.030 0.100± 0.030 8164± 1752 Table 20: Full-evaluation resource allocation by difficulty stratum. Values report mean± deviation across repeated runs. Verification may precede either terminal route; for ECMR-Full, arbitration and early stopping are complementary shares, and mean rounds increase with difficulty within the six-round budget. Interpretation. Table 20 separates three possible explanations for ECMR’s cost. Verification is already near saturation for several multi-agent baselines, so its trigger rate is not the distinctive signal. Mean rounds increase only modestly from Easy to Hard, whereas ECMR-Full’s terminal decision changes sharply from early stopping to arbitration. The additional computation is therefore associated with how unresolved cases terminate, not simply with enabling verification or adding a fixed number of rounds. Figure 6: Difficulty-conditioned controller actions. Panel (a) reports the explicit-verification trigger rate for every policy and difficulty stratum. Panel (b) shows ECMR-Full reallocating complementary terminal routes from early stopping on Easy cases to arbitration on Hard cases; whiskers mark routing uncertainty. Figure 6 makes the mechanism discriminative. Verification coverage is high and changes little, but the complementary terminal routes exchange 0.550 probability mass from early stopping to arbitration between Easy and Hard cases. Fixed-ECMR cannot express this conditional exchange, and the state-shuffled ablation loses 11.7 accuracy points despite retaining the same operation set. These two observations connect the routing visualization to the performance result: the gain depends on mapping evidence states to actions, not on action availability alone. B.8 Additional Cost-Efficiency Statistics Method Avg. Tokens Avg. Latency (s) Δ vs Weak Extra tokens / 1p Tokens / correct Strong-Direct 679± 201 4.72± 1.38 11.1 — 3054 Strong-Self-Consistency 2110± 582 16.02± 5.39 5.6 255.0 12658 Weak-Direct 693± 219 3.34± 0.43 0.0 — 6238 Weak-Self-Consistency 2153± 643 12.17± 2.09 0.0 — 19374 Weak-Fixed-ECMR 4044± 727 15.79± 2.22 11.1 301.5 18196 Weak-ECMR 8816± 7368 27.31± 13.33 33.3 243.7 19836 Strong-ECMR 9751± 7370 41.82± 16.09 33.3 271.7 21941 Table 21: Cost efficiency across strong and weak backbones. Token cost and latency are reported as mean± across the three domain aggregates. Accuracy gain is measured relative to Weak-Direct; additional tokens per percentage-point (p) gain and tokens per correct sample are computed from the recorded diagnostic logs. Interpretation. Table 21 separates absolute cost from marginal efficiency. Weak-ECMR requires fewer extra tokens per percentage-point gain than Weak-Fixed-ECMR (243.7 versus 301.5), indicating that dynamic control uses additional computation more productively than a fixed ECMR schedule. At the same time, tokens per correct sample remain high for both weak and strong ECMR variants, reinforcing that controller efficiency should be evaluated jointly with the cost of the frozen backbone and the value assigned to reliability. B.9 State Feature Ablation Removed Feature Performance Drop Importance Rank Evidence independence ItindI^ind_t 6.1± 0.5 p 1 Support entropy H~tsup H^sup_t 5.8± 0.4 p 2 Contradiction structure CtctrC^ctr_t 5.2± 0.4 p 3 Role drift/overlap 4.8± 0.4 p 4 Answer concentration gtg_t 4.3± 0.4 p 5 Remaining budget btb_t 3.1± 0.3 p 6 Table 22: State-feature ablation importance over the declared 9,000-record evaluation. Semantic-state features are ranked by recorded accuracy drop in percentage points (p) and across-run uncertainty when each feature is removed. Interpretation. Table 22 ranks the information used by the controller. Removing evidence independence causes the largest performance drop, followed closely by support entropy and contradiction structure. These three leading features all measure the quality and diversity of support rather than the popularity of the current answer. Answer concentration ranks below them, empirically reinforcing the paper’s central claim that agreement alone is an insufficient stopping criterion. B.10 Risk-Flagged Compliance Gains Method Risk-Flagged FCR Error Propagation Rate No-Compliance 0.389± 0.115 0.333± 0.111 Compliance-Monitor 0.333± 0.111 0.278± 0.106 Feedback-Only 0.278± 0.106 0.222± 0.098 Partial-Loop 0.222± 0.098 0.167± 0.088 Full-Compliance-Loop 0.167± 0.088 0.111± 0.074 Table 23: Compliance gains on the separate 18-record risk-flagged diagnostic. The ± terms are binomial standard errors for false-consensus and error-propagation incidence and do not inherit the main evaluation sample count. Interpretation. Table 23 shows a stepwise reduction in risk as compliance feedback is added. The full loop reduces risk-flagged FCR from 0.389 to 0.167 and error propagation from 0.333 to 0.111 relative to No-Compliance. The parallel decrease indicates that role correction acts upstream: it limits the propagation of a weak challenge or copied rationale before that error becomes a confident group decision. B.11 System Stability Method Performance Variance Collaboration Budget Fluctuation Compliance Rate Frozen-Baseline 12.7 28.3% 0.71± 0.04 Frozen+EMA 7.5 18.4% 0.80± 0.03 Frozen+Memory 5.3 13.2% 0.86± 0.03 Full-Online-ECMR 2.4 8.6% 0.92± 0.02 Table 24: System stability on the separate 150-record Formal50 streaming diagnostic. Performance variance and collaboration-budget fluctuation are point estimates; compliance-rate uncertainty retains recorded across-run variation. Interpretation. Table 24 connects online adaptation to operational stability. Full-Online-ECMR reduces performance variance from 12.7 to 2.4 and collaboration-budget fluctuation from 28.3% to 8.6%, while raising compliance from 0.71 to 0.92. The joint movement shows that adaptation is not simply increasing average performance; it also makes the controller’s resource use and role behavior more predictable under the streaming protocol. References Chen et al. [2023a] Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F. Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation, 2023a. arXiv:2309.17288. Chen et al. [2024] Justin Chen, Swarnadeep Saha, and Mohit Bansal. Reconcile: Round-table conference improves reasoning via consensus among diverse llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024. doi: 10.18653/v1/2024.acl-long.381. Chen et al. [2023b] Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. Advances in Neural Information Processing Systems, 2023b. doi: 10.48550/arxiv.2305.05176. Chen et al. [2023c] Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Transactions on Machine Learning Research, 2023c. doi: 10.48550/arxiv.2211.12588. URL https://openreview.net/forum?id=YfZ4ZPt8zd. Chen et al. [2021] Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021. doi: 10.18653/v1/2021.emnlp-main.300. Du et al. [2023] Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023. Farquhar et al. [2024] Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, and Yarin Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 2024. doi: 10.1038/s41586-024-07421-0. Jiang et al. [2023] Dongfu Jiang, Xiang Ren, and Bill Lin. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023. doi: 10.18653/v1/2023.acl-long.792. Jin et al. [2021] Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021. doi: 10.3390/app11146421. Li et al. [2024] Yunxuan Li, Yibing Du, Jiageng Zhang, Longxiang Hou, Peter Grabowski, Yeqing Li, and Eugene Ie. Improving multi-agent debate with sparse communication topology. In Findings of the Association for Computational Linguistics: EMNLP 2024, 2024. doi: 10.18653/v1/2024.findings-emnlp.427. Liang et al. [2024] Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024. doi: 10.18653/v1/2024.emnlp-main.992. Liu et al. [2023] Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023. doi: 10.18653/v1/2023.emnlp-main.153. Madaan et al. [2023] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 2023. doi: 10.48550/arxiv.2303.17651. Qian et al. [2024] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yu-Sheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024. doi: 10.18653/v1/2024.acl-long.810. Schick et al. [2023] Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 2023. doi: 10.48550/arxiv.2302.04761. Shinn et al. [2023] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 2023. doi: 10.48550/arxiv.2303.11366. Tang et al. [2024] Xiangru Tang, Anni Zou, Zhuosheng Zhang, Ziming Li, Yilun Zhao, Xingyao Zhang, Arman Cohan, and Mark Gerstein. Medagents: Large language models as collaborators for zero-shot medical reasoning. In Findings of the Association for Computational Linguistics: ACL 2024, 2024. doi: 10.18653/v1/2024.findings-acl.33. Wang et al. [2023] Boshi Wang, Xiang Yue, and Huan Sun. Can chatgpt defend its belief in truth? evaluating llm reasoning via debate. In Findings of the Association for Computational Linguistics: EMNLP 2023, 2023. doi: 10.18653/v1/2023.findings-emnlp.795. Wang et al. [2024] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. A survey on large language model based autonomous agents. Frontiers of Computer Science, 2024. doi: 10.1007/s11704-024-40231-1. Wang et al. [2022] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H., Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. International Conference on Learning Representations, 2022. doi: 10.48550/arxiv.2203.11171. Wei et al. [2022] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 2022. doi: 10.48550/arxiv.2201.11903. Wu et al. [2023] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen llm applications via multi-agent conversation, 2023. Yao et al. [2023a] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 2023a. doi: 10.48550/arxiv.2305.10601. Yao et al. [2023b] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023b. doi: 10.48550/arxiv.2210.03629. Zheng et al. [2021] Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho. When does pretraining help? assessing self-supervised learning for law and the casehold dataset, 2021.