Paper deep dive

Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection

Liangyu Teng, Hengsong Liu, Juncen Guo, Jingyu Zhang, Yang Liu, Jing Liu, Liang Song

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 156

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%

Last extracted: 10/1/2026, 7:04:05 AM

Summary

The paper introduces C2-MAS, a framework for selecting heterogeneous teams of Large Language Models (LLMs) to maximize performance while minimizing redundancy. It proposes two offline profiling metrics: error decorrelation (to reduce co-failures) and distributional divergence (to capture strategy diversity). These metrics are combined with individual model quality into a standardized objective function, optimized via multi-start greedy search. Experiments across 13 benchmarks show that C2-MAS consistently outperforms quality-only baselines and random selection, demonstrating the value of heterogeneity in multi-LLM systems.

Entities (10)

C2-MAS · methodframework · 98%Llama-3.1-8B-Instruct · model · 95%Distributional Divergence · metric · 92%Error Decorrelation · metric · 92%MMLU · benchmark · 90%Multi-start Greedy Search · algorithm · 90%Jensen-Shannon Divergence · statistical-measure · 85%Quality-Only · baseline · 85%Yule's Q · statistical-measure · 85%Stacking · aggregation-method · 80%

Relation Signals (7)

C2-MAS → uses → Distributional Divergence

confidence 95% · C2-MAS ... profile each candidate to estimate its individual capability together with two complementary heterogeneity signals: ... (2) distributional divergence

C2-MAS → uses → Error Decorrelation

confidence 95% · C2-MAS ... profile each candidate to estimate its individual capability together with two complementary heterogeneity signals: (1) error decorrelation

C2-MAS → optimizes → Multi-start Greedy Search

confidence 90% · We then select a size-k team by optimizing a composite objective ... using multi-start greedy search

C2-MAS → outperforms → Quality-Only

confidence 90% · C2-MAS reaches 75.52% accuracy under Stacking, improving over the strongest selection baseline (Quality-Only, 74.72%)

Error Decorrelation → measuredby → Yule's Q

confidence 85% · we then compute Yule’s Q association statistic and define the pairwise error-correlation matrix

Distributional Divergence → measuredby → Jensen-Shannon Divergence

confidence 85% · measure pairwise disagreement directly in the choice distribution space using the Jensen–Shannon divergence (JSD)

C2-MAS → evaluatedon → MMLU

confidence 80% · We evaluate C2-MAS on 13 public multiple-choice benchmarks ... MMLU

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictive differences. Although heterogeneous teaming is often observed to be effective in practice, existing approaches lack complementarity metrics that are computable, interpretable, and optimizable, leaving team composition to rely on heuristics. We propose a heterogeneity-driven team selection framework that performs offline profiling to characterize individual capability along with two complementary signals: one captures decorrelation in error patterns to reduce co-failures, while the other measures divergence in predictive behavior to capture strategy diversity. We formulate team selection as a standardized quality--complementarity combinatorial objective and apply an efficient greedy search to select a small team from a candidate pool. Experiments across multiple benchmarks demonstrate that our framework consistently outperforms quality-only baselines under controlled candidate pools and team sizes, establishing reusable selection principles for multi-LLM systems.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

153,359 characters extracted from source content.

Expand or collapse full text

Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection Liangyu Teng1, Hengsong Liu1, Juncen Guo1, Jingyu Zhang1, Yang Liu2, Jing Liu3, Liang Song1 Abstract The performance ceiling of an LLM team is constrained not only by individual model capabilities, but also by inter-member error resonance and predictive differences. Although heterogeneous teaming is often observed to be effective in practice, existing approaches lack complementarity metrics that are computable, interpretable, and optimizable, leaving team composition to rely on heuristics. We propose a heterogeneity-driven team selection framework that performs offline profiling to characterize individual capability along with two complementary signals: one captures decorrelation in error patterns to reduce co-failures, while the other measures divergence in predictive behavior to capture strategy diversity. We formulate team selection as a standardized quality–complementarity combinatorial objective and apply an efficient greedy search to select a small team from a candidate pool. Experiments across multiple benchmarks demonstrate that our framework consistently outperforms quality-only baselines under controlled candidate pools and team sizes, establishing reusable selection principles for multi-LLM systems. Fudan Institute on Networking Systems of AI 1 Introduction Large language models (LLMs) are increasingly deployed as teams: multiple models or agents collaborate through voting, debate, or orchestration to improve reasoning accuracy and robustness (Du et al., 2024; Liang et al., 2024; Wang et al., 2025a). In practical deployments, however, collaboration gains are tightly constrained by inference budget: each additional member introduces extra model calls and tokens, increasing end-to-end latency and cost. Real systems must therefore keep teams small, drawing from a limited pool of deployable candidates. These constraints make member selection a primary design decision: under a fixed budget, which models should we call so that the team is both individually strong and behaviorally complementary? A natural starting point is to pick the top-k models by individual accuracy, but as illustrated in Figure 1(a), this strategy ignores a second axis that bounds team performance: similarity across members. Models can exhibit error resonance, failing on the same instances and concentrating probability on the same incorrect options; even when they agree on a label, they may share highly similar predictive behavior (confidence and preferences over alternatives), offering little strategy diversity. Top-k-by-accuracy selection therefore tends to yield a strong yet redundant team whose blind spots largely overlap. These two sources of redundancy (co-failure and predictive similarity) motivate selection criteria that go beyond quality and capture inter-model complementarity directly. Figure 1: Illustration of heterogeneity-aware team selection. While the value of heterogeneous teaming is widely recognized, practical team composition still relies largely on heuristics such as mixing model families. Most recent work has instead focused on downstream combination mechanisms such as output fusion, reranking, routing, and cascading, all of which take a callable model set and budget as given (Jiang et al., 2023; Chen et al., 2024b; Shnitzer et al., 2024; Ong et al., 2025). The upstream question of which models to select in the first place remains comparatively underexplored. We address this gap by formulating team selection as an offline, budgeted optimization problem: composing a compact and complementary team from development data, independently of the inference-time aggregation rule. This requires complementarity signals that are computable, interpretable, and tractable to optimize, together with a selection procedure that scales to combinatorial search over candidate teams. Driven by these requirements, we propose C2-MAS (Covariance-minimized and Capability-maximized Multi-Agent System), an offline profiling-and-selection pipeline that directly targets the two sources of redundancy identified above. Given development data, we profile each candidate to estimate its individual capability together with two complementary heterogeneity signals: (1) error decorrelation, which addresses co-failure by capturing whether models fail on different instances; and (2) distributional divergence, which addresses predictive similarity by measuring disagreement in choice distributions over alternatives. When multiple tasks are available, we average pairwise estimates across tasks to stabilize the complementarity signal. We then select a size-k team by optimizing a standardized quality–complementarity objective with an efficient multi-start greedy search. Experiments on 13 multiple-choice benchmarks with 11 open-weight models show that heterogeneity signals yield consistent gains across aggregation rules (Figure 1(b)). On the 7 primary benchmarks, C2-MAS reaches 75.52% accuracy under Stacking, improving over the strongest selection baseline (Quality-Only, 74.72%) by +0.80 p without regressing on any task, and over the random-k baseline by +3.58 p. On 6 additional held-out benchmarks, the gains persist under distribution shift, suggesting that the proposed signals capture transferable complementarity rather than task-specific artifacts. Our key contributions are summarized as follows: ❶ Problem Formulation: We frame team selection as an upstream, budget-constrained optimization problem, decoupled from downstream aggregation mechanisms. ❷ Practical Solution: We propose two interpretable heterogeneity metrics and a standardized quality–complementarity objective with efficient greedy search. Our analysis connects the two signals to collective errors and identifies conditions for accuracy gains over quality-only selection. ❸ Empirical Validation: We demonstrate consistent, non-regressive improvements across multiple benchmarks and aggregation rules, with gains that persist on held-out tasks under distribution shift. 2 Related Work 2.1 Multi-Agent and Multi-Model Collaboration LLM-based multi-agent systems organize collaboration through predefined roles and workflows (Li et al., 2023; Hong et al., 2024; Wu et al., 2024), and refine interaction through debate and voting (Du et al., 2024; Choi et al., 2025), communication structure optimization, and dynamic team formation (Zhuge et al., 2024; Zhang et al., 2025b; Liu et al., 2024). Beyond agent interaction, multi-model methods perform fusion at the output, reasoning, or decoding stage (Jiang et al., 2023; Wang et al., 2025a; Huang et al., 2024), while routing and cascading select models per input to balance quality and cost (Chen et al., 2024b; Shnitzer et al., 2024; Ong et al., 2025). Distinct from these approaches, we focus on task-level offline team selection, constructing fixed-size teams whose membership is determined independently of the subsequent aggregation rule. 2.2 Ensemble Diversity and Team Selection Classical ensemble theory emphasizes individual accuracy and low error correlation, motivating diversity measures and validation-driven ensemble selection (Hansen and Salamon, 1990; Krogh and Vedelsby, 1994; Kuncheva and Whitaker, 2003; Windeatt, 2005; Caruana et al., 2004). Recent LLM ensemble selection methods exploit error or semantic diversity (Tekin et al., 2024; Cohen et al., 2026), or adopt information-theoretic and label-based surrogate objectives (Turkmen et al., 2026; Zhang et al., 2026). Building on this line of work, we jointly quantify error correlation and distributional divergence through a single offline profiling pass, combine these signals with model quality for team selection, and evaluate their utility and generality across multiple aggregation rules. 3 Methodology C2-MAS selects compact LLM teams through offline profiling and team search, followed by inference-time aggregation (Figure 2). For each task, development data provide quality and heterogeneity estimates (Section 3.2) for the standardized selection objective (Section 3.3); the selected team’s predictions are then combined by an aggregator (Section 3.4). 3.1 Preliminaries We consider a collection of tasks T. Each task t∈t has a label set tY_t (e.g., A,B,C,D\A,B,C,D\) and two disjoint splits: a development set DtdevD_t^dev and a held-out test set DttestD_t^test. We assume a fixed pool of m candidate models ℳ=1,…,mM=\1,…,m\ and select a team St⊆ℳS_t of size k for each task t. Throughout, DtdevD_t^dev is used only for candidate profiling, complementarity estimation, and training an aggregator (if needed), while DttestD_t^test is used only for final evaluation. To obtain comparable choice distributions, we use a short prompt to elicit a single option from the model: “Complete the sentence ‘The correct answer is ’ by outputting exactly one letter from: <options>’.11 1 See Section D.1 for full details. Using the fixed probe prefix “The correct answer is ”, for each instance x we query the model’s log-probability ℓi(y∣x) _i(y x) for each label y∈ty _t (i.e., the next-token log-probability under this prefix) and normalize within tY_t: pit(y∣x)=exp⁡(ℓi(y∣x))∑y′∈texp⁡(ℓi(y′∣x)).p_i^t(y x)= ( _i(y x)) _y _t ( _i(y x)). (1) By construction, pit(⋅∣x)p_i^t(· x) is a proper distribution over tY_t. Based on this distribution, model i predicts y^i(x)=argmaxy∈t⁡pit(y∣x) y_i(x)= _y _tp_i^t(y x), and we define the correctness indicator Ci(x)=[y^i(x)=y(x)]C_i(x)=I[ y_i(x)=y(x)], which equals 11 if the prediction is correct and 00 otherwise. These definitions provide a unified interface for the subsequent quality estimation, complementarity signals, and team selection. Figure 2: Overview of C2-MAS framework. Given a candidate pool of m models, we (1) profile each model on development data to obtain quality estimates and choice distributions; (2) compute two heterogeneity index metrics (HIerrHI_err and HIdistHI_dist) and stabilize them via cross-task pooling; (3) select a team of size k by optimizing a composite objective that balances quality and complementarity using multi-start greedy search; and (4) aggregate the selected team’s predictions to produce final outputs, evaluated on held-out test data. 3.2 Profiling Signals From DtdevD_t^dev, we estimate per-model quality qit\q_i^t\ and two heterogeneity index (HI) metrics: error decorrelation HIerrHI_err from an association matrix RtR^t, and distributional divergence HIdistHI_dist from a choice-divergence matrix JtJ^t. These signals serve as inputs to the subsequent team selection procedure. Quality. We measure the individual capability of model i on task t by its development-set accuracy, denoted as qitq_i^t, computed as: qit=1|Dtdev|∑x∈DtdevCi(x).q_i^t= 1|D_t^dev| _x∈ D_t^devC_i(x). (2) Error Decorrelation (HIerrHI_err). To capture complementarity at the error level, we summarize whether two models tend to succeed and fail on the same instances. For a pair (i,j)(i,j), we form a 2×22× 2 contingency table on DtdevD_t^dev: Nabt(i,j)=∑x∈Dtdev⁡[Ci(x)=a,Cj(x)=b],N_ab^t(i,j)= _x∈ D_t^devI\! [C_i(x)=a,\;C_j(x)=b ], (3) where a,b∈0,1a,b∈\0,1\. Let Dijt=N11tN00t+N10tN01tD_ij^t=N_11^tN_00^t+N_10^tN_01^t, we then compute Yule’s Q association statistic and define the pairwise error-correlation matrix Rt∈ℝm×mR^t ^m× m as: Rijt=N11tN00t−N10tN01tDijt,Dijt>0,0,Dijt=0.R_ij^t= cases N_11^tN_00^t-N_10^tN_01^tD_ij^t,&D_ij^t>0,\\ 0,&D_ij^t=0. cases (4) where we write Nabt≡Nabt(i,j)N_ab^t≡ N_ab^t(i,j) for readability. Given a team S, we aggregate pairwise correlations into an error-decorrelation score: HIerr(S,Rt)=1−1(|S|2)∑i,j⊆SRijt.HI_err(S;R^t)=1- 1 |S|2 _\i,j\ SR_ij^t. (5) Higher HIerrHI_err indicates that team members make mistakes on different instances. Distributional Divergence (HIdistHI_dist). Correctness correlations omit how models distribute probability over answer choices. We therefore measure pairwise disagreement directly in the choice distribution space using the Jensen–Shannon divergence (JSD) (Lin, 1991; Endres and Schindelin, 2003), computed with base-2 logarithms. For a pair (i,j)(i,j), we average per-instance JSD on DtdevD_t^dev to obtain a matrix Jt∈ℝm×mJ^t ^m× m: Jijt=1|Dtdev|∑x∈DtdevJSD(pit(⋅∣x),pjt(⋅∣x)).J_ij^t= 1|D_t^dev| _x∈ D_t^devJSD\! (p_i^t(· x),\,p_j^t(· x) ). (6) Given a team S, we aggregate pairwise divergences into a distributional-divergence score: HIdist(S,Jt)=1(|S|2)∑i,j⊆SJijt.HI_dist(S;J^t)= 1 |S|2 _\i,j\ SJ_ij^t. (7) A larger HIdistHI_dist indicates greater disagreement in the models’ choice distributions. Cross-Task Pooling for Stable HI. Pairwise heterogeneity estimates can be noisy when |Dtdev||D_t^dev| is limited, which may make team selection overly sensitive to idiosyncrasies of a single task’s development split. When a collection of tasks T is available, we pool pairwise heterogeneity across tasks to obtain a shared selection signal. For each pairwise matrix, we compute the pooled estimates by averaging over tasks: R¯=1||∑u∈Ru,J¯=1||∑u∈Ju. R= 1|T| _u R^u, J= 1|T| _u J^u. (8) In our experiments, we use R¯ R and J¯ J as the heterogeneity signals for team selection. 3.3 Heterogeneity-Aware Team Selection Given the development-set profiling results, our goal is to select a team St⊆ℳS_t of size k that balances individual capability and inter-model complementarity. We combine task-specific quality estimates qit\q_i^t\ with the cross-task pooled heterogeneity matrices R¯ R and J¯ J, which summarize correctness dependence and differences in choice distributions, respectively. We formulate team selection as maximizing a composite score comprising average member quality, error decorrelation HIerr(S,R¯)HI_err(S; R), and distributional divergence HIdist(S,J¯)HI_dist(S; J). Since these terms may have different scales across tasks and candidate pools, we standardize each term with a standard-error-scaled z-score before combining them. Let u⁡(S)u(S) denote a team-level quantity computed as an average (e.g., the average quality, HIerr(S,R¯)HI_err(S; R), or HIdist(S,J¯)HI_dist(S; J)), and let nu(S)n_u(S) be the number of items being averaged (e.g., |S||S| for member-level quantities and (|S|2) |S|2 for pairwise quantities). We define: z⁡(u⁡(S))=u⁡(S)−μuσu/nu(S),z(u(S))= u(S)- _u _u/ n_u(S), (9) where (μu,σu)( _u, _u) are computed over the corresponding base items in the candidate pool: the set qit\q_i^t\ for quality, the set 1−R¯ij\1- R_ij\ for error decorrelation, and the set J¯ij\ J_ij\ for distributional divergence. The C2-MAS selection score for task t can then be formulated as: scoret(S)=z⁡(1|S|∑i∈Sqit)+λ1z(HIerr(S,R¯))+λ2z(HIdist(S,J¯)),score_t(S)=z\! ( 1|S| _i∈ Sq_i^t )+ _1\,z\! (HI_err(S; R) )+ _2\,z\! (HI_dist(S; J) ), (10) wrapfloat algorithmr0.55 Multi-start forward greedy selection for task t 1: Input: Candidates ℳM, team size k, qualities qit\q_i^t\, pooled matrices R¯,J¯ R, J, weights (λ1,λ2)( _1, _2) 2: Output: Selected team St∗S_t^* 3: St∗←∅S_t^*← , best←−∞best←-∞ 4: for each seed s∈ℳs do 5: S←sS←\s\ 6: while |S|<k|S|<k do 7: j∗←argmaxj∈ℳ∖S⁡scoret(S∪j)j^*← _j S\;score_t(S∪\j\) 8: S←S∪j∗S← S∪\j^*\ 9: end while 10: if scoret(S)>bestscore_t(S)>best then 11: best←scoret(S)best _t(S), St∗←S_t^*← S 12: end if 13: end for 14: return St∗S_t^* where λ1,λ2≥0 _1, _2≥ 0 are trade-off coefficients for HIerrHI_err and HIdistHI_dist. The selection objective is max|S|=k⁡scoret(S) _|S|=kscore_t(S). To efficiently solve this combinatorial optimization problem, we adopt a multi-start forward greedy procedure: starting from a seed model, we iteratively add the candidate that yields the largest score improvement until reaching size k, and we repeat this process for each seed in ℳM, returning the best team found. The full procedure is summarized in Algorithm 3.3. 3.4 Aggregation Our method is agnostic to the downstream aggregator, as team selection relies solely on capability and complementarity statistics obtained through offline profiling. To examine whether the gains from team selection persist across aggregation mechanisms, we evaluate the same selected teams using Choice-Soft, PoE, DS, and Stacking. Choice-Soft and PoE use fixed rules based on the mean and normalized product of member probabilities, respectively, whereas DS and Stacking learn aggregation parameters from development data to account for members’ predictive characteristics and reliability differences. Full definitions and training details are provided in Section D.5. 3.5 Theoretical Insights We analyze how correctness dependence and choice-distribution divergence jointly affect team accuracy, and when their benefits compensate for a reduction in individual model quality. A choice-probability model. Fix a task distribution and three-member teams on four choices. With the true answer listed first, a correct member outputs pc=(a,u,u,u)p c=(a,u,u,u); an incorrect member outputs pw=(a,b,c,c)p^w=(a,b,c,c) with the last three coordinates permuted to place b on a wrong answer w. Assume 0<c<a<b,a+b+2c=1,u=(b+2c)/3,a>(2b+u)/3.0<c<a<b, a+b+2c=1, u=(b+2c)/3, a>(2b+u)/3. (11) Members’ correctness and preferred wrong answers may be arbitrarily dependent. For a team S, write q¯S=13∑i∈Sℙ⁡(Ci=1) q_S= 13 _i∈ SP(C_i=1), eS=1−q¯Se_S=1- q_S, and let DSD_S be the mean pairwise probability of both members being incorrect. Let KSK_S be the mean probability of both being incorrect and preferring the same wrong answer, and let JSJ_S be their mean JSD on this task. Define d=JSD⁡(pw,pw′)d=JSD(p^w,p^w ) for w≠w′w≠ w , h=JSD⁡(pc,pw)h=JSD(p c,p^w), and θ=2h/dθ=2h/d. Proposition 3.1 (Shared errors and wrong-answer concentration). Under (11), 0<θ<10<θ<1 and KS=θeS+(1−θ)DS−JSd.K_S=θ e_S+(1-θ)D_S- J_Sd. (12) For either Choice-Soft or PoE, ℙ⁡(y^S≠Y)≤KSP( y_S≠ Y)≤ K_S. Both aggregators fail exactly when all three members prefer the same wrong answer. Remark 1 (Complementarity of the two signals). At fixed marginal accuracies, Yule’s Q increases strictly with pairwise double-fault probability (Lemma C.1). Correctness dependence therefore characterizes shared errors, while choice-space JSD supplies information about their concentration on particular answers. Equation (12) connects both signals to a source of collective failure beyond individual quality. Accuracy gains over quality-only selection. Let SQS_Q be the size-three Quality-Only team, and abbreviate its quantities by a subscript Q. Define the baseline correction εQ=min⁡KQ,eQ−(DQ+KQ)/2≥0 _Q= \K_Q,\,e_Q-(D_Q+K_Q)/2\≥ 0. Proposition 3.2 (A sufficient condition for improvement). Under (11), the team S returned by Algorithm 3.3, evaluated with the same Choice-Soft or PoE aggregator as SQS_Q, satisfies Acc⁡(S)−Acc⁡(SQ)≥(1−θ)(DQ−DS)+JS−JQd−θ⁡(q¯Q−q¯S)−εQ.Acc(S)-Acc(S_Q)≥(1-θ)(D_Q-D_S)+ J_S-J_Qd-θ( q_Q- q_S)- _Q. (13) A positive right-hand side guarantees strictly higher team accuracy. Remark 2 (Quality–heterogeneity trade-off). Equation (13) shows that a team with lower mean member accuracy can still outperform Quality-Only. A sufficient condition is that the weighted gains from reduced shared errors and increased choice-distribution divergence exceed the quality penalty and the baseline correction εQ _Q. For a baseline whose members make identical predictions, εQ=0 _Q=0, so these gains need only outweigh the quality penalty. Sections C.2 and C.3 give complete proofs; Section C.7 connects the condition to the standardized score, pooled profiles, and the terminal teams generated by multi-start search. 4 Experiments We evaluate C2-MAS on various benchmarks to address the following research questions: RQ1 (Effectiveness and Generalization). Does heterogeneity-aware team selection yield consistent gains over quality-only baselines, both on the primary task set and on out-of-distribution (OOD) benchmarks? RQ2 (Mechanism). How do error decorrelation (HIerrHI_err) and distributional divergence (HIdistHI_dist) individually contribute to team performance, and what underlies these gains? RQ3 (Robustness). Is C2-MAS robust to the choice of heterogeneity weights (λ1,λ2)( _1, _2) and to limited profiling data? 4.1 Setup Tasks and Benchmarks. We evaluate C2-MAS on 7 primary datasets and test its OOD generalization on 6 held-out datasets. The 13 public multiple-choice benchmarks span four domains: (1) math reasoning, AQUA-RAT (Ling et al., 2017) and MathQA (Amini et al., 2019); (2) natural science, ARC-Challenge (Clark et al., 2018) and OpenBookQA (Mihaylov et al., 2018); (3) reading comprehension and logical reasoning, RACE (Lai et al., 2017), ReClor (Yu et al., 2020), and LogiQA2 (Liu et al., 2023); and (4) general or domain knowledge, MMLU (Hendrycks et al., 2021), MMLU-Pro (Wang et al., 2024), C-EVAL (Huang et al., 2023), CommonsenseQA (Talmor et al., 2019), MedQA (Jin et al., 2021), and StrategyQA (Geva et al., 2021). For efficiency, we randomly subsample both the dev and test splits to 1,068 examples each. Models. We use m=11m=11 open-weight chat models in the 7–9B parameter class for the candidate pool. The pool includes Llama-3.1-8B-Instruct (Grattafiori et al., 2024), Qwen3-8B (Yang et al., 2025), Gemma-2-9B-IT (Team et al., 2024), Ministral-3-8B-Instruct (Liu et al., 2026), Granite-3.3-8B-Instruct (IBM Granite Team, 2024), GLM-4-9B-Chat (GLM et al., 2024), Nemotron-Nano-9B-v2 (Basant et al., 2025), EuroLLM-9B-Instruct (Martins et al., 2025), OLMo-3-7B-Instruct (Olmo et al., 2025), Apertus-8B-Instruct (Apertus et al., 2025), and InternLM3-8B-Instruct (Cai et al., 2024). Baselines. We compare: (1) Self-Consistency (Wang et al., 2023): the best single model on the dev set sampled k=3k=3 times with stochastic decoding and aggregated by majority vote; (2) Random-k: uniformly sample k=3k=3 models without replacement and report mean accuracy over 100 samples; (3) Caruana (Caruana et al., 2004): forward ensemble selection on DtdevD_t^dev with replacement (allow repeats to express non-uniform member frequencies); (4) Quality-Only: select k=3k=3 by maximizing mean development accuracy (i.e., λ1=λ2=0 _1= _2=0); and (5) C2-MAS: our heterogeneity-aware selection with k=3k=3. Implementation details. We select teams by maximizing the standardized objective in Equation 10 with weights (λ1,λ2)=(0.13,0.05)( _1, _2)=(0.13,0.05). For aggregation, DS uses additive smoothing with ε=10−3 =10^-3 when estimating the class prior and confusion matrices; Stacking trains a linear combiner on DtdevD_t^dev and evaluates on DttestD_t^test (L2L_2 coefficient β=0.003β=0.003); the other aggregators are parameter-free. For statistical testing, we use paired bootstrap with 2,000 resamples to compute confidence intervals (CI) for accuracy differences on the test set (paired by instance). Table 1: Test accuracy (%) on the 7 primary benchmarks; teams of size k=3k=3 are selected from m=11m=11 candidates. ↑ : gain over Random-k (p); bold: best within each aggregation block. Held-out and full results: Tables 5 and E.1. ARC-C: ARC-Challenge; CSQA: CommonsenseQA; OBQA: OpenBookQA. Method Dataset Avg. ARC-C CSQA LogiQA2 MedQA MMLU MMLU-Pro OBQA Closed-Source Models (for reference) Gemini-2.5-Flash 92.7092.70 81.6581.65 60.9660.96 77.4377.43 81.3781.37 58.6158.61 92.7092.70 77.9277.92 GPT-4o 94.1094.10 83.3383.33 64.9864.98 84.7484.74 84.6484.64 50.1950.19 91.8591.85 79.1279.12 Single-Model Aggregation Self-Consistency 89.1489.14 78.6578.65 67.3267.32 61.7061.70 73.4173.41 44.7644.76 88.5888.58 71.9471.94 Team Selection Baselines Choice-Soft Random-k 87.6187.61 78.8778.87 62.4962.49 60.5060.50 72.0872.08 43.1543.15 85.0885.08 69.9769.97 Caruana 89.9889.98 81.6581.65 70.5170.51 63.1163.11 73.6973.69 48.2248.22 90.5490.54 73.9673.96 Quality-Only 91.4891.48 81.4681.46 70.8870.88 66.3966.39 76.5976.59 47.0047.00 89.7089.70 74.7974.79 C2-MAS (Ours) 91.4891.48↑ 3.87 83.9083.90↑ 5.02 70.8870.88↑ 8.39 66.3966.39↑ 5.89 76.5976.59↑ 4.51 48.5048.50↑ 5.35 90.4590.45↑ 5.37 75.4575.45↑ 5.48 PoE Random-k 87.5887.58 79.2079.20 63.0463.04 60.8160.81 72.3072.30 43.4143.41 84.7984.79 70.1670.16 Caruana 90.5490.54 83.0583.05 71.4471.44 65.0765.07 74.0674.06 49.3449.34 90.8290.82 74.9174.91 Quality-Only 91.5791.57 82.6882.68 70.6970.69 65.3665.36 76.6976.69 48.6948.69 89.3389.33 75.0075.00 C2-MAS (Ours) 91.5791.57↑ 3.99 83.9083.90↑ 4.70 70.6970.69↑ 7.65 65.3665.36↑ 4.55 76.6976.69↑ 4.39 48.9748.97↑ 5.56 89.5189.51↑ 4.72 75.2475.24↑ 5.08 DS Random-k 87.5687.56 78.0078.00 62.9062.90 60.8760.87 71.5271.52 42.4642.46 84.9684.96 69.7569.75 Caruana 86.2486.24 79.4979.49 68.8268.82 63.9563.95 72.9472.94 47.4747.47 89.6189.61 72.6572.65 Quality-Only 90.9290.92 81.1881.18 70.3270.32 65.1765.17 75.4775.47 47.2847.28 89.5189.51 74.2674.26 C2-MAS (Ours) 90.9290.92↑ 3.36 82.8782.87↑ 4.87 70.3270.32↑ 7.42 65.1765.17↑ 4.30 75.4775.47↑ 3.95 47.1047.10↑ 4.64 90.4590.45↑ 5.49 74.6174.61↑ 4.86 Stacking Random-k 89.3089.30 80.2080.20 65.5465.54 62.8262.82 73.5573.55 44.8244.82 87.3587.35 71.9471.94 Caruana 90.5490.54 82.7782.77 70.6070.60 64.4264.42 73.8873.88 48.7848.78 91.5791.57 74.6574.65 Quality-Only 91.6791.67 81.7481.74 70.6070.60 65.5465.54 76.1276.12 47.1947.19 90.1790.17 74.7274.72 C2-MAS (Ours) 91.6791.67↑ 2.37 84.1884.18↑ 3.98 70.6070.60↑ 5.06 65.5465.54↑ 2.72 76.1276.12↑ 2.57 48.9748.97↑ 4.15 91.5791.57↑ 4.22 75.5275.52↑ 3.58 4.2 Effectiveness of C2-MAS (RQ1) We report primary results on 7 datasets in Table 1 and OOD generalization on the remaining 6 in Table 5. Primary Results. Under strict development-test separation, C2-MAS delivers small but consistent improvements over the Quality-Only baseline. As shown in Table 1, C2-MAS reaches 75.52% with the Stacking aggregator, outperforming Quality-Only (74.72%) by +0.80+0.80 p (95% CI: [0.44, 1.16]), while also surpassing both Self-Consistency and Random-k by +3.58+3.58 p. OOD Generalization. To assess robustness under distribution shifts, we evaluate C2-MAS on 6 held-out benchmarks. C2-MAS achieves 66.74% average accuracy, surpassing Quality-Only (66.43%, +0.31 p) and significantly outperforming Self-Consistency (64.45%, +2.29+2.29 p) and Random-k (61.12%, +5.62+5.62 p), confirming that heterogeneity signals capture intrinsic complementarity that transfers beyond the primary task set. Non-regression and Consistency. Beyond average gains, safety is critical for practical deployment. Under Stacking, the wins come from the three benchmarks where C2-MAS selects a different team from Quality-Only (CommonsenseQA, MMLU-Pro, and OpenBookQA), with a mean gain of +1.87+1.87 p. On the remaining four benchmarks the teams are identical and predictions match Quality-Only by construction (Section E.2). In each differing case, C2-MAS replaces a redundant high-accuracy member with a more complementary one rather than trading quality for diversity. C2-MAS thus acts as a safe plugin: it preserves the strongest baseline’s performance and adds gains where complementarity helps. The pattern holds across all four aggregators, with the largest margin under Stacking where the learned combiner benefits most from heterogeneous inputs. 4.3 Mechanism Analysis: Why Heterogeneity Matters? (RQ2) While the main results confirm the effectiveness of our approach, we further investigate the source of these gains through quantitative ablation studies (Table 2) and qualitative visualization (Figure 4). (a) Sensitivity of heterogeneity weights. (b) Accuracy vs. profiling size. (c) Selection stability (Jaccard vs. full-dev team). Figure 3: Robustness and data efficiency of C2-MAS. (a) Test accuracy under Stacking across heterogeneity weights; (0,0)(0,0) denotes Quality-Only and ⋆ marks the configuration used in our main experiments. (b–c) Test accuracy and selection stability versus profiling size, with stability measured by mean Jaccard similarity to the full-dev team. HIerrHI_err HIdistHI_dist Choice- Soft PoE DS Stacking Overall ✗ ✗ 74.79 75.00 74.26 74.72 74.69 ✓ ✗ 75.24 75.20 74.64 75.27 75.09 ✓ ✓ 75.45 75.24 74.61 75.52 75.21 Table 2: Ablation study of C2-MAS. The first row corresponds to the Quality-Only baseline. Contribution of Profiling Signals. Table 2 presents the performance impact of removing individual heterogeneity signals. (1) Error Decorrelation: With HIerrHI_err, all four aggregators improve over Quality-Only, supporting the value of reducing shared errors. (2) Distributional Divergence: Adding HIdistHI_dist further improves overall accuracy, indicating that choice probabilities contain complementary information. (3) Synergy: The combined signals achieve the highest overall score, consistent with their distinct roles in profiling complementarity. Figure 4: MDS visualization of model heterogeneity. Visualization of the Heterogeneity Landscape. To examine the limitations of Quality-Only selection, we visualize model heterogeneity using multidimensional scaling (MDS). The embedding uses pairwise Yule’s Q and choice-space JSD averaged across the seven primary benchmarks. Models with similar prediction patterns cluster together. As shown in Figure 4, high-accuracy models (e.g., Qwen3, Gemma-2) tend to cluster tightly in the center. This reveals a key phenomenon: the strongest models are often the most similar, likely sharing knowledge boundaries and blind spots. Consequently, a Quality-Only strategy that simply selects the top-k models often results in a highly redundant team. In contrast, distinct models with slightly lower accuracy (e.g., GLM-4) are scattered along the periphery. C2-MAS excels by identifying and recruiting these “outlier” members while maintaining a quality baseline. These diverse perspectives fill the blind spots of strong models, thereby breaking the performance ceiling of homogeneous teams. 4.4 Robustness and Efficiency (RQ3) For real-world deployment, sensitivity to hyperparameters and dependence on cold-start data are key concerns. We verify the robustness of C2-MAS as follows. Sensitivity to Weights. Figure 3(a) shows broad regions of high accuracy away from the Quality-Only origin (0,0)(0,0). The selection score is affine in (λ1,λ2)( _1, _2), so the greedy output is constant within regions where its comparison signs remain fixed (Section C.5). This explains why nearby weights can select the same team. The observed high-accuracy regions indicate low sensitivity to local weight changes in the evaluated range. Data Efficiency and Stability. Figures 3(b) and (c) examine profiling data size. C2-MAS retains its advantage with a few hundred development instances, while Jaccard similarity to the team selected using the full dev set rises rapidly and saturates early. Together, these results show that reliable team composition and competitive accuracy can be achieved with limited development data, supporting the practical efficiency of our framework. 5 Conclusion We proposed C2-MAS, a heterogeneity-driven framework for selecting complementary LLM teams under budget constraints. By explicitly modeling error decorrelation and distributional divergence, our approach quantifies inter-member complementarity beyond individual capability. Extensive experiments validate the effectiveness, robustness, and efficiency of C2-MAS, demonstrating that it consistently outperforms quality-only baselines by recruiting complementary models to fill the blind spots of top-performing candidates. Future work could extend heterogeneity profiling to open-ended generation and explore how complementary teams collaborate through multi-turn debate and iterative verification. Ethics Statement Our experiments use only publicly available multiple-choice benchmarks under their original licenses, and we evaluate open-weight models alongside two closed-source APIs (GPT-4o, Gemini-2.5-Flash) accessed through their official endpoints in accordance with the providers’ terms of use. No human subjects, crowdsourced annotation, or personally identifiable information are involved. A risk specific to aggregation-based methods is that biases or errors present in individual models may be amplified, attenuated, or obscured when their outputs are combined. Although our heterogeneity metrics are designed to diversify error patterns, they do not directly target social bias, and practitioners deploying C2-MAS should pair team selection with bias auditing of the resulting predictions. Reproducibility Statement The candidate models, benchmarks, selection weights, and aggregation settings are described in Sections 4.1 and 3. The appendix provides the choice-scoring implementation, paired-bootstrap procedure, hyperparameter selection protocol, and complete results (Appendices D and E.1). Authors and Affiliations Liangyu Teng1, Hengsong Liu1, Juncen Guo1, Jingyu Zhang1, Yang Liu2, Jing Liu3, Liang Song1 1 College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China 2 Tongji University, Shanghai, China 3 College of Future Information Technology, Fudan University, Shanghai, China References Amini et al. (2019) A. Amini, S. Gabriel, S. Lin, R. Koncel-Kedziorski, Y. Choi, and H. Hajishirzi Mathqa: towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, volume 1 (long and short papers), p. 2357–2367. Cited by: §4.1. Apertus et al. (2025) P. Apertus, A. Hernández-Cano, A. Hägele, A. H. Huang, A. Romanou, A. Solergibert, B. Pasztor, B. Messmer, D. Garbaya, E. F. Ďurech, et al. Apertus: democratizing open and compliant llms for global language environments. arXiv preprint arXiv:2509.14233. Cited by: §4.1. Basant et al. (2025) A. Basant, A. Khairnar, A. Paithankar, A. Khattar, A. Renduchintala, A. Malte, A. Bercovich, A. Hazare, A. Rico, A. Ficek, et al. Nvidia nemotron nano 2: an accurate and efficient hybrid mamba-transformer reasoning model. arXiv preprint arXiv:2508.14444. Cited by: §4.1. Bian and Chen (2022) Y. Bian and H. Chen When does diversity help generalization in classification ensembles?. IEEE Transactions on Cybernetics 52 (9), p. 9059–9075. External Links: Document Cited by: Appendix C. Cai et al. (2024) Z. Cai, M. Cao, H. Chen, K. Chen, K. Chen, X. Chen, X. Chen, Z. Chen, Z. Chen, P. Chu, et al. Internlm2 technical report. arXiv preprint arXiv:2403.17297. Cited by: §4.1. Caruana et al. (2004) R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes Ensemble selection from libraries of models. In Proceedings of the twenty-first international conference on Machine learning, p. 18. Cited by: §2.2, §4.1. Chen et al. (2024a) J. Chen, S. Saha, and M. Bansal ReConcile: round-table conference improves reasoning via consensus among diverse LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 7066–7085. External Links: Link, Document Cited by: §B.1, §B.2. Chen et al. (2024b) L. Chen, M. Zaharia, and J. Zou FrugalGPT: how to use large language models while reducing cost and improving performance. Transactions on Machine Learning Research. Note: Featured Certification External Links: ISSN 2835-8856, Link Cited by: §B.2, §1, §2.1. Chen et al. (2024c) W. Chen, Y. Su, J. Zuo, C. Yang, C. Yuan, C. Chan, H. Yu, Y. Lu, Y. Hung, C. Qian, Y. Qin, X. Cong, R. Xie, Z. Liu, M. Sun, and J. Zhou AgentVerse: facilitating multi-agent collaboration and exploring emergent behaviors. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024, p. 20094–20136. External Links: Link Cited by: §B.1. Chen et al. (2024d) X. Chen, R. Aksitov, U. Alon, J. Ren, K. Xiao, P. Yin, S. Prakash, C. Sutton, X. Wang, and D. Zhou Universal self-consistency for large language models. In ICML 2024 Workshop on In-Context Learning, External Links: Link Cited by: §B.2. Chen et al. (2025) Z. Chen, X. Lu, J. Li, P. Chen, Z. Li, K. Sun, Y. Luo, Q. Mao, M. Li, L. Xiao, D. Yang, X. Huang, Y. Ban, H. Sun, and P. S. Yu Harnessing multiple large language models: a survey on llm ensemble. arXiv preprint arXiv:2502.18036. External Links: Link Cited by: §B.2. Choi et al. (2025) H. K. Choi, X. Zhu, and S. Li Debate or vote: which yields better decisions in multi-agent large language models?. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §B.1, §2.1. Clark et al. (2018) P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: §4.1. Cohen et al. (2026) S. Cohen, N. C. Inger, N. Goldshlager, B. Shapira, and L. Rokach DFPE: a diverse fingerprint ensemble for enhancing LLM performance. In Findings of the Association for Computational Linguistics: EACL 2026, V. Demberg, K. Inui, and L. Marquez (Eds.), Rabat, Morocco, p. 5326–5336. External Links: Link, Document, ISBN 979-8-89176-386-9 Cited by: §2.2. Dekoninck et al. (2025) J. Dekoninck, M. Baader, and M. Vechev A unified approach to routing and cascading for LLMs. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §B.2. Ding et al. (2025) D. Ding, A. Mallick, S. Zhang, C. Wang, D. Madrigal, M. D. C. H. Garcia, M. Xia, L. V. S. Lakshmanan, Q. Wu, and V. Rühle BEST-route: adaptive LLM routing with test-time optimal compute. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §B.2. Du et al. (2024) Y. Du, S. Li, A. Torralba, J. B. Tenenbaum, and I. Mordatch Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. Cited by: §B.1, §1, §2.1. Endres and Schindelin (2003) D. M. Endres and J. E. Schindelin A new metric for probability distributions. IEEE Transactions on Information theory 49 (7), p. 1858–1860. Cited by: §3.2. Estornell and Liu (2024) A. Estornell and Y. Liu Multi-llm debate: framework, principals, and interventions. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 28938–28964. External Links: Document, Link Cited by: §B.1. Geva et al. (2021) M. Geva, D. Khashabi, E. Segal, T. Khot, D. Roth, and J. Berant Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics 9, p. 346–361. Cited by: §4.1. GLM et al. (2024) T. GLM, A. Zeng, B. Xu, B. Wang, C. Zhang, D. Yin, D. Zhang, D. Rojas, G. Feng, H. Zhao, et al. Chatglm: a family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793. Cited by: §4.1. Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §4.1. Hansen and Salamon (1990) L. K. Hansen and P. Salamon Neural network ensembles. IEEE transactions on pattern analysis and machine intelligence 12 (10), p. 993–1001. Cited by: §2.2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt Measuring massive multitask language understanding. In International Conference on Learning Representations, External Links: Link Cited by: §4.1. Hong et al. (2024) S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber MetaGPT: meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §B.1, §2.1. Hu et al. (2024) Q. J. Hu, J. Bieker, X. Li, N. Jiang, B. Keigwin, G. Ranganath, K. Keutzer, and S. K. Upadhyay Routerbench: a benchmark for multi-llm routing system. arXiv preprint arXiv:2403.12031. Cited by: §B.2. Huang et al. (2024) Y. Huang, X. Feng, B. Li, Y. Xiang, H. Wang, T. Liu, and B. Qin Ensemble learning for heterogeneous large language models with deep parallel collaboration. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 119838–119860. External Links: Document, Link Cited by: §B.2, §2.1. Huang et al. (2023) Y. Huang, Y. Bai, Z. Zhu, J. Zhang, J. Zhang, T. Su, J. Liu, C. Lv, Y. Zhang, Y. Fu, et al. C-eval: a multi-level multi-discipline chinese evaluation suite for foundation models. Advances in Neural Information Processing Systems 36, p. 62991–63010. Cited by: §4.1. IBM Granite Team (2024) IBM Granite Team Granite 3.0 language models. Note: Technical report External Links: Link Cited by: §4.1. Jiang et al. (2023) D. Jiang, X. Ren, and B. Y. Lin LLM-blender: ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki (Eds.), Toronto, Canada, p. 14165–14178. External Links: Link, Document Cited by: §B.2, §1, §2.1. Jin et al. (2021) D. Jin, E. Pan, N. Oufattole, W. Weng, H. Fang, and P. Szolovits What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences 11 (14). External Links: Link, ISSN 2076-3417, Document Cited by: §4.1. Krogh and Vedelsby (1994) A. Krogh and J. Vedelsby Neural network ensembles, cross validation, and active learning. Advances in neural information processing systems 7. Cited by: §2.2. Kuncheva and Whitaker (2003) L. I. Kuncheva and C. J. Whitaker Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine learning 51 (2), p. 181–207. Cited by: §C.1, §2.2. Lai et al. (2017) G. Lai, Q. Xie, H. Liu, Y. Yang, and E. Hovy RACE: large-scale reading comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, p. 785–794. Cited by: §4.1. Li et al. (2023) G. Li, H. A. Al Kader Hammoud, H. Itani, D. Khizbullin, and B. Ghanem CAMEL: communicative agents for "mind" exploration of large language model society. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Cited by: §B.1, §2.1. Li et al. (2024) Y. Li, Y. Du, J. Zhang, L. Hou, P. Grabowski, Y. Li, and E. Ie Improving multi-agent debate with sparse communication topology. In Findings of the Association for Computational Linguistics: EMNLP 2024, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 7281–7294. External Links: Link, Document Cited by: §B.1. Liang et al. (2024) T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, S. Shi, and Z. Tu Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Y. Al-Onaizan, M. Bansal, and Y. Chen (Eds.), Miami, Florida, USA, p. 17889–17904. External Links: Link, Document Cited by: §B.1, §1. Lin (1991) J. Lin Divergence measures based on the shannon entropy. IEEE Transactions on Information Theory 37 (1), p. 145–151. External Links: Document Cited by: §C.4, §D.1, §3.2. Ling et al. (2017) W. Ling, D. Yogatama, C. Dyer, and P. Blunsom Program induction by rationale generation: learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), R. Barzilay and M. Kan (Eds.), Vancouver, Canada, p. 158–167. External Links: Link, Document Cited by: §4.1. Liu et al. (2026) A. H. Liu, K. Khandelwal, S. Subramanian, V. Jouault, A. Rastogi, A. Sadé, A. Jeffares, A. Jiang, A. Cahill, A. Gavaudan, et al. Ministral 3. arXiv preprint arXiv:2601.08584. Cited by: §4.1. Liu et al. (2023) H. Liu, J. Liu, L. Cui, Z. Teng, N. Duan, M. Zhou, and Y. Zhang Logiqa 2.0—an improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing 31, p. 2947–2962. Cited by: §4.1. Liu et al. (2024) Z. Liu, Y. Zhang, P. Li, Y. Liu, and D. Yang A dynamic LLM-powered agent network for task-oriented agent collaboration. In First Conference on Language Modeling, External Links: Link Cited by: §B.1, §2.1. Lu et al. (2024) J. Lu, Z. Pang, M. Xiao, Y. Zhu, R. Xia, and J. Zhang Merge, ensemble, and cooperate! a survey on collaborative strategies in the era of large language models. arXiv preprint arXiv:2407.06089. Cited by: §B.2. Martins et al. (2025) P. H. Martins, J. Alves, P. Fernandes, N. M. Guerreiro, R. Rei, A. Farajian, M. Klimaszewski, D. M. Alves, J. Pombal, N. Boizard, et al. Eurollm-9b: technical report. arXiv preprint arXiv:2506.04079. Cited by: §4.1. Mihaylov et al. (2018) T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, p. 2381–2391. Cited by: §4.1. Olmo et al. (2025) T. Olmo, A. Ettinger, A. Bertsch, B. Kuehl, D. Graham, D. Heineman, D. Groeneveld, F. Brahman, F. Timbers, H. Ivison, et al. Olmo 3. arXiv preprint arXiv:2512.13961. Cited by: §4.1. Ong et al. (2025) I. Ong, A. Almahairi, V. Wu, W. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica RouteLLM: learning to route LLMs from preference data. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §B.2, §1, §2.1. O’Brien and Lewis (2023) S. O’Brien and M. Lewis Contrastive decoding improves reasoning in large language models. arXiv preprint arXiv:2309.09117. Cited by: §B.2. Park et al. (2023) J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein Generative agents: interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, UIST ’23, New York, NY, USA. External Links: ISBN 9798400701320, Link, Document Cited by: §B.1. Park et al. (2025) S. Park, X. Liu, Y. Gong, and E. Choi Ensembling large language models with process reward-guided tree search for better complex reasoning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, p. 10256–10277. External Links: Link, Document, ISBN 979-8-89176-189-6 Cited by: §B.2. Piao et al. (2025) J. Piao, Y. Yan, J. Zhang, N. Li, J. Yan, X. Lan, Z. Lu, Z. Zheng, J. Y. Wang, D. Zhou, et al. Agentsociety: large-scale simulation of llm-driven generative agents advances understanding of human behaviors and society. arXiv preprint arXiv:2502.08691. Cited by: §B.1. Qian et al. (2024) C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, J. Xu, D. Li, Z. Liu, and M. Sun ChatDev: communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 15174–15186. External Links: Link, Document Cited by: §B.1. Qian et al. (2025) C. Qian, Z. Xie, Y. Wang, W. Liu, K. Zhu, H. Xia, Y. Dang, Z. Du, W. Chen, C. Yang, Z. Liu, and M. Sun Scaling large language model-based multi-agent collaboration. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 41488–41505. External Links: Link Cited by: §B.1. Shen et al. (2024) Z. Shen, H. Lang, B. Wang, Y. Kim, and D. Sontag Learning to decode collaboratively with multiple language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 12974–12990. External Links: Link, Document Cited by: §B.2. Shnitzer et al. (2024) T. Shnitzer, A. Ou, M. Silva, K. Soule, Y. Sun, J. Solomon, N. Thompson, and M. Yurochkin Large language model routing with benchmark datasets. In First Conference on Language Modeling, External Links: Link Cited by: §B.2, §1, §2.1. Talmor et al. (2019) A. Talmor, J. Herzig, N. Lourie, and J. Berant CommonsenseQA: a question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio (Eds.), Minneapolis, Minnesota, p. 4149–4158. External Links: Link, Document Cited by: §4.1. Tang et al. (2006) E. K. Tang, P. N. Suganthan, and X. Yao An analysis of diversity measures. Machine Learning 65, p. 247–271. External Links: Document Cited by: §C.1, Appendix C. Tang et al. (2025) J. Tang, H. Gao, X. Pan, L. Wang, H. Tan, D. Gao, Y. Chen, X. Chen, Y. Lin, Y. Li, B. Ding, J. Zhou, J. Wang, and J. Wen GenSim: a general social simulation platform with large language model based agents. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations), N. Dziri, S. (. Ren, and S. Diao (Eds.), Albuquerque, New Mexico, p. 143–150. External Links: Link, Document, ISBN 979-8-89176-191-9 Cited by: §B.1. Team et al. (2024) G. Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahriari, A. Ramé, et al. Gemma 2: improving open language models at a practical size. arXiv preprint arXiv:2408.00118. Cited by: §4.1. Tekin et al. (2024) S. F. Tekin, F. Ilhan, T. Huang, S. Hu, and L. Liu LLM-TOPLA: efficient LLM ensemble by maximising diversity. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 11951–11966. External Links: Document, Link Cited by: §2.2. Turkmen et al. (2026) Y. Turkmen, B. Buyukates, and M. Bastopcu Don’t always pick the highest-performing model: an information theoretic view of LLM ensemble selection. arXiv preprint arXiv:2602.08003. External Links: Document, Link Cited by: §2.2. Wang et al. (2025a) J. Wang, J. Wang, B. Athiwaratkun, C. Zhang, and J. Y. Zou Mixture-of-agents enhances large language model capabilities. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 33944–33963. External Links: Link Cited by: §B.2, §1, §2.1. Wang et al. (2023) X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, Cited by: §4.1. Wang et al. (2024) Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, et al. Mmlu-pro: a more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems 37, p. 95266–95290. Cited by: §4.1. Wang et al. (2025b) Z. Wang, S. Moriyama, W. Wang, B. Gangopadhyay, and S. Takamatsu Talk structurally, act hierarchically: a collaborative framework for llm multi-agent systems. arXiv preprint arXiv:2502.11098. Cited by: §B.1. Wang et al. (2025c) Z. Wang, M. Azmat, A. Li, R. Horesh, and M. Yurochkin Speculate, then collaborate: fusing knowledge of language models during decoding. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §B.2. Windeatt (2005) T. Windeatt Diversity measures for multiple classifier system analysis and design. Information fusion 6 (1), p. 21–36. Cited by: §2.2. Wood et al. (2023) D. Wood, T. Mu, A. M. Webb, H. W. J. Reeve, M. Luján, and G. Brown A unified theory of diversity in ensemble learning. Journal of Machine Learning Research 24 (359), p. 1–49. External Links: Link Cited by: Appendix C. Wu et al. (2024) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, and C. Wang AutoGen: enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, External Links: Link Cited by: §B.1, §2.1. Yang et al. (2025) A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §4.1. Yu et al. (2020) W. Yu, Z. Jiang, Y. Dong, and J. Feng ReClor: a reading comprehension dataset requiring logical reasoning. In International Conference on Learning Representations, External Links: Link Cited by: §4.1. Yule (1900) G. U. Yule On the association of attributes in statistics. Philosophical Transactions of the Royal Society of London. Series A 194, p. 257–319. Cited by: §C.1. Zhang et al. (2025a) G. Zhang, Y. Yue, Z. Li, S. Yun, G. Wan, K. Wang, D. Cheng, J. Yu, and T. Chen Cut the crap: an economical communication pipeline for llm-based multi-agent systems. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 75389–75428. External Links: Link Cited by: §B.1. Zhang et al. (2025b) G. Zhang, Y. Yue, X. Sun, G. Wan, M. Yu, J. Fang, K. Wang, T. Chen, and D. Cheng G-designer: architecting multi-agent communication topologies via graph neural networks. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §B.1, §2.1. Zhang et al. (2026) Y. Zhang, K. Lu, Y. Zhang, J. Gao, L. Xia, and F. Yu Mixture of complementary agents for robust LLM ensemble. arXiv preprint arXiv:2605.24048. External Links: Document, Link Cited by: §2.2. Zhou et al. (2024) X. Zhou, H. Zhu, L. Mathur, R. Zhang, H. Yu, Z. Qi, L. Morency, Y. Bisk, D. Fried, G. Neubig, and M. Sap SOTOPIA: interactive evaluation for social intelligence in language agents. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024, p. 40975–41019. External Links: Link Cited by: §B.1. Zhuge et al. (2024) M. Zhuge, W. Wang, L. Kirsch, F. Faccio, D. Khizbullin, and J. Schmidhuber GPTSwarm: language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, External Links: Link Cited by: §B.1, §2.1. Appendix Contents *app:notations . A *app:extended_related_work . B *app:rw_multi_agent . B.1 *app:rw_multi_model . B.2 *app:theory . C *app:correctness_dependence . C.1 *app:joint_failure . C.2 *app:quality_gain . C.3 *app:soft_recovery . C.4 *app:selection_stability . C.5 *app:pooling_geometry . C.6 *app:selection_gain . C.7 *app:stacking_theory . C.8 *app:implementation . D *app:scoring_implementation . D.1 *app:bootstrap . D.2 *app:hyperparameter_selection . D.3 *app:selection_cost . D.4 *app:aggregation_rules . D.5 *app:additional_experiments . E *app:complete_results . E.1 *app:team_composition . E.2 *app:robustness_ablation . E.3 *app:pooled_stability_results . E.4 *app:limitations . F Appendix A Notations Table 3: Notations and Definitions Notation Definition T Set of tasks used for cross-task pooling. t Task / benchmark index. Dtdev,DttestD_t^dev,\,D_t^test Development and test splits for task t (strict dev→ ). tY_t Label / option set for task t (multiple-choice). x,yx,\,y An instance and its gold label (y∈ty _t). ℳ=1,…,mM=\1,…,m\ Index set of the m candidate agents (callable LLMs). m Number of candidate agents. k Team size budget. S⊆ℳS A team of agents, with |S|=k|S|=k. i∈ℳi Index of a candidate agent. pi(⋅∣x)p_i(· x) Agent i’s aligned choice distribution over tY_t for instance x (task index omitted when clear). y^i(x) y_i(x) Predicted label of agent i, y^i(x)=arg⁡maxy∈tpi(y∣x) y_i(x)= _y _tp_i(y x). Ci(x)∈0,1C_i(x)∈\0,1\ Correctness indicator of agent i on example x. Nab(i,j)N_ab(i,j) Count of examples where (Ci(x),Cj(x))=(a,b)(C_i(x),C_j(x))=(a,b) for a,b∈0,1a,b∈\0,1\. R⁡(i,j)R(i,j) Error-correlation / complementarity statistic between agents i and j (e.g., Yule’s Q). qitq_i^t Estimated individual quality of agent i on task t (development accuracy). R¯ R Pooled Yule’s Q association matrix estimated from development data. J¯ J Pooled distribution-divergence matrix estimated from development data (across tasks if applicable). HIerr(S,R¯)HI_err(S; R) Error-level heterogeneity / complementarity of team S (computed from R¯ R). HIdist(S,J¯)HI_dist(S; J) Distribution-level heterogeneity / disagreement of team S (computed from J¯ J). z⁡(⋅)z(·) Standard-error-scaled normalization of a team average using candidate-pool base-item statistics. λ1,λ2 _1,\, _2 Objective weights trading off capability and complementarity. scoret(S)score_t(S) Standardized selection objective for task t (see Equation 10). St∗S_t^* Size-k team returned by multi-start greedy search on scoret(S)score_t(S). pcst(y∣x)p_ cs^t(y x) Choice-Soft aggregation distribution (mean of members’ pitp_i^t). ppoet(y∣x)p_ poe^t(y x) PoE aggregation distribution (product of members’ pitp_i^t, renormalized). πt(y)π^t(y) Class prior estimated from DtdevD_t^dev (used in DS). Ait(a,b)A_i^t(a,b) DS confusion matrix entry estimating ℙ⁡(y^i=b∣y=a)P( y_i=b y=a) on DtdevD_t^dev. ε Additive smoothing constant for DS priors/confusions (e.g., ε=10−3 =10^-3). pdst(y∣x)p_ ds^t(y x) DS posterior (up to proportionality) combining πtπ^t and Ait\A_i^t\. ϕ⁡(x)φ(x) Stacking feature vector formed by concatenating members’ pit(⋅∣x)p_i^t(· x). W,bW,\,b Linear stacking combiner parameters. β L2L_2 regularization coefficient for stacking. πW,b(y∣x) _W,b(y x) Stacking aggregation distribution, πW,b(y∣x)=softmax(W⊤ϕ(x)+b)y _W,b(y x)=softmax(W φ(x)+b)_y. y^St(x) y_S_t(x) Final team prediction on x under a chosen aggregator. q¯S,eS q_S,\,e_S Mean member accuracy and error probability in the fixed-task theoretical model. DS,KS,JSD_S,\,K_S,\,J_S Mean pairwise double-fault probability, same-wrong-answer probability, and JSD in that model. d,h,θd,\,h,\,θ Divergence between distinct wrong-answer templates, divergence between correct and incorrect templates, and θ=2h/dθ=2h/d. εQ _Q Baseline correction in the accuracy comparison with Quality-Only. Appendix B Additional Related Works B.1 Multi-Agent LLM Systems Much prior work builds multi-agent collaboration on predefined roles and fixed workflows, ranging from role-playing paradigms (Li et al., 2023; Hong et al., 2024) and workflow-based dialogue orchestration (Wu et al., 2024; Qian et al., 2024; Chen et al., 2024c) to multi-agent simulation environments for modeling collective and social behaviors (Park et al., 2023; Zhou et al., 2024; Tang et al., 2025; Piao et al., 2025). To improve reliability, studies model interaction as structured group decision processes such as debate (Du et al., 2024; Liang et al., 2024), vote-based aggregation (Choi et al., 2025), and consensus mechanisms (Chen et al., 2024a), and analyze protocol interventions to understand mechanisms and failure modes (Estornell and Liu, 2024). More recent work treats the interaction structure as an optimization target, using learnable communication graphs (Zhuge et al., 2024; Zhang et al., 2025b), sparse or hierarchical messaging (Li et al., 2024; Qian et al., 2025; Wang et al., 2025b), pruning and economical communication pipelines (Zhang et al., 2025a), or dynamic team formation (Liu et al., 2024) to improve scalability and efficiency. B.2 Multi-Model Collaboration, Output Fusion, and Routing Multi-model collaboration is often organized as an ensemble pipeline that generates candidates, selects among them, and fuses outputs (Lu et al., 2024; Chen et al., 2025). Methods operate at the output level via pairwise ranking (Jiang et al., 2023), consistency-based selection (Chen et al., 2024d), or consensus aggregation (Chen et al., 2024a); during reasoning through iterative refinement (Wang et al., 2025a) or tree-search-based decision making (Park et al., 2025); or at decoding time by combining token-level distributions (Huang et al., 2024; O’Brien and Lewis, 2023; Shen et al., 2024; Wang et al., 2025c). For deployment, routing and cascading dynamically choose models per input to trade quality for cost (Chen et al., 2024b; Shnitzer et al., 2024; Ong et al., 2025), with systematic benchmarks characterizing the quality–cost trade-off (Hu et al., 2024; Ding et al., 2025) and unified analyses of when such strategies are effective (Dekoninck et al., 2025). Appendix C Theoretical Analysis and Proofs The analysis follows the two questions in Section 3.5: how the two heterogeneity signals characterize collective errors, and when their benefits compensate for a reduction in member quality. Classical ensemble analyses relate diversity to margins and loss decompositions (Tang et al., 2006; Bian and Chen, 2022; Wood et al., 2023). We develop the connection through a choice-probability model, prove an accuracy comparison with Quality-Only, and relate its terms to the standardized selection objective. Further results cover soft recovery, profiling perturbations, cross-task pooling, and the member-specific information available to Stacking. JSD uses base-22 logarithms; cross-entropy and the auxiliary logarithmic calculations below use natural logarithms. C.1 Correctness-Level Dependence Correctness correlations summarize which examples models solve together. We first record their connection to shared errors, using the classical diversity statistics studied by Yule (1900); Kuncheva and Whitaker (2003); Tang et al. (2006). Fix a task distribution D; an empirical development distribution is also allowed. Write qi=⁡[Ci]q_i=E[C_i], Ei=1−CiE_i=1-C_i, and DFij=ℙ⁡(Ei=Ej=1)DF_ij=P(E_i=E_j=1). Lemma C.1 (Fixed-marginal dependence). For fixed qi,qj∈(0,1)q_i,q_j∈(0,1), Yule’s Q, correctness covariance, and double-fault probability are strictly increasing functions of the joint-correct probability t=ℙ⁡(Ci=Cj=1)t=P(C_i=C_j=1) over the interior of its feasible interval. In particular, DFij=(1−qi)(1−qj)+Cov⁡(Ci,Cj).DF_ij=(1-q_i)(1-q_j)+Cov(C_i,C_j). (14) Proof. The four joint probabilities are p11=tp_11=t, p10=qi−tp_10=q_i-t, p01=qj−tp_01=q_j-t, and p00=1−qi−qj+tp_00=1-q_i-q_j+t, where max⁡0,qi+qj−1≤t≤min⁡qi,qj \0,q_i+q_j-1\≤ t≤ \q_i,q_j\. Thus Cov⁡(Ci,Cj)=t−qiqjCov(C_i,C_j)=t-q_iq_j and DFij=1−qi−qj+tDF_ij=1-q_i-q_j+t, giving Equation 14; both have derivative one. In the interior, all four cells are positive. For the odds ratio O⁡(t)=p11p00/(p10p01)O(t)=p_11p_00/(p_10p_01), dtlog⁡O⁡(t)=1t+11−qi−qj+t+1qi−t+1qj−t>0. ddt O(t)= 1t+ 11-q_i-q_j+t+ 1q_i-t+ 1q_j-t>0. (15) Since Q=(O−1)/(O+1)Q=(O-1)/(O+1) and dQ/dO=2/(O+1)2>0dQ/dO=2/(O+1)^2>0, Q is strictly increasing as well. Endpoint values follow by continuity. Finally, Cov⁡(Ei,Ej)=Cov⁡(Ci,Cj)Cov(E_i,E_j)=Cov(C_i,C_j) because Ei=1−CiE_i=1-C_i. ∎ With individual qualities held fixed, HIerrHI_err therefore favors pairs with fewer shared errors. For two members, an oracle that succeeds whenever either member is correct has error probability DFijDF_ij. Distributional analysis below examines how soft aggregation can recover information even on examples where every member’s top prediction is wrong. Classical majority-vote bound. For completeness, the standard one-sided variance argument gives another interpretation of correctness dependence. For binary-label prediction, let k≥3k≥ 3 be odd and let correctness indicators have common mean p∈(1/2,1)p∈(1/2,1) and common pairwise correlation ρ, and put C¯=k−1∑iCi C=k^-1 _iC_i. Expanding the variance and applying Cantelli’s inequality gives V⁡(ρ):=Var⁡(C¯) V(ρ):=Var( C) =p⁡(1−p)k(1+(k−1)ρ), = p(1-p)k (1+(k-1)ρ ), (16) ℙ⁡(majority vote errs)=ℙ⁡(C¯≤1/2) (majority vote errs)=P( C≤ 1/2) ≤V⁡(ρ)V⁡(ρ)+(p−1/2)2. ≤ V(ρ)V(ρ)+(p-1/2)^2. Indeed, the variance contains k diagonal terms p⁡(1−p)p(1-p) and k⁡(k−1)k(k-1) off-diagonal terms ρp(1−p)ρ p(1-p), all divided by k2k^2. Cantelli’s inequality applied to C¯−p≤−(p−1/2) C-p≤-(p-1/2) yields the second line. Writing A=(p−1/2)2>0A=(p-1/2)^2>0, the derivative of this upper bound is Ap(1−p)(k−1)/[k(V⁡(ρ)+A)2]>0Ap(1-p)(k-1)/[k(V(ρ)+A)^2]>0 throughout the feasible correlation range. This is a classical bound on binary majority voting; the subsequent results analyze the choice distributions and selection rule used by C2-MAS directly. C.2 Joint Characterization of Collective Failure We work on one fixed distribution D of labeled examples. The same identities hold for the uniform distribution on a finite set. Task indices are suppressed within this analysis; the target-task and pooled profiles are distinguished explicitly in Section C.7. Model and pairwise quantities. All candidate members follow (11), with common constants a,b,c,ua,b,c,u. For each example, let wi≠yw_i≠ y denote the preferred answer of an incorrect member. Writing the true answer first, the distributions are pc=(a,u,u,u),pw1=(a,b,c,c),pw2=(a,c,b,c),pw3=(a,c,c,b).p c=(a,u,u,u), p^w_1=(a,b,c,c), p^w_2=(a,c,b,c), p^w_3=(a,c,c,b). The correctness indicators and the wiw_i have an arbitrary joint distribution across members and examples. Since c<u<bc<u<b and a>(2b+u)/3a>(2b+u)/3, we have a>ua>u, so these templates have the stated unique preferred answers. The parameter region has nonempty interior; for example, (a,b,c)=(0.42,0.48,0.05)(a,b,c)=(0.42,0.48,0.05) satisfies all inequalities. For a three-member team, define q¯S q_S =13∑i∈Sqi,eS=1−q¯S, = 13 _i∈ Sq_i, e_S=1- q_S, DS D_S =13∑i,j⊂Sℙ⁡(Ci=Cj=0), = 13 _\i,j\⊂ SP(C_i=C_j=0), KS K_S =13∑i,j⊂Sℙ(Ci=Cj=0,wi=wj), = 13 _\i,j\⊂ SP(C_i=C_j=0,\ w_i=w_j), JS J_S =13∑i,j⊂S⁡[JSD2(pi,pj)]. = 13 _\i,j\⊂ SE[JSD_2(p_i,p_j)]. (17) The event defining KSK_S evaluates wi,wjw_i,w_j only when both members are incorrect. Here JSJ_S is the population counterpart of the task-specific HIdistHI_dist. The pairwise terms in DSD_S are determined by the marginal qualities and Yule’s Q on nondegenerate correctness tables, by Lemma C.1. Divergence constants. Let g⁡(x,z)=12[xlog22x+z+zlog22zx+z].g(x,z)= 12 [x _2 2xx+z+z _2 2zx+z ]. The two relevant divergences are d=2g(b,c)>0,h=g⁡(b,u)+2g(c,u)>0,θ=2hd.d=2g(b,c)>0, h=g(b,u)+2g(c,u)>0, θ= 2hd. (18) The true-answer coordinate contributes zero to both divergences. Two correct members, or two incorrect members preferring the same answer, have identical distributions. A correct–incorrect pair contributes h, and two incorrect members preferring different answers contribute d. Lemma C.2 (Positive coefficients in the joint decomposition). For b>c>0b>c>0 and u=(b+2c)/3u=(b+2c)/3, the constants in (18) satisfy 0<2h<d0<2h<d. Proof. Positivity follows from the distinct templates. To prove d−2h>0d-2h>0, set t=b/c>1t=b/c>1 and φ⁡(z)=zln⁡z (z)=z z. Expanding the scalar divergences and cancelling the terms containing ln⁡c c gives d−2h=cln⁡2F(t),d-2h= c 2F(t), where F⁡(t)=2φ(2t+13)+4φ(t+56)−3φ(t+23)−2φ(t+12).F(t)=2 \! ( 2t+13 )+4 \! ( t+56 )-3 \! ( t+23 )-2 \! ( t+12 ). Direct differentiation yields F⁡(1)=F′(1)=0F(1)=F (1)=0 and F′(t)=−2t2+7t+13(t+2)(t+1)(2t+1)(t+5).F (t)= -2t^2+7t+13(t+2)(t+1)(2t+1)(t+5). (19) This derivative is positive for 1<t<t∗=(7+153)/41<t<t_*=(7+ 153)/4 and negative for t>t∗t>t_*. Thus F′F first increases from zero and then decreases to limt→∞F′(t)=53ln⁡2−ln⁡3>0, _t→∞F (t)= 53 2- 3>0, where the final inequality is equivalent to 32>2732>27. Consequently F′(t)>0F (t)>0 for every t>1t>1, and integration from 11 gives F⁡(t)>0F(t)>0. Dividing 0<2h<d0<2h<d by d proves 0<θ<10<θ<1. ∎ Proof of Proposition 3.1. For a pair (i,j)(i,j), write Dij=ℙ⁡(Ci=Cj=0)D_ij=P(C_i=C_j=0) and Kij=ℙ(Ci=Cj=0,wi=wj)K_ij=P(C_i=C_j=0,w_i=w_j). The probability that exactly one member is incorrect is (1−qi)+(1−qj)−2Dij(1-q_i)+(1-q_j)-2D_ij. The pairwise divergence calculation above therefore gives ⁡[JSD2(pi,pj)]=h⁡[(1−qi)+(1−qj)−2Dij]+d⁡(Dij−Kij).E[JSD_2(p_i,p_j)]=h [(1-q_i)+(1-q_j)-2D_ij ]+d(D_ij-K_ij). (20) Each member occurs in two of the three unordered pairs. Averaging yields JS=2h(eS−DS)+d⁡(DS−KS)J_S=2h(e_S-D_S)+d(D_S-K_S). Rearranging and using Lemma C.2 establishes (12). It remains to characterize aggregation. Every member assigns a to the true answer. If all three are incorrect and prefer the same wrong answer, that answer has score b>ab>a under Choice-Soft and b3>a3b^3>a^3 under PoE. Both aggregators therefore err. In every other case, fix an incorrect answer r. If at least one member is correct, the three probabilities assigned to r have sum at most 2b+u2b+u and product at most b2ub^2u. If all members are incorrect but their preferred answers are not all equal, at most two assign b to r, so the corresponding bounds are 2b+c<2b+u2b+c<2b+u and b2c<b2ub^2c<b^2u. Thus the true answer has a strictly larger Choice-Soft score by (11). For PoE, the same condition and the arithmetic–geometric mean inequality imply a3>(2b+u3)3≥b2u.a^3> ( 2b+u3 )^3≥ b^2u. Hence both aggregators err exactly on the event that all three members prefer the same wrong answer. On that event, every pair contributes one to the same-wrong-answer indicator. Pointwise, y^S≠y≤13∑i,j⊂SCi=Cj=0,wi=wj.1\ y_S≠ y\≤ 13 _\i,j\⊂ S1\C_i=C_j=0,\ w_i=w_j\. Taking expectations gives ℙ⁡(y^S≠Y)≤KSP( y_S≠ Y)≤ K_S. ∎ C.3 Accuracy Gains over Quality-Only Selection We derive a lower bound on the baseline’s error probability to accompany the upper bound in Proposition 3.1. Their combination gives an accuracy comparison between teams. Lemma C.3 (Error bounds from pairwise profiles). Under (11), let ℛSR_S be the error probability of either Choice-Soft or PoE. Then max⁡0,DS+3KS−2eS2≤ℛS≤min⁡KS, 1−3eS+3DS. \0, D_S+3K_S-2e_S2 \ _S≤ \K_S,\ 1-3e_S+3D_S\. (21) Both endpoints are attainable for every feasible triple (eS,DS,KS)(e_S,D_S,K_S). In particular, 0≤KS−ℛS≤εS:=min⁡KS,eS−DS+KS2.0≤ K_S-R_S≤ _S:= \K_S,\ e_S- D_S+K_S2 \. (22) Proof. Suppress the team subscript. There are seven possible error patterns, summarized below. The last four columns give the contribution of each pattern to the mean member error, double-fault probability, same-wrong-answer probability, and aggregation error. Probability Error pattern e D K ℛR p0p_0 No member incorrect 00 00 00 00 p1p_1 One member incorrect 1/31/3 00 00 00 p2sp_2s Two incorrect, same answer 2/32/3 1/31/3 1/31/3 00 p2dp_2d Two incorrect, different answers 2/32/3 1/31/3 00 00 p3sp_3s Three incorrect, same answer 11 11 11 11 p21p_21 Three incorrect, two answers 11 11 1/31/3 00 p111p_111 Three incorrect, three answers 11 11 00 00 These probabilities are nonnegative and sum to one. The table gives e e =p1+2p2s+2p2d3+p3s+p21+p111, = p_1+2p_2s+2p_2d3+p_3s+p_21+p_111, D D =p2s+p2d3+p3s+p21+p111, = p_2s+p_2d3+p_3s+p_21+p_111, K K =p2s3+p3s+p213,ℛ=p3s. = p_2s3+p_3s+ p_213, =p_3s. (23) In particular, K−ℛ=(p2s+p21)/3≥0K-R=(p_2s+p_21)/3≥ 0. A second identity is e−D+K2−(K−ℛ)=p13+p2d+p1112≥0.e- D+K2-(K-R)= p_13+ p_2d+p_1112≥ 0. Since ℛ≥0R≥ 0, these two identities imply (22) and the lower bound in (21). Finally, 1−3e+3D=p0+p3s+p21+p111≥ℛ1-3e+3D=p_0+p_3s+p_21+p_111 gives the second upper bound. To show attainability, fix any feasible (e,D,K)(e,D,K) and choose any f between the two endpoints in (21). Set τ=max⁡f, 2D−e, 0,z=max⁡0, 3(K−D+τ−f).τ= \f,\,2D-e,\,0\, z= \0,\,3(K-D+τ-f)\. Feasibility gives 0≤K≤D≤e0≤ K≤ D≤ e and D≥2e−1D≥ 2e-1. The bounds on f then imply f≤τ≤min⁡D, 1−3e+3D,f+32(D−K).f≤τ≤ \D,\ 1-3e+3D,\ f+ 32(D-K)\. For the final inequality, the only additional comparison is 2D−e≤f+32(D−K)2D-e≤ f+ 32(D-K), which is exactly the stated lower bound on f. It follows that 0≤z≤min⁡τ−f, 3(K−f)0≤ z≤ \τ-f,\,3(K-f)\. Define p3s p_3s =f, =f, p21 p_21 =z, =z, p111 p_111 =τ−f−z, =τ-f-z, p2s p_2s =3(K−f)−z, =3(K-f)-z, p2d p_2d =3(D−τ)−p2s, =3(D-τ)-p_2s, p1 p_1 =3e−6D+3τ, =3e-6D+3τ, p0 p_0 =1−3e+3D−τ. =1-3e+3D-τ. Every term is nonnegative; for p2dp_2d this follows from z≥3(K−D+τ−f)z≥ 3(K-D+τ-f). They sum to one, and substitution into (23) recovers (e,D,K,ℛ)=(e,D,K,f)(e,D,K,R)=(e,D,K,f). Each pattern is realizable using the three incorrect choices. Thus the endpoints are sharp given these aggregate pairwise quantities. ∎ Proof of Proposition 3.2. Write ℛS=1−Acc⁡(S)R_S=1-Acc(S), with the same aggregator for both teams. By Lemma C.3, ℛQ≥KQ−εQR_Q≥ K_Q- _Q and ℛS≤KSR_S≤ K_S. Therefore Acc⁡(S)−Acc⁡(SQ)=ℛQ−ℛS≥KQ−KS−εQ.Acc(S)-Acc(S_Q)=R_Q-R_S≥ K_Q-K_S- _Q. Applying (12) to each team and using eQ−eS=−(q¯Q−q¯S)e_Q-e_S=-( q_Q- q_S) gives (13). Its positive right-hand side implies a strict accuracy improvement. If the baseline members make identical predictions, then DQ=KQ=eQD_Q=K_Q=e_Q, hence εQ=0 _Q=0. ∎ The correction quantifies how much pairwise error concentration can overstate the baseline’s aggregate error. Its exact excess is (p2s+p21)/3(p_2s+p_21)/3: these are cases with a same-wrong-answer pair that the third member or soft aggregation resolves. The bound on this excess uses only (eQ,DQ,KQ)(e_Q,D_Q,K_Q), with KQK_Q supplied by the joint decomposition. The full interval in (21) also gives the sharper comparison Acc⁡(S)−Acc⁡(SQ)≥max⁡0,DQ+3KQ−2eQ2−min⁡KS, 1−3eS+3DS.Acc(S)-Acc(S_Q)≥ \0, D_Q+3K_Q-2e_Q2 \- \K_S,\ 1-3e_S+3D_S\. Equation (13) separates this comparison into the two complementarity gains and the member-quality difference. Perturbations of the probability templates. The joint mechanism extends to a neighborhood of the common templates. Suppose each actual distribution is within total variation δ of its template on every example, where 0≤δ<c,2δ<min⁡b−a,a−2b+u3.0≤δ<c, 2δ< \b-a,\ a- 2b+u3 \. (24) A coordinate changes by at most δ. An incorrect member retains its preferred answer because b−a>2δb-a>2δ; a correct member retains its preferred answer because a−u>a−(2b+u)/3>2δa-u>a-(2b+u)/3>2δ. For every pattern with successful aggregation, the correct Choice-Soft score is at least a−δa-δ and every wrong score is at most (2b+u)/3+δ(2b+u)/3+δ. The same inequality gives (a−δ)3>(b+δ)2(u+δ)(a-δ)^3>(b+δ)^2(u+δ) by the arithmetic–geometric mean inequality, preserving PoE recovery. The common wrong answer still wins in the remaining pattern. Thus e,D,Ke,D,K and both aggregation accuracies are unchanged. By the JSD gradient argument in (49), the observed divergence JS′J _S satisfies |JS′−JS|≤ηδ,ηδ=2δlog21c−δ.|J _S-J_S|≤ _δ, _δ=2δ _2 1c-δ. Consequently, defining KS′=θeS+(1−θ)DS−JS′/dK _S=θ e_S+(1-θ)D_S-J _S/d gives |KS′−KS|≤ηδ/d|K _S-K_S|≤ _δ/d and ℛS≤KS′+ηδ/dR_S≤ K _S+ _δ/d. The map K↦min⁡K,e−(D+K)/2K \K,e-(D+K)/2\ is 11-Lipschitz. After clipping its value below at zero, the baseline correction computed with KQ′K _Q differs from εQ _Q by at most ηδ/d _δ/d. Replacing JS,JQ,εQJ_S,J_Q, _Q in (13) by these perturbed quantities therefore preserves the lower bound after subtracting 3ηδ/d3 _δ/d. C.4 Soft Recovery from Dispersed Incorrect Choices This section isolates recovery on examples where every member is incorrect, extending the mechanism to general team sizes, larger label sets, and perturbed choice distributions. We work on a fixed distribution of labeled examples; the same calculations apply to the uniform distribution on a finite profiling set. Throughout this section, Jensen–Shannon divergence is measured in bits (Lin, 1991): JSD2(p,q)=12∑rprlog22prpr+qr+12∑rqrlog22qrpr+qr.JSD_2(p,q)= 12 _rp_r _2 2p_rp_r+q_r+ 12 _rq_r _2 2q_rp_r+q_r. (25) Incorrect-choice occupancy and aggregation margins. Consider k≥2k≥ 2 members and L≥4L≥ 4 choices. On an example with true label y, member i has an incorrect preferred choice wi≠yw_i≠ y and distribution pi(r)=a,r=y,b,r=wi,c,r∉y,wi,0<c<a<b,a+b+(L−2)c=1.p_i(r)= casesa,&r=y,\\ b,&r=w_i,\\ c,&r∉\y,w_i\, cases 0<c<a<b, a+b+(L-2)c=1. (26) Thus every member predicts incorrectly, while the true label is each member’s second choice. For each incorrect label r, let nr=∑i=1kwi=r,vr=nrk,∑r≠yvr=1.n_r= _i=1^k1\w_i=r\, v_r= n_rk, _r≠ yv_r=1. (27) Define the divergence between two members with different incorrect preferred choices and the mean pairwise divergence on this example by d d =blog22b+c+clog22cb+c>0, =b _2 2bb+c+c _2 2cb+c>0, (28) jS(x) j_S(x) =1(k2)∑i<jJSD2(pi,pj). = 1 k2 _i<jJSD_2(p_i,p_j). (29) Lemma C.4 (Occupancy and strict recovery). Under (26), jS(x)=dk2−∑r≠ynr2k⁡(k−1)=dk−1(1−∑r≠yvr2).j_S(x)=d\, k^2- _r≠ yn_r^2k(k-1)=d\, kk-1 (1- _r≠ yv_r^2 ). (30) Choice-Soft assigns the true label a strictly larger score than every incorrect label if and only if maxr≠y⁡vr<a−cb−c. _r≠ yv_r< a-cb-c. (31) For PoE, the corresponding necessary and sufficient condition is maxr≠y⁡vr<log2⁡(a/c)log2⁡(b/c). _r≠ yv_r< _2(a/c) _2(b/c). (32) Equality in either threshold gives a tie between the true label and a highest scoring incorrect label for that aggregator. Proof. When wi=wjw_i=w_j, the two distributions coincide. When wi≠wjw_i≠ w_j, they differ only at these two incorrect labels, with probabilities (b,c)(b,c) and (c,b)(c,b). Substitution into (25) gives JSD2(pi,pj)=d 1wi≠wj.JSD_2(p_i,p_j)=d\,1\w_i≠ w_j\. (33) The number of unordered pairs with different incorrect preferred choices is (k2)−∑r≠y(nr2)=k2−∑r≠ynr22, k2- _r≠ y n_r2= k^2- _r≠ yn_r^22, which proves (30). Choice-Soft assigns probability a to the true label and c+(b−c)vrc+(b-c)v_r to incorrect label r. Its margin against its strongest incorrect competitor is therefore a−c−(b−c)maxr≠yvr.a-c-(b-c) _r≠ yv_r. (34) This margin is positive exactly under (31). For PoE, the unnormalized scores are aka^k for the true label and bnrck−nrb^n_rc^k-n_r for incorrect label r. The base-two log score ratio, divided by k, against the strongest incorrect competitor is log2⁡(a/c)−maxr≠y⁡vrlog2⁡(b/c). _2(a/c)- _r≠ yv_r _2(b/c). (35) Since b>c>0b>c>0, this expression is positive exactly under (32). The common positive PoE normalizer does not affect the comparison. These expressions also establish the statements about ties. ∎ The divergence records the squared concentration ∑rvr2 _rv_r^2 of incorrect preferred choices, whereas recovery depends on their largest concentration maxr⁡vr _rv_r. This distinction explains both the information supplied by choice-space divergence and the information lost by averaging it. Within this family, the PoE threshold is larger than the Choice-Soft threshold: strict concavity of log2 _2 gives log2⁡(a/c)log2⁡(b/c)>a−cb−c(c<a<b). _2(a/c) _2(b/c)> a-cb-c (c<a<b). Three-member recovery and sharp bounds. Proposition C.5 (Distributional heterogeneity and recovery). Let k=3k=3 and suppose (26) holds on the event H that all three members predict incorrectly, with ℙ⁡(H)>0P(H)>0 and a>(2b+c)/3a>(2b+c)/3. Define U=⁡[jS(X)∣H]/dU=E[j_S(X) H]/d, with jS,dj_S,d from (29) and (28). For either Choice-Soft or PoE, the conditional recovery probability G=ℙ⁡(y^S=Y∣H)G=P( y_S=Y H) satisfies the sharp bounds U≤G≤min⁡1,3U/2.U≤ G≤ \1,3U/2\. (36) Both aggregators recover exactly when the three preferred wrong answers are not all identical. Proof of Proposition C.5. Condition on H, the event that all three members predict incorrectly. If all three incorrect preferred choices coincide, that choice receives score b>ab>a under Choice-Soft and unnormalized score b3>a3b^3>a^3 under PoE, so both aggregators fail. Otherwise, no incorrect label is preferred by more than two members. The condition a>(2b+c)/3a>(2b+c)/3 makes the Choice-Soft margin strictly positive even for an incorrect label preferred by two members. Moreover, a3>(2b+c3)3≥b2c,a^3> ( 2b+c3 )^3≥ b^2c, so PoE also recovers in this case. Thus both aggregators recover on H exactly when the three incorrect preferred choices are not all equal. Let r3r_3, r21r_21, and r111r_111 denote, conditional on H, the probabilities of all three choices being equal, exactly two being equal, and all three being different, respectively. They are nonnegative and sum to one. Equation (33) gives mean pairwise divergences 00, 2d/32d/3, and d in these three cases. Consequently, the quantities U and G in the proposition satisfy U=23r21+r111,G=r21+r111.U= 23r_21+r_111, G=r_21+r_111. (37) In particular, G−U=13r21≥0,32U−G=12r111≥0,G-U= 13r_21≥ 0, 32U-G= 12r_111≥ 0, and G≤1G≤ 1, proving U≤G≤min⁡1,3U/2U≤ G≤ \1,3U/2\. Both endpoints are attainable for every U∈[0,1]U∈[0,1]. The lower endpoint follows from (r3,r21,r111)=(1−U,0,U)(r_3,r_21,r_111)=(1-U,0,U). For the upper endpoint, take (r3,r21,r111)=(1−3U/2, 3U/2, 0),0≤U≤2/3,(0, 3(1−U), 3U−2),2/3≤U≤1.(r_3,r_21,r_111)= cases(1-3U/2,\,3U/2,\,0),&0≤ U≤ 2/3,\\ (0,\,3(1-U),\,3U-2),&2/3≤ U≤ 1. cases (38) There are at least three incorrect labels because L≥4L≥ 4, so all three patterns can be realized. These constructions use the same a,b,ca,b,c and can be combined with any fixed distribution of predictions outside H. Finally, changing the joint pattern of (w1,w2,w3)(w_1,w_2,w_3) on H leaves each correctness indicator equal to zero there. With H and all predictions outside H fixed, the entire correctness vector is unchanged. Hence every member’s accuracy and every pair’s correctness contingency table are unchanged, as is Yule’s Q, including the zero-denominator convention in (4). ∎ Relation to overall accuracy and the profiled divergence. Let τ=ℙ⁡(H)>0τ=P(H)>0, and write Jall=⁡[jS(X)],Jout=⁡[HcjS(X)].J_all=E[j_S(X)], J_out=E[1\H^c\j_S(X)]. (39) Here JallJ_all is precisely the population version of HIdist(S,Jt)HI_dist(S;J^t) for the fixed task distribution. For aggregator ∈cs,poeA∈\ cs, poe\, define Aout=ℙ⁡(Hc,y^(X)=Y)A_out^A=P(H^c, y_A(X)=Y) and let Acc=ℙ⁡(y^(X)=Y)Acc_A=P( y_A(X)=Y). Partitioning by H gives Jall=Jout+τdU,Acc=Aout+τG.J_all=J_out+τ dU, _A=A_out^A+τ G. (40) The sharp conditional bounds therefore imply Aout+Jall−Joutd A_out^A+ J_all-J_outd ≤Acc _A (41) ≤Aout+min⁡τ,3(Jall−Jout)2d. ≤ A_out^A+ \! \τ, 3(J_all-J_out)2d \. These bounds remain sharp when the predictions outside H are fixed. For example, suppose that with probability q∈(0,1)q∈(0,1) all three members output the same positive distribution whose unique preferred choice is correct, and the remaining probability 1−q1-q follows the model on H. Then qi=q_i=q and Qij=1Q_ij=1 for every pair, while Jout=0J_out=0 and Aout=qA_out^A=q. Equation (41) reduces to q+Jalld≤Acc≤q+min⁡1−q,3Jall2d.q+ J_alld _A≤ q+ \! \1-q, 3J_all2d \. (42) Thus the existing choice-space divergence can quantify recovery opportunities while the full correctness profile is held fixed. The calculation applies task by task; the pooled HIdistHI_dist is the arithmetic mean of the corresponding taskwise divergences. Stability under perturbations of member distributions. The recovery mechanism persists when the members’ probabilities differ from the common template. We use total variation in the convention TV⁡(p,q)=12‖p−q‖1TV(p,q)= 12\|p-q\|_1. Corollary C.6 (Perturbed soft recovery). Let k=3k=3, and let pi0p_i^0 have the form (26) on an event H of positive probability. Suppose the actual distributions pip_i satisfy, almost surely on H, TV⁡(pi,pi0)≤ε(i=1,2,3),0≤ε<c,b−a>2ε,TV(p_i,p_i^0)≤ (i=1,2,3), 0≤ <c, b-a>2 , (43) and a−2b+c3>2ε.a- 2b+c3>2 . (44) Each member still has unique preferred choice wi≠yw_i≠ y, and both Choice-Soft and PoE recover on H exactly when (w1,w2,w3)(w_1,w_2,w_3) are not all equal. Let G be this conditional recovery probability and set JH=⁡[13∑i<jJSD2(pi,pj)|H],η=2εlog21c−ε.J_H=E\! [ 13 _i<jJSD_2(p_i,p_j)\, |\,H ], η=2 _2 1c- . (45) With d as in (28), max⁡0,JH−ηd≤G≤min⁡1,3(JH+η)2d. \! \0, J_H-ηd \≤ G≤ \! \1, 3(J_H+η)2d \. (46) For PoE alone, the conclusions hold with (44) replaced by (a−ε)3>(b+ε)2(c+ε).(a- )^3>(b+ )^2(c+ ). (47) Proof. For probability distributions, a total-variation bound of ε implies an absolute difference of at most ε in every coordinate. Therefore pi(wi)≥b−ε>a+ε≥pi(y),p_i(w_i)≥ b- >a+ ≥ p_i(y), and pi(wi)>pi(r)p_i(w_i)>p_i(r) for every other incorrect label r, since a>ca>c. Thus each wiw_i remains the unique preferred choice. If all three wiw_i agree, their common label strictly exceeds the true label for every member, so both the arithmetic mean and the product favor an incorrect label. If the wiw_i are not all equal, any incorrect label is preferred by at most two members. Its Choice-Soft score is at most (2b+c)/3+ε(2b+c)/3+ , whereas the true label’s score is at least a−εa- . Condition (44) makes the latter strictly larger. For PoE, any incorrect label has unnormalized score at most (b+ε)2(c+ε)(b+ )^2(c+ ), while the true label has score at least (a−ε)3(a- )^3. All factors are positive because ε<c<a <c<a. Condition (47) therefore suffices. It is also implied by (44), since a−ε>2(b+ε)+(c+ε)3≥((b+ε)2(c+ε))1/3.a- > 2(b+ )+(c+ )3≥ ((b+ )^2(c+ ) )^1/3. This proves the stated recovery rule for both aggregators. To bound the change in divergence, write F⁡(p,q)=JSD2(p,q)F(p,q)=JSD_2(p,q) and mε=c−ε>0m_ =c- >0. Along the straight line from any template pair (p0,q0)(p^0,q^0) to its perturbed pair (p,q)(p,q), every coordinate of both distributions belongs to [mε,1][m_ ,1]. Direct differentiation gives ∂F∂pr=12log22prpr+qr,∂F∂qr=12log22qrpr+qr. ∂ F∂ p_r= 12 _2 2p_rp_r+q_r, ∂ F∂ q_r= 12 _2 2q_rp_r+q_r. (48) On this line, mε≤2prpr+qr≤1mε,m_ ≤ 2p_rp_r+q_r≤ 1m_ , and the same inequalities hold after interchanging prp_r and qrq_r. Consequently, both gradient vectors have infinity norm at most 12log2⁡(1/mε) 12 _2(1/m_ ). Integrating the directional derivative along the line yields |F⁡(p,q)−F⁡(p0,q0)| |F(p,q)-F(p^0,q^0)| ≤12log21mε(‖p−p0‖1+‖q−q0‖1) ≤ 12 _2 1m_ (\|p-p^0\|_1+\|q-q^0\|_1 ) ≤2εlog21c−ε=η. ≤ 2 _2 1c- =η. (49) Averaging over the three pairs and then conditioning on H preserves this bound. If U denotes the normalized conditional divergence of the template distributions, we obtain |JH−dU|≤η|J_H-dU|≤η. The recovered cases are exactly the same occupancy patterns as for the templates, so (37) gives U≤G≤min⁡1,3U/2U≤ G≤ \1,3U/2\. Combining these inequalities with (JH−η)/d≤U≤(JH+η)/d(J_H-η)/d≤ U≤(J_H+η)/d proves (46). ∎ Every template satisfying the strict condition in Proposition C.5 admits a positive perturbation radius: any 0<ε<min⁡c,b−a2,12(a−2b+c3)0< < \! \c, b-a2, 12 (a- 2b+c3 ) \ satisfies the corollary. Hence recovery is stable on a neighborhood of member distributions, with an explicit error allowance for the observed JSD. C.5 Stability of Multi-Start Selection We analyze two finite profiling outputs for the same target task, candidate set, team size, and heterogeneity weights. Each output is standardized separately. Let P=(m2)P= m2, Lr=(r2)L_r= r2, and E(S)=i,j:i,j∈S,i<jE(S)=\\i,j\:i,j∈ S,\ i<j\. Edge vectors contain the P upper-triangular entries, each counted once; all edge norms below use this convention. For a vector x∈ℝdx ^d, define population standardization by Cd=Id−⊤d,σ⁡(x)=‖Cdx‖2d,Z⁡(x)=Cdx/σ⁡(x),σ⁡(x)>0,0,σ⁡(x)=0.C_d=I_d- 11 d, σ(x)= \|C_dx\|_2 d, Z(x)= casesC_dx/σ(x),&σ(x)>0,\\ 0,&σ(x)=0. cases (50) Thus ‖Z⁡(x)‖2=d\|Z(x)\|_2= d for every nonconstant vector. The corresponding quality and combined edge signals are ui=Z(qt)i,vij=λ1Z(1−R¯)ij+λ2Z(J¯)ij,u_i=Z(q^t)_i, v_ij= _1Z(1- R)_ij+ _2Z( J)_ij, (51) where matrix standardization acts on the upper-triangular vector. The quality statistics are computed over the m candidates, and each HI channel is standardized over its P base pairs. For |S|=r≥2|S|=r≥ 2, the implemented objective in its active normalization regime is exactly F⁡(S)=1r∑i∈Sui+1Lr∑e∈E⁡(S)ve,F⁡(i)=ui.F(S)= 1 r _i∈ Su_i+ 1 L_r _e∈ E(S)v_e, F(\i\)=u_i. (52) In particular, u always uses the target task’s quality estimates. Write the second profiling output as u′,v′u ,v , its score as F′F , and the changes as δu=u′−uδ u=u -u, δv=v′−vδ v=v -v, and δF(S)=F′(S)−F⁡(S)δ F(S)=F (S)-F(S). Normalization in the implementation. The default normalizer uses population standard deviation (std_ddof=0). It sets a term to zero when σ/nitems≤10−12σ/ n_ items≤ 10^-12. Hence (52) agrees with the numerical implementation when constant channels are zero and every nonconstant channel remains active at each relevant team size. Sufficient conditions are σ⁡(qt)>10−12kσ(q^t)>10^-12 k for a nonconstant quality channel and σ⁡(h)>10−12Lkσ(h)>10^-12 L_k for either nonconstant pooled HI channel, in each profiling output. A channel suppressed at every relevant size can instead be represented by zero in the effective profile. If a channel changes activation with team size, the same comparison argument below applies to the effective size-dependent vectors u(r),v(r)u^(r),v^(r), using their perturbations at size r=s+1r=s+1 for an extension and at size r=kr=k for the final comparison. Proposition C.7 (Stability of multi-start selection). Fix the candidates, 2≤k≤m2≤ k≤ m, weights, and tie-breaking, with each nonconstant standardization denominator above the numerical cutoff at every team size in both profiles. At a reference state of size s<ks<k, the change in the score difference between any two candidate extensions is at most Bs=2‖δu‖2+2‖δv‖2s+1.B_s= 2\|δ u\|_2+2\|δ v\|_2 s+1. (53) If every winning extension in every seed’s reference path leads its alternatives by more than BsB_s, and the final winner leads every other distinct terminal team by more than 2(‖δu‖2+‖δv‖2) 2(\|δ u\|_2+\|δ v\|_2), all paths and the returned team remain unchanged. Proof of Proposition C.7. Candidate comparisons. At a reference state S with |S|=s<k|S|=s<k, consider two remaining candidates j and ℓ . Define GS(j,ℓ)=F⁡(S∪j)−F⁡(S∪ℓ).G_S(j, )=F(S∪\j\)-F(S∪\ \). Both extended teams have size s+1s+1. Expanding (52) cancels the quality of the existing members and all edges within S, giving GS(j,ℓ)=uj−uℓs+1+∑i∈S(vij−viℓ)Ls+1.G_S(j, )= u_j-u_ s+1+ _i∈ S(v_ij-v_i ) L_s+1. Comparing marginal gains gives the same expression, since the current score F⁡(S)F(S) also cancels. Subtracting the corresponding expression for the two profiles yields the exact perturbation δGS(j,ℓ)=δuj−δuℓs+1+∑i∈S(δvij−δviℓ)Ls+1.δ G_S(j, )= δ u_j-δ u_ s+1+ _i∈ S(δ v_ij-δ v_i ) L_s+1. (54) The quality coefficient vector has two nonzero entries, each of magnitude 1/s+11/ s+1, and therefore norm 2/(s+1) 2/(s+1). The edge coefficient vector has 2s2s nonzero entries: the s edges connecting j to S and the s edges connecting ℓ to S. These edges are distinct because j,ℓ∉Sj, ∉ S. Its norm is 2sLs+1=2s+1. 2sL_s+1= 2 s+1. Cauchy–Schwarz and the triangle inequality therefore give |δGS(j,ℓ)|≤2‖δu‖2+2‖δv‖2s+1=Bs.|δ G_S(j, )|≤ 2\|δ u\|_2+2\|δ v\|_2 s+1=B_s. (55) Complete-team comparisons. For two size-k teams A,BA,B, subtraction similarly gives δ⁡(F⁡(A)−F⁡(B))= δ (F(A)-F(B) )= ∑i∈A∖Bδui−∑i∈B∖Aδuik _i∈ A Bδ u_i- _i∈ B Aδ u_i k (56) +∑e∈E⁡(A)∖E⁡(B)δve−∑e∈E⁡(B)∖E⁡(A)δveLk. + _e∈ E(A) E(B)δ v_e- _e∈ E(B) E(A)δ v_e L_k. Let r=|A∩B|r=|A∩ B|. The quality difference has 2(k−r)2(k-r) nonzero coefficients. Moreover, E⁡(A)∩E⁡(B)=E⁡(A∩B)E(A)∩ E(B)=E(A∩ B), so each team has Lk−LrL_k-L_r edges absent from the other. The same norm calculation gives the overlap-sensitive bound |δ⁡(F⁡(A)−F⁡(B))| |δ (F(A)-F(B) ) | ≤2−2rk‖δu‖2+2−2LrLk‖δv‖2 ≤ 2- 2rk\,\|δ u\|_2+ 2- 2L_rL_k\,\|δ v\|_2 (57) ≤2(‖δu‖2+‖δv‖2)=Bfin. ≤ 2 (\|δ u\|_2+\|δ v\|_2 )=B_ fin. Preservation of every seed run. Fix one seed. Both runs start with the same singleton. Suppose inductively that they have reached the same reference state S, and let jSj_S be the reference winner at this state. For every competing candidate ℓ , the assumed reference margin and (55) imply GS′(jS,ℓ)=GS(jS,ℓ)+δGS(jS,ℓ)≥GS(jS,ℓ)−Bs>0.G _S(j_S, )=G_S(j_S, )+δ G_S(j_S, )≥ G_S(j_S, )-B_s>0. The perturbed run therefore chooses jSj_S as well. Induction to size k proves that this seed reaches the same terminal team. Applying the argument to every seed preserves all terminal teams. Let C be the set of distinct terminal teams in the reference run, and let SgS_ g be its returned team. Repeated occurrences of the same team among different seeds do not create competing teams. For any T∈∖SgT \S_ g\, the assumed final margin and (57) give F′(Sg)−F′(T)≥F⁡(Sg)−F⁡(T)−Bfin>0.F (S_ g)-F (T)≥ F(S_ g)-F(T)-B_ fin>0. Thus the perturbed multi-start run returns SgS_ g. If there is only one distinct terminal team, this final condition is vacuous. The induction follows exactly the seed paths and terminal-team comparisons performed by Algorithm 3.3. ∎ A sharper certificate using signed comparisons. The norm bounds can be replaced by the exact projected perturbations. At a reference state S, write wS(j)=δujs+1+∑i∈SδvijLs+1,γS(ℓ)=GS(jS,ℓ).w_S(j)= δ u_j s+1+ _i∈ Sδ v_ij L_s+1, _S( )=G_S(j_S, ). The reference winner remains a strict winner at this state if and only if maxℓ∈ℳ∖(S∪jS)⁡wS(ℓ)−wS(jS)−γS(ℓ)<0. _ (S∪\j_S\) \w_S( )-w_S(j_S)- _S( ) \<0. (58) Indeed, the expression inside the braces is exactly F′(S∪ℓ)−F′(S∪jS)F (S∪\ \)-F (S∪\j_S\). The corresponding final condition is maxT∈∖Sg⁡δF(T)−δF(Sg)−[F⁡(Sg)−F⁡(T)]<0. _T \S_ g\ \δ F(T)-δ F(S_ g)-[F(S_ g)-F(T)] \<0. (59) An empty maximum is −∞-∞. These conditions retain cancellation between quality and HI perturbations and require only reference-winner comparisons. Applying them at every reference state, followed by (59), gives the same induction. Repeated reference states can share a single condition. Strict inequalities remove tie ambiguity; a non-strict comparison is sufficient when a retained tie is resolved in favor of the reference choice by the fixed priority rule. Both versions are deterministic statements about finite profiles, including estimates that share development examples or models. Their conditions preserve the selected team relative to the reference profile. Piecewise-constant selection as weights vary. Fix the profiling outputs and vary (λ1,λ2)∈ℝ≥02( _1, _2) _≥ 0^2. Every candidate comparison and complete-team comparison is affine in these two weights. There are finitely many possible states and teams, so the nontrivial comparison equalities form a finite collection of lines. Within each open cell of their arrangement, all comparison signs are fixed. Induction over the seed runs and the final comparison then shows that the returned team is constant throughout the cell. Identically tied comparisons are handled by the fixed priority rule. A team’s selection region may be a union of cells and need not be convex. This accounts for constant-selection regions in a weight sweep; one color in an accuracy heatmap can also represent several teams with the same accuracy. C.6 Pooling Geometry and Selection Perturbations For each source task τ∈τ , collect the upper-triangular HI entries as h1τ=(1−Rijτ)i<j,h2τ=(Jijτ)i<j,h¯ℓ=1||∑τ∈hℓτ,ℓ∈1,2.h_1^τ=(1-R_ij^τ)_i<j, h_2^τ=(J_ij^τ)_i<j, h_ = 1|T| _τ h_ ^τ, ∈\1,2\. Thus h¯1=1−R¯ h_1=1- R follows from averaging the task-level Yule-Q statistics in (8). In this notation the selector uses v=λ1Z(h¯1)+λ2Z(h¯2)v= _1Z( h_1)+ _2Z( h_2) while retaining u=Z⁡(qt)u=Z(q^t) from the target task. All identities below use population standard deviation as in (50). Their use in the numerical selector inherits the active-channel conditions above; a numerically suppressed channel contributes zero to its effective profile. The direction of the standardized pool. For one HI channel, abbreviate hτ=hℓτh^τ=h_ ^τ, and set στ=σ⁡(hτ),zτ=Z⁡(hτ),W=∑τ∈στzτ. _τ=σ(h^τ), z_τ=Z(h^τ), W= _τ _τz_τ. Constant task vectors have στ=0 _τ=0 and contribute zero to W. If W≠0W≠ 0, then Z⁡(h¯)=PW‖W‖2.Z( h)= P\, W\|W\|_2. (60) To prove the identity, center the average using CPC_P from (50): CPh¯=1||∑τ∈CPhτ=W||,σ⁡(h¯)=‖W‖2||P.C_P h= 1|T| _τ C_Ph^τ= W|T|, σ( h)= \|W\|_2|T| P. Their ratio gives (60). If W=0W=0, the pooled vector is constant and its standardized value is zero. Equal weighting of the raw task vectors therefore combines their standardized directions with weights proportional to their within-task dispersion across model pairs. Task-wide offsets vanish under centering, whereas opposing centered directions can cancel in W. Positive-affine invariance. Suppose, for each channel ℓ , the pooled and target-task vectors obey h¯ℓ=αℓhℓt+βℓ,αℓ>0. h_ = _ h_ ^t+ _ 1, _ >0. (61) Then Z⁡(h¯ℓ)=Z⁡(hℓt)Z( h_ )=Z(h_ ^t). For a nonconstant vector this follows because CPh¯ℓ=αℓCPhℓtC_P h_ = _ C_Ph_ ^t and σ⁡(h¯ℓ)=αℓσ(hℓt)σ( h_ )= _ σ(h_ ^t); if the target vector is constant, both standardized vectors are zero. Consequently, selection with pooled HI and selection with target-task HI have identical scores at every team size when both use the same qtq^t. Their greedy paths and returned teams are identical, including ties resolved by the same rule. This identity applies to the numerical implementation when the scaling preserves the active or zero status of each channel at every relevant team size. A sufficient source-level condition for (61) is hℓτ=ατℓhℓ⋆+βτℓh_ ^τ= _τ h_ + _τ 1 with ατℓ>0 _τ >0 for the source tasks and the target. Averaging preserves this common centered direction. The condition concerns each channel separately and imposes no sharing requirement on target qualities qtq^t. Different target tasks can therefore select different teams. Positive-affine agreement preserves standardized spacings as well as pair order; agreement of pair order alone does not preserve sums of edge values. Direction changes bound the effective edge perturbation. Consider any two nonconstant vectors h,h′∈ℝPh,h ^P, and let ρ⁡(h,h′)=Z(h)⊤Z(h′)P.ρ(h,h )= Z(h) Z(h )P. Their standardized norms are both P P, so expansion of the squared distance gives ‖Z⁡(h′)−Z⁡(h)‖22=2P(1−ρ⁡(h,h′)).\|Z(h )-Z(h)\|_2^2=2P (1-ρ(h,h ) ). (62) When the pooled vectors are nonconstant in both profiles for each channel, let dℓ=Z⁡(h¯ℓ′)−Z⁡(h¯ℓ)d_ =Z( h _ )-Z( h_ ) and ρℓ=ρ⁡(h¯ℓ,h¯ℓ′) _ =ρ( h_ , h _ ). Then δv=λ1d1+λ2d2,‖δv‖2≤2P∑ℓ=12λℓ1−ρℓ.δ v= _1d_1+ _2d_2, \|δ v\|_2≤ 2P _ =1^2 _ 1- _ . (63) The direct norm ‖λ1d1+λ2d2‖2\| _1d_1+ _2d_2\|_2 in the selection proposition retains any cancellation between channels. If a channel is constant, the exact difference dℓd_ remains defined by the zero convention, without requiring a correlation value. Changing only the HI pool leaves δu=0δ u=0; re-estimating the development profiles can also change the target quality vector, in which case its δuδ u term is retained. The pool-to-target alignment can also be expressed using task-to-task alignments. For one channel, let ρτν=zτ⊤zν/P _τν=z_τ z_ν/P for nonconstant task vectors. For a nonconstant target vector and W≠0W≠ 0, substitution into (60) gives ρ⁡(ht,h¯)=∑τστρtτ∑τ,νστσνρτν,ρ(h^t, h)= _τ _τ _tτ _τ,ν _τ _ν _τν, (64) where zero-dispersion source vectors are omitted from the sums. Indeed, zt⊤W=P∑τστρtτz_t W=P _τ _τ _tτ and ‖W‖22=P∑τ,νστσνρτν\|W\|_2^2=P _τ,ν _τ _ν _τν. Equations (62)–(64) connect task agreement to the perturbation that enters the actual selection comparisons. Entrywise variance reduction under resampling is a different quantity: after standardization, stability also depends on the pooled direction and the decision margins. C.7 From Selection Scores to Accuracy Gains We connect the accuracy comparison in Proposition 3.2 to the quality and HI terms in the implemented objective. The connection separates the curvature of the Q-to-error relation, the choice of objective weights, and the change from a target-task profile to a pooled development profile. All teams in this subsection have size three and follow (11). From Yule’s Q to a positive three-term score. Write ei=1−qie_i=1-q_i. For a pair with error probabilities x,zx,z and Yule’s coefficient r, let f⁡(x,z,r)f(x,z,r) denote its double-fault probability. On positive contingency tables it is the unique solution s of Φ⁡(s,x,z):=ln⁡s⁡(1−x−z+s)(x−s)(z−s)=ln⁡1+r1−r. (s,x,z):= s(1-x-z+s)(x-s)(z-s)= 1+r1-r. (65) The feasible interior is max⁡0,x+z−1<s<min⁡x,z \0,x+z-1\<s< \x,z\, with 0<x,z<10<x,z<1 and −1<r<1-1<r<1. The left-hand side increases from −∞-∞ to +∞+∞ on this interval. Define T=∂sΦ=1s+11−x−z+s+1x−s+1z−s>0.T= _s = 1s+ 11-x-z+s+ 1x-s+ 1z-s>0. Implicit differentiation gives ∂xf=(1−x−z+s)−1+(x−s)−1T∈(0,1),∂rf=2(1−r2)T>0, _xf= (1-x-z+s)^-1+(x-s)^-1T∈(0,1), _rf= 2(1-r^2)T>0, (66) and the expression for ∂zf _zf follows by symmetry. Choose an interior reference (e∗,e∗,r∗)(e_*,e_*,r_*) and put f∗=f⁡(e∗,e∗,r∗)f_*=f(e_*,e_*,r_*), ae=∂xf⁡(e∗,e∗,r∗)=∂zf⁡(e∗,e∗,r∗)a_e= _xf(e_*,e_*,r_*)= _zf(e_*,e_*,r_*), and ar=∂rf⁡(e∗,e∗,r∗)a_r= _rf(e_*,e_*,r_*). For each actual pair define its residual by the exact identity Dij=f∗+ae[(ei−e∗)+(ej−e∗)]+ar(Rijt−r∗)+ρij.D_ij=f_*+a_e[(e_i-e_*)+(e_j-e_*)]+a_r(R^t_ij-r_*)+ _ij. (67) This definition also applies to boundary tables using their actual DijD_ij. If the segments from the reference to (ei,ej,Rijt)(e_i,e_j,R^t_ij) lie in a compact interior region on which ‖∇2f‖op≤M\|∇^2f\|_ op≤ M, Taylor’s theorem gives the quantitative bound |ρij|≤M2[(ei−e∗)2+(ej−e∗)2+(Rijt−r∗)2].| _ij|≤ M2 [(e_i-e_*)^2+(e_j-e_*)^2+(R^t_ij-r_*)^2 ]. (68) Such a finite M exists on every compact region of positive contingency tables, by the implicit function theorem and (66). Let HS=HIerr(S,Rt)=1−13∑i,j⊂SRijtH_S=HI_err(S;R^t)=1- 13 _\i,j\⊂ SR^t_ij. Averaging (67) over a team, then substituting into (12), yields −KS=c0+Aqq¯S+AHHS+AJJS+ηS,-K_S=c_0+A_q q_S+A_HH_S+A_JJ_S+ _S, (69) where c0c_0 is independent of the team and Aq=θ+2(1−θ)ae>0,AH=(1−θ)ar>0,AJ=1/d>0,A_q=θ+2(1-θ)a_e>0, A_H=(1-θ)a_r>0, A_J=1/d>0, (70) with ηS=−1−θ3∑i,j⊂Sρij,|ηS|≤τS:=1−θ3∑i,j⊂S|ρij|. _S=- 1-θ3 _\i,j\⊂ S _ij, | _S|≤ _S:= 1-θ3 _\i,j\⊂ S| _ij|. To see the coefficient of quality explicitly, the average of (ei−e∗)+(ej−e∗)(e_i-e_*)+(e_j-e_*) over the three pairs is 2(eS−e∗)2(e_S-e_*). The average RijtR^t_ij is 1−HS1-H_S, and eS=1−q¯Se_S=1- q_S. These substitutions give (70). Thus the local directions of all three terms agree with the signs used in the selection objective, with a second-order remainder in an interior neighborhood. Normalization and fixed objective weights. Let FtF_t be the score computed using target-task qualities and target-task HI, with the base-item normalization in (9). For size three, it has the exact affine form Ft(S)=cF+wqq¯S+wHHS+wJJS,wq=3σq,wH=λ13σH,wJ=λ23σJ.F_t(S)=c_F+w_q q_S+w_HH_S+w_JJ_S, w_q= 3 _q, w_H= _1 3 _H, w_J= _2 3 _J. (71) Here each standard deviation is over its corresponding candidate-pool base items, and cFc_F collects their means. A constant channel has coefficient zero; nonconstant channels satisfy the active-normalization conditions in Section C.5. Choose any scale α>0α>0 and define =(Aq,AH,AJ),=(wq,wH,wJ),XS=(q¯S,HS,JS),=−α.A=(A_q,A_H,A_J), =(w_q,w_H,w_J), X_S=( q_S,H_S,J_S), =A- . For wq>0w_q>0, taking α=Aq/wqα=A_q/w_q matches the quality coefficient exactly. The scale is used only to compare the two mathematical quantities; the selector retains its original weights. For two teams define the one-sided coefficient correction Ewt(S,T)=[−⊤(XS−XT)]+,[z]+=max⁡z,0.E_ wt(S,T)= [-b (X_S-X_T) ]_+, [z]_+= \z,0\. (72) Subtracting (69) and using (71), together with ⊤(XS−XT)≥−Ewt(S,T)b (X_S-X_T)≥-E_ wt(S,T), gives KT−KS≥α⁡[Ft(S)−Ft(T)]−Ewt(S,T)−τS−τT.K_T-K_S≥α[F_t(S)-F_t(T)]-E_ wt(S,T)- _S- _T. (73) This form keeps the specified values of λ1,λ2 _1, _2 and measures their effect through the explicit coefficient differences b. Pooled development profiles and the returned team. Write F F for the actual score based on estimated target qualities and pooled HI. Let (ut,vt)(u_t,v_t) and (u^,v^)( u, v) be the corresponding standardized signals from (51), with Rt,JtR^t,J^t used for vtv_t. Put δu=u^−utδ u= u-u_t and δv=v^−vtδ v= v-v_t. For two teams with overlap o=|S∩T|o=|S∩ T|, (57) specializes to Bprof(S,T) B_ prof(S,T) :=2−2o3‖δu‖2+2−2(o2)3‖δv‖2, := 2- 2o3\|δ u\|_2+ 2- 2 o23\|δ v\|_2, (74) |[F^(S)−F^(T)]−[Ft(S)−Ft(T)]| |[ F(S)- F(T)]-[F_t(S)-F_t(T)] | ≤Bprof(S,T). ≤ B_ prof(S,T). Every normalization is recomputed on its own profile, so this expression includes changes in both base-item values and normalization statistics. Proposition C.8 (Selection-score condition for an accuracy gain). Under (11) and the active-normalization conditions above, let SgS_ g be the output of multi-start greedy selection on F F, and let SQS_Q be the Quality-Only team from the same candidate pool. For either Choice-Soft or PoE, Acc⁡(Sg)−Acc⁡(SQ)≥ (S_ g)-Acc(S_Q)≥ α⁡[F^(Sg)−F^(SQ)]−εQ α[ F(S_ g)- F(S_Q)]- _Q (75) −αBprof(Sg,SQ)−Ewt(Sg,SQ)−τSg−τSQ. -α B_ prof(S_ g,S_Q)-E_ wt(S_ g,S_Q)- _S_ g- _S_Q. In particular, a score advantage exceeding the displayed correction terms guarantees a strict accuracy improvement. Proof. The proof of Proposition 3.2 gives Acc⁡(Sg)−Acc⁡(SQ)≥KQ−KSg−εQAcc(S_ g)-Acc(S_Q)≥ K_Q-K_S_ g- _Q. Apply (73) with S=SgS=S_ g and T=SQT=S_Q. Then (74) implies Ft(Sg)−Ft(SQ)≥F^(Sg)−F^(SQ)−Bprof(Sg,SQ)F_t(S_ g)-F_t(S_Q)≥ F(S_ g)- F(S_Q)-B_ prof(S_ g,S_Q). Substitution proves (75). ∎ To make the role of multi-start search explicit, let C be its set of distinct terminal teams. Algorithm 3.3 satisfies F^(Sg)=maxT∈⁡F^(T) F(S_ g)= _T F(T). Set Emax=maxT∈⁡αBprof(T,SQ)+Ewt(T,SQ)+τT+τSQ.E_ = _T \α B_ prof(T,S_Q)+E_ wt(T,S_Q)+ _T+ _S_Q\. Therefore the sufficient condition α⁡[maxT∈⁡F^(T)−F^(SQ)]>εQ+Emaxα [ _T F(T)- F(S_Q) ]> _Q+E_ (76) ensures that the returned team outperforms Quality-Only. The maximization here is over the teams actually generated by the seed runs. Finally, the HI component of the profile change is exactly δv=λ1[Z⁡(1−R¯^)−Z⁡(1−Rt)]+λ2[Z⁡(J¯^)−Z⁡(Jt)].δ v= _1[Z(1- R)-Z(1-R^t)]+ _2[Z( J)-Z(J^t)]. With exact target qualities, δu=0δ u=0. Positive-affine agreement between each pooled channel and its target counterpart makes δv=0δ v=0 by (61). More generally, (62) and (63) bound its norm through the standardized profile directions. These terms connect target-task accuracy to the pooled signals used by the selector, while (72) and (68) account separately for the objective weights and the curvature of the correctness association. C.8 Member-Specific Information in Regularized Stacking The concatenated probability features used by Stacking contain both the team’s mean prediction and differences between its members. We analyze the additional reduction in regularized cross-entropy available from these member-specific differences after the mean prediction is already available to a linear softmax classifier. Setting and comparison class. Fix a team S of size k≥2k≥ 2 and relabel its members as 1,…,k1,…,k. Let c≥2c≥ 2 be the number of classes, and let pi(X)∈Δc−1p_i(X)∈ ^c-1 be each member’s aligned choice distribution. All expectations in this subsection are taken under one fixed distribution D over (X,Y)(X,Y). This may be a population distribution or the empirical distribution of a development set. Define the cross-entropy loss on logits using natural logarithms: ℓ⁡(z,y)=log⁡(∑a=1cexp⁡(za))−zy. (z,y)= \! ( _a=1^c (z_a) )-z_y. For β>0β>0, the optimal value of the Stacking objective is Fβ(S)=infW1,…,Wk,bDℓ(∑i=1kWi⊤pi(X)+b,Y)+β2∑i=1k‖Wi‖F2,F_β(S)= _W_1,…,W_k,b \E_D \! ( _i=1^kW_i p_i(X)+b,Y )+ β2 _i=1^k\|W_i\|_F^2 \, (77) where Wi∈ℝc×cW_i ^c× c are the blocks of the weight matrix in Equations 101 and 102, and b∈ℝcb ^c is unpenalized. For an empirical D, this is exactly the average cross-entropy plus the L2L_2 penalty in Equation 102. In the implementation, the data-fit gradient is averaged over the development examples and the penalty contributes W l2\,W to the weight gradient, so the coefficient here is β=β= l2. Let p¯(X)=k−1∑i=1kpi(X) p(X)=k^-1 _i=1^kp_i(X). The comparison class uses only this mean probability vector as the input to a linear softmax classifier: Gβ/k(p¯)=minM,b⁡Dℓ(M⊤p¯(X)+b,Y)+β2k‖M‖F2.G_β/k( p)= _M,b \E_D (M p(X)+b,Y)+ β2k\|M\|_F^2 \. (78) This is a restricted class within Stacking: setting Wi=M/kW_i=M/k reproduces its logits and penalty. Thus the factor β/kβ/k follows from the original regularizer. The comparison class is used only for analysis; its input equals the Choice-Soft mean, followed by a learned linear softmax map. Assume that Equation 78 has a finite minimizer (M0,b0)(M_0,b_0). For example, positive probability for every class is sufficient when β>0β>0, after fixing ⊤b=01 b=0 to remove the irrelevant common shift in the logits. Define the mean classifier, the concatenated member differences, and their residual alignment by π0(X) _0(X) =softmax⁡(M0⊤p¯(X)+b0), =softmax(M_0 p(X)+b_0), (79) δϕ(X) δφ(X) =[p1(X)−p¯(X);…;pk(X)−p¯(X)], =[p_1(X)- p(X);…;p_k(X)- p(X)], Γ =D[δϕ(X)(eY−π0(X))⊤], =E_D\! [δφ(X)(e_Y- _0(X)) ], where eYe_Y is the one-hot vector of the true class. Also set Σδ=D[δϕ(X)δϕ(X)⊤],Lδ=β+12λmax(Σδ). _δ=E_D[δφ(X)δφ(X) ], L_δ=β+ 12 _ ( _δ). (80) All these moments are finite because the probability features are bounded. To use the same distribution throughout the analysis, write HD(S)=HIdist(S,JD),JijD=D[JSD2⁡(pi(X),pj(X))],H_D(S)=HI_dist(S;J^D), J^D_ij=E_D\! [JSD_2(p_i(X),p_j(X)) ], where JSD2JSD_2 uses logarithms to base 22 and HDH_D follows the pairwise averaging rule in Equation 7. Theorem C.9 (Additional information available to regularized Stacking). Under the setting above, let ΔS=Gβ/k(p¯)−Fβ(S) _S=G_β/k( p)-F_β(S). Then ‖Γ‖F22Lδ≤ΔS≤‖Γ‖F22β≤2ln⁡2(k−1)βHD(S). \| \|_F^22L_δ\;≤\; _S\;≤\; \| \|_F^22β\;≤\; 2 2\,(k-1)β\,H_D(S). (81) Consequently, ΔS>0 _S>0 if and only if Γ≠0 ≠ 0. Proof. Step 1: separate the mean and member-specific weights. For arbitrary Stacking weights, define M=∑i=1kWi,Ui=Wi−Mk,U=[U1;…;Uk].M= _i=1^kW_i, U_i=W_i- Mk, U=[U_1;…;U_k]. Then ∑iUi=0 _iU_i=0, and every collection of weights admits this decomposition. Conversely, any M and U satisfying ∑iUi=0 _iU_i=0 define weights Wi=M/k+UiW_i=M/k+U_i. The logits and penalty obey ∑iWi⊤pi+b _iW_i p_i+b =M⊤p¯+U⊤δϕ+b, =M p+U δφ+b, (82) ∑i‖Wi‖F2 _i\|W_i\|_F^2 =‖M‖F2k+‖U‖F2. = \|M\|_F^2k+\|U\|_F^2. (83) The first identity uses ∑iUi=0 _iU_i=0; the cross terms in the second identity vanish for the same reason. Therefore, with Ψ⁡(M,b,U)=Dℓ(M⊤p¯+U⊤δϕ+b,Y)+β2k‖M‖F2+β2‖U‖F2, (M,b,U)=E_D (M p+U δφ+b,Y)+ β2k\|M\|_F^2+ β2\|U\|_F^2, we have Fβ(S)=infM,b,U:∑iUi=0Ψ(M,b,U)F_β(S)= _M,b,U:\, _iU_i=0 (M,b,U) and Gβ/k(p¯)=Ψ⁡(M0,b0,0)G_β/k( p)= (M_0,b_0,0). In particular, ΔS≥0 _S≥ 0. Step 2: bound the improvement using the residual gradient. The finite optimum of the mean classifier satisfies D[p¯(π0−eY)⊤]+βkM0=0,D[π0−eY]=0.E_D[ p( _0-e_Y) ]+ βkM_0=0, _D[ _0-e_Y]=0. (84) If ⊤b=01 b=0 is imposed, stationarity initially gives the second equation on that subspace. The bias gradient always has coordinate sum zero, so this also makes its full gradient zero. Differentiation under the expectation is justified by bounded probability features and bounded first derivatives of cross-entropy with respect to logits. Apply convexity to the cross-entropy term at (M0,b0,0)(M_0,b_0,0), and expand the quadratic penalties. The first-order terms in M−M0M-M_0 and b−b0b-b_0 cancel by Equation 84, leaving Ψ⁡(M,b,U)≥Gβ/k(p¯)−⟨Γ,U⟩F+β2k‖M−M0‖F2+β2‖U‖F2. (M,b,U)≥ G_β/k( p)- ,U _F+ β2k\|M-M_0\|_F^2+ β2\|U\|_F^2. (85) Here ⟨A,B⟩F=tr⁡(A⊤B) A,B _F=tr(A B). Completing the square in U gives −⟨Γ,U⟩F+β2‖U‖F2=β2‖U−Γβ‖F2−‖Γ‖F22β.- ,U _F+ β2\|U\|_F^2= β2 \|U- β \|_F^2- \| \|_F^22β. Taking the infimum in Equation 85 yields Fβ(S)≥Gβ/k(p¯)−‖Γ‖F2/(2β)F_β(S)≥ G_β/k( p)-\| \|_F^2/(2β), proving the upper bound on ΔS _S. This argument uses the quadratic penalty on the weights and the bias stationarity condition; it requires no strong convexity in the unpenalized bias. Step 3: exhibit a feasible improvement. For a probability vector π, the Hessian of cross-entropy with respect to logits is B⁡(π)=diag⁡(π)−ππ⊤B(π)=diag(π)-π . For every v∈ℝcv ^c, v⊤B(π)v v B(π)v =∑a<bπaπb(va−vb)2 = _a<b _a _b(v_a-v_b)^2 ≤2∑aπa(1−πa)va2≤12‖v‖22. ≤ 2 _a _a(1- _a)v_a^2≤ 12\|v\|_2^2. The last inequality uses πa(1−πa)≤1/4 _a(1- _a)≤ 1/4, so 0⪯B⁡(π)⪯I/20 B(π) I/2. Fix (M,b)=(M0,b0)(M,b)=(M_0,b_0) and write f⁡(U)=Ψ⁡(M0,b0,U)f(U)= (M_0,b_0,U). For any matrix direction V, its second directional derivative obeys D2f(U)[V,V] D^2f(U)[V,V] ≤β‖V‖F2+12D‖V⊤δϕ‖22 ≤β\|V\|_F^2+ 12E_D\|V δφ\|_2^2 =β‖V‖F2+12tr⁡(V⊤ΣδV)≤Lδ‖V‖F2. =β\|V\|_F^2+ 12tr(V _δV)≤ L_δ\|V\|_F^2. Since ∇f(0)=−Γ∇ f(0)=- , this global curvature bound implies f⁡(U)≤Gβ/k(p¯)−⟨Γ,U⟩F+Lδ2‖U‖F2.f(U)≤ G_β/k( p)- ,U _F+ L_δ2\|U\|_F^2. Partition Γ into blocks Γi∈ℝc×c _i ^c× c. Their sum is zero: ∑iΓi=D[(∑i(pi−p¯))(eY−π0)⊤]=0. _i _i=E_D\! [ ( _i(p_i- p) )(e_Y- _0) ]=0. Thus U=Γ/LδU= /L_δ satisfies the constraint ∑iUi=0 _iU_i=0 and gives Fβ(S)≤f⁡(Γ/Lδ)≤Gβ/k(p¯)−‖Γ‖F22Lδ.F_β(S)≤ f( /L_δ)≤ G_β/k( p)- \| \|_F^22L_δ. This proves the lower bound on ΔS _S. Step 4: relate the residual gradient to pairwise JSD. First, ‖eY−π0‖22≤2\|e_Y- _0\|_2^2≤ 2. The triangle inequality and Cauchy–Schwarz therefore give ‖Γ‖F2≤D‖δϕ‖22D‖eY−π0‖22≤2D‖δϕ‖22.\| \|_F^2 _D\|δφ\|_2^2\,E_D\|e_Y- _0\|_2^2≤ 2E_D\|δφ\|_2^2. (86) For completeness, the constants connecting this energy to JSD follow directly from Pinsker’s inequality. For probability vectors p,qp,q, put m=(p+q)/2m=(p+q)/2 and use natural logarithms for KLKL and JSDnatJSD_nat. Then JSDnat⁡(p,q) _nat(p,q) =12KL(p∥m)+12KL(q∥m) = 12KL(p\|m)+ 12KL(q\|m) ≥14‖p−m‖12+14‖q−m‖12=18‖p−q‖12. ≥ 14\|p-m\|_1^2+ 14\|q-m\|_1^2= 18\|p-q\|_1^2. Because p−qp-q has coordinate sum zero, its positive and negative parts have equal total mass a. Hence ‖p−q‖22≤2a2=‖p−q‖12/2\|p-q\|_2^2≤ 2a^2=\|p-q\|_1^2/2. Converting to bits yields JSD2⁡(p,q)=JSDnat⁡(p,q)ln⁡2≥‖p−q‖224ln⁡2.JSD_2(p,q)= JSD_nat(p,q) 2≥ \|p-q\|_2^24 2. (87) Finally, expanding the squared norms around the mean gives D‖δϕ‖22 _D\|δφ\|_2^2 =D∑i‖pi−p¯‖22=1k∑i<jD‖pi−pj‖22 =E_D _i\|p_i- p\|_2^2= 1k _i<jE_D\|p_i-p_j\|_2^2 ≤4ln⁡2k∑i<jJijD=2ln⁡2(k−1)HD(S). ≤ 4 2k _i<jJ^D_ij=2 2\,(k-1)H_D(S). Together with Equation 86, this gives ‖Γ‖F2≤4ln⁡2(k−1)HD(S)\| \|_F^2≤ 4 2\,(k-1)H_D(S) and proves the final inequality in Equation 81. If Γ=0 =0, the upper bound and ΔS≥0 _S≥ 0 give ΔS=0 _S=0. If Γ≠0 ≠ 0, the lower bound is strictly positive because 0<β≤Lδ<∞0<β≤ L_δ<∞. ∎ Interpretation. Pairwise JSD quantifies the size of the probability differences available to Stacking. The matrix Γ identifies whether those differences align with the predictive residual left by the mean classifier. The theorem concerns the additional value of member-specific features beyond that classifier, measured by the optimal regularized cross-entropy under D. Improvements already present in the mean probability vector are part of the comparison class. The theorem complements the soft-aggregation analysis by characterizing the gain from retaining member identity in a learned linear combiner. Appendix D Implementation Details D.1 Choice-Space Aligned Scoring Implementation For each example x with label space t=y1,…,y|t|Y_t=\y_1,…,y_|Y_t|\, we: 1. Force the model to generate each choice token yky_k (e.g., “A”, “B”, “C”, “D”) 2. Extract the log-probability ℓi(yk|x) _i(y_k|x) from the model’s output distribution 3. Normalize across tY_t: pi(yk|x)=exp⁡(ℓi(yk|x))∑y′∈texp⁡(ℓi(y′|x))p_i(y_k|x)= ( _i(y_k|x)) _y _t ( _i(y |x)) (88) This ensures ∑y∈tpi(y|x)=1 _y _tp_i(y|x)=1 with negligible OTHER mass. Prompt Format. We use a chat-based prompt format with two messages: Profiling Prompt Template User Message: ⬇ [Question] Options: [Option 1] [Option 2] ... Complete the sentence ’The correct answer is ’ by outputting exactly one letter from: A, B, C, D. Output nothing else. Assistant Message (prefix): (Note: there is a trailing space after “is”.) ⬇ The correct answer is Why normalize within the choice space. Outside-choice token mass can introduce divergence unrelated to preferences among the actual answer options. The following entropy identity (Lin, 1991) isolates this effect. Lemma D.1 (Divergence due to outside-choice mass). Let Y be the evaluation label space (e.g., A,B,C,D\A,B,C,D\) and ′=∪OTHERY =Y∪\OTHER\ be an extended space. Suppose two agents have distributions: pi p_i =(1−αi)q+αiδOTHER =(1- _i)q+ _i _OTHER (89) pj p_j =(1−αj)q+αjδOTHER =(1- _j)q+ _j _OTHER (90) where q is a distribution over Y (identical for both agents), δOTHER _OTHER is a point mass on OTHER, and αi,αj∈[0,1] _i, _j∈[0,1]. Then: JSD⁡(pi,pj)=JSD⁡([αi,1−αi],[αj,1−αj])JSD(p_i,p_j)=JSD ([ _i,1- _i],[ _j,1- _j] ) (91) Thus all divergence arises from the difference in OTHER mass, although their common choice distribution is q (the conditional distribution whenever αi,αj<1 _i, _j<1). Proof. Let α¯=(αi+αj)/2 α=( _i+ _j)/2, and let H denote Shannon entropy in bits. Since q and δOTHER _OTHER have disjoint supports, H⁡(pi) H(p_i) =H⁡(αi)+(1−αi)H(q), =H( _i)+(1- _i)H(q), (92) H⁡(pj) H(p_j) =H⁡(αj)+(1−αj)H(q), =H( _j)+(1- _j)H(q), (93) H⁡(pi+pj2) H\! ( p_i+p_j2 ) =H⁡(α¯)+(1−α¯)H(q). =H( α)+(1- α)H(q). (94) Substituting these identities into the entropy expression for JSD cancels the H⁡(q)H(q) terms: JSD⁡(pi,pj)=H⁡(αi+αj2)−H⁡(αi)+H⁡(αj)2=JSD⁡([αi,1−αi],[αj,1−αj]).JSD(p_i,p_j)=H ( _i+ _j2 )- H( _i)+H( _j)2=JSD ([ _i,1- _i],[ _j,1- _j] ). (95) where H⁡(p)=−plog2p−(1−p)log2⁡(1−p)H(p)=-p _2p-(1-p) _2(1-p) is the binary entropy. For αi=0.1 _i=0.1 and αj=0.9 _j=0.9, the divergence is 1−H⁡(0.1)≈0.5311-H(0.1)≈ 0.531 bits, entirely due to the difference in OTHER mass. ∎ Choice-space normalization removes this outside-mass contribution before computing JtJ^t. D.2 Paired Bootstrap Procedure For comparing two methods A and B on a test set of N examples: 1. Compute per-example correctness: cA(xn),cB(xn)∈0,1c_A(x_n),c_B(x_n)∈\0,1\ for n=1,…,Nn=1,…,N 2. For b=1,…,Bb=1,…,B (we use B=2000B=2000): (a) Sample indices i1,…,iN\i_1,…,i_N\ uniformly with replacement from 1,…,N\1,…,N\ (b) Compute bootstrap accuracies: AccA(b) _A^(b) =1N∑n=1NcA(xin) = 1N _n=1^Nc_A(x_i_n) (96) AccB(b) _B^(b) =1N∑n=1NcB(xin) = 1N _n=1^Nc_B(x_i_n) (97) (c) Compute bootstrap difference: Δ(b)=AccA(b)−AccB(b) ^(b)=Acc_A^(b)-Acc_B^(b) 3. Compute 95% confidence interval: [percentile(Δ(b),2.5),percentile(Δ(b),97.5)][percentile(\ ^(b)\,2.5),percentile(\ ^(b)\,97.5)] 4. Mark as significant if the CI excludes 0 The paired structure (using the same bootstrap sample for both methods) preserves the correlation between cAc_A and cBc_B, providing more statistical power than unpaired bootstrap. D.3 Hyperparameter Selection We maintain strict development–test separation when selecting the objective weights. We first perform a grid search using only development splits to identify a stable high-performing region for the two heterogeneity weights, and then fix (λ1,λ2)=(0.13,0.05)( _1, _2)=(0.13,0.05) for all datasets and aggregation rules. No test-set performance is used for per-task hyperparameter tuning. To examine whether this choice reflects a task-specific artifact, we further conduct leave-one-task-out hyperparameter selection over the seven primary benchmarks. For each held-out benchmark, the weight pair is selected using the development splits of the remaining six benchmarks and then applied to the held-out benchmark. Across the held-out tasks, the grid contains broad high-performing plateaus rather than isolated optima: multiple neighboring weight pairs achieve the same development-set performance. The fixed weight pair used in our main experiments lies within this high-performing region for 6 out of 7 held-out tasks. This indicates that the selected weights capture a stable quality–heterogeneity trade-off rather than a task-specific tuning artifact. D.4 Computational Cost of Team Selection With m candidates and target size k, Algorithm 3.3 tries m seeds and at most k−1k-1 additions per seed. If within-team quality and pairwise sums are cached, evaluating an extension from a size-s team needs s pairwise lookups plus constant-time quality and normalization operations. Evaluating all remaining candidates therefore costs O⁡(ms)O(ms) per round and O⁡(m∑s=1k−1s)=O⁡(mk2)O(m _s=1^k-1s)=O(mk^2) per seed. The total cost is O⁡(m2k2)O(m^2k^2), with O⁡(m2)O(m^2) storage for the pairwise matrices. These are operation counts for cached score evaluation. At the experimental size m=11m=11, k=3k=3, exhaustive enumeration is also feasible; multi-start search extends the same scoring rule to larger pools. D.5 Aggregation Rules Choice-Soft. Choice-Soft aggregates by averaging the members’ aligned choice distributions. For a team StS_t and instance x, we compute the mean distribution as follows: pcst(y∣x)=1|St|∑i∈Stpit(y∣x).p_ cs^t(y x)= 1|S_t| _i∈ S_tp_i^t(y x). (98) We then predict y^St(x)=arg⁡maxy∈tpcst(y∣x) y_S_t(x)= _y _tp_ cs^t(y x). PoE. PoE (Product of Experts) combines distributions multiplicatively, rewarding labels that receive consistently high probability across members. We form the aggregated distribution as: ppoet(y∣x)=∏i∈Stpit(y∣x)∑y′∈t∏i∈Stpit(y′∣x),p_ poe^t(y x)= _i∈ S_tp_i^t(y x) _y _t _i∈ S_tp_i^t(y x), (99) and predict y^St(x)=arg⁡maxy∈tppoet(y∣x) y_S_t(x)= _y _tp_ poe^t(y x). DS. DS (Dawid–Skene) models each member as a noisy labeler with an agent-specific confusion matrix estimated on DtdevD_t^dev. Let y^i(x)=arg⁡maxy∈tpit(y∣x) y_i(x)= _y _tp_i^t(y x) denote the label predicted by model i. We estimate a prior πt(y)π^t(y) and, for each model i, a confusion matrix Ait∈ℝ|t|×|t|A_i^t ^|Y_t|×|Y_t| via additive smoothing (with a small constant ϵε), where Ait(a,b)A_i^t(a,b) estimates ℙ⁡(y^i=b∣y=a)P( y_i=b y=a). At inference time, we compute the posterior up to proportionality: pdst(y∣x)∝πt(y)∏i∈StAit(y,y^i(x)).p_ ds^t(y x)\; \;π^t(y) _i∈ S_tA_i^t\! (y,\, y_i(x) ). (100) We then renormalize over tY_t and predict y^St(x)=arg⁡maxy∈tpdst(y∣x) y_S_t(x)= _y _tp_ ds^t(y x). Stacking. Stacking trains a lightweight combiner on DtdevD_t^dev. For a team StS_t and instance x, we concatenate members’ choice distributions: ϕ(x)=[pi1t(⋅∣x);…;pikt(⋅∣x)]∈ℝk|t|.φ(x)= [p_i_1^t(· x);…;p_i_k^t(· x) ] ^k|Y_t|. (101) We learn a linear classifier parametrized by W∈ℝk|t|×|t|W ^k|Y_t|×|Y_t| and b∈ℝ|t|b ^|Y_t| via L2L_2-regularized cross-entropy. Let πW,b(y∣x)=softmax(W⊤ϕ(x)+b)y _W,b(y x)=softmax(W φ(x)+b)_y. We train the stacking combiner by minimizing: minW,b⁡ℒ⁡(W,b)= _W,b\;L(W,b)= 1|Dtdev|∑(x,y)∈Dtdev−logπW,b(y∣x) 1|D_t^dev| _(x,y)∈ D_t^dev- _W,b(y x) (102) +β2‖W‖F2. + β2\|W\|_F^2. At test time, we predict y^St(x)=arg⁡maxy∈tπW,b(y∣x) y_S_t(x)= _y _t _W,b(y x). Appendix E Supplementary Results This section provides additional results that complement the main experiments. We first report the complete benchmark tables omitted from the main text for space, then analyze the team compositions selected by C2-MAS, and finally provide robustness and ablation analyses for team size and objective validity. E.1 Complete Benchmark Results Section E.1 reports the full per-benchmark accuracy across all 11 open-weight candidate models, the Self-Consistency baseline, and C2-MAS under four aggregation rules, separated into (a) 7 primary benchmarks and (b) 6 OOD benchmarks. Table E.1: Complete per-benchmark results across all 13 benchmarks. We report the full open-weight candidate pool (m=11m=11), the single-model Self-Consistency baseline, and C2-MAS under four aggregation rules. (a) covers the seven primary benchmarks used in Table 1; (b) covers the six OOD benchmarks, where heterogeneity matrices are pooled only on the primary set and applied to OOD teams (dev→ ). Avg. reports the average accuracy within each panel. Abbreviations: ARC-C: ARC-Challenge; CSQA: CommonsenseQA; OBQA: OpenBookQA. (a) Primary benchmarks (7). Method Dataset Avg. ARC-C CSQA LogiQA2 MedQA MMLU MMLU-Pro OBQA Open-Weight Individual Models Llama-3.1-8B 80.1580.15 75.2875.28 54.0354.03 61.9861.98 67.7067.70 37.9237.92 81.7481.74 65.5465.54 Qwen3-8B 89.6189.61 79.7879.78 67.2367.23 61.0561.05 71.7271.72 45.7945.79 83.2483.24 71.2071.20 Gemma-2-9B 88.4888.48 78.5678.56 63.2063.20 59.7459.74 74.4474.44 44.0144.01 85.4985.49 70.5670.56 Ministral-3-8B 79.4079.40 62.0862.08 59.9359.93 55.5255.52 67.8867.88 41.2041.20 68.4568.45 62.0762.07 Granite-3.3-8B 80.9080.90 72.6672.66 53.0053.00 51.2251.22 65.4565.45 35.0235.02 80.5280.52 62.6862.68 GLM-4-9B 85.8685.86 74.4474.44 56.5556.55 55.8155.81 64.6164.61 31.8431.84 84.9384.93 64.8664.86 Nemotron-Nano-9B 88.9588.95 74.7274.72 65.0765.07 60.2160.21 72.9472.94 45.8845.88 88.9588.95 70.9670.96 EuroLLM-9B 75.9475.94 72.8572.85 45.8845.88 50.0050.00 59.9359.93 26.9726.97 73.8873.88 57.9257.92 OLMo-3-7B 72.4772.47 71.5471.54 42.8842.88 41.4841.48 56.1856.18 32.8732.87 66.5766.57 54.8654.86 Apertus-8B 72.0072.00 65.5465.54 46.8246.82 48.2248.22 55.7155.71 29.6829.68 68.6368.63 55.2355.23 InternLM3-8B 88.2088.20 75.9475.94 67.1367.13 59.6459.64 71.1671.16 42.7042.70 84.3684.36 69.8869.88 Single-Model Aggregation Self-Consistency 89.1489.14 78.6578.65 67.3267.32 61.7061.70 73.4173.41 44.7644.76 88.5888.58 71.9471.94 C2-MAS Team Selection C2-MAS (Choice-Soft) 91.4891.48 83.9083.90 70.8870.88 66.3966.39 76.5976.59 48.5048.50 90.4590.45 75.4575.45 C2-MAS (PoE) 91.5791.57 83.9083.90 70.6970.69 65.3665.36 76.6976.69 48.9748.97 89.5189.51 75.2475.24 C2-MAS (DS) 90.9290.92 82.8782.87 70.3270.32 65.1765.17 75.4775.47 47.1047.10 90.4590.45 74.6174.61 C2-MAS (Stacking) 91.6791.67 84.1884.18 70.6070.60 65.5465.54 76.1276.12 48.9748.97 91.5791.57 75.5275.52 Table E.1 (continued). (b) OOD benchmarks (6). Method Dataset Avg. AQUA-RAT C-Eval MathQA RACE ReClor StrategyQA Open-Weight Individual Models Llama-3.1-8B 33.9933.99 49.5349.53 33.2433.24 60.7760.77 62.4562.45 67.3267.32 51.2251.22 Qwen3-8B 42.6042.60 75.0975.09 40.0740.07 63.1163.11 81.4681.46 67.8867.88 61.7061.70 Gemma-2-9B 30.9030.90 55.3455.34 32.2132.21 64.5164.51 77.0677.06 70.1370.13 55.0355.03 Ministral-3-8B 31.3731.37 58.8058.80 27.1527.15 58.8058.80 67.7967.79 58.5258.52 50.4150.41 Granite-3.3-8B 32.4032.40 45.5145.51 31.0931.09 60.9660.96 66.4866.48 64.8964.89 50.2250.22 GLM-4-9B 30.8130.81 65.7365.73 30.4330.43 60.3960.39 61.8961.89 67.4267.42 52.7852.78 Nemotron-Nano-9B 45.3245.32 54.2154.21 40.7340.73 64.1464.14 82.1282.12 63.2063.20 58.2958.29 EuroLLM-9B 27.8127.81 46.5446.54 24.9124.91 56.9356.93 56.4656.46 64.3364.33 46.1646.16 OLMo-3-7B 33.6133.61 37.9237.92 30.8130.81 58.7158.71 55.2455.24 63.6763.67 46.6646.66 Apertus-8B 27.0627.06 47.1947.19 28.4628.46 53.7553.75 50.0950.09 61.7061.70 44.7144.71 InternLM3-8B 32.1232.12 84.0884.08 29.3129.31 63.7663.76 72.4772.47 62.5562.55 57.3857.38 Single-Model Aggregation Self-Consistency 45.5145.51 81.4681.46 39.2339.23 67.6067.60 81.6581.65 71.2571.25 64.4564.45 C2-MAS Team Selection C2-MAS (Choice-Soft) 45.6945.69 80.2480.24 41.9541.95 67.9867.98 84.1884.18 69.7669.76 64.9764.97 C2-MAS (PoE) 45.2245.22 78.9378.93 41.5741.57 67.7967.79 84.5584.55 69.6669.66 64.6264.62 C2-MAS (DS) 44.1944.19 80.5280.52 42.0442.04 68.3568.35 83.9083.90 70.7970.79 64.9764.97 C2-MAS (Stacking) 46.7246.72 85.9685.96 43.9143.91 68.3568.35 84.8384.83 70.6970.69 66.7466.74 Table 5: OOD test accuracy (%) on the remaining 6 benchmarks. For C2-MAS, we pool heterogeneity matrices using only the seven primary benchmarks in Table 1 and apply them to select teams on these 6 benchmarks (dev→ ). ↑ indicates gain over Random-k (p). Bold marks the best within each aggregation block. Method Dataset Avg. AQUA-RAT C-Eval MathQA RACE ReClor StrategyQA Closed-Source Models (for reference) Gemini-2.5-Flash 26.1226.12 73.4173.41 30.1530.15 63.8663.86 79.2179.21 62.7362.73 55.9155.91 GPT-4o 29.9629.96 72.4772.47 29.4929.49 65.0765.07 84.3684.36 80.0680.06 60.2460.24 Single-Model Aggregation Self-Consistency 45.5145.51 81.4681.46 39.2339.23 67.6067.60 81.6581.65 71.2571.25 64.4564.45 Team Selection Baselines Choice-Soft Random-k 39.1039.10 65.7165.71 36.8336.83 66.3166.31 75.5575.55 66.6366.63 58.3658.36 Caruana 47.0047.00 84.5584.55 40.9240.92 67.2367.23 83.7183.71 70.1370.13 65.5965.59 Quality-Only 45.3245.32 80.2480.24 42.8842.88 67.9867.98 82.9682.96 69.7669.76 64.8664.86 C2-MAS (Ours) 45.6945.69↑ 6.59 80.2480.24↑ 14.53 41.9541.95↑ 5.12 67.9867.98↑ 1.67 84.1884.18↑ 8.63 69.7669.76↑ 3.13 64.9764.97↑ 6.61 PoE Random-k 39.2439.24 66.6366.63 36.7036.70 66.4766.47 76.3876.38 66.7666.76 58.7058.70 Caruana 47.0047.00 83.2483.24 41.4841.48 68.0768.07 84.6484.64 69.9469.94 65.7365.73 Quality-Only 46.7246.72 78.9378.93 42.0442.04 67.7967.79 84.4684.46 69.6669.66 64.9364.93 C2-MAS (Ours) 45.2245.22↑ 5.98 78.9378.93↑ 12.30 41.5741.57↑ 4.87 67.7967.79↑ 1.32 84.5584.55↑ 8.17 69.6669.66↑ 2.90 64.6264.62↑ 5.92 DS Random-k 38.7138.71 68.3868.38 36.3336.33 66.6466.64 75.4775.47 68.2968.29 58.9758.97 Caruana 44.7644.76 84.0884.08 39.7939.79 67.2367.23 83.1583.15 70.7970.79 64.9764.97 Quality-Only 44.9444.94 80.5280.52 40.3640.36 68.3568.35 82.8782.87 70.7970.79 64.6464.64 C2-MAS (Ours) 44.1944.19↑ 5.48 80.5280.52↑ 12.14 42.0442.04↑ 5.71 68.3568.35↑ 1.71 83.9083.90↑ 8.43 70.7970.79↑ 2.50 64.9764.97↑ 6.00 Stacking Random-k 41.0441.04 71.9071.90 38.4338.43 67.2167.21 78.8178.81 69.3569.35 61.1261.12 Caruana 46.3546.35 84.9384.93 43.2643.26 68.4568.45 83.8083.80 70.6970.69 66.2566.25 Quality-Only 46.5446.54 85.9685.96 43.6343.63 68.3568.35 83.4383.43 70.6970.69 66.4366.43 C2-MAS (Ours) 46.7246.72↑ 5.68 85.9685.96↑ 14.06 43.9143.91↑ 5.48 68.3568.35↑ 1.14 84.8384.83↑ 6.02 70.6970.69↑ 1.34 66.7466.74↑ 5.62 E.2 Team Composition Analysis To further examine how heterogeneity-aware selection changes the selected team, we report in Table 6 the benchmarks where C2-MAS selects a different team from Quality-Only under Stacking aggregation. On the seven primary benchmarks, the two methods select identical teams on four benchmarks, with a mean Jaccard similarity of 0.790.79. The three differing teams share the same pattern: C2-MAS keeps the strongest shared anchor (Nemotron, Gemma-2, or Qwen3) and substitutes one model from the central high-accuracy cluster with a peripheral one (GLM-4, Ministral-3). This pattern aligns with the heterogeneity landscape in Figure 4, where peripheral models provide complementary error coverage despite slightly lower individual accuracy. Table 6: Team composition changes made by C2-MAS relative to Quality-Only under Stacking aggregation. We report only benchmarks where the selected teams differ. Δ Acc. denotes the accuracy difference (p) between C2-MAS and Quality-Only; arrows indicate gains or losses. Benchmark Quality-Only Team C2-MAS Team Δ Acc. CommonsenseQA Gemma-2 Qwen3 InternLM3 Gemma-2 Qwen3 GLM-4 ↑2.43 2.43 MMLU-Pro Nemotron Qwen3 Gemma-2 Nemotron Qwen3 Ministral-3 ↑1.78 1.78 OpenBookQA Nemotron Gemma-2 InternLM3 Nemotron Gemma-2 GLM-4 ↑1.40 1.40 E.3 Robustness and Ablation Analysis We examine the robustness of C2-MAS along two axes. Figure 5 sweeps team size k∈2,3,4,5k∈\2,3,4,5\ under all four aggregation rules and shows that C2-MAS maintains a consistent margin over Quality-Only and Caruana across budgets, confirming that the gains reported with k=3k=3 are not an artifact of a specific team size. Figure 6 then assesses whether the optimization objective is a faithful proxy for downstream accuracy: across the 13 benchmarks, the Spearman rank correlation between objective scores and ground-truth accuracy over all (113) 113 teams averages ρ=0.751ρ=0.751, supporting the validity of our selection criterion. (a) Team size sweep (k): Choice-Soft. (b) Team size sweep (k): PoE. (c) Team size sweep (k): DS. (d) Team size sweep (k): Stacking. Figure 5: Performance scaling with team size (k) across different aggregators. We compare the average test accuracy of C2-MAS (ours) against Quality-Only and Caruana baselines as the team size k varies from 2 to 5. The evaluation is performed under four distinct inference-time aggregation strategies: (a) Choice-Soft, (b) PoE, (c) DS, and (d) Stacking. C2-MAS demonstrates consistent improvements over baselines across all strategies, particularly in Stacking and DS, indicating that our heterogeneity-aware selection is robust to the choice of downstream combination mechanism. Figure 6: Validity of the optimization objective. To assess the reliability of our selection criterion, we compute the Spearman rank correlation coefficient (ρ) between the team scores derived from our objective function and their actual downstream test accuracy, calculated across all (113) 113 possible team combinations for each task. The histogram illustrates the distribution of these correlations across the 13 benchmarks. With a high mean correlation of 0.751, the results confirm that our proposed objective serves as an effective proxy for ground-truth performance, allowing C2-MAS to accurately rank candidate teams based solely on profiling data. E.4 Stability of Pooled Heterogeneity Estimates Figure 7 reports the variability of pairwise HI entries under repeated subsampling of development examples. The reduced variability after pooling complements the selection analysis: Section C.6 maps changes in pooled, centered signals to changes in standardized pairwise scores, and Proposition C.7 identifies when those changes preserve the selected team. Figure 7: Empirical stability of pooled HI under dev subsampling. We subsample 70% of each task’s dev set for 50 repetitions, recompute per-task HI matrices (Yule’s-Q R and JSD), and form pooled matrices from these noisy estimates. Pooling substantially reduces the across-repetition standard deviation of pairwise entries, complementing the selection-stability analysis in Sections C.5 and C.6. Appendix F Limitations Our current formulation of HIerrHI_err and HIdistHI_dist targets discriminative tasks with a shared finite label space, and does not directly cover open-ended generation, code synthesis, or multi-turn interactive settings. However, the decoupled pipeline extends naturally to generative settings by redesigning the metrics: HIerrHI_err can be adapted to pass@k decorrelation (co-failures on unit tests), and HIdistHI_dist to embedding-based distances (e.g., BERTScore) between open-ended outputs. The selection procedure itself is unchanged, and we leave empirical study to future work. Our profiling also assumes access to representative development data for each candidate model. Although cross-task pooling reduces estimation noise and improves transfer to held-out benchmarks, the selected team may still be sensitive to distribution shift when development tasks poorly match deployment conditions. Finally, C2-MAS addresses offline team selection rather than adaptive per-instance routing or interactive collaboration. The selected team is fixed for a task and can be paired with different aggregation rules, but our experiments do not study dynamic protocols such as debate, tool use, or iterative verification. Studying how the heterogeneity signals interact with such protocols is a natural next step.

Instant research discovery