Paper deep dive
HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems
Tianjun Yao, Zhaoyi Li, Zhiqiang Shen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/20/2026, 3:06:08 PM
Summary
The paper introduces HieraMAS, a hierarchical multi-agent system framework that optimizes both intra-node LLM mixtures and inter-node communication topology. It addresses credit assignment challenges in joint optimization by using a two-stage algorithm: multi-level reward attribution for fine-grained node-level feedback and graph classification for holistic topology selection. Experiments show HieraMAS outperforms existing methods in reasoning and coding benchmarks with better cost-performance trade-offs.
Entities (10)
Relation Signals (8)
HieraMAS → uses → Supernodes
confidence 95% · HieraMAS introduces supernodes, where each functional role is implemented by multiple heterogeneous LLMs
Supernodes → consistsof → LLMs
confidence 93% · each functional role is implemented by multiple heterogeneous LLMs using a propose-synthesis structure
HieraMAS → employs → Multi-level Reward Attribution
confidence 92% · we use a two-stage algorithm: (1) multi-level reward attribution, which provides fine-grained feedback
HieraMAS → employs → Graph Classification
confidence 90% · (2) graph classification for topology selection, which treats choosing the communication structure as a holistic decision
HieraMAS → solves → Credit Assignment
confidence 90% · Optimizing HieraMAS creates unique credit-assignment challenges... To address this, we use a two-stage algorithm
HieraMAS → evaluatedon → HumanEval
confidence 88% · We evaluate our approach on three diverse benchmark datasets... HumanEval++
HieraMAS → evaluatedon → MATH
confidence 88% · We evaluate our approach on three diverse benchmark datasets... MATH
HieraMAS → evaluatedon → MMLU-Redux
confidence 88% · We evaluate our approach on three diverse benchmark datasets... MMLU-Redux
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent systems (MAS) built on large language models (LLMs) have shown strong performance across many tasks. Most existing approaches improve only one aspect at a time, such as the communication topology, role assignment, or LLM routing, while treating each agent as a single, indivisible unit. This misses the opportunity to use mixtures of LLMs within an agent to strengthen role-specific abilities. We propose HieraMAS, a hierarchical collaboration framework that combines intra-node LLM mixtures with an inter-node communication topology. HieraMAS introduces supernodes, where each functional role is implemented by multiple heterogeneous LLMs using a propose-synthesis structure. Optimizing HieraMAS creates unique credit-assignment challenges: final task performance depends heavily on the underlying LLMs' capabilities, which can lead reinforcement methods to incorrectly reward suboptimal configurations. To address this, we use a two-stage algorithm: (1) multi-level reward attribution, which provides fine-grained feedback at both the node level and the overall system level; (2) graph classification for topology selection, which treats choosing the communication structure as a holistic decision rather than optimizing edges one by one. Experiments on reasoning and coding benchmarks show that HieraMAS substantially outperforms existing methods while also delivering better cost-performance trade-offs.
Tags
Links
- Source: https://arxiv.org/abs/2602.20229v1
- Canonical: https://arxiv.org/abs/2602.20229v1
Trouble viewing inline? Open PDF directly →
Full Text
75,494 characters extracted from source content.
Expand or collapse full text
HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Tianjun Yao * 1 Zhaoyi Li * 1 Zhiqiang Shen 1 Abstract Multi-agent systems (MAS) built on large lan- guage models (LLMs) have demonstrated remark- able performance across diverse tasks. Exist- ing approaches optimize communication topol- ogy, role assignment, or LLM routing in isola- tion, while treating each agent as a monolithic unit—failing to exploit internal LLM mixtures that can enhance individual role capabilities. We proposeHieraMAS, a hierarchical agent collab- oration framework with intra-node LLM mix- tures and inter-node communication topology. HieraMAS introduces supernodes, where each functional role comprises multiple heterogeneous LLMs in a propose-synthesis structure. The op- timization ofHieraMASposes unique credit as- signment challenges, as final task performance heavily depends on LLM capabilities, potentially causing erroneous reinforcement of suboptimal configurations. We address this via a two-stage al- gorithm: (1) multi-level reward attribution provid- ing fine-grained feedback at both node and system levels; and (2) graph classification treating topol- ogy selection as a holistic task rather than per- edge optimization. Experiments on reasoning and coding benchmarks demonstrate thatHieraMAS significantly outperforms existing methods while achieving better cost-performance trade-offs. 1. Introduction Recent advances in large language model (LLM) based agents have revealed a parallel phenomenon in artificial in- telligence. A growing body of research demonstrates that multi-agent systems (MAS) substantially outperform single- agent approaches across diverse tasks (Du et al., 2024a; Liang et al., 2024; Talebirad & Nadiri, 2023; Chen et al., 2024), catalyzing the development of numerous MAS frame- works (Wu et al., 2024; Hong et al., 2024; Qian et al., 2024). * Equal contribution 1 Mohamed bin Zayed University of Artifi- cial Intelligence, Abu Dhabi, UAE. Preprint, February, 2026. Beyond application-specific designs, researchers have iden- tified fundamental challenges in multi-agent coordination and proposed systematic optimization approaches along several key dimensions: communication topology learn- ing (Zhuge et al., 2024; Zhang et al., 2024a; 2025), which optimizes the information flow structure between agents; role assignment and specialization (Qian et al., 2025b; Hong et al., 2024; Liu et al., 2023b), which determines how agents are assigned distinct functional responsibilities; and LLM routing (Ong et al., 2024; Yue et al., 2025), which selects appropriate backbone models for different agent roles to bal- ance cost and capability. Concurrently, researchers have dis- covered that LLMs generate substantially better responses when provided with outputs from other models as auxiliary input (Wang et al., 2024; Li et al., 2025a;b). This phe- nomenon offers an intriguing connection to MAS: MAS inherently involves collaboration among potentially hetero- geneous LLMs through different roles and communication patterns, while the aforementioned phenomenon provides an alternative form of LLM collaboration through input-output composition. However, this connection remains unexplored in current research. Existing approaches either focus on role assignment (Liu et al., 2023b; Chen et al., 2023) or LLM routing (Yue et al., 2025), but none effectively integrates the collaborativeness property, as they treat each agent as a monolithic unit rather than exploiting the potential for inter- nal LLM mixtures to enhance individual role capabilities. In this work, we proposeHieraMAS(Hierarchical Col- laboration with MAS), which unifies these two forms of collaboration within a coherent MAS framework. Unlike conventional MAS where each node corresponds to a single agent,HieraMASintroduces supernodes, each comprising multiple potentially heterogeneous agents that implement a propose-synthesis structure to augment each functional role. Intuitively, enhancing each role’s capability through inter- nal mixtures may induce cascading effects on the overall system, e.g., some communication edges may no longer be necessary, leading to sparser and more efficient structures. Motivated by this hypothesis,HieraMASjointly optimizes three interconnected dimensions:①graph topology that determines inter-supernode communication patterns;②role pruning that identifies which functional roles to retain; and ③LLM selection within supernodes that configures the in- 1 arXiv:2602.20229v1 [cs.MA] 23 Feb 2026 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems CHALLENGE 1 CHALLENGE 2 Figure 1. Illustration of two credit assignment challenges in joint optimization and our solutions. Challenge 1: Final task rewards mask individual node errors—Node 2 produces incorrect output but receives high rewardR 2 = 0.92.HieraMASaddresses this via multi-level rewards that provide effective per-role attribution (R eff 2 =−0.23). Challenge 2: Per-edge optimization suffers from entangled attribution, where edges may be falsely reinforced or suppressed.HieraMASreformulates topology selection as a holistic graph classification task, using a graph generator to produce candidates and a graph classifier to select the optimal topology. ternal agent mixtures. This joint optimization can naturally be formulated as a credit assignment problem (Sutton, 1984) using reinforcement learning (Sutton et al., 1998). How- ever, compared to methods that optimize a single dimension, credit assignment becomes substantially more difficult in our setting (Figure 1).①Relying solely on final task rewards leads to inaccurate per-role attribution, as high rewards may mask individual node errors that were compensated by other agents.②Per-edge optimization faces similar attribution challenges but is even more challenging, as the contribu- tion of individual communication edges is entangled with both the sending and receiving nodes’ behaviors, making it challenging to isolate edge-level effects. To address these challenges,HieraMASemploys a two-phase algorithm: in the first phase, we use multi-level rewards rather than final outcome rewards alone, providing fine-grained attribution signals for optimizing supernode (Def. 1) configurations; in the second phase, given the inherent difficulty of edge-level credit assignment, we propose treating topology selection as a holistic graph classification task rather than per-edge optimization. We summarize our contributions as follows: • We introduce a novel MAS paradigm where collabora- tiveness emerges at two levels: intra-node collaboration through internal LLM mixtures within supernodes, and inter-node collaboration through communication across functional roles. •We proposeHieraMAS, a unified framework that jointly optimizes intra-node configurations (LLM selection and role retention) and inter-node structures (communication topology), enabling holistic system optimization. •We design multi-level rewards for fine-grained per-role credit assignment, and reformulate topology optimization as a graph classification task to circumvent the intractable edge-level attribution problem. •We conduct extensive experiments on programming, math- ematical reasoning, and general knowledge benchmarks spanning diverse subjects.HieraMASachieves state-of- the-art performance while maintaining cost-efficiency. 2. Preliminaries In this section, we formalize theHieraMASframework as a Markov Decision Process (MDP) and introduce its optimization objectives. 2.1. Notation Establishment Search Space. We define the search space of a MAS asS = (M, R, G) , whereMdenotes the pool ofN m available LLM backbones (including a specialskiptoken),Rrepresents the set ofN r predefined agent roles (e.g., Mathematical Analyst, Math Solver, Inspector), andGdenotes the space of graph topologies encoding inter-agent communication (e.g., Chain, FullyConnected, Star). Definition 1 (Supernode). A supernodeS i is a mixture- of-LLMs unit with roler i ∈ R, consisting ofWproposer 2 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems positions and one synthesizer: S i = r i ,m (w) i,j W j=1 ,m (a) i , m (w) i,j ,m (a) i ∈ M. (1) Here,iindexes the supernode within the system,j ∈ 1,...,W indexes proposer positions within supernode S i , the superscript(w)denotes proposer LLMs that gener- ate diverse proposals, and(a)denotes the synthesizer LLM that synthesizes proposer outputs into a unified response. Internally, each proposer connects to the synthesizer, which aggregates the diverse proposals from proposers and gener- ates the final response. This internal structure is inspired by prior work (Wang et al., 2024) and is not optimized. 2.2. MDP Formulation We formulate HieraMAS as an MDP (X,A,P,R): StateX. The state encodes the current configuration of the MAS: x = r i N i=1 ,m (w) i,j ,m (a) i i,j , E,Q ,(2) wherer i N i=1 are the role assignments for up toNsupern- odes,m (w) i,j ,m (a) i are the LLM assignments within each supernode,E∈0, 1 N×N is the adjacency matrix, andQ is the input query. ActionA. The action space consists of three components: (1) Role selection: for each supernodeS i , select a role r i ∈ Ror deactivate it; (2) LLM selection: for each position within a supernode (up toW + 1positions), select an LLM m∈ Morskip; and (3) Edge selection: select a subset of edgesE⊆ Gto enable communication among supernodes. TransitionP. Given statexand actiona, the transition P (x ′ |x,a)deterministically updates the system configura- tion and executes the MAS to obtain outputs. RewardR. The reward balances task performance and computational cost: R(x,a) = f (U (S;Q,a ∗ ),C(S;Q)),(3) whereS =S i N i=1 denotes the set of all supernodes in the system,U (·)measures correctness against ground-trutha ∗ , and C(·) quantifies token expenditure. 2.3. Optimization Objective Given a datasetDof queriesQwith ground-truth answers a ∗ ,HieraMASaims to learn a policyπ θ that maximizes expected reward: max θ E (Q,a ∗ )∼D,a∼π θ (·|x) [R(x,a)].(4) 3. Method In this section, we introduce ourHieraMASframework. Given a queryQ, our framework progressively constructs a customized MAS by: (1) selecting optimal LLMs for each position within supernodes, and (2) selecting an appropriate inter-supernode communication topology. We employ a two-stage training algorithm that addresses the unique credit assignment challenges arising from joint optimization. 3.1. LLM Selection within Supernodes Each supernodeS i containsWproposer positions and one synthesizer position. We learn to select the optimal LLM for each position based on task characteristics and role requirements. We encode the query with role in- formation ash Q,r = f ψ (Q,r, desc(r))using a sentence encoder (Reimers & Gurevych, 2019), and pre-compute LLM profile embeddingsh m ℓ = f ψ (profile(m ℓ ))for each m ℓ ∈ M. The selection probability over LLMs is: π m (m ℓ |Q,r) = exp(s ℓ /τ ) P m ′ ∈M exp(s m ′ /τ ) , where s ℓ = MLP(h Q,r , h m ℓ ) (5) and τ is the temperature. Role and proposer Pruning via Skip Token. Critically, Mincludes a specialskiptoken that enables automatic pruning at multiple granularities. Whenskipis selected for a proposer position, that proposer is omitted and incurs zero cost, allowing the system to adaptively reduce the mixture size within a supernode. Whenskipis selected for a synthesizer position, the entire supernode is deactivated, effectively pruning that functional role from the MAS. This unified mechanism enablesHieraMASto jointly learn both the optimal LLM configuration and which roles/proposers are necessary for a given query, without requiring separate pruning modules. 3.2. Graph Topology Selection To circumvent the challenge of the per-edge credit assign- ment, we propose treating topology selection as a holistic graph classification problem rather than per-edge optimiza- tion. Instead of learning edge probabilities directly, we train a graph classifier that scores a pool of topology candidates and selects the most suitable one for each query. This formu- lation sidesteps edge-level credit assignment by evaluating topologies as indivisible units. Graph Candidate Pool.We pre-generate a diverse pool ofKrandom directed acyclic graphs (DAGs)G = G 1 ,G 2 ,...,G K , where each graphG k represents a po- tential communication topology. The graphs are sampled with varying edge densities to ensure diversity, covering sparse, medium, and dense connectivity patterns. Graph Classifier. Given a queryQand a candidate graph G k represented by its adjacency matrixA k ∈0, 1 N×N , the classifier predicts a suitability score. We first construct 3 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Figure 2. The overall framework ofHieraMAS. By optimizing a policy learnerπ m with multi-level rewards (Stage 1) and a graph classifierf G (·)with contrastive rewards (Stage 2),HieraMASlearns to select optimal supernode configurations and communication topologies. During inference, the trained modules jointly determine the supernode configurations and graph topology, then execute the MAS to produce the final answer. node features by concatenating role embeddings with the query embedding, then apply a Graph Convolutional Net- work (GCN) (Kipf & Welling, 2017) to obtain graph-aware node representations: Z = GCN(X, A k ), z G = Pool(Z), s k = MLP(z G , h Q ), (6) whereXcontains node features andPool(·)aggregates node representations into a graph-level embedding. At inference, we select G ∗ = arg max G k ∈G s k . 3.3. Two-Stage Training Algorithm TrainingHieraMASrequires addressing the entangled op- timization of supernode configurations and graph topology. We propose a two-stage algorithm that decouples these com- ponents to enable more effective learning. The complete procedure is summarized in Algorithm 1. 3.3.1. STAGE 1: SUPERNODE OPTIMIZATION WITH RANDOM GRAPHS In the first stage, we focus on learning optimal LLM selec- tion within supernodes while using randomly sampled graph topologies. For each training sample, we randomly select a graphG k from the candidate poolGand execute the MAS with this fixed topology. This design serves two purposes: (1) it exposes the LLM selector to diverse communication patterns, preventing overfitting to a single topology; and (2) it decouples supernode learning from topology learning, allowing the selector to learn robust LLM assignments that generalize across different graph structures. Multi-Level Reward. To address the per-node credit as- signment challenge (Challenge 1 in Figure 1), we employ a multi-level reward structure that provides feedback at both the system level and individual supernode level. For each supernodeS i , we compute a node-level rewardR node i by evaluating its synthesizer output against the ground truth. The effective reward combines both signals: R eff i = α· R node i + (1− α)· R final ,(7) whereα∈ [0, 1]controls the mixing coefficient andR final is the reward from the decision node’s output. For roles whose outputs are not directly comparable to the final answer (e.g., planners, critics), we setα = 0to rely solely on the final reward signal. Cost-Aware Reward Function. Each reward (bothR node i andR final ) is computed using a cost-sensitive reward func- tion that balances correctness and computational cost: R(u,C) = ( exp(−λ· C),if u = 1 − exp(λ· C), if u =−1 (8) whereu ∈ −1, 1is the utility (correctness),Cis the token cost, andλis the cost sensitivity hyperparameter. This 4 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems formulation encourages the policy to select cost-efficient LLMs: correct solutions yield positive rewards that decrease with cost, while failures incur negative rewards that grow more severe with higher cost. Stage 1 Training Objective. The total loss for Stage 1 combines policy gradient with entropy regularization to encourage exploration: L stage1 =− N X i=1 logπ m (m i |Q,r i )·R eff i −λ H X p H(π (p) m ), (9) whereH(π (p) m ) = − P m ℓ π m (m ℓ ) logπ m (m ℓ ) is the en- tropy of LLM selection at positionp, preventing premature convergence to suboptimal configurations. 3.3.2. STAGE 2: GRAPH CLASSIFIER TRAINING After Stage 1 converges, we freeze the LLM selector and train the graph classifier. This stage generates labeled train- ing data by executing the fixed MAS with different graph topologies. Data Generation. For each taskQin the training set, we sampleMrandom graphs fromGand execute the MAS with each topology using the frozen LLM selector. We record the rewardR k for each graphG k , and label the top-performing graphs (with positive reward) as positive examples. Classifier Training. We train the graph classifier using binary cross-entropy loss: L stage2 = BCE(y k ,σ(s k )),(10) where y k is the label, σ(·) is the sigmoid function. This two-stage approach offers several advantages: (1) it avoids the credit assignment problem in edge-level optimiza- tion by treating topologies holistically; (2) it leverages the optimized LLM selector to generate meaningful training signals for the graph classifier; and (3) it enables efficient inference by simply scoring pre-generated graph candidates rather than sampling edges stochastically. 3.4. Theoretical Analysis We provide theoretical justification for our two-stage design in addressing the credit assignment challenges. Theorem 3.1. Consider optimizing a multi-agent system with N supernodes and a communication graph G∈G. (i)(Per-node credit assignment) Under final-reward-only training, when a failing supernode’s error is compen- sated by other agents, its policy gradient points in the wrong direction. Multi-level rewards with sufficient weight on node-level feedback ensure the gradient sign matches the desired update direction. (i)(Per-edge credit assignment) Per-edge policy gradi- ent optimization incurs an irreducible error rate of Ω((1− ρ)q), whereρis the fraction of optimal edges andqis the probability of high final reward. In con- trast, holistic graph scoring with random graph gen- eration reduces this to a vanishing estimation error O(1/ p N samples ), enabling correct topology identifica- tion with sufficient samples. Intuitively, multi-level rewards prevent the masking effect where system-level success hides individual failures, while graph-level scoring transforms the ill-posed per-edge credit assignment into a well-posed estimation problem. The for- mal proofs are deferred to Appendix C. 4. Experiments 4.1. Experimental Setup Datasets and Metrics. We evaluate our approach on three diverse benchmark datasets to comprehensively as- sess its performance across different task types: (1) Hu- manEval++ (Liu et al., 2023a): An enhanced version of the original HumanEval benchmark (Chen, 2021), featur- ing more robust evaluation via improved test suites for function-implementation tasks in code generation. (2) MATH (Hendrycks et al., 2021): A mathematical rea- soning benchmark containing challenging high school competition problems requiring multi-step reasoning. (3) MMLU-Redux (Gema et al., 2025): A decontaminated and disambiguated subset of the original MMLU bench- mark (Hendrycks et al., 2020), covering 30 subjects across STEM, humanities, and social sciences. For HumanEval++, we report Pass@1, and for MATH and MMLU-Redux, we report accuracy. Baselines. We compare against a comprehensive set of baselines spanning single-agent and multi-agent approaches: Single-agent methods: Base (direct prompting), Chain- of-Thought (CoT) (Wei et al., 2022). Fixed multi-agent methods: Self-Consistency (Wang et al., 2022), Self- Consistency+CoT, LLM-Debate (Du et al., 2024b; Liang et al., 2024), Full-Graph (fully-connected topology with all agents communicating), and Random-Graph (randomly gen- erated communication topology). Learning-based multi- agent methods: AFlow (Zhang et al., 2024b), which op- timizes workflows using Monte Carlo Tree Search; GDe- signer (Zhang et al., 2024a), which uses GNN-based topol- ogy optimization; and MASRouter (Yue et al., 2025), which learns to route between predefined topologies. LLM Pools. We use a diverse pool of LLMs for heterogeneous agent assignment:Qwen3-8B (Yang et al., 2025), Qwen3-Next-80B-A3B-Instruct (Yang et al., 5 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 1. Main results on three benchmarks. We report accuracy (%) for each dataset.✓indicates the method uses the corresponding component: Multi = multi-agent, Topo = optimizing topology, Role = optimizing roles and LLMs in each role, Node = optimizing intra-node configuration. Best results are in bold, second best are underlined. HumanEval++MATHMMLU-Redux MethodMultiTopoRoleNode GPT-5-MiniQwen3-80BGPT-5-MiniQwen3-80BGPT-5-MiniQwen3-80B Avg. Base✗89.0684.1477.7874.4492.0082.4083.14 CoT✗ ✗ ✗87.5085.9492.2290.0093.6089.6089.81 Self-Consistency✓✗89.0687.5093.3391.1194.4083.2089.77 Self-Consistency+CoT ✓ ✗ ✗ ✗90.6285.9494.4492.2293.6092.8091.60 LLM-Debate✓✗87.5087.5094.4494.4492.8092.0091.45 Full-Graph✓ ✗ ✗ ✗89.0692.1995.5696.6794.4088.8092.78 Random-Graph✓✗85.9492.1993.3394.4491.2088.0090.85 AFlow✓ ✓ ✗ ✗95.3198.4495.5684.4491.2091.2092.69 GDesigner✓✗90.6293.7591.1187.7792.0088.8090.68 MASRouter✓ ✓ ✓ ✗96.8898.4491.1188.8888.3381.6790.89 Ours✓93.7596.8896.6795.5695.2089.6094.61 2025), DeepSeek-R1-Distill-Qwen-14B (Guo et al., 2025), Llama-3.1-8B-Instruct (Dubey et al., 2024), DeepSeek- V3.2 (DeepSeek-AI, 2024), and Gemma-3-27B-IT (Team et al., 2025), GPT-5-Mini (OpenAI, 2025), GPT-5- Nano (OpenAI, 2025) and GPT-4o-Mini (Achiam et al., 2023). All models are accessible via open APIs, supporting reproducibility of our experiments. We conduct experiments with two settings: GPT-5-Mini and Qwen3-Next-80B-A3B- Instruct. For the GPT-5-Mini setting, methods that support learnable LLM selection (HieraMASand MASRouter) use all LLMs in the pool; for the Qwen3-80B setting, these methods exclude all GPT models from the pool. For other methods, GPT-5-Mini and Qwen3-Next-80B-A3B-Instruct are utilized respectively. Implementation Details. Details on implementations of HieraMAS are deferred to Appendix D.1. 4.2. Main Results Table 1 presents the main results comparing our approach against all baselines. Our method achieves the best aver- age performance of 94.61%, outperforming AFlow and MAS with full graph by a large margin. Notably, our ap- proach demonstrates consistent improvements across all three benchmarks, achieving the best performance on Hu- manEval++ and MATH, and competitive results on MMLU- Redux.①Full-Graph achieves 92.78% average accuracy by enabling all agents to communicate, but incurs signif- icant computational overhead, for instance, on MMLU- Redux with GPT-5-Mini, our method costs $1.29 while Full- Graph requires $4.23 (3.27×more expensive). AFlow op- timizes workflows via MCTS with Claude 3.5-Sonnet (An- thropic, 2024), resulting in prohibitive training costs: on HumanEval++ with GPT-5-Mini, our training cost is 18.41x cheaper than AFlow (More training cost is included in Ap- pendix D).②LLM-Debate and Random-Graph rely on fixed debate patterns or random structures, trailing our ap- Table 2. Ablation study on MATH and MMLU-Redux benchmarks. We evaluate the contribution of graph topology scoring and LLM selection. Acc (%) and Cost (USD) are reported. MATHMMLU-Redux Variant AccCostAccCost w/o Graph93.331.5792.001.36 w/o LLM Selection97.782.5694.403.92 Ours96.671.5295.201.29 proach by over 3%.③GDesigner and MASRouter both em- ploy learning-based topology optimization. However, MAS- Router routes between only 4 predefined regular topologies (e.g., debate, chain), constraining its search space. Our method instead learns to score and select from diverse ran- dom graph structures, discovering more effective collab- oration strategies that better exploit heterogeneous LLM capabilities. This gap also highlights the importance of learned topologies, consistent with recent findings that irreg- ular topologies outperform regular ones (Qian et al., 2025a). 4.3. In-depth Analysis We conduct more experimental analysis in this section, using GPT-5-Mini setting. Additional experimental results are demonstrated in Appendix D, including: (1) The effect of #Proposers in supernodes, and (2) The effect of the size of graph poolG. Ablation Study. We conduct ablation experiments to evalu- ate the contribution of each component in our framework, as shown in Table 2. We consider two variants: (1) w/o Graph, which removes the graph scoring mechanism and instead uses three randomly sampled graphs with averaged outputs; (2) w/o LLM Selection, which uses the learned opti- mal graph structureG ∗ but assigns all agents the strongest LLM backbone (i.e., GPT-5-Mini). 6 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Removing the graph scoring mechanism (w/o Graph) leads to substantial performance degradation on both benchmarks: 3.45% drop on MATH and 3.36% drop on MMLU-Redux, demonstrating the importance of learned topology selec- tion. For w/o LLM Selection, using only the learned struc- tureG ∗ with a fixed strongest LLM (GPT-5-Mini) yields marginal improvement on MATH, but at the expense of sig- nificantly higher cost ($2.56 vs. $1.52). On MMLU-Redux, using GPT-5-Mini for all nodes incurs 203.9% more cost while achieving lower accuracy compared toHieraMAS. This demonstrates that our heterogeneous LLM assignment strategy in the supernode, mixing weaker and stronger models based on role requirements, achieves better cost- performance trade-offs than uniformly using the strongest LLM in the MAS. Analysis on Learned Topology in MMLU-Redux. We vi- sualize the most frequently selected communication topolo- gies learned byHieraMASon MMLU-Redux under GPT- 5-Mini setting in Figure 3. The pairwise Jaccard similarity between the top-5 graphs ranges from 0.11 to 0.44, indicat- ing substantial structural diversity in the learned topologies. Despite this diversity, we identify several consistent patterns across the top-ranked graphs. Common structural properties: (1) Sink nodes: The Psychologist and Doctor roles consistently serve as sink nodes (high in-degree, zero out-degree) across most top graphs, receiving information from multiple sources but not propagating further. This suggests these roles function as fi- nal synthesizers or decision-makers. (2) Source nodes: The Critic role consistently acts as a primary source node (high out-degree, low in-degree), broadcasting information to mul- tiple agents. The Economist also frequently serves as a hub with high out-degree. (3) Sparse and irregular structure: The density ratios range from 0.23 to 0.32, indicating that the learned topologies are considerably sparser than a fully- connected graph (density=1.0). Analysis on Intra-Node Configuration. We analyze the dataset-level LLM selection preferencesπ m learned by HieraMASfor proposer and synthesizer positions within supernodes, as shown in Figure 4. The normalized logits reveal thatHieraMASlearns distinct LLM preferences tai- lored to different task characteristics. For proposer nodes, although the preferred LLMs vary across tasks, they con- sistently form strong-weak combinations that balance capability and cost. On HumanEval++, Llama-3.1-8B- Instruct, DeepSeek-R1-Distill-Qwen-14B, and DeepSeek- V3.2 emerge as the most preferred models. For MATH, Qwen3-Next-80B-A3B-Instruct and Qwen3-8B dominate the selection, while MMLU-Redux favors GPT-5-Nano and GPT-5-Mini. These patterns reveal task-specific model strengths: the GPT-5 series excels on general knowledge tasks requiring broad coverage, whereas Qwen3-Next-80B- Table 3. Generalization performance (%) on unseen MMLU cate- gories. Best in bold, second best underlined. MethodGPT-5-MiniQwen3-80B SC+CoT68.0064.00 AFlow72.0068.00 GDesigner72.0068.00 MASRouter56.0068.00 Ours68.0072.00 A3B-Instruct offers superior cost-effectiveness for mathe- matical reasoning. For synthesizer nodes, we observe highly consistent preferences with proposer nodes, suggesting that LLM selection is primarily driven by task characteristics rather than positional roles within the supernode architec- ture. Notably, theskiptoken is rarely selected across all tasks, indicating thatHieraMASprefers to reduce costs through sparser communication topologies (density 0.23- 0.32) rather than removing functional roles from the MAS. Generalization Analysis on MMLU-Redux. We evalu- ate out-of-domain generalization by reserving 5 subjects from MMLU-Redux that are excluded from both training and in-domain testing, using the remaining 25 subjects for training, as shown in Table 3. GDesigner and AFlow, de- spite being learning-based methods, demonstrate compet- itive generalization, comparable to or exceeding SC+CoT. This may be attributed to their use of the strongest available LLM during execution without learning, which provides an implicit bias toward generalization. In contrast, MAS- Router andHieraMASinvolve more extensive learning components (LLM/Role selection and topology optimiza- tion), posing greater challenges for out-of-domain gener- alization. Notably, MASRouter exhibits significant degra- dation under GPT-5-Mini setting, as its most frequently selected LLMs are Qwen3-Next-80B-A3B-Instruct, Qwen3- 8B, and Gemma-3-27B, models that may not generalize well to unseen subjects. In comparison,HieraMASmaintains performance by preferring GPT-5-Mini and GPT-5-Nano, which are more amenable to generalization across diverse subjects due to their broad pretraining coverage. Addi- tionally, we hypothesize that the learned communication topologies inHieraMAS, with Psychologist and Doctor as sink nodes and Critic as the source node, represent more generalizable collaboration patterns that transfer effectively to unseen domains. Cost Analysis. We compare inference costs with MAS- Router and GDesigner across the three benchmarks, as shown in Table 4. MASRouter achieves the lowest cost, which can be attributed to two factors: (1) its reward func- tion explicitly incorporates cost penalties, and (2) following their original implementation, we constrain the maximum number of agents to 6, resulting in aggressive role prun- 7 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems (a) Top-3 learned graph topologies. Top-1Top-2Top-3Top-4Top-5 Top-1 Top-2 Top-3 Top-4 Top-5 –0.290.250.290.43 0.29–0.110.240.12 0.250.11–0.200.39 0.290.240.20–0.17 0.430.120.390.17– 0.0 0.1 0.2 0.3 0.4 0.5 Jaccard Similarity (b) Pairwise Jaccard similarity of top-5 graphs. Figure 3. Analysis of learned topologies on MMLU-Redux. (a) Visualization of the top-3 most frequently selected graph structures with their density. (b) Pairwise Jaccard similarity between top-5 graphs, showing low structural overlap. LLaMA-8B DS-R1-14B DS-V3 Gemma-27B Qwen3-8B Qwen3-80B GPT-5-Mini Skip GPT-5-Nano GPT-4o-Mini HumanEval MATH MMLU 1.000.160.130.07-0.06-0.25-0.45-0.67-0.77-1.00 -0.590.050.17-0.610.561.00-0.04-1.00-0.21-0.24 -0.80-0.54-0.59-0.60-0.95-1.000.68-0.551.000.64 −1.0 −0.5 0.0 0.5 1.0 Normalized Logits Figure 4. Dataset-level LLM selection preferences learned by HieraMAS. Normalized Logits indicate selection preference, with higher values indicating stronger preference. Table 4. Cost comparison ($). Lowest cost in bold. MethodHumanEval++MATHMMLU GDesigner1.332.163.66 MASRouter0.120.320.15 Ours0.531.521.29 ing. In contrast,HieraMASwithskipaction does not impose such hard constraints; instead, it learns to reduce costs through sparser communication topologies rather than pruning agents, achieving a better trade-off between cost and performance. Compared to GDesigner, which incurs the highest cost, the difference stems from its lack of LLM and role selection: it defaults to using the strongest (and most expensive) model for all agents. Furthermore, GDe- signer requires 3 rounds of agent communication, whereas HieraMAScompletes inference in a single round, further contributing to the cost reduction. 5. Related Work Multi-Agent Systems. LLM-based multi-agent systems have emerged as a powerful paradigm, with foundational frameworks enabling agent collaboration through conversa- tion programming, standardized operating procedures, and dynamic composition (Li et al., 2023; Hong et al., 2024; Qian et al., 2024; Chen et al., 2024). More recently, Mixture- of-Agents (Wang et al., 2024) introduces a layered archi- tecture exploiting the collaborativeness phenomenon where LLMs generate better responses when provided with outputs from other models. Our work builds upon this insight by incorporating intra-node LLM mixtures within a broader MAS framework, while jointly optimizing communication topology and role configurations. Optimizing Multi-Agent Systems. Recent research has focused on optimizing MAS from multiple perspectives: communication topology optimization via graph-based rep- resentations and reinforcement learning (Zhuge et al., 2024; Zhang et al., 2024a; 2025; Qian et al., 2025b), dynamic role assignment with agent importance scoring (Liu et al., 2023b), and LLM routing for cost-quality trade-offs (Ong et al., 2024; Yue et al., 2025). Beyond these directions, AgentVerse (Chen et al., 2024) explores dynamic agent re- cruitment that adaptively assembles agent teams based on task complexity, while MetaGPT (Hong et al., 2024) in- troduces structured communication protocols inspired by software engineering workflows to reduce redundant interac- tions. While most of these works optimize individual dimen- sions or focus on single-agent routing,HieraMASjointly optimizes topology and intra-node LLM configurations for multi-agent collaboration within a unified framework. 6. Conclusion We presentedHieraMAS, a framework that unifies intra- node and inter-node collaboration in multi-agent systems. By introducing supernodes with internal LLM mixtures and a two-stage training algorithm,HieraMASaddresses the fundamental credit assignment challenges: multi-level rewards resolve per-node attribution, while holistic graph classification circumvents intractable per-edge credit assign- ment. Experiments on diverse domains demonstrate that HieraMASachieves state-of-the-art performance with su- perior cost-efficiency. We believe that this new paradigm opens promising directions for improving the reasoning ability for more complex, real-world tasks. 8 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Impact Statement This paper presents work on MAS, which are increasingly prevalent across various application domains. We believe this research contributes positively to the broader field of machine learning and collaborative AI systems. We do not foresee any direct negative societal consequences arising from this work. References Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. Anthropic. Claude 3.5 sonnet model card.https://w. anthropic.com/news/claude-3-5-sonnet, 2024. Chen, G., Dong, S., Shu, Y., Zhang, G., Sesay, J., Karls- son, B. F., Fu, J., and Shi, Y. Autoagents: A frame- work for automatic agent generation. arXiv preprint arXiv:2309.17288, 2023. Chen, M. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. Chen, W., Su, Y., Zuo, J., Yang, C., Yuan, C., Chan, C.-M., Yu, H., Lu, Y., Hung, Y.-H., Qian, C., Qin, Y., Cong, X., Xie, R., Liu, Z., Sun, M., and Zhou, J. Agentverse: Facil- itating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, 2024. DeepSeek-AI. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. In Forty-first International Conference on Machine Learning, 2024a. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. Improving factuality and reasoning in language models through multiagent debate. In International Conference on Machine Learning, 2024b. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Gema, A. P., Leang, J. O. J., Hong, G., Devoto, A., Mancino, A. C. M., Saxena, R., He, X., Zhao, Y., Du, X., Madani, M. R. G., et al. Are we done with mmlu? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 5069–5096, 2025. Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J.Measuring mas- sive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring math- ematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., Ran, C., Xiao, L., Wu, C., and Schmidhuber, J. Metagpt: Meta programming for a multi-agent collaborative frame- work. In The Twelfth International Conference on Learn- ing Representations, 2024. Kingma, D. P. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. Kipf, T. N. and Welling, M. Semi-supervised classifica- tion with graph convolutional networks. In International Conference on Learning Representations, 2017. Li, D., Tan, Z., Qian, P., Li, Y., Chaudhary, K., Hu, L., and Shen, J. Smoa: Improving multi-agent large language models with s parse m ixture-o f-a gents. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, p. 54–65. Springer, 2025a. Li, G., Hammoud, H., Itani, H., Khizbullin, D., and Ghanem, B. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991–52008, 2023. Li, W., Lin, Y., Xia, M., and Jin, C. Rethinking mixture-of- agents: Is mixing different large language models benefi- cial? arXiv preprint arXiv:2502.00674, 2025b. Liang, T., He, Z., Jiao, W., Wang, X., Wang, Y., Wang, R., Yang, Y., Shi, S., and Tu, Z. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Em- pirical Methods in Natural Language Processing, p. 17889–17904, 2024. Liu, J., Xia, C. S., Wang, Y., and Zhang, L. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems, 36:21558– 21572, 2023a. 9 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics, 12:157–173, 2024. Liu, Z., Zhang, Y., Li, P., Liu, Y., and Yang, D. Dy- namic llm-agent network: An llm-agent collaboration framework with agent team optimization. arXiv preprint arXiv:2310.02170, 2023b. Ong, I., Almahairi, A., Wu, V., Chiang, W.-L., Wu, T., Gonzalez, J. E., Kadous, M. W., and Stoica, I. Routellm: Learning to route llms with preference data, 2024. URL https://arxiv.org/abs/2406.18665. OpenAI. Introducing gpt-5.https://openai.com/ index/introducing-gpt-5/, 2025. Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., Xu, J., Li, D., Liu, Z., and Sun, M. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024. Qian, C., Xie, Z., Wang, Y., Liu, W., Dang, Y., Du, Z., Chen, W., Yang, C., Liu, Z., Sun, M., Zhu, K., and Xia, H. Scaling large-language-model-based multi-agent col- laboration. In International Conference on Learning Representations, 2025a. Qian, C., Xie, Z., Wang, Y., Liu, W., Zhu, K., Xia, H., Dang, Y., Du, Z., Chen, W., Yang, C., Liu, Z., and Sun, M. Scaling large language model-based multi-agent collab- oration. In The Thirteenth International Conference on Learning Representations, 2025b. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, p. 3982–3992, 2019. Sutton, R. S. Temporal credit assignment in reinforcement learning. University of Massachusetts Amherst, 1984. Sutton, R. S., Barto, A. G., et al. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998. Talebirad, Y. and Nadiri, A. Multi-agent collaboration: Harnessing the power of intelligent llm agents, 2023. URL https://arxiv.org/abs/2306.03314. Team, G., Kamath, A., Ferret, J., Pathak, S., Vieillard, N., Merhej, R., Perrin, S., Matejovicova, T., Ramé, A., Riv- ière, M., et al. Gemma 3 technical report. arXiv preprint arXiv:2503.19786, 2025. Wang, J., Wang, J., Athiwaratkun, B., Zhang, C., and Zou, J. Mixture-of-agents enhances large language model ca- pabilities, 2024. URLhttps://arxiv.org/abs/ 2406.04692. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. Self-consistency im- proves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. Williams, R. J. Simple statistical gradient-following algo- rithms for connectionist reinforcement learning. Machine Learning, 8(3-4):229–256, 1992. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. Autogen: Enabling next- gen llm applications via multi-agent conversation. In Forty-first International Conference on Machine Learn- ing, 2024. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yue, Y., Zhang, G., Liu, B., Wan, G., Wang, K., Cheng, D., and Qi, Y. Masrouter: Learning to route llms for multi- agent systems. In Annual Meeting of the Association for Computational Linguistics, 2025. Zhang, G., Yue, Y., Sun, X., Wan, G., Yu, M., Fang, J., Wang, K., Chen, T., and Cheng, D. G-designer: Archi- tecting multi-agent communication topologies via graph neural networks, 2024a. URLhttps://arxiv.org/ abs/2410.11782. Zhang, G., Yue, Y., Li, Z., Yun, S., Wan, G., Wang, K., Cheng, D., Yu, J. X., and Chen, T. Cut the crap: An economical communication pipeline for llm-based multi- agent systems. In International Conference on Learning Representations, 2025. Zhang, J., Xiang, J., Yu, Z., Teng, F., Chen, X., Chen, J., Zhuge, M., Cheng, X., Hong, S., Wang, J., et al. Aflow: Automating agentic workflow generation. arXiv preprint arXiv:2410.10762, 2024b. Zhuge, M., Wang, W., Kirsch, L., Faccio, F., Khizbullin, D., and Schmidhuber, J. Language agents as optimiz- able graphs. In Forty-first International Conference on Machine Learning, 2024. 10 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Appendix A. Notation Table Table 5 summarizes the key notation used throughout this paper. Table 5. Summary of notation used in this paper. SymbolDescriptionSymbolDescription Search Space & Supernode Structure SSearch space (M, R, G)S i Supernode i with role r i MPool of N m LLM backbonesr i Role for supernode i, r i ∈ R RSet of N r predefined agent roles WNumber of proposers per supernode GSpace of graph topologies m (w) i,j LLM for j-th proposer in supernode i GPool of K candidate DAGsm (a) i LLM for synthesizer in supernode i NMax number of supernodes MDP Components & Policy XState space π m LLM selection policy AAction spaceπ θ Overall policy with parameters θ PTransition functionf G Graph classifier RReward functionτSoftmax temperature QInput queryh Q,r Query-role embedding a ∗ Ground-truth answer h m ℓ LLM profile embedding EAdjacency matrix∈0, 1 N×N Rewards R final Final task rewardU (·)Utility (correctness) function R node i Node-level reward for supernode iC(·)Cost function (token expenditure) R eff i Effective reward (node + final) λCost sensitivity hyperparameter αMixing coefficient for multi-level reward Training Objectives L stage1 Stage 1 loss (LLM selector)T 1 ,T 2 Training iterations for Stage 1, 2 L stage2 Stage 2 loss (graph classifier)DTraining dataset λ H Entropy regularization coefficientD G Graph classification dataset H(π (p) m )Entropy at position p Graph Classifier A k Adjacency matrix for graph G k z G Graph-level embedding XNode feature matrix s k Suitability score for G k ZGraph-aware node representationsG ∗ Selected optimal graph B. Algorithm Algorithm 1 presents the complete two-stage training procedure for HieraMAS. C. Theoretical Analysis In this section, we provide formal theoretical analysis justifying our two-stage training design, corresponding to Theorem 3.1 in Section 3.4. The theorem identifies two fundamental credit assignment challenges in multi-agent systems: (i) per- node credit assignment, where individual agent failures may be masked by system-level success, and (i) per-edge credit assignment, where the contribution of individual communication links is difficult to isolate. Below, we formalize these challenges and prove how our proposed solutions: multi-level rewards and holistic graph selection to address them. Problem Setting. We consider multi-agent coordination problems formalized as MDPs with bounded rewardsR∈ [−B,B], finite state spaceX, and finite action spaceA(corresponding to discrete LLM and topology selections). The communication topology is represented as a directed acyclic graph (DAG) over N supernodes. 11 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Algorithm 1 HieraMAS Two-Stage Training Require: Training setD, graph poolG, LLM pool M Ensure: Trained LLM selector π m , graph classifier f G Stage 1: Supernode Optimization 1: for iteration = 1,...,T 1 do 2:Sample batchQ b B b=1 fromD 3:for each queryQ in batch do 4:Sample random graph G k ∼ Uniform(G) 5:Sample LLM assignments m∼ π m (·|Q) 6:Execute MAS with topology G k and LLMs m, obtain output ˆa 7:Compute rewards using ground-truth: R node i and R final 8:Compute effective rewards R eff i via Eq. equation 7 9:end for 10:Update π m by minimizingL stage1 (Eq. equation 9) 11: end for Stage 2: Graph Classifier Training 12: Freeze LLM selector π m 13: D G ←∅▷ Graph classification dataset 14: for each queryQ inD do 15:for k = 1,...,M do 16:Sample graph G k fromG, execute MAS, compute reward R k 17:end for 18:Label top graphs with R k > 0 as positive; add toD G 19: end for 20: for iteration = 1,...,T 2 do 21:Update graph classifier f G onD G via Eq. equation 10 22: end for Section C.1 provides the formal proof for Theorem 3.1(i), showing how multi-level rewards guarantee correct gradient direction for all supernodes by preventing the masking effect. Section C.2 proves Theorem 3.1(i), demonstrating that per-edge optimization incurs irreducible credit assignment error, while our holistic graph selection approach reduces this to a vanishing estimation error. C.1. Per-Node Credit Assignment: Multi-Level Rewards This subsection provides the formal proof for Theorem 3.1(i), which addresses the per-node credit assignment challenge. As illustrated in Figure 1(a), when using only final task rewards, a failing supernode (e.g., a Reviewer that introduces errors) may receive positive gradient updates if other agents compensate for its mistakes, leading to incorrect policy reinforcement. Our multi-level reward mechanism (Eq. equation 7) addresses this by combining node-level rewardsR node i with the final reward R final , ensuring that each supernode receives feedback proportional to its individual contribution. We provide theoretical justification for why multi-level rewards lead to more accurate policy gradients compared to using final rewards alone. The key insight is that when agents have heterogeneous performance, final-reward-only training can push poorly-performing agents’ policies in incorrect directions. Proposition 1 (Gradient Bias under Final Reward). Consider a multi-agent system withNsupernodes, where the policy for selecting LLM configuration at supernodeiis parameterized byθ i . LetR final denote the final task reward andR node i denote the node-level reward for supernode i. Define the policy gradient under final reward as: g final i =∇ θ i logπ θ i (m i |Q)· R final ,(11) and the policy gradient under multi-level reward as: g multi i =∇ θ i logπ θ i (m i |Q)· R eff i ,(12) 12 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems where R eff i = α· R node i + (1− α)· R final . Suppose there exists a “failed” supernodejsuch thatR node j < 0, while the final reward is positive due to other agents compensating for this failure, i.e., R final > 0. Then: (i) Under final reward, the gradient g final j reinforces the current (suboptimal) policy for supernode j. (i) Under multi-level reward with α > R final R final −R node j , the gradient g multi j correctly penalizes the suboptimal policy. Proof. We analyze the direction of policy updates for supernode j. Part (i): Final Reward Case. The policy gradient update rule increases the probability of actions that receive positive reward and decreases the probability for actions with negative reward. Under the REINFORCE estimator (Williams, 1992), the expected update direction for supernode j is: E g final j = E ∇ θ j logπ θ j (m j |Q)· R final .(13) SinceR final > 0by assumption, this gradient will increase the log-probability of the sampled actionm j . However, supernode jproduced a suboptimal output (as evidenced byR node j < 0). Therefore, the policy is reinforced in a direction that maintains or increases the probability of selecting the suboptimal configuration, the gradient direction is incorrect with respect to improving supernode j’s individual performance. Part (i): Multi-Level Reward Case. Under the multi-level reward, the effective reward for supernode j is: R eff j = α· R node j + (1− α)· R final .(14) For the gradient to correctly penalize the suboptimal policy, we require R eff j < 0, which implies: α· R node j + (1− α)· R final < 0(15) (1− α)· R final <−α· R node j (16) (1− α)· R final < α·|R node j | (since R node j < 0)(17) Rearranging: R final − α· R final < α·|R node j |(18) R final < α· (R final +|R node j |)(19) R final < α· (R final − R node j )(20) α > R final R final − R node j .(21) When this condition holds,R eff j < 0 , and the gradientg multi j will decrease the probability of the suboptimal action, correctly updating the policy in the direction that discourages the failing configuration. Corollary C.1 (Sufficient Condition for Gradient with Correct Sign). If the reward signals are normalized such that R final ,R node i ∈ [−1, 1], then settingα ≥ 0.5ensures the gradient sign matches the desired update direction whenever R node j ≤ 0 and R final ≤ 1. Proof. The worst case occurs whenR final = 1 andR node j = 0 (marginal failure). The threshold becomesα ∗ = 1 1−0 = 1. However, for any strictly negative R node j < 0, we have: α ∗ = R final R final − R node j < R final R final = 1.(22) 13 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems More practically, when R node j =−1 (complete failure) and R final = 1: α ∗ = 1 1− (−1) = 1 2 = 0.5.(23) Thus,α≥ 0.5ensures the gradient sign matches the desired update direction for all cases where the node-level performance is at least as negative as the final reward is positive, in magnitude. This analysis provides theoretical grounding for our multi-level reward design in Eq. equation 7. By incorporating node-level feedback,HieraMASavoids the pathological case where high-performing agents mask the failures of others, enabling more accurate credit assignment and faster convergence to optimal supernode configurations. C.2. Per-Edge Credit Assignment: Holistic Graph Selection This subsection provides the formal proof for Theorem 3.1(i), which addresses the per-edge credit assignment challenge. As illustrated in Figure 1(b), when optimizing communication topology via per-edge policy gradients, individual edges receive credit based on the final task reward. However, after Stage 1 optimization, the LLM selector is already well-trained, causing most configurations to achieve positive rewards regardless of topology. This leads to an irreducible credit assignment error: non-beneficial edges are incorrectly reinforced simply because they co-occur with successful task completions. Our holistic graph selection approach (Section 3.2) circumvents this problem by treating topology selection as a graph classification problem rather than per-edge optimization. By evaluating entire topologies as indivisible units and selecting from a pre-generated candidate poolG, we transform the ill-posed per-edge credit assignment into a well-posed estimation problem with vanishing error. We now analyze the credit assignment problem in per-edge topology optimization and justify our holistic graph selection approach. Proposition 2 (Credit Assignment Error in Per-Edge Optimization). Consider optimizing graph topology via per-edge policy gradient after the LLM mixture has been optimized (Stage 1 converged). LetG ∗ ⊆Edenote the set of edges in the optimal topology, whereEis the set of all possible edges. For each edgee∈E, letπ φ (e)denote the probability of including edge e, parameterized by φ. The per-edge policy gradient is: g e =∇ φ logπ φ (e)· R final ,if e is sampled.(24) Assumptions: (A1)After Stage 1 optimization, the final reward is high with probability q > 0.5, i.e., Pr(R final > 0) = q. (A2)Each edge e is sampled independently with probability p = 0.5. Justification of Assumptions: (A1) Stage 1 optimizes the LLM mixture over the expectation of randomly sampled graphs. We posit that the capability of the LLM combination has a larger impact on task success than the communication topology, i.e., strong LLMs can compensate for suboptimal topologies, but weak LLMs cannot be saved by optimal topologies. Therefore, after Stage 1 converges, the optimized LLM selector achieves positive reward on more than half of the instances, i.e., q > 0.5. (A2)Since we use randomly generated graphs during Stage 1 training, edges are sampled uniformly at random. For simplicity, we assume p = 0.5, corresponding to a uniform edge inclusion probability. LetE + = G ∗ denote the set of beneficial edges (those in the optimal topology) andE − = E \ G ∗ denote the set of non-beneficial edges. Define the gradient direction error for an edge e as: Error(e) = 1 if e∈E + and g e < 0 (beneficial edge incorrectly weakened) 1 if e∈E − and g e > 0 (non-beneficial edge incorrectly reinforced) 0 otherwise (correct gradient direction) (25) 14 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Then the expected fraction of edges with incorrect gradient direction is: E " 1 |E| X e∈E Error(e) # ≥ (1− ρ)· p· q,(26) where ρ = |G ∗ | |E| is the fraction of edges in the optimal topology. Proof. We analyze the gradient direction for edges inE + andE − separately. Case 1: Non-beneficial edges (e∈E − ). Consider an edge e∈E − that is sampled (included in the current graph G). The gradient update is: g e =∇ φ logπ φ (e)· R final .(27) Since∇ φ logπ φ (e)points in the direction that increasesπ φ (e), andR final > 0with probabilityq(by Assumption 2), the gradient g e > 0 will increase the probability of selecting edge e. However,e /∈ G ∗ , meaning this edge is not part of the optimal topology. Therefore, reinforcingeis incorrect, the gradient pushes the policy toward including a suboptimal edge. The probability that a non-beneficial edge e∈E − receives an incorrect gradient update is: Pr(Error(e) = 1| e∈E − ) = Pr(e sampled)· Pr(R final > 0) = p· q.(28) Case 2: Beneficial edges (e∈E + ). For edgese∈E + = G ∗ , correct gradient direction requiresg e > 0(reinforcement). WhenR final > 0, sampled beneficial edges receive correct positive gradients. However, whenR final < 0(which occurs with probability1− q), beneficial edges that were sampled receive incorrect negative gradients. The probability that a beneficial edge receives an incorrect gradient: Pr(Error(e) = 1| e∈E + ) = Pr(e sampled)· Pr(R final < 0) = p· (1− q).(29) Total Expected Error. The expected error across all edges is: E " X e∈E Error(e) # = X e∈E − Pr(Error(e) = 1) + X e∈E + Pr(Error(e) = 1)(30) =|E − |· p· q +|E + |· p· (1− q)(31) = p· [(|E|−|G ∗ |)· q +|G ∗ |· (1− q)](32) = p· [|E|· q−|G ∗ |· q +|G ∗ |−|G ∗ |· q](33) = p· [|E|· q +|G ∗ |· (1− 2q)].(34) Dividing by|E| and letting ρ = |G ∗ | |E| : E " 1 |E| X e∈E Error(e) # = p· [q + ρ· (1− 2q)](35) = p· [q + ρ− 2ρq](36) = p· [q(1− 2ρ) + ρ].(37) 15 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems When q is high (close to 1) and ρ is small (sparse optimal topology), the dominant term is: E[Error Rate]≈ p· q· (1− 2ρ) + p· ρ≈ p· q· (1− ρ),(38) forρ ≪ 1. Sincep· qrepresents the probability of sampling an edge under high reward, and(1− ρ)is the fraction of non-beneficial edges, this shows that: E[Error Rate]≥ (1− ρ)· p· q.(39) The error rate(1− ρ)· p· qreveals a fundamental tension in per-edge optimization after Stage 1. Withp = 0.5andq > 0.5, the error rate exceeds(1− ρ)/4. The error is amplified when the optimal topology is sparse (ρ → 0) or when Stage 1 achieves high success rate. Corollary C.2 (Justification for Holistic Graph Selection). The high error rate in per-edge optimization motivates treating topology selection as a holistic graph classification problem (Section 3.2). By evaluating entire topologies as indivisible units and selecting from a pre-generated candidate pool, our approach avoids the per-edge credit assignment problem entirely. C.3. Generalization Guarantee for Graph Classifier We now provide theoretical guarantees for our graph classifier approach. Theorem C.3 (Generalization Guarantee for Graph Classifier). LetG =G 1 ,G 2 ,...,G K be the pre-generated graph candidate pool, and let π ∗ m denote the optimized LLM selector from Stage 1. Assumptions: (B1)(Optimal topology coverage) The candidate poolG contains an optimal graph G ∗ ∈G. (B2) (Reward separability) Letμ k = E[R(Q,G k )]denote the expected reward for graphG k . There exists a margin γ > 0 such that: μ ∗ − μ k ≥ γ, ∀G k ̸= G ∗ .(40) (B3)(Bounded reward) The reward is bounded: R(Q,G k )∈ [−B,B] for allQ,G k . Justification of Assumptions: (B1)In practical settings, the number of roles is typically small. For instance, MATH uses 3 roles and MMLU-Redux uses 8 predefined roles at maximum, resulting in a finite and tractable search space of candidate topologies. Therefore the probability of including the optimal or near-optimal topology is high. Furthermore, if the exact optimal graph is not inG, a near-optimal graph with similar structure is likely to be present. (B2)This assumption is necessary and reflects our belief that the optimal communication topology leads to better performance given the same supernode configuration. If all topologies performed identically, topology optimization would be meaningless. The margin γ captures the performance gap between optimal and suboptimal topologies. (B3)This assumption is directly satisfied by our reward formulation in Eq. 8, where the utility and normalized cost ensure bounded rewards. In Stage 2, for each graph G k , we collect N k i.i.d. samples and compute the empirical mean reward: ˆμ k = 1 N k N k X j=1 R(Q j ,G k ).(41) Then, if N k ≥ 8B 2 log(2K/δ) γ 2 for all k ∈ [K], with probability at least 1− δ: arg max G k ∈G ˆμ k = G ∗ .(42) 16 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Proof. By Hoeffding’s inequality, for bounded random variables in [−B,B]: Pr [|ˆμ k − μ k | > ε]≤ 2 exp − 2N k ε 2 (2B) 2 = 2 exp − N k ε 2 2B 2 .(43) Applying union bound over all K graphs: Pr [∃k :|ˆμ k − μ k | > ε]≤ 2K exp − N k ε 2 2B 2 .(44) Setting ε = γ/4 and N k ≥ 8B 2 log(2K/δ) γ 2 : 2K exp − 1 2B 2 · 8B 2 log(2K/δ) γ 2 · γ 2 16 = 2K exp(− log(2K/δ)) = δ.(45) Thus, with probability at least 1− δ, for all k simultaneously: |ˆμ k − μ k |≤ γ/4. Under this event, for any G k ̸= G ∗ : ˆμ ∗ − ˆμ k ≥ (μ ∗ − γ 4 )− (μ k + γ 4 )(46) = (μ ∗ − μ k )− γ 2 (47) ≥ γ− γ 2 = γ 2 > 0.(48) Therefore, G ∗ = arg max G k ˆμ k . This theorem reveals a fundamental difference from per-edge policy gradient: • Per-edge RL (Proposition 2): The gradient errorΩ((1− ρ)pq)is irreducible, and persists regardless of sample size due to inherent credit assignment ambiguity. •Graph classifier (Theorem C.3): The estimation error|ˆμ k − μ k | =O(B/ √ N k )vanishes with more samples. With N k =O(B 2 log(K)/γ 2 ), the optimal graph is identified with high probability. The key insight is that: treating graph selection as supervised learning transforms an ill-posed credit assignment problem into a well-posed estimation problem. D. Additional Experimental Details D.1. Implementation Details Training Procedure. Our training procedure consists of two stages: Stage 1 (Supernode Optimization): We optimize the LLM selectorπ m using the Adam optimizer (Kingma, 2014) with learning rate2× 10 −3 and weight decay5× 10 −4 . During this stage, graph topologies are randomly sampled from the candidate poolGto expose the LLM selector to diverse communication patterns. The mixing coefficientαfor multi-level rewards is set to0.5based on the theoretical analysis in Corollary C.1. The training iterationsT 1 is searched over5, 10 with batch size 8. Stage 2 (Graph Classifier Training): After Stage 1 converges, we freeze the LLM selectorπ m and train the GCN-based graph classifier. For each training instance, we sampleM = 5random graphs from the candidate pool. We select the top-2 highest reward graphs as positive samples and the remaining 3 as negative samples. The GCN encoder consists of 2 graph convolutional layers with hidden dimension 256, followed by mean pooling and a linear classifier. We tune the dropout rate across0.05, 0.1, 0.2, 0.5and select the best model based on validation performance. We train forT 2 = 20epochs with early stop. 17 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 6. Effect of the number of proposers per supernode on MMLU-Redux. MethodAcc (%)Cost ($) w/ 8 proposers88.802.32 w/ 6 proposers91.201.97 Ours (2 proposers)95.201.29 Table 7. Effect of graph pool size K on MMLU-Redux. MethodAcc (%) w/ K = 5093.60 w/ K = 50094.40 Ours (K = 200)95.60 Graph Candidate Pool.We maintain a candidate pool ofK = 200randomly generated directed acyclic graphs (DAGs) for both training and inference. The graphs are generated with varying edge densities sampled uniformly from[0.3, 0.75]to ensure diversity, covering sparse, medium, and dense connectivity patterns. Sentence Encoder. For query and role encoding, we use the pre-trainedall-MiniLM-L6-v2sentence trans- former (Reimers & Gurevych, 2019) to compute embeddingsh Q,r andh m ℓ . The encoder produces 384-dimensional embeddings. The model is frozen without finetuning. Data Splits. For all benchmarks, we use an 80%/20% train/validation split. The training set is used for both Stage 1 and Stage 2 optimization, while the validation set is used for model selection (dropout tuning) and early stopping. Final evaluation is performed on the official test sets of each benchmark. D.2. Effect of Proposers in Supernode We investigate the impact of the number of proposers per supernode on MMLU-Redux, as shown in Table 6. Increasing the number of proposers from 2 to 6 or 8 leads to significant performance degradation: accuracy drops from 95.20% (2 proposers) to 91.20% (6 proposers) and 88.80% (8 proposers). Interestingly, the cost does not increase proportionally with the number of proposers. We observe that during training, the system increasingly selects theskipaction for synthesizer positions as the number of proposers grows, effectively pruning some roles from the MAS. This adaptive behavior differs from the 2-proposer configuration where pruning is less frequent. The performance drop with more proposers can be attributed to two factors. First, when the system learns to handle the increased complexity of many proposers, some meaningful roles may be inadvertently pruned during the optimization process, reducing the diversity of perspectives available for synthesis. Second, when too many proposers are active, the synthesizer receives excessively long context from multiple proposer outputs, leading to the lost-in-the-middle phenomenon (Liu et al., 2024) where critical information is overlooked. These findings suggest that a moderate number of proposers (e.g., 2) strikes the optimal balance between diversity of perspectives and the synthesizer’s aggregation capacity. D.3. Effect of Graph Pool Size We investigate the impact of the graph candidate pool sizeKon MMLU-Redux, as shown in Table 7. The results demonstrate thatHieraMASis not sensitive to the choice ofK: performance remains stable across different pool sizes, with accuracy ranging from 93.60% (K = 50) to 95.60% (K = 200). D.4. Case Studies We present qualitative case studies from each benchmark to illustrate howHieraMASadaptively selects LLM configurations and communication topologies based on task characteristics. 18 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 8. Case studies on MMLU-Redux. The typical configuration assigns GPT-5-Mini or GPT-5-Nano as proposers and GPT-5-Nano as the aggregator, favoring cost-efficient models for knowledge-intensive QA tasks. MMLU-Redux Case Studies Case 1: Nutrition QuestionCase 2: Communication Theory Query: Which of the following appears to lower bad cholesterol? A. Vitamin DB. Niacin C. ThiamineD. Riboflavin Query: According to “measurement,” what is the step that occurs between an individual gaining information and changing behavior? A. coorientationB. opinion change C. reaction formationD. semantic encoding D.5. Training Cost Analysis Table 11 presents the training cost comparison across different learning-based methods. AFlow incurs substantially higher training costs due to its Monte Carlo Tree Search optimization with Claude 3.5-Sonnet. Our method is more cost-efficient than GDesigner, which lacks LLM selection and defaults to using the strongest model (GPT-5-Mini) for all agents, whereas our approach incorporates LLM selection with cost-aware rewards that adaptively choose cheaper models when appropriate. Compared to MASRouter, our training cost is slightly higher due to two factors: (1) different LLM selection preferences during training, and (2) our two-stage algorithm requires additional sample collection in Stage 2 for graph classifier training, which incurs extra cost. Table 11. Training cost comparison (in USD) across benchmarks. DatasetGDesignerAFlowMASRouterOurs MATH$1.22$18.62$1.97$2.03 HumanEval++$1.95$28.64$0.54$1.55 MMLU-Redux$3.51$13.94$0.46$1.85 D.6. Prompts We provide the text profiles used for embedding-based LLM selection and role assignment. These profiles are encoded using a sentence transformer to compute similarity scores during the selection process. LLM Profiles.Table 12 presents the text descriptions for each LLM option in our candidate pool. These profiles capture model capabilities, cost characteristics, and recommended use cases. 19 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 9. Case studies on HumanEval. The typical configuration assigns Qwen3-8B, Qwen3-80B, or LLaMA3.1-8B as proposers, leveraging diverse code-specialized models for programming tasks. HumanEval Case Studies Case 1: Sum and ProductCase 2: Filter by Substring Query:def sum_product(numbers: List[int]) -> Tuple[int, int]: """ For a given list of integers, return a tuple consisting of a sum and a product of all the integers in a list...""" Query:def filter_by_substring(strings: List[str], substring: str) -> List[str]: """ Filter an input list of strings only for ones that contain given substring...""" Role Profiles. Table 13 presents the text descriptions for each agent role. These profiles define role responsibilities and specializations used in the supernode configuration. 20 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 10. Case studies on MATH. The typical configuration assigns Qwen3-8B combined with Qwen3-80B or DeepSeek-V3.2 as proposers, utilizing strong mathematical reasoning models. MATH Case Studies Case 1: Geometry ReflectionCase 2: Cone Volume Query: TriangleABCwith verticesA(−2, 0),B(1, 4)and C(−3, 2)is reflected over they-axis to form triangleA ′ B ′ C ′ . What is the length of a segment drawn from C to C ′ ? Query: The diameter of a cone is 30 decimeters. If the height is two times the radius, what is the volume of the cone in cubic decimeters? Table 12. LLM profiles used for embedding-based model selection. Each profile describes model strengths, cost tier, and recommended use cases. ModelProfile Description qwen3-8bFast and efficient 8B parameter model from Alibaba. Very fast inference, low latency, cost-effective. Good for basic text generation and simple Q&A. Very low cost. qwen3-80bPowerful 80B mixture-of-experts model from Alibaba. Strong reasoning, excellent at complex tasks, good at math and coding. Good for complex reasoning and mathematical problem-solving. Medium-high cost. deepseek-r1-14bMath-focused 14B model optimized for reasoning. Excellent mathematical reasoning and step-by-step problem solving. Good for math problems and logical reasoning. Low-medium cost. llama-8bGeneral-purpose 8B model from Meta. Well-balanced capabilities with good instruction following. Good for general tasks, conversation, and basic reasoning. Very low cost. deepseek-v3State-of-the-art reasoning model with excellent performance. Top-tier reasoning, excellent at complex problems, strong coding ability. Good for complex reasoning and advanced math. Medium cost. gemma-3-27bBalanced 27B model from Google. Good balance of speed and capability with strong instruction following. Good for medium complexity tasks. Medium cost. gpt-5-nanoExtremely fast and cheap model from OpenAI. Very low latency and lowest cost. Good for simple text processing and basic Q&A. Lowest cost option. GPT-5-MiniCapable model from OpenAI with strong reasoning. Strong reasoning and excellent at complex tasks. Good for complex reasoning and multi-step problems. Medium-high cost. gpt-4o-miniCapable and reliable model from OpenAI. Strong reasoning, good at coding, reliable instruction following. Good for complex reasoning and code generation. Medium-high cost. skipSpecial token: Do not assign any LLM to this position. Removes this agent from the pipeline entirely. No API call made, no cost incurred. Use when position is not needed for the current task. 21 HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems Table 13. Role profiles used for embedding-based role assignment. Roles are categorized by their primary application domain. RoleProfile Description Mathematical Reasoning Roles (MATH benchmark) Mathematical AnalystSpecialized in breaking down complex math problems. Identifies mathematical structure, defines variables, and sets up equations. Best for initial problem understanding and mathematical modeling. Math SolverSpecialized in executing mathematical computations. Solves equations, performs calculations, and verifies numerical results. Best for carrying out calculations and deriving numerical answers. InspectorSpecialized in reviewing and validating solutions. Checks solution correctness, identifies errors, and verifies final answers. Best for final verification and error catching. Programming Roles (HumanEval benchmark) Project ManagerSpecialized in design patterns and code structure. Coordinates solution strategy and organizes workflow. Best for complex problems requiring structured multi-step approaches. Algorithm DesignerSpecialized in algorithm design and pseudocode. Designs algorithms, creates pseudocode, and plans computational steps. Best for problems requiring novel algorithms or optimization. Test AnalystSpecialized in test cases and edge conditions. Identifies edge cases, designs test scenarios, and validates solutions. Best for ensuring solution robustness across all inputs. Programming ExpertSpecialized in writing executable code. Implements algorithms, writes clean code, and handles edge cases. Best for tasks requiring actual code implementation. Bug FixerSpecialized in debugging and code correction. Identifies bugs, fixes errors, and improves code robustness. Best for correcting issues in existing code. Knowledge QA Roles (MMLU-Redux benchmark) Knowledgeable ExpertSpecialized in domain knowledge and research. Provides domain expertise and searches for relevant information. Best for problems requiring specialized domain knowledge. CriticSpecialized in critical analysis and feedback. Critiques solutions, identifies weaknesses, and suggests improvements. Best for improving solution quality through critical review. MathematicianFor STEM questions. Handles mathematical reasoning, calculations, and formal logic. Best for math, physics, and engineering questions. PsychologistFor behavioral and social questions. Analyzes human behavior, cognitive processes, and social dynamics. Best for psychology and sociology questions. HistorianFor historical and cultural questions. Provides historical context, analyzes events, and identifies patterns. Best for history and political science questions. DoctorFor health and biology questions. Covers medical knowledge, biological processes, and health advice. Best for medicine and biology questions. LawyerFor legal and ethical questions. Handles legal reasoning, case analysis, and ethical frameworks. Best for law and ethics questions. EconomistFor economic and business questions. Covers economic analysis, market dynamics, and financial reasoning. Best for economics, business, and finance questions. ProgrammerFor computer science questions. Covers algorithm design, coding concepts, and system architecture. Best for computer science and technology questions. 22