Paper deep dive
Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems
Zhezheng Hao, Tianfu Wang, Huanshuo Dong, Ziyan Liu, Hong Wang, Xiankun Lin, Qiang Lin, Can Wang, Hande Dong, Jiawei Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 7/8/2026, 10:18:37 PM
Summary
The paper introduces Meta-Team, an experience-driven framework for evolving LLM-based multi-agent systems (MAS). It addresses the challenge of attributing failures in long, distributed MAS trajectories by proposing a collaborative self-evolution scheme. Instead of flattening trajectories into a single analyzer (global scheme) or analyzing them in isolation (local scheme), Meta-Team preserves agent-local contexts and coordinates post-task communication to exchange distributed evidence. This enables multi-scale self-evolution at the agent, interaction, and team levels. Experiments across six long-horizon benchmarks demonstrate that Meta-Team consistently outperforms single-agent systems, hand-crafted MAS, and prior evolution methods, offering reliable and scalable MAS self-evolution.
Entities (15)
Relation Signals (16)
Meta-Team ā evaluatedon ā SWE-bench Pro
confidence 95% Ā· we evaluate Meta-Team on six challenging benchmarks: SWE-bench Pro...
Meta-Team ā evaluatedon ā ResearchRubrics
confidence 95% Ā· ResearchRubrics for open-ended research evaluation.
Meta-Team ā evaluatedon ā LoCoBench
confidence 95% Ā· LoCoBench for long-context repository-level coding...
Meta-Team ā evaluatedon ā GAIA
confidence 95% Ā· GAIA for multi-step open-web reasoning...
Meta-Team ā evaluatedon ā LOCA-Bench
confidence 95% Ā· LOCA-Bench for long-context productivity assistants...
Meta-Team ā evaluatedon ā BeyondSWE
confidence 95% Ā· SWE-bench Pro and BeyondSWE for realistic software-engineering tasks...
Meta-Team ā implements ā Collaborative Self-Evolution
confidence 95% Ā· Meta-Team, an experience-driven MAS evolution framework based on collaborative self-evolution.
Meta-Team ā outperforms ā Hand-crafted MAS
confidence 95% Ā· Meta-Team consistently outperforms single-agent systems, hand-crafted MAS, and prior MAS evolution methods
ā ā
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:LLM-based multi-agent systems (MAS) have emerged as an effective paradigm for complex and long-horizon tasks. However, in real-world tasks, MAS often exhibit various failures during execution and such failures are difficult to eliminate during design. This motivates experience-driven MAS evolution, where a system improves based on its own execution experience. Yet such evolution is challenging because MAS experience is prolonged and intricate, interleaving multiple agents' execution chains and communication messages, which makes it difficult to identify what should be improved. To address this challenge, we propose Meta-Team, an experience-driven MAS evolution framework based on collaborative self-evolution. Meta-Team preserves the execution context of each agent and coordinates post-task communication, enabling agents to exchange distributed evidence for evolution. Building on this design, Meta-Team conducts multi-scale self-evolution, transforming execution experience into reusable improvements to agent behaviors, inter-agent coordination, and team-level organization. Across six long-horizon agent benchmarks, Meta-Team consistently outperforms single-agent systems, hand-crafted MAS, and prior MAS evolution methods; further analyses demonstrate that Meta-Team enables more reliable and scalable MAS self-evolution.
Tags
Links
- Source: https://arxiv.org/abs/2605.29790v1
- Canonical: https://arxiv.org/abs/2605.29790v1
Trouble viewing inline? Open PDF directly ā
Full Text
88,780 characters extracted from source content.
Expand or collapse full text
Evolve as a Team: Collaborative Self-Evolution for LLM-based Multi-Agent Systems Zhezheng Hao1 Tianfu Wang2 Huanshuo Dong3 Ziyan Liu3 Hong Wang3 Xiankun Lin3 Qiang Lin3 Can Wang1 Hande Dong3222Corresponding Authors Jiawei Chen1222Corresponding Authors 1 Zhejiang University 2 Hong Kong University of Science and Technology 3 Tencent Abstract LLM-based multi-agent systems (MAS) have emerged as an effective paradigm for complex and long-horizon tasks. However, in real-world tasks, MAS often exhibit various failures during execution and such failures are difficult to eliminate during design. This motivates experience-driven MAS evolution, where a system improves based on its own execution experience. Yet such evolution is challenging because MAS experience is prolonged and intricate, interleaving multiple agentsā execution chains and communication messages, which makes it difficult to identify what should be improved. To address this challenge, we propose Meta-Team, an experience-driven MAS evolution framework based on collaborative self-evolution. Meta-Team preserves the execution context of each agent and coordinates post-task communication, enabling agents to exchange distributed evidence for evolution. Building on this design, Meta-Team conducts multi-scale self-evolution, transforming execution experience into reusable improvements to agent behaviors, inter-agent coordination, and team-level organization. Across six long-horizon agent benchmarks, Meta-Team consistently outperforms single-agent systems, hand-crafted MAS, and prior MAS evolution methods; further analyses demonstrate that Meta-Team enables more reliable and scalable MAS self-evolution.33footnotetext: Code is available at https://github.com/z-haooo/Meta-Team. 1 Introduction Recent advancements in Large Language Models (LLMs) [49, 20, 5, 44] have driven rapid progress in LLM agents [66, 41, 4]. However, as LLM agents are increasingly applied to complex, long-horizon real-world tasks, a fundamental bottleneck of single-agent systems has emerged: they struggle to handle long-context information within a limited context window [26, 46] and face cognitive burden during long-horizon execution [88, 57, 2]. To address this bottleneck, LLM-based Multi-Agent Systems (MAS) have become an effective paradigm for complex and long-horizon tasks [21, 38]. MAS leverage the āwisdom of the crowdā and follow a divide-and-conquer principle ā they decompose complex tasks into manageable subtasks, assign them to specialized agents, and coordinate the agents to complete the task collaboratively [83, 84, 71]. This paradigm effectively reduces per-agent context overload and makes complex tasks more tractable. MAS have developed from early workflow-based systems [25, 52, 34] to recent industry-scale agent teams [63, 7], demonstrating their growing utility in solving real-world tasks. Despite this progress, building effective MAS still requires substantial human effort [3, 85]. Developers often need to manually design core MAS components, including agent prompts, interaction protocols, workflow structures, and coordination mechanisms, making such systems costly to build and difficult to adapt to diverse tasks. Moreover, practical deployments show that MAS often exhibit failure modes such as inter-agent miscommunication, role drift, and cascading errors [51, 64, 72]. Since these failure modes arise from actual execution, they cannot be fully eliminated through design-time specifications alone. These limitations have motivated recent studies on automated MAS generation and evolution. Early efforts primarily follow two lines: (1) Performance-driven strategies, which leverage a meta-level optimizer to generate, search, or refine MAS configurations based on performance metrics (e.g., pass rates or final rewards) [28, 58, 87, 80, 78]; (2) Memorization-driven strategies, which directly store MAS trajectories or interaction fragments for subsequent retrieval [39, 43, 74]. However, both paradigms lack fine-grained attribution over MAS experience: they provide limited insight into why and how a system succeeds or fails, leaving evolution to rely on exhaustive trial-and-error or superficial trajectory reuse rather than targeted improvement. More recently, to bridge this attribution gap, studies have employed a single centralized LLM analyzer to inspect full MAS trajectories, pinpoint failures, and propose revisions for MAS evolution [65, 29, 45]. While promising, this single-analyzer paradigm remains highly unreliable on long trajectories: existing failure-attribution benchmarks show that centralized analysis performs poorly in pinpointing the decisive failure step, and the performance declines as trajectories grow longer [82, 79, 9]. We argue that this limitation stems from a fundamental architectural mismatch between MAS execution and MAS evolution ā while MAS mitigate context overload during execution by distributing task information across agents, existing evolution methods regress by flattening the entire teamās trajectory into a single context for one analyzer. Consequently, they reintroduce the single-context bottleneck that MAS were originally designed to overcome. This bottleneck is even more pronounced during evolution, given that the full team trajectory is particularly prolonged and intricate [35, 14]. In this work, we advocate a simple principle for MAS evolution: a MAS should not only execute as a team, but also evolve as a team. Following this principle, we propose Meta-Team, a MAS evolution framework based on collaborative self-evolution. Embracing the philosophy of team reflexivity in organizational psychology [68, 56], Meta-Team preserves local execution contexts within the agents that produced them and coordinates inter-agent communication so that agents can exchange information required for evolution. This collaborative organization matches the distributed structure of MAS experience, avoiding the context and reasoning burden of flattening the full MAS trajectory into a single analyzer. Specifically, Meta-Team conducts self-evolution at three levels: (i) Agent-level evolution, where each agent reflects on its own execution behavior and updates its individual agent scaffold; (i) Interaction-level evolution, where agents revisit their collaboration history to refine communication and teammate profiles; and (i) Team-level evolution, where the team revises its organization and shared coordination rules via collective discussion. Instead of evaluating on saturated benchmarks, we conduct experiments on six challenging long-horizon agent benchmarks, where Meta-Team consistently improves over existing agents, hand-crafted MAS, and prior MAS evolution methods. For example, Meta-Team outperforms hand-crafted MAS by 6.6%6.6\% on average across nine benchmarks. These results demonstrate the effectiveness of collaborative self-evolution across diverse long-horizon tasks. In summary, our main contributions are: ā¢ We identify an architectural mismatch in experience-driven MAS evolution: MAS execute through distributed contexts, whereas existing evolution methods centralize the full experience, making it difficult to attribute failures and identify what to improve. ā¢ We propose Meta-Team, a collaborative self-evolution framework that preserves agent-local contexts, coordinates post-task communication to exchange distributed evidence, and translates the distributed evidence into multi-scale evolution. ā¢ We conduct extensive experiments and analyses. These results validate the effectiveness and scalability of Meta-Team in MAS self-evolution. 2 Preliminaries 2.1 LLM-based MAS An LLM-based Multi-Agent System (MAS) consists of multiple LLM agents that collaborate to solve a task. We define a MAS as ā³ā”(,),Mā”(A,O), where =aii=1NA=\a_i\_i=1^N is a set of agents and O denotes the orchestration mechanism that specifies how the agents are organized and how they interact and coordinate during task execution. Each agent aia_i consists of a backbone LLM ĻĻ and an agent scaffold sis_i, including its role instruction, task strategy, tools, and memory mechanism [58, 22]. The orchestration mechanism O can take various forms in existing systems [25, 69, 38]. For clarity, we decompose the orchestration mechanism as =(,)O=(P,S), where P denotes inter-agent protocols and S denotes the shared team scaffold. The protocol P specifies when and how agents exchange information and hand off intermediate results. For instance, workflow-based MAS often follow predefined information flows, whereas flexible agent-team systems allow agents to reason about when to contact teammates and what information to share. The shared team scaffold S specifies team-level rules visible to all agents, such as the team roster, organization, and shared constitution. 2.2 Experience-Driven Self-Evolution of MAS Given a task x, executing a MAS ā³M yields an execution experience defined as eā”(x,Ļ,r),eā”(x,Ļ,r), where Ļ is the multi-agent trajectory and r is the task outcome, such as the final deliverable and evaluator score. Unlike a single-agent trajectory, Ļ is not a single execution chain, but an interleaving of multiple agentsā local execution chains, communication messages, and intermediate artifacts. We denote the local execution chain of agent aia_i as Ļi _i, and write the complete trajectory as Ļ=Interleaveā”(Ļ1,ā¦,ĻN,ā¬),Ļ=Interleave( _1,ā¦, _N,B), where ā¬B denotes cross-agent events such as messages and shared artifacts. This formulation highlights the distributed structure of MAS experience: each agent holds a local execution context, while the complete team trajectory also depends on cross-agent interactions. Experience-driven self-evolution aims to improve a MAS from its own execution experience across tasks [60, 22, 45]. Given a sequence of experiences ā°1:k=e1,ā¦,ekE_1:k=\e_1,ā¦,e_k\, the MAS is updated by ā³k+1=Ī©ā(ā³k,ā°1:k).M_k+1= (M_k,E_1:k). Since task outcomes are usually sparse, self-evolution requires determining what should be improved from the trajectory and applying changes to the MAS. 3 Method 3.1 Collaborative Scheme for MAS Self-Evolution Real-world tasks often call for long-context understanding and long-horizon interaction [53, 36]. Solving them requires agents to perform hundreds of interaction steps and consume millions of context tokens, imposing a heavy burden of long-context understanding and long-horizon reasoning on a single agent. MAS alleviate this dual burden by distributing task information and execution responsibility across multiple agents during execution, allowing each agent to focus on more manageable subtasks within shorter and more coherent contexts [83, 84, 71, 57]. However, the distributed execution creates a new challenge for self-evolution: the resulting experience is longer and more complex than any single agentās local trajectory, making it difficult to identify what should be improved [31, 79, 14]. Existing analyzer-based evolution methods follow a Global Scheme: they flatten this joint experience and feed it to a single centralized LLM analyzer to attribute failures and propose revisions [29, 45]. While providing a global view, the global scheme concentrates the context and reasoning burden back on a single analyzer, reintroducing the bottleneck that MAS were originally designed to alleviate. A natural and naive counterpart is Local Scheme: decomposing the analysis by agent, with each analyzer reflecting on the corresponding local trajectory to identify local failures and propose agent-specific revisions, in the spirit of single-agent reflection [60, 1, 22]. Although the local scheme reduces the per-analyzer context length, it lacks a global view to understand how agentsā local decisions, messages, and dependencies jointly shape the final outcome, making it difficult to identify what should truly be improved. This reveals a global-local trade-off in organizing MAS experience for self-evolution. The global scheme preserves cross-agent coverage yet imposes a heavy centralized burden; the local scheme preserves local fidelity yet lacks cross-agent awareness. To address this trade-off, we propose a Collaborative Scheme for MAS self-evolution: each agent carries its local execution context into the evolution stage, while agents actively communicate to exchange processed local findings and trace how their outputs affected downstream execution. In this way, collaborative experience organization preserves the fidelity of agent-local contexts while recovering the cross-agent awareness needed to identify reliable targets for MAS evolution. Figure 1(a) illustrates how the global, local, and collaborative schemes organize MAS experience for self-evolution. (a) Schematic illustration. (b) Attribution comparison. Figure 1: Three different schemes of experience-driven MAS self-evolution. Empirical validation. To validate the global-local trade-off and the benefit of collaborative experience organization, we compare the three schemes in a failure-attribution setting. Settings. We evaluate the global, local, and collaborative schemes on TraceElephant [14], a failure-attribution benchmark containing 220220 real multi-agent failure traces from three public MAS frameworks. Each trace is annotated with a ground-truth (mistake_agent,mistake_step)( mistake\_agent, mistake\_step) tuple, and the goal is to identify the failure-causing agent (Agent Accuracy) and the decisive failure step (Step Accuracy). The three schemes differ only in how they organize the same MAS trajectory for attribution: global uses the full flattened trajectory, local uses isolated agent-wise trajectories, and collaborative uses agent-wise trajectories with cross-agent exchange. All schemes use the same backbone LLM, Claude Sonnet 4.6, and we report avg@3 Agent Accuracy and Step Accuracy. Implementation details are provided in Appendix C. Findings. The results are shown in Figure 1, where the dataset is split by trajectory length into ā¤128āKā¤ 128K and >128āK>128K regimes to examine how context length affects different schemes. Notably, all trajectories are within the 1āM1M-token context limit of the backbone LLM, so the global scheme always has access to the full trajectory without truncation. On trajectories shorter than 128āK128K, the global scheme outperforms the local scheme, suggesting that full-trajectory access is useful when the context remains manageable. However, on trajectories longer than 128āK128K, the global scheme drops sharply and falls behind the local scheme on both metrics, indicating that long interleaved MAS traces impose a severe context and reasoning burden even without context overflow. By contrast, the collaborative scheme consistently achieves the best performance in both length regimes. These results show that, compared with purely global or local analysis, the collaborative scheme better aligns with the distributed structure of MAS experience and provides more reliable guidance for self-evolution. Discussion. Beyond its benefit for identifying failure causes through context decomposition, collaborative self-evolution reframes agents from passive targets of diagnosis into active contributors to system evolution. This collaborative philosophy brings two further advantages for MAS evolution. Endogenous Feedback. Existing evolution methods often rely on final task outcomes or verifier scores as feedback [22, 74, 45], which are too sparse to specify how agents should improve their behavior. Collaborative self-evolution enriches this feedback through post-task communication: agents explain how othersā outputs affected their decisions and receive feedback on how their own outputs affected downstream execution. In this way, the teamās interactions become endogenous feedback signals for agent evolution. Bottom-Up Self-Evolution. Collaborative self-evolution also enables agents to surface system-level improvements. Since agents observe different parts of the execution, they can identify team-level issues such as missing roles, redundant responsibilities, unclear handoffs, or ineffective coordination rules. Following the principle of team reflexivity [68, 56], the team can aggregate these observations through collective discussion and revise its composition and organization. In this way, distributed execution experience becomes the basis for bottom-up self-evolution of the whole MAS. 3.2 Meta-Team: Multi-Scale Collaborative Self-Evolution The previous subsection establishes how MAS experience should be organized for self-evolution: agent-local contexts are preserved and connected through post-task communication. Meta-Team instantiates this principle by aligning evolution with the compositional structure of MAS. After each task, agents receive the final deliverable and evaluation result, collaboratively revisit the execution process, and convert distributed evidence into updates at three complementary scopes: agent-level behavior, interaction-level collaboration, and team-level coordination. These scopes are complementary: a single failure may simultaneously reveal an agent-specific limitation, a cross-agent dependency issue, and a team-level coordination gap [42, 30]. The overall process is illustrated in Figure 2. Figure 2: Overview of Meta-Team. Agent-Level Evolution (L1). At the agent level, the evolution targets are individual agent scaffolds sis_i. Although the update is applied to one agent aia_i, it is derived from collaborative self-evolution rather than from that agentās local context alone. The agent reviews evidence from its own execution chain and actively solicits cross-agent evidence to verify how its assumptions, intermediate outputs, and decisions affected downstream execution. This enables agent-specific improvements with both local fidelity and cross-agent awareness, such as what the agent should observe, verify, report, or avoid in future tasks. Interaction-Level Evolution (L2). At the interaction level, the evolution target is the collaboration mechanism P between agents. Through communication, agents revisit their previous collaboration to examine how information flowed and how agents relied on one another. Rather than optimizing a binary communication graph [74, 85], Meta-Team updates agentsā teammate profiles, which capture how agents understand, query, and rely on one another. These profiles help agents calibrate their understanding of each teammateās strengths and limitations, enabling more effective collaboration in subsequent tasks. Team-Level Evolution (L3). At the team level, the evolution target is the shared team scaffold S, including the team roster, the organizational structure among agents, and the shared team constitution, i.e., the global prompt that defines common objectives, collaboration principles, and decision rules for all members. Through collective discussion and decision-making, Meta-Team examines whether the team has the appropriate composition, organization, and shared rules for the task. It then decides whether to introduce new roles, remove redundant ones, reorganize coordination, or revise the shared constitution. In this way, Team-Level Evolution improves the teamās overall operating mechanism rather than only patching individual agents or pairwise interactions. After the three-scale evolution, Meta-Team commits validated updates to the reusable team scaffold, including agent patches, teammate profiles, collaboration notes, and revisions to the shared constitution. A compact implementation pipeline is provided in Appendix D. 4 Experiments and Results In this section, we conduct our experiments from the following perspectives. First, we compare Meta-Team against nine strong baselines on six agent benchmarks to examine its overall effectiveness (Ā§4.2). Second, we conduct ablation studies to isolate the contribution of Meta-Teamās core designs, including collaborative self-evolution and multi-scale evolution (Ā§4.3). Third, we study scalability and generalization by evaluating whether Meta-Team remains effective across increasing context lengths and generalizes beyond the evolution distribution (Ā§4.4). Finally, we analyze its efficiency under constrained budgets, testing whether Meta-Team achieves robust performance with limited time and cost (Ā§4.5). 4.1 Experimental Setup Benchmarks. Although prior MAS and MAS-evolution methods have been evaluated on various benchmarks, many of these benchmarks have become saturated under rapidly advancing LLM capabilities, detailed in Appendix F.1. Instead, we evaluate Meta-Team on six challenging benchmarks: SWE-bench Pro [17] and BeyondSWE [11] for realistic software-engineering tasks, LOCA-Bench [77] for long-context productivity assistants with MCP-style tools, GAIA [48] for multi-step open-web reasoning, LoCoBench [53] for long-context repository-level coding, and ResearchRubrics [59] for open-ended research evaluation. Appendix F.2 provides detailed benchmark descriptions, experimental settings, and initial configurations for each benchmark. Baselines. We compare Meta-Team against three families of baselines. Hand-crafted agents and multi-agent systems: SA (Single-Agent) [75], which synergizes reasoning and acting under the same executor, tools, and per-case budget as Meta-Team; MAS, our vanilla hand-designed multi-agent scaffold (identical to Meta-Teamās initial configuration but without any evolution); AggAgent [33], which samples K independent agent trajectories and synthesizes a final answer by cross-trajectory aggregation; OWL [27], a PlannerāCoordinatorāWorker hierarchy that was state-of-the-art on GAIA with a fixed role decomposition; and AOrchestra [55], which abstracts each sub-agent as a four-tuple āØInstructions,Context,Tools,Modelā© ,Context,Tools,Model and lets a central orchestrator synthesize sub-agents at runtime. Performance-driven search: AgentSquare [58], which searches the modular design space of Planning / Reasoning / Tool Use / Memory under a scalar task reward. Experience-based evolution (our closest comparison points): ReCreate [22], which creates domain-specialized agents from past interaction histories in an experience-driven loop; AgentNet [74], a decentralized DAG of agents with retrieval-augmented memory whose edges evolve from per-task success signals; and MASFly [45], which adapts the MAS at test time by distilling an SOP repository and a personalized experience pool that a single analyzer uses to revise trajectories. The details of the implementation of these methods are provided in Appendix F.3. Evolution and Evaluation Setup. Following [70, 22], we hold out around 20 instances per benchmark or subset as the evolution set. All evolution methods (AgentSquare, ReCreate, AgentNet, MASFly, Meta-Team) consume only this set during evolution and are evaluated on the disjoint held-out split with all learned artifacts frozen. Unless stated otherwise, all experiments use Claude Sonnet 4.6 [6] as the base LLM (temperature == 0.2, max_tokens == 32768). Full event traces (LLM calls, tool calls, inter-agent messages) are recorded for experience-driven evolution. To balance evaluation cost and variance, all results on all selected benchmarks are reported as avg@3 following [29, 45]. Table 1: Main results across diverse agent benchmarks. The best results are in bold. Method SWE-Pro BeyondSWE LOCA GAIA LoCoBench ResRub. Avg. Ansible Qute. DepMig. CrossR. Val Val Feat. Refact. All SA 44.744.7 62.762.7 43.543.5 41.741.7 78.378.3 70.070.0 64.564.5 62.062.0 43.943.9 56.856.8 MAS 40.840.8 61.061.0 37.637.6 40.440.4 82.182.1 74.774.7 57.657.6 60.960.9 49.549.5 56.156.1 AggAgent 49.649.6 57.757.7 42.842.8 40.640.6 80.480.4 70.370.3 60.260.2 54.754.7 47.247.2 55.955.9 OWL 39.539.5 60.160.1 36.736.7 36.936.9 72.972.9 64.064.0 53.853.8 56.256.2 47.947.9 52.052.0 AOrchestra 44.744.7 54.054.0 44.144.1 42.042.0 82.982.9 73.373.3 62.062.0 65.565.5 51.551.5 57.857.8 AgentSquare 39.039.0 56.356.3 38.438.4 37.437.4 68.368.3 65.065.0 56.256.2 55.955.9 41.941.9 50.950.9 ReCreate 42.542.5 59.659.6 41.641.6 39.139.1 74.674.6 69.069.0 58.958.9 62.062.0 45.945.9 54.854.8 AgentNet 40.840.8 58.758.7 44.344.3 39.639.6 69.669.6 67.367.3 57.057.0 56.956.9 40.340.3 52.752.7 MASFly 43.443.4 62.962.9 39.739.7 39.439.4 75.875.8 71.071.0 60.160.1 64.564.5 50.850.8 56.456.4 Meta-Team 53.953.9 66.266.2 45.645.6 43.343.3 87.987.9 77.377.3 67.167.1 67.067.0 55.855.8 62.762.7 Abbreviations. Ans. = ansible, Qute. = qutebrowser, DepMig. = DepMigrate, CrossR. = CrossRepo, Feat. = Feature Implementation, Refact. = Cross-File Refactoring, ResRub. = ResearchRubrics. Avg. is the unweighted arithmetic mean across the nine columns. 4.2 Main Results Table 1 reports the main results across six challenging agent benchmarks. Meta-Team achieves the best performance on all evaluated benchmark columns. This suggests that the benefit of Meta-Team is not tied to a single task format, but rather generalizes across heterogeneous agent settings. We first compare Meta-Team with the single-agent SA baseline and the initial hand-designed MAS. The initial MAS uses the same starting team scaffold as Meta-Team, but does not evolve from execution experience. Notably, the initial MAS is not consistently better than the single-agent baseline: it underperforms single-agent on six out of nine benchmark columns, suggesting that a human-designed MAS may suffer from organizational or coordination issues. In contrast, after collaborative self-evolution, Meta-Team outperforms both single agent and the MAS on every evaluated column. The gains against MAS are particularly clear on long-horizon tasks, such as SWE-bench Pro Ansible with +13.1+13.1 points and LoCoBench Feat. with +9.5+9.5 points. This shows that Meta-Team does not merely rely on having multiple agents; rather, it turns an initially imperfect team into an adaptive system whose organization and coordination improve from execution experience. Meta-Team also outperforms prior automated and experience-driven evolution methods, which exploit task feedback or past trajectories through performance-driven search, memory retrieval, or centralized analyzer-based revision. Meta-Team obtains consistent gains over these baselines, including an average improvement of 6.36.3 points over MASFly, the strongest MAS evolution baseline. This comparison shows that simply using experience is not sufficient; how the experience is organized is critical. The consistent gains of Meta-Team indicate that preserving agent-local contexts and enabling post-task communication provides a more effective way to convert distributed execution experience into MAS improvements. Table 2: Ablation on collaborative self-evolution. Experience schemes Ansible ā ResRub. ā No evolution (MAS) 40.840.8 49.549.5 Centralized experience 49.849.8 52.952.9 Partitioned experience 44.544.5 51.051.0 Collaborative experience 53.953.9 55.855.8 Table 3: Ablation on multi-scale evolution. Variant Ansible ā ResRub. ā Full Meta-Team 53.953.9 55.855.8 w/o L1 (agent) 48.548.5 47.947.9 w/o L2 (interaction) 50.750.7 54.454.4 w/o L3 (team) 51.951.9 52.152.1 4.3 Ablation Study We conduct ablations on SWE-bench Pro Ansible and ResearchRubrics to examine two core designs of Meta-Team: collaborative self-evolution and multi-scale evolution. All settings are kept identical to the main experiments, except for the experience scheme or the enabled evolution scales. Table 2 compares different schemes of organizing execution experience for MAS evolution. Centralized experience refers to flattening the whole-team trajectory into a single LLM analyzer call for evolution. Partitioned experience refers to spawning one LLM analyzer for each agent on its local experience, without cross-agent information exchange. Collaborative experience refers to Meta-Team, which preserves the agent-wise partition of execution contexts and coordinates agents to exchange local information based on their reasoning abilities. Although centralized and partitioned experience improve over no evolution, they remain clearly below collaborative experience. This result is consistent with the attribution analysis in Figure 1: collaborative experience yields more reliable evolution targets by combining agent-local evidence with cross-agent discussion. The resulting improvement suggests that Meta-Teamās gains come not merely from using past trajectories, but from organizing them in a form better suited to MAS failure attribution and self-evolution. Table 3 evaluates the three evolution scales. Removing any scale hurts performance, indicating that the three evolution scales are complementary to the MAS design. L1 is the most influential scale: removing agent-level evolution drops performance by 5.45.4 points on Ansible and 7.97.9 points on ResearchRubrics, suggesting that improving individual agent scaffolds is the most direct lever for MAS improvement. L2 and L3 show benchmark-dependent effects: L2 contributes more on Ansible, where structured software tasks rely on precise coordination, while L3 contributes more on ResearchRubrics, where open-ended evaluation benefits from better team organization. Together, these ablation studies demonstrate that Meta-Teamās gains come from both collaborative experience analysis and multi-scale evolution. (a) Experiments on dataset LOCA-Bench with different context scales. (b) Experiments on dataset LoCoBench with various evaluation sets. Figure 3: Experiments on scaling context and out-of-distribution evaluation set. Figure 4: Experiments with constrained budget. 4.4 Scalability and Generalization of Meta-Team We further examine whether Meta-Team remains effective beyond the distribution used for evolution. We study two axes: scalability across context length on LOCA-Bench and out-of-distribution generalization across programming languages on LoCoBench. In LOCA-Bench, Meta-Team is evolved only on 96K-token cases with held-out seeds, and is evaluated on standard seeds from 8K to 256K. In LoCoBench, Meta-Team is evolved on Python tasks and evaluated on C, C++, and Java tasks. Figure 3(a) shows the scalability results on LOCA-Bench. As the context length increases, the single-agent baseline degrades substantially, while the fixed MAS is more stable but remains below Meta-Team. Meta-Team maintains the best performance across all context lengths except 8K context. This suggests that collaborative self-evolution does not merely improve performance at the evolution context length, but learns more robust coordination patterns that continue to help under extreme context growth. Interestingly, Meta-Team introduces two additional workers during evolution on 96K context, which explains why Meta-Team is especially effective in the longest-context setting (256K), where both the single-agent baseline and the fixed MAS face stronger context-management pressure. Figure 3(b) reports the cross-language results on LoCoBench. Although evolution is performed on Python tasks, Meta-Team consistently outperforms both the single-agent baseline and the fixed MAS on C, C++, and Java for both Feature Implementation and Cross-File Refactoring. This indicates that the evolved improvements are not limited to language-specific shortcuts. Instead, Meta-Team learns reusable collaboration behaviors, such as task decomposition, handoff, and verification, that transfer to unseen programming languages. 4.5 Evolution under Constrained Budget In real deployment, MAS are running with time, cost, and token limits. Therefore, we examine whether Meta-Team can evolve effective teams under a limited cost budget. For all evolution methods, we constrain the evolution budget to 1/31/3 of the main setting, freeze the resulting artifacts, and evaluate them under the same per-case deployment limits. We compare MAS, ReCreate, AgentNet, MASFly, and Meta-Team on four benchmarks. The evaluation results are shown in Figure 4, where the x-axis reports the average inference cost per evaluation case. The results show that Meta-Team achieves the best performance-cost trade-off across all four settings. It obtains the highest performance with relatively low average evaluation cost, outperforming the other MAS self-evolution methods. This advantage arises because Meta-Team can use timeout feedback on the evolution set to adjust the team system, allowing it to better adapt to the constrained-budget setting. These results suggest that collaborative self-evolution helps Meta-Team learn more cost-efficient team behaviors under constrained evolution budgets. 5 Related Work 5.1 LLM-based Multi-Agent Systems MAS coordinate multiple language agents through role specialization, communication, and workflow-level organization. Early systems such as CAMEL [34], ChatDev [52], MetaGPT [25], and AutoGen [69] demonstrate that specified roles, protocols, and workflows can improve collaborative problem solving across tasks such as software development, reasoning, and tool use. More recent systems extend this paradigm to tool-rich and long-horizon settings [73, 66, 41]. For example, Magentic-One [18] coordinates specialized agents through a central orchestrator, while OWL [27] adopts a workforce-style organization with planning, coordination, and execution roles. Recent agent-team systems, including agent swarm [63], agent teams [7], and AOrchestra [55], further show the practical value of multi-agent coordination in real-world task automation. These works mainly study how agents should be organized during task execution, whereas our work studies how such organizations can be updated from execution experience. 5.2 Automated MAS Generation and Evolution Although LLM-based MAS have shown promise, recent studies show that they remain brittle in realistic settings, with failures often stemming from poor specification, system-design flaws, inter-agent misalignment, and weak verification or termination mechanisms [10, 82]. Other studies further show that small errors at inter-agent handoffs can propagate into system-level failures [42], and that self-organizing agent teams may fail to effectively leverage their strongest members [50]. These challenges motivate automated methods for designing, adapting, and evolving MAS. Recent work has explored automated generation and optimization of agent systems. One line of work formulates MAS construction as performance-driven search over prompts, modules, workflows, communication graphs, or routing policies. Representative methods include ADAS [28], AgentSquare [58], GPTSwarm [87], AFlow [80], MASS [78]. Subsequent work trains workflow optimizers [78, 40, 19, 67, 76, 15, 81, 32]. These methods optimize agent-system configurations using task-level feedback and reducing human design effort. Inspired by experience learning in LLM and single-agent [60, 47, 1, 22], another line of work uses execution experience to support MAS self-evolution. Cross-task experiential learning methods store MAS trajectories or interaction fragments for later retrieval [39], while AgentNet [74] evolves decentralized agent connectivity and expertise through accumulated experience. ASpec [65] cultivates persistent specialist teams for adaptive agent systems, and MASFly [45] adapts MAS at test time by leveraging experience-guided revision. Meta-Team follows this experience-driven direction, but organizes evolution according to the distributed structure of MAS itself, enabling agents to preserve local execution contexts while collaboratively evolving the system. 6 Conclusion In this paper, we study how LLM-based multi-agent systems can improve from their own execution experience. We advocate a simple principle for MAS evolution: a multi-agent system should not only execute as a team, but also evolve as a team. Building on this principle, we propose Meta-Team, which preserves agent-local contexts, enables post-task communication, and transforms distributed execution experience into targeted improvements for the team. We believe Meta-Team paves the way toward self-evolving agent teams that continuously improve their behaviors, communication, and coordination from their own distributed experience. References [1] L. A. Agrawal, S. Tan, D. Soylu, N. Ziems, R. Khare, K. Opsahl-Ong, A. Singhvi, H. Shandilya, M. J. Ryan, M. Jiang, et al. (2025) Gepa: reflective prompt evolution can outperform reinforcement learning. arXiv preprint arXiv:2507.19457. Cited by: Ā§3.1, Ā§5.2. [2] C. An, J. Zhang, M. Zhong, L. Li, S. Gong, Y. Luo, J. Xu, and L. Kong (2025) Why does the effective context length of llms fall short?. In ICLR, Cited by: Ā§1. [3] Anthropic (2025) How we built our multi-agent research system. Note: https://w.anthropic.com/engineering/built-multi-agent-research-system Cited by: Ā§F.2, Ā§F.2, Ā§1. [4] Anthropic (2026) Claude Code Overview. Note: https://code.claude.com/docs/en/overview Cited by: Ā§1. [5] Anthropic (2026) Introducing claude opus 4.6. Note: https://w.anthropic.com/news/claude-opus-4-6 Cited by: Ā§1. [6] Anthropic (2026) Introducing claude sonnet 4.6. Note: https://w.anthropic.com/news/claude-sonnet-4-6 Cited by: Ā§4.1. [7] Anthropic (2026) Orchestrate teams of Claude Code sessions. Note: https://code.claude.com/docs/en/agent-teams Cited by: Ā§1, Ā§5.1. [8] J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: Ā§F.1. [9] A. Banerjee, A. Nair, and T. Borogovac (2025) Where did it all go wrong? a hierarchical look into multi-agent error attribution. In NeurIPS Workshop on Evaluating the Evolving LLM Lifecycle: Benchmarks, Emergent Abilities, and Scaling, Cited by: Ā§1. [10] M. Cemri, M. Z. Pan, S. Yang, L. A. Agrawal, B. Chopra, R. Tiwari, K. Keutzer, A. Parameswaran, D. Klein, K. Ramchandran, et al. (2025) Why do multi-agent llm systems fail?. In NeurIPS Datasets and Benchmarks Track, Cited by: Ā§5.2. [11] G. Chen, F. Meng, J. Zhao, M. Li, D. Cheng, H. Song, J. Chen, Y. Lin, H. Chen, X. Zhao, et al. (2026) BeyondSWE: can current code agent survive beyond single-repo bug fixing?. arXiv preprint arXiv:2603.03194. Cited by: Ā§F.2, Ā§F.2, Ā§4.1. [12] H. Chen, X. Zheng, Y. Liu, P. Jiao, S. Li, H. Liu, Z. Zhao, Z. Xu, I. Khalil, and S. Pan (2026) GoAgent: group-of-agents communication topology generation for llm-based multi-agent systems. arXiv preprint arXiv:2603.19677. Cited by: Ā§F.1. [13] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: Ā§F.1. [14] M. Chen, J. Wang, F. Mu, Y. Wang, Z. Liu, H. Feng, and Q. Wang (2026) Seeing the whole elephant: a benchmark for failure attribution in llm-based multi-agent systems. arXiv preprint arXiv:2604.22708. Cited by: Appendix C, Ā§1, Ā§3.1, Ā§3.1. [15] Z. Chen, Z. Ji, Q. Mao, H. Wu, J. Cheng, B. Qin, Z. Li, J. Li, K. Sun, Z. Wang, et al. (2025) Scoring, reasoning, and selecting the best! ensembling large language models via a peer-review process. arXiv preprint arXiv:2512.23213. Cited by: Ā§5.2. [16] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. (2021) Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: Ā§F.1. [17] X. Deng, J. Da, E. Pan, Y. Y. He, C. Ide, K. Garg, N. Lauffer, A. Park, N. Pasari, C. Rane, et al. (2025) Swe-bench pro: can ai agents solve long-horizon software engineering tasks?. arXiv preprint arXiv:2509.16941. Cited by: Ā§F.2, Ā§F.2, Ā§F.2, Ā§4.1. [18] A. Fourney, G. Bansal, H. Mozannar, C. Tan, E. Salinas, F. Niedtner, G. Proebsting, G. Bassman, J. Gerrits, J. Alber, et al. (2024) Magentic-one: a generalist multi-agent system for solving complex tasks. arXiv preprint arXiv:2411.04468. Cited by: Appendix C, Appendix C, Ā§5.1. [19] H. Gao, Y. Liu, Y. He, L. Dou, C. Du, Z. Deng, B. Hooi, M. Lin, and T. Pang (2025) Flowreasoner: reinforcing query-level meta-agents. arXiv preprint arXiv:2504.15257. Cited by: Ā§5.2. [20] Google (2025) A new era of intelligence with gemini 3. Note: https://blog.google/products/gemini/gemini-3/ Cited by: Ā§1. [21] T. Guo, X. Chen, Y. Wang, R. Chang, S. Pei, N. Chawla, O. Wiest, and X. Zhang (2024) Large language model based multi-agents: a survey of progress and challenges.. In 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024), Cited by: Ā§1. [22] Z. Hao, H. Wang, J. Luo, J. Zhang, Y. Zhou, Q. Lin, C. Wang, H. Dong, and J. Chen (2026) ReCreate: reasoning and creating domain agents driven by experience. arXiv preprint arXiv:2601.11100. Cited by: Ā§F.3, Ā§2.1, Ā§2.2, Ā§3.1, Ā§3.1, Ā§4.1, Ā§4.1, Ā§5.2. [23] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2021) Measuring massive multitask language understanding. In International Conference on Learning Representations, Cited by: Ā§F.1. [24] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) Measuring mathematical problem solving with the math dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, Cited by: Ā§F.1. [25] S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, et al. (2024) MetaGPT: meta programming for a multi-agent collaborative framework. In ICLR, Cited by: Ā§F.2, Ā§1, Ā§2.1, Ā§5.1. [26] C. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y. Zhang, and B. Ginsburg (2024) RULER: whatās the real context size of your long-context language models?. arXiv preprint arXiv:2404.06654. Cited by: Ā§1. [27] M. Hu, Y. Zhou, W. Fan, Y. Nie, Z. Ye, B. Xia, T. Sun, Z. Jin, Y. Li, Z. Zhang, et al. (2025) OWL: optimized workforce learning for general multi-agent assistance in real-world task automation. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, Cited by: Ā§F.3, Ā§4.1, Ā§5.1. [28] S. Hu, C. Lu, and J. Clune (2025) Automated design of agentic systems. In ICLR, Cited by: Ā§1, Ā§5.2. [29] Y. Hu, M. Trager, Y. Zhang, Y. Zhang, S. Yang, W. Xia, and S. Soatto (2026) Evolutionary generation of multi-agent systems. arXiv preprint arXiv:2602.06511. Cited by: Ā§1, Ā§3.1, Ā§4.1. [30] Y. In, M. Tanjim, J. Subramanian, S. Kim, U. Bhattacharya, W. Kim, S. Park, S. Sarkhel, and C. Park (2026) Rethinking failure attribution in multi-agent systems: a multi-perspective benchmark and evaluation. arXiv preprint arXiv:2603.25001. Cited by: Ā§3.2. [31] F. Kong, R. Zhang, H. Yin, G. Zhang, X. Zhang, Z. Chen, Z. Zhang, X. Zhang, S. Zhu, and X. Feng (2025) Aegis: automated error generation and attribution for multi-agent systems. arXiv preprint arXiv:2509.14295. Cited by: Ā§3.1. [32] M. Kong, Z. Qu, Z. Zhou, P. Liang, X. Li, Z. Shang, Z. Hong, K. Huang, Z. Wang, and Z. Dai (2026) Workflow-r1: group sub-sequence policy optimization for multi-turn workflow construction. arXiv preprint arXiv:2602.01202. Cited by: Ā§5.2. [33] Y. Lee, H. Yen, X. Ye, and D. Chen (2026) Agentic aggregation for parallel scaling of long-horizon agentic tasks. arXiv preprint arXiv:2604.11753. Cited by: Ā§F.2, Ā§F.3, Ā§4.1. [34] G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem (2023) Camel: communicative agents for" mind" exploration of large language model society. Advances in neural information processing systems 36, p. 51991ā52008. Cited by: Ā§1, Ā§5.1. [35] J. Li, E. Yilmaz, B. Chen, and D. Le (2026) Towards self-improving error diagnosis in multi-agent systems. arXiv preprint arXiv:2604.17658. Cited by: Ā§1. [36] K. Li, J. Shi, Y. Xiao, M. Jiang, J. Sun, Y. Wu, D. Fu, S. Xia, X. Cai, T. Xu, et al. (2026) Agencybench: benchmarking the frontiers of autonomous agents in 1m-token real-world contexts. arXiv preprint arXiv:2601.11044. Cited by: Ā§3.1. [37] S. Li, Y. Liu, Q. Wen, C. Zhang, and S. Pan (2026) Assemble your crew: automatic multi-agent communication topology design via autoregressive graph generation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 23142ā23150. Cited by: Ā§F.1. [38] X. Li, S. Wang, S. Zeng, Y. Wu, and Y. Yang (2024) A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth 1 (1), p. 9. Cited by: Ā§1, Ā§2.1. [39] Y. Li, C. Qian, Y. Xia, R. Shi, Y. Dang, Z. Xie, Z. You, W. Chen, C. Yang, W. Liu, et al. (2025) Cross-task experiential learning on llm-based multi-agent collaboration. arXiv preprint arXiv:2505.23187. Cited by: Ā§F.1, Ā§1, Ā§5.2. [40] Y. Li, L. Li, Z. Wu, Q. Liao, J. Hao, K. Shao, F. Xu, and Y. Li (2025) AgentSwift: efficient llm agent design via value-guided hierarchical search. arXiv preprint arXiv:2506.06017. Cited by: Ā§5.2. [41] X. Liang, J. Xiang, Z. Yu, J. Zhang, S. Hong, S. Fan, X. Tang, B. Liu, Y. Luo, and C. Wu (2025) OpenManus: an open-source framework for building general ai agents. Zenodo. External Links: Document, Link Cited by: Ā§1, Ā§5.1. [42] B. Lin, K. Yang, Z. Tan, Y. Lai, C. Zhang, G. Zhang, X. Yu, M. Yu, X. Wang, Y. Zhang, et al. (2025) AgentAsk: multi-agent systems need to ask. arXiv preprint arXiv:2510.07593. Cited by: Ā§3.2, Ā§5.2. [43] J. Lin, Y. Guo, Y. Han, S. Hu, Z. Ni, L. Wang, M. Chen, H. Liu, R. Chen, Y. He, et al. (2025) Se-agent: self-evolution trajectory optimization in multi-step reasoning with llm-based agents. arXiv preprint arXiv:2508.02085. Cited by: Ā§1. [44] A. Liu, A. Mei, B. Lin, B. Xue, B. Wang, B. Xu, B. Wu, B. Zhang, C. Lin, C. Dong, et al. (2025) DeepSeek-v3. 2: pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. Cited by: Ā§1. [45] G. Liu, H. Lin, H. Zeng, H. Wang, and Q. Yao (2026) MAS-on-the-fly: dynamic adaptation of llm-based multi-agent systems at test time. arXiv preprint arXiv:2602.13671. Cited by: Ā§F.2, Ā§F.2, Ā§F.2, Ā§F.3, Ā§1, Ā§2.2, Ā§3.1, Ā§3.1, Ā§4.1, Ā§4.1, Ā§5.2. [46] N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024) Lost in the middle: how language models use long contexts. Transactions of the association for computational linguistics 12, p. 157ā173. Cited by: Ā§1. [47] A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. (2023) Self-refine: iterative refinement with self-feedback. Advances in neural information processing systems 36, p. 46534ā46594. Cited by: Ā§F.2, Ā§5.2. [48] G. Mialon, C. Fourrier, T. Wolf, Y. LeCun, and T. Scialom (2023) Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, Cited by: Ā§F.2, Ā§4.1. [49] Open AI (2026) Introducing gpt-5.4. Note: https://openai.com/index/introducing-gpt-5-4 Cited by: Ā§1. [50] A. Pappu, B. El, H. Cao, C. di Nolfo, Y. Sun, M. Cao, and J. Zou (2026) Multi-agent teams hold experts back. arXiv preprint arXiv:2602.01011. Cited by: Ā§5.2. [51] C. Phillips (2026-04) Multi-agent coordination patterns: five approaches and when to use them. Note: https://claude.com/blog/multi-agent-coordination-patternsAnthropic Blog, Accessed: 2026-05-05 Cited by: Ā§1. [52] C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, et al. (2024) Chatdev: communicative agents for software development. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers), p. 15174ā15186. Cited by: Ā§F.2, Ā§1, Ā§5.1. [53] J. Qiu, Z. Liu, Z. Liu, R. Murthy, J. Zhang, H. Chen, S. Wang, M. Zhu, L. Yang, J. Tan, et al. (2025) Locobench: a benchmark for long-context large language models in complex software engineering. arXiv preprint arXiv:2509.09614. Cited by: Ā§F.2, Ā§F.2, Ā§3.1, Ā§4.1. [54] S. Roy and D. Roth (2015) Solving general arithmetic word problems. In Proceedings of the 2015 conference on empirical methods in natural language processing, p. 1743ā1752. Cited by: Ā§F.1. [55] J. Ruan, Z. Xu, Y. Peng, F. Ren, Z. Yu, X. Liang, J. Xiang, Y. Chen, B. Liu, C. Wu, et al. (2026) AOrchestra: automating sub-agent creation for agentic orchestration. arXiv preprint arXiv:2602.03786. Cited by: Ā§4.1, Ā§5.1. [56] M. C. Schippers, M. A. West, and J. F. Dawson (2015) Team reflexivity and innovation: the moderating role of team context. Journal of Management 41 (3), p. 769ā788. Cited by: Ā§1, Ā§3.1. [57] H. Shang, X. Liu, Z. Liang, J. Zhang, H. Hu, and S. Guo (2025) United minds or isolated agents? exploring coordination of llms under cognitive load theory. arXiv preprint arXiv:2506.06843. Cited by: Ā§1, Ā§3.1. [58] Y. Shang, Y. Li, K. Zhao, L. Ma, J. Liu, F. Xu, and Y. Li (2025) AgentSquare: automatic llm agent search in modular design space. In ICLR, Cited by: Ā§F.3, Ā§1, Ā§2.1, Ā§4.1, Ā§5.2. [59] M. Sharma, C. B. C. Zhang, C. Bandi, C. Wang, A. Aich, H. Nghiem, T. Rabbani, Y. Htet, B. Jang, S. Basu, et al. (2025) Researchrubrics: a benchmark of prompts and rubrics for evaluating deep research agents. arXiv preprint arXiv:2511.07685. Cited by: Ā§F.2, Ā§4.1. [60] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: language agents with verbal reinforcement learning. Advances in neural information processing systems 36, p. 8634ā8652. Cited by: Ā§2.2, Ā§3.1, Ā§5.2. [61] L. Song, Y. Dai, V. Prabhu, J. Zhang, T. Shi, L. Li, J. Li, S. Savarese, Z. Chen, J. Zhao, et al. (2026) Coact-1: computer-using agents with coding as actions. In ICLR, Cited by: Ā§F.2. [62] L. Song, J. Liu, J. Zhang, S. Zhang, A. Luo, S. Wang, Q. Wu, and C. Wang (2024) Adaptive in-conversation team building for language model agents. arXiv preprint arXiv:2405.19425. Cited by: Appendix C, Appendix C. [63] K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chen, et al. (2026) Kimi k2.5: visual agentic intelligence. arXiv preprint arXiv:2602.02276. Cited by: Ā§1, Ā§5.1. [64] D. Tran and D. Kiela (2026) Single-agent llms outperform multi-agent systems on multi-hop reasoning under equal thinking token budgets. arXiv preprint arXiv:2604.02460. Cited by: Ā§1. [65] M. Vu, H. Ayyanar, P. JIANG, A. Reddy, and M. Goel (2026) Automated stateful specialization for adaptive agent systems. In ICLR, Cited by: Ā§1, Ā§5.2. [66] X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, et al. (2024) Openhands: an open platform for ai software developers as generalist agents. arXiv preprint arXiv:2407.16741. Cited by: Ā§1, Ā§5.1. [67] Y. Wang, L. Yang, G. Li, M. Wang, and B. Aragam (2025) Scoreflow: mastering llm agent workflows via score-based preference optimization. arXiv preprint arXiv:2502.04306. Cited by: Ā§5.2. [68] M. West (1996) Reflexivity and work group effectiveness: a conceptual integration. In The handbook of work group psychology, p. 555ā579. Cited by: Ā§1, Ā§3.1. [69] Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. (2024) Autogen: enabling next-gen llm applications via multi-agent conversations. In First conference on language modeling, Cited by: Ā§2.1, Ā§5.1. [70] Z. Wu, X. Lin, Z. Dai, W. Hu, Y. Shu, S. Ng, P. Jaillet, and B. K. H. Low (2024) Prompt optimization with ease? efficient ordering-aware automated selection of exemplars. Advances in Neural Information Processing Systems 37, p. 122706ā122740. Cited by: Ā§4.1. [71] Z. Xu, S. Zhu, J. Wang, J. Wang, B. Athiwaratkun, C. Wang, J. Zou, and C. Zhang (2026) When does divide and conquer work for long context llm? a noise decomposition framework. In ICLR, Cited by: Ā§F.2, Ā§1, Ā§3.1. [72] W. Yan (2025-06) Donāt build multi-agents. Note: https://cognition.ai/blog/dont-build-multi-agentsCognition AI Blog, Accessed: 2026-05-05 Cited by: Ā§1. [73] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press (2024) Swe-agent: agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems 37, p. 50528ā50652. Cited by: Appendix C, Appendix C, Ā§F.2, Ā§F.2, Ā§F.2, Ā§5.1. [74] Y. Yang, H. Chai, S. Shao, Y. Song, S. Qi, R. Rui, and W. Zhang (2025) AgentNet: decentralized evolutionary coordination for llm-based multi-agent systems. External Links: 2504.00587, Link Cited by: Ā§F.3, Ā§1, Ā§3.1, Ā§3.2, Ā§4.1, Ā§5.2. [75] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), Cited by: Ā§F.2, Ā§F.3, Ā§4.1. [76] Y. Yue, G. Zhang, B. Liu, G. Wan, K. Wang, D. Cheng, and Y. Qi (2025) Masrouter: learning to route llms for multi-agent systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 15549ā15572. Cited by: Ā§F.1, Ā§5.2. [77] W. Zeng, Y. Huang, and J. He (2026) LOCA-bench: benchmarking language agents under controllable and extreme context growth. arXiv preprint arXiv:2602.07962. Cited by: Ā§F.2, Ā§F.2, Ā§F.2, Ā§4.1. [78] G. Zhang, L. Niu, J. Fang, K. Wang, L. Bai, and X. Wang (2025) Multi-agent architecture search via agentic supernet. In International Conference on Machine Learning, p. 75834ā75852. Cited by: Ā§F.1, Ā§1, Ā§5.2. [79] G. Zhang, J. Wang, J. Chen, W. Zhou, K. Wang, and S. Yan (2025) AgenTracer: who is inducing failure in the llm agentic systems?. arXiv preprint arXiv:2509.03312. Cited by: Ā§1, Ā§3.1. [80] J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, et al. (2025) AFlow: automating agentic workflow generation. In ICLR, Cited by: Ā§1, Ā§5.2. [81] M. Zhang, H. Luo, T. Shen, Q. Lin, X. Tang, R. Mao, and E. Cambria (2026) FlowSteer: interactive agentic workflow orchestration via end-to-end reinforcement learning. arXiv preprint arXiv:2602.01664. Cited by: Ā§5.2. [82] S. Zhang, M. Yin, J. Zhang, J. Liu, Z. Han, J. Zhang, B. Li, C. Wang, H. Wang, Y. Chen, et al. (2025) Which agent causes task failures and when? on automated failure attribution of llm multi-agent systems. Proceedings of Machine Learning Research 267, p. 76583ā76599. Cited by: Ā§1, Ā§5.2. [83] Y. Zhang, R. Sun, Y. Chen, T. Pfister, R. Zhang, and S. Ć. Arık (2024) Chain of agents: large language models collaborating on long-context tasks. Advances in Neural Information Processing Systems 37, p. 132208ā132237. Cited by: Ā§1, Ā§3.1. [84] J. Zhao, C. Zu, X. Hao, Y. Lu, W. He, Y. Ding, T. Gui, Q. Zhang, and X. Huang (2024) LONGAGENT: achieving question answering for 128k-token-long documents through multi-agent collaboration. In EMNLP, p. 16310ā16324. Cited by: Ā§1, Ā§3.1. [85] H. Zhou, X. Wan, R. Sun, H. Palangi, S. Iqbal, I. VuliÄ, A. Korhonen, and S. Ć. Arık (2025) Multi-agent design: optimizing agents with better prompts and topologies. arXiv preprint arXiv:2502.02533. Cited by: Ā§1, Ā§3.2. [86] Z. Zhou, C. Li, X. Chen, S. Wang, Y. Chao, Z. Li, H. Wang, Q. Shi, Z. Tan, X. Han, et al. (2025) LLMĆ mapreduce: simplified long-sequence processing using large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 27664ā27678. Cited by: Ā§F.2. [87] M. Zhuge, W. Wang, L. Kirsch, F. Faccio, D. Khizbullin, and J. Schmidhuber (2024) Gptswarm: language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, Cited by: Ā§F.2, Ā§F.2, Ā§1, Ā§5.2. [88] D. Zou, Y. Chen, J. Wang, G. YANG, M. Li, Q. Da, J. Cheng, P. Li, and Y. Gong (2026) Reducing belief deviation in reinforcement learning for active reasoning of llm agents. In ICLR, Cited by: Ā§1. Contents Appendix A Limitations This work has two main limitations. First, Meta-Team focuses on scaffold- and organization-level evolution, including agent behaviors, collaboration patterns, and team-level coordination rules. It does not adapt the underlying execution infrastructure, such as tool APIs, environment harnesses, verifiers, or memory backends, which typically require task-specific engineering. Second, Meta-Team does not update the parameters of the underlying LLMs. Its evolution relies on post-task reflection and explicit scaffold updates, making it interpretable and model-agnostic, but limiting the extent to which recurring team behaviors can be internalized into the base models. Combining collaborative self-evolution with fine-tuning or reinforcement learning is a promising future direction. Appendix B Broader Impact Meta-Team introduces a collaborative self-evolution paradigm for LLM-based multi-agent systems by enabling agent teams to improve from their own distributed execution experience. Instead of relying on static, manually designed architectures or centralized post-hoc analyzers, Meta-Team allows agents to refine their roles, collaboration patterns, and team-level coordination through structured reflection. This approach can reduce the human effort required to design and maintain effective MAS, while improving their adaptability on long-horizon and tool-rich tasks. By making multi-agent systems more self-improving, inspectable, and reusable, Meta-Team has the potential to broaden access to intelligent automation across applications such as software engineering, research assistance, education, and enterprise workflows. Appendix C Global-Local-Collaborative Experiment Settings To examine this design choice, we compare three attribution paradigms on TraceElephant, a benchmark of 220 real MAS failure traces from captain-agent [62], magentic-one [18], and swe-agent [73]. Dataset. TraceElephant [14] provides 220220 failed multi-agent traces collected from three open-source MAS frameworks: Captain-Agent [62] (n=85n=85, short analytical tasks), Magentic-One [18] (n=91n=91, Magentic-One-style orchestrated tasks) and SWE-Agent [73] (n=44n=44, SWE-Agent traces on SWE-bench Verified). Each trace is a time-ordered list of step records (agent name, full input, full output), ranges from 22 to 9494 steps (median 2121) involving 22ā44 agents, and is annotated with a ground-truth (mistake_agent,mistake_step)( mistake\_agent, mistake\_step) tuple. Our split yields 106106 traces in the ā¤128ā¤\!128K split and 114114 in the >128>\!128K split. Agent-Accuracy and Step-Accuracy are exact-match rates against the annotated tuple, averaged per split by simple arithmetic mean. Settings. The three schemes are implemented as follows. The global scheme issues a single LLM call on the flattened trace and reads the (agent,step)( agent, step) pair directly from its JSON response. The local scheme spawns one analyzer per agent, each seeing only its own sub-trace and submitting (i_erredā0,1,my_step,c)( i\_erredā\0,1\,\, my\_step,\,c), where cā[0,1]cā[0,1]. The final (agent,step)( agent, step) is the submission of (i_erred=1 i\_erred=1) with the largest confidence c; if nobody self-accuses, we charge the first step of the least-confident denier, exposing the āno coordinationā failure mode. The collaborative scheme adds multiple rounds of communication: each analyzer first posts a short summary of its own findings, reads the summaries posted by the other analyzers, and then re-audits its sub-trace, submitting a tuple (i_erred,my_step,c,r)( i\_erred, my\_step,c,r) ,where cā[0,1]c\!ā\![0,1] is its self-confidence and r=1r\!=\!1 if it disagrees with the global verdict and backs the disagreement with explicit counter-evidence from its own sub-trace. Each analyzer then votes for the pair it accuses with weight w=cā(1+Ī±ār)w=c\,(1+Ī± r), where Ī±=1Ī±=1 and the highest-scoring pair wins. Appendix D Pipeline and Implementation of Meta-Team Meta-Team uses an open-roster orchestration mechanism. It maintains a candidate pool of agents and recruits an active subset for each task. The shared team scaffold contains the candidate roster and a team constitution, which specifies common objectives, collaboration principles, and decision rules. Each agent is implemented as an editable directory, with prompt.md storing its role prompt and config.yaml specifying the backbone model and allowed tools. The evolving components are stored as plain-text files under evolution/, including behavioral patches, teammate profiles, and pairwise collaboration notes. Agent skills are kept under skills/<name>/SKILL.md and loaded by progressive disclosure: only skill names and descriptions are shown by default, while the full skill body is retrieved when needed. During execution, the active roster is assembled dynamically. The orchestration protocol provides a small set of primitives, including ListPool, StartAgent, StopAgent, Finalize, and Terminate. Each episode starts with pool-level visibility through ListPool; agents are recruited through StartAgent when additional expertise is needed; and the episode ends when an agent calls Finalize to submit the final deliverable. Communication is handled by an append-only message bus. When an agent sends a message, the event is appended to the bus and delivered to the receiverās mailbox. Each agent runs its own loop: it either works by calling the backbone model and tools, or waits for new messages when idle. The recorded experience therefore contains each agentās local trace, together with cross-agent events such as messages, handoffs, and shared artifacts. After each task, Meta-Team updates the system at three levels. Agent-level evolution revises an agent using its local trace and the cross-agent evidence directly related to it. Interaction-level evolution updates teammate profiles and collaboration notes for agents that actually interacted. Team-level evolution aggregates the short summaries and selected evidence from the previous two levels to revise the constitution, coordination rules, or candidate roster. This avoids sending the full flattened trajectory to a single analyzer: local traces are preserved, relevant evidence is exchanged, and only concise summaries are used for team-level revision. Before updates are committed, Meta-Team checks role consistency, tool availability, formatting validity, and budget constraints. If a retry is requested during evolution, it is allowed only when budget remains, and the retry cost is counted toward the evolution budget. Algorithm 1 summarizes the procedure. Meta-Team instantiates the MAS ā³=(,)M=(A,O) with an open-roster orchestration mechanism. The active agent set A is selected at run time from a fixed candidate pool āA , and the shared scaffold is written as =(ā,)S=(A ,C), where C denotes the team constitution. Each agent scaffold is decomposed as si=(Ļi,Īi,i,Ī¦i,Ri),s_i=( _i,\; _i,\;K_i,\; _i,\;R_i), where Ļi _i is the base role prompt, Īi _i stores behavioral patches, iK_i is the skill library, Ī¦i _i contains teammate profiles, and RiR_i records pairwise correlation notes. Given a task xkx_k, Meta-Team executes the current MAS to collect experience ek=(xk,Ļk,rk)e_k=(x_k, _k,r_k). It then applies the update operator Ī© at three scales: agent-level updates, interaction-level updates, and team-level updates. The resulting MAS is committed as ā³k+1M_k+1. Algorithm 1 summarizes the procedure. Algorithm 1 Meta-Team: Collaborative Self-Evolution of an LLM-based MAS. 1:Initial MAS ā³0M_0 with pool āA ; task stream xkk=1K\x_k\_k=1^K; evaluator V. 2:Evolved MAS ā³KM_K. 3:for k=1k=1 to K do 4: ā³āā³kā1M _k-1 5: (,Ļk,r^)āRunā(ā³;xk)(A, _k, r)ā Run(M;x_k) ā³ recruit āāA and execute 6: rkāā(r^)r_k ( r); ekā(xk,Ļk,rk)e_kā(x_k, _k,r_k) ā³ freeze the task experience 7: for all aiāa_i in parallel do 8: (Īi,i,ui)āĪ©L1ā(si,Ļi,rk)( _i,K_i,u_i)ā _ L1(s_i, _i,r_k) ā³ agent-level reflection 9: end for 10: for all interacting pairs (ai,aj)(a_i,a_j) in Ļk _k do 11: (Ī¦i,Ri,Ī¦j,Rj)āĪ©L2ā(si,sj,Ļk,rk)( _i,R_i, _j,R_j)ā _ L2(s_i,s_j, _k,r_k) ā³ interaction-level reflection 12: end for 13: āĪ©L3ā(,uiaiā,Ļk,rk)Sā _ L3 (S,\u_i\_a_i , _k,r_k ) ā³ team-level revision 14: if the team requests retry then 15: re-execute xkx_k with the updated ā³M 16: end if 17: ā³kāā³M_k ā³ commit the updated MAS 18:end for 19:return ā³KM_K Appendix E Statistical Reliability of Main Results We further assess the statistical reliability of the main-result margins in Table 1. Since all methods are evaluated on the same held-out instances, each comparison forms a paired sample. We use the instance as the unit of analysis: for each method and instance, the three rollouts are aggregated into a single avg@3 score, and a two-sided paired t-test is applied to the per-instance differences. As a robustness check, the Wilcoxon signed-rank test gives the same significance label in all checked cases; we therefore report the paired t-test results. The most reliable gains appear on the longer and more coordination-intensive benchmarks. Compared with the initial MAS, Meta-Team improves by +6.0+6.0 points on LOCA-Bench (n=240n=240, p<0.001p<0.001), by +9.6+9.6 and +6.1+6.1 LCBS points on the two LoCoBench slices under the 0āāā1000ā100 LCBS scale (n=80,p<0.001n=80,p<0.001 and p<0.01p<0.01), and by +6.2+6.2 points on ResearchRubrics (n=81n=81, p<0.001p<0.001). Compared with MASFly, the corresponding margins are +12.2+12.2 points, +7.0+7.0 LCBS points, +2.5+2.5 LCBS points, and +4.9+4.9 points, all significant at p<0.05p<0.05. The gains on SWE-bench Pro Ansible (n=76n=76) and BeyondSWE DepMigrate (n=158n=158) are also significant against both baselines at p<0.05p<0.05 or better. The remaining columns, SWE-Pro Qute., BeyondSWE CrossRepo, and GAIA, show positive but not statistically significant margins. These columns have smaller effect sizes and many paired instances on which both methods obtain the same outcome, leaving limited paired variation for the test. We therefore interpret them as directionally positive rather than statistically conclusive. Overall, the paired tests support the qualitative reading of Table 1: Meta-Team improves over the baselines across all benchmark columns, with statistically reliable gains on the settings that most benefit from collaborative self-evolution, while smaller-margin columns remain positive but inconclusive at the current sample size. Appendix F Experiment Settings F.1 Saturation of Commonly Used Benchmarks Prior MAS and MAS-evolution studies [78, 39, 76, 37, 12] have evaluated on a wide range of standard benchmarks. However, as LLM capabilities advance rapidly, many of these commonly used benchmarks have become increasingly saturated. As shown in Table 4, a state-of-the-art model already achieves near-perfect performance on several widely used datasets, including HumanEval [13], MBPP [8], MultiArith [54], GSM8K [16], MATH [24], and MMLU [23]. Such saturation reduces their discriminative power for evaluating new MAS evolution methods, since small performance differences near the ceiling may no longer reflect substantive improvements in reasoning, tool use, collaboration, or long-horizon execution. Therefore, in our main experiments, we focus on more challenging, long-horizon, and verifiable benchmarks where current agents still exhibit substantial room for improvement. This design allows us to better assess whether an MAS evolution method can improve agent behavior beyond saturated short-form reasoning and coding tasks. Table 4: Performance of Claude Sonnet 4.6 on commonly used benchmarks. We report our own reproduced scores under standard evaluation protocols. Model HumanEval MBPP MultiArith GSM8K MATH MMLU Claude Sonnet 4.6 99.499.4 99.699.6 100.0100.0 100.0100.0 98.798.7 97.297.2 F.2 Experiment Settings on Different Benchmarks Table 5: Unified runtime settings shared across all benchmarks. Component Setting Value Backbone LLM Model Claude Sonnet 4.6 API layer LiteLLM + OpenAI-compatible gateway Retry policy 5 attempts, 1.5ā60 s exp. back-off Decoding Executor temperature 0.2 Planner temperature 0.2 Max output tokens 32768 Context window Max history messages 150 (with pair-preserving trim) Tool schema refresh every LLM call Reflection Per-phase step budget 50 Per-phase timeout 300ā900 s Per-reflection cost $50 Force-finalize Tools disabled Yes Timeout 240 s Table 6: Time and cost budget for evolution and evaluation for all benchmarks. Benchmark max_seconds max_messages max_cost ($) initial team size SWE-bench Pro 1800 600 50 3 BeyondSWE 1800 600 30 3 GAIA 600 300 30 4 LoCoBench 1200 400 30 4 LOCA-bench 3200 600 150 4 ResearchRubrics 1800 600 50 1 Table 7: SWE-bench Pro and BeyondSWE subsets used in our experiments. For each benchmark, we hold out 20 random instances per subset as the Evolution set; the remainder forms the Evaluation set. Benchmark Selected Subset Total Evolution Evaluation SWE-bench Pro Ansible 96 20 76 Qutebrowser 91 20 71 Total 187 40 147 BeyondSWE CrossRepo 200 20 180 DepMigrate 178 20 158 Total 378 40 338 SWE-bench Pro SWE-bench Pro [17] evaluates coding agents on real-world GitHub pull requests that require multi-file bug fixes or feature implementations. The public split contains 731 instances across 11 repositories, and each instance is evaluated by running the generated patch against Fail-to-Pass (F2P) and Pass-to-Pass (P2P) tests in an isolated Docker environment. An instance is counted as resolved only if all F2P tests pass and no P2P test regresses. Experimental settings. Subject to budget constraints, we evaluate on the two largest Python repositories, shown in Table 7. For each repository, we select 20 random instances as the Evolution set; the remaining instances form the Evaluation set. Evolution is performed separately for each repository and evaluated on its corresponding held-out instances. The reported metric is Pass Rate. Initial MAS configurations. SWE-bench Proās official scaffold [17] is single-agent (SWE-Agent), but repository-level patch generation is a well-studied multi-agent task: prior work such as MetaGPT [25] and ChatDev [52] decomposes software development into specialized roles (product manager, architect, engineer). We adopt a minimal instantiation of this recipeāPlanner ++ Developer ++ Reviewerāthat retains the three functions most predictive of patch quality: task decomposition, code implementation, and independent verification [47]. The planner scopes the fix and dispatches, the developer edits inside the Docker workspace, and the reviewer inspects git diff and reruns tests before submission. Single-agent baseline. Our SA replicates SWE-bench Proās officially endorsed reproducibility scaffold [17], namely SWE-Agent (Scale) [73], which is the same single-agent recipe used in the leaderboardās reference Sonnet runs. It is a function-calling agent with the canonical bash ++ str_replace_editor ++ finish tool set, executed inside the per-instance Docker workspace and submitting via git diff HEAD. We keep all decoding and tool-filter hyperparameters at the SWE-Agent benchmark config and only swap in our shared backbone. BeyondSWE BeyondSWE [11] extends software-engineering evaluation beyond single-repository bug fixing, covering tasks that require cross-repository knowledge, domain expertise, and whole-repository modifications. The full benchmark contains 500 instances from 246 Python repositories. Experimental settings. We evaluate on the two largest subsets, CrossRepo (200) and DepMigrate (178), which together cover cross-repository reasoning and whole-repository modificationāthe two capability dimensions most distinct from SWE-bench Pro, yielding 378 instances in total (Table 7). For each subset, we select 20 random instances as the Evolution set; the remaining instances form the Evaluation set. Evolution is performed separately for each subset and evaluated on its corresponding held-out instances. Evaluation follows the same F2P++P2P protocol as SWE-bench Pro. The reported metric is Pass Rate. Initial MAS configurations. Since BeyondSWE instances share the repository-level patch-generation format with SWE-bench Pro, we reuse the same Planner ++ Developer ++ Reviewer configuration, differing only in the constitution prompts that adapt the team to BeyondSWEās per-task working directory, cross-repository context, and dependency-migration conventions. Using an identical architecture across the two benchmarks isolates the effect of task distribution from that of scaffold design. Single-agent baseline. BeyondSWE shares the same patch-submission protocol as SWE-bench Pro, so for parity we reuse the SWE-Agent (Scale) [73] SA recipe described above, only adjusting the per-task working directory and the cross-repository context fields prescribed by BeyondSWE [11]. Using identical SA machinery across the two SWE benchmarks isolates the effect of task distribution and matches the convention adopted by BeyondSWEās own evaluation. LOCA-Bench LOCA-Bench [77] evaluates LLM agents under controllable context growth. It procedurally scales environment state size while keeping task semantics invariant, so the effective context grows smoothly from 8K to 256K tokens. The full benchmark contains 15 Short-to-Long (S2L) tasks Ć 5 seeds Ć 7 context lengths (8K/16K/32K/64K/96K/128K/256K) == 525 configurations. Evaluation is binary (0/1), computed by offline scripts comparing the agentās final environment state to ground truth. Experimental settings. To control evaluation cost, we select 8 out of 15 tasks covering e-commerce, academic, and education domains with 8 business MCP services (Table 8). Table 8: The eight LOCA-Bench tasks used in our experiments. # Task Total Domain Key MCP Services 1 WoocommerceNewWelcome 30 E-commerce CRM woocommerce, google_cloud, email 2 WoocommerceStockAlert 30 E-commerce purchasing woocommerce, google_sheet, email 3 FilterLowSellingProducts 30 E-commerce clearance woocommerce, email 4 ApplyPhDEmail 30 Academic email, pdf_tools 5 SetConfCrDdl 30 Academic calendar, email 6 CourseAssistant 30 Education excel, email 7 CanvasArrangeExam 30 Education canvas, email, excel 8 CanvasListTest 30 Education canvas Total 240 Leveraging LOCA-Benchās procedural generation, we construct a dually-isolated split: ā¢ Evolution set (16 cases): 8 tasks Ć 96K Ć 2 human-specified seeds 101, 102 (outside the original benchmark). ā¢ Evaluation set (240 cases): 8 tasks Ć 5 standard seeds 42, 123, 456, 789, 2024 Ć 6 context lengths 8K, 16K, 32K, 64K, 128K, 256K (deliberately excluding 96K). The (seed, context_length) tuples of the two sets are fully disjoint, guaranteeing no data leakage. This design jointly tests cross-seed and cross-scale generalization. All runs use LOCA-Benchās local mock MCP servers, requiring no Docker or external network. The reported metric is Pass Rate. Initial MAS configurations. LOCA-Bench officially ships only single-agent scaffolds, so no multi-agent baseline is available for replication. We adopt the orchestratorāworker pattern popularized by recent work [3, 61], instantiating it as Chairman ++ 3 Workers. The pattern fits LOCA-Bench naturally: every task decomposes into explore ā bulk-process ā verify, where the bulk phase (80ā300 items) exceeds a single agentās effective context [86]. The chairman additionally preserves recoverability when MCP back-ends crashāa known benchmark feature [77]āand the workers act without incurring excessive coordination overhead. Single-agent baseline. Our SA is LOCA-Benchās officially shipped default scaffold, the react strategy [77]: a single ReAct agent [75] with direct access to the full local mock-MCP tool stack (WooCommerce, Google Sheets, Calendar, Email, Canvas, Excel, etc.). We do not modify any tool wiring or context-management policy beyond the shared budget table, so the SA column corresponds to the same single-agent baseline reported in the LOCA-Bench paper, only re-evaluated under our identical backbone and budget. GAIA GAIA [48] is a knowledge-intensive question-answering benchmark that requires multi-step reasoning with tool use, such as web search, file reading, and code execution. Its public validation set contains 165 questions stratified into three difficulty levels. Experimental settings. Following [87, 45], we evaluate on the 120-samples subset released by MASFly (Table 9); the 45 excluded samples either require multimodal tools outside our search setting or are marked by annotators as unsolvable under automated exact-match grading. We use Serper API***https://serper.dev for Google Search. The returned results are capped at the top-5 entries per query and the page fetcher retrieves a specific URL and returns up to 8K characters of plain text. On these 120 questions we use 20 for evolution (training) and 100 as a disjoint held-out test set, so that every evolved configuration is evaluated on tasks it has never seen. Evaluation follows the official answer-matching protocol and the reported metric is Pass Rate. Initial MAS configurations. We initialize Meta-Team with the four-agent configuration released in the MASFly codebase [45]: a Planner that holds no information-gathering tools and coordinates through message passing, a FileAnalyzer for attached documents, a WebSearcher equipped with Google Search and page fetching, and a Summarizer that enforces the strict GAIA exact-match answer format. Starting from this hand-crafted multi-agent baseline ensures that any improvement we observe is attributable to evolution rather than to the initial design. Table 9: GAIA validation subset used in our experiments. All instances in the three levels are shuffled for evolution and evaluation. Level Full (Official) Our Subset Train Test Level 1 53 44 6 38 Level 2 86 58 9 49 Level 3 26 18 5 13 Total 165 120 20 100 Single-agent baseline. The SA follows the GAIA single-agent recipe used by [87, 45]: a function-calling ReAct agent with web_search (Serper API, top-5), web_fetch, bash, read_file, submit, the same tool surface our MAS exposes to its Web/File workers. The system prompt enforces the official GAIA exact-match answer format, so SA and MAS share both tools and answer normalizationāthe comparison isolates collaboration, not tool access. LoCoBench LoCoBench [53] is a long-context code-generation benchmark containing 8,000 scenarios across 10 programming languages, 100 synthetic projects, and 8 task categories. Each scenario provides an entire codebase ranging from 10K to 1M tokens and asks the agent to implement the given requirement. Performance is measured by the LoCoBench Score (LCBS), a weighted average of Software Engineering Excellence (40%), Functional Correctness (30%), Code Quality (20%), and Long-Context Utilization (10%), normalized to [0,100][0,100]. Experimental settings. We focus on two scenarios that are most relevant to collaborative coding: Feature Implementation and Cross-File Refactoring (Table 10). For each category, we focus on Python-based tasks to control cost. We use 20 cases for evolution and the remaining 80 cases for evaluation. In addition, we conduct out-of-domain evaluation on C, C++, and Java tasks. The reported metric is average LCBS (LoCoBench Score). Initial MAS configurations. The defining challenge of LoCoBench is long-context comprehension: each scenario ships an entire codebase (10Kā1M tokens, up to 100 files) that rarely fits in a single agentās working context. We therefore adopt a readāwrite separation inspired by divide-and-conquer long-text processing [71], instantiated as a Planner, a Developer, and two parallel Readers. The planner partitions the file list and dispatches disjoint subsets to readers; each reader extracts structured notes (APIs, call sites, coding conventions) from its assigned files and sends them directly to the developer, which synthesizes the notes into the final patch. The readers provide enough parallelism to absorb 15ā50-file scenarios while keeping each readerās context well under the modelās window. Table 10: LoCoBench subsets used in our experiments. # Task Category Total Evolution Evaluation 1 Feature Implementation 100 20 80 2 Cross-File Refactoring 100 20 80 Total 200 40 160 Single-agent baseline. LoCoBenchās official harness [53] evaluates a single LLM with full-codebase context as the reference scaffold. To keep the comparison fair under our shared cost cap (full-codebase prompting alone exhausts the per-case budget on 100Kā1M-token scenarios), our SA mirrors the SWE-Agent (Scale) [73] agentic recipeābash ++ str_replace_editor ++ finish on the same checked-out workspaceāwhich is the standard single-agent baseline used by every recent long-context coding evaluation. This grants the SA the same file-reading bandwidth as our MAS readers, so any gap reflects coordination, not tool starvation. ResearchRubrics ResearchRubrics [59] evaluates deep-research agents on 101 open-ended research questions spanning ten domains, including AI&ML, historical analysis, STEM, and creative writing. Each question is paired with a human-authored rubric covering explicit requirements, implicit requirements, synthesis, communication quality, instruction following, and citation quality. Responses are scored by an LLM judge, and the final score is a rubric-weighted average. Experimental settings. We use 20 questions as the Evolution set and the rest of questions as Evaluation set. The reported metric is Average Score on the 0ā100 scale. Initial MAS configurations. Since ResearchRubrics grades responses along six orthogonal axes and the most useful specialists are not obvious a priori, we start from a single Lead Researcher equipped with web search, page fetching, and file I/O, following the solo-start pattern of [3]. During self-evolution, when reflection identifies a persistent weakness on a specific axis, the Lead Researcher can introduce new specialists, e.g., a citation verifier when citation scores are low, a requirement tracker when explicit requirements are frequently missed, or a report writer when synthesis and communication scores lag. Single-agent baseline. The SA is the deep-research single-agent recipe from AggAgent [33]: a ReAct loop with search (Serper) and visit (page fetch with goal-conditioned extraction), and the verbatim System Prompt ++ System-Prompt-DR from the AggAgent rollout codebase, which encodes the citation, structure, and confidence requirements that ResearchRubricsā rubric grades along. We use AggAgentās reference single-agent recipe rather than a custom one so that the SA column corresponds to a published, reproducible deep-research baseline. F.3 Experiment Settings of Baseline Reproductions All baselines share the same executor LLM (Claude Sonnet 4.6 via an OpenAI-compatible gateway), tool bundle, evaluation splits, and per-case time / message / cost limits as Meta-Team. We only describe the method-specific adaptations below. SA. Vanilla ReAct [75] with the same system prompt template, tool bundle, step cap (5050), and context-trimming rule (150150 retained messages, pair-preserving) as Meta-Teamās executor. MAS. Meta-Teamās initial configuration on each benchmark with reflection and the commit gate disabled. Concretely, we zero out the L1/L2/L3 analyzers and skip the verifier call, so the team never mutates after task k. In AggAgent [33], we use the authorsā implementation of aggagent package (rollout/ ++ aggregation/). Per case we sample K=5K=5 single-agent rollouts at temperature 0.20.2, then run the default CROSS_VALIDATION aggregation strategy. In OWL [27], we keep the released PlannerāCoordinatorāWorker hierarchy unchanged with minor adaptations on each benchmark. AOrchestra (Agent-Orchestra). The MainAgent runner is used unchanged; the orchestrator is allowed to synthesize up to 44 sub-agents per task, each inheriting M=M=Claude Sonnet 4.6. Because AOrchestra hard-codes config/example/model_config.yaml, our adapter chdirs into the AOrchestra directory before import and redirects LLM calls through our gateway in LLMsConfig.default(). For AgentSquare [58], we run the search loop over Planning,Reasoning,ToolUse,Memory\ Planning, Reasoning, ToolUse, Memory\ for 3030 trials on the evolution set of each benchmark. After search, we freeze the best-performing configuration and evaluate it on the held-out split. For each benchmark, we construct a carefully curated search pool by prompting Claude Opus 4.6 to augment the baseline MAS with more than 6060 scaffold templates. Each template is further decomposed into modular components, which serve as the candidate units for AgentSquareās search procedure. Although ReCreate [22] was originally proposed for single-agent evolution, its scaffold-level optimization interface makes it naturally extensible to experience-driven MAS evolution. We therefore adapt ReCreate as a MAS evolution baseline. Specifically, we use the default hyperparameter with BATCH_SIZE=4=4 and N_REPEAT=2=2. We use Claude Sonnet 4.6 as the execution agent model, replacing the default gpt-5-mini, and Claude Sonnet 4.6 as the agent-as-optimizer model. We wrap Meta-Teamās configuration through its adapters/ entry points on benchmarks not shipped in the original repo and reuse the released adapters on SWE-bench-like domains. AgentNet [74]. We instantiate one AgentNet per benchmark with 33 seed agents and the GAIA ability vector from the paper (reasoning/knowledge/language each at 0.60.6), and pre-populate the memory store with our evolution set. Official defaults are kept: router temperature 0.70.7, DAG pruning threshold 0.30.3, memory retrieval top-k=5k=5. The episodic memory is persisted within a benchmark but cleared between benchmarks. Evolution / held-out protocol across baselines. We evaluate every method under a strict evolution/test split. Methods that include a learning phase (Meta-Team, ReCreate, MASFly, AgentNet, AgentSquare) consume only the benchmarkās designated evolution set, freeze all learned artifacts, and are then evaluated on the disjoint held-out set. ReCreate and AgentSquare support this protocol natively through --skip-instances / trainātest file flags and are used as-is. For MASFly, we rebuild the SOP Repository and Personalised Experience Pool on our own evolution set (go run . --eval 0) rather than using the shipped repository, so that no released SOP can have seen our held-out instances. AgentNetās released runner performs online router/memory updates during evaluation; we extend it with a snapshot step that freezes the routing weights, ability vectors, and episodic memory after the evolution set, disabling further updates during held-out evaluation. Methods without a learning phase (SA, MAS, AggAgent, OWL, AOrchestra) are evaluated on the held-out set directly. MASFly [45]. MASFlyās reference Go implementation is driven by go run . --eval 0 to build the SOP Repository and Personalised Experience Pool from the evolution set, then --eval 2 for held-out evaluation. We expose Meta-Teamās executor through MASFlyās OpenAI-compatible environment variables (OPENAI_BASE_URL, OPENAI_MODEL); feedback style, analyzer prompt, and SOP top-k=3k=3 retrieval are kept at the paperās defaults. Shared harness notes. Three harness-level rules apply uniformly to every baseline: (a) SWE-bench Pro / BeyondSWE patches are scored through the official harness container with empty-patch=\,=\,unresolved; (b) LOCA-Bench task setup and evaluation are wrapped in a process-level lock to avoid the os.environ race at workersā„ 8\,ā„\,8; (c) all ResearchRubrics submissions are graded by the rubric judge (Claude Opus 4.6, temperature 0.20.2).