Paper deep dive
COOP$^2$: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems
Hanqing Yang, Narjes Nourzad, Shiyu Chen, Marie Siew, Jingdi Chen, Carlee Joe-Wong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 7/20/2026, 6:16:56 AM
Summary
The paper introduces COOP^2, an evaluation framework for Large Language Model-based Multi-Agent Systems (LLM-MAS) that formalizes cooperation as a dynamic process governed by verifiable constraints (temporal, spatial, participation, dependency). It proposes a cognitive-primitive interface to link high-level agent reasoning with grounded environment actions. Additionally, it presents COOP^2-Repair, a mechanism that predicts constraint failures from group plans and initiates targeted repair channels to improve task success and constraint satisfaction.
Entities (10)
Relation Signals (9)
COOP^2-Repair → buildson → COOP^2
confidence 96% · Building on this framework, we develop COOP^2-Repair
COOP^2 → includes → Cognitive-Primitive Interface
confidence 95% · COOP2 models LLM-based multi-agent systems through two coupled levels: a cognitive layer... and a primitive layer
COOP^2-Repair → predicts → Constraint Failures
confidence 94% · COOP^2-Repair, which predicts constraint failures from group plans
COOP^2 → defines → Cooperative Constraints
confidence 93% · COOP2 then defines cooperative tasks with verifiable cooperative requirements... We instantiate four cooperation constraints: spatial, temporal, participation, and dependency
COOP^2-Repair → opens → Repair Channels
confidence 92% · opens targeted repair channels for guided revisions
COOP^2-Repair → improves → task success
confidence 90% · COOP^2-Repair improves task success and constraint satisfaction
LLM-MAS → uses → Natural Language
confidence 88% · plans, messages, and revisions occur in natural language
COOP^2 → instantiatesin → MA-CRAFTER
confidence 85% · Cooperative tasks, instantiated in MA-CRAFTER and CUBE
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Many complex tasks require extended effort, diverse capabilities, or coordinated actions beyond what a single agent can provide. However, simply adding more agents does not guarantee better performance, as effective cooperation depends on how agents interact with each other and with task structure to satisfy evolving constraints over time. This challenge is amplified for LLM-based multi-agent systems (LLM-MAS): plans, messages, and revisions occur in natural language, whereas task progress depends on grounded environment actions. Current evaluations mostly treat cooperation as an implicit ingredient of final task success, leaving both cooperation and the effect of multi-agent interaction on task dynamics difficult to study. We introduce COOP$^2$, an evaluation framework that grounds high-level agent cooperation dynamics in LLM-MAS within task progress in the environment. COOP$^2$ then defines cooperative tasks with verifiable cooperative requirements, allowing us to analyze how cooperation unfolds over time with respect to task progress, as well as where and why cooperation breaks down. Building on this framework, we develop COOP$^2$-Repair, which predicts constraint failures from group plans and opens targeted repair channels for guided revisions. Across two environments and three communication structures, COOP$^2$-Repair improves task success and constraint satisfaction while exposing the additional decision overhead and communication load required for repair. The project web page can be found at: this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2603.00349v2
- Canonical: https://arxiv.org/abs/2603.00349v2
Trouble viewing inline? Open PDF directly →
Full Text
98,783 characters extracted from source content.
Expand or collapse full text
COOP2: Defining, Observing, and Repairing Cooperation in LLM Multi-Agent Systems Hanqing Yang CMU Narjes Nourzad USC Shiyu Chen11footnotemark: 1 CMU Marie Siew SUTD Jingdi Chen UArizona Carlee Joe-Wong CMU Equal contribution. Contact: hanqing3@andrew.cmu.edu. CMU: Carnegie Mellon University; USC: University of Southern California; SUTD: Singapore University of Technology and Design; UArizona: University of Arizona. Abstract Many complex tasks require extended effort, diverse capabilities, or coordinated actions beyond what a single agent can provide. However, simply adding more agents does not guarantee better performance, as effective cooperation depends on how agents interact with each other and with task structure to satisfy evolving constraints over time. This challenge is amplified for LLM-based multi-agent systems (LLM-MAS): plans, messages, and revisions occur in natural language, whereas task progress depends on grounded environment actions. Current evaluations mostly treat cooperation as an implicit ingredient of final task success, leaving both cooperation and the effect of multi-agent interaction on task dynamics difficult to study. We introduce COOP2, an evaluation framework that grounds high-level agent cooperation dynamics in LLM-MAS within task progress in the environment. COOP2 then defines cooperative tasks with verifiable cooperative requirements, allowing us to analyze how cooperation unfolds over time with respect to task progress, as well as where and why cooperation breaks down. Building on this framework, we develop COOP2-Repair, which predicts constraint failures from group plans and opens targeted repair channels for guided revisions. Across two environments and three communication structures, COOP2-Repair improves task success and constraint satisfaction while exposing the additional decision overhead and communication load required for repair. The project web page can be found at: https://happyeureka.github.io/coop2. 1 Introduction Recent progress in large language models (LLMs) has enabled agents that reason, plan, communicate, and act in interactive environments (Sumers et al., 2023; Durante et al., 2024). As these agents are applied to more complex tasks that require long-horizon execution, diverse capabilities, or joint actions, many settings exceed what a single agent can reliably accomplish. Consider, for example, agents collaboratively building a web application from a user specification, where some components can be implemented independently, while others require cooperation across frontend, backend, database, and deployment agents. Progress may stall on failing workflows that block downstream modules, or on newly introduced bugs, dependency conflicts, and mismatches with user expectations (Tran et al., 2026; Jimenez et al., 2024; Yang et al., 2025c). Similar dynamics arise in embodied and game-playing settings, where agents must allocate work, synchronize interdependent changes, and adapt to evolving constraints over time (Nourzad et al., 2025; Savva et al., 2019). LLM-MAS systems often prescribe cooperation through workflows or encourage it through hand-crafted rewards (Liu et al., 2026; Wu et al., 2024). While effective when task structure and expected agent behaviors are well understood, such designs become limiting for complex tasks with adaptive structure, larger teams, or cross-task generalization, since adaptive behavior under evolving conditions is difficult to fully prescribe or induce through predefined components. These limitations grow more pronounced as LLM-MAS move toward autonomous settings where agents must define roles, allocate work, monitor progress, and adapt to evolving task requirements. Task outcomes then depend on how agents coordinate their actions with respect to task structure, shared state, and evolving task constraints (Kim et al., 2025), that is, on their cooperative behavior (Chen et al., 2026; Dafoe et al., 2020). Cooperation must therefore be studied as a dynamic process by which agents adapt their interactions to satisfy evolving task requirements, rather than as a system design. The cognitive–primitive structure of LLM-MAS. Studying LLM-MAS cooperation as a process is complicated by a two-layer structure absent from classical multi-agent systems (Hong et al., 2023; Chen et al., 2023; Zhuge et al., 2024). LLMs equip agents with cognitive capabilities that operate over a high-level symbolic space, while task progress unfolds through grounded actions in a lower-level environment. This separation introduces asynchrony between reasoning and execution. Agents may spend varying amounts of time planning and communicating, while committed plans execute as a sequence of primitive actions with variable durations and uncertain outcomes. Even when a central coordinator assigns subtasks, worker agents may execute them at different speeds or encounter local obstacles, forcing the coordinator to revise task allocation and interrupt agents mid-execution. Understanding cooperation in LLM-MAS, therefore, requires connecting asynchronous agent interaction to environment-level state transitions and task progress. Figure 1: Overview of COOP2. COOP2 models LLM-MAS through a cognitive–primitive interface (top) that couples a symbolic cognitive layer, where agents plan, communicate, interrupt, and replan through the Multi-Agent Environment Interaction Loop (MAEIL, bottom-left), with a primitive layer, where joint actions drive environment states and task-state transitions (sec. 3). Cooperative tasks, instantiated in MA-CRAFTER and CUBE, are guarded by participation and task-progress constraints; joint executions that satisfy all active constraints succeed while violations identify which cooperative requirements failed. These constraint signals support process-level diagnosis and feed COOP2-Repair (sec. 5), which opens targeted repair channels to revise plans before primitive execution. The evaluation gap. Even when agent interactions are connected to environment-level task progress, evaluating the cooperative process still requires formal specification of an environment’s evolving cooperation requirements. Existing multi-agent evaluations, however, mostly treat cooperation as an implicit ingredient of final task success. Many LLM-MAS benchmarks evaluate outcome-based performance on reasoning-centric tasks, such as task completion, answer accuracy, or coding fixes verified by unit tests (Zhu et al., 2025; Agashe et al., 2025; Kim et al., 2025; Qian et al., 2024). Such outcome measures cannot reveal whether agents actually cooperated, how their interactions contributed to progress, or why task failures occurred: a system may succeed through individual effort without meaningful cooperation, or fail despite partially effective cooperation when specific task requirements go unmet. Formalizing task cooperation requirements potentially allows for these cooperation failures to be detected and repaired so as to improve task success. Our Contributions: The COOP2 framework. In this paper, we introduce COOP2, a framework for making cooperation a first-class object of study in LLM-MAS. In doing so, we make four major contributions. (1) COOP2 agent trace. COOP2 defines a model of agent actions that connects high-level communication and decision-making, including agents’ symbolic activity, plans, messages, interruptions, and replanning; to grounded actions and task-process transitions. Thus, we can analyze the effects of cooperative dynamics on unfolding task progress. (2) A constraint-based cooperation formalism. COOP2 then formalizes cooperative tasks as constraint-guarded state transitions, and cooperation as the process by which agents communicate, plan, revise, and act to jointly satisfy task constraints over time. We instantiate four cooperation constraints: spatial, temporal, participation, and dependency, that provide verifiable, environment-agnostic signals for diagnosing cooperative progress. (3) COOP2-Repair. Building on this formalism, we develop COOP2-Repair, a constraint-aware communication method that monitors agents’ remaining plan feasibility and opens targeted repair channels by deciding when agents should communicate, who should participate, and which constraint should be addressed. (4) Environment instantiations and validation. Across two cooperative environments and multiple LLM backbones, team sizes, and communication structures, COOP2 supports analysis beyond final task success. COOP2-Repair demonstrates how capability- and constraint-aware prediction can guide more efficient task progress, while exposing the additional decision overhead and communication load required for repair. 2 Related Work Recent LLM-MAS frameworks use workflows, roles, automatic topology search, or multi-agent optimization to improve collaborative task solving (Gao et al., 2025; Wu et al., 2024; Hong et al., 2023; Zhuge et al., 2024; Chen et al., 2023). Existing benchmarks and environments evaluate LLM agents in embodied, collaborative, or reasoning-centric settings (Yang et al., 2025d; Zhu et al., 2025; Agashe et al., 2025; Chollet, 2019; Sun et al., 2025; Mosquera et al., 2025; Savva et al., 2019). Classical MAS studies cooperation as an interactive process (Durfee, 1993; Yokoo et al., 2002), while single-agent LLM methods study reasoning, acting, planning, and self-correction (Wei et al., 2022; Yao et al., 2022; Wang et al., 2023; Shinn et al., 2023). In contrast, COOP2 standardizes cooperation itself by defining cooperative constraints, traces language-level interaction into grounded execution, measures process-level cooperation dynamics, and repairs predicted coordination failures. Due to space constraints, we provide a more comprehensive discussion of related work in Appendix A.1. 3 Cognitive–Primitive Interface for LLM-MAS Figure 2: Decoupling cognitive and environment clocks in multi-agent execution. At each cognitive step, an agent maintains an internal state consisting of its current interaction stage and its committed symbolic plan. Plans may be newly generated (green), resumed (yellow), or terminated (red, indicating success or failure). COOP2 models LLM-based multi-agent systems through two coupled levels: a cognitive layer in which agents form symbolic plans, communicate, interrupt, and replan, and a primitive layer in which environment states evolve through primitive actions and task-state transitions. The interface aligns these two levels so that high-level agent interaction can be connected to primitive task progress. Primitive layer. Let I=1,…,nI=\1,…,n\ denote the agent set and let t∈=0,…,Tt =\0,…,T\ index primitive environment steps. We decompose the primitive state as st=(t,xg,tg∈t,st−),s_t=(G_t,\x_g,t\_g _t,s_t^-), where tG_t is the active task set, xg,tx_g,t is the state of task g (including its cooperation requirements, define in Section 4), and st−s_t^- contains remaining environment variables that may affect execution but are not directly tracked at the task level (e.g., the presence of other objects in a navigation environment). Given a joint primitive action at=(a1,t,…,an,t)a_t=(a_1,t,…,a_n,t), we consider its task-level effect: (t+1,xg,t+1g∈t+1)←at(t,xg,tg∈t) (G_t+1,\x_g,t+1\_g _t+1 ) \,a_t\, (G_t,\x_g,t\_g _t ) which may complete active tasks, activate new tasks, or modify existing tasks. Cognitive layer. The cognitive layer is the high-level space in which LLM agents reason to form and revise plans, communicate, and call tools. An agent action is represented as a tool operation a^i(θi)∈ a_i( _i)∈ A, where a^i a_i denotes the operation selected by agent i and θi _i its arguments. The symbolic tool space A may be dynamically revised, expanded, or reduced during interaction. To capture multi-agent dynamics in which decisions and interactions may occur asynchronously across agents, we use a cognitive index t^∈ t∈ T, distinct from the primitive index t and aligned to it through the grounding bridge (Eq. 2). For each agent i, we define its cognitive activity status as x^i,t^=(^i,t^,π^i,t^),^i,t^∈R,W,X,I,π^i,t^=(ρ^π^,[a^i,1(θi,1),a^i,2(θi,2),…]). x_i, t=( Q_i, t, π_i, t), Q_i, t∈\R,W,X,I\, π_i, t= ( ρ_ π,[ a_i,1( _i,1), a_i,2( _i,2),…] ). (1) Here, ^i,t Q_i, t indexes the interaction stage, which is automatically updated as agents reason (R)(R) to send or receive messages and form or revise plans, wait (W)(W) for other agents to commit their plans, execute (X)(X) their plans, or are interrupted (I)(I) by other agent messages (full stage dynamics in Appendix A.2). The plan π^i,t π_i, t contains a task specification ρ^π ρ_ π, encoding the agent’s current understanding of what task it is working toward, and a sequence of tool operations intended to satisfy that specification. Grounding bridge. The grounding bridge connects the cognitive and primitive layers by aligning their time scales, grounding symbolic tool operations into real-time primitive execution, and specifying when agents transition between them. Cognitive-to-primitive clock alignment. We align cognitive and primitive time by defining κ:^→κ: T : κ(t^)=maxt∈:t⪯t^,ℐt=[t^t,t^t+1),X^ℐt=x^i,τi∈I,τ∈ℐt,M^ℐt=mτ∈ℐt.κ( t)= \t :t t\, _t=[ t_t, t_t+1), X_I_t=\ x_i,τ\_i∈ I,τ _t, M_I_t=\m_τ\_τ _t. (2) where t⪯t^t t means primitive step t has completed by cognitive time t t, ℐtI_t is the cognitive interval between primitive steps, X^ℐt X_I_t and M^ℐt M_I_t collect the cognitive activity and communication events over that interval, and mτ=(sτ,Rτ,pτ)m_τ=(s_τ,R_τ,p_τ) denotes a communication event at cognitive time τ the sender sτs_τ, recipient set RτR_τ, and payload pτp_τ. Cognitive-to-primitive action grounding. Each tool operation is grounded by Γ into a non-deterministic sequence of primitive actions: a^i(θi)∈^,(ai,τ=t:t′,t′)∼PΓ(⋅∣a^i(θi),st,X^ℐt,M^ℐt),t′≥t. a_i( _i)∈ A, (\a_i,τ\_τ=t:t ,t ) P_ (· a_i( _i),s_t, X_I_t, M_I_t ), t ≥ t. (3) Since the unrolling depends on the evolving primitive state, the endpoint t′t cannot be known ahead of time. As a consequence, the plan containing this tool operation may be interrupted, revised, resumed, or terminated. Cognitive-to-primitive interplay. By the end of ℐtI_t, each agent carries a step-aligned plan π^i,t π_i,t obtained from the latest cognitive-time plan π^i,t π_i, t with κ(t^)=tκ( t)=t, written as π^i,t=(ρ^π^,[a^i,ki(θi,ki),a^i,ki+1(θi,ki+1),…]),π^i,t+1=Ui(π^i,t,X^ℐt,M^ℐt,st+1), π_i,t= ( ρ_ π,[ a_i,k_i( _i,k_i), a_i,k_i+1( _i,k_i+1),…] ), π_i,t+1=U_i ( π_i,t, X_I_t, M_I_t,s_t+1 ), (4) where the update UiU_i pops the selected operation if it reaches a terminal state, resumes the remaining plan if execution continues, revises the plan after interruption, or terminates it upon failure or completion. The first tool operation in the remaining list is selected for primitive execution. Let C and P denote the cognitive and primitive layers. Layer transitions are given by →⟺∀i∈I,Active(π^i,t),→⟺i∈I:¬Active(π^i,t+1)∨MSGi(ℐt)≠∅, C→ P ∀ i∈ I,\ Active( π_i,t), P→ C \i∈ I: ( π_i,t+1) _i(I_t)\≠ , (5) where ActiveActive indicates a non-terminated plan available for execution, and ¬Active covers plans that complete, fail, or become infeasible under the updated primitive state. MSGi(ℐt)MSG_i(I_t) indicates that agent i receives a message during ℐtI_t, so the nonempty set in the second condition identifies the agents that re-enter the cognitive layer asynchronously. The active plans of agents in I are grounded through Γ to form the joint action at=(a1,t,…,an,t)a_t=(a_1,t,…,a_n,t) that drives the task progress. Together, the primitive layer, cognitive layer, and grounding bridge expose how symbolic activity over T produces task-state changes over T, providing the substrate on which we define and analyze cooperation in the next section. 4 Cooperative Tasks and Cooperation Processes The cognitive–primitive interface grounds symbolic tool operations into primitive actions that change task structure. Building on this interface, we define cooperation as a process over task states, agents capability, and cooperative constraint satisfaction. Cooperative task. We model a cooperative task as a multi-agent task whose progress is guarded by cooperative constraint satisfaction. For each active task g∈tg _t, let xg,t∈gx_g,t _g denote its task state, ℐg,t⊆II_g,t I the agents participating in g, and Zg,t=zi,ti∈ℐg,tZ_g,t=\z_i,t\_i _g,t their group capability, where zi,tz_i,t represents the capability state of agent i. We define pg,tc=Pr(Cg,tc∣xg,t,Zg,t),c∈g=Δt,Δℓ,n,d.p_g,t^c= \! (C_g,t^c x_g,t,Z_g,t ), c _g=\ t, ,n,d\. (6) Here Cg,tcC_g,t^c denotes the task-specific cooperative constraint of type c for task g at step t, and pg,tcp_g,t^c is the probability that the participating group satisfies this constraint. The set gC_g contains four cooperative constraint types: temporal, spatial or relational, participation or capability, and dependency constraints, respectively, each acting as a prerequisite for task progress. For instance, temporal constraints may require agents to simultaneously take the same action (e.g., to lift an object), spatial or relational constraints may require agents to be in specific locations at the same time, participation or capability constraints require agent capabilities needed to accomplish a task (e.g., agents with minimum combined context lengths needed to split and read a long document), and dependency constraints may require agents to have access to certain tools to complete a task. These constraints are general for describing cooperative decision-making across environments and team size, and provide verifiable signals for evaluating cooperation in terms of how constraint satisfaction shapes task progress. Cooperation process. We define the cooperation process at two levels. At the single-task level, task progress is monitored over the task’s active span under grounded multi-agent execution: xg,t+1∼Pr(⋅∣xg,t,pg,tcc∈g),ϕg,t=(xg,t,ℐg,t,Zg,t,pg,tcc∈g),t∈Tg.x_g,t+1 (· x_g,t,\p_g,t^c\_c _g), _g,t=(x_g,t,I_g,t,Z_g,t,\p_g,t^c\_c _g), t∈ T_g. (7) Here Tg=[tgstart,tgend]T_g=[t_g^start,t_g^end] denotes the active span of task g, from its generation until xg,tx_g,t reaches a terminal state (completion, failure, or removal). The evolution of ϕg,t _g,t over t∈Tgt∈ T_g defines the task-level cooperation process. At the episode level, we collect all active tasks and their cooperative states as t=g:t∈Tg,Φt=(t,ϕg,tg∈t),Lt=(X^ℐt,M^ℐt,at,Φt,Φt+1).G_t=\g:t∈ T_g\, _t= (G_t,\ _g,t\_g _t ), L_t=( X_I_t, M_I_t,a_t, _t, _t+1). (8) Here tG_t is the active task set, Φt _t is the episode-level cooperative state, and LtL_t logs the transition from cognitive-layer activity (X^ℐt,M^ℐt)( X_I_t, M_I_t) to primitive-layer action and task transitions (at,Φt,Φt+1)(a_t, _t, _t+1). 5 COOP2-Repair COOP2-Repair uses the COOP2 trace to guide communication before primitive execution. At each primitive step t, agents first operate in the cognitive layer until all active agents commit a plan. COOP2-Repair then groups committed plans by their task specifications and predicts whether each group is likely to satisfy the cooperative constraints required by its intended task. Feasibility check. Following Eq. (4), each step-aligned plan π^i,t π_i,t contains a task specification and a remaining sequence of tool operations. In COOP2-Repair, we take the task specification to be the intended task index, so ρ^π^i,t∈t ρ_ π_i,t _t. The group pursuing task g and the corresponding group plans are ℐg,t=i∈I:ρ^π^i,t=g,Π^g,t=π^i,ti∈ℐg,t.I_g,t=\i∈ I: ρ_ π_i,t=g\, _g,t=\ π_i,t\_i _g,t. (9) COOP2-Repair predicts the constraint likelihoods in Eq. (6) under these plans, denoted by p^g,tc p_g,t^c for c∈gc _g. The predicted failing constraints are ^g,t−=c∈g:p^g,tc<τc, C_g,t^-=\c _g: p_g,t^c< _c\, with thresholds τc∈[0,1] _c∈[0,1]. If ^g,t−≠∅ C_g,t^-≠ , the group pursuing task g is predicted to be unlikely to complete it, and ^g,t− C_g,t^- identifies which cooperative constraints are expected to fail. Adaptive repair communication. When failure is predicted for a task g COOP2-Repair opens a temporary repair channel initialized with the group pursuing g: ℛg,t=ℐg,t,ctxg,t=(g,ℐg,t,^g,t−).R_g,t=I_g,t, _g,t= (g,I_g,t, C_g,t^- ). (10) Here ctxg,tctx_g,t identifies the task, the group, and the cooperative constraints that the group’s remaining plans are expected to violate. Agents in ℛg,tR_g,t communicate and replan around the identified task and failure reason, and may contact agents outside the current channel. Contacted agents are interrupted and added to ℛg,tR_g,t. The resulting messages become part of M^ℐt M_I_t, and plan revisions are captured by the active-plan update UiU_i in Eq. (4). After repair, task g is placed on cooldown, during which COOP2-Repair does not open another repair channel for the same task. Agents interrupted during a repair re-enter the cognitive layer to update their plans before the next primitive step. COOP2 is intended as a general framework rather than a proposal for a specific learning model; its core contribution is to show that the COOP2 representation can support both diagnosis and interaction guidance. COOP2-Repair instantiates this idea with lightweight heuristic estimators. Learning a general constraint-satisfaction predictor is a promising direction and is left for future work. 6 Cooperative Environment Instantiations We instantiate COOP2 in two multi-agent environments with distinct cooperative task structures. In both environments, each task g has a score r(g)r(g) proportional to the number and difficulty of its cooperative constraints. Figure 3: Two cooperative environment instantiations. CUBE: agents push weighted blocks where heavier blocks require more participants in spatial alignment. MA-Crafter: agents collect resources whose difficulty scales with prerequisites, tools, and required participants. Agents aim to maximize the total score of completed tasks within a bounded episode, which terminates when either the environment-step budget T or the wall-clock budget T T is reached, whichever comes first. The wall-clock budget makes cognitive activity and cooperation overhead consequential: time spent reasoning, communicating, waiting, or replanning reduces the remaining opportunity for task completion. The episode score is defined over the effective horizon induced by the environment-step and wall-clock budgets: R(τ)=∑t=0T⋆∑g∈tr(g)Done(g),T⋆=minT,κ(T^).R(τ)= _t=0^T _g _tr(g)\,Done(g), T = \T,κ( T)\. Environment instantiations. We build on MA-Crafter (Yang et al., 2025a), a multi-agent extension of Crafter (Hafner, 2021), and CUBE (Yang et al., 2025b), a cooperative block-pushing environment for LLM agents, shown in Figure 3. Table 1: Cooperation constraint specifications for task g at time t in MA-Crafter and CUBE. Constraint MA-Crafter CUBE Cni(g,t)C_n^i(g,t) capi(t)⪰cap(g,t)cap_i(t) (g,t) capi(t)cap_i(t) CΔℓi(g,t)C_ ^i(g,t) ∥i,t−yg∥≤d _i,t-y_g ≤ d miny∈g∥i,t−y∥≤1 _y _g _i,t-y ≤ 1 CΔti(g,t)C_ t^i(g,t) ai,t=collect(g)a_i,t= collect(g) ai,t=push(b(g))a_i,t= push(b(g)) In MA-Crafter, agents explore, collect resources, and craft tools through a technology tree. We define each collectible resource active at time t as a task g∈tg _t. Advanced resources receive higher scores because they require more prerequisites, stronger tools, and more valid participants. As summarized in Table 1, an agent counts as a valid participant only if it has the required capability, is within distance d of the resource, and executes the corresponding collect action. This setting emphasizes capability acquisition, dependency satisfaction, and synchronized execution. In CUBE, agents push weighted blocks to a goal region. We treat each available block side at time t as a task g∈tg _t, corresponding to pushing block b(g)b(g) from side s(g)∈(b(g))s(g) (b(g)), with participation threshold p(g)=wb(g)p(g)=w_b(g). Blocks with larger weights receive higher scores because they impose stronger spatial-alignment and participation constraints. An agent counts as a valid participant only if it is adjacent to the relevant side of the block and executes push(b(g)). This setting emphasizes spatial alignment, simultaneous action, and multi-agent participation. Environment details are provided in Appendices C.1 and C.3. Both environments use the task participation constraint p(g)p(g) to determine how many agents must satisfy each required constraint. Specifically, N(Cc)(g,t)=∑i∈I[Cci(g,t)]N(C_c)(g,t)= _i∈ II[C_c^i(g,t)] is the number of agents that satisfy constraint c for task g at step t. In the next section, we explain our design of experiments that use COOP2 to examine agent cooperation in these environments. 7 Experiment Design We design experiments that exercise COOP2 across multiple agent backbones, communication structures, and team sizes, allowing us to evaluate cooperation beyond final task performance and to measure the cost and quality of cooperation under different system conditions. Settings. We vary four factors: environment, LLM backbone, communication structure, and team size. We evaluate the two environments described in Sec. 6. For backbones, we compare GPT-5.4-mini, GPT-5.4, and Llama-4-Scout-17B-16E, covering smaller and larger frontier models as well as an open-source model, to test how model scale and type affect cooperative progress, inference time, and failure modes. For communication, we compare three settings: Individual, where agents act independently; Centralized, where a leader coordinates workers; and Chain, where agents discuss in order before committing plans. We evaluate both 3-agent and 6-agent teams to study how cooperation changes with team size and communication structure. This gives 3636 unique settings in total each repeated 55 times; we report mean and standard deviation. Recorded outcomes. For each run, we record task progress, cognitive activity, communication, and constraint satisfaction from the COOP2 specification defined in Sec. 4. We summarize cooperation with eight aggregate quantities: Score (cumulative task value), Steps (episode length), and Score/Step (task value per step), which measure task progress; Plans/Agent (average committed plans per agent), which measures cognitive activity; Msg. (total messages exchanged), Intr. (interruptions), and Total Dec. (wall-clock time spent reasoning or handling interruptions), which measure communication; and finally constraint violation deficits by type. Constraint violations are computed over targeted task-attempt events and report average deficits for the constraints defined in Sec. 6; lower values indicate better constraint satisfaction. Full metric definitions are provided in Appendix A.3. 8 Main Results We focus the main result table on MA-Crafter, which contains richer dependency constraints, a larger action space, and a more open-ended structure. Table 2 reports the MA-Crafter results; the corresponding CUBE results are provided in Appendix D.1. Table 2: MA-Crafter recorded outcomes for different agent-model settings and communication structures. Values are mean ± standard deviation over five runs. Arrows indicate better direction when a clear direction exists; bold marks the best value within each agent-model setting. Structure Score ↑ Steps Score/ Step ↑ Plans/ Agent Msg. Intr. Total Dec. Spat. Viol. ↓ Temp. Viol. ↓ Dep. Viol. ↓ 3-agent GPT-5.4-mini Individual 146.8±141.6146.8± 141.6 117.4±13.0117.4± 13.0 1.26±1.291.26± 1.29 42.8±1.242.8± 1.2 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 196.1±8.1196.1± 8.1 0.10±0.070.10± 0.07 0.01±0.020.01± 0.02 0.23±0.230.23± 0.23 Centralized 39.4±24.339.4± 24.3 88.4±12.188.4± 12.1 0.45±0.300.45± 0.30 33.7±0.933.7± 0.9 88.4±6.988.4± 6.9 34.6±5.634.6± 5.6 251.9±5.3251.9± 5.3 0.12±0.080.12± 0.08 0.01±0.010.01± 0.01 0.57±0.170.57± 0.17 Chain 29.4±5.729.4± 5.7 61.8±11.961.8± 11.9 0.49±0.100.49± 0.10 33.3±1.533.3± 1.5 57.4±3.057.4± 3.0 48.0±4.948.0± 4.9 200.0±3.7200.0± 3.7 0.05±0.050.05± 0.05 0.00±0.000.00± 0.00 0.44±0.260.44± 0.26 3-agent Llama-Scout Individual 30.4±8.030.4± 8.0 62.6±7.162.6± 7.1 0.48±0.110.48± 0.11 33.3±2.233.3± 2.2 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 242.9±9.6242.9± 9.6 0.04±0.040.04± 0.04 0.01±0.020.01± 0.02 0.24±0.190.24± 0.19 Centralized 24.0±6.624.0± 6.6 59.4±3.959.4± 3.9 0.41±0.110.41± 0.11 26.4±2.526.4± 2.5 50.4±10.450.4± 10.4 13.6±7.013.6± 7.0 244.1±15.5244.1± 15.5 0.08±0.090.08± 0.09 0.01±0.020.01± 0.02 0.25±0.110.25± 0.11 Chain 18.4±5.918.4± 5.9 29.4±8.729.4± 8.7 0.63±0.090.63± 0.09 18.7±5.518.7± 5.5 29.6±7.329.6± 7.3 19.4±5.219.4± 5.2 179.6±35.7179.6± 35.7 0.11±0.110.11± 0.11 0.00±0.000.00± 0.00 0.03±0.060.03± 0.06 3-agent GPT-5.4 Individual 809.0±405.8809.0± 405.8 99.8±10.099.8± 10.0 8.22±4.258.22± 4.25 17.8±1.217.8± 1.2 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 170.5±7.1170.5± 7.1 0.43±0.060.43± 0.06 0.04±0.050.04± 0.05 0.24±0.090.24± 0.09 Centralized 266.8±174.1266.8± 174.1 68.2±3.968.2± 3.9 3.96±2.673.96± 2.67 13.5±0.313.5± 0.3 36.8±7.036.8± 7.0 19.4±2.919.4± 2.9 220.7±12.0220.7± 12.0 0.36±0.080.36± 0.08 0.00±0.010.00± 0.01 0.11±0.070.11± 0.07 Chain 181.6±84.5181.6± 84.5 54.2±3.954.2± 3.9 3.37±1.583.37± 1.58 14.3±0.514.3± 0.5 26.8±1.326.8± 1.3 22.0±2.122.0± 2.1 201.3±7.3201.3± 7.3 0.37±0.050.37± 0.05 0.01±0.020.01± 0.02 0.07±0.060.07± 0.06 6-agent GPT-5.4-mini Individual 86.8±34.586.8± 34.5 82.2±4.282.2± 4.2 1.06±0.451.06± 0.45 33.4±2.133.4± 2.1 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 322.4±8.2322.4± 8.2 0.11±0.040.11± 0.04 0.00±0.010.00± 0.01 0.38±0.270.38± 0.27 Centralized 141.8±89.2141.8± 89.2 63.4±8.363.4± 8.3 2.18±1.312.18± 1.31 23.7±0.623.7± 0.6 127.4±38.2127.4± 38.2 61.4±16.761.4± 16.7 401.9±51.3401.9± 51.3 0.13±0.050.13± 0.05 0.02±0.020.02± 0.02 0.25±0.120.25± 0.12 Chain 27.4±6.127.4± 6.1 22.6±1.322.6± 1.3 1.20±0.221.20± 0.22 16.8±1.216.8± 1.2 75.2±6.075.2± 6.0 61.2±9.961.2± 9.9 332.9±18.1332.9± 18.1 0.13±0.080.13± 0.08 0.06±0.060.06± 0.06 0.38±0.130.38± 0.13 6-agent Llama-Scout Individual 32.6±8.032.6± 8.0 53.2±3.353.2± 3.3 0.61±0.120.61± 0.12 35.2±3.035.2± 3.0 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 498.5±39.5498.5± 39.5 0.09±0.040.09± 0.04 0.02±0.040.02± 0.04 0.30±0.200.30± 0.20 Centralized 50.8±6.550.8± 6.5 49.6±4.349.6± 4.3 1.03±0.181.03± 0.18 27.0±2.827.0± 2.8 115.2±52.8115.2± 52.8 43.2±27.943.2± 27.9 484.5±33.6484.5± 33.6 0.18±0.120.18± 0.12 0.01±0.010.01± 0.01 0.16±0.120.16± 0.12 Chain 19.8±11.919.8± 11.9 14.2±3.814.2± 3.8 1.24±0.601.24± 0.60 13.1±1.513.1± 1.5 60.6±6.260.6± 6.2 49.4±6.449.4± 6.4 386.0±12.5386.0± 12.5 0.13±0.100.13± 0.10 0.00±0.000.00± 0.00 0.05±0.090.05± 0.09 6-agent GPT-5.4 Individual 691.4±303.5691.4± 303.5 68.4±3.768.4± 3.7 9.92±3.999.92± 3.99 11.7±1.311.7± 1.3 0.0±0.00.0± 0.0 0.0±0.00.0± 0.0 215.0±9.7215.0± 9.7 0.47±0.040.47± 0.04 0.01±0.010.01± 0.01 0.17±0.080.17± 0.08 Centralized 230.4±167.0230.4± 167.0 46.0±3.046.0± 3.0 4.83±3.334.83± 3.33 9.8±0.49.8± 0.4 58.0±6.958.0± 6.9 35.8±5.135.8± 5.1 326.5±16.5326.5± 16.5 0.43±0.060.43± 0.06 0.02±0.020.02± 0.02 0.06±0.070.06± 0.07 Chain 34.6±14.834.6± 14.8 21.2±3.721.2± 3.7 1.63±0.631.63± 0.63 8.0±0.58.0± 0.5 36.6±2.036.6± 2.0 30.8±2.730.8± 2.7 349.9±17.0349.9± 17.0 0.25±0.040.25± 0.04 0.00±0.000.00± 0.00 0.00±0.000.00± 0.00 Stronger models progress better. GPT-5.4 achieves substantially higher Score and Score/Step than GPT-5.4-mini and Llama-Scout, both smaller models, in most matched settings. GPT-5.4 reaches a Score roughly 5×5× that of GPT-5.4-mini (809.0809.0 vs. 146.8146.8) and over 25×25× that of Llama-Scout (30.430.4). The gap persists at 6 agents, with GPT-5.4 outperforming GPT-5.4-mini by ∼8× 8× and Llama-Scout by ∼20× 20×. This suggests that stronger backbones produce more grounded and executable plans that better account for task feasibility, satisfy capability requirements, and adapt from low-level resource collection toward higher-value tasks as capabilities evolve. More agents are not always better. Increasing team size from 3 to 6 agents does not monotonically improve Score or Score/Step. Although larger teams provide more capacity, they also enlarge the error space through collection conflicts and decision overhead. Even for GPT-5.4 in Individual mode, where there is no communication by definition, scaling from 3 to 6 agents reduces Score by ∼15% 15\% (809.0→691.4809.0→ 691.4), as additional agents compete for the same resources without coordinating. In the Centralized and Chain communication structures, the 6 agents exchange nearly twice as many messages as the 3 agents for most settings, with corresponding increases in interruptions. Under the wall-clock budget T, this overhead can reduce completed environment steps, especially when agents are interrupted during execution. Communication is not always cooperation. The Centralized and Chain structures introduce more messages, interruptions, and decision time, but this additional process activity does not uniformly improve outcomes. For weaker backbones, Centralized communication can recover or improve Score over Individual baselines: 6-agent GPT-5.4-mini gains ∼60% 60\% (86.8→141.886.8→ 141.8). For the strongest backbone, communication overhead instead reduces Score: 3-agent GPT-5.4 drops by ∼67% 67\% under Centralized (809.0→266.8809.0→ 266.8), as the model’s independent plans are already grounded enough that synchronization adds cost without proportional benefit. Decision time confirms this overhead: 6-agent Centralized configurations require ∼25 25–50%50\% more Total Dec. than Individual. Constraint violations expose distinct failure modes. Aggregate scores conceal which cooperative requirements break down. Dependency violations track model strength: GPT-5.4-mini suffers severe dependency failures (up to 93%93\% in 3-agent Centralized), while GPT-5.4 reduces these by an order of magnitude (down to ∼7% 7\%–11%11\% in matched settings). Spatial violations, in contrast, remain substantial across all backbones, reaching 3636–43%43\% even for GPT-5.4. This suggests spatial coordination is a failure mode largely separate from capability and dependency reasoning. Temporal violations, in contrast, stay low across the board (<5%<5\%). These aggregate outcomes help reveal cooperation by comparing models, team sizes, and communication structures, but they do not show how successes and failures arise over time. We next use COOP2 traces to inspect how cooperation unfolds over time and how it correlates to task progress. 8.1 COOP2 Trace Figure 4: COOP2 process traces for MA-Crafter with 3-agent teams, averaged over five runs. Columns compare backbones, and rows compare communication structures. The blue heatmap in each cell shows the number of agents whose active plans target each resource or tool at each environment step. The black curve shows cumulative team score with shaded 95% confidence intervals across runs, and the red dashed line marks the mean episode termination step under the cognitive-time budget T. The top of each cell reports observed spatial, temporal, and dependency satisfaction for attempted collection actions, indicating when and where cooperation succeeds or breaks down during execution. Figure 4 illustrates an example process-level view enabled by COOP2. While the trace can be further elaborated to inspect per-agent or per-task behavior, here we visualize agents’ active task focus, observed constraint satisfaction at collection attempts, and cumulative team score over time. Our goal is to explain how the aggregate outcomes in Table 2 arise from the underlying execution dynamics. Individual reveals model capability. The Individual structure does not require communication: agents avoid message passing and plan interruptions, independently forming and executing plans. This allows them to complete more environment steps within the T T budget, creating more opportunities for resource collection. However, the traces show that smaller models often spend these steps on immediately available resources, especially wood, rather than progressing through the prerequisite chain (wood→stone→coal→iron→diamond, plus toolswood , plus tools). GPT-5.4 instead shifts from low-level resources toward tools and higher-value resources, suggesting that stronger backbones produce more grounded plans and better adapt objectives as agent capabilities evolve. Centralized requires a capable coordinator. In the Centralized setting, GPT-5.4-mini shows more incomplete constraint satisfaction, especially for dependency-rich tasks: the leader may assign tasks that workers cannot yet execute, and workers may follow these assignments despite missing tools or prerequisites. GPT-5.4 instead shifts earlier toward advanced resources and shows fewer dependency failures, suggesting that centralized communication is useful only when the coordinator can identify feasible shared objectives and route them around constraint requirements. Chain aligns plans but delays execution. The Chain structure shows more similar score progression across episodes because later agents condition on earlier messages and make more aligned task choices. However, ordered discussion also increases waiting, since the environment cannot advance until the chain completes, and agents tend to follow earlier speakers, making them less likely to shift toward new resources or higher-value tasks. As a result, Chain often completes fewer environment steps, especially for slower backbones, which limits task progress under the wall-clock budget. The traces reveal a tradeoff obscured by aggregate outcomes: planning quality versus execution opportunity. Stronger models produce more grounded plans, but slower reasoning can reduce completed environment steps under the cognitive-time budget. COOP2 traces expose this tradeoff by linking cognitive activity, grounded execution, and constraint satisfaction over time. 8.2 COOP2-Repair Figure 5: COOP2-Repair process trace for a centralized 3-agent GPT-5.4-mini run. Gold markers indicate steps where COOP2 predicts an upcoming constraint failure and opens a targeted repair round. The top rows show observed satisfaction across the spatial, temporal, and dependency constraints. Figure 5 shows a centralized 3-agent GPT-5.4-mini run with COOP2-Repair. COOP2-Repair predicts and monitors whether agents’ joint plans are likely to satisfy the constraints of their target tasks given current capabilities, and opens targeted repair rounds when failure is predicted. The first repair event triggers when agents have accumulated enough resources to advance but their plans remain focused on low-level collection; the subsequent shift in active plans from wood collection toward tool crafting reflects this intervention. After this round, agents begin collecting stone, and later repair guidance helps them unlock more advanced resources such as coal, producing the sharp increase in cumulative score near the end of the episode. 9 Conclusion We introduce COOP2COOP^2, an evaluation framework for studying cooperation in LLM-based multi-agent systems by making it an observable process. By exposing the interplay of cooperation signals in agents’ cognitive activity, environment execution, and cooperative constraints, COOP2COOP^2 shifts evaluation beyond end-task success toward interpretable cooperation dynamics and customizable metrics over interaction traces, surfacing tradeoffs between planning quality and execution opportunity that aggregate metrics obscure. Building on this representation, COOP2COOP^2-Repair predicts constraint failures and opens targeted repair channels, enabling improved cooperative task solving. COOP2 makes cooperation itself an object of study in LLM-MAS through grounded, process-level interaction dynamics and task design. Acknowledgments and Disclosure of Funding This work was supported in part by the Office of Naval Research under grant N000142412073 and the National Science Foundation under grants CNS-2533813 and CNS-2312761. Marie Siew was supported by the SUTD-MOE Early Career Award under the Singapore Ministry of Education START Scheme. In addition, we would like to thank Tania Lorido-Botran for her early discussions and contributions to the project, particularly on cooperation among LLM agents. References S. Agashe, Y. Fan, A. Reyna, and X. E. Wang (2025) Llm-coordination: evaluating and analyzing multi-agent coordination abilities in large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 8038–8057. Cited by: §A.1, §1, §2. J. Chen, H. Yang, Z. Liu, and C. Joe-Wong (2026) The five ws of multi-agent communication: who talks to whom, when, what, and why - a survey from MARL to emergent language and LLMs. Transactions on Machine Learning Research. Note: Survey Certification External Links: ISSN 2835-8856, Link Cited by: §1. W. Chen, Y. Su, J. Zuo, C. Yang, C. Yuan, C. Chan, H. Yu, Y. Lu, Y. Hung, C. Qian, et al. (2023) Agentverse: facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, Cited by: §A.1, §A.1, §1, §2. F. Chollet (2019) On the measure of intelligence. arXiv preprint arXiv:1911.01547. Cited by: §A.1, §2. A. Dafoe, E. Hughes, Y. Bachrach, T. Collins, K. R. McKee, J. Z. Leibo, K. Larson, and T. Graepel (2020) Open problems in cooperative ai. arXiv preprint arXiv:2012.08630. Cited by: §1. Z. Durante, Q. Huang, N. Wake, R. Gong, J. S. Park, B. Sarkar, R. Taori, Y. Noda, D. Terzopoulos, Y. Choi, et al. (2024) Agent ai: surveying the horizons of multimodal interaction. arXiv preprint arXiv:2401.03568. Cited by: §1. E. H. Durfee (1993) Cooperative distributed problem solving between (and within) intelligent agents. In Neuroscience: From Neural Networks to Artificial Intelligence: Proceedings of a US-Mexico Seminar held in the city of Xalapa in the state of Veracruz on December 9–11, 1991, p. 84–98. Cited by: §A.1, §2. H. Gao, J. Geng, W. Hua, M. Hu, X. Juan, H. Liu, S. Liu, J. Qiu, X. Qi, Y. Wu, et al. (2025) A survey of self-evolving agents: on path to artificial super intelligence. arXiv preprint arXiv:2507.21046. Cited by: §A.1, §2. D. Hafner (2021) Benchmarking the spectrum of agent capabilities. arXiv preprint arXiv:2109.06780. Cited by: §6. S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, et al. (2023) MetaGPT: meta programming for a multi-agent collaborative framework. In The twelfth international conference on learning representations, Cited by: §A.1, §A.1, §1, §2. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. R. Narasimhan (2024) SWE-bench: can language models resolve real-world github issues?. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1. Y. Kim, K. Gu, C. Park, C. Park, S. Schmidgall, A. A. Heydari, Y. Yan, Z. Zhang, Y. Zhuang, M. Malhotra, et al. (2025) Towards a science of scaling agent systems. arXiv preprint arXiv:2512.08296. Cited by: §A.1, §1, §1. S. Liu, Z. Liang, X. Lyu, and C. Amato (2026) Llm collaboration with multi-agent reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 32150–32158. Cited by: §1. M. Mosquera, J. S. Pinzon, Y. Fonseca, M. Ríos, N. Quijano, L. F. Giraldo, and R. Manrique (2025) Can llm-augmented autonomous agents cooperate? an evaluation of their cooperative capabilities through melting pot. IEEE Transactions on Artificial Intelligence. Cited by: §A.1, §A.1, §2. N. Nourzad, H. Yang, S. Chen, and C. Joe-Wong (2025) DR. well: dynamic reasoning and learning with symbolic world model for embodied llm-based multi-agent collaboration. arXiv preprint arXiv:2511.04646. Cited by: §1. C. Qian, Z. Xie, Y. Wang, W. Liu, K. Zhu, H. Xia, Y. Dang, Z. Du, W. Chen, C. Yang, et al. (2024) Scaling large language model-based multi-agent collaboration. arXiv preprint arXiv:2406.07155. Cited by: §A.1, §1. M. Savva, A. Kadian, O. Maksymets, Y. Zhao, E. Wijmans, B. Jain, J. Straub, J. Liu, V. Koltun, J. Malik, et al. (2019) Habitat: a platform for embodied ai research. In Proceedings of the IEEE/CVF international conference on computer vision, p. 9339–9347. Cited by: §A.1, §A.1, §1, §2. N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36, p. 8634–8652. Cited by: §A.1, §2. T. Sumers, S. Yao, K. Narasimhan, and T. Griffiths (2023) Cognitive architectures for language agents. Transactions on Machine Learning Research. Cited by: §1. H. Sun, S. Zhang, L. Niu, L. Ren, H. Xu, H. Fu, F. Zhao, C. Yuan, and X. Wang (2025) Collab-overcooked: benchmarking and evaluating large language models as collaborative agents. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 4922–4951. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §A.1, §A.1, §2. H. Tran, L. Nashold, R. Krishnan, A. Bigeard, and A. Gu (2026) Vibe code bench: evaluating ai models on end-to-end web application development. External Links: 2603.04601, Link Cited by: §1. L. Wang, W. Xu, Y. Lan, Z. Hu, Y. Lan, R. K. Lee, and E. Lim (2023) Plan-and-solve prompting: improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091. Cited by: §A.1, §2. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, p. 24824–24837. Cited by: §A.1, §2. Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. (2024) Autogen: enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling, Cited by: §A.1, §A.1, §1, §2. H. Yang, J. Chen, M. Siew, T. L. Botran, and C. Joe-Wong (2025a) LLM-powered decentralized generative agents with adaptive hierarchical knowledge graph for cooperative planning. In The First MARW: Multi-Agent AI in the Real World Workshop at AAAI 2025, Cited by: §6. H. Yang, N. Nourzad, S. Chen, and C. Joe-Wong (2025b) CUBE: collaborative multi-agent block-pushing environment for collective planning with llm agents. In Workshop on Scaling Environments for Agents, Cited by: §C.3, §6. J. Yang, C. E. Jimenez, A. L. Zhang, K. Lieret, J. Yang, X. Wu, O. Press, N. Muennighoff, G. Synnaeve, K. R. Narasimhan, D. Yang, S. I. Wang, and O. Press (2025c) SWE-bench multimodal: do ai systems generalize to visual software domains?. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1. R. Yang, H. Chen, J. Zhang, M. Zhao, C. Qian, K. Wang, Q. Wang, T. V. Koripella, M. Movahedi, M. Li, et al. (2025d) Embodiedbench: comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint arXiv:2502.09560. Cited by: §A.1, §2. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao (2022) React: synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, Cited by: §A.1, §A.2, §2. M. Yokoo, E. H. Durfee, T. Ishida, and K. Kuwabara (2002) The distributed constraint satisfaction problem: formalization and algorithms. IEEE Transactions on knowledge and data engineering 10 (5), p. 673–685. Cited by: §A.1, §2. K. Zhu, H. Du, Z. Hong, X. Yang, S. Guo, Z. Wang, Z. Wang, C. Qian, X. Tang, H. Ji, et al. (2025) MultiAgentBench: evaluating the collaboration and competition of llm agents. arXiv preprint arXiv:2503.01935. Cited by: §A.1, §1, §2. M. Zhuge, W. Wang, L. Kirsch, F. Faccio, D. Khizbullin, and J. Schmidhuber (2024) Gptswarm: language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, Cited by: §A.1, §A.1, §1, §2. Appendix The appendix is organized into four clusters: Cluster A (Framework Details) extends the formal framework from Sections 3-5 with MAEIL stage dynamics, complete metric definitions, and extended related work. Cluster B (Implementation Details) describes the threaded execution loop and its scalability to large teams, the structured LLM interface, the role prompts for each communication structure, and timeline visualizations. Cluster C (Environments) documents the two cooperative environments, MA-Crafter and CUBE described in section 6, including their task structures, action spaces, and environment prompts. Cluster D (Additional Results) provides additional COOP2 traces and qualitative annotated examples of agent communication , traces beyond those in the main paper and qualitative annotated examples of agent communication. Appendix A Framework details This cluster extends the formal framework introduced in Sections 3 - 5 with detailed dynamics, full metric definitions, and a comprehensive positioning against prior work. A.1 Extended Related Works We provide extended discussion of how COOP2 relates to prior work along three dimensions: how LLM-MAS are built and structured, how cooperation has been formalized and abstracted, and how multi-agent systems are evaluated. LLM-MAS frameworks and multi-agent environments. Recent work has explored LLM-MAS through human-defined workflows, automatic workflow discovery, and multi-agent optimization (Gao et al., 2025). LLM-MAS frameworks such as AutoGen (Wu et al., 2024), MetaGPT (Hong et al., 2023), GPT-Swarm (Zhuge et al., 2024), and AgentVerse (Chen et al., 2023) prescribe cooperation implicitly through workflows, roles, or prompts. Multi-agent environments such as Collab-Overcooked (Sun et al., 2025), Melting Pot (Mosquera et al., 2025), and Habitat (Savva et al., 2019) define cooperative success through environment-specific rules and dynamics. While these systems support a wide range of cooperative tasks, the specification of when cooperation is required and how it is satisfied varies across systems and environments, making it difficult to compare cooperative behavior across settings or to disentangle cooperation from individual task performance. COOP2 complements these systems by introducing a constraint-guarded representation of cooperative tasks that makes cooperation requirements explicit and verifiable across environments. Cooperative decision-making, classical and recent. Classical work on cooperative distributed problem solving views cooperation as a temporally extended interactive process among agents (Durfee, 1993; Yokoo et al., 2002). More recently, single-agent LLM methods such as Chain-of-Thought (Wei et al., 2022), ReAct (Yao et al., 2022), Plan-and-Solve (Wang et al., 2023), and Reflexion (Shinn et al., 2023) provide cognitive abstractions for reasoning, acting, planning, and self-correction, but operate over a single agent’s deliberation rather than multi-agent cooperation. Existing LLM-MAS frameworks focus on high-level symbolic interaction (Wu et al., 2024; Hong et al., 2023; Zhuge et al., 2024; Chen et al., 2023), while embodied environments emphasize low-level execution and state transitions (Sun et al., 2025; Mosquera et al., 2025; Savva et al., 2019). This separation between the symbolic and grounded views of multi-agent behavior leaves a gap in connecting language-level coordination to environment-level task progress, which COOP2’s cognitive–primitive interface is designed to bridge. Evaluation of multi-agent systems. Several LLM-MAS evaluation benchmarks have been proposed, including EmbodiedBench (Yang et al., 2025d), MultiAgentBench (Zhu et al., 2025), and related evaluation suites (Agashe et al., 2025; Chollet, 2019; Kim et al., 2025; Qian et al., 2024). These evaluations rely primarily on reasoning-centric or outcome-based tasks such as task completion, answer accuracy, or unit-test completion. While such measures are useful for comparing systems, they cannot distinguish true cooperation from individual success or localize when and why coordination fails during execution. COOP2 provides a complementary view that exposes cooperative dynamics during execution, allowing outcome-based comparisons to be interpreted alongside process-level signals. A.2 MAEIL: Interaction Loop and Stage Transitions In this section, we further explain the multi-agent environment interaction loop (MAEIL) and the events that trigger transitions between its stages. As introduced in before, at any cognitive time t t, each agent i maintains an internal state xi,t^=(i,t^,πi,t^)x_i, t=(Q_i, t, _i, t), where i,t^Q_i, t denotes the current MAEIL stage of agent i, and πi,t _i, t denotes its active plan. We begin with a single agent interacting with the environment, and then extend the discussion to multiple agents that reason asynchronously while acting in a shared, stepwise environment. Communication is subsequently introduced as an explicit interaction mechanism among agents. Single-agent case. When only one embodied agent is present, the agent alternates between a reasoning stage R and an execution stage X, such that i,t^∈,Q_i, t∈\ R, X\. During R, the agent may reason in any style, invoke any available tools, and take an arbitrary amount of cognitive time to produce a plan πi,t _i, t. The agent executes a plan in the environment and returns to reasoning once execution completes, following interaction pattern resembling ReAct (Yao et al., 2022). Multiple agents without communication. When multiple embodied agents act in a shared environment without communication, an additional wait stage W is required, so that i,t^∈,,Q_i, t∈\ R, X, W\. Since the environment advances only when all agents maintain an active plan, an agent that has committed its plan but is waiting for others, transitions into W. i:,→∃i∈Iactive:¬Ready→∀i∈Iactive:Ready.i:\;\ R, X\\; ∃ i∈ I_active:\ \; W\; ∀ i∈ I_active:\ Ready\; X. Multiple agents with communication. When communication is enabled, agents may receive messages at any time, including during execution. To model this, we introduce an interrupt stage I, yielding i,t^∈,,,Q_i, t∈\ R, X, W, I\. Upon receiving a message, an agent transitions to I, processes the incoming information, and decides whether to resume its current plan or replan. Agents in stages R and I may send messages, while agents in any stage may receive messages: i2:,⇒Msgi2→i1(t^)i1:,,,.i_2:\ R, I\\; Msg_i_2→ i_1( t)\;i_1:\ R, X, W, I\. Message arrival interrupts waiting or execution and moves the agent to I: i1:,→Msgi2→i1(t^)→Ready(i1,t^′),i_1:\;\ W, X\\; Msg_i_2→ i_1( t)\; I Ready(i_1, t ) W, where the transition is defined for the receiving agent i. After processing the message, the agent enters W once it maintains an active plan, either by resuming the previous plan or by committing a new one. Figure 2 shows an instance of multiple communicating agents and their MAEIL transitions. A.3 Metrics At the aggregate level, we report Score, Steps, Score/Step, Plans/Agent, Msg., Intr., Total Dec., and constraint violation deficits. These quantities are computed from the COOP2 trace defined in Sec. 4, specifically the step-aligned cognitive activity X^It X_I_tand message trace M^It M_I_t over each interval. Let N denote the number of agents. We aggregate planning and communication over cognitive intervals as Plans/Agent=1N∑i=1N∑t=1Tπ^i,t is logged in It,Msg.=∑t=1T|M^It|.Plans/Agent= 1N _i=1^N _t=1^T1\ π_i, t is logged in I_t\, .= _t=1^T| M_I_t|. (11) Interruption frequency and decision time are computed from the MAEIL stage component Q^i,t Q_i, t, with Δt^i,t t_i, t denoting the duration of agent i’s cognitive step at time t t: Intr.=∑i=1N∑t=1T∑t^∈It[Q^i,t^=I],TotalDec.=∑i=1N∑t=1T∑t^∈It[Q^i,t^∈R,I]Δt^i,t^.Intr.= _i=1^N _t=1^T _ t∈ I_t1[ Q_i, t=I], \ Dec.= _i=1^N _t=1^T _ t∈ I_t1[ Q_i, t∈\R,I\] t_i, t. (12) Constraint violations are aggregated over targeted task-attempt events. An attempt event ege_g occurs when one or more agents execute the action associated with task g (e.g., collect(g) in MA-Crafter) For each attempt event ege_g targeting task g with participation threshold p(g)p(g), and for each constraint type c, we define vc(eg)=1−minsc(eg),1,Violc=1|Ec|∑eg∈Ecvc(eg),v_c(e_g)=1- \s_c(e_g),1\, _c= 1|E_c| _e_g∈ E_cv_c(e_g), (13) where sc(eg)s_c(e_g) is the satisfaction score for constraint c at event ege_g, and EcE_c is the set of evaluated attempt events for constraint type c, as defined in Sec. 6. Appendix B Implementation Details This cluster describes the implementation behind COOP2’s experimental pipeline, including the scalability design, the asynchronous execution loop, the structured LLM interface, and the prompts used to instantiate each communication structure. B.1 Scalable Design COOP2 is designed to scale to larger multi-agent settings. Its cooperation formalism specifies task progress through participation, capability, spatial, temporal, and dependency constraints. In MA-Crafter, the same cooperative task design can be instantiated with larger agent populations by varying participation constraints; similarly, in CUBE, task difficulty can be scaled by increasing block weights. Figure 6 shows a 10-agent MA-Crafter run under COOP2’s MAEIL, showing that the framework can support larger teams. Figure 6: 10-agent running example. Figure 7: MA-Crafter can host 100 agents, and more! Figure 8: CUBE is a grid world where teams push weighted blocks into a goal zone while respecting embodied constraints. A single scaling parameter n jointly sets team size, block weights, and grid size, creating a transparent curriculum from small to large-scale cooperation. Each panel illustrates a snapshot of a cooperative block pushing scenario at increasing scales (n from 2 to 256). B.2 Cooperation Analysis: Algorithmic Details Algorithm 1 specifies the threaded execution loop used to support cooperation analysis under asynchronous communication. At each environment step, agents run in parallel within live threads so they can continuously receive and process messages while forming plans. The system waits until all agents report readiness, which serves as a synchronization barrier that ends the current reasoning interval. Once all agents are ready, their threads are joined to ensure planning has completed, and the environment is advanced via a single joint step that returns observations, rewards, and termination signals. The episode ends when any agent terminates or is truncated, or when the maximum step budget is reached. Throughout this loop, the reasoning-phase interactions (e.g., internal state updates and message traffic) can be logged during the asynchronous period, and per-step environment outputs can be recorded after each call to env.step() for downstream cooperation analysis. Require: Base environment env_base, agent name list agent_names, maximum steps max_steps Ensure: last_step Wrap base environment with COOP2: env←COOP2(env_base,agent_names) env← COOP2( env\_base, agent\_names) Initialize done←False done← False while env.current_step<max_steps env.current\_step< max\_steps and not done do Run agents asynchronously to allow continuous communication while not all agents are ready do Ensure each agent is running in a live thread env.wait_for_state_change(timeout=0.05) Synchronize agents before stepping environment Join all agent threads Step environment (obs,rewards,terminated,truncated,info)←env.step()( obs, rewards, terminated, truncated, info)← env.step() Check termination conditions if any agent terminated or truncated then done←True done← True last_step←env.current_step last\_step← env.current\_step return last_step Algorithm 1 Threaded Multi-Agent Environment Loop B.3 Communication Structures To study how communication topology affects cooperative performance, we consider multiple communication topologies and analyze their impact. Nodes correspond to agents, and a directed edge indicates that one agent can send messages to another (with reciprocity in the bidirectional case). We consider the following communication topologies: 1. Individual: Agents do not communicate with one another. 2. Chain: During reasoning, each agent broadcasts messages to all other agents. Each agent observes all previously generated messages. 3. Centralized: For n agents, one agent acts as a leader and the remaining agents are followers. During reasoning, the leader broadcasts a message to all followers and waits for their responses. Each follower waits for the leader’s message, generates a response, and forms a plan. If a follower is interrupted by a new leader message during execution, it decides whether to resume the current plan or replan before responding to the leader. (a) Individual baseline. No communication: each agent plans independently without waiting or sending messages. (b) Chain topology. Agents speak in order: each waits for the previous speaker, then broadcasts to all following agents before planning. (c) Centralized topology. Leader broadcasts to all followers, waits for their responses, then plans; followers acknowledge the leader and plan. Figure 9: Timeline visualizations for the three communication structures considered in this work. B.4 Communication Structure Prompt Each communication structure includes role-specific instructions that are appended to the agent’s system prompt. The role descriptions below define how each agent interprets its position in the team under Individual, Centralized, and Chain settings. B.4.1 Centralized Role Descriptions in Centralized Topology ⬇ LEADER_ROLE = """ ## Your Role: LEADER You are the leader of a team. Your responsibilities: - Coordinate team activities by broadcasting directives to followers - Wait for acknowledgments from all followers before finalizing your plan - Make strategic decisions that benefit the whole team """ FOLLOWER_ROLE = """ ## Your Role: FOLLOWER You are a follower in a team. Your responsibilities: - Wait for and follow the leader’s directives - Acknowledge the leader’s messages - Execute tasks that support the team’s goals """ B.4.2 Chain Role Descriptions in Chain Topology ⬇ if speaker_order == 0: return """ ## Your Role: FIRST SPEAKER You speak first in the chain. Your responsibilities: - Set the initial direction and strategy for the team - Broadcast your analysis to all other agents - Your message will be heard by everyone """ elif speaker_order == n_agents - 1: return """ ## Your Role: FINAL SPEAKER You speak last in the chain. Your responsibilities: - Consider all previous speakers’ messages - Make a final decision incorporating all viewpoints - You have heard from everyone else """ else: return f""" ## Your Role: SPEAKER speaker_order + 1 of n_agents You speak in the middle of the chain. Your responsibilities: - Listen to the previous speaker - Add your perspective and broadcast to remaining agents - Build on the discussion so far """ B.4.3 Individual Role Descriptions in Individual Topology ⬇ INDIVIDUAL_ROLE = """ ## Your Role: INDEPENDENT AGENT You operate independently without communication with other agents. Make decisions based solely on your own observations and goals. """ Appendix C Environments This cluster details the two cooperative environments used in our experiments, MA-Crafter and CUBE, including their task structures, action spaces, observation modalities, and the prompts presented to LLM agents. C.1 MA-Crafter MA-Crafter is an embodied multi-agent environment that extends the Crafter world to support cooperative decision making among many agents in a shared, partially observable 2D grid world. Agents explore, gather resources, and craft tools through a technology tree, while maintaining individual inventories, health states, and local views. We apply Coop2’s task design to instantiate scalable cooperative tasks by enforcing embodied participation constraints on execution, and provide a dual interface: a symbolic planning layer for high-level, interpretable multi-agent plans, and a low-level execution layer that grounds plan executions under these constraints. Figure 10: MA-Crafter Figure 11: MA-Crafter Experiment C.1.1 Actions Action Arguments Effect noop (num_steps) Do nothing for a specified number of steps (default: 1). move (direction, num_steps) Move in a direction (left, right, up, down) for a specified number of steps (default: 1). collect (Resource) Collect a resource in front of the agent. All participating agents receive the resource if participation requirements are met. craft (object_type) Craft an item (e.g., wood_pickaxe, stone_sword). All participating agents receive the crafted item if participation requirements are met. sleep ( ) Sleep to recover energy. place (object_type) Place an object in the environment (e.g., stone, table, furnace, plant). share (recipient_agent_id, resource_type, quantity) Share resources/tools with another agent. Quantity defaults to 1. navigate (object_type, item_id, timeout) Navigate to a specific object using pathfinding. Multi-step action that completes when agent reaches target or times out (default timeout: 64 steps). Table 3: Symbolic actions in MCrafter used in our experiments. Action space covers all high-level behaviors available to agents. Although the symbolic action set is compact, these actions remain expressive because coordination emerges through agents’ choices of where to move, when to wait, and which block to push, all under embodied constraints from collisions, alignment, and quorum requirements. ⬇ class Direction(str, Enum): """Movement directions.""" UP = "up" DOWN = "down" LEFT = "left" RIGHT = "right" class Resource(str, Enum): """Collectible resources.""" WOOD = "wood" STONE = "stone" COAL = "coal" IRON = "iron" DIAMOND = "diamond" class PlaceableItem(str, Enum): """Items that can be placed.""" TABLE = "table" FURNACE = "furnace" PLANT = "plant" class CraftableItem(str, Enum): """Items that can be crafted.""" WOOD_PICKAXE = "wood_pickaxe" STONE_PICKAXE = "stone_pickaxe" IRON_PICKAXE = "iron_pickaxe" WOOD_SWORD = "wood_sword" STONE_SWORD = "stone_sword" IRON_SWORD = "iron_sword" class Task(str, Enum): """Available tasks/achievements in the environment.""" # Collection tasks COLLECT_WOOD = "collect_wood" COLLECT_STONE = "collect_stone" COLLECT_COAL = "collect_coal" COLLECT_IRON = "collect_iron" COLLECT_DIAMOND = "collect_diamond" COLLECT_DRINK = "collect_drink" COLLECT_SAPLING = "collect_sapling" # Crafting tasks MAKE_WOOD_PICKAXE = "make_wood_pickaxe" MAKE_STONE_PICKAXE = "make_stone_pickaxe" MAKE_IRON_PICKAXE = "make_iron_pickaxe" MAKE_WOOD_SWORD = "make_wood_sword" MAKE_STONE_SWORD = "make_stone_sword" MAKE_IRON_SWORD = "make_iron_sword" # Placement tasks PLACE_TABLE = "place_table" PLACE_FURNACE = "place_furnace" PLACE_PLANT = "place_plant" PLACE_STONE = "place_stone" class TaskSpecification(BaseModel): """Task specification with object type and target ID.""" task: Task = Field(description="The task type to accomplish") object_type: str = Field(description="Type of target object (e.g., ’tree’, ’stone’, ’cow’)") object_id: int = Field(description="ID of the specific object instance to target") def __str__(self) -> str: """String representation of the task specification.""" return f"self.task.value(self.object_type#self.object_id)" class InterruptDecision(str, Enum): """Decision options when agent is interrupted.""" RESUME = "resume" # Continue with the current plan REPLAN = "replan" # Generate a completely new plan # ============================================================================ # Pydantic models for structured LLM output - Action Classes # ============================================================================ class MoveAction(BaseModel): """Move in a direction.""" action_type: Literal["move"] = "move" direction: Direction = Field(description="Direction to move") num_steps: int = Field(ge=1, le=5, description="Number of steps (1-5)") class CollectAction(BaseModel): """Collect a resource at current location.""" action_type: Literal["collect"] = "collect" target: Resource = Field(description="Resource to collect") class PlaceAction(BaseModel): """Place an item in the world.""" action_type: Literal["place"] = "place" item: PlaceableItem = Field(description="Item to place") class CraftAction(BaseModel): """Craft an item.""" action_type: Literal["craft"] = "craft" item: CraftableItem = Field(description="Item to craft") class SleepAction(BaseModel): """Sleep to restore energy.""" action_type: Literal["sleep"] = "sleep" class NoopAction(BaseModel): """Do nothing this step.""" action_type: Literal["noop"] = "noop" class NavigateAction(BaseModel): """Navigate to a specific object in the world using pathfinding.""" action_type: Literal["navigate"] = "navigate" object_type: str = Field(description="Type of object to navigate to (e.g., ’tree’, ’stone’, ’coal’, ’iron’, ’diamond’, ’water’, ’table’, ’furnace’)") item_id: int = Field(description="ID of the specific object instance to navigate to") timeout: int = Field(default=30, ge=1, le=100, description="Maximum steps to attempt navigation (default 30)") class ShareAction(BaseModel): """Share resources with another agent.""" action_type: Literal["share"] = "share" recipient_agent_id: str = Field(description="Agent ID to share with (e.g., ’agent_0’, ’agent_1’)") resource_type: str = Field(description="Type of resource to share (e.g., ’wood’, ’stone’, ’coal’, ’iron’)") quantity: int = Field(default=1, ge=1, description="Amount to share (default 1)") # Union type for all actions LLMAction = Union[MoveAction, CollectAction, PlaceAction, CraftAction, SleepAction, NoopAction, NavigateAction, ShareAction] # ============================================================================ # Response Models # ============================================================================ class LLMPlanResponse(BaseModel): """Structured response from LLM for plan generation.""" task: TaskSpecification = Field(description="The task specification with type and optional target object") actions: List[LLMAction] = Field(description="List of actions to execute") reasoning: str = Field(description="Brief explanation of why this plan was chosen") class LLMMessageResponse(BaseModel): """Structured response from LLM for message generation.""" recipients: List[str] = Field(description="List of agent IDs to send message to") content: str = Field(description="Message content to send") reasoning: str = Field(description="Brief explanation of why sending this message") class LLMInterruptResponse(BaseModel): """Structured response from LLM for interrupt handling.""" decision: InterruptDecision = Field( description="Whether to resume the current plan or generate a new plan" ) reasoning: str = Field( description="Brief explanation of why this decision was made based on the messages received" ) # Optional new plan - only required if decision is REPLAN new_plan: Optional[LLMPlanResponse] = Field( default=None, description="The new plan to execute (required if decision is ’replan’)" ) Listing 1: Pydantic schema for LLM response C.1.2 Task Design We apply cooperative task design to resource collection in MA-Crafter by defining each collectible resource as a task g∈g , where completion requires p(g)p(g) agents to collect simultaneously within radius d of g and with the required tools. Cni(g,t)=capi(t)⪰cap(g,t), C_n^i(g,t)=cap_i(t) (g,t), (14) CΔni(g,t;d)=∥xi,t−yg∥≤d, C_ n^i(g,t;d)= x_i,t-y_g ≤ d, (15) CΔti(g,t)=ai,t=act(g), C_ t^i(g,t)=a_i,t=act(g), (16) N(C)(g,t)=∑i∈I[Ci(g,t)] N(C)(g,t)= _i∈ II\! [C^i(g,t) ] (17) Cn(g,t;d,p(g))=[⋀x∈Δℓ,Δt,nN(Cx)(g,t)≥p(g)] C_n(g,t;d,p(g))=I\! [ _x∈\ , t,n\N(C_x)(g,t)≥ p(g) ] (18) C.2 COOP2 Task Design in MA-Crafter Resource Required agents Required tool tree / wood 1 – stone 1 wood_pickaxe coal 2 wood_pickaxe iron 3 stone_pickaxe diamond 3 iron_pickaxe cow 1 – plant 1 – Table 4: MA-Crafter cooperative collection configuration. The spatial distance threshold is 11 Manhattan step for all resources. Scored resource Base value wood 1 stone 10 coal 25 iron 25 diamond 80 Table 5: MA-Crafter team-score values. C.2.1 Environment Prompt The environment prompt is a fixed textual description that outlines the basic rules, actions, and constraints of the MA-Crafter world. It is provided to all agents at the beginning of each planning step and serves as a reference for understanding the world. While it describes prerequisites, the actual environment configuration is dynamically defined in env_rule_txt (see Section C.2) and passed to agents at each step. These parameters-including object weights, required collaborators, and crafting constraints-may be redefined over time, enabling emergent adaptability challenges in multi-agent systems. MA-Crafter Environment Specification Goal The team’s objective is to collect a diamond. Any agent on the team successfully collecting a diamond achieves the goal. Cooperative Configuration The environment uses a configuration file that defines cooperative requirements for each resource, provided in observations as “COOPERATIVE CONFIGURATION”. Key parameters: – Distance threshold: Maximum distance (in tiles) for an agent to be considered “near” a resource. – Resource requirements: For each resource type, required_agents (minimum agents needed, all must be within distance and issue collect action) and required_tool (tool that ALL participating agents must have). – Different levels have different requirements: Level 1 (most resources require 1 agent), Level 2 (most require 2 agents), Level 3 (varying numbers up to 4 agents). Actions noop: Does nothing for the current step. Can be used with num_steps parameter to wait/synchronize with other agents. move: Moves the agent one step in the specified direction. Use to explore when object IDs are unknown. navigate: Multi-step action that moves the agent to reach a specified target by ID. The agent ends up facing the target. If the target is blocked, moves to the nearest location. Fails if object type and ID mismatch or object ID does not exist. sleep: Multi-step action that puts the agent to sleep until energy is fully recovered. place: Places an object of the specified type into the environment where the agent is facing. craft: Creates a new item using resources from the leader’s bag. – The leader must be facing the required tool station. – At least x agents (including the leader) must be near the tool station. – All agents must execute the craft action simultaneously. – Only the leader’s bag is used for resource consumption. collect: Collects a resource from the environment. – Agent MUST be nearby the resource (use navigate to get close first). – At least x agents must be within distance d of the object (check COOPERATIVE CONFIGURATION). – All participating agents must perform the collect action simultaneously. – The resource is added to participating agents’ bags (if space allows). – Usage: (1) If object ID known: use navigate(object_type, object_id); (2) If unknown: use move to explore; (3) Once nearby, use collect. share: Transfers resources from your inventory to another agent. – Parameters: recipient_agent_id (e.g., “agent_0”), resource_type (e.g., “wood”, “stone”), quantity (amount to transfer). – Conditions: You must have the resource; cannot share health; cannot share with yourself. Bag Capacity Each resource has a weight; the total carried weight must not exceed the bag capacity when collecting or sharing. Prerequisites Collecting Resources: - cow: navigate to cow, then collect (restores food) - drink (water): navigate to water, then collect (restores drink) - wood: navigate to tree, then collect (no tool required) - stone: navigate to stone, then collect (requires wood_pickaxe) - coal: navigate to coal, then collect (requires wood_pickaxe) - iron: navigate to iron, then collect (requires stone_pickaxe) - diamond: navigate to diamond, then collect (requires iron_pickaxe) Placing Structures: - table: consumes 2 wood; must be placed on grass - furnace: consumes 4 stone Crafting Tools (must be near required station): - wood_pickaxe: crafted at a table; uses 1 wood - stone_pickaxe: crafted at a table; uses 1 stone and 1 wood - iron_pickaxe: crafted at a furnace; uses 1 iron, 1 coal, and 1 wood Agent Health High numbers mean better condition. To replenish: – Low hunger: collect from cow – Low thirst: collect water – Low energy: sleep World Rules 1. Satisfy prerequisites (tools and materials). 2. Navigate to resources using navigate. 3. Use the collect action once nearby. 4. Be cooperative: help others when needed and ask for help when necessary. 5. Your collaborators should not include yourself. 6. Work with no more agents than needed. C.3 CUBE Figure 12: CUBE At its base level, CUBE is a grid-world environment built on PettingZoo’s parallel API (terry2021pettingzoo) and modified from the multi-agent block-pushing environment introduced in (Yang et al., 2025b). Compared to the original formulation, we simplify the environment design while retaining its cooperative block-pushing objective. Agents must cooperate to push square blocks into a designated goal region. The resulting dynamics create coordination challenges through collisions, congestion, and enforced cooperation. The same underlying environment has also been used in (nourzad2025well). The environment consists of agents and movable blocks placed on a grid. Their dynamics create coordination challenges through collisions, congestion, and enforced collaboration. Figure 13: CUBE Experiment C.3.1 Task Design We denote the set of agents by I and the set of blocks available at the start of an episode by ℬB. We treat each block’s four sides as distinct tasks, denoted by g∈g , where each task corresponds to pushing block b(g)b(g) from side s(g)∈(b(g))s(g) (b(g)). Each block has an integer weight wb(g)≥1w_b(g)≥ 1, indicating the number of agents needed to push it. Meaning, each task requires participation threshold p(g)=wb(g)p(g)=w_b(g). The block occupies a contiguous square of side length wb(g)w_b(g), so its physical size grows in direct proportion to its weight. This proportionality ensures consistency, as larger blocks both span more grid cells and require a greater quorum of agents to move. Each agent i∈Ii∈ I occupies a single grid cell, with position denoted itx_i^t at step t. Episodes terminate successfully when all blocks have been delivered to the goal region. Episodes truncate if the max_steps is reached without delivering all blocks. Cni(g,t)=capi(t), C_n^i(g,t)=cap_i(t), (19) CΔℓi(g,t)=ming∈g∥i,t−g∥≤1, C_ ^i(g,t)= _g _g _i,t-g ≤ 1, (20) CΔti(g,t)=ai,t=push(b(g)), C_ t^i(g,t)=a_i,t= push(b(g)), (21) N(C)(g,t)=∑i∈I[Ci(g,t)], N(C)(g,t)= _i∈ II\! [C^i(g,t) ], (22) Cn(g,t)=[⋀x∈Δℓ,Δt,nN(Cx)(g,t)≥wb(g)] C_n(g,t)=I\! [ _x∈\ , t,n\N(C_x)(g,t)≥ w_b(g) ] (23) Preset Block specification Setting 9 blocks in total: 3 blocks each of weights 1, 2, and 3 Table 6: CUBE cooperative block-pushing configuration. A block’s weight is both its side length and the number of agents required to push it. (a) Successful chain (b) Failed chain Figure 14: Illustration of chains: a chain succeeds or fails depending on whether available agents meet the block’s weight requirement and the maximum force exerted on the block face. Block chain. A block B(g)∈ℬB(g) of weight wb(g)w_b(g) may occupy one or more grid cells. When another block lies directly in front of BjB_j along a push direction d∈↑,↓,←,→d∈\ , ,←,→\, the blocks form a block chain. The chain behaves as a composite structure whose motion depends on whether the total applied force at its leading face is sufficient to move all blocks within the chain. Let i index agents in the corresponding agent chain, each contributing unit force fi=1f_i=1 along direction d. A chain advances one cell in direction d if and only if ∑ifi≥∑B∈chainwb(g) _if_i≥ _B w_b(g) and all destination cells are unoccupied and within bounds. Upon success, all blocks in the chain advance by one cell; otherwise, the entire chain of blocks remains in place. Agent chain. Agents exert unit forces on blocks through pushing chains. A pushing chain forms when multiple agents align collinearly behind a block and all push in the direction of the block face. The effective force at the contact face is the number of aligned agents in that direction. If this total meets or exceeds the required block (or chain) weight, the structure advances. Otherwise, the attempt fails and all participating agents remain in place, as shown in Fig. 14. Agent Movement and Collisions While the environment supports a dual-layer action interface, primitive and symbolic (see subsection C.3.2 for details), each symbolic action is unrolled into a sequence of primitive actions, with all movements and collisions resolved at the primitive level. At each timestep, each agent issues a primitive action ai∈a_i , which specifies a target cell on the grid. A move is valid if the target cell lies within bounds and is not occupied by a block (including newly moved blocks). If multiple agents attempt to move into the same unoccupied cell, the agent with the smallest index i successfully claims the cell, while all others involved in the conflict remain unmoved. If any agent attempts to move into a cell occupied by a stationary agent, the move fails and both agents stay in place. This rule prevents overlap and introduces a consistent tie-breaking mechanism for simultaneous movements. C.3.2 Cognitive Layer Design CUBE features a dual-layer interface that integrates symbolic reasoning with vector-based representations across observation, action, and feedback channels. The symbolic layer abstracts the dynamics of the environment into discrete entities and relations such as the distance between the agents and the quorum, allowing high-level reasoning, planning, and language interaction. Complementing this, the vector-based layer provides dense spatial and state features suitable for reinforcement learning and low-level control. Together, these layers allow agents to ground symbolic reasoning in embodied experience, supporting cooperative learning across reinforcement, language, and hybrid agent architectures. Observation Space CUBE provides two observation modalities: a symbolic observation and a multi-channel observation. This dual interface supports diverse agent architectures, enabling reinforcement learning agents to rely on grid-based encodings, LLM-based agents to operate over symbolic state descriptions, or novel approaches that combine both. Symbolic Observation. At each step t, every agent i receives a symbolic dictionary describing the current state. This includes global environment information (grid size, positions of all agents) as well as a compact summary for each block (block ID, weight, position, and distance to the goal column). The dictionary also records all symbolic actions taken so far in the episode, along with their corresponding primitive actions and the status (start, in progress, or end) at each timestep. This structured interface allows reasoning directly about concepts such as which blocks remain, how far they are from the goal, and where teammates are, supporting high-level planning and coordination. Multi-channel Observation. In addition, a five-channel grid encodes agent locations, block weights, the goal column, a channel marking which agent occupies each cell, and a channel marking which block occupies each cell. This representation resembles standard reinforcement learning observations and is primarily included for compatibility with reinforcement learning pipelines and for visualization. Action Space CUBE supports two sets of action spaces. The primitive action space provides low-level grid movements, enabling agents to interact directly with the environment through discrete directional moves. The symbolic action space abstracts these primitives into higher-level cooperative strategies, such as aligning on a block face, synchronizing for a push, or waiting for teammates. Together, these two levels allow experiments to target both reinforcement learning agents, which operate naturally over primitive actions, and LLM-based agents, which benefit from reasoning over symbolic actions. Primitive Actions. Each agent selects from a discrete 5-action set =STAY=0,UP=1,DOWN=2,LEFT=3,RIGHT=4.A=\ STAY=0,\ UP=1,\ DOWN=2,\ LEFT=3,\ RIGHT=4\. At time t, each agent i issues an action ai,ta_i,t specifying a movement direction, and all agents act in parallel to move one unit in their respective directions. Moves succeed only if the target cell is free; collisions with walls, agents, or insufficiently supported blocks cause the agent to remain in place. A push succeeds if the aligned agents’ combined force exceeds the total weight of the aligned blocks and the destination cell is free, in which case both the blocks and agents advance one step. Symbolic Actions. Beyond primitive grid movements, CUBE provides a small library of symbolic actions that capture essential coordination behaviors while remaining easy to ground in the environment. Each symbolic action is executed by compiling it into a sequence of primitive moves until the specified condition is met. In our final setup, we use only three symbolic actions: move, wait, and push. move compiles into repeated primitive movements in a fixed direction, wait compiles into STAY actions for a fixed number of steps, and push compiles into repeated primitive pushes against the specified block until the push terminates. Action Arguments Effect move (direction, steps) Move in the specified direction for a fixed number of steps (compiled into primitive moves). wait (steps) Remain idle for a fixed number of steps (compiled into STAY). push (block_id, steps) Push against the specified block by repeatedly issuing primitive actions directed into the block face; succeeds only if enough agents align and apply sufficient force. Table 7: Symbolic actions in CUBE used in our experiments. Each action specifies a high-level effect that is decomposed into primitive actions. Although the symbolic action set is compact, these actions remain expressive because coordination emerges through agents’ choices of where to move, when to wait, and which block to push, all under embodied constraints from collisions, alignment, and quorum requirements. C.3.3 Environment Prompt Similarly to the MA-Crafter, the environment prompt outlines the basic rules, actions, and constraints of the CUBE world. The prompt describes cooperative block-pushing mechanics, where blocks of different weights require different numbers of agents to push simultaneously. CUBE Environment Specification Goal Push all blocks to the goal column (rightmost column) to deliver them. Blocks require multiple agents pushing simultaneously if block weight >1>1. Grid – Simple K×K× K grid (default 8×88× 8) – Goal column is the rightmost column (column K−1K-1) – Agents start at various positions on the left side – Blocks are placed in the middle area Blocks Each block has: – ID: Unique identifier – Weight: Determines size (weight × weight square) and push requirement – Position: (row, col) of top-left corner Push Requirements – Weight 1: 1 agent can push alone – Weight 2: 2 agents must push simultaneously from same side – Weight 3: 3 agents must push simultaneously from same side – etc. Cooperative Mechanics To successfully push a block of weight W: 1. W agents must be adjacent to the SAME side of the block 2. All W agents must issue push/move commands INTO the block simultaneously 3. If successful, both the block and pushing agents move forward 4. Block is delivered when any part reaches the goal column Actions move(direction, num_steps): Move in a direction for num_steps. – direction: “up”, “down”, “left”, “right” – num_steps: 1–10 push(block_id, num_steps): Push a block by moving into it. – block_id: ID of the target block – num_steps: How many steps to push – Direction is automatic based on agent position relative to block – Agent must be adjacent to the block wait(num_steps): Do nothing for num_steps. – Useful for synchronizing with other agents – num_steps: 1–10 Observation Channels 5-channel observation (K×K×5K× K× 5): – Channel 0: Agent mask (1.0 at agent positions) – Channel 1: Block weights (value = weight at block cells) – Channel 2: Goal strip (1.0 in rightmost column) – Channel 3: Agent IDs (+1) – Channel 4: Block IDs (+1) Rewards – Step cost: Small negative reward per step – Delivery reward: Positive reward when block reaches goal – Episode ends when all blocks delivered or max steps reached Strategy Tips 1. Communication: Announce which block you’re targeting and from which side 2. Coordination: For weight-2+ blocks, agree on timing (e.g., “push at step 10”) 3. Positioning: Get into position on the same side before pushing 4. Use wait: Synchronize pushes with wait actions 5. Efficiency: Push lighter blocks first if they’re blocking heavier ones Appendix D Additional Results and Qualitative Analysis This cluster provides additional COOP2 traces beyond those shown in the main paper and qualitative analyses of the communication patterns that emerge across different settings. D.1 COOP2 Traces We provide additional process-level traces beyond the 3-agent MA-Crafter results shown in the main part of the paper. The traces below cover the 6-agent setting and CUBE runs, allowing more detailed inspection of how cooperation unfolds across team sizes and environments. Figure 15: COOP2 process traces for MA-Crafter with 6-agent teams, averaged over five runs. The traces show active task focus, observed constraint satisfaction at collection attempts, and cumulative team score across backbones and communication structures. Figure 16: COOP2 process traces for MA-Crafter with 3-agent teams, single run. The traces show how agents’ active task focus, constraint satisfaction, and cumulative score evolve over time, providing an additional view of cooperation dynamics in a spatial coordination environment. Structure Score ↑ Steps Score/Step Plans/Agent Msg. Intr. Total Dec. Spat. Viol. Temp. Viol. Dep. Viol. 3-agent GPT-5.4-mini Individual 3.0 116.0 0.03 30.7 0.0 0.0 137.0 0.20 0.26 0.00 Centralized 3.0 80.0 0.04 23.3 54.0 29.0 183.9 0.27 0.30 0.00 Chain 1.0 54.0 0.02 23.7 48.0 46.0 197.8 0.17 0.23 0.00 3-agent GPT-5.4 Individual 1.0 78.0 0.01 16.3 0.0 0.0 130.1 0.29 0.48 0.00 Centralized 1.0 52.0 0.02 12.3 39.0 24.0 215.3 0.23 0.34 0.00 Chain 1.0 37.0 0.03 12.3 24.0 19.0 199.5 0.46 0.46 0.00 6-agent GPT-5.4-mini Individual 3.0 86.0 0.03 21.7 0.0 0.0 192.3 0.10 0.12 0.00 Centralized 0.0 75.0 0.00 20.5 126.0 97.0 342.3 0.10 0.64 0.00 Chain 3.0 39.0 0.08 13.5 66.0 52.0 310.8 0.14 0.22 0.00 6-agent GPT-5.4 Individual 1.0 49.0 0.02 15.0 0.0 0.0 233.5 0.21 0.41 0.00 Centralized 6.0 59.0 0.10 10.7 55.0 40.0 324.3 0.41 0.55 0.00 Chain 1.0 23.0 0.04 6.7 35.0 28.0 332.9 0.31 0.33 0.00 Table 8: CUBE cooperation results. Figure 17: COOP2 process traces for CUBE with 3-agent teams, single run. The traces show how agents’ active task focus, constraint satisfaction, and cumulative score evolve over time, providing an additional view of cooperation dynamics in a spatial coordination environment. Figure 18: COOP2 process traces for CUBE with 6-agent teams, single run. The traces show how agents’ active task focus, constraint satisfaction, and cumulative score evolve over time, providing an additional view of cooperation dynamics in a spatial coordination environment. D.2 Annotated Communications We provide annotated examples of agent communication under different structures and difficulty levels, illustrating common patterns in how LLM agents recognize, discuss, and attempt to satisfy cooperative constraints. These examples qualitatively complement the aggregate metrics and traces in the main paper. Figure 19: Communication messages of LLM-MAS operating under centralized communication structures. The left panel shows a 2-agent setting in Easy mode, the middle panel shows 3 agents in Easy mode, and the right panel shows 3 agents in Hard mode. In the Easy setting, the leader assigns distinct tasks to workers. In Hard settings, although agents recognize the presence of constraints, they struggle to determine how to satisfy them. The leader is aware that coordination is required, but lacks grounding in how to execute effective teamwork within the environment. Figure 20: COOP2 bridges the cognitive layer, where agents perform reasoning, planning, and communication, and the primitive layer, where plans are grounded into primitive actions. The figure depicts a centralized communication structure under COOP2: the leader broadcasts messages to workers, interrupting workers’ ongoing execution and triggering replanning or resumption. Figure 21: Individual communication structure. Figure 22: Chain communication structure.