Paper deep dive

PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting

Youran Sun, Xingyu Ren, Kejia Zhang, Xinpeng Liu, Jiaxuan Guo

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 69

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/9/2026, 4:02:38 AM

Summary

PerspectiveGap is a benchmark designed to evaluate large language models' ability to compose orchestration prompts for multi-agent systems. It introduces 110 scenarios across 10 topologies, utilizing role-fragment assignment and free-form prompt writing tasks to measure strict pass rates and information leakage. Experiments show GPT-5.5 leads with a 62.0% pass rate, while Opus 4.7 struggles, highlighting that multi-agent orchestration prompting is a distinct and under-evaluated capability.

Entities (10)

PerspectiveGap · benchmark · 98%GPT-5.5 · model · 97%Multi-Agent Orchestration · concept · 95%Opus 4.7 · model · 95%10 Topologies · structure · 93%Prompt Economy · principle · 92%Free-Form Prompt Writing · task · 90%Role-Fragment Assignment · task · 90%Overall Leakage · metric · 88%Strict Pass · metric · 88%

Relation Signals (9)

PerspectiveGap → contains → 110 Scenarios

confidence 96% · PerspectiveGap contains 110 scenarios, each evaluated through two distractor-mixed task formats

GPT-5.5 → achieves → 62.0% pass rate

confidence 95% · GPT-5.5 substantially outperforms all competitors... GPT-5.5 62.0%

PerspectiveGap → evaluates → Multi-Agent Orchestration

confidence 95% · PerspectiveGap, a benchmark for evaluating LLMs' ability to compose orchestration prompts for multi-agent systems.

Opus 4.7 → exhibits → weakness in orchestration prompting

confidence 93% · Opus 4.7 shows a notable weakness in orchestration prompting despite its strong coding performance.

PerspectiveGap → uses → Role-Fragment Assignment

confidence 92% · each evaluated through two distractor-mixed task formats: role-fragment assignment and free-form prompt writing.

PerspectiveGap → uses → Free-Form Prompt Writing

confidence 92% · each evaluated through two distractor-mixed task formats: role-fragment assignment and free-form prompt writing.

Prompt Economy → frames → 10 Topologies

confidence 91% · These scenarios are organized into 10 topologies... framed by the Prompt Economy principle

PerspectiveGap → measures →

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Real-world LLM applications are moving beyond single-agent workflows toward orchestrated multi-agent systems, yet current models still struggle to determine what each sub-agent needs to know. To measure this, we introduce PerspectiveGap, a benchmark for evaluating LLMs' ability to compose orchestration prompts for multi-agent systems. PerspectiveGap contains 110 scenarios, each evaluated through two distractor-mixed task formats: role-fragment assignment and free-form prompt writing. These scenarios are organized into 10 topologies, which are distilled from the authors' real-world engineering practice and framed by the Prompt Economy principle: building loop-centered orchestrations that maximize utility with minimal role and engineering overhead. In experiments with 27 commercial models from 10 companies, GPT-5.5 substantially outperforms all competitors, whereas Opus 4.7 shows a notable weakness in orchestration prompting despite its strong coding performance. Nevertheless, PerspectiveGap remains challenging: the evaluated models achieve an average combined pass rate of only 14.9\% (GPT-5.5 62.0\%) and an average overall leakage rate of 246.5\% (a per-scenario information leak-event count, not a proportion; GPT-5.5 49.1\%). These findings suggest that multi-agent orchestration prompting is a distinct and under-evaluated capability, and PerspectiveGap provides a foundation for measuring and improving it systematically.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

68,756 characters extracted from source content.

Expand or collapse full text

PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting Youran Sun 1,∗ Xingyu Ren 2,∗ Kejia Zhang 1 Xinpeng LiuJiaxuan Guo 3,† Abstract Real-world LLM applications are moving be- yond single-agent workflows toward orches- trated multi-agent systems, yet current mod- els still struggle to determine what each sub- agent needs to know. To measure this, we introduce PerspectiveGap, a benchmark for evaluating LLMs’ ability to compose orches- tration prompts for multi-agent systems. Per- spectiveGap contains 110 scenarios, each eval- uated through two distractor-mixed task for- mats: role-fragment assignment and free-form prompt writing. These scenarios are organized into 10 topologies, which are distilled from the authors’ real-world engineering practice and framed by the Prompt Economy princi- ple: building loop-centered orchestrations that maximize utility with minimal role and en- gineering overhead. In experiments with 27 commercial models from 10 companies, GPT- 5.5 substantially outperforms all competitors, whereas Opus 4.7 shows a notable weakness in orchestration prompting despite its strong coding performance. Nevertheless, Perspective- Gap remains challenging: the evaluated models achieve an average combined pass rate of only 14.9% (GPT-5.5 62.0%) and an average overall leakage rate of 246.5% (a per-scenario informa- tion leak-event count, not a proportion; GPT- 5.5 49.1%). These findings suggest that multi- agent orchestration prompting is a distinct and under-evaluated capability, and PerspectiveGap provides a foundation for measuring and im- proving it systematically. 1 Introduction Prompt engineering has shifted from tuning a sin- gle prompt to designing multi-agent orchestras, in which a task is decomposed into specialized roles with distinct information boundaries and artifact handoffs (Li et al., 2023; Qian et al., 2023; Hong et al., 2023; Wu et al., 2023a; Chen et al., 2023; Lu et al., 2024; Tran et al., 2025). Constructing these orchestras requires orches- tration prompting: writing sub-agent instructions 1 University of Maryland. 2 The Chinese University of Hong Kong. 3 Stanford University. ∗ Equal contri- bution. † Corresponding author.Emails: Youran Sun, sun1245@umd.edu; Jiaxuan Guo, guojx@stanford.edu. that specify each role’s task scope, context bound- aries, and expected handoffs. Yet current LLMs struggle to determine what each sub-agent needs to know. The resulting failures are not cosmetic: main agents leak distractors, expose out-of-role information, drop shared context, confuse artifact ownership, and sometimes place instructions where the sub-agent cannot see them. These errors pro- duce incomplete, contaminated, or self-defeating sub-agent prompts. Section 6 analyzes these failure modes in detail. Existing evaluations do not target this orchestration-prompting ability. Theory-of-mind (ToM) benchmarks typically score question answering or belief tracking (Le et al., 2019; Kim et al., 2023; Sclar et al., 2024), multiple-choice action prediction (Gu et al., 2024), dialogue acts (YS et al., 2026), or functional behavior labels (Riemer et al., 2025). Agent benchmarks instead score downstream task success, tool use, environment-level performance, or instruction following (Liu et al., 2023; Qin et al., 2023; Orogat et al., 2026; Qi et al., 2025). Neither line directly evaluates whether a main agent can write sub-agent prompts that respect asymmetric context and role-specific information needs. Section 2 discusses this boundary. We introduce PerspectiveGap, a benchmark for multi-agent orchestration prompting. Perspective- Gap contains 110 scenarios, each consisting of a role list, a shuffled set of information fragments f 1 ,...,f N , and a reference answer specifying which fragments each role needs. Each scenario includes two tasks: role-fragment assignment and free-form prompt writing. The former asks the model to output the fragment IDs for each role; the latter asks the model to write the actual sub- agent prompts. The two tasks share the same frag- ments, so comparing them isolates models that can identify each role’s needs but fail to act on them when writing the prompt. Each scenario also in- jects one distractor, such as prompt-engineering advice, that may look useful to the tested LLM but is useless or even burdensome for any sub- agent. The benchmark covers orchestrations with 1 arXiv:2606.08878v1 [cs.CL] 7 Jun 2026 2–6 roles and 7–13 information fragments. A de- terministic, rule-only scorer evaluates each role’s prompt for inclusion of required fragments and ex- clusion of irrelevant content; a 716-row hand audit validates this containment detector. The construc- tion pipeline records each scenario’s topology, role definitions, fragments, and reference answers ex- plicitly, allowing new scenarios to be added with auditable answer keys. The 110 scenarios are organized into 10 topolo- gies shown in Figure 2. These topologies are distilled from the authors’ real-world prompt- engineering practice and from production agent- system patterns. We frame their loop-centered structure with Prompt Economy. Under this fram- ing, engineering cost is largely fixed by the number of role prompts, while benefit accumulates across repeated role invocations. Loop-centered orchestra- tion patterns are therefore attractive when the goal is to maximize utility with minimal engineering overhead. The release uses six base patterns plus four pool variants, all built around one or more critic loops. As a set, they cover the main practical one- and two-loop orchestration patterns targeted by PerspectiveGap. We evaluate 27 commercial models from 10 com- panies on PerspectiveGap. GPT-5.5 leads with a 62.0% combined pass rate, while deepseek-v4-pro is a distant second at 32.0%, and models ranked 3–8 cluster between 19% and 26%. The average combined pass rate is 14.9%. The average overall leakage rate is 246.5%, compared with 49.1% for GPT-5.5. Opus 4.7 underperforms relative to its strong coding performance (Wang et al., 2025) 1 , suggesting that orchestration prompting is not re- ducible to coding skill. These results indicate that multi-agent orchestration prompting is a distinct and under-evaluated capability. This paper makes four contributions: 1. We release PerspectiveGap, a 110-scenario benchmark with two task formats per scenario, per-scenario answer keys, construction notes, and full logs. 2.We evaluate 27 commercial models from 10 companies and report a standardized leader- board, revealing low average performance. 3. We identify five recurring failure modes in multi-agent orchestration prompting. 1 This finding also matches the authors’ day-to-day experi- ence; PerspectiveGap finally quantifies that intuition. 4.We articulate Prompt Economy as a cost- benefit framing for the loop-centered topology family. Figure 1: Overview of PerspectiveGap. (A) A scenario presents a set of information fragmentsf 1 ,...,f N , one of which is a distractor (f 7 ), and a set of sub-agent roles (here dispatcher, coder, and reviewer); each role needs only a specific subset of the fragments. (B) The role-fragment assignment task asks the model to out- put that subset for each role, as fragment identifiers. (C) The free-form prompt writing task asks the model to write each role’s actual prompt from the same frag- ments, scored by checking that each prompt contains its required fragments and excludes out-of-role content. The example prompts illustrate the two error types the benchmark targets: a leaked distractor in the dispatcher prompt and an omitted required fragment (f 2 ) in the coder prompt. 2 Related Work Multi-agent orchestration. LLM applications increasingly split work across specialized workers, 2 critic loops, and pipelines. Early multi-agent sys- tems such as CAMEL, ChatDev, MetaGPT, Auto- Gen, and AgentVerse showed that role-specialized LLMs can collaborate through natural-language messages and structured workflows (Li et al., 2023; Qian et al., 2023; Hong et al., 2023; Wu et al., 2023a; Chen et al., 2023). Subsequent surveys and pattern catalogues describe recurring orchestration motifs such as routing, planning, reflection, cri- tique, and tool-mediated handoff (Tran et al., 2025; Liu et al., 2024; Dao et al., 2026; Gullí, 2025). Industrial guidance makes a similar distinction be- tween workflows with prescribed control flow and more autonomous agentic systems, emphasizing that role boundaries, context selection, and handoff discipline are central engineering concerns (An- thropic, 2024). Recent applied systems instantiate these motifs in optimization, scientific computing, data discovery, and automated research (Lu et al., 2024; Thind et al., 2025; Du et al., 2026; Sun et al., 2026). These works motivate our setting, but they do not test the fragile step we study: whether an LLM can write role-specific instructions that pre- serve context boundaries and handoff contracts in- stead of flattening every role into the same prompt. Agent and tool-use benchmarks. AgentBench (Liu et al., 2023), ToolLLM (Qin et al., 2023), and related benchmark suites evaluate execution after the agent’s task, tools, and instructions are already given. They ask whether an agent can act in an envi- ronment, call the right tool, or complete a workflow under an existing scaffold. Other recent evalua- tions compare multi-agent frameworks, delegation behavior, instruction following, and failure attri- bution inside already-instantiated systems (Orogat et al., 2026; Gao et al., 2026; Qi et al., 2025; Zhang et al., 2025b). PerspectiveGap moves the evalua- tion one step earlier: before any worker acts, can the main agent allocate context and constraints into prompts that downstream workers can safely use? Information asymmetry and ToM benchmarks. Theory-of-mind benchmarks study information asymmetry, but they do not ask the model to produce prompt artifacts. ToMi, FANToM, Hi- ToM, OpenToM, BigToM, and ExploreToM test belief tracking, higher-order mental-state reason- ing, or adversarially generated belief structures through question answering and classification (Le et al., 2019; Kim et al., 2023; Wu et al., 2023b; Xu et al., 2024; Gandhi et al., 2023; Sclar et al., 2024). Work on perspective-taking, hypothesis- driven ToM reasoning, and strategic social reason- ing shows that making viewpoints explicit can im- prove model behavior or expose failures in social planning (Wilf et al., 2023; Kim et al., 2025; Yao et al., 2025). SimpleToM, together with recent position and behavior-based work, goes further, separating explicit mental-state prediction from the functional step of acting on it (Gu et al., 2024; Riemer et al., 2025; Ackerman, 2026). Our two tasks mirror this split: assignment tests whether the model knows what each role needs, and prompt writing tests whether it acts on that knowledge. SOTOPIA-TOM is closest to our setting because it studies information management in multi-agent interaction (YS et al., 2026). Yet even SOTOPIA- TOM, like the rest, tests belief tracking, action choice, or dialogue behavior inside an interaction. PerspectiveGap tests orchestration prompting: con- structing the instructions that define each role’s view of the task before the interaction begins. Distributed information and multi-agent fail- ures.Several recent benchmarks show that multi- agent LLM systems fail under distributed infor- mation even when individual models are strong. HiddenBench finds a large gap between collec- tive reasoning with distributed information and single-agent reasoning with complete information (Li et al., 2025). Silo-Bench similarly shows that agents may exchange enough information but fail to integrate it into a correct joint answer (Zhang et al., 2026). MAST categorizes multi-agent fail- ures and identifies inter-agent misalignment, in- cluding role confusion and incomplete delegation, as a recurring source of breakdowns (Cemri et al., 2025). Privacy and leakage benchmarks such as AgentLeak study how information can escape across a running multi-agent stack (Yagoubi et al., 2026). These works diagnose failures after agents interact; PerspectiveGap isolates a precursor arti- fact: the role prompts that determine each agent’s initial information boundary. Prompt optimization and role-state manage- ment.Prompting and workflow methods can im- prove downstream behavior through refinement, role evolution, memory, or training-time optimiza- tion (Madaan et al., 2023; Chang and Geng, 2025; Mo et al., 2025). Other multi-agent methods make collaborator knowledge or belief state explicit, which is closely related to our premise that the main agent must reason about what each sub-agent knows and needs (Zhang et al., 2025a; Singh et al., 3 2026). However, these methods are usually eval- uated by final task success or workflow reliabil- ity. PerspectiveGap instead evaluates the generated prompt artifact directly: whether it assigns informa- tion correctly can be read from the prompts them- selves, not inferred from whether the downstream task succeeds. 3 PerspectiveGap Benchmark Design Scenario schema. As shown in Figure 1, each scenario is a small orchestration problem with three explicit parts: a list of sub-agent roles, a shuffled list of information fragmentsf 1 ,...,f N , and a ref- erence role-to-fragment assignment stored with the scenario. The model sees the roles and the shuffled fragments, but not the reference assignment. The reference assignment applies the need-only rule: each role receives exactly the fragments it needs to do its stated job. This rule sets the role-context boundary used for scoring. In this sense, the ref- erence assignment is the task contract: given the stated roles and the need-only rule, the model must preserve that boundary in both evaluated formats. All 110 scenario mappings were constructed under this rule and inspected by the full five-author team for consistency. Appendix J gives an actual bench- mark scenario, including the background, fragment headings, the “need-only” instruction shown to the model, the reference assignment, and the textual evidence behind representative boundary decisions. Deterministic rendering. The benchmark sepa- rates topology, instance, and scenario. A topology is one of the 10 role-and-handoff patterns used in the paper; an instance is a domain-specific real- ization of a topology; a scenario is the rendered benchmark item given to the model. For each scenario, a shuffle seed fixes the displayed frag- ment order; the displayed identifiers are then rela- beled asf 1 ,...,f N in that order, as choices in a multiple-choice question would be relabeled after shuffling. Changing the seed changes the presen- tation order and displayed identifiers, but not the role definitions, fragment content, distractor, or reference assignment. This gives a deterministic way to test whether a model is assigning informa- tion by the need-only rule rather than by position in the prompt. The same topology-instance split also gives the benchmark breadth: Appendix I lists the professional-domain coverage. Across these domains, the same construction rule determines which fragments each role needs. Figure 2: Base topology patterns in PerspectiveGap. Each pattern is a role-and-handoff graph: nodes are sub- agent roles, solid edges are the actor–reviewer feedback loops and the handoffs between them, and dashed edges are distribution or support links (a dispatcher feeding workers, or a shared librarian). The six base patterns span a single coder–reviewer loop (cr), a dispatcher- fed loop (dcr), sequential and parallel two-loop systems (dpc and dtc, respectively), a scientist–coder–reviewer arrangement (scr), and a supervisor–student–librarian hub (spl). Four of the six have pool variants (insets crp, dcrp, dpcp, dtcp) that add parallel candidates on the producer side, for 10 topologies in total. Extensibility.The topologies are reusable skele- tons, not one-off examples. These 10 topologies cover the main practical one- and two-loop or- chestration patterns targeted by PerspectiveGap 2 ; more complex orchestrations can be composed from them as building blocks. New instances can be added by choosing a domain, naming the roles, writing the fragments, and applying the same reference-assignment rule; the renderer and scorers do not change. 2 We quotient away role names, artifact names, domains, and exchangeable pools. A primitive loop is an actor–reviewer feedback pair; one-loop systems are this pair with optional dispatcher, pool, or producer-side support, and two-loop sys- tems have three normal forms: no coupling, one-way handoff, and mutual handoff. This claim excludes arbitrary multi-agent graphs such as routers, deep hierarchies, memory managers, and systems with more than two feedback loops. 4 Distractor insertion.Each main benchmark sce- nario includes one distractor fragment. A represen- tative distractor used in the benchmark is: Best practices: Use consistent, descriptive tag names across your prompts. Nest tags when con- tent has a natural hierarchy, such as documents inside<documents>and each document inside <document index="n">. This looks useful to the tested model while it is writing prompts, but it is not useful to any down- stream sub-agent. The tested model must separate information that helps itself compose the orches- tra from information that a sub-agent needs to do its job. Leaking irrelevant content can add cogni- tive load and degrade the sub-agent’s output; in higher-stakes settings, it can also expose out-of- role information that enables reward hacking or violates safety constraints. Two evaluated formats. Each scenario is ren- dered in two formats from the same shuffled frag- ments. Role-fragment assignment asks for a struc- tured mapping from roles to fragment identifiers. Free-form prompt writing asks for the final sub- agent prompts. Pairing the two formats makes the failure mode visible: a model may know the bound- ary as a set of identifiers, yet fail to preserve that boundary when it writes natural-language instruc- tions. The benchmark therefore reports the two formats separately and also reports their combined score. Evaluation metrics. We treat Strict pass as the primary endpoint, report Net match score as a partial-credit companion, and use Required cov- erage, Boundary precision, Overall leakage, and Distractor leakage as diagnostic metrics. For each evaluatione(one model response to one scenario, shuffle seed, and task format), we first aggre- gate events over all requested roles:TP e counts correctly included reference role-fragment events, FP e counts extra out-of-role events, andFN e counts omitted reference events. Strict pass is 1 iffFP e = FN e = 0. The average Strict pass over a task format’s evaluations is that format’s pass rate, and the average over all evaluations is the combined pass rate. This all-or-nothing criterion matches the benchmark’s boundary-preservation objective: either an output preserves all requested role boundaries, or it contains an omission or cross- role inclusion. Net match score ismax(0, (TP e − FP e − FN e )/(TP e + FN e )) for each evaluation, averaged over evaluations. It is not used to relax the endpoint; it provides a continuous companion for model-level consistency checks. Required cov- erage and Boundary precision are micro-averages over evaluations: P e TP e / P e (TP e + FN e )and P e TP e / P e (TP e + FP e ) . Overall leakage is avg e FP e , counting every out-of-role leak event per role; Distractor leakage is its restriction to the injected distractor,avg e D e , whereD e counts only injected-distractor leak events and thusD e ≤ FP e . Because leaks are counted per role, both can ex- ceed 100%. These diagnostics decompose failures but are not intended to replace the strict endpoint. Scoring. Free-form prompt writing uses a deter- ministic rule-only scorer rather than an LLM judge, so that prompt-boundary decisions are reproducible and do not depend on another model’s implicit del- egation policy. It is a containment audit, not a general prompt-quality judge: it checks whether re- quired fragment evidence is present and out-of-role fragment evidence is absent. For each fragment, the scorer builds a fingerprint from the unigram, bigram, and trigram phrases that distinguish that fragment from the others in the same scenario. A role prompt must include enough of each required fragment’s distinctive fingerprint and must not in- clude enough of any fragment outside that role’s reference assignment. The asymmetric thresholds (include at least 0.7, leak less than 0.3) allow ordi- nary connective text while still catching wrong-role copying and distractor leakage. Phrase-level finger- prints also handle near-parallel fragments, such as two file-handoff instructions that share most words but refer to different artifacts. Its agreement with expert human containment labels is validated on a 716-row hand-audited scorer test set (Table 13). Released scope.The released evaluation set con- tains 10 topologies, 100 domain instances, and 110 scenarios. Table 1 summarizes these counts and the benchmark components. The benchmark data, renderer, scorer, and model-running scripts are pub- licly available at WhymustIhaveaname/Perspective- Gap. 4 The Prompt Economy Framing Prompt-engineering effort. An orchestrated agent system is maintained through role prompts and handoff protocols: the former define each sub- agent’s responsibility, and the latter specify which artifacts each role reads, writes, and ignores. If a system hasmrole prompts andnhandoff protocols, 5 ComponentCount Topologies10 Domain instances100 Scenarios110 Evaluated formats per scenario2 Distractors per scenario1 Scorer test-set rows716 Table 1: Benchmark scope of PerspectiveGap. Topolo- gies are reusable role-and-handoff skeletons; instances are domain-specific realizations; scenarios are rendered benchmark items. The main 110-scenario evaluation uses one distractor per scenario; a separate distractor- count ablation (Appendix D) reruns role-fragment as- signment with 0 to 3 injected distractors. then its prompt-engineering effort can be approx- imated asO(m + αn), whereαweights handoff maintenance. A naive application of Conway’s law can therefore produce brittle orchestrations with too many roles, unclear file ownership, and unsta- ble handoffs. The useful design target is not “more agents,” but a small set of roles whose responsibili- ties and handoffs remain stable. Role reuse. A role prompt is written and main- tained once, but the role may be run many times. Ifv i counts useful runs of rolei, then the system’s value is better viewed as accumulating with P i v i than as scaling with the number of roles alone. A frequently reused role can amortize its prompt ef- fort over many calls, whereas a rarely used role adds maintenance cost with little return. This is the intuition behind Prompt Economy: keep prompt- maintenance effort bounded while increasing use- ful role reuse. Here, “economy” refers to spar- ing and efficient prompt use: a small, reusable prompt surface whose maintenance cost is amor- tized across repeated role invocations. Loop-centered design. Critic loops are the sim- plest way to create that asymmetry. An actor–critic pair needs only two role prompts and a small hand- off protocol: the actor produces an artifact, the critic finds flaws, and the actor revises. Once writ- ten, the same loop can run for many rounds, so effort stays close to the two-role design while value accumulates across iterations. This actor–critic pat- tern appears in software agents, research agents, and Ralph-style practitioner workflows (Qian et al., 2023; Lu et al., 2024; Huntley, 2025). From framing to benchmark topologies. The topologies in PerspectiveGap are built around this loop-centered view. Each topology specifies a set of roles and handoffs; each scenario then asks the model to write prompts that give each role the con- text it needs and the handoff constraints it must follow, while excluding distractor material. Fig- ure 2 shows the six base topology patterns used in the benchmark. The release also includes four pool variants that add parallel evaluator-candidate structure to the producer side. Together, these 10 topologies increase role and handoff complexity while preserving the same loop-centered design principle. 5 Experiments Setup. We evaluate 27 commercial models on both tasks across all 110 scenarios at two shuffle seeds (1 and 42), yielding27×110×2×2 = 11,880 evaluations. Main leaderboard.Appendix A reports the full model leaderboard and per-task score-consistency metrics for all 27 evaluated models. GPT-5.5 is the clear outlier at 62.0%, nearly twice the second- place model, deepseek-v4-pro, at 32.0%. Table 2 summarizes the same leaderboard at the company level by taking each company’s best model. Only three companies exceed1/4in their best-model score, and Anthropic is notably weak despite its strong coding reputation (Wang et al., 2025). Score consistency. Because Strict pass is inten- tionally all-or-nothing, we pair it with partial-credit and diagnostic metrics that expose different failure modes. We then ask whether Net match score pre- serves the same model-level signal as Strict pass, rather than producing a different ranking driven by boundary details. Table 3 reports Pearson correla- tions across the 27 evaluated models. Net match score has the strongest linear association with Strict pass (r = 0.744), while Required coverage, Bound- ary precision, and Distractor leakage are weaker diagnostics of the strict endpoint. The relationship between Net match score and Strict pass is non-linear, so here we use an idealized error model to explain why a non-linear recovery is expected. Letcdenote Net match score ands denote Strict pass. A scenario has an easy part that most models solve and a harder fractionp, where a model makes an error on each hard event with probabilityεand a strict pass requires aboutn hard events to be correct. Under these simplifying 6 CompanyBest modelRole-fragment assignmentFree-form prompt writingCombined OpenAI gpt-5.555.5%68.6%62.0% DeepSeek deepseek-v4-pro37.3%26.8%32.0% Kimi kimi-k2.626.8%25.0%25.9% Google gemini-3.1-pro20.0%23.6%21.8% Z.ai glm-525.9%14.5%20.2% Xiaomi mimo-v2.5-pro25.0%15.0%20.0% Anthropic claude-opus-4-724.1%14.1%19.1% Qwen qwen3.6-plus25.5%8.6%17.0% xAI grok-4.316.4%14.5%15.5% MiniMax minimax-m2.716.4%5.9%11.1% Table 2: Best model per company under the main Strict-pass metric. Combined is the unweighted average of role-fragment assignment and free-form prompt-writing. MetricNet match Coverage Precision Leakage Pearson r0.7440.4970.615-0.501 Table 3: Pearson correlation with Strict pass across 27 evaluated models. assumptions, c = 1−2pε, s = (1−ε) n = 1− 1− c 2p n . (1) Equivalently, this suggests thatsshould be well ap- proximated by the polynomial span1,c,...,c n . Table 4 confirms this, with the multiple correlation rising to 0.942 atn = 5. This supports treating Strict pass as a conservative operational endpoint. It remains strict about any single boundary fail- ure while tracking the same model-level capability signal as the partial-credit score. n12345 r0.7440.8550.9130.9350.942 Table 4: Multiple correlation between Strict pass and polynomial bases of Net match score. As a special case, setting p = 1 in Eq. (1) gives lns = n ln ((1 + c)/2),(2) a one-parameter log-parity fit that reaches a log- space correlation of 0.920 (Figure 3). Information leakage. Table 5 separates distrac- tor leakage from overall information leakage for the best model from each company and includes the all-model mean used in the abstract. These are role-fragment event rates, not binary scenario or role rates: if one scenario assigns two extra fragments to one role and three to another, it con- tributes five leak events. Even GPT-5.5, which has Figure 3: Log-parity fit between Strict pass and Net match score under the p = 1 special case of Eq. 1. only 2.3% distractor leakage, reaches 49.1% over- all leakage; averaged over all 27 models, overall leakage reaches 246.5%. 23456 Number of roles 0 10 20 30 40 Pass rate (%) Role-fragment assignment Free-form prompt writing Figure 4: Pass rate by number of roles, aggregated across all 27 commercial models. Difficulty by role count. Figure 4 aggregates pass rates by the number of roles in a scenario, pool- ing all models and shuffle seeds. Overall, scenarios with more roles tend to be harder. The four-role bin is an exception because it contains only one topol- ogy,dispatcher_scientist_coder_reviewer, 7 ModelDistractor leakage Overall leakage gpt-5.52.3%49.1% deepseek-v4-pro42.3%88.2% kimi-k2.630.0%90.0% gemini-3.1-pro24.1%24.5% glm-532.7%100.9% mimo-v2.5-pro76.4%168.2% claude-opus-4-7104.5%135.0% qwen3.6-plus59.5%80.5% grok-4.37.3%91.4% minimax-m2.732.7%99.1% All-model mean75.3%246.5% Table 5: Role-fragment assignment leakage rates for the best model from each company in Table 2, with the all-model mean over all 27 evaluated models shown in the final row. Distractor leakage counts only the injected distractor fragment; overall leakage counts any fragment outside the receiving role’s reference need- set. Both rates average role-fragment leak events over scenarios, so values can exceed 100%. which is relatively simple. The free-form prompt- writing task is also slightly harder than role- fragment assignment in most role-count bins. Ablation studies. We also use PerspectiveGap for four targeted ablation studies: few-shot prompt- ing, distractor count, scratchpad prompting, and reasoning effort. These experiments test how stan- dard prompting techniques, inference-time inter- ventions, and the number of distractors affect per- formance; details are in Appendix C, D, E and F. 6 Common Failure Modes Distractor leakage. Main agents often pass in- formation unrelated to a sub-agent’s work into that sub-agent’s prompt; this is clearest when distrac- tors are inserted. In PerspectiveGap, distractors are prompt-engineering tips that are useful to the main agent while it writes prompts. Models often treat this as generally useful context and pass it to sub-agents, showing that they do not reliably rea- son from the sub-agent’s perspective. This failure is common even among strong models. Table 5 reports distractor leakage for the best model from each company: GPT-5.5 is the best case at 2.3%, while several other leading models leak distractors at much higher rates. Out-of-role information leakage. A more se- rious form of leakage occurs when a fragment needed by one role is copied into another role’s prompt. This is not merely redundant context; it changes what the receiving role is allowed to know. In a software-engineering orchestra, for example, a coder might receive the task goal and public tests, while a reviewer or test engineer receives private tests. If the main agent gives the private tests to the coder, the coder can optimize for the hidden evalu- ation instead of solving the intended problem. That is a reward-hacking channel created by the orches- tration prompt itself. The overall leakage column in Table 5 captures this broader failure mode: even GPT-5.5 reaches 49.1% overall leakage, despite its low distractor leakage, and several models exceed 80%. The difference between distractor leakage and overall leakage shows that models are not only leaking irrelevant distractors; they also fail to pre- serve role-specific information boundaries. Artifact ownership and handoff confusion. Models also confuse which role owns which ar- tifact. In thedispatcher_planloop_codeloop topology, for example, the plan loop writes PLAN.mdand the code loop writesSOLUTION.md. A failed orchestration prompt can swap these bound- aries, sending instructions forSOLUTION.mdto the plan creator or plan critic, and instructions for PLAN.mdto the code loop. This breaks the file pro- tocol itself. Table 6 gives a fragment-level view of this problem for a near-parallel handoff pair indpc anddpcp, averaged over all 27 evaluated models. 3 Fordpc, the two parallel fragments have leak rates of 23.7% and 26.6%, showing that models often send the handoff to a non-owner role. This is the practical bottleneck for automating multi-agent or- chestration prompting: the model does not reliably take each sub-agent’s perspective and ask what that role needs, so engineers still have to inspect and repair the generated prompts. Topology f -idmissingleak dpcf 10 7.6%23.7% dpcf 11 9.6%26.6% dpcpf 10 9.8%0.7% dpcpf 11 17.0%7.6% Table 6: Missing and leakage rates for the near-parallel handoff pair f 10 and f 11 in dpc and dpcp. Dropped shared context.Models also fail in the other direction: they omit context that a role actu- ally needs. Shared background is especially easy to mishandle because it belongs to multiple roles, not just to the role whose task looks most directly re- lated. Table 7 shows this pattern forf 1 , the shared 3 Topology aliases are listed in Appendix Table 11. 8 background fragment, averaged over all 27 evalu- ated models. Across topologies, models often omit this fragment from roles that need it, with miss rates ranging from 12.1% to 38.7% for actor-style roles and 17.3% to 44.9% for reviewer-style roles. This is a different failure from over-sharing: the sub-agent receives a clean-looking prompt, but the prompt lacks the background needed to evaluate or complete the work. TopologyActor missReviewer miss cr38.7%26.9% crp12.1%32.0% dcr16.0%44.9% dcrp13.8%36.5% dpc15.2%26.3% dpcp20.2%21.4% scr16.3%22.9% dtc15.2%26.1% dtcp17.5%24.6% spl12.6%17.3% Table 7: Role-fragment assignment miss rates forf 1 , the shared background fragment, from actor-style and reviewer-style roles. Topology aliases are listed in Ap- pendix Table 11. Bootstrap paradox. Some failures are control- flow errors rather than simple include-or-exclude mistakes. One form places the instruction to read an artifact inside the artifact itself: before reading it the agent cannot see the instruction, and after reading it the instruction is redundant. Another form adds a no-go rule that names an otherwise absent action, e.g., “do not dox,” even thoughx is so out-of-scope that the sub-agent would not have considered it. This is the prompt equivalent of the Inception example: telling someone not to think about elephants. Both cases reveal a failure of perspective-taking: the agent cannot reason from another agent’s point of view. 7 Conclusion We introduced PerspectiveGap, a benchmark for evaluating whether LLMs can write role-specific prompts for multi-agent orchestration. Its ten topologies are loop-centered patterns favored by the Prompt Economy principle: benefit accrues across repeated role invocations while engineering cost stays fixed by the number of role prompts. Across 110 scenarios and 27 commercial models, the results show that current models still struggle to assign context according to the need-only rule, even when the required information is explicitly present in the prompt. The failures are not cos- metic: models leak distractors, expose out-of-role information, drop shared context, confuse artifact ownership, and sometimes place instructions where the sub-agent cannot see them. That even coding- strong models such as Opus 4.7 fail here indicates that orchestration prompting is a capability distinct from the coding ability current benchmarks reward. These results suggest that orchestration prompt- ing remains a fragile intermediate step. Generated sub-agent prompts should not be assumed to pre- serve role-specific information boundaries without inspection. Limitations Coverage.PerspectiveGap covers 10 topologies and 100 domain instances. It does not cover all possible multi-agent orchestration patterns, and fu- ture work can add new topology templates under the same construction rule. Prompt artifact rather than execution. Per- spectiveGap evaluates the prompts written for sub- agents, not the downstream behavior of the sub- agents that would consume those prompts. This scope is deliberate. Sub-agent prompts are the handoff artifact that assigns context and constraints, and they can be inspected across domains without defining a separate downstream task for each do- main. Running downstream agents across all do- mains would test whether prompt-boundary errors translate into task failures, but requires a separate runtime and success criterion for each domain. Reference mapping and scoring. The role-to- fragment mappings are authored under the need- only rule: a fragment is assigned to a role if that role needs it to discharge its documented respon- sibility. The full five-author team internally au- dited the mappings for consistency, but this does not establish external annotator agreement; future versions could measure such agreement under the same rule. The free-form prompt-writing scorer is deterministic and LLM-free, but it can penalize high-quality paraphrases that no longer preserve enough fragment-specific surface evidence. Code and Data Availability The benchmark data, rendering code, scoring scripts, and model-running utilities are available at WhymustIhaveaname/PerspectiveGap. 9 References Christopher Ackerman. 2026. Selective deficits in LLM mental self-modeling in a behavior-based test of the- ory of mind. Preprint, arXiv:2603.26089. Anthropic. 2024. Building effective agents.https: //w.anthropic.com/engineering/building -effective-agents. M. Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya G. Parameswaran, Dan Klein, K. Ramchandran, Matei Zaharia, Joseph E. Gonza- lez, and Ion Stoica. 2025. Why do multi-agent llm systems fail? Preprint, arXiv:2503.13657. Edward Y. Chang and Longling Geng. 2025. Sagallm: Context management, validation, and transaction guarantees for multi-agent llm planning. Preprint, arXiv:2503.11951. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen-Ming Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. 2023. AgentVerse: Facilitating multi- agent collaboration and exploring emergent behav- iors. Preprint, arXiv:2308.10848. Minh-Dung Dao, Quy Minh Le, Hoang Thanh Lam, Duc-Trong Le, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D. Nguyen. 2026. Agentic design pat- terns: A system-theoretic framework.Preprint, arXiv:2601.19752. Jianda Du, Youran Sun, and Haizhao Yang. 2026. Au- tonumerics: An autonomous, pde-agnostic multi- agent pipeline for scientific computing. Preprint, arXiv:2602.17607. Kanishk Gandhi, Jan-Philipp Fränken, Tobias Gersten- berg, and Noah D. Goodman. 2023. Understanding social reasoning in language models with language models. In Advances in Neural Information Process- ing Systems 36. Yuxuan Gao, Megan Wang, Yiyao Yu, Zijian Ma, and A. Qu. 2026. Decisionbench: A benchmark for emer- gent delegation in long-horizon agentic workflows. Preprint, arXiv:2605.19099. Yuling Gu, Oyvind Tafjord, Hyunwoo Kim, Jared Moore, Ronan Le Bras, Peter Clark, and Yejin Choi. 2024. SimpleToM: Exposing the gap between ex- plicit ToM inference and implicit ToM application in LLMs. Preprint, arXiv:2410.13648. Antonio Gullí. 2025.Agentic Design Patterns: A Hands-On Guide to Building Intelligent Systems. Springer Cham. Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2023. Metagpt: Meta pro- gramming for a multi-agent collaborative framework. Preprint, arXiv:2308.00352. Geoffrey Huntley. 2025. Ralph wiggum as a “software engineer”. https://ghuntley.com/ralph/. Hyunwoo Kim, Melanie Sclar, Zhi-Xuan Tan, Lance Ying, Sydney Levine, Yang Liu, Joshua B. Tenen- baum, and Yejin Choi. 2025. Hypothesis-driven theory-of-mind reasoning for large language mod- els. Preprint, arXiv:2502.11881. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. FANToM: A benchmark for stress-testing machine theory of mind in interactions. Preprint, arXiv:2310.15421. Matthew Le, Y-Lan Boureau, and Maximilian Nickel. 2019. Revisiting the evaluation of theory of mind through question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natu- ral Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative agents for “mind” explo- ration of large language model society. Preprint, arXiv:2303.17760. Yuxuan Li, Aoi Naito, and Hirokazu Shirado. 2025. Systematic failures in collective reasoning under dis- tributed information in multi-agent llms. Preprint, arXiv:2505.11556. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Ao- han Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, and 3 oth- ers. 2023. AgentBench: Evaluating LLMs as agents. Preprint, arXiv:2308.03688. Yue Liu, Sin Kit Lo, Qinghua Lu, Liming Zhu, De- hai Zhao, Xiwei Xu, Stefan Harrer, and Jon Whittle. 2024. Agent design pattern catalogue: A collection of architectural patterns for foundation model based agents. Preprint, arXiv:2405.10467. Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foer- ster, Jeff Clune, and David Ha. 2024. The ai scientist: Towards fully automated open-ended scientific dis- covery. Preprint, arXiv:2408.06292. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdan- bakhsh, and Peter Clark. 2023.Self-refine: It- erative refinement with self-feedback.Preprint, arXiv:2303.17651. 10 Zhanfeng Mo, Xingxuan Li, Yuntao Chen, and Lidong Bing. 2025. Multi-agent tool-integrated policy opti- mization. Preprint, arXiv:2510.04678. Abdelghny Orogat, Ana Rostam, and Essam Man- sour. 2026. Understanding multi-agent llm frame- works: A unified benchmark and experimental analy- sis. Preprint, arXiv:2602.03128. Yunjia Qi, Hao Peng, Xiaozhi Wang, Amy Xin, Yung- Chuan Liu, Bin Xu, Lei Hou, and Juanzi Li. 2025. Agentif: Benchmarking instruction following of large language models in agentic scenarios. Preprint, arXiv:2505.16944. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. Chatdev: Communica- tive agents for software development.Preprint, arXiv:2307.07924. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. Preprint, arXiv:2307.16789. Matthew Riemer, Zahra Ashktorab, Djallel Bounef- fouf, Payel Das, Miao Liu, Justin D. Weisz, and Murray Campbell. 2025. Position: Theory of mind benchmarks are broken for large language models. Preprint, arXiv:2412.19726. Melanie Sclar, Jane Yu, Maryam Fazel-Zarandi, Yulia Tsvetkov, Yonatan Bisk, Yejin Choi, and Asli Ce- likyilmaz. 2024. Explore theory of mind: Program- guided adversarial data generation for theory of mind reasoning. Preprint, arXiv:2412.12175. Joykirat Singh, Zaid Khan, Archiki Prasad, J. Chen, Ak- shay Nambi, Hyunji Lee, Elias Stengel-Eskin, and Mohit Bansal. 2026. Agent-brace: Decoupling be- liefs from actions in long-horizon tasks via verbalized state uncertainty. Preprint, arXiv:2605.11436. Youran Sun, Yixin Wen, and Haizhao Yang. 2026. Research: A multi-stage machine learning frame- work for earth science data discovery. Preprint, arXiv:2601.14176. Raghav Thind, Youran Sun, Ling Liang, and Haizhao Yang. 2025. Optimai: Optimization from natural language using llm-powered ai agents. Preprint, arXiv:2504.16918. Khanh-TungTran,DungDao,Minh-Duong Nguyen, Quoc-Viet Pham, Barry O’Sullivan, and Hoang D. Nguyen. 2025. Multi-agent collab- oration mechanisms: A survey of llms. Preprint, arXiv:2501.06322. Lilin Wang, Lucas Ramalho, Alan Celestino, Phuc An- thony Pham, Yu Liu, Umang Kumar Sinha, Andres Portillo, Onassis Osunwa, and Gabriel Maduekwe. 2025. Swe-bench++: A framework for the scal- able generation of software engineering bench- marks from open-source repositories.Preprint, arXiv:2512.17419. Alex Wilf, Sihyun Lee, P. Liang, and Louis-Philippe Morency. 2023. Think twice: Perspective-taking improves large language models’ theory-of-mind ca- pabilities. Preprint, arXiv:2311.10227. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadal- lah, Ryen W. White, Doug Burger, and Chi Wang. 2023a.AutoGen: Enabling next-gen LLM ap- plications via multi-agent conversation. Preprint, arXiv:2308.08155. Yufan Wu, Yinghui He, Yilin Jia, Rada Mihalcea, Yu- long Chen, and Naihao Deng. 2023b.Hi-ToM: A benchmark for evaluating higher-order theory of mind reasoning in large language models. In Find- ings of the Association for Computational Linguistics: EMNLP 2023. Hainiu Xu, Runcong Zhao, Lixing Zhu, Jinhua Du, and Yulan He. 2024. OpenToM: A comprehensive bench- mark for evaluating theory-of-mind reasoning capa- bilities of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Faouzi El Yagoubi, Ranwa Al Mallah, and Godwin Badu-Marfo. 2026. Agentleak: A full-stack bench- mark for privacy leakage in multi-agent llm systems. Preprint, arXiv:2602.11510. Jianzhu Yao, Kevin Wang, Ryan Hsieh, Haisu Zhou, Tianqing Zou, Zerui Cheng, Zhangyang Wang, and P. Viswanath. 2025. Spin-bench: How well do llms plan strategically and reason socially?Preprint, arXiv:2503.12349. Yashwanth YS, Ruichen Wang, Shihua Zeng, Xuhui Zhou, Koichi Onoue, Vasudha Varadarajan, and Maarten Sap. 2026. SOTOPIA-TOM: Evaluating information management in multi-agent interaction with theory of mind. Preprint, arXiv:2605.02307. Jusheng Zhang, Yijia Fan, Kaitong Cai, Xiaofei Sun, and Keze Wang. 2025a. Osc: Cognitive orchestra- tion through dynamic knowledge alignment in multi- agent llm collaboration. Preprint, arXiv:2509.04876. Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang, Huazheng Wang, Yiran Chen, and Qingyun Wu. 2025b. Which agent causes task failures and when? on automated failure attribution of llm multi- agent systems. Preprint, arXiv:2505.00212. Yuzhe Zhang, Feiran Liu, Yizhou Shan, Xinyi Huang, Xin Yang, Yue Zhu, Xuxin Cheng, Cao Liu, Ke Zeng, 11 Terry Jingchen Zhang, and Wenyuan Jiang. 2026. Silo-bench: A scalable environment for evaluating distributed coordination in multi-agent llm systems. Preprint, arXiv:2603.01045. 12 Contents of Appendix Appendix AFull Model Leaderboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Appendix BAdditional Topology Details . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Appendix CFew-Shot Prompting Ablation . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Appendix DDistractor-count Ablation. . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 Appendix EScratchpad Ablation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Appendix FReasoning Effort Ablation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Appendix GScorer Agreement with Human Labels. . . . . . . . . . . . . . . . . . . . . .16 Appendix HRole-Fragment Assignment Trivial Baselines . . . . . . . . . . . . . . . . . . .16 Appendix IProfessional-Domain Coverage. . . . . . . . . . . . . . . . . . . . . . . . . .16 Appendix JConcrete Reference-Mapping Example. . . . . . . . . . . . . . . . . . . . . .17 A Full Model Leaderboard Table 8 reports the full model leaderboard used for the main experimental comparison. The combined score is the unweighted average of role-fragment assignment Strict pass and free-form prompt-writing Strict pass. Tables 9 and 10 report the score-consistency metrics for the two task formats. CompanyModelRole-fragment assignment Free-form prompt writing Combined OpenAI gpt-5.555.5%68.6%62.0% DeepSeek deepseek-v4-pro37.3%26.8%32.0% Kimi kimi-k2.626.8%25.0%25.9% DeepSeek deepseek-v4-flash24.5%25.0%24.8% Google gemini-3.1-pro20.0%23.6%21.8% Z.ai glm-525.9%14.5%20.2% Xiaomi mimo-v2.5-pro25.0%15.0%20.0% Anthropic claude-opus-4-724.1%14.1%19.1% Qwen qwen3.6-plus25.5%8.6%17.0% Anthropic claude-sonnet-4-65.9%25.5%15.7% xAI grok-4.316.4%14.5%15.5% Kimi kimi-k2.513.6%16.8%15.2% Qwen qwen3.7-max20.0%8.6%14.3% OpenAI gpt-5.425.5%2.3%13.9% Qwen qwen3.5-9b16.4%9.1%12.7% Xiaomi mimo-v2.514.1%11.4%12.7% MiniMax minimax-m2.716.4%5.9%11.1% Google gemini-3.5-flash19.1%0.9%10.0% Z.ai glm-5.16.4%12.7%9.5% Anthropic claude-haiku-4-53.2%8.2%5.7% MiniMax minimax-m2.55.9%5.0%5.5% xAI grok-4.201.4%8.6%5.0% Google gemini-3.1-flash-lite2.7%4.1%3.4% Z.ai glm-4.7-flash3.2%3.6%3.4% Xiaomi mimo-v2-flash3.6%1.8%2.7% OpenAI gpt-5.4-mini0.9%2.7%1.8% OpenAI gpt-5.4-nano0.0%0.9%0.5% Table 8: Full PerspectiveGap leaderboard over all 27 evaluated commercial models. Models are sorted by combined pass rate. B Additional Topology Details Table 11 lists the topology aliases used in the appendix tables and the number of roles in each template. Figure 5 reports the combined pass rate for each model and topology. C Few-Shot Prompting Ablation This ablation asks whether few-shot examples improve accuracy on PerspectiveGap’s free-form prompt- writing task. We run it on gpt-5.4-mini, claude-haiku-4-5, and gemini-3.5-flash, where weak 0-shot performance leaves more room for few-shot examples to help. The comparison must avoid giving away the answer through superficial overlap. Several hand-written templates share role names or fragment wording, so we manually choose examples whose topology, roles, and content differ from the evaluation slice. The four settings are listed in Table 12; each setting evaluates the ten instances under one target 13 ModelStrict pass Net match Required coverage Boundary precision Distractor leakage gpt-5.555.5%92.6%96.2%96.2%2.3% deepseek-v4-pro37.3%85.4%95.8%93.4%42.3% kimi-k2.626.8%66.7%79.7%92.0%30.0% deepseek-v4-flash24.5%77.6%94.9%88.5%17.7% gemini-3.1-pro20.0%60.9%63.8%97.1%24.1% glm-525.9%80.2%94.1%92.4%32.7% mimo-v2.5-pro25.0%75.3%92.6%87.7%76.4% claude-opus-4-724.1%79.4%94.2%90.1%104.5% qwen3.6-plus25.5%80.5%93.3%93.8%59.5% claude-sonnet-4-65.9%67.5%89.9%87.5%83.6% grok-4.316.4%68.6%87.0%92.5%7.3% kimi-k2.513.6%73.6%92.9%89.2%88.2% qwen3.7-max20.0%73.7%86.2%95.0%55.0% gpt-5.425.5%78.3%93.4%91.6%47.3% qwen3.5-9b16.4%66.3%86.5%89.5%51.4% mimo-v2.514.1%69.3%89.5%89.3%50.9% minimax-m2.716.4%50.8%62.5%89.1%32.7% gemini-3.5-flash19.1%59.1%65.8%87.3%101.4% glm-5.16.4%58.3%87.7%85.0%97.3% claude-haiku-4-53.2%40.1%77.1%78.8%133.6% minimax-m2.55.9%52.6%80.0%87.3%57.3% grok-4.201.4%56.2%92.0%77.4%184.5% glm-4.7-flash3.2%33.5%69.2%66.3%124.1% gemini-3.1-flash-lite2.7%32.0%59.0%70.9%193.2% mimo-v2-flash3.6%39.3%83.6%75.7%130.9% gpt-5.4-mini0.9%21.4%77.6%64.9%174.5% gpt-5.4-nano0.0%7.9%35.3%18.3%29.5% Table 9: Full score-consistency metrics for role-fragment assignment over all 27 evaluated models. gpt-5.5 deepseek-v4-pro kimi-k2.6 deepseek-v4-flash gemini-3.1-pro glm-5 mimo-v2.5-pro claude-opus-4-7 qwen3.6-plus claude-sonnet-4-6 grok-4.3 kimi-k2.5 qwen3.7-max gpt-5.4 mimo-v2.5 qwen3.5-9b minimax-m2.7 gemini-3.5-flash glm-5.1 claude-haiku-4-5 minimax-m2.5 grok-4.20 gemini-3.1-flash-lite glm-4.7-flash mimo-v2-flash gpt-5.4-mini gpt-5.4-nano cr crp dcr dcrp dpc dpcp scr tl tlp spl 844820529364511162111691130254116501618716222 8450417048344832303064341834393634181616181418187140 864539254518323930361618272799530752990000 75522034482511113232141618111614023292000000 3992002702009220022071100200000 451490140057700200000000000000 8975805048525068432539596645273030245149000900 27014027020200000000000000000 1495020202002200000000000000 7718111602311211149509511259147700922 0 10 20 30 40 50 60 70 80 90 Combined pass rate (%) Figure 5: Combined pass rate by topology and model. Darker squares indicate higher pass rates. topology. The main benchmark has 10 topology templates, 100 instances, and 110 scenarios in total once the 10 hand-written templates are included. The 2-shot setting uses the same 5-role evaluation slice as 1-shot-2, isolating whether a second example helps. Figure 6 shows that the worked example helpsgemini-3.5-flashin the smaller settings, with +80 p on 1-shot-1 and +30 p on 1-shot-3. The same model gets little or no lift in the 5-role settings. Forclaude-haiku-4-5andgpt-5.4-mini, no evaluated output crosses the strict pass threshold in any setting; this does not rule out sub-threshold improvements. The leakage panel shows why pass-rate lift alone is incomplete:gemini-3.5-flashreduces leakage on 1-shot-1, whileclaude-haiku-4-5and gpt-5.4-mini leak more under the same intervention. D Distractor-count Ablation To study how leakage changes as irrelevant context grows, we rerun role-fragment assignment with 0, 1, 2, or 3 injected distractors on gpt-5.5, gpt-5.4, claude-opus-4-7, claude-sonnet-4-6, gemini-3.1-pro, and gemini-3.5-flash. We choose this six-model panel to cover flagship and fast models from OpenAI, Anthropic, and Google. Figure 7 plots the model-averaged distractor leakage and overall leakage at each distractor count. Both rates rise sharply after the first distractor; overall leakage continues to increase 14 ModelStrict pass Net match Required coverage Boundary precision Distractor leakage gpt-5.568.6%91.5%92.8%99.0%0.9% deepseek-v4-pro26.8%82.7%95.4%92.2%71.4% kimi-k2.625.0%69.0%80.8%95.5%32.7% deepseek-v4-flash25.0%78.6%94.4%89.2%33.2% gemini-3.1-pro23.6%62.0%63.9%94.2%48.6% glm-514.5%70.9%93.1%85.1%141.8% mimo-v2.5-pro15.0%74.3%89.9%90.4%51.8% claude-opus-4-714.1%82.3%94.4%92.7%65.5% qwen3.6-plus8.6%69.0%82.7%90.6%90.9% claude-sonnet-4-625.5%80.3%91.7%95.6%35.0% grok-4.314.5%74.6%88.7%94.6%12.3% kimi-k2.516.8%72.0%89.1%89.1%110.9% qwen3.7-max8.6%63.8%85.7%85.0%177.3% gpt-5.42.3%71.9%95.6%84.0%174.5% qwen3.5-9b9.1%47.1%65.7%88.6%46.8% mimo-v2.511.4%69.6%89.2%90.6%42.7% minimax-m2.75.9%33.5%46.6%88.6%36.8% gemini-3.5-flash0.9%51.3%65.5%80.3%197.3% glm-5.112.7%73.1%88.8%92.9%54.5% claude-haiku-4-58.2%66.1%84.4%95.8%16.4% minimax-m2.55.0%48.2%70.1%91.8%30.0% grok-4.208.6%67.9%93.2%82.2%143.2% glm-4.7-flash3.6%32.4%61.7%85.6%76.8% gemini-3.1-flash-lite4.1%34.9%40.4%82.6%95.0% mimo-v2-flash1.8%48.1%78.3%85.5%73.6% gpt-5.4-mini2.7%25.2%58.2%84.6%35.5% gpt-5.4-nano0.9%20.7%48.4%81.6%24.1% Table 10: Full score-consistency metrics for free-form prompt writing over all 27 evaluated models. AliasTopologyRoles cr coder_reviewer2 crp coder_reviewer_pool2 dcr dispatcher_coder_reviewer3 dcrp dispatcher_coder_reviewer_pool3 dpc dispatcher_planloop_codeloop5 dpcp dispatcher_planloop_codeloop_pool6 scr dispatcher_scientist_coder_reviewer4 dtc dispatcher_theoryloop_codeloop5 dtcp dispatcher_theoryloop_codeloop_pool5 spl supervisor_phdstudent_librarian3 Table 11: Additional details for the 10 topology templates used in PerspectiveGap. through three distractors, reaching 122.3% on average. E Scratchpad Ablation Prior work reports that giving models a scratchpad can improve accuracy on self-modeling tasks (Ack- erman, 2026). We test whether the same trick helps in PerspectiveGap’s free-form prompt-writing task. We use the same model set as Appendix C: gpt-5.4-mini, claude-haiku-4-5, and gemini-3.5-flash. For each model, we rerun the task on all 110 scenarios. The treatment adds an instruction to first write a hidden scratchpad block listing which fragments each sub-agent needs and why. The scratchpad block is stripped before scoring, so the scorer checks only the final orchestra text. Figure 8 shows that the scratchpad does not improve strict pass rate. It leaves claude-haiku-4-5 unchanged at 2.7%, and moves gemini-3.5-flash and gpt-5.4-mini from 1.8% to 0.0%. The leakage effect is model-specific: gemini-3.5-flash drops from 42.7% to 0.9%, while claude-haiku-4-5 rises from 17.3% to 40.9% and gpt-5.4-mini rises from 44.5% to 83.6%. Scratchpads change how these small models handle distractors, but in this setting they do not solve free-form prompt writing. F Reasoning Effort Ablation We ask how free-form prompt writing changes when the same model is given less or more reasoning effort. We compare gpt-5.5 and claude-sonnet-4-6 at low, medium, and high effort on the same 30-scenario 15 1-shot-1 2 roles 1-shot-2 5 roles 1-shot-3 3 roles 2-shot 5 roles 0 20 40 60 80 Few-shot 0-shot (p) 1-shot-1 2 roles 1-shot-2 5 roles 1-shot-3 3 roles 2-shot 5 roles 25 0 25 50 75 0-shot few-shot (p) gemini-3.5-flashclaude-haiku-4-5gpt-5.4-mini Figure 6: Few-shot effects on free-form prompt writing. The left panel shows pass-rate lift, measured as few-shot minus 0-shot. The right panel shows leakage reduction, measured as 0-shot minus few-shot. Higher is better in both panels. SettingEval. topologyRolesExample topology 1-shot-1 cr2 spl 1-shot-2 dtcp5 cr 1-shot-3 spl3 dpc 2-shot dtcp5 cr, spl Table 12: Few-shot settings used in the ablation. Each evaluation setting uses the ten instances under one target topology. Example topologies were selected manually to avoid shared topology, role names, and fragment wording with the evaluation topology. Topology aliases are listed in Table 11. subset used in Appendix C. Figure 9 shows medium effort as the pass-rate sweet spot for both models. gpt-5.5 falls from 63.3% at medium effort to 53.3% at both low and high effort. claude-sonnet-4-6 rises from 0.0% at low effort to 16.7% at medium effort, then drops to 6.7% at high effort. More effort also makes distractor use worse: gpt-5.5 leaks more at high effort than at medium effort, and claude-sonnet-4-6 leakage rises from 23.3% to 40.0% across the sweep. G Scorer Agreement with Human Labels For the free-form prompt-writing task, the scorer test set contains 716 rows sampled from benchmark outputs and labeled by human annotators for fragment containment. It is used only to validate the rule scorer; the scorer itself reads only the benchmark reference mapping at evaluation time and does not consult any human labels. MetricValue Percent agreement99.44% (712/716) F1 (pass class)99.57% Precision (pass)99.78% Recall (pass)99.36% Table 13: Scorer agreement with human labels on all 716 free-form prompt writing test-set rows. H Role-Fragment Assignment Trivial Baselines To validate that the leaderboard is not driven by surface shortcuts, we analytically score three trivial baselines on the same role-fragment assignment set-equality scorer used in the main results. Table 14 reports the results. All three baselines reach a 0.0% pass rate, so the nonzero leaderboard scores cannot be explained by copy-all, role-name keyword matching, or random assignment. I Professional-Domain Coverage Table 15 summarizes the domains used in PerspectiveGap. 16 0123 Number of distractors 125 100 75 50 25 0 Mean leakage rate (%) Distractor leakageOverall leakage Figure 7: Role-fragment assignment leakage as the number of injected distractors increases. Downward bars indicate leakage. Each bar averages over a six-model panel: one flagship and one fast model from OpenAI, Anthropic, and Google. gemini-3.5-flashclaude-haiku-4-5gpt-5.4-mini 0 1 2 3 4 5 Pass rate (%) gemini-3.5-flashclaude-haiku-4-5gpt-5.4-mini 80 60 40 20 0 Distractor leak rate (%) no scratchpadscratchpad Figure 8: Free-form prompt writing pass rate (left) and distractor leak rate (right) on three small models, with and without a hidden scratchpad block. Downward bars in the right panel indicate leakage. J Concrete Reference-Mapping Example This appendix makes the reference policy auditable on an actual benchmark scenario. We use the dispatcher_theoryloop_codeloopscenario. It shows the background given to the model, the fragment headings, the “need-only” instruction, the reference assignment, and textual evidence for representative boundary decisions. I need you to set up a 5-agent pipeline that solves mathematical optimization problems by developing theoretical analysis and solver code in parallel, with each side informing the other. A problem arrives as a natural-language statement in problem.md; read it before starting your part. The five agents: •dispatcher orchestrates the run by alternating one theory step with one code step and tracking progress. It does no modeling, coding, or reviewing itself. •theory loop — the theorist writes the theoretical analysis toTHEORY.md; the theory-reviewer judges it. The theorist reads SOLUTION.md on each revision pass to incorporate empirical evidence. • code loop — the coder writes solver code and its results toSOLUTION.md; the code-reviewer judges them. The coder readsTHEORY.mdon each revision pass to align the implementation with the latest theoretical claims. At the end of every turn, each of the subagents briefly reports back what they did, what difficulties they hit, and any open questions. The pipeline hands work off through three files: • problem.md — the original problem, read-only. All four domain agents read it. • THEORY.md— the theoretical analysis. Theorist writes; theory-reviewer reviews; coder and code-reviewer read it to know which bounds and invariants the implementation must respect. • SOLUTION.md— the solver code and its results. Coder writes; code-reviewer reviews; theorist and theory- reviewer read it to ground the theory in observed numbers. Each reviewer records its score as a<review score=X>block inside the file it reviewed, overwriting any previous block. The dispatcher reads those scores to drive the loops. Unlike a plan-then-code pipeline where the plan must converge before the code starts, here neither side ever finishes first. Both loops alternate forever, capped at a fixed iteration count. Theory may tighten after seeing code’s empirical numbers; code may correct after seeing theory’s refined bounds. 17 gpt-5.5sonnet-4.6 0 20 40 60 Pass rate (%) gpt-5.5sonnet-4.6 40 30 20 10 0 Distractor leak rate (%) lowmediumhigh Figure 9: Free-form prompt writing pass rate (left) and distractor leak rate (right) across three reasoning-effort levels. Each model has one bar for low, medium, and high effort; downward bars in the right panel indicate leakage. BaselineRole-fragment assignment pass rate Copy-all (every role gets every fragment)0.0% Role-keyword (assign fragment to roles whose name token appears in its heading)0.0% Random (each fragment to one uniform-random role, 20 random draws)0.0% Table 14: Three trivial baselines on the role-fragment assignment set-equality scorer. Professional domainScenariosExample role list (one scenario per domain) Medical, clinical trials, regulatory safety14care coordinator, diagnostician, diagnostic challenge physician, treatment planner, patient-safety reviewer Litigation, criminal defense, appellate13 docket coordinator, appellate strategist, red-team appellate counsel, brief drafter, opposing-counsel brief reviewer Cultural heritage, provenance, authentication11 authentication case manager, provenance researcher, provenance skeptic, conserva- tion science examiner, forensic methods auditor Intelligence, security operations, vulnerability research10secops director, threat modeler, security architect, red-teamer, vulnerability analyst Tax law, IRS audit, forensic accounting7audit docket clerk, tax credit preparer, IRS examining agent Civil and structural engineering, plan check7project coordinator, structural scheme designer, design revision engineer, code- compliance examiner, calculation package engineer, independent structural re- viewer Investigative journalism, editorial7assigning editor, investigative reporter, skeptical investigations editor, story writer, fact-checking editor Finance, quant trading, M&A, risk6deal orchestrator, valuation strategist, investment committee skeptic, term-sheet drafter, due-diligence auditor Drug discovery, pharmacology, toxicology4program director, pharmacologist, toxicology skeptic, trial designer, regulatory auditor Environmental permitting, impact assessment4permitting coordinator, impact assessment lead, regulatory challenge reviewer, EIA report author, permit-compliance auditor Aerospace mission operations, flight dynamics3 mission commander, astrodynamics architect, maneuver adjuster, orbital- mechanics reviewer, burn-sequence compiler, physics-simulator proxy Patent law and IP3patent attorney, patent examiner Policy, campaign, legislation3 policy coordinator, macro-theorist, intervention tuner, economic skeptic, legislation drafter, compliance stress-tester Other (game and screen production, fire incident response, mathematical proof verification, anomaly detection, culi- nary operations) 8showrunner, narrative architect, plot-hole critic, scene writer, continuity auditor Software engineering and research10coder, reviewer, dispatcher, scientist, theorist, phdstudent, librarian Table 15: Professional-domain coverage of PerspectiveGap. The prompt then shows the fragment headings in Table 16. We omit the fragment bodies here, but keep the headings because they show how the scenario separates actor instructions, reviewer instructions, orchestration, and reporting. After the fragment block, the prompt tells the model: “Each agent’s prompt should contain only the information that agent needs to do its job.” and the output format. Table 17 shows the resulting reference assignment. The main boundary decisions in this example are not arbitrary. They follow from the stated role responsibilities: 1. f12 belongs to all four domain agents because the background says “At the end of every turn, each of the subagents briefly reports back what they did, what difficulties they hit, and any open questions.” It does not belong to the dispatcher: the dispatcher receives those reports, butf12tells domain agents what to report. 18 FragmentHeading f1problem context f2theorist deliverable rule f3theory toolbox f4coder deliverable rule f5solution-doc writing rule f6theory-review rule f7code-review rule f8reviewer scoring rule f9dispatch rule f10theorist workflow f11coder workflow f12report-back instruction Table 16: Fragment headings for the dispatcher_theoryloop_codeloop example. RoleReference fragmentsRationale Dispatcher f9The background says the dispatcher “orchestrates the run by alter- nating one theory step with one code step and tracking progress” and “does no modeling, coding, or reviewing itself.” It therefore needs the dispatch rule only. Theorist f1, f2, f3, f10, f12The theorist needs the problem context, its deliverable rule, the theory toolbox, its workflow across turns, and the instruction to report back. Theory-reviewer f1, f3, f6, f8, f12 The theory-reviewer needs the problem context, enough theory-tool context to judge the analysis, the theory-review rule, the reviewer scoring rule, and the instruction to report back. Coder f1, f4, f5, f11, f12The coder needs the problem context, its deliverable rule, solution- document writing rules, its workflow across turns, and the instruc- tion to report back. Code-reviewer f1, f7, f8, f12The code-reviewer needs the problem context, the code-review rule, the reviewer scoring rule, and the instruction to report back. Table 17: Reference role-fragment assignment for one PerspectiveGap scenario. 2. f6 belongs to the theory-reviewer, not to the theorist. Its first sentence is “Review THEORY.md against problem.md and against the numbers reported in SOLUTION.md (if any).” Giving it to the theorist leaks the critic’s instructions into the actor’s prompt. 3. f2belongs to the theorist, not to the theory-reviewer. It begins with “Produce the theoretical analysis for the problem and write it to THEORY.md.” Although it is about theory, it specifies the actor’s deliverable rather than the reviewer’s job. 4. The code side follows the same boundary.f4begins with “Read problem.md and THEORY.md, then implement the solver:”;f5says “Keep SOLUTION.md lean.”; andf11says “The solver code and its results live in SOLUTION.md, and you may be invoked on it more than once.” These are coder-side instructions. By contrast,f7begins with “Review SOLUTION.md against problem.md and against THEORY.md,” and f8 says to “Score the work from 1 to 10”; those are reviewer-side instructions. 19

Instant research discovery