Paper deep dive
WorkflowOps: Learning Agent Collaboration Priors for Multi-Agent Workflow Orchestration
Qi Cheng, Shengyu Chen, Wei Cheng, Zhengzhang Chen, Xiaowei Jia, Haoyu Wang, Haifeng Chen
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent systems are increasingly deployed for complex knowledge work, yet their orchestration layers remain largely memoryless: each new task is decomposed, assigned, and executed from scratch with no benefit from prior successful executions. We present WorkflowOps, a multi-agent workflow orchestration framework that learns agent collaboration priors from historical workflows and expands its agent pool on demand to cover new capability requirements. Our approach introduces three coupled mechanisms. First, a transition probability matrix captures pairwise agent collaboration frequencies from past workflows and applies them as soft guidance during DAG workflow construction through intra-layer ordering optimization, probability-thresholded edge suggestion, and transitive reduction for parallelism maximization. Second, a sufficiency-driven agent creation loop detects capability gaps via semantic matching scores, generates specialized agents through an LLM, and simultaneously injects them into the collaboration matrix, so that newly created agents are immediately usable with predicted collaboration priors. Third, a layered semantic matching strategy uses pre-trained sentence embeddings for fast, deterministic capability matching as a first pass, invoking LLM verification only for low-confidence cases, thereby reducing LLM routing calls by over 80\% compared to pure-LLM approaches. Experiments on mixed code, math, and question-answering suites show that WorkflowOps improves end-to-end pass rates over recent workflow-construction baselines, with the largest gains on structured, decomposable tasks where past agent handoff patterns transfer.
Tags
Links
- Source: https://arxiv.org/abs/2610.07860v1
- Canonical: https://arxiv.org/abs/2610.07860v1
Trouble viewing inline? Open PDF directly →
Full Text
66,916 characters extracted from source content.
Expand or collapse full text
WorkflowOps: Learning Agent Collaboration Priors for Multi-Agent Workflow Orchestration Qi Cheng Shengyu Chen Wei Cheng Zhengzhang Chen Xiaowei Jia Haoyu Wang Haifeng Chen Abstract Multi-agent systems are increasingly deployed for complex knowledge work, yet their orchestration layers remain largely memoryless: each new task is decomposed, assigned, and executed from scratch with no benefit from prior successful executions. We present WorkflowOps, a multi-agent workflow orchestration framework that learns agent collaboration priors from historical workflows and expands its agent pool on demand to cover new capability requirements. Our approach introduces three coupled mechanisms. First, a transition probability matrix captures pairwise agent collaboration frequencies from past workflows and applies them as soft guidance during DAG workflow construction through intra-layer ordering optimization, probability-thresholded edge suggestion, and transitive reduction for parallelism maximization. Second, a sufficiency-driven agent creation loop detects capability gaps via semantic matching scores, generates specialized agents through an LLM, and simultaneously injects them into the collaboration matrix, so that newly created agents are immediately usable with predicted collaboration priors. Third, a layered semantic matching strategy uses pre-trained sentence embeddings for fast, deterministic capability matching as a first pass, invoking LLM verification only for low-confidence cases, thereby reducing LLM routing calls by over 80% compared to pure-LLM approaches. We formalize the workflow construction problem as constrained optimization over a dependency-valid DAG guided by Markov collaboration priors and show that even lightweight frequency-based statistics, without any training, provide effective orchestration guidance. Experiments on mixed code, math, and question-answering suites show that WorkflowOps improves end-to-end pass rates over recent workflow-construction baselines, with the largest gains on structured, decomposable tasks where past agent handoff patterns transfer. 1Rutgers University, New Brunswick, NJ 2University of Pittsburgh, Pittsburgh, PA 3NEC Labs America, Princeton, NJ q.cheng@rutgers.edu, shc160@pitt.edu, xj159@cs.rutgers.edu, weicheng, zchen, haoyu, haifeng@nec-labs.com 1 Introduction Figure 1: Overview of WorkflowOps. Given an agent pool A, transition matrix P, and task T, the pipeline decomposes T, matches subtasks to agents (creating a∗a^* when ϕ<τφ<τ), constructs a DAG using P as soft guidance, and executes it in parallel layers. Outputs: workflow W, expanded ′A , and updated P′P . Coral marks the newly created agent. Large language model (LLM) based multi-agent systems have rapidly evolved from research prototypes into practical frameworks for complex knowledge-work tasks. Systems such as AutoGen (Wu et al. 2024), CrewAI (CrewAI 2024), and LangGraph (LangChain 2024) allow developers to compose specialized agents into collaborative workflows for tasks such as research synthesis, software engineering, compliance analysis, and business planning. As these systems are applied to increasingly complex and recurring tasks, the orchestration layer—how one task is decomposed, how subtasks are assigned to agents, how dependencies are structured, and how the resulting workflow is executed—has become a central bottleneck. Reconstructing a workflow from scratch for every query increases API cost, latency, and the risk of error accumulation, even when similar workflows have succeeded before. The central limitation of many current orchestration pipelines is the lack of persistent orchestration state: workflow-level collaboration priors, evolving capability coverage, and low-cost capability matching are rarely carried across tasks. Historical workflow traces expose recurring handoffs, capability gaps, and matching patterns that can be reused as soft guidance rather than hard constraints. Recent work improves orchestration adaptivity through topology selection, dynamic coordination, learned optimization, routing, and sub-agent creation (authors 2026a; authors 2026c; authors 2026d; Yue and others 2025; Zhang and others 2026; authors 2026b). However, these threads are typically studied separately: some methods recompute decisions at task time, some require task-specific training, and others create agents without integrating them into a persistent collaboration model. We introduce WorkflowOps, a history-conditioned orchestration framework that maintains persistent state through a collaboration-prior matrix, an evolving agent pool, and a semantic capability index. The system uses historical or bootstrapped successful workflow traces as soft guidance for approximate constrained optimization over dependency-valid DAGs, balancing historical alignment, assignment sufficiency, and structural parallelism. Concretely, WorkflowOps builds a row-stochastic agent transition matrix for layer ordering, soft-edge proposal, and redundancy pruning via transitive reduction (Aho et al. 1972); expands the agent pool only when semantic sufficiency checks reveal capability gaps; and uses Sentence-BERT-style embeddings (Reimers and Gurevych 2019) as the primary routing mechanism, with optional LLM verification for low-confidence cases. We evaluate WorkflowOps on two heterogeneous suites: a Mixed code/math/QA benchmark and a harder Mixed_Hard suite of competition-grade and frontier-difficulty problems. On the Mixed test split, WorkflowOps in DAG mode achieves a 0.890 overall pass rate (Linear: 0.870), improving over MetaAgent (Zhang et al. 2025d), the strongest query-level baseline, by 13.9 absolute points, and over ADAS (Hu et al. 2025), the strongest task-level baseline, by 19.6 points. On the harder Mixed_Hard suite, WorkflowOps reaches 0.770, exceeding every fully-completed baseline by at least 15 points. Ablations show that removing the transition matrix, disabling agent-pool evolution, and replacing semantic embeddings with TF-IDF reduce accuracy by 4.5, 1.7, and 8.5 points, respectively, validating each of the three coupled mechanisms. The gains are strongest on code and math; DAG execution provides modest additional accuracy gains via matrix-suggested edges over the Linear mode. Our contributions are as follows: 1. We formulate history-conditioned multi-agent workflow construction as approximate constrained optimization over dependency-valid DAGs, guided by persistent Markov collaboration priors extracted from successful workflow traces. 2. We instantiate this formulation with a transition-prior workflow-construction algorithm that uses empirical agent-to-agent transition probabilities for layer ordering, soft-edge proposal, and redundancy pruning, while preserving hard task dependencies. 3. We introduce a sufficiency-driven agent-pool expansion loop that detects capability gaps through semantic matching, creates specialized agents only when needed, and immediately integrates new agents into the collaboration-prior matrix. 4. We provide an end-to-end empirical study and ablation analysis on Mixed and Mixed_Hard benchmarks, showing a 13.9-point overall gain over the strongest query-level baseline on Mixed and a 15+ point lead over fully-completed baselines on Mixed_Hard, while isolating the effects of transition priors, agent evolution, and semantic matching. Together, these results position WorkflowOps as a lightweight and interpretable framework for training-free, history-conditioned multi-agent orchestration. Rather than relying on expensive policy learning or reconstructing workflows from scratch, WorkflowOps demonstrates that simple frequency-based collaboration statistics, when coupled with persistent agent-pool evolution and semantic capability matching, can serve as effective operational priors for future workflow construction. 2 Related Work Recent multi-agent orchestration methods improve adaptivity through topology selection, dynamic coordination, learned workflow optimization, routing, or agent creation. AdaptOrch (authors 2026a) selects among predefined orchestration patterns, CORAL (authors 2026c) studies information-flow-based dynamic coordination, and FlowSteer (authors 2026d) learns workflow-editing policies through reinforcement learning. Other work optimizes workflow structure or interaction policies directly, including HeGFlow (Guan et al. 2025), AgentPO (Sun et al. 2026), and agentic architecture search methods such as ADAS (Hu et al. 2025) and MaAS (Zhang et al. 2025b). These approaches show that orchestration structure matters, but many either recompute decisions at task time or require training/search for each target setting. Our baselines instantiate this workflow-search line: AFlow (Zhang et al. 2025c) searches workflow code with MCTS, ADAS (Hu et al. 2025) performs automated agentic-system search, MaAS (Zhang et al. 2025b) searches over architecture supernets, and MetaAgent (Zhang et al. 2025d) constructs a finite-state-machine workflow per query. A further line reuses workflow experience: Agent Workflow Memory (Wang et al. 2024) induces and stores reusable workflow skills, EvoFlow (Zhang et al. 2025a) evolves a population of diverse workflows, and TopoPrior (Zhang et al. 2026) learns topology priors that transfer across domains. WorkflowOps is complementary to both lines: rather than storing executable workflows or learning topology generators, it accumulates interpretable agent-to-agent transition statistics and applies them as training-free soft guidance under hard task dependencies. A complementary line of work studies agent routing and self-evolving agent systems. MasRouter (Yue and others 2025) and EvoRoute (Zhang and others 2026) learn routing policies for multi-agent systems, while AOrchestra (authors 2026b), SEAA (SEAA authors 2026), SEVerA (Banerjee et al. 2026), and AgentOrchestra (authors 2025a) explore dynamic sub-agent creation, self-extension, verification, or hierarchical orchestration. WorkflowOps differs by integrating these threads into a single training-free, history-conditioned orchestration layer: empirical transition priors guide DAG construction, semantic matching assigns subtasks to agents, and demand-driven agent creation expands the pool while immediately updating the persistent collaboration prior. Extended related work is provided in Appendix A. 3 Problem Formulation General problem. Given a task T and an agent pool A, workflow orchestration must produce a dependency-valid workflow DAG W and an agent assignment π that maximize downstream task success. This statement is independent of any particular mechanism. WorkflowOps additionally assumes access to a history of successful workflows, from which it derives a transition matrix P; P is an input to our approach, not part of the problem itself. We first define the core objects. Definition 1 (Agent Pool) An agent pool =a1,…,aNA=\a_1,…,a_N\ consists of N specialized agents. Each agent aia_i is characterized by a natural language capability description did_i and dense embedding i=fenc(di)∈ℝhe_i=f_enc(d_i) ^h, where fencf_enc is a pre-trained sentence encoder. Definition 2 (Task Decomposition) Given a natural language task T, a decomposition function LLMD_LLM produces subtasks =s1,…,sKS=\s_1,…,s_K\ and a dependency DAG Gdep=(,Edep)G_dep=(S,E_dep), where each directed edge (si,sj)∈Edep(s_i,s_j)∈ E_dep indicates that sis_i must complete before sjs_j begins. Each subtask sjs_j carries a required capability description cjc_j. Definition 3 (Capability Matching) For subtask sjs_j with requirement cjc_j, the matching score against agent aia_i is ϕ(sj,ai)=cos(fenc(cj),i).φ(s_j,a_i)= \! (f_enc(c_j),e_i ). (1) The assignment π:→π:S maps each subtask to its best-matching agent: π(sj)=argmaxai∈ϕ(sj,ai)π(s_j)= _a_i φ(s_j,a_i). Definition 4 (Sufficiency) Assignment π is τ-sufficient if and only if ϕ(sj,π(sj))≥τφ(s_j,π(s_j))≥τ for all sj∈s_j , where τ∈[0,1]τ∈[0,1] is the sufficiency threshold. The gap set is =sj∣ϕ(sj,π(sj))<τG=\s_j φ(s_j,π(s_j))<τ\. Definition 5 (Transition Matrix) Given a history of successful workflows ℋ=W1,…,WMH=\W_1,…,W_M\, each containing an ordered sequence of agent invocations, the transition matrix ∈ℝN×NP ^N× N is Pij=count(ai→aj in ℋ)∑k=1Ncount(ai→ak in ℋ).P_ij= count(a_i→ a_j in H) _k=1^Ncount(a_i→ a_k in H). (2) Here count(ai→aj in ℋ)count(a_i→ a_j in H) denotes the number of times agent aja_j immediately follows aia_i across workflows in ℋH. P is row-stochastic and encodes empirical first-order Markov collaboration priors. Definition 6 (Workflow) A workflow W=(V,E,π)W=(V,E,π) consists of nodes V=V=S, edges E=Edep∪EmatE=E_dep∪ E_mat containing hard dependency edges and soft matrix-suggested edges, and agent assignment π. WorkflowOps design objective. Because downstream task success is not directly measurable at construction time, our approach optimizes a history-conditioned surrogate balancing three terms: alignment with historical collaboration priors, semantic sufficiency of subtask–agent assignments, and structural parallelism: Q(W,π,′)=α⋅Align(W,)+β⋅Suff(π)+γ⋅Par(W),Q(W,π,A )=α·Align(W,P)+β·Suff(π)+γ·Par(W), (3) subject to preserving hard task dependencies, maintaining acyclicity, and allowing only monotonic expansion of the agent pool ′⊇A , where Align(W,) (W,P) =1|E|∑(si,sj)∈EPπ(si),π(sj), = 1|E| _(s_i,s_j)∈ EP_π(s_i),π(s_j), (4) Suff(π) (π) =1||∑j=1Kϕ(sj,π(sj)), = 1|S| _j=1^Kφ(s_j,π(s_j)), (5) Par(W) (W) =||depth(W), = |S|depth(W), (6) with depth(W)depth(W) the length of the longest path in the DAG. Align measures agreement with historical collaboration patterns, Suff measures capability coverage, and Par measures available parallelism. We use this objective as a design principle rather than solving it globally; WorkflowOps approximates it through staged decomposition, semantic matching, agent-pool expansion, transition-guided DAG construction, and layered execution. 4 Method WorkflowOps operates as a pipeline of four stages: (1) task decomposition with layered semantic matching, (2) sufficiency-driven agent pool expansion, (3) transition-prior-guided workflow construction, and (4) layered parallel execution. We describe each stage below, with detailed algorithms where useful. The general flow of WorkflowOps can be found in Figure 1. 4.1 Task decomposition and layered semantic matching Task decomposition. Given a natural language task T, an LLM decomposes it into K subtasks =s1,…,sKS=\s_1,…,s_K\ (3≤K≤73≤ K≤ 7), each with a required capability description cjc_j and dependency links forming a DAG GdepG_dep. The decomposition prompt enforces valid DAG structure; a post-processing step detects and breaks cycles via DFS if they arise from LLM output errors. All dependency links are thus proposed by the LLM under a schema-constrained prompt: structural validity is enforced programmatically, while the semantic correctness of the decomposition is delegated to the LLM. Layered semantic matching. We adopt a two-layer matching strategy that balances quality and efficiency: Layer 1: Sentence embedding matching. We use a pre-trained sentence encoder fencf_enc (Sentence-BERT (Reimers and Gurevych 2019), specifically the all-MiniLM-L6-v2 variant) to encode both agent capability descriptions and subtask requirements into 384-dimensional dense vectors. Matching scores are computed as: ϕ(sj,ai)=cos(fenc(cj),fenc(di)).φ(s_j,a_i)= \! (f_enc(c_j),\;f_enc(d_i) ). (7) Here aia_i denotes an agent, did_i its capability description, and cjc_j the subtask requirement (cf. Definition 3). Agent embeddings are pre-computed and cached; subtask embeddings are computed at decomposition time. This layer is deterministic, has zero LLM cost, and runs in sub-millisecond time per subtask-agent pair through batched matrix multiplication. Layer 2: LLM on-demand verification. For subtasks where the best matching score falls below the sufficiency threshold τ, an optional LLM verification stage can judge whether an existing agent is truly adequate or whether a capability gap exists. This stage is invoked only for low-confidence cases where embedding similarity is ambiguous. Complexity comparison. Pure LLM routing evaluates K×NK× N subtask-agent pairs with LLM calls. Our approach requires K×NK× N vector operations (Layer 1, negligible cost) plus at most K′×1K × 1 LLM calls (Layer 2, where K′≪K K). For a typical setting with K=5K=5 subtasks and N=35N=35 agents, this reduces from 175 LLM calls to ∼5 5 vector-similarity computations plus ∼1 1 LLM verification call. 4.2 Transition-prior-guided workflow construction Matrix construction. The transition matrix P (Definition 5) is built offline from a corpus of historical (or synthetic) workflows by counting pairwise agent transitions and row-normalizing. In our deployment, we bootstrap from ∼65 65 synthetic workflows spanning eight task domains, yielding a 35×3535× 35 matrix with approximately 20% non-zero entries. Constructing P is a single counting pass over historical handoffs, lookups are O(1)O(1), and memory grows as O(N2)O(N^2); at N=35N=35 the matrix occupies a few kilobytes, so its construction and query cost are negligible relative to a single LLM call. Triple-role mechanism. The transition matrix serves three distinct roles during workflow construction: (a) Intra-layer ordering optimization. Within each topological layer (subtasks that share the same depth and can be freely reordered), agents are sorted by their transition probability from the preceding layer’s agents. If the previous layer’s agent was apa_p, and the current layer contains subtasks assigned to agents axa_x and aya_y, the system prefers the ordering where Pp,x>Pp,yP_p,x>P_p,y, placing axa_x before aya_y. This optimizes information flow sequence based on historically successful patterns. (b) Probability-thresholded edge suggestion. For each pair of subtasks (si,sj)(s_i,s_j) with no explicit dependency edge, if Pπ(si),π(sj)≥θP_π(s_i),π(s_j)≥θ (the edge threshold) and adding the edge preserves topological ordering (i.e., depth(si)<depth(sj)depth(s_i)<depth(s_j)), a soft edge is added. These matrix-suggested edges encode the system’s belief, grounded in historical data, that agent π(si)π(s_i)’s output is valuable as input to agent π(sj)π(s_j). (c) Transitive reduction. After both hard dependency edges and soft matrix-suggested edges are incorporated, the system performs a transitive reduction (Aho et al. 1972): if edges (u,v)(u,v) and (v,w)(v,w) exist, and a direct edge (u,w)(u,w) also exists, the direct edge (u,w)(u,w) is removed because the dependency is already transitively satisfied. This maximizes parallelism by removing redundant sequential constraints, allowing more subtasks to execute concurrently within each topological layer. Algorithm 1 formalizes this procedure. Algorithm 1 Transition-guided DAG workflow construction 0: Subtasks S, dependency graph Gdep=(,Edep)G_dep=(S,E_dep), assignment π, transition matrix P, edge threshold θ 0: Workflow W=(V,E,π)W=(V,E,π) 1: V←V ; E←EdepE← E_dep 2: Compute topological depths: depth(s)←depth(s)← longest path from any source to s in GdepG_dep 3: for each pair (si,sj)∈×(s_i,s_j) ×S with si≠sjs_i≠ s_j and (si,sj)∉E(s_i,s_j)∉ E do 4: if Pπ(si),π(sj)≥θP_π(s_i),π(s_j)≥θ and depth(si)<depth(sj)depth(s_i)<depth(s_j) then 5: E←E∪(si,sj)E← E∪\(s_i,s_j)\ ⊳ Add matrix-suggested soft edge 6: end if 7: end for 8: E←TransitiveReduction(V,E)E← TransitiveReduction(V,E) ⊳ Remove redundant edges 9: Compute parallel groups via layered topological sort of (V,E)(V,E) 10: Sort agents within each layer by Pprev,π(s)P_prev,π(s) descending ⊳ Intra-layer ordering 11: return W=(V,E,π)W=(V,E,π) Dynamic matrix update. When new agents are created (§4.3), they must be integrated into P. The system uses the LLM to predict likely predecessor and successor agents for the new agent anewa_new; the prediction is conditioned on the new agent’s capability card and the capability descriptions of the existing pool (cf. Algorithm 2), and the predicted transition probabilities are injected into the corresponding rows and columns of P. All affected rows are re-normalized to maintain row-stochasticity. This ensures that the matrix grows alongside the agent pool, so workflow construction can immediately exploit the new agent’s predicted collaboration patterns. Why soft guidance. The matrix is used as soft guidance rather than hard constraints for three reasons: (i) with ∼20% 20\% matrix density, hard constraints would block valid but historically unseen workflows; (i) dependency structure is always the non-negotiable foundation—the matrix cannot override task logic; (i) novel task patterns should still be expressible even if they deviate from historical priors. 4.3 Sufficiency-driven agent-pool expansion When the current agent pool cannot τ-sufficiently cover a task’s requirements, WorkflowOps enters a closed-loop agent creation process. This is a defining capability: the system does not stop at weak assignments but actively expands its capability boundaries. Algorithm 2 formalizes the sufficiency loop. Algorithm 2 Sufficiency-driven agent pool expansion 0: Subtasks S, agent pool A, threshold τ, max rounds R, matrix P 0: Expanded pool ′A , updated matrix ′P , sufficient assignment π′π 1: ′←A ; ′←P 2: for r=1,…,Rr=1,…,R do 3: Build matching index from ′A using fencf_enc 4: Compute π′←argmaxa∈′ϕ(sj,a)π ← _a φ(s_j,a) for all sj∈s_j 5: ←sj∣ϕ(sj,π′(sj))<τG←\s_j φ(s_j,π (s_j))<τ\ ⊳ Identify capability gaps 6: if =∅G= then 7: break ⊳ All subtasks sufficiently covered 8: end if 9: for each sj∈s_j do 10: if optional adequacy check indicates existing agents are insufficient then 11: anew←LLMGenerate(sj.capability)a_new← LLMGenerate(s_j.capability) ⊳ Generate agent definition 12: Persist anewa_new to agent pool on disk 13: ′←′∪anewA ∪\a_new\ 14: (pred,succ)←LLMPredict(anew,′)(pred,succ)← LLMPredict(a_new,A ) ⊳ Predict collaborators 15: ′←InjectAndNormalize(′,anew,pred,succ)P ← InjectAndNormalize(P ,a_new,pred,succ) 16: end for 17: end for 18: return ′,′,π′A ,P ,π Key design decisions. Demand-driven creation. Unlike AOrchestra (authors 2026b), which creates agents for every task regardless of pool coverage, WorkflowOps creates agents only when a measurable capability gap is detected (ϕ<τφ<τ). This prevents unnecessary pool inflation. Integrated collaboration knowledge. Each new agent is not only added to the pool but also injected into the transition matrix P with LLM-predicted predecessor and successor relationships. This means the next workflow that uses the new agent benefits from collaboration priors immediately, rather than requiring multiple executions to accumulate transition statistics organically. Safety mechanisms. The loop is bounded by three mechanisms: (i) a maximum round count R (default 3), (i) a per-subtask deduplication guard, and (i) a maximum agent creation count per run. Persistence and lifecycle. Created agents are persisted to disk and become part of the available pool for future runs. The system supports three environment modes—exploratory (free evolution), controlled (periodic curation), and frozen (version-locked for production)—enabling a governance path from experimentation to enterprise deployment. 4.4 Layered Parallel Execution After workflow construction, WorkflowOps executes the DAG by topological layers. All nodes in the same layer are dependency-independent and can therefore run concurrently; downstream nodes receive either a single predecessor output or a merged message containing all upstream results. This execution mode preserves hard dependencies while exploiting the parallelism exposed by transition-guided DAG construction. We compare this DAG mode with a Linear mode that uses the same decomposition and agent assignment but executes subtasks sequentially. Both modes use the same message abstraction and logging interface, enabling a controlled comparison between sequential execution and transition-guided DAG execution. Detailed message schemas, fan-in merging, error propagation, audit logging, and the end-to-end pipeline algorithm (Algorithm 3) are provided in Appendix B. 5 Experiments 5.1 Experimental Setup Task suites. We use two heterogeneous benchmark suites with the same inference pipeline and grading harness. Mixed combines code generation (HumanEval (Chen et al. 2021), MBPP (Austin et al. 2021)), mathematical reasoning (GSM8K (Cobbe et al. 2021), MATH (Hendrycks et al. 2021)), and question answering (DROP (Dua et al. 2019), HotpotQA (Yang et al. 2018)); it contains 120 validation and 300 test problems, sampled equally across sources. Mixed_Hard contains harder competition-grade and frontier-difficulty problems from AIME 2025 (Balunović et al. 2026), LiveCodeBench (Jain et al. 2024), CodeContests (Li et al. 2022), and GPQA Diamond (Rein et al. 2023); it contains 180 validation and 694 test problems. The orchestrator does not observe the benchmark identity, so it must handle both broad category coverage and harder problem distributions without source-specific routing. Baselines. We compare against four recent multi-agent orchestration systems: AFlow (Zhang et al. 2025c), an MCTS-based workflow generator; ADAS (Hu et al. 2025), an automated agentic-system search method; MaAS (Zhang et al. 2025b), architecture search over agentic supernets; and MetaAgent (Zhang et al. 2025d), a finite-state-machine multi-agent constructor. MetaAgent constructs a fresh workflow per query, while AFlow, ADAS, and MaAS search for one workflow per benchmark and reuse it across queries. Backbone and configuration. All systems use gpt-5 as the underlying language model. WorkflowOps uses sufficiency threshold τ=0.45τ=0.45, edge threshold θ=0.3θ=0.3, the all-MiniLM-L6-v2 sentence-transformer matcher (Reimers and Gurevych 2019), MAX_AGENTS_PER_RUN=4=4, MAX_CREATION_ROUNDS=3=3, and MAX_DAG_BRANCHING=3=3. Decomposition produces 3−73-7 subtasks per query, workflows execute with 16 worker processes per configuration, and each query has a 900 s wall-clock timeout. Metrics. The primary metric is pass rate: the fraction of test rows where the extracted final answer matches ground truth, with code tasks graded by sandboxed execution against canonical tests. We also log matrix alignment, sufficiency, dynamic-agent creation, gap closure, LLM calls, token counts, latency, and DAG-vs.-Linear wall time; detailed operational statistics are reported in Appendix E. 5.2 End-to-end Workflow Quality Table 1 reports the headline pass rates on Mixed and Mixed_Hard. Full category-level breakdowns are provided in Appendix C. Baseline numbers are taken from the respective systems’ evaluation runs on the same splits with the same gpt-5 backbone. Table 1: End-to-end pass rate on Mixed and Mixed_Hard. Full category breakdowns are in Appendix C. System Mixed Mixed_Hard AFlow (Zhang et al. 2025c) 0.676 0.218 ADAS (Hu et al. 2025) 0.694 0.729† MaAS (Zhang et al. 2025b) 0.400 DNF MetaAgent (Zhang et al. 2025d) 0.751 0.618 WorkflowOps Linear 0.870 0.763 WorkflowOps DAG 0.890 0.770 †ADAS on Mixed_Hard is the strongest partial-strategy result among completed candidate strategies. DNF = did not finish (MaAS deadlocked on Mixed_Hard). Mixed. On Mixed, WorkflowOps (DAG) reaches 0.8900.890, improving over the strongest query-level baseline, MetaAgent (0.7510.751), by 13.913.9 absolute points and over the strongest task-level baseline, ADAS (0.6940.694), by 19.619.6 points. The gain is largest on CODE: WorkflowOps obtains 0.9400.940 in Linear mode and 0.9600.960 in DAG mode, more than 4040 points above the nearest baseline. On MATH, WorkflowOps ties MetaAgent at 0.9400.940. QA remains the weakest category: MetaAgent leads by 44–88 points, suggesting that the current agent pool and transition priors are better aligned with structured code/math workflows than with open-ended reading-comprehension style reasoning. DAG mode improves over Linear by 22 points overall, with the gain concentrated on CODE and QA. Mixed_Hard. On the harder benchmark suite, WorkflowOps (DAG) reaches 0.7700.770, outperforming every fully completed baseline by at least 1515 points and exceeding the strongest partial ADAS result by 44 points. The breakdown shows strong performance across competition-grade problems: 0.8500.850 on MATH, 0.7800.780 on CODE, and 0.7000.700 on QA. DAG and Linear modes are within 11 point of each other on Mixed_Hard, indicating that the harder suite benefits more from robust assignment and agent coverage than from additional DAG structure. Significance. A paired bootstrap test on Mixed finds the gap between WorkflowOps (DAG) and MetaAgent significant (p<0.001p<0.001). The effect is driven primarily by CODE; MATH is statistically tied and QA marginally favors MetaAgent. Full confidence intervals, category results, and effect sizes are in Appendix C. 5.3 Ablation Studies We ablate the mechanisms introduced in Section 4: transition-matrix guidance, sufficiency-driven agent-pool expansion, semantic embedding matching, pure-LLM matching, and the edge threshold θ. Each cell uses the same agent pool, matrix initialization, decomposition prompt, and execution engine. Table 2: Ablation results on the Mixed test split (300 problems per cell). “Linear” and “DAG” columns give overall pass rate; brackets are 95% bootstrap CIs (nboot=2000n_boot=2000). CODE/MATH/QA are unweighted category means in Linear mode. Calls and tokens are mean per query in Linear mode. Variant Linear DAG CODE MATH QA calls tokens Reference 0.870 [0.830, 0.907] 0.890 [0.853, 0.923] 0.940 0.940 0.730 14.8 43.6k No-Matrix 0.825 [0.780, 0.867] 0.840 [0.797, 0.880] 0.920 0.910 0.645 13.9 41.2k No-Evolve 0.853 [0.813, 0.893] 0.850 [0.807, 0.890] 0.930 0.960 0.670 7.8 15.9k TF-IDF 0.785 [0.737, 0.830] 0.760 [0.710, 0.810] 0.890 0.910 0.555 14.7 43.0k LLM-Match 0.840 [0.800, 0.883] 0.853 [0.813, 0.890] 0.930 0.940 0.650 17.7 48.0k θ=0.0θ=0.0 0.825 [0.780, 0.867] 0.815 [0.770, 0.857] 0.870 0.910 0.695 14.6 42.5k θ=0.5θ=0.5 0.855 [0.813, 0.893] 0.870 [0.830, 0.907] 0.910 0.920 0.735 14.7 42.6k θ=0.7θ=0.7 0.795 [0.747, 0.840] 0.815 [0.770, 0.857] 0.870 0.890 0.625 14.5 42.3k The ablations validate all three core mechanisms. Removing the transition matrix drops Mixed accuracy by 4.54.5–5.05.0 points, showing that historical collaboration priors provide useful structural guidance. Disabling agent evolution drops accuracy by 1.71.7 points but reduces cost sharply, identifying a cheaper operating point when the static pool already covers the task distribution. Replacing semantic embeddings with TF-IDF drops accuracy by 8.58.5–13.013.0 points, while pure LLM matching is both less accurate and more expensive than the layered matcher. The edge-threshold sweep peaks near the default θ=0.3θ=0.3, with θ=0.5θ=0.5 close behind. Full per-benchmark ablations and detailed interpretations are in Appendix D. Together, these ablations support the central design claim: WorkflowOps improves not through a single dominant component, but through the coupling of historical transition priors, semantic capability matching, and bounded agent-pool expansion. In terms of effect size, semantic capability matching is the largest single factor, transition-matrix guidance contributes a consistent additional gain, and pool expansion trades a smaller accuracy gain for the largest share of cost, while DAG construction adds a further modest improvement over Linear at near-equal wall time. The headline improvement is therefore driven primarily by reliable subtask–agent matching, with the transition prior and DAG structure supplying complementary structural guidance. 5.4 Efficiency Analysis WorkflowOps exposes a clear cost–accuracy tradeoff. Appendix E tabulates the full operational profile of the Reference configuration on the Mixed test split. The system uses 14.8 LLM calls and 43.6k tokens per query on average, with a median wall time of 605 seconds. Most calls come from two sources: the sufficiency-driven agent-creation loop, which creates an average of 3.5 agents per query, and agent execution, which invokes one agent per workflow node. The embedding matcher itself contributes essentially no LLM calls, supporting the design choice of using semantic embeddings as the routing fast path. Normalizing by the cheapest ablation cell, No-Evolve (7.8 calls and 15.9k tokens), Reference spends 1.9×1.9× the calls and 2.7×2.7× the tokens for a +1.7+1.7 point accuracy gain. This makes agent-pool expansion an explicit deployment knob: disabling expansion is attractive when the static pool already covers the task distribution, while the reference setting is preferable when tasks may expose new capability gaps. DAG execution provides modest accuracy gains with near-equal median wall time to Linear mode on Mixed, because most workflows are shallow; detailed DAG-vs.-Linear timing and additional operational statistics are provided in Appendix E. 6 Discussion and Limitations Lightweight priors as orchestration memory. A key finding is that simple frequency-based transition priors can provide useful workflow guidance without gradient-based training. Unlike learned workflow optimizers, the transition matrix requires only execution history, is interpretable, updates incrementally as new workflows complete, and degrades gracefully when no relevant transition has been observed. This makes it a practical middle ground between static templates and expensive task-specific workflow search. Limitations. WorkflowOps shows that lightweight collaboration priors can improve multi-agent orchestration without gradient-based training, but several limitations remain. First, the transition matrix captures only first-order pairwise handoffs; higher-order collaboration patterns such as repeated agent sequences are not modeled. Second, dynamic agent creation depends on LLM generation quality, so poorly specified agents may persist unless the pool is curated; relatedly, our evaluation does not measure long-term effects of pool expansion—whether created agents are reused by later tasks, refined, or pruned—so we claim demand-driven expansion rather than sustained self-evolution. Third, the current system uses a single shared transition matrix, whereas domain-specific matrices may provide more targeted guidance; because P indexes agent roles rather than backbone outputs, it need not be rebuilt when the underlying LLM changes, though transition quality may shift and should be re-validated. Finally, our evaluation focuses on code, math, and QA-style knowledge work; generalization to substantially different domains requires further study. Additional discussion of matrix sparsity, matching design, and future extensions is provided in Appendix F. 7 Conclusion We presented WorkflowOps, a multi-agent workflow orchestration framework that addresses the memoryless orchestration problem in current multi-agent systems. Our three coupled mechanisms—transition probability matrix guidance, sufficiency-driven agent-pool expansion, and layered semantic matching—enable the system to learn from execution history, expand its capabilities on demand, and efficiently route subtasks to agents, all without requiring gradient-based training. The benefits are largest on structured, decomposable tasks where past agent handoff patterns transfer. The transition matrix demonstrates that even simple first-order Markov collaboration statistics, when applied as soft guidance during DAG workflow construction, provide meaningful improvement in workflow quality and parallelism. The sufficiency loop shows that demand-driven agent creation with integrated collaboration knowledge updates lets the agent pool and its collaboration priors grow together on demand. The layered matching strategy shows that pre-trained sentence embeddings can serve as an effective fast path for capability matching, with LLM verification reserved for genuinely ambiguous cases. Future work. Several directions are promising: (i) upgrading the matrix to learned completion methods (matrix factorization, neural completion) for better coverage of sparse transitions; (i) online matrix learning from execution feedback, creating an RL-style closed loop; (i) domain-specific matrices that capture field-specific collaboration patterns; (iv) integration with emerging inter-agent protocols such as A2A (Google 2026) and MCP for standardized agent communication. References Aho et al. (1972) A. V. Aho, M. R. Garey, and J. D. Ullman The transitive reduction of a directed graph. SIAM Journal on Computing 1 (2), p. 131–137. Cited by: §1, §4.2. Austin et al. (2021) J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: §5.1. authors (2026a) A. authors AdaptOrch: task-adaptive multi-agent orchestration in the era of LLM performance convergence. arXiv preprint arXiv:2602.16873. Note: Author metadata to be filled in the next round Cited by: §A.1, §1, §2. authors (2025a) A. authors AgentOrchestra: a hierarchical multi-agent framework for general-purpose task solving. arXiv preprint arXiv:2506.12508. Note: Author metadata to be filled in the next round Cited by: §A.4, §2. authors (2026b) A. authors AOrchestra: automating sub-agent creation for agentic orchestration. arXiv preprint arXiv:2602.03786. Note: Author metadata to be filled in the next round Cited by: §A.4, §1, §2, §4.3. authors (2026c) C. authors Beyond rule-based workflows: an information-flow-orchestrated multi-agents paradigm via agent-to-agent communication from CORAL. arXiv preprint arXiv:2601.09883. Note: Author metadata to be filled in the next round Cited by: §A.1, §1, §2. authors (2026d) F. authors FlowSteer: interactive agentic workflow orchestration via end-to-end reinforcement learning. arXiv preprint arXiv:2602.01664. Note: Author metadata to be filled in the next round Cited by: §A.1, §1, §2. authors (2026e) P. authors ProMAS: proactive error forecasting for multi-agent systems using Markov transition dynamics. arXiv preprint arXiv:2603.20260. Note: Author metadata to be filled in the next round Cited by: §A.2. authors (2025b) S. authors A survey on agent workflow – status and future. Proceedings of IEEE ICAIBD. Note: Author metadata to be filled in the next round Cited by: §A.2. authors (2026f) S. authors From static templates to dynamic runtime graphs: a survey of workflow optimization for LLM agents. arXiv preprint arXiv:2603.22386. Note: Author metadata to be filled in the next round Cited by: §A.2. Balunović et al. (2026) M. Balunović, J. Dekoninck, I. Petrov, N. Jovanović, and M. Vechev MathArena: evaluating llms on uncontaminated math competitions. External Links: 2505.23281, Link Cited by: §5.1. Banerjee et al. (2026) D. Banerjee, C. Xu, and G. Singh SEVerA: verified synthesis of self-evolving agents. arXiv preprint arXiv:2603.25111. Cited by: §A.4, §2. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba Evaluating large language models trained on code. External Links: 2107.03374, Link Cited by: §5.1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: §5.1. CrewAI (2024) CrewAI CrewAI: multi-agent orchestration framework. Note: https://w.crewai.com/ Cited by: §1. Dua et al. (2019) D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner DROP: a reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proc. of NAACL, Cited by: §5.1. Google (2026) Google Agent-to-agent (A2A) protocol. Note: https://google.github.io/A2A/ Cited by: §7. Guan et al. (2025) B. Guan, R. Wang, and Y. Shen A heterogeneous graph optimization perspective for multi-agent system workflows. arXiv preprint. Note: Submitted to ICLR 2026 Cited by: §A.2, §2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the math dataset. External Links: 2103.03874, Link Cited by: §5.1. Hu et al. (2025) S. Hu, C. Lu, and J. Clune Automated design of agentic systems. External Links: 2408.08435, Link Cited by: Table 3, Table 4, §1, §2, §2, §5.1, Table 1. Jain et al. (2024) N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica LiveCodeBench: holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974. Cited by: §5.1. LangChain (2024) LangChain LangGraph: multi-agent workflows. Note: https://blog.langchain.dev/langgraph-multi-agent-workflows Cited by: §1. Li et al. (2022) Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, T. Hubert, P. Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. Sutherland Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals Competition-level code generation with alphacode. Science 378 (6624), p. 1092–1097. External Links: ISSN 1095-9203, Link, Document Cited by: §5.1. Reimers and Gurevych (2019) N. Reimers and I. Gurevych Sentence-BERT: sentence embeddings using siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 3982–3992. Cited by: §1, §4.1, §5.1. Rein et al. (2023) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof q&a benchmark. External Links: 2311.12022, Link Cited by: §5.1. SEAA authors (2026) SEAA authors Self-extending agent architecture: capability gap detection, tool synthesis, and autonomous evolution under governance constraints. Note: https://os.maria-code.ai/blog/self-extending-agent-architecture Cited by: §A.4, §2. Sun et al. (2026) L. Sun, C. Liu, C. Zhang, Y. Wu, W. Lu, and N. Wu AgentPO: enhancing multi-agent collaboration via reinforcement learning. In International Conference on Learning Representations (ICLR), Cited by: §A.2, §2. survey authors (2026) O. survey authors The orchestration of multi-agent systems: architectures, protocols, and enterprise adoption. arXiv preprint arXiv:2601.13671. Note: Author metadata to be filled in the next round Cited by: §A.2. Wang et al. (2024) Z. Z. Wang, J. Mao, D. Fried, and G. Neubig Agent workflow memory. arXiv preprint arXiv:2409.07429. Cited by: §A.5, §2. Wu et al. (2024) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, et al. AutoGen: enabling next-gen LLM applications via multi-agent conversation. In Conference on Language Modeling (COLM), Cited by: §1. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §5.1. Yue et al. (2025) Y. Yue et al. MasRouter: learning to route LLMs for multi-agent systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), p. 15549–15572. Cited by: §A.3, §1, §2. Zhang et al. (2025a) G. Zhang, K. Chen, G. Wan, H. Chang, H. Cheng, K. Wang, S. Hu, and L. Bai EvoFlow: evolving diverse agentic workflows on the fly. arXiv preprint arXiv:2502.07373. Cited by: §A.5, §2. Zhang et al. (2025b) G. Zhang, L. Niu, J. Fang, K. Wang, L. Bai, and X. Wang Multi-agent architecture search via agentic supernet. External Links: 2502.04180, Link Cited by: Table 3, §2, §2, §5.1, Table 1. Zhang et al. (2026) G. Zhang et al. EvoRoute: experience-driven self-routing LLM agent systems. arXiv preprint arXiv:2601.02695. Cited by: §A.3, §1, §2. Zhang et al. (2025c) J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu AFlow: automating agentic workflow generation. External Links: 2410.10762, Link Cited by: Table 3, Table 4, §2, §5.1, Table 1. Zhang et al. (2026) T. Zhang, Z. Zhou, J. Wan, T. Hu, C. Wang, X. He, and R. Hong Learning transferable topology priors for multi-agent LLM collaboration across domains. arXiv preprint arXiv:2605.17359. Cited by: §A.5, §2. Zhang et al. (2025d) Y. Zhang, X. Liu, and C. Xiao MetaAgent: automatically constructing multi-agent systems based on finite state machines. External Links: 2507.22606, Link Cited by: Table 3, Table 4, §1, §2, §5.1, Table 1. Technical Appendix Appendix A Extended Related Work A.1 Adaptive and Dynamic Orchestration Recent orchestration methods mainly improve adaptivity through topology routing or learned control policies. AdaptOrch (authors 2026a) proposes task-adaptive topology selection among predefined orchestration patterns and reports gains over static templates. CORAL (authors 2026c) moves from fixed workflows to information-flow-based dynamic coordination via agent-to-agent communication. FlowSteer (authors 2026d) applies end-to-end reinforcement learning to optimize workflow editing decisions. These methods are highly relevant but differ from our focus: WorkflowOps emphasizes persistent, training-free collaboration priors accumulated from workflow history and used as soft constraints during DAG construction. A.2 Workflow Structure Learning and Optimization HeGFlow (Guan et al. 2025) formulates workflow design as heterogeneous graph adjacency optimization with a training pipeline, while AgentPO (Sun et al. 2026) optimizes collaborative interaction policies under fixed topology assumptions. ProMAS (authors 2026e) uses Markov transition dynamics for error forecasting rather than workflow synthesis. In contrast, WorkflowOps directly uses historical transition statistics as an interpretable and incrementally updatable prior for structure construction. A broader survey perspective (authors 2026f; authors 2025b; survey authors 2026) highlights an open gap between static template orchestration and expensive training-heavy optimization. Our method is positioned in that gap: lightweight, history-informed graph construction without retraining. A.3 Agent Routing and Selection MasRouter (Yue and others 2025) and EvoRoute (Zhang and others 2026) study learned routing in multi-agent systems, primarily targeting model allocation and cost-performance trade-offs. Our setting is complementary: we focus on capability matching between subtasks and role-specialized agents. The layered semantic matching strategy in WorkflowOps can be interpreted as a routing policy over agent capabilities, using an embedding fast path with optional LLM fallback, rather than routing over model backbones. A.4 Self-Evolving Agent Architectures AOrchestra (authors 2026b) demonstrates dynamic sub-agent creation through compositional agent specifications, but treats created sub-agents as ephemeral: they are not persisted, and no collaboration knowledge about them is accumulated. SEAA (SEAA authors 2026) and SEVerA (Banerjee et al. 2026) provide formal perspectives on self-extension and verification, mainly at the tool or synthesis-control layer. AgentOrchestra (authors 2025a) studies hierarchical orchestration with dynamic tool pathways. WorkflowOps differs by coupling demand-driven agent creation with immediate integration into a persistent collaboration-prior matrix, so newly created roles influence future workflow construction. A.5 Workflow Memory and Experience Reuse Agent Workflow Memory (Wang et al. 2024) induces reusable workflow skills from experience and stores them for later tasks, EvoFlow (Zhang et al. 2025a) evolves a population of diverse agentic workflows, and TopoPrior (Zhang et al. 2026) learns topology priors that transfer across domains. These systems persist executable workflows or learned topology generators. WorkflowOps instead persists pairwise collaboration statistics that act as training-free soft guidance during construction: hard task dependencies are always preserved, the prior is interpretable and incrementally updatable without retraining, and previously unseen workflow shapes remain expressible. Appendix B Pipeline and Execution Engine Details B.1 End-to-End Pipeline Algorithm 3 summarizes the complete WorkflowOps pipeline; the individual stages are described in Section 4. Algorithm 3 WorkflowOps end-to-end pipeline 0: Task T, agent pool A, transition matrix P, threshold τ, edge threshold θ, max rounds R, workflow type ∈linear,dag∈\ linear, dag\ 0: Final result mfinalm_final, workflow W, logs ℒL 1: // Stage 1: Decomposition 2: ,Gdep←LLM(T)S,G_dep _LLM(T) ⊳ LLM decomposes task into subtasks with dependencies 3: Validate DAG: if HasCycle(Gdep) HasCycle(G_dep), linearize dependencies 4: // Stage 2: Matching + Sufficiency Loop 5: ′,′,π←SufficiencyLoop(,,τ,R,)A ,P ,π← SufficiencyLoop(S,A,τ,R,P) ⊳ Algorithm 2 6: // Stage 3: Workflow Construction 7: if workflow type == dag then 8: W←TransitionGuidedDAG(,Gdep,π,′,θ)W← TransitionGuidedDAG(S,G_dep,π,P ,θ) ⊳ Algorithm 1 9: else 10: W←BuildLinear(,π,′)W← BuildLinear(S,π,P ) ⊳ Linear with intra-layer ordering 11: end if 12: // Stage 4: Execution 13: mfinal,ℒ←ExecuteDAG(W,T)m_final,L← ExecuteDAG(W,T) ⊳ Algorithm 4 14: // Stage 5: Persistence 15: Save (W,mfinal,ℒ)(W,m_final,L) to results directory with timestamps and metadata 16: return mfinal,W,ℒm_final,W,L B.2 Layered DAG Execution After workflow construction, the execution engine processes the DAG through layered topological traversal. Nodes in the same layer have no unresolved dependencies and are executed in parallel. A downstream node receives the output of its predecessor when it has one parent, or a merged message when it has multiple parents. Algorithm 4 DAG execution engine 0: Workflow W=(V,E,π)W=(V,E,π), original task T 0: Final result message mfinalm_final, execution logs ℒL 1: Compute topological layers L0,L1,…,LDL_0,L_1,…,L_D from (V,E)(V,E) 2: results←results←\\; ℒ←[]L←[] 3: minit←CreateMessage(T)m_init← CreateMessage(T) 4: for l=0,1,…,Dl=0,1,…,D do 5: parallel for each node v∈Llv∈ L_l: ⊳ Thread pool within layer 6: preds←u∣(u,v)∈Epreds←\u (u,v)∈ E\ 7: if preds=∅preds= : min←minitm_in← m_init 8: else if |preds|=1|preds|=1: min←results[preds[0]]m_in [preds[0]] 9: else: min←MergeMessages(results[u]∣u∈preds)m_in← MergeMessages(\results[u] u \) 10: mout←ExecuteAgent(π(v),min)m_out← ExecuteAgent(π(v),m_in) 11: results[v]←moutresults[v]← m_out; append log to ℒL 12: end for 13: terminals←v∈V∣∄(v,w)∈Eterminals←\v∈ V \,(v,w)∈ E\ 14: mfinal←MergeMessages(results[v]∣v∈terminals)m_final← MergeMessages(\results[v] v \) 15: return mfinal,ℒm_final,L B.3 Unified Messaging Protocol All inter-agent communication uses a single AgentMessage schema carrying natural language content, optional structured data, typed artifacts, and trace metadata, including source agent, target agent, parent message ID, workflow ID, and step index. This composable protocol avoids the pairwise adapter problem that arises when agents use heterogeneous message formats. B.4 Fan-in Merge When parallel branches converge, upstream messages are merged into a single input for the downstream agent. The merge concatenates labeled content sections, preserves source attribution, and unions artifact collections, maintaining provenance across branches. B.5 Error Propagation If an individual agent step fails, the system creates an error-typed message and propagates it downstream. Subsequent agents receive the error context and may still produce partial results. This prioritizes operational continuity and traceability over immediate workflow abortion. B.6 Audit Logging Every execution step records timestamps, input/output previews, success or error status, and agent metadata. Combined with the workflow definition, including nodes, edges, assignments, matrix alignment score, and sufficiency report, this provides a complete decision trail from task intake to final output. Appendix C Full End-to-end Results Tables 3 and 4 provide the full category-level pass-rate breakdowns corresponding to the compact main results table. Table 3: End-to-end accuracy on the Mixed test split (300 problems). Per-category numbers are unweighted means across the benchmarks within each category. “Overall” is the unweighted mean over all rows; brackets are 95% bootstrap CIs (nboot=2000n_boot=2000) on WorkflowOps rows where per-row pass/fail data is available. Best result per column is in bold. System Overall CODE MATH QA AFlow (Zhang et al. 2025c) 0.676 0.470 0.810 0.747 ADAS (Hu et al. 2025) 0.694 0.480 0.870 0.731 MaAS (Zhang et al. 2025b) 0.400 — — — MetaAgent (Zhang et al. 2025d) 0.751 0.500 0.940 0.812 WorkflowOps (Linear) 0.870 [0.830, 0.907] 0.940 0.940 0.730 WorkflowOps (DAG) 0.890 [0.853, 0.923] 0.960 0.940 0.770 Table 4: End-to-end accuracy on the Mixed_Hard test split (694 problems): AIME 2025 (math), LiveCodeBench and CodeContests (code), and GPQA Diamond (qa). Best result per column in bold. ADAS results are from the strongest per-benchmark strategy among the 10 of 25 candidate strategies that completed; MaAS deadlocked and is omitted. System Overall CODE MATH QA AFlow (Zhang et al. 2025c) 0.218 0.000 0.200 0.365 ADAS (Hu et al. 2025) (Self-Refine, partial) 0.729 0.751 0.567 0.723 MetaAgent (Zhang et al. 2025d) 0.618 0.648 0.450 0.584 WorkflowOps (Linear) 0.763 [0.731, 0.795] 0.770 0.850 0.700 WorkflowOps (DAG) 0.770 [0.738, 0.802] 0.780 0.850 0.700 On Mixed, WorkflowOps dominates the CODE category in both modes and ties MetaAgent on MATH, while MetaAgent remains stronger on QA. On Mixed_Hard, WorkflowOps is strongest overall, with especially large gains on competition-level math and competitive results on code. A paired bootstrap test (nboot=2000n_boot=2000) on the per-row pass/fail vector finds the gap between WorkflowOps (DAG) and MetaAgent on Mixed highly significant (p<0.001p<0.001), driven by CODE (p<0.001p<0.001); MATH is statistically tied and QA marginally favors MetaAgent (p<0.05p<0.05). Cohen’s h on overall Mixed pass rate is +0.53+0.53 over AFlow, +0.49+0.49 over ADAS, +1.09+1.09 over MaAS, and +0.37+0.37 over MetaAgent. Appendix D Detailed Ablation Results Table 5 reports per-benchmark ablation results. Disabling the transition matrix reduces all categories by 2–8 points, indicating that the matrix shapes workflow construction across subtasks rather than carrying a single category. Disabling agent evolution reduces accuracy modestly but cuts calls and tokens substantially, making it a cost-optimized operating point when the static agent pool is already sufficient. Replacing embeddings with TF-IDF causes the largest degradation, validating semantic matching over sparse keyword overlap. Pure LLM matching is more expensive and less accurate than the layered matcher because it bypasses the agent-creation loop. The edge-threshold sweep peaks at the default θ=0.3θ=0.3, with θ=0.5θ=0.5 close behind. Table 5: Per-benchmark pass-rate per ablation cell on the Mixed test split (Linear mode, n=50n=50 per benchmark per cell). Variant humaneval mbpp gsm8k math drop hotpotqa Reference 0.940 0.940 0.980 0.900 0.640 0.820 No-Matrix 0.940 0.900 0.980 0.840 0.530 0.760 No-Evolve 0.900 0.960 1.000 0.920 0.560 0.780 TF-IDF 0.860 0.920 1.000 0.820 0.450 0.660 LLM-Match 0.920 0.940 1.000 0.880 0.580 0.720 θ=0.0θ=0.0 0.840 0.900 1.000 0.820 0.640 0.750 θ=0.5θ=0.5 0.900 0.920 1.000 0.840 0.660 0.810 θ=0.7θ=0.7 0.820 0.920 1.000 0.780 0.580 0.670 Appendix E Efficiency Details We report per-query operational characteristics from the Reference cell on the Mixed test split. Table 6 summarizes calls, tokens, latency, and workflow statistics; Table 7 compares DAG and Linear median wall time. Table 6: Operational statistics for the Reference cell on the Mixed test split (n=300n=300). “Calls/query” and “Tokens/query” are means; “Wall-time/query” is median. Statistic Reference LLM calls / query (mean) 14.8 decomposition 1.0 matcher (sufficiency check) ∼0 0 (embedding) agent-creation loop ∼7 7 (3.5 new agents avg.) agent execution ∼6 6 (one per node) Prompt tokens / query (mean) ∼10k 10k Completion tokens / query (mean) ∼33k 33k Total tokens / query (mean) 43,620 Wall-time / query (median) 605 s Workflow nodes / query (mean) 5.62 Workflow edges / query (mean) 5.38 Matrix alignment score A¯ A 0.181 Avg. sufficiency score ϕ¯ φ 0.748 Avg. dynamic agents created 3.50 Gap closure rate 0.96 Errors 0/300 Normalizing by the cheapest cell, No-Evolve (7.8 calls, 15.9k tokens), Reference spends 1.9×1.9× the calls and 2.7×2.7× the tokens for +1.7+1.7 points of overall accuracy. The agent-creation loop is the dominant driver of latency variance: queries that trigger multiple creation rounds run approximately 3×3× longer than queries where the static pool is already sufficient. Table 7: DAG vs. Linear median wall-time per query on the Mixed test split and selected ablation cells. “Ratio” is DAG-median / Linear-median; values close to 11 indicate near-equal wall time. Cell Linear (s) DAG (s) Ratio Reference (test, n=300n=300) 605 598 0.99× No-Matrix (test, n=300n=300) 588 590 1.00× No-Evolve (test, n=300n=300) 248 250 1.01× TF-IDF (test, n=300n=300) 425 408 0.96× LLM-Match (test, n=300n=300) 207 199 0.96× DAG mode provides consistent accuracy gains in cells where the matrix mechanism is active, but wall-time differences are small because Mixed workflows are shallow. Thus, the main benefit of DAG mode in this setting is structural guidance rather than large parallel speedup. Appendix F Additional Discussion Lightweight priors vs. trained optimizers. The transition matrix is built from simple frequency counting without gradient-based training. This design is useful when execution history exists but task-specific training is expensive: the matrix is interpretable, updates incrementally as new workflows complete, and degrades gracefully when no historical transition is available. The matrix interface can also support learned completion methods, such as matrix factorization or neural completion, without changing the surrounding orchestration pipeline. Embedding matching vs. TF-IDF. Sentence embeddings capture semantic relationships beyond keyword overlap, such as “regulatory audit” and “compliance assessment,” while retaining low cost because agent embeddings are pre-computed and cached. The matching module is abstracted, so future systems can replace the current encoder with domain-specific embeddings or cross-encoder rerankers. Matrix sparsity. With approximately 65 bootstrapped workflows over a 35-agent grid, the transition matrix has about 20% non-zero entries. This sparsity is acceptable because the matrix is used as soft guidance; workflow construction still preserves hard dependencies when the matrix has no relevant history. As more workflows execute, matrix density increases organically, and LLM-predicted links for newly created agents provide immediate approximate guidance that can later be refined. Appendix G Agent card specification Each agent in WorkflowOps is defined by a structured YAML card with the schema listed in Table 8. Table 8: Agent card schema. Field Type Description id string Unique identifier (snake_case) name string Human-readable agent name description string Capability description (primary signal for matching, 2–3 sentences) keywords list[string] Search keywords for enriching embedding vectors category enum One of: research, content, analysis, business, engineering, quality, planning instructions string System prompt governing agent behavior llm_config object Model configuration: model, temperature, max_tokens metadata dict Optional key-value metadata (e.g., priority, domain) In addition, each predefined agent has a corresponding three-layer Agent Card providing richer structured metadata: • Meta layer: Role identity (role_definition), semantic classification (semantic_tags), and overarching purpose (primary_objective). • Reasoning layer: Capabilities, anti-patterns (domain-specific guardrails), and declared dependencies (upstream/downstream agents). • Execution layer: System prompt, output schema, and tool selection. Appendix H Message schema The unified AgentMessage schema enables O(1)O(1)-composable inter-agent communication; Table 9 lists its fields. Table 9: AgentMessage schema. Field Type Description message_id UUID Unique identifier message_type enum TASK, RESULT, ERROR, or HANDOFF source_agent string Sending agent ID target_agent string Receiving agent ID content string Primary natural language body structured_data dict Optional key-value structured data artifacts list Typed attachments with name, content_type, content task_context string Original user task (immutable, passed through entire workflow) workflow_id string Workflow this message belongs to step_index int Current step number parent_message_id UUID Upstream message ID for traceability metadata dict Arbitrary metadata dictionary Appendix I Predefined agent pool The system ships with 35 predefined agents organized into 7 categories, summarized in Table 10. Table 10: Predefined agent pool composition. Category Count Representative capabilities Research 5 Information gathering, literature review, hypothesis generation Content 8 Writing, summarization, translation, report synthesis Analysis 2 Data analysis, benchmarking, comparative evaluation Business 11 Risk assessment, market analysis, legal compliance, financial modeling Engineering 4 Code generation, data processing, visualization Quality 4 Review, editing, fact-checking, peer review simulation Planning 1 Strategic planning and task decomposition Total 35 Appendix J Configuration parameters Table 11 lists the key configuration parameters and their defaults. Table 11: Key configuration parameters and their defaults. Parameter Default Description τ (SUFFICIENCY_THRESHOLD) 0.45 Minimum matching score for subtask-agent assignment θ (EDGE_THRESHOLD) 0.3 Minimum transition probability for matrix-suggested edges R (MAX_CREATION_ROUNDS) 3 Maximum rounds of dynamic agent creation TOP_K_MATCHES 3 Number of top agent candidates per subtask MAX_DAG_BRANCHING 3 Maximum branching factor in DAG workflows LLM_TEMPERATURE 0.3 Default temperature for LLM calls LLM_TIMEOUT 120s Request timeout for LLM API calls MAX_RETRIES 2 Maximum retry attempts for failed LLM calls