Paper deep dive
OOPMAS: Object-Oriented Multi-Agent Systems for Query-Level Workflow Generation
Qi Cheng, Shengyu Chen, Wei Cheng, Yiqun Xie, Xiaowei Jia, Haoyu Wang, Haifeng Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 10/13/2026, 4:30:09 AM
Summary
The paper introduces OOPMAS, a training-free framework for generating object-oriented multi-agent systems (MAS) and coordination workflows at the granularity of individual queries. Unlike existing task-level methods that apply a fixed workflow to all queries, OOPMAS dynamically generates agent sets and executable Python-based workflows tailored to specific query difficulties and types. It utilizes a dynamic skill library to accumulate structured lessons from execution feedback, enabling in-context improvement without gradient updates or fine-tuning. OOPMAS achieves 89.6% accuracy on a mixed-task benchmark (code, math, QA), outperforming the strongest baseline by 18.1 percentage points, and scales consistently across different LLM backbones.
Entities (10)
Relation Signals (5)
OOPMAS → achievesaccuracyon → Mixed Benchmark
confidence 95% · On a mixed-task benchmark of queries spanning code, math, and QA, OOPMAS achieves 89.6% accuracy
OOPMAS → istrainingfree → true
confidence 95% · We introduce OOPMAS, a training-free framework that generates both the agent set and the coordination workflow at the granularity of individual queries.
OOPMAS → outperforms → ADAS
confidence 95% · OOPMAS achieves 89.6% accuracy, outperforming the strongest baseline by 18.1 percentage points.
OOPMAS → uses → Skill Library
confidence 95% · A dynamic skill library accumulates structured lessons from execution feedback across optimization rounds, enabling in-context improvement without any gradient updates or fine-tuning.
OOPMAS → scaleswith → GPT-5.2
confidence 85% · A model-swap study across four LLM backbones shows consistent scaling, reaching 92.4% with the strongest model.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-agent systems (MAS) powered by large language models have shown strong performance across code generation, mathematical reasoning, and question answering. However, existing methods for automating MAS design mostly operate at the task level, producing a single fixed workflow per benchmark that is applied uniformly to all queries. This assumption fails under realistic conditions. Query difficulty varies widely within a task, and real-world workloads mix heterogeneous task types. We introduce OOPMAS, a training-free framework that generates both the agent set and the coordination workflow at the granularity of individual queries. Agents are represented as object-oriented class definitions with dedicated roles, tools, and persistent state, and workflows are expressed as executable main functions over these agent objects. A dynamic skill library accumulates structured lessons from execution feedback across optimization rounds, enabling in-context improvement without any gradient updates or fine-tuning. On a mixed-task benchmark of queries spanning code, math, and QA, OOPMAS achieves 89.6% accuracy, outperforming the strongest baseline by 18.1 percentage points. A model-swap study across four LLM backbones shows consistent scaling, reaching 92.4% with the strongest model.
Tags
Links
- Source: https://arxiv.org/abs/2610.07787v1
- Canonical: https://arxiv.org/abs/2610.07787v1
Trouble viewing inline? Open PDF directly →
Full Text
42,471 characters extracted from source content.
Expand or collapse full text
OOPMAS: Object-Oriented Multi-Agent Systems for Query-Level Workflow Generation Qi Cheng Affiliation: Rutgers University Affiliation: New Brunswick, NJ Email: q.cheng@rutgers.edu Shengyu Chen Affiliation: University of Pittsburgh Affiliation: Pittsburgh, PA Email: shc160@pitt.edu Wei Cheng Affiliation: NEC Labs America Affiliation: Princeton, NJ Email: weicheng@nec-labs.com Yiqun Xie Affiliation: University of Maryland Affiliation: College Park, MD Email: xie@umd.edu Xiaowei Jia Affiliation: Rutgers University Affiliation: New Brunswick, NJ Email: xj159@cs.rutgers.edu Haoyu Wang Affiliation: NEC Labs America Affiliation: Princeton, NJ Email: haoyu@nec-labs.com Haifeng Chen Affiliation: NEC Labs America Affiliation: Princeton, NJ Email: haifeng@nec-labs.com Abstract Multi-agent systems (MAS) powered by large language models have shown strong performance across code generation, mathematical reasoning, and question answering. However, existing methods for automating MAS design mostly operate at the task level, producing a single fixed workflow per benchmark that is applied uniformly to all queries. This assumption fails under realistic conditions. Query difficulty varies widely within a task, and real-world workloads mix heterogeneous task types. We introduce OOPMAS, a training-free framework that generates both the agent set and the coordination workflow at the granularity of individual queries. Agents are represented as object-oriented class definitions with dedicated roles, tools, and persistent state, and workflows are expressed as executable main functions over these agent objects. A dynamic skill library accumulates structured lessons from execution feedback across optimization rounds, enabling in-context improvement without any gradient updates or fine-tuning. On a mixed-task benchmark of queries spanning code, math, and QA, OOPMAS achieves 89.6% accuracy, outperforming the strongest baseline by 18.1 percentage points. A model-swap study across four LLM backbones shows consistent scaling, reaching 92.4% with the strongest model. 1 Introduction Figure 1: Task-level workflows fail on real-world query streams. Each fixed workflow catches only its native domain; queries from other domains and multi-domain queries (gray) are missed. No single workflow handles the full mixed distribution. Multi-agent systems (MAS) powered by large language models have emerged as a leading paradigm for solving complex tasks across code generation, mathematical reasoning, question answering, and scientific analysis (Hong et al., 2024; Wu et al., 2023; Wang et al., 2024a; Li et al., 2023) Designing these systems, however, remains a largely manual process (Nori et al., 2023; Wei et al., 2023; Madaan et al., 2023; Wang et al., 2024b). Each new task requires developers to decide which agents to create, what roles and tools to assign, and how agents should coordinate. This manual effort limits both scalability and adaptability. Recent works are moving to automate the generation and optimization of multi-agent workflows (Zhang et al., 2025a; Hu et al., 2025; Zhang et al., 2025b; Zhang et al., 2025c). These methods have shown strong results on individual benchmarks including HumanEval (Chen et al., 2021), GSM8k (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and HotpotQA (Yang et al., 2018) by optimizing a single agentic workflow for each of the benchmarks. This one workflow per task approach share a common assumption that a single fixed workflow is the optimal solution uniformly to all queries within that benchmark. We argue that this task-level assumption breaks down in realistic deployment settings for two reasons (illustrated in Figure 1). First, within a single task category, the optimal MAS structure varies with query difficulty. An easy coding problem can be solved by a straightforward workflow, while a repository-level coding task benefits from a substantially more complex workflow. A single task-level workflow must commit to one topology and cannot adapt. Second, real-world query streams are heterogeneous mixtures of task types with unpredictable distribution. A workflow optimized for code generation provides no benefit when a math reasoning or question answering query arrives while supporting mixed workloads under the task-level paradigm would require maintaining a separate optimized system per domain plus a routing mechanism. Our argument is supported by the evaluation of these task-level methods on a mixed-task benchmark containing code, math, and QA queries, where the single optimized workflow often performs well on one task but poorly on others. A second structural limitation for both task-level and query-level methods. Existing approaches search for workflow while holding the agent pool fixed (Zhang et al., 2025b; Hu et al., 2025; Gao et al., 2025). We argue that constraining the agent set limits the reachable solution space. If no agent in the pool possesses the domain expertise required by a particular query, no workflow in the search space over that pool can produce an optimal solution. The agents themselves must also be generated dynamically. In this paper, we introduce OOPMAS (Object-Oriented Programming Multi-Agent Systems), a training-free framework that generates both the agent set and the workflow for individual queries. Given a query, a frozen LLM acting as a meta-agent first generates a pool of specialist agents with tailored role prompts, tool bindings, and model configurations, then produces an executable workflow program that orchestrates these agents to solve the query. Agents are represented as object-oriented class definitions, a choice that naturally encapsulates each agent’s role, tools, and persistent state, and that aligns with code patterns prevalent in LLM pretraining data, leading to more reliable generation. The workflow is iteratively refined across multiple optimization rounds using a dynamic skill library that accumulates structured lessons from past successes and failures, enabling in-context improvement without any gradient updates, distillation, or fine-tuning. We evaluate OOPMAS on two benchmarks. The standard Mixed benchmark draws 120120 queries from six standard sources (HumanEval, MBPP, MATH, GSM8K, HotpotQA, DROP); the Hard Mixed benchmark draws 180180 competition-level queries (AIME 2025/2026, LiveCodeBench, CodeContests, GPQA Diamond) with diverse output formats. OOPMAS achieves 89.6%89.6\% on Mixed (vs. 71.5%71.5\% for ADAS, +18.1+18.1 points) and 90.8%90.8\% on Hard Mixed (vs. 72.3%72.3\% for ADAS, +18.5+18.5 points). The same gap is preserved despite the much harder problem distribution, showing that per-query generation generalises rather than relying on benchmark-specific memorisation. A model-swap study across four LLM backbones (GPT-4.1 through GPT-5.2) shows that OOPMAS scales consistently with model capability, reaching 92.4%92.4\% with GPT-5.2 without any system-level changes. Our contributions are as follows: 1. Problem formulation. We formalise MAS generation as a query-level optimisation problem over a joint agent-and-workflow search space, and argue that both query-level adaptation and dynamic agent generation are necessary for handling heterogeneous task distributions. 2. OOPMAS framework. We propose a training-free, two-phase pipeline consisting of dynamic OOP-style agent generation and iterative workflow optimisation with a structured skill library. The system requires no reinforcement learning, distillation, or fine-tuning and is immediately deployable on arbitrary task distributions. 3. Mixed and Hard Mixed benchmarks. We introduce two benchmarks that test MAS generation under realistic heterogeneous workloads, including a competition-difficulty Hard Mixed benchmark spanning AIME, LiveCodeBench, CodeContests, and GPQA. OOPMAS substantially outperforms existing task-level and query-level baselines across both benchmarks and all domains, with the gap preserved as task difficulty grows. 2 Related Work Multi-agent systems with LLMs. A first wave of LLM multi-agent work hand-designs roles, prompts, and coordination protocols for specific applications. AutoGen (Wu et al., 2023), MetaGPT (Hong et al., 2024), and CAMEL (Li et al., 2023) provide programming abstractions for orchestrating multiple specialised LLM agents and demonstrate strong results on software engineering, research, and embodied tasks. Single-agent prompting techniques such as chain-of-thought (Wei et al., 2023), tree-of-thoughts (Yao et al., 2023a), self-refine (Madaan et al., 2023), multi-persona collaboration (Wang et al., 2024b), ReAct (Yao et al., 2023b), and Voyager (Wang et al., 2023) are special cases of multi-agent design with one or two roles. While effective, these systems require human experts to choose the agent set and the coordination structure for each new task, limiting scalability and adaptability. Automated MAS generation at the task level. A second wave of work automates parts of the design process. AFlow (Zhang et al., 2025b) formulates workflow optimisation as a search problem over code-represented operator graphs and uses Monte Carlo Tree Search to discover effective workflows from a fixed operator vocabulary. ADAS (Hu et al., 2025) uses a meta-agent that iteratively programmes and archives new agent designs in code, leveraging the Turing-completeness of the search space. GPTSwarm (Zhuge et al., ) casts multi-agent construction as an optimisation over computational graphs. MaAS (Zhang et al., 2025a) searches over a multi-agent supernet to identify high-performance subnetworks, and MetaAgent Search (Zhang et al., 2025c) learns a controller that emits the best architecture per benchmark. FlowReasoner (Gao et al., 2025) also operates at this granularity. All of these methods optimise a single workflow per task category and apply it uniformly to every query in that category. We argue (§4) that this assumption breaks under heterogeneous workloads: query difficulty varies within a task and a single workflow cannot serve a mixed query stream that contains code, math, and QA. Query-level adaptation. Two earlier works move toward query-level adaptation. FlowReasoner (Gao et al., 2025) produces a workflow per query but draws agents from a fixed predefined set; the workflow is therefore restricted to the expressivity of those agents. Self-refine (Madaan et al., 2023) and tree-of-thoughts (Yao et al., 2023a) adapt the inference-time computation per query but use a single underlying agent. OOPMAS combines these two ideas: it generates both the agent set and the workflow per query, and the agents are themselves dynamically synthesised from a meta-agent rather than chosen from a closed library. Skill libraries and self-improvement. Voyager (Wang et al., 2023) accumulates a library of executable skills across game episodes and reuses them at later episodes. ADAS’s archive plays a similar role at coarse granularity. Our skill library is closer in spirit to Voyager but stores structured failure/fix records rather than executable code, and is consumed by the meta-agent as in-context optimisation memory rather than as a callable library. Crucially, skill construction in OOPMAS is training-free: the skill library improves the meta-agent’s prompt across rounds without any model parameter updates. Benchmarks for MAS evaluation. Existing MAS-generation methods are typically evaluated on individual benchmarks (HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), MATH (Hendrycks et al., 2021), GSM8K (Cobbe et al., 2021), HotpotQA (Yang et al., 2018), DROP (Dua et al., 2019)). We argue this single-benchmark evaluation does not test the realistic deployment setting. The Mixed and Hard Mixed benchmarks introduced in this paper include heterogeneous query types and output formats per run, exposing failure modes of task-level methods that single-benchmark evaluation hides. The Hard Mixed benchmark additionally tests competition-level difficulty (AIME, LiveCodeBench ( , ), CodeContests (Li et al., 2022), GPQA (Rein et al., 2023)) that has not yet saturated, providing headroom to distinguish methods. 3 Method Figure 2: The OOPMAS pipeline. (a) A meta-agent emits a tailored agent set AqA_q for each query. (b) A second meta-agent emits an executable workflow WqW_q over AqA_q, conditioned on the skill library K. (c) WqW_q is run and scored; failures are diagnosed. (d) Each diagnosis distills a reusable skill into K, which is injected into workflow generation in subsequent rounds. Training-free. In this section, we first formalize the query-level MAS generation problem in section 3.1, then we describe the two core phases of OOPMAS: dynamic agent generation and workflow generation and optimization. Figure 2 summarises the pipeline. 3.1 Problem Formulation Multi-Agent System. We define a multi-agent system (MAS) for a query q as a tuple Sq=(q,q)S_q=(A_q,W_q), where q=a1,…,anA_q=\a_1,…,a_n\ is a set of agents and qW_q is a workflow that coordinates them. Each agent ai=(pi,i,mi(hi),si)a_i=(p_i,T_i,m_i(h_i),s_i) is characterized by a role prompt pip_i specifying its role and expertise, a tool set iT_i defining its action space, a backbone model mim_i with hyperparameter such as temperature hih_i, and states sis_i that contains conversation history and intermediate results. The workflow qW_q is an executable program over qA_q. Query-Level Workflow Optimization. Given a query q drawn from an arbitrary distribution Q and a task-specific evaluation function Score(⋅)Score(·), the MAS generation problem is to find: Sq∗=argmaxSq∈Score(Sq,q),S_q^*= _S_q \;Score(S_q,q), (1) where =(,):⊆∗, over S=\(A,W):A ^*,\,W over A\ is the space of all valid multi-agent systems and ∗A^* is the space of all possible agent configurations. This is a per-query optimization: the system Sq∗S_q^* is generated fresh for each q, allowing both the agent composition and the workflow topology to adapt to the specific demands of the query. 3.2 Dynamic Agent Generation We represent both agents and workflows as executable Python code: agents follow an object-oriented programming (OOP) convention and the corresponding workflow is a main function that imports agent classes. OOP Agent Representation. Each agent in OOPMAS is a self-contained Python object whose attributes encode the role prompt pip_i, model configuration mi(hi)m_i(h_i), and persistent state sis_i, and whose methods expose the tool set iT_i and predefined tasks. This design lets an agent accumulate conversation history and intermediate results in sis_i across multiple interactions inside a single workflow execution, going beyond a single LLM API call. We choose OOP for three reasons: it naturally encapsulates per-agent state without external bookkeeping; it lets inter-agent interactions be expressed as flexible Python object interactions (passing objects as arguments, reading exposed state, calling each other’s methods) rather than being constrained to a fixed workflow schema; and OOP code is well represented in LLM pretraining corpora, making workflow code generation more reliable than custom DSLs or graph schemas. Agent Pool Generation. Given validation queries q1,…,qN\q_1,…,q_N\ from a task distribution, the meta-agent ℳagentM_agent generates a shared agent pool ¯ A in a single call. The pool is regenerated per task set; unlike methods that rely on a fixed operator set (Zhang et al., 2025b), the roles and expertise in ¯ A are tailored to the query distribution. Per-Query Coverage Verification. Although ¯ A targets common capabilities across queries, individual queries may need specialised expertise absent from the pool. For each qiq_i we run a coverage check: the system inspects ¯ A together with qiq_i, and if the pool lacks a required capability it generates additional specialist agents and merges them into a query-specific set qiA_q_i. This two-tier process balances pool reuse with per-query adaptability. 3.3 Workflow Generation and Optimisation Given the agent set qA_q, OOPMAS generates a workflow qW_q and iteratively refines it over R optimisation rounds. Rather than training the meta-agent via RL or distillation, we keep the meta-agent fixed and pursue cross-query improvement through a dynamic skill library K that accumulates structured experience from execution feedback. The pipeline is therefore training-free: no gradient updates, distillation, or fine-tuning are required, yet workflow generation improves across rounds. Workflow generation and execution. In each round, ℳworkflowM_workflow receives q, qA_q, and K, and generates an executable main function that imports agent classes, calls their interfaces, invokes tool functions, and composes outputs into a final answer. Because the workflow is unrestricted Python code over OOP agent objects, the meta-agent can express arbitrary coordination patterns without being limited to predefined operators. A lightweight static check rejects programmes that are syntactically invalid, lack an entry point, or never call agent.run; the meta-agent regenerates with the error messages, up to K repair attempts. The validated workflow runs in a standalone subprocess and is scored by a task-specific function ScoreScore. Optimisation via the skill library. After each round we classify outcomes into workflow crash, wrong output, or success. For crashes and wrong outputs, the meta-agent receives the failing workflow, the error trace, and K, and produces a structured diagnosis (root cause, fix summary, fix reasoning) followed by a corrected workflow that is re-executed. The skill library K is a structured store of FailureRecord and FixRecord entries, organised into four prompt-injectable sections: round-level score trajectories, crash patterns by error type, successful root-cause/fix pairs, and failed fix attempts. The full skill schema and a worked example are given in Appendix C. Algorithm 1 summarises the full pipeline; Phase 2 runs all queries in parallel and accumulates experience across rounds. Algorithm 1 OOPMAS: Training-Free Query-Level MAS Generation 0: Queries q1,…,qN\q_1,…,q_N\, evaluation function ScoreScore, rounds R 0: Optimized workflows qi∗i=1N\W_q_i^*\_i=1^N 1: Phase 1: Agent Generation 2: ¯←ℳagent(q1,…,qN) A _agent(\q_1,…,q_N\) ⊳ Pooled generation (single LLM call) 3: for each query qiq_i do 4: qi←CoverageCheck(¯,qi)A_q_i← CoverageCheck( A,q_i) ⊳ Augment if needed 5: end for 6: Phase 2: Iterative Workflow Optimization 7: Initialize skill library ←∅K← 8: for round r=1r=1 to R do 9: for each query qiq_i in parallel do 10: qi1…N←ℳworkflow(qi,qi,)W_q_i^1… N _workflow(q_i,A_q_i,K) ⊳ Generate workflow 11: si1…N←Score(qi1…N,qi)s_i^1… N (W_q_i^1… N,q_i) ⊳ Execute and evaluate 12: for wqij∈qi1…Nw_q_i^j _q_i^1… N do 13: if sij∈si1…N<θs_i^j∈ s_i^1… N<θ then 14: qi′←Diagnose&Fix(qi,errori,)W_q_i ← Diagnose\&Fix(W_q_i,error_i,K) 15: si′←Score(qi′,qi)s_i (W_q_i ,q_i) 16: end if 17: end for 18: end for 19: ←∪failures, fixes, scores from round rK ∪\failures, fixes, scores from round r\ 20: end for 4 Experiments We evaluate OOPMAS on four experimental settings: (i) a mixed-task benchmark of standard tasks (§4.2); (i) a hard mixed-task benchmark of competition-level tasks (§4.3) to test generalisation as task difficulty increases; (i) a component ablation that isolates the contributions of the agent pool, skill library, and optimisation memory (§4.4); and (iv) a model-swap study together with a cost–accuracy comparison against baselines (§4.5, §4.6). All experiments use GPT-5 unless stated otherwise. 4.1 Experimental Setup Mixed-Task Benchmark. Prior automated MAS-generation methods evaluate each task type separately: AFlow (Zhang et al., 2025b) and ADAS (Hu et al., 2025) run a separate optimisation per benchmark for HumanEval (Chen et al., 2021), MBPP (Austin et al., 2021), MATH (Hendrycks et al., 2021), GSM8K (Cobbe et al., 2021), HotpotQA (Yang et al., 2018), and DROP (Dua et al., 2019), producing one optimised workflow per benchmark. This task-specific generation does not reflect real-world deployment: in practice, users submit queries from a mixture of task types with unpredictable proportions, and the MAS-generation system must handle them with a single pipeline. To evaluate this setting, we construct a mixed-task benchmark of 120120 queries: 4040 code (HumanEval, MBPP), 4040 math (MATH, GSM8K), and 4040 QA (HotpotQA, DROP). All samples are drawn from the validation splits. Hard Mixed-Task Benchmark. The standard mixed benchmark is now largely saturated by strong frontier LLMs, so accuracy alone no longer separates methods. To stress-test query-level adaptation under realistic difficulty, we additionally construct a hard mixed-task benchmark of 180180 queries drawn from competition-level sources: 6060 math from AIME 2025/2026, 6060 code from LiveCodeBench (Jain et al., 2024) and CodeContests (Li et al., 2022), and 6060 QA from GPQA Diamond (Rein et al., 2023). Each domain has heterogeneous formats (integer answers for AIME, full stdin/stdout programs for CodeContests, four-way multiple choice for GPQA), forcing the MAS-generation system to produce specialised workflows per query rather than a single template. Baselines. We compare against four automated MAS-generation methods: (1) AFlow (Zhang et al., 2025b), MCTS-based optimisation of code-represented workflows over a fixed operator vocabulary; (2) ADAS (Hu et al., 2025), a meta-agent that iteratively programmes and archives agent designs; (3) MaAS (Zhang et al., 2025a), search over a multi-agent supernet; and (4) MetaAgent (Zhang et al., 2025c). For all baselines, the system receives the same mixed batch and must produce workflows that handle every query type, matching the setting OOPMAS operates in. Evaluation Metrics. We use pass@1 for code (HumanEval, MBPP, LiveCodeBench, CodeContests), solve rate for math (MATH, GSM8K, AIME), and F1 for QA (HotpotQA, DROP), and accuracy for multiple-choice (GPQA). We report per-domain accuracy averaged over the sub-benchmarks within each domain, and an Overall column averaged across all queries. Implementation Details. Unless stated otherwise, all methods use GPT-5 as both the meta-agent and the worker model. OOPMAS runs for R=5R=5 optimisation rounds on the mixed benchmark and R=3R=3 on the hard mixed benchmark, with concurrency 3030 and at most 77 agents per query. We report the highest score across the two configurations of OOPMAS (with the dynamic skill library on or off); other hyperparameters are reported in Appendix A. 4.2 Mixed-Task Benchmark Results Table 1 reports results on the standard mixed-task benchmark. OOPMAS reaches 89.6%89.6\% overall accuracy and outperforms every baseline by a wide margin: the best baseline, ADAS, reaches 71.5%71.5\%, an absolute gap of 18.118.1 percentage points. The advantage is largest on code generation (+26.026.0 points over ADAS) and persists across math and QA. Task-level baselines exhibit characteristic failure modes on heterogeneous distributions. AFlow’s MCTS optimisation discovers a single workflow specialised for one domain; that workflow scores 90.090.0 on code but 0.00.0 on math and QA, collapsing the overall score to 30.030.0. MaAS exhibits the same pathology in reverse, achieving reasonable math accuracy but near-zero on code. ADAS and MetaAgent are more balanced because their archives include heterogeneous solutions, but they still under-perform OOPMAS by a wide margin. OOPMAS’s per-query generation tailors both the agent set and the workflow topology to the specific demands of each query, so simple problems receive a single-expert call while harder ones receive multi-agent pipelines with planners, reviewers, and iterative testing loops. Table 1: Accuracy (%\%) on the Mixed and Hard Mixed benchmarks. All methods use GPT-5 as the worker model and receive the same mixed batch of queries. Best results in bold. Mixed Hard Mixed Method Code Math QA Overall Code Math QA Overall ADAS 50.0 82.5 82.0 71.5 — — — 72.3 MetaAgent 50.0 80.0 80.7 70.2 64.8 45.0 58.4 61.8 MaAS† 2.5 77.5 — 40.0 — — — — AFlow 90.0 0.0 0.0 30.0 0.0 20.0 36.5 8.9 OOPMAS (ours) 97.5 85.0 86.4 89.6 74.5 98.3 98.3 90.8 4.3 Hard Mixed-Task Benchmark Hard Mixed (right half of Table 1) is substantially more challenging: it pairs competition-level math (AIME) with stdin/stdout competitive programming (LiveCodeBench, CodeContests) and graduate-level science MCQ (GPQA). Despite the increased difficulty, OOPMAS reaches 90.8%90.8\% overall, again leading the strongest baseline (ADAS, 72.3%72.3\%) by 18.518.5 percentage points – almost identical to the gap on the easier mixed benchmark. The per-query agent and workflow generation thus generalises to harder problem distributions. The baseline failure modes on Hard Mixed are informative. AFlow’s single MCTS-discovered workflow assumed HumanEval-style function-completion outputs and is incompatible with stdin/stdout competitive programming, collapsing to 0%0\% on code. MetaAgent’s code accuracy improves slightly (50→64.850→ 64.8) but math drops sharply (80→4580→ 45) because the AIME integer-answer format is out of distribution. ADAS is the most consistent baseline (71.5→72.371.5→ 72.3) but still trails by 18.518.5 points. These patterns reinforce the central argument: a single fixed workflow cannot accommodate heterogeneous output formats and difficulty levels in realistic mixed workloads, while query-level generation handles them naturally. OOPMAS’s edge widens on math and QA, where per-query workflows exploit problem structure (chain-of-specialists for AIME, multi-perspective consensus for ambiguous GPQA items); on code the lead is smaller because failure modes (off-by-one, edge cases) are orthogonal to workflow topology. 4.4 Component Ablation Figure 3: Component ablation on Mixed. Removing each component individually leaves overall accuracy within ±3.5± 3.5 points of Full (dotted red line). The agent pool and per-query generation are the dominant accuracy drivers; skill-library and memory components contribute marginal additional points on the saturated Mixed-validation set. We ablate the four post-pool components of OOPMAS on the Mixed validation set and report results in Figure 3. Each ablation removes exactly one component while keeping the others enabled. Full corresponds to the median of four random-seed reruns of the complete system. Removing the optimisation memory (−-mem.), skill creation (−-skill_init), skill updates (−-skill_upd), skill selection (−-skill_sel), all three skill components together (−-all skills), or per-query coverage verification (−-cov.) leaves overall accuracy within a ±3.5± 3.5-point band of the Full configuration. Code accuracy is the most stable across ablations, ranging from 93.793.7 to 100.0100.0. The key takeaway is robustness: the OOP agent pool generated for the query distribution already captures most of the headroom on Mixed, and the optimisation components matter most when the validation distribution is small or when test problems differ from validation problems. Crucially, every ablation still beats every baseline in Table 1 by at least 2020 points. 4.5 Model Swap Study Figure 4: Model-swap study on Mixed. OOPMAS scales consistently with the worker model’s capability (51.0→92.451.0→ 92.4 from GPT-5-nano to GPT-5.2). At weak models (GPT-5-nano), AFlow’s fixed MCTS scaffold is more robust because it does not require the meta-agent to author high-quality agent code; at stronger models, dynamic generation dominates. We re-run OOPMAS and AFlow on the Mixed benchmark with four different worker models – GPT-4.1, GPT-5-nano, GPT-5, and GPT-5.2 – regenerating agents and workflows from scratch each time. Figure 4 shows the per-domain breakdown. OOPMAS consistently outperforms AFlow on GPT-4.1, GPT-5, and GPT-5.2, and scales smoothly with model capability (72.5%→92.4%72.5\%→ 92.4\% from GPT-4.1 to GPT-5.2). The exception is GPT-5-nano, where AFlow’s fixed MCTS workflow (65.4%65.4\%) outperforms OOPMAS (51.0%51.0\%). With a weak meta-agent, the per-query agent code is too unreliable; AFlow’s pre-discovered scaffold is more robust because it does not require online code synthesis. As the worker model strengthens, dynamic generation rapidly dominates, exceeding the fixed workflow by ≈27≈ 27 points at GPT-5.2. 4.6 Cost–Accuracy Trade-off Figure 5: Accuracy versus cost on Mixed (left) and Hard Mixed (right). OOPMAS sits in the upper-right of both panels. Costs cover the full pipeline; baseline costs are estimates derived from the configuration used in Table 1. Figure 5 plots accuracy against total dollar cost. On Mixed, OOPMAS uses $35\$35 to reach 89.6%89.6\% vs. $15\$15 for ADAS at 71.5%71.5\% – a ∼7× \!7× premium for +18+18 accuracy points. On Hard Mixed, OOPMAS at $271\$271 reaches 90.8%90.8\% vs. ADAS at $25\$25 for 72.3%72.3\%. The cost premium buys robustness (OOPMAS is the only method that maintains accuracy as difficulty grows) rather than marginal accuracy, and the per-query nature of the cost makes selective deployment easy: route cheap queries to a single-expert workflow and expensive queries to the full pipeline. The framework remains training-free, so deploying on a new domain only requires changing the validation queries. A per-stage cost decomposition is in Appendix B. 5 Conclusion 6 Conclusion We presented OOPMAS, a training-free framework for query-level multi-agent system generation. Unlike prior automated MAS design methods that optimize a single fixed workflow for an entire task or benchmark, OOPMAS treats each query as its own system-design problem. For every query, it jointly synthesises a tailored set of object-oriented agents and an executable coordination workflow, then refines the workflow over R optimization rounds using execution feedback and a dynamic skill library. This design expands the search space from workflow-only optimization over a fixed agent pool to joint agent-and-workflow generation, allowing the system to adapt both the expertise available to the workflow and the structure of the workflow itself. Limitation. Several limitations remain. First, OOPMAS depends on a strong meta-agent. When the backbone is weak, the system may fail to synthesize reliable agent classes or executable workflows, and fixed scaffolds such as AFlow can be more robust. Second, the current skill library is run-local: it improves workflow generation within an optimization run but does not yet persist across deployments, domains, or future query streams. Third, per-query generation increases inference cost substantially compared with fixed-workflow baselines, especially on hard problems that require multiple repair attempts. Finally, while Mixed and Hard Mixed cover diverse domains and formats, they remain benchmark-based evaluations; broader deployment settings may introduce tool-use, long-context, safety, privacy, and distribution-shift challenges not fully captured here. References Austin et al. (2021) J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: §2, §4.1. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba Evaluating large language models trained on code. External Links: 2107.03374, Link Cited by: §1, §2, §4.1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: §1, §2, §4.1. Dua et al. (2019) D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner DROP: a reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proc. of NAACL, Cited by: §2, §4.1. Gao et al. (2025) H. Gao, Y. Liu, Y. He, L. Dou, C. Du, Z. Deng, B. Hooi, M. Lin, and T. Pang FlowReasoner: reinforcing query-level meta-agents. External Links: 2504.15257, Link Cited by: §1, §2, §2. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt Measuring mathematical problem solving with the math dataset. External Links: 2103.03874, Link Cited by: §1, §2, §4.1. Hong et al. (2024) S. Hong, M. Zhuge, J. Chen, X. Zheng, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, C. Ran, L. Xiao, C. Wu, and J. Schmidhuber MetaGPT: meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1, §2. Hu et al. (2025) S. Hu, C. Lu, and J. Clune Automated design of agentic systems. External Links: 2408.08435, Link Cited by: §1, §1, §2, §4.1, §4.1. Jain et al. (2024) N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica LiveCodeBench: holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974. Cited by: §4.1. Li et al. (2023) G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem CAMEL: communicative agents for "mind" exploration of large language model society. External Links: 2303.17760, Link Cited by: §1, §2. Li et al. (2022) Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago, T. Hubert, P. Choy, C. de Masson d’Autume, I. Babuschkin, X. Chen, P. Huang, J. Welbl, S. Gowal, A. Cherepanov, J. Molloy, D. J. Mankowitz, E. Sutherland Robson, P. Kohli, N. de Freitas, K. Kavukcuoglu, and O. Vinyals Competition-level code generation with alphacode. Science 378 (6624), p. 1092–1097. External Links: ISSN 1095-9203, Link, Document Cited by: §2, §4.1. Madaan et al. (2023) A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark Self-refine: iterative refinement with self-feedback. External Links: 2303.17651, Link Cited by: §1, §2, §2. Nori et al. (2023) H. Nori, Y. T. Lee, S. Zhang, D. Carignan, R. Edgar, N. Fusi, N. King, J. Larson, Y. Li, W. Liu, R. Luo, S. M. McKinney, R. O. Ness, H. Poon, T. Qin, N. Usuyama, C. White, and E. Horvitz Can generalist foundation models outcompete special-purpose tuning? case study in medicine. External Links: 2311.16452, Link Cited by: §1. Rein et al. (2023) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof q&a benchmark. External Links: 2311.12022, Link Cited by: §2, §4.1. Wang et al. (2023) G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar Voyager: an open-ended embodied agent with large language models. External Links: 2305.16291, Link Cited by: §2, §2. Wang et al. (2024a) Y. Wang, M. Wang, M. A. Manzoor, F. Liu, G. Georgiev, R. J. Das, and P. Nakov Factuality of large language models: a survey. External Links: 2402.02420, Link Cited by: §1. Wang et al. (2024b) Z. Wang, S. Mao, W. Wu, T. Ge, F. Wei, and H. Ji Unleashing the emergent cognitive synergy in large language models: a task-solving agent through multi-persona self-collaboration. External Links: 2307.05300, Link Cited by: §1, §2. Wei et al. (2023) J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou Chain-of-thought prompting elicits reasoning in large language models. External Links: 2201.11903, Link Cited by: §1, §2. Wu et al. (2023) Q. Wu, G. Bansal, J. Zhang, Y. Wu, B. Li, E. Zhu, L. Jiang, X. Zhang, S. Zhang, J. Liu, A. H. Awadallah, R. W. White, D. Burger, and C. Wang AutoGen: enabling next-gen llm applications via multi-agent conversation. External Links: 2308.08155, Link Cited by: §1, §2. Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §1, §2, §4.1. Yao et al. (2023a) S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan Tree of thoughts: deliberate problem solving with large language models. External Links: 2305.10601, Link Cited by: §2, §2. Yao et al. (2023b) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. External Links: 2210.03629, Link Cited by: §2. Zhang et al. (2025a) G. Zhang, L. Niu, J. Fang, K. Wang, L. Bai, and X. Wang Multi-agent architecture search via agentic supernet. External Links: 2502.04180, Link Cited by: §1, §2, §4.1. Zhang et al. (2025b) J. Zhang, J. Xiang, Z. Yu, F. Teng, X. Chen, J. Chen, M. Zhuge, X. Cheng, S. Hong, J. Wang, B. Zheng, B. Liu, Y. Luo, and C. Wu AFlow: automating agentic workflow generation. External Links: 2410.10762, Link Cited by: §1, §1, §2, §3.2, §4.1, §4.1. Zhang et al. (2025c) Y. Zhang, X. Liu, and C. Xiao MetaAgent: automatically constructing multi-agent systems based on finite state machines. External Links: 2507.22606, Link Cited by: §1, §2, §4.1. [26] M. Zhuge, W. Wang, L. Kirsch, F. Faccio, D. Khizbullin, and J. Schmidhuber GPTSwarm: language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, Cited by: §2. Technical Appendices Appendix A Reproducibility Details Hardware and runtime. Experiments were run on a single workstation with 4848 CPU threads; all LLM calls are offloaded to the OpenAI API. A typical Mixed run completes in 3030–6060 minutes; a typical Hard Mixed run takes 1.51.5–33 hours. Wall-clock is dominated by API latency rather than local compute. Evaluation specifics. For HumanEval and MBPP we treat a query as solved iff every assertion in the dataset’s test list passes, evaluated in an isolated subprocess. For MATH and GSM8K we use the standard boxed answer extraction. For HotpotQA and DROP we use F1 on the predicted answer. For AIME we extract the final integer in the range [0,999][0,999] and compare to ground truth. For LiveCodeBench and CodeContests we run the predicted programme on every public and private test case and require all to pass. For GPQA we extract the final letter ∈A,B,C,D∈\A,B,C,D\. Appendix B Per-Stage Cost Breakdown Figure 6: Token usage by OOPMAS on the Mixed benchmark, decomposed into Optimisation (workflow generation, skill update, repair) and Execution (per-agent inference). Total tokens scale roughly linearly with model size up to GPT-5; GPT-5.2 uses fewer optimisation tokens than GPT-5 because its workflows succeed earlier and require less repair. Figure 6 reports the total token usage for OOPMAS across four backbone models. The Optimisation phase dominates (workflow generation, repair, skill creation), while Execution (per-agent calls inside the workflow) is small in comparison because most queries terminate within a single agent call. A GPT-5 run on Mixed costs approximately $3535 at current public API pricing, or $0.300.30 per query. On Hard Mixed, the per-query cost rises to $1.501.50 because problems are longer and require more repair attempts. Within a single Mixed run, the per-stage decomposition is approximately: agent generation 0.3%0.3\%, skill creation/update 1.0%1.0\%, skill selection 4.5%4.5\%, workflow generation 58%58\%, workflow execution 36%36\% (numbers from a representative GPT-5 run; full breakdowns are produced by the codebase under each run’s cost_summary.json). Appendix C Skill Library Examples The skill library is a structured store of FailureRecord and FixRecord entries serialised between rounds. A single skill manifest entry has the schema: "skill_id": "code_generate_test_refine", "name": "Code: Generate-Test-Refine Loop", "applicable_domains": ["code"], "when_to_use": "Code-generation problems with verifiable test cases.", "strategy": "1. Pass the full task to a code-writer agent. 2. Run the function’s tests via execute_code. 3. On failure, pass error+code to a code-refiner agent. 4. Print the final code between markers.", "detection_keywords": ["writer.run", "execute_code", "CODE_START"] After each optimisation round, the meta-agent reviews up to 3030 failure traces and 1010 successful fixes from the round and produces an updated manifest. We constrain each skill to a single domain to prevent the workflow generator from importing cross-domain agents into a single workflow (e.g., importing a math agent into a code workflow), which is a major source of contamination in early prototypes.