Paper deep dive
From Specification to Execution: AI Assisted Scientific Workflow Management
Komal Thareja, Hamza Safri, Rajiv Mayani, Anirban Mandal, Ewa Deelman
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 6/21/2026, 2:27:20 AM
Summary
The paper presents an AI-assisted scientific workflow management approach that utilizes a specification-driven methodology to separate workflow intent, design, and implementation. This approach improves transparency and reproducibility compared to direct LLM code synthesis. The system integrates a structured specification phase, an LLM-based debugging agent for autonomous failure diagnosis and recovery, and a Model Context Protocol (MCP) layer for remote management. The architecture combines the Pegasus Workflow Management System (WMS) with HTCondor for distributed execution. The system was evaluated using a federated learning workflow for medical imaging (using TCIA and NIH ChestX-ray datasets) to demonstrate its ability to handle complex, iterative, and dependency-intensive scientific pipelines.
Entities (9)
Relation Signals (4)
HTCondor → executesjobsfor → Pegasus WMS
confidence 100% · The AI-assisted authoring component... is submitted to the Pegasus Workflow Management System (WMS)... Execution is delegated to HTCondor
pegasus-ai → providesskillsfor → Pegasus WMS
confidence 100% · The primary plugin that we developed for this work is pegasus-ai [7], which provides skills for Pegasus-based workflow authoring.
LLM-based Debugging Agent → diagnosesfailuresin → Pegasus WMS
confidence 90% · The debugging agent analyzes this information to detect anomalies and identify failures... in the Pegasus WMS
Model Context Protocol (MCP) → enablesremotemanagementof → Pegasus WMS
confidence 90% · integrate Pegasus, a widely used WMS, with a Model Context Protocol (MCP) layer, providing a unified interface for workflow submission, monitoring, and control.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Scientific workflow management systems (WMS) support scalable and reproducible execution of complex pipelines, but workflow design, implementation, and debugging remain largely manual and require significant expertise. Recent approaches using large language models (LLMs) show promise for workflow generation from natural language, but often rely on direct code synthesis, which limits transparency, reproducibility, and integration with workflow systems. We present an AI-assisted approach to scientific workflow management that combines specification-driven workflow generation, automated debugging, and distributed execution. The method introduces a structured specification phase that separates workflow intent, design, and implementation, allowing validation prior to code generation. We also develop an LLM-based debugging agent that diagnoses and resolves failures across multiple system layers. To support distributed execution and user interaction, we integrate Pegasus, a widely used WMS, with a Model Context Protocol (MCP) layer, providing a unified interface for workflow submission, monitoring, and control. We evaluate the approach using a federated learning workflow for medical imaging, chosen for its parallel, iterative, and dependency-intensive structure. The system generated and executed large-scale workflows with thousands of jobs, reduced debugging effort, and allowed non-expert users to construct workflows with expert-level design patterns. These results indicate that end-to-end AI-assisted workflow generation and execution is feasible, and point toward AI-driven platforms for managing the scientific workflow lifecycle.
Tags
Links
- Source: https://arxiv.org/abs/2606.18425v1
- Canonical: https://arxiv.org/abs/2606.18425v1
Trouble viewing inline? Open PDF directly →
Full Text
48,029 characters extracted from source content.
Expand or collapse full text
From Specification to Execution: AI Assisted Scientific Workflow Management Komal Thareja ∗ , Hamza Safri † , Rajiv Mayani † , Anirban Mandal ∗ , Ewa Deelman † ∗ RENCI, University of North Carolina at Chapel Hill, NC, USA † Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA Abstract—Scientific workflow management systems (WMS) support scalable and reproducible execution of complex pipelines, but workflow design, implementation, and debugging remain largely manual and require significant expertise. Recent ap- proaches using large language models (LLMs) show promise for workflow generation from natural language, but often rely on direct code synthesis, which limits transparency, reproducibility, and integration with workflow systems. We present an AI-assisted approach to scientific workflow management that combines specification-driven workflow generation, automated debugging, and distributed execution. The method introduces a structured specification phase that separates workflow intent, design, and implementation, allowing validation prior to code generation. We also develop an LLM-based debugging agent that diagnoses and resolves failures across multiple system layers. To support distributed execution and user interaction, we integrate Pegasus, a widely used WMS, with a Model Context Protocol (MCP) layer, providing a unified interface for workflow submission, monitoring, and control. We evaluate the approach using a federated learning workflow for medical imaging, chosen for its parallel, iterative, and dependency-intensive structure. The sys- tem generated and executed large-scale workflows with thousands of jobs, reduced debugging effort, and allowed non-expert users to construct workflows with expert-level design patterns. These results indicate that end-to-end AI-assisted workflow generation and execution is feasible, and point toward AI-driven platforms for managing the scientific workflow lifecycle. Index Terms—scientific workflows, workflow management, large language models, medical imaging, Pegasus WMS, feder- ated learning I. INTRODUCTION Scientific workflows represent complex computational pipelines as directed graphs of jobs with explicit data and control dependencies. Workflow management systems (WMS) such as Pegasus [1], Nextflow [2], and Galaxy [3] support scal- able and reproducible execution through job orchestration, data management, provenance tracking, and fault tolerance across distributed environments. These systems focus on workflow execution, however, and assume that workflows are already defined; workflow design, implementation, and debugging remain manual and time-consuming. Prior work has explored automated workflow generation. Early approaches rely on semantic representations and plan- ning, such as Wings [4], where workflows are constructed from formal descriptions of components and constraints. These methods give correctness guarantees but require manually en- gineered ontologies and lack flexibility. More recent work uses large language models (LLMs) for workflow generation from natural language descriptions [5], [6]. These approaches show that LLMs can translate user intent into executable pipelines, but they often rely on direct code generation, which limits transparency, reproducibility, and reliability. Existing LLM- based methods are also typically not tightly integrated with workflow management systems and do not address distributed orchestration or runtime interaction. We propose an AI-assisted scientific workflow manage- ment approach that combines specification-driven workflow generation, automated debugging, and distributed execution in a single architecture. The approach improves transparency and reproducibility by separating workflow intent, design, and implementation through a structured specification phase, which allows explicit validation before execution. We also in- troduce an LLM-powered debugging agent that performs fail- ure diagnosis and recovery through log analysis and targeted corrective actions, forming a closed-loop execution model. To support distributed operation, we integrate Pegasus WMS with a Model Context Protocol (MCP) layer that provides a unified interface for workflow submission, monitoring, and control across distributed resources. We evaluate the approach using a federated learning work- flow for medical imaging, chosen for its parallelism, iterative execution, and complex inter-job dependencies, all of which stress workflow design and orchestration. The main contri- butions are: (1) a specification-driven workflow generation methodology that separates intent, design, and implementa- tion; (2) an automated LLM-based debugging agent for failure diagnosis and recovery; and (3) an architecture combining workflow generation, execution, and remote management. I. AI-ASSISTED WORKFLOW MANAGEMENT A. Overview of AI-Assisted Workflow Generation We propose a specification-driven approach to AI-assisted workflow generation. The process is decomposed into three explicit stages: prompt and dataset description, specification generation, and code generation (Fig. 1). This design separates user intent from implementation, supporting transparent and reproducible workflow construction. The process begins with a prompt and dataset description, where the user provides a natural language specification of the desired workflow along with input data references. This stage captures high-level intent, domain constraints, and ex- perimental objectives without requiring detailed knowledge of the underlying workflow system. arXiv:2606.18425v1 [cs.SE] 16 Jun 2026 From this input, the system generates a structured specifi- cation that formalizes the workflow design. The specification defines the workflow structure, data dependencies, processing steps, and configuration parameters; the validation plan (ex- periments, metrics, and expected figures); a record of resolved design decisions and their trade-offs; and an implementation roadmap describing phases and current status. Unlike direct code synthesis, this specification is a human-readable artifact, allowing users to validate and refine the workflow design prior to execution. Capturing tests and phases alongside structure makes the generated workflow not only executable but also checkable and incrementally implementable. Once the specification is approved, the system performs code generation, producing executable workflow artifacts such as workflow definitions, wrapper scripts, configuration files, and container specifications. These artifacts are directly deployable within the workflow management system, ensuring consistency between the validated design and the final imple- mentation. The entire lifecycle is supported by domain-specific skills, which provide workflow-oriented guidance for prompt inter- pretation, specification synthesis, and artifact generation. By grounding the language model in reusable workflow knowl- edge and templates, the system produces outputs that conform to the requirements of the target execution environment. This approach improves transparency, user control, and reproducibility relative to direct LLM-based code generation for complex scientific workflows. Fig. 1: Integrated AI workflow design pipeline. The plugin marketplace provides domain-specific skills for workflow gen- eration. B. Domain-Specific Workflow Plugins To support AI-assisted workflow generation, we organize domain knowledge into plugins. A plugin is a packaged ex- tension that provides workflow-specific instructions, templates, and optional tools for the language model. Within each plugin, functionality is organized into skills, where each skill imple- ments a specific workflow-oriented operation. This modular structure allows the model to invoke only the capability needed for the current step, while keeping the generation process structured and easier to control. In our system, the plugin marketplace serves as a curated collection of such plugins for scientific workflow develop- ment. The primary plugin that we developed for this work is pegasus-ai [7], which provides skills for Pegasus-based workflow authoring. These skills support tasks such as gener- ating a project scaffold, creating wrapper scripts, producing container definitions, reviewing generated code, diagnosing failures, and converting workflows between formats. Together, they translate high-level user intent into executable workflow artifacts, including workflow definitions, configuration files, and execution environments. To improve correctness, each skill is supported by struc- tured reference material, including documentation, example workflows, validation checklists, and reusable templates. This grounding helps constrain the generation process and en- courages outputs that follow Pegasus-specific requirements, such as correct API usage, data staging patterns, and con- tainer configuration. By packaging workflow knowledge into reusable plugins and skills, the system improves consistency, reproducibility, and extensibility across workflow development tasks. C. Specification-Driven Development Our approach uses a specification-driven development methodology [8] for workflow generation. Rather than directly synthesizing executable code from user input, the system de- composes the process into three stages: prompt interpretation, specification construction, and code generation. Given a user prompt and dataset description, the system first produces a structured, human-readable specification that captures the workflow design, including data dependencies, execution stages, and configuration parameters. The specifi- cation also records a validation plan (experiments, metrics, and expected figures), the design decisions resolved during specification with their trade-offs, and an implementation roadmap with phases and status. This specification serves as an intermediate artifact that can be reviewed and validated before implementation, and provides explicit acceptance criteria for the generated code. Only after validation does the system generate executable workflow artifacts, including workflow definitions, wrapper scripts, configuration files, and container specifications. This separation between design and implementation improves trans- parency and reproducibility, and ensures that generated work- flows are consistent with the intended high-level structure. In contrast to direct LLM-based interaction, where users iteratively prompt the model to produce code fragments until a working solution emerges, this approach makes design decisions explicit and reviewable. By introducing a specifi- cation as an intermediate representation, the system avoids implicit assumptions embedded in generated code and enables controlled, auditable workflow construction. D. Skills in Pegasus-AI Plugin The pegasus-ai [7] plugin provides eight domain- specific skills for Pegasus workflow development. These skills can be grouped into three functional categories: workflow construction, validation & recovery, and reuse & support. a) Workflow construction: The workflow construction skills support the initial creation of executable Pegasus work- flows. The /pegasus-scaffold skill generates a complete workflow project from a high-level pipeline description, in- cluding the workflow generator, wrapper scripts, Dockerfile, and site catalogs. The /pegasus-wrapper skill creates wrapper scripts that interface Pegasus job execution with computational tools, requiring the user to provide the tool name, inputs/outputs, and whether to use a Python or shell wrapper The /pegasus-dockerfile skill generates con- tainer definitions with the required software dependencies for reproducible execution. b) Validation and recovery: The validation and re- covery skills support quality control and fault diagnosis. The /pegasus-debug skill analyzes workflow failures using execution logs and known failure patterns, while the /pegasus-review skill inspects workflow code for com- mon anti-patterns and correctness issues. c) Reuse and support: The reuse and support skills extend the workflow development process to migration, guid- ance, and experiment setup. The /pegasus-convert skill converts workflows between supported formats. The /pegasus-help skill provides context-aware Pegasus doc- umentation and guidance. The /kiso skill generates experi- ment configurations for the Kiso [9] experiment management platform. These groups provide a modular interface for AI-assisted workflow construction and debugging, while keeping the gen- eration process structured and easier to control. E. Integrated System Architecture 1) System Components: The integrated system, as shown in Figure 2 is organized around four main components: AI- assisted workflow authoring and generation, an AI-enabled autonomous debugging and recovery agent, the Pegasus work- flow management system with HTCondor execution [10], and a remote interaction management component using a layered Model Context Protocol (MCP) interface. The AI-assisted authoring component described in the previous sections gen- erates workflow artifacts from user input, including workflow definitions, wrapper scripts, configuration files, and container specifications compatible with Pegasus. These artifacts are submitted to the Pegasus Workflow Management System (WMS), which serves as the central orchestration layer by constructing workflow DAGs, managing data staging, coor- dinating job execution, and monitoring workflow progress. Execution is delegated to HTCondor, which handles resource allocation, job scheduling, and distributed execution across compute resources with built-in fault tolerance. The AI debug- ging and recovery agent described in the next section handles autonomous debugging and failure recovery for the workflows in execution. In parallel, the MCP interface enables interaction between external clients and the workflow system. 2) AI Debugging and Recovery: The architecture includes an AI debugging agent that handles failure diagnosis and re- covery during workflow execution. As workflows execute, Pe- gasus and HTCondor continuously generate logs and runtime metadata describing job execution, resource usage, and failure conditions. The debugging agent analyzes this information to detect anomalies and identify failures by interpreting execution traces and matching them to known failure patterns, such as Fig. 2: Integrated architecture combining AI-assisted workflow generation, debugging, and execution with remote monitoring via MCP. file staging errors, container issues, resource misconfiguration, dependency conflicts, or incorrect wrapper scripts. Based on this analysis, the agent proposes targeted corrective actions, including updating workflow parameters, modifying resource requests, or correcting execution logic. The system then re- submits affected jobs or sub-workflows, forming a closed- loop recovery mechanism in which failures are detected, diagnosed, and corrected during execution. This capability is important for large-scale hierarchical workflows, where manual debugging would be costly and error-prone. 3) MCP-Based Remote Management: The system supports remote interaction through a layered MCP architecture com- posed of a gateway layer and a set of execution nodes. The execution nodes are responsible for performing workflow operations on distributed resources and expose capabilities for building, planning, executing, monitoring, and analyzing workflows through Pegasus and HTCondor. The gateway layer serves as the entry point for external clients by receiv- ing requests, resolving the appropriate execution node based on user context and resource configuration, and forwarding the request for execution. This abstraction allows clients to interact with a single interface without requiring knowledge of individual execution nodes. Execution nodes dynamically register with the gateway, enabling flexible resource discovery and multi-node coordi- nation. In addition, the gateway integrates auxiliary services such as documentation and guidance, allowing both workflow operations and support functions to be accessed through a unified interface. Remote clients, such as chat-based user interfaces, interact with the system using structured tool calls, where requests (e.g., workflow submission, status queries, or debugging actions) are transmitted to the gateway, routed to the selected execution node, executed within the workflow environment, and returned as structured responses containing execution status, logs, or control results. This design enables real-time monitoring and control of workflows without requir- Fig. 3: Single-round federated learning workflow showing fan- out (client training) and fan-in (aggregation) phases. ing direct access to the execution infrastructure. I. EVALUATION SETUP A. Use Case: Federated Learning We use a federated learning (FL) workflow for medical imaging to evaluate the proposed system. This setting lets us assess both the system’s ability to generate workflow artifacts and its ability to design complex workflows. Federated learning (FL) [11] is a distributed learning paradigm in which multiple clients collaboratively train a shared model without exchanging raw data. Each client per- forms local training and periodically shares model updates with a central aggregation process, typically using Federated Averaging FedAvg [11]. Training proceeds in iterative rounds, where selected clients train in parallel (fan-out) and their updates are aggregated into a global model (fan-in) (Fig. 3). This results in a hierarchical and iterative workflow with strong dependencies across rounds. Such workflows involve parallel job coordination and inter- dependent stages across multiple clients, along with challenges such as data heterogeneity, communication overhead, and synchronization. These characteristics make FL a suitable benchmark for the proposed skill-based workflow generation approach and the AI-assisted debugging agent. B. Datasets We evaluate the proposed system using two public medical imaging datasets with complementary characteristics. TCIA [12] provides 4,144 3D CT/MRI lung volumes from multiple collections. A representative slice from each volume is extracted, resized to 224× 224, and distributed across K clients to simulate a federated learning setting. The task is binary classification (lung pathology present/absent). NIH ChestX-ray [13] consists of 50,000 2D chest X-ray images resized to 224× 224. Data is partitioned by patient ID across K clients to ensure independence, with a binary classification task (“No Finding” vs. pathology). Together, these datasets cover both multi-source heterogene- ity (TCIA) and large-scale patient-partitioned data (NIH), sup- porting evaluation under diverse federated learning conditions. TABLE I: Experimental configurations. Alg.RExpKEFocus FedAvg50E1102Baseline E352Scalability E6 ‡ 105Optimization 10E4105Communication FedProx50E2102Algorithm ‡ Adam optimizer, LR=10 −4 , frozen backbone, class-weighted loss, augmentation, gradient clipping, cosine LR scheduler. C. Experimental Setup The experimental workflows are generated using the ap- proach described in Section I-E. All experiments use a ResNet-18 model with an ImageNet-pretrained backbone, batch size 32, learning rate 0.001, and stochastic gradient descent with momentum 0.9 and weight decay 10 −4 . These configurations are designed to isolate the effects of algorithm choice, client scalability, communication efficiency, and train- ing optimization. To evaluate the capabilities of the generated workflows, we define five experimental configurations that explore different aspects of the federated learning (FL) workflow, including algorithm choice, scalability, communication efficiency, and training optimization (Table I). E1 uses standard FedAvg with default hyperparameters and no optimizations, serving as a controlled reference point against which all other configurations vary exactly one factor. E2 evaluates the impact of algorithm choice by replacing FedAvg with FedProx [14]. E3 studies scalability by reducing the number of clients to K=5, increasing data per client while reducing aggregation diversity. E4 investigates communication efficiency by reducing the number of rounds to T=10 and increasing local epochs to E=5. E6 investigates whether modern training optimizations, including backbone freezing (training only the classifier head), the Adam optimizer with a reduced learning rate (10 −4 ), class-weighted cross-entropy loss, data augmentation, gradient clipping, and cosine learning rate scheduling, can improve federated convergence. Experiments were conducted on a distributed HTCondor pool provisioned on the FABRIC testbed [15], comprising four GPU-equipped worker nodes across multiple sites: HAWI (3 NVIDIA GPUs), MAX-1 (2), MAX-2 (2), and NCSA (2), for a total of 9 GPUs. All jobs were executed within Docker containers using CondorIO for data staging, with each training job requesting 1 GPU, 4 CPU cores, and 15 GB of memory. Workflow execution was orchestrated by Pegasus WMS, which handled planning, scheduling, data staging, and provenance tracking across hierarchical sub-workflows, while HTCondor managed distributed resource allocation and parallel execution. D. Evaluation Metrics We evaluate the system at two complementary levels: work- flow generation performance and federated learning perfor- mance. Workflow Generation Performance. We evaluate the gen- erated workflows along three dimensions: (i) execution suc- cess, measured by successful workflow completion without runtime errors (i) structural quality, including correct job de- pendencies and alignment with federated learning patterns; and (i) development effort, reflecting the number of iterations and manual interventions required to obtain a working workflow. Federated Learning Performance. We assess the quality of the trained models using standard metrics, including accuracy and F1 score, to evaluate the effectiveness of the workflows in producing correct machine learning results. This evaluation framework enables us to distinguish be- tween the quality of the workflow generation process itself and the performance of the trained models. Scope of Evaluation: The MCP-based remote management layer is part of the overarching architecture proposed in this work and was described to present a complete picture of the integrated system. A quantitative evaluation of the MCP layer, including gateway routing behavior, multi-node coordination, and remote-client interaction patterns, is outside the scope of this paper. The experimental evaluation results in the remainder of the paper focuses on the specification-driven workflow design, the AI debugging agent, and execution on Pegasus and HTCondor; the MCP interface is left for a future dedicated study. IV. RESULTS A. Workflow Design and Generation During the specification phase, the LLM presented two orchestration strategies for the iterative FL rounds: (1) an ensemble-manager pattern, in which a single top-level work- flow re-invokes itself across rounds using an external co- ordinator, and (2) a sub-workflow pattern, in which each round is encapsulated as a Pegasus SubWorkflow with explicit data dependencies between rounds. The ensemble-manager approach offers lower scheduling overhead by avoiding per- round workflow planning, and supports dynamic runtime deci- sions such as early stopping based on convergence. However, it introduces a single point of failure in the external coordinator and provides less granular fault tolerance and provenance. The sub-workflow approach incurs additional scheduling latency per round, but provides per-round fault tolerance (failed rounds can be rescued independently while preserving completed work), per-round provenance tracking, and per-round cleanup of intermediate files all handled natively by Pegasus. The user selected the sub-workflow approach, as the fault tolerance and provenance benefits outweigh the scheduling overhead for long-running FL workflows where GPU computation domi- nates execution time. Figure 4 shows the resulting top-level workflow DAG for a single dataset branch. Data is downloaded and partitioned into client shards, after which T federated learning rounds execute as SubWorkflows, each implementing the fan-out/fan- in pattern shown in Fig. 3. A centralized baseline and result visualization complete the pipeline. The full workflow repli- cates this branch for each dataset, enabling parallel execution. Fig. 4: Generated top-level workflow DAG for a single dataset branch. Each Round t is a Pegasus SubWorkflow implement- ing the fan-out/fan-in pattern of Fig. 3. TABLE I: Workflow generation comparison (manual work- flow used as reference baseline). MetricClaudeCodexKimiManual User prompts7737– Files generated223315– SpecificationYesYesYesNo Design guidance500N/A Orchestration opts.YesNoNoYes OrchestrationSubWFSubWF † SubWF ‡ Ensemble ∗ SubWF supportInitialLater † Later ‡ Yes CondorIO stagingInitialLater † Later ‡ Yes Parallel datasetsInitialLater † InitialYes GPU containersInitialLater † Initial § Yes PyTorch modelsInitialLater † InitialYes FedProx supportInitialLater † InitialYes Staging resolvedYesYesNo ‡ Yes ReproducibilityHighLowLowManual † Follow-up prompts required; ‡ SubWorkflow staging unresolved ; § Fixed in later sessions ; ∗ Preferred for large-scale workflows; B. Workflow Generation Comparison To evaluate the effectiveness of AI-assisted workflow gen- eration, we compare four approaches: Claude Code with the pegasus-ai plugin, OpenAI Codex/Opencode [16] and Kimi K2.6 with the same Pegasus reference material available in the working directory, and a manually implemented work- flow by an expert in federated learning and Pegasus WMS (Table I). All the LLM systems had access to identical Pega- sus documentation and reference materials. Claude integrates them as structured skills invoked during generation, whereas Codex and Kimi consume them as static context. All three LLM-based systems produced functional Pegasus workflows using the same number of initial user prompts, but differ substantially in design completeness, interaction TABLE I: Development effort comparison across ap- proaches. Manual estimates are based on prior Pegasus work- flow development experience. MetricClaudeCodexKimiManual Sessions268– User prompts523737– Output tokens∼109K∼237K∼49K– Estimated cost (USD)$10–15$15–20<$2– Development time2 days3 weeks4 days3–4 months Production-readyYesYesPartial † Yes † Generates valid DAG; not yet executed on distributed infrastructure. effort, and reproducibility. Claude Code produced a modular, feature-complete workflow [17] in a single session, guided by structured skills that surface design alternatives and enforce consistent patterns. Codex produced a simplified initial work- flow [18] and required multiple follow-up interactions to add sub-workflows, data staging, parallel dataset execution, and realistic model implementations. Kimi, despite receiving the same Pegasus skills and SubWorkflow reference documenta- tion, required the most iterative refinement [19]. Kimi did not resolve the SubWorkflow staging conflict; it oscillated between flat DAG and sub-workflow approaches across multiple debug- ging sessions without converging on a correct solution. The systems differ in how they handle design guidance. Claude exposes orchestration alternatives, including ensemble- manager and sub-workflow approaches, so the user can make informed decisions before code generation. Neither Codex nor Kimi surfaces such guidance; workflow design evolves incrementally through additional prompts. From an expert perspective, the generated workflows are structurally correct, but optimal choices may differ depending on scale; experts often prefer the Pegasus Ensemble Manager for large iterative workflows due to its improved scalability and reduced orches- tration overhead [20]. Reproducibility, defined as the ability to regenerate the same workflow from the same input, further differentiates the approaches. Claude’s specification-driven process supports consistent regeneration by capturing workflow design in an explicit intermediate representation, whereas both Codex and Kimi rely on prompt sequences, making reproduction de- pendent on interaction history. The manual workflow, while correct, depends on developer expertise and lacks a systematic mechanism for reproducible construction. These results indi- cate that structured, skill-based workflow generation allows non-expert users to produce workflows comparable to expert implementations with lower development effort and better reproducibility. C. Cost of AI-Assisted Development Table I compares the development effort for all LLM- assisted approaches and the manual baseline. Claude Code sessions used Claude Opus 4 ($15/M input, $75/M output tokens); Codex sessions used OpenAI’s Gpt 5.4 model; Kimi sessions used Kimi K2.6 via Opencode. Claude produced a production-ready workflow in 2 sessions (52 prompts,∼$10–15), while Codex required 6 sessions over 3 weeks (37 prompts,∼$15–20) to incrementally add features that Claude generated in the initial session. Despite fewer total prompts, Codex consumed more output tokens (∼237K vs∼109K) due to repeated regeneration of the monolithic workflow generator across sessions. Kimi achieved the lowest token cost (<$2,∼49K output tokens) across 8 sessions and 37 prompts over 4 days, reflecting Kimi K2.6’s lower per- token pricing. However, the majority of Kimi’s sessions (6 of 8) were spent debugging Pegasus-specific issues, SubWork- flow file registration, staging policy conflicts, and shared- filesystem assumptions. Despite this effort, Kimi was unable to resolve the SubWorkflow staging issue, oscillating between flat DAG and sub-workflow approaches without converging on a correct solution. While Kimi’s workflow generates a valid Pegasus DAG, the unresolved staging conflict means it has not been successfully executed on distributed HTCondor infrastructure. The debugging phase dominated Claude’s cost (45 of 52 prompts), whereas Codex spread effort across feature ad- ditions that Claude’s plugin-augmented generation handled in the initial session. All LLM-assisted approaches incur negligible cost compared to the GPU resources required for the experiments (9 GPUs running for multiple days), and reduce development time relative to manual implementation. Constructing comparable workflows manually, which involves learning Pegasus WMS and then implementing and validating complex pipelines, would require an estimated 3–4 months of effort by an experienced user. D. AI Debugging Agent The LLM-based debugging agent resolved several classes of runtime failures during workflow execution, spanning Pegasus configuration, HTCondor scheduling, Docker containerization, and filesystem management. Disk exhaustion: Repeated container staging duplicated large images across sub-workflows, exhausting available disk space. The agent identified this issue from disk usage logs and recommended bypassing staging. Missing cleanup: Intermediate files accumulated across rounds due to disabled cleanup. The agent detected this condition and enabled in-place cleanup to remove files after use. Experiment collisions: Concurrent experiments overwrote shared output paths. The agent identified conflicting directories and introduced per-experiment namespacing. Model configuration errors: A mismatch between model output dimensions and class-weight computation caused train- ing failures. The agent traced the issue to the loss configuration and corrected it. Missing dependencies: Workflow jobs failed due to unde- clared script dependencies. The agent diagnosed the errors and added the required files to the workflow definition. The agent was able to analyze logs, identify root causes, and apply targeted fixes across multiple system layers, reducing debugging effort relative to manual inspection. a) LLM vs. User vs. Expert.: We compare workflow gen- eration from three perspectives: a user with limited experience in Pegasus WMS, the skill-based approach, and an expert with experience in both federated learning and Pegasus WMS. A non-expert user typically constructs a sequential pipeline with predefined rounds and explicit job definitions, without using advanced orchestration features. The skill-based system, by contrast, uses structured abstractions such as sub-workflows and guides the user toward more scalable and maintainable workflow designs. From an expert perspective, the generated workflow is structurally correct and uses advanced Pegasus features, but some federated-learning-specific refinements, such as early stopping or alternative orchestration strategies (e.g., Ensemble Manager), are not explicitly incorporated. This reflects the system’s primary focus on workflow design and orchestration rather than federated learning optimization. Prior work on federated learning workflow design in Pegasus WMS [20] discusses the importance of such design choices for large-scale iterative workflows. The generated workflow still applies de- sign patterns that would otherwise require significant Pegasus WMS expertise. The skill-based approach brings non-expert users closer to expert-level workflow design, reducing the need for manual expertise but leaving room for expert refinement. E. Federated Learning Performance Table IV summarizes the final-round accuracy and F1 score across all experimental configurations, compared against cen- tralized training baselines, while Figure 5 shows convergence behavior for FedAvg (E1) and FedProx (E2). Federated learn- ing consistently underperforms centralized training on both datasets, reflecting the challenges of distributed optimization under data heterogeneity and communication constraints. 010203040 FL Round 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Test Accuracy TCIA Collections FedAvg (E1) FedProx (E2) Centralized (0.96) 010203040 FL Round 0.3 0.4 0.5 0.6 0.7 Test Accuracy NIH Chest X-Ray14 FedAvg (E1) FedProx (E2) Centralized (0.67) Fig. 5: Convergence comparison: FedAvg (E1) vs. FedProx (E2) over 50 FL rounds on both datasets. Dashed line shows centralized baseline accuracy. In the baseline configuration (E1), FedAvg achieves 47.2% accuracy (F1=0.43) on TCIA and 46.1% accuracy (F1=0.32) on NIH, compared to centralized results of 96.0% and 67.0%, TABLE IV: Final performance comparison across datasets. TCIA (FL)NIH (FL)Centralized ExpAccF1AccF1Acc (TCIA)Acc (NIH) E10.470.430.460.320.960.67 E20.460.310.460.310.950.67 E30.570.360.470.320.940.67 E40.550.440.460.430.950.66 E60.560.360.460.320.860.66 respectively. TCIA exhibits gradual convergence, plateauing around round 10, whereas NIH converges rapidly to a sub- optimal solution, indicating sensitivity to class imbalance. Although the datasets are reasonably sized, the effective data available per client in the federated setting is limited. For example, in the TCIA setup with K=10 clients, each client receives approximately 330 samples, which constrains local training capacity and contributes to the observed performance gap between federated and centralized models. In E2, FedProx (μ=0.01) performs worse than FedAvg, with early stagnation observed on both datasets, suggesting that the proximal constraint limits effective local updates. In E3, reducing the number of clients from K=10 to K=5 improves TCIA performance (57.1% accuracy), indicating that increased data per client can compensate for reduced aggregation diversity, while NIH performance remains largely unchanged. In E4, reducing communication rounds (T=10) and increasing local epochs (E=5) yields the best F1 scores on both datasets (TCIA: 0.44, NIH: 0.43), demonstrating the importance of local computation. E6 applies several training optimizations, including back- bone freezing, the Adam optimizer with a reduced learning rate (10 −4 ), class-weighted loss, data augmentation, gradient clipping, and cosine learning rate scheduling, to address the convergence issues observed in E1–E4. Despite these improve- ments, the federated models still converge to majority-class prediction (TCIA: 55.9% accuracy, NIH: 46.3%), collapsing after the first aggregation round. The centralized baselines under the same training regime achieve 86.1% (TCIA) and 65.5% (NIH), confirming that the optimizations are effective when data is pooled but insufficient to overcome the challenges of federated averaging with small per-client shards. The TCIA centralized accuracy drops from 96% (E1) to 86% under backbone freezing, reflecting the trade-off between preserving pretrained features and limiting model capacity. The persistent FL collapse despite class-weighted loss and frozen-backbone averaging suggests that the main challenge lies in the inter- action between weight averaging and limited per-client data, rather than in optimizer or loss function choice alone. F. Workflow Execution Statistics Table V summarizes both the workflow structure and ex- ecution characteristics across all configurations. Large-scale experiments (E1 and E2) consist of 101 sub-workflows (50 rounds across two datasets), generating over 2,400 and 4,400 jobs respectively. Each sub-workflow incurs planning and scheduling overhead, giving a wall-to-compute ratio of approx- imately 4×. Reducing the number of rounds (E4) decreases workflow complexity by 80% and lowers execution time to 4.8 hours, showing the impact of communication frequency. Reducing the number of clients (E3) lowers the number of training jobs but not the number of sub-workflows, indicating that workflow complexity is driven primarily by the number of rounds. All configurations (E1 through E4) completed without job- level retries. E6 required two DAG-level rescues due to TABLE V: Federated learning workflow structure and execu- tion statistics. ConfigurationFL StructureExecution ExpKTETrainAggEvalJobsWall (h)Ratio E11050210001004242020.04.4× E21050210001004242022.34.1× E355025001004122016.64.3× E4101052002044844.83.9× E61050510001004242025.13.1× runtime errors, which were diagnosed and corrected by the debugging agent (Section IV-D). These results show that work- flow structure, in particular the number of rounds, dominates execution cost, while Pegasus provides scalable orchestration for large iterative workflows. V. BACKGROUND AND RELATED WORK Research on automated scientific workflow generation spans three main directions: semantic workflow synthesis, scien- tific workflow management systems (SWfMS), and LLM- based workflow generation. While each addresses part of the problem, neither provides a unified solution that combines workflow synthesis, execution in systems such as Pegasus, and distributed orchestration. A. Semantic and Scientific Workflow Systems Early approaches rely on semantic representations and plan- ning, such as the Wings framework [4], where workflows are generated from formal descriptions of components, in- puts, outputs, and constraints. These methods provide strong correctness guarantees and scalability but require manually engineered ontologies, limiting flexibility. In contrast, workflow management systems such as Pegasus, Nextflow, and Galaxy focus on execution, providing schedul- ing, data management, and reproducibility across distributed environments [1], [3], [21]. However, they assume workflows are already defined, resulting in a separation between workflow generation and execution. B. LLM-Based and Agentic Workflow Generation Recent work explores the use of large language models (LLMs) to generate workflows from natural language spec- ifications. Zeng et al. [5] demonstrate workflow generation using structured prompting and API grounding, while studies in bioinformatics show similar applications in systems such as Galaxy and Nextflow [22]. These approaches are flexible but rely heavily on prompting strategies and domain knowledge. Agentic approaches further extend this idea by letting LLMs orchestrate tools dynamically. Zhang et al. [6] model workflow generation as a search over structured tool graphs. While ef- fective for iterative refinement, these methods are not tailored to DAG-based scientific workflows and lack integration with workflow management systems. Balis et al. [23] introduce a skill-based approach combining intent extraction with deterministic DAG construction. Al- though closer to scientific workflows, their approach targets HyperFlow and does not address distributed orchestration. C. Orchestration and Emerging Protocols Recent systems adopt structured orchestration mechanisms to coordinate workflows and external tools. Protocols such as the Model Context Protocol (MCP) enable standardized interaction between LLMs and external services. Existing approaches, however, do not integrate such protocols with scientific workflow systems like Pegasus, nor do they provide end-to-end support for workflow generation, execution, and runtime interaction. VI. CONCLUSION We presented an AI-assisted approach to scientific work- flow management that combines specification-driven workflow generation, automated debugging, hierarchical execution with Pegasus WMS, and MCP-based remote management. The specification-driven methodology supports structured and re- producible workflow design; the skill-based system generates complex hierarchical workflows; and the debugging agent resolves failures across multiple system layers. The MCP interface supports remote submission, monitoring, and control, with the submit node coordinating execution and managing workflow artifacts across distributed resources. Federated learning was used as a representative use case due to its iterative, distributed, and dependency-intensive na- ture, which stresses workflow design and orchestration. We evaluated our system by generating and executing federated learning workflows on the FABRIC testbed. Our evaluation results demonstrated that the system could generate and autonomously execute workflows with thousands of jobs. We measured the effectiveness of the AI-assisted workflow generation capabilities using several leading AI tools, including Claude Code and codex, and compared the costs of AI-assisted development. Our results also showed how the AI debugging agent was able to autonomously detect and recover from several clases of runtime failures. The perfor- mance results for the federated learning show that workflow structure, in particular the number of rounds, is the dominant factor in execution cost. Future work will focus on evaluating the performance of the MCP layer, and building an AI-driven platform that guides users in designing more complex workflows, and manages the full workflow lifecycle - from specification to optimization of workflow runs across heterogeneous infrastructures. ACKNOWLEDGMENTS This work is supported by the US National Science Foun- dation grant #2513101. We acknowledge the FABRIC Testbed (NSF #2330891). The authors used Claude, an AI assistant, to help edit and refine the manuscript; all intellectual contri- butions and final editorial decisions remain solely with the human authors. REFERENCES [1] E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. F. da Silva, M. Livny, and K. Wenger, “Pegasus, a workflow management system for science automation,” Future Generation Computer Systems, vol. 46, p. 17–35, 2015. [2] P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational work- flows,” Nature Biotechnology, vol. 35, no. 4, p. 316–319, 2017. [3] E. Afgan, D. Baker, B. Batut, M. van den Beek, D. Bouvier, M. ˇ Cech, J. Chilton, D. Clements, N. Coraor, B. A. Gr ̈ uning et al., “The galaxy platform for accessible, reproducible and collaborative biomedical anal- yses,” Nucleic Acids Research, vol. 46, no. W1, p. W537–W544, 2018. [4] Y. Gil, V. Ratnakar, E. Deelman, G. Mehta, and J. Kim, “Wings for pegasus: Creating large-scale scientific workflows using semantic representations,” in Proceedings of the 19th International Conference on Scientific and Statistical Database Management (SSDBM), 2007. [5] H. Zeng et al., “Flowmind: Automatic workflow generation with large language models,” arXiv preprint arXiv:2404.13050, 2024. [6] C. Zhang et al., “Aflow: Automating agentic workflow generation,” arXiv preprint arXiv:2410.10762, 2024. [7] K. Thareja and R. Mayani, “Claude code plugin pegasus-ai,” https://github.com/pegasus-isi/claude-plugin-marketplace/tree/main/ plugins/pegasus-ai, 2026, claude Code plugin pegasus-ai. [8] D. B. Piskala, “Spec-driven development: From code to contract in the age of ai coding assistants,” arXiv preprint arXiv:2602.00180, 2026. [9] R. Mayani, K. Vahi, M. Rynge, K. Thareja, X. Casas-Moreno, H. Jin, A. Mandal, F. Lordan, K. Raghavan, R. M. Badia, and E. Deelman, “Kiso: A foundation for complex, agentic, and reproducible experi- ments,” Frontiers in Complex Systems, vol. 4, p. 1800335, 2026. [10] D. Thain, T. Tannenbaum, and M. Livny, “Distributed computing in prac- tice: The condor experience,” Concurrency and Computation: Practice and Experience, vol. 17, no. 2–4, p. 323–356, 2005. [11] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y. Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial Intelligence and Statistics. PMLR, 2017, p. 1273– 1282. [12] K. Clark, B. Vendt, K. Smith, J. Freymann, J. Kirby, P. Koppel, S. Moore, S. Phillips, D. Maffitt, M. Pringle et al., “The cancer imaging archive (tcia): maintaining and operating a public information repository,” Jour- nal of Digital Imaging, vol. 26, no. 6, p. 1045–1057, 2013. [13] X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p. 2097–2106, 2017. [14] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” Proceedings of Machine Learning and Systems, vol. 2, p. 429–450, 2020. [15] I. Baldin, A. Nikolich, J. Griffioen, I. I. S. Monga, K.-C. Wang, T. Lehman, and P. Ruth, “Fabric: A national-scale programmable ex- perimental network infrastructure,” IEEE Internet Computing, vol. 23, no. 6, p. 38–47, 2020. [16] Opencode Contributors, “Opencode: An open-source ai coding agent,” https://github.com/opencode-ai/opencode, 2025, accessed: 2025-05-18. [17] K. Thareja, H. Safri, and E. Deelman, “Medical imaging fed- erated learning workflow (claude),” https://github.com/pegasus-isi/ medical-imaging-fl-workflow, 2026, generated with Claude Code and the pegasus-ai plugin. [18] —, “Fl pegasus workflow (codex),” https://github.com/kthare10/ fl-pegasus-workflow-gpt-5.4, 2026, generated with OpenAI Codex. [19] —,“Flchestworkflow(kimi),”https://github.com/kthare10/ fl-chest-workflow-kimi, 2026, generated with Opencode and Kimi K2.6. [20] H. Safri, G. Papadimitriou, F. Desprez, and E. Deelman, “A workflow management system approach to federated learning: Application to industry 4.0,” in 2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT). IEEE, 2024, p. 259–263. [21] P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational work- flows,” Nature Biotechnology, vol. 35, no. 4, p. 316–319, 2017. [22] Anonymous, “From prompt to pipeline: Large language models for scientific workflow development in bioinformatics,” arXiv preprint, 2025. [23] B. Balis et al., “From research question to scientific workflow: Leverag- ing agentic ai for science automation,” arXiv preprint arXiv:2604.21910, 2026.