Paper deep dive
Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows
Merve Astekin, Yan Naing Tun, Arda Goknil, Erik Johannes Husom, Lwin Khin Shar, Hasan Sözer, Ratnadira Widyasari, Hui Song
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 10/5/2026, 4:58:07 AM
Summary
This paper presents a systematic empirical study comparing agentic Large Language Model (LLM) systems against non-agentic baselines across five software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. The study evaluates six open-weight LLMs, two prompt strategies, and three hardware platforms, measuring accuracy, inference latency, and energy consumption. Key findings indicate that multi-agent designs consume on average 6.36x more energy and run 6.07x longer than non-agentic baselines, with limited accuracy gains that are task-specific. Non-agentic and single-agent configurations dominate the Pareto front for sustainability and efficiency, suggesting that lightweight, task-aware configurations are often preferable for sustainable software engineering tools.
Entities (21)
Relation Signals (8)
Energy Consumption â correlateswith â Inference Latency
confidence 95% · Energy and latency are tightly coupled (Pearson r=0.987 in logâlog space)
Non-Agentic Baseline â dominates â Pareto Front
confidence 95% · lightweight non-agentic and single-agent configurations still dominate the Pareto front, accounting for 59 of 66 Pareto-optimal configurations.
Multi-agent system â hasmetricimpact â Energy Consumption
confidence 95% · multi-agent designs consume on average 6.36à as much energy... as the non-agentic baseline
Multi-agent system â hasmetricimpact â Inference Latency
confidence 95% · multi-agent designs... run 6.07à as long as the non-agentic baseline
Code Vulnerability Detection â benefitsfrom â Multi-agent system
confidence 90% · multi-agent improves average vulnerability-detection accuracy
Gemma â isvariantof â CodeGemma
confidence 90% · Gemma and its code-specialized variant CodeGemma
Qwen â isvariantof â Qwen-Coder
confidence 90% · Qwen and its code-specialized variant Qwen-Coder
AutoGen â supports â Multi-Agent Collaboration
confidence 90% · AutoGen [61], which enables multi-agent collaboration for solving complex tasks
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are increasingly used in software engineering, including agentic systems that coordinate multiple agents, but impose higher computational and environmental costs. In this paper, we present a comprehensive empirical study of agentic LLM systems across five software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. For each task, we compare LLM configurations that range from a non-agentic single-query baseline to multi-agent workflows, using six open-weight LLMs, two prompt strategies, and three hardware platforms. We assess each configuration in terms of accuracy, inference latency, and energy consumption. Our results reveal substantial trade-offs between agentic complexity and energy efficiency: multi-agent designs consume on average 6.36$\times$ as much energy and run 6.07$\times$ as long as the non-agentic baseline, with worst-case slowdowns of up to 160$\times$ for individual task--hardware pairs. Accuracy gains from additional agents are limited and task-specific: multi-agent improves average vulnerability-detection accuracy, but lightweight non-agentic and single-agent configurations still dominate the Pareto front, accounting for 59 of 66 Pareto-optimal configurations. Model and prompt choice act as task-specific levers whose effective direction varies between tasks rather than as global defaults. We translate these findings into design guidelines for sustainable, task-aware LLM-based development tools.
Tags
Links
- Source: https://arxiv.org/abs/2610.03010v1
- Canonical: https://arxiv.org/abs/2610.03010v1
Trouble viewing inline? Open PDF directly â
Full Text
189,044 characters extracted from source content.
Expand or collapse full text
Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows MERVE ASTEKIN, SINTEF, Norway YAN NAING TUN, Singapore Management University, Singapore ARDA GOKNIL, SINTEF, Norway ERIK JOHANNES HUSOM, SINTEF, Norway LWIN KHIN SHAR, Singapore Management University, Singapore HASAN SĂZER, Ozyegin University, Turkiye RATNADIRA WIDYASARI, Singapore Management University, Singapore HUI SONG, SINTEF, Norway Large language models (LLMs) are increasingly used in software engineering, including agentic systems that coordinate multiple agents, but impose higher computational and environmental costs. In this paper, we present a comprehensive empirical study of agentic LLM systems across five software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. For each task, we compare LLM configurations that range from a non-agentic single-query baseline to multi-agent workflows, using six open-weight LLMs, two prompt strategies, and three hardware platforms. We assess each configuration in terms of accuracy, inference latency, and energy consumption. Our results reveal substantial trade-offs between agentic complexity and energy efficiency: multi-agent designs consume on average 6.36Ă as much energy and run 6.07Ăas long as the non-agentic baseline, with worst-case slowdowns of up to 160Ăfor individual taskâhardware pairs. Accuracy gains from additional agents are limited and task-specific: multi-agent improves average vulnerability-detection accuracy, but lightweight non-agentic and single-agent configurations still dominate the Pareto front, accounting for 59 of 66 Pareto-optimal configurations. Model and prompt choice act as task-specific levers whose effective direction varies between tasks rather than as global defaults. We translate these findings into design guidelines for sustainable, task-aware LLM-based development tools. ACM Reference Format: Merve Astekin, Yan Naing Tun, Arda Goknil, Erik Johannes Husom, Lwin Khin Shar, Hasan Sözer, Ratnadira Widyasari, and Hui Song. 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows. 1, 1 (October 2026), 50 pages. https://doi.org/10.1145/n.n 1 Introduction Large Language Models (LLMs) are transforming software engineering by supporting a variety of tasks, including code generation, debugging, code analysis, and documentation [4,18,44,60,66]. Authorsâ Contact Information: Merve Astekin, SINTEF, Oslo, Norway, merve.astekin@sintef.no; Yan Naing Tun, Singapore Management University, Singapore, Singapore, yannaingtun@smu.edu.sg; Arda Goknil, SINTEF, Oslo, Norway, arda. goknil@sintef.no; Erik Johannes Husom, SINTEF, Oslo, Norway, erik.johannes.husom@sintef.no; Lwin Khin Shar, Singapore Management University, Singapore, Singapore, lkshar@smu.edu.sg; Hasan Sözer, Ozyegin University, Istanbul, Turkiye, hasan.sozer@ozyegin.edu.tr; Ratnadira Widyasari, Singapore Management University, Singapore, Singapore, ratnadiraw. 2020@phdcs.smu.edu.sg; Hui Song, SINTEF, Oslo, Norway, hui.song@sintef.no. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM X-X/2026/10-ART https://doi.org/10.1145/n.n , Vol. 1, No. 1, Article . Publication date: October 2026. arXiv:2610.03010v1 [cs.SE] 2 Oct 2026 2Astekin et al. These capabilities have been accessed through single-query interactions with general-purpose models. On the other hand, the rise of LLM-based multi-agent systems, where multiple specialized agents collaborate to tackle different parts of a task, promises to enhance reasoning, problem decomposition, and robustness in developer workflows [18]. Early work in this area, such as agent frameworks for secure code generation and multi-agent debate mechanisms, suggests potential benefits including improved code correctness, enhanced security, and better scalability [9,27,41, 47,65]. At the same time, LLMs incur high environmental costs, including substantial energy consumption, water usage, andCO 2 emissions, with impacts sometimes magnitudes greater for reasoning-intensive prompts compared to simpler ones [26,28,48]. This highlights an urgent need to consider sustainability alongside performance when designing agentic LLM systems. Problem Statement. Despite the growing enthusiasm for LLM-based multi-agent systems in software engineering, the trade-offs between their effectiveness and sustainability remain under- investigated. Early work has demonstrated that agentic architectures, where autonomous agents orchestrate, refine, and reason collectively, can enhance task performance in domains like code generation and complex problem solving, with success rates increasing by up to 70% compared to single-agent approaches [51]. Similarly, literature reviews emphasize the transformative potential of such systems for autonomous software development workflows [18]. However, this enhanced capability often demands greater computational overhead. Previous studies have highlighted that LLM inference consumes substantial energy and contributes notably toCO 2 emissions, especially for reasoning-intensive prompts [2, 10, 26, 48]. Crucially, no existing study simultaneously evaluates agentic LLM configurations across multiple tasks, models, and hardware platforms with respect to both performance (accuracy, latency) and environmental cost (energy consumption). Without such a multi-dimensional, empirically grounded evaluation, practitioners lack the evidence needed to design effective yet sustainable multi-agent development tools. This gap calls for a structured analysis that bridges performance gains with ecological and practical deployment considerations. Goal of This Study. The primary goal of this study is to systematically investigate the design, performance, and sustainability trade-offs of multi-agent LLM systems for automating software engineering tasks. Although multi-agent LLM architectures are increasingly studied across diverse domains, systematic empirical evidence on their effectiveness and efficiency in practical developer tasks remains scarce. This study aims to fill this gap by providing empirical evidence and actionable insights across a representative spectrum of software engineering tasks, including code generation, where LLMs are used to synthesize executable source code from natural language prompts; technical debt identification, which involves analyzing code snippets for signs of code smells; code vulnerability detection, which targets identifying security flaws or unsafe coding patterns; log parsing, where structured representations are extracted from raw log files; and log analysis, which involves interpreting parsed logs to detect errors or anomalous system behaviors. We select these five software engineering tasks because they represent a diverse set of challenges across the software development lifecycle. They vary in terms of input/output structure, reasoning complexity, and reliance on external tools, making them well-suited for evaluating the performance and scalability of different LLM configurations. This diversity enables a comprehensive assessment of how agentic system design impacts effectiveness, efficiency, and sustainability in real-world development scenarios. For each task, we implement a series of increasingly complex LLM- based configurations, i.e., non-agentic single-query models, single-agent setups, dual-agent configurations, and multi-agent workflows involving collective intelligence and division of labor among agents. These configurations are deployed using multiple families of LLMs (i.e., Gemma and its code-specialized variant CodeGemma, Qwen and its code-specialized variant Qwen-Coder, , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows3 and DeepSeek-Coder) and evaluated on different hardware platforms, including a developer workstation and two high-performance servers. Our goal is to evaluate each system setup along three critical dimensions: accuracy, to measure task-specific effectiveness; inference latency, to understand the responsiveness and feasibility of real-time deployment; and energy consumption, to assess the computational cost of running different configurations. We conduct this study with the expectation that trade-offs will emerge across multiple variation points, including the software engineering task, the underlying LLM model, the agentic system design, and the prompting strategy. This multi-dimensional evaluation enables us to quantify the costâbenefit trade-offs involved in deploying agentic LLM systems for software engineering and to derive generalizable insights into when and how such systems can be used effectively and sustainably. Research Questions. We structure our study around four research questions (RQs), focusing on the effectiveness, efficiency, and sustainability of LLM-based agentic systems. Specifically, we investigate: âąHow do different LLM system configurations perform across software engineering tasks in terms of accuracy? (RQ1) âą How do LLM system configurations differ in energy consumption across software engineering tasks and hardware platforms? (RQ2) âą What is the inference latency of different agent configurations across software engineering tasks and hardware platforms? (RQ3) âą What trade-offs exist between performance, energy efficiency, and latency across software engineering tasks and hardware platforms, and are there task-specific patterns that inform the design of sustainable multi-agent LLM systems? (RQ4) Our Findings. Our results consistently show that agentic complexity has bounded benefits and systematic costs. The non-agentic baseline (NA) has the highest average accuracy for four of the five tasks, while the multi-agent configuration (MA) performs best for vulnerability detection. Moving from NA to MA increases energy consumption by 6.36Ăand latency by 6.07Ăon average, with worst-case slowdowns of up to 160Ă. Of 66 Pareto-optimal configurations across all tasks, only one is multi-agent, and 36 (55%) are non-agentic. Agentic configuration is therefore a first-order factor, but accuracy gains from additional agents are task-specific while the cost increase is much more systematic. Model and prompt choice, in contrast, have task-specific effects whose direction depends on the task: no single model wins across tasks, and prompt strategy alone can swing log-parsing accuracy from 3.4% to 54.3%. Hardware effects are modest for lightweight workloads but become first-order for heavier configurations, where a high-end server can be 3.4Ăfaster than a workstation on MA. Energy and latency are tightly coupled (Pearsoní=0.987 in logâlog space), so optimizing one effectively optimizes the other. Together, these findings indicate that competitive LLM-based developer tools can be built with lightweight, task-aware configurations, and that multi-agent designs should be adopted only with concrete evidence of a task-specific benefit. Our Contributions. This paper makes the following key contributions: âą A comprehensive multi-dimensional evaluation of LLM system configurations across diverse software engineering tasks. We design and implement multiple LLM-based system configurations (ranging from non-agentic baselines to complex multi-agent workflows) for five representative software engineering tasks: code generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. This task diversity allows us to assess how configuration complexity impacts effectiveness and efficiency in varied developer contexts. , Vol. 1, No. 1, Article . Publication date: October 2026. 4Astekin et al. âąA systematic comparison of agentic configurations using different LLMs and hard- ware platforms. We experiment with a range of instruction-tuned open-weight LLMs spanning multiple model families and sizes, and deploy them across heterogeneous hardware setups (i.e., workstations and servers). This enables us to investigate how model characteristics and deployment environments influence task performance and resource usage. âąA detailed empirical analysis of trade-offs between accuracy, latency, and energy consumption. We measure each configurationâs performance using a unified evaluation framework that captures inference latency and energy use. Our results reveal how agen- tic complexity, model size, prompting strategies, and hardware choice interact to affect performance and sustainability outcomes. âą Identification of task-specific and generalizable patterns across agentic design choices. By comparing configurations across tasks, we uncover when lightweight agentic setups (e.g., single or dual agents) suffice and when more complex multi-agent workflows yield measur- able benefits. We highlight configurations that offer favorable trade-offs, enabling developers to make informed design decisions based on their constraints and priorities. âąDesign guidelines for sustainable and effective LLM-based developer tools. Drawing from our findings, we distill a set of practical recommendations for building energy-efficient, task-aware, and scalable agentic LLM systems. These guidelines are intended to inform the future development of intelligent assistants, autonomous developer agents, and AI-augmented software engineering platforms. Collectively, these contributions provide a comprehensive view of the effectiveness, efficiency, and sustainability of agentic LLM systems, offering guidance for their practical design and deployment in software engineering. The remainder of this paper is organized as follows. In Section 2, we summarize the background on LLMs and energy monitoring. Section 3 presents our experimental setup, results, and analysis. In Section 4, we summarize our findings for each research question. Section 5 synthesizes these findings across RQs and distils design guidelines for sustainable agentic LLM systems. Section 6 presents threats to the validity of our experiments. Finally, Section 7 concludes the paper and outlines directions for future research. 2 Background and Related Work 2.1 Multi-Agent LLM Systems Multi-agent systems (MAS) powered by Large Language Models (LLMs) are an emerging paradigm for enabling collaboration, specialization, and iterative reasoning across autonomous agents. In contrast to single-turn, single-agent interactions, multi-agent LLM systems leverage structured communication and role decomposition to handle complex tasks more effectively. Recent surveys and systematizations outline the architectural components of LLM-based multi- agent systems, including agent roles, memory, planning, and interaction protocols. Li et al. [36] provide a comprehensive taxonomy of multi-agent LLM capabilities, highlighting key challenges in scalability, coordination, and self-evolution. Similarly, Tran et al. [57] and Liu et al. [37] survey coordination strategies in multi-agent LLMs, offering categorizations of mechanisms and patterns that frame the design space of such systems. Several open-source agent frameworks have been developed to support these patterns. They provide infrastructure for task orchestration, agent communication, and integration with external tools or data sources. Examples include AutoGen [61], which enables multi-agent collaboration for solving complex tasks, and Semantic Kernel [50], which combines LLMs with modular plugins for scalable AI systems. AutoGPT [6] supports autonomous multi-step task execution through , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows5 planning and refinement. Similarly, OpenAgents [63] and AgentGPT [1] focus on deploying au- tonomous AI agents, emphasizing modularity, integration, and usability for real-world applications. These frameworks excel in enabling multi-agent collaboration, where specialized agentsâsuch as those for code generation, result interpretation, or output refinementâwork together to achieve shared objectives. Frameworks like AutoGen and AutoGPT decompose complex workflows into manageable tasks, enhancing the efficiency and flexibility of LLM-powered systems. Interoperabil- ity is another critical feature, allowing seamless integration with external databases, APIs, and knowledge graphs. Tools like OpenAgents, AgentGPT, and Semantic Kernel streamline the processing of heterogeneous data, aligning real-time and historical information with system-specific knowledge to generate actionable insights. Multi-agent LLM systems represent a significant evolution in AI-assisted software engineering, offering new capabilities for structured collaboration and task specialization. While frameworks such as AutoGen, OpenAgents, and AgentGPT provide a foundation for their development, rigorous empirical evaluation, particularly with respect to performance-efficiency trade-offs and environ- mental impact, is still lacking. This paper contributes to filling that gap by systematically assessing multi-agent configurations across software engineering tasks, models, and hardware platforms. 2.2 Multi-Agent LLM Systems in Software Engineering Recent advancements in LLM-based MAS have sparked considerable interest in the software engineering community, owing to their enhanced problem-solving capabilities and structured collaboration patterns [18]. For instance, multi-agent frameworks built on LLMs have significantly advanced the field of automated code generation [18,29]. These approaches decompose the complex process of software synthesis into specialized sub-tasks, such as planning, coding, testing, and de- bugging, handled by collaborating agents, thereby addressing limitations in coherence, correctness, and scalability inherent to single-agent LLM systems. AgentCoder by Huang et al. [22] exemplifies this architecture with a system comprising programmer, test designer, and test executor agents. These agents iteratively generate, test, and refine code, resulting in performance gains on Hu- manEval and MBPP benchmarks over state-of-the-art single-agent models. Similarly, CodeTree [34] introduces a tree-search-based planning strategy for LLM agents, which guides decision-making across code generation stages via both environment feedback and LLM-generated reasoning. Another approach, CodeCoR, integrates self-reflective evaluation into the multi-agent code pipeline, combining prompt design, test generation, repair, and re-coding steps to ensure robustness and quality of results [45]. These frameworks often feature feedback loops where code is revised based on syntactic or semantic failures, a technique that has also been extended to security-enhanced frameworks such as AutoSafeCoder, which includes fuzz testing and static analysis agents to enforce safety constraints [41]. INTERVENOR [59] employs a collaborative agent architecture consisting of a Code Learner and a Code Teacher. The Code Learner is responsible for generating an initial code implementation and compiling it to assess correctness. Upon detecting errors, the Code Teacher analyzes both the faulty code and the resulting error reports, then provides targeted repair instructions to guide the refinement of the solution. Self-repair [42] and TGen [40] improve code quality by leveraging feedback from the execution of predefined test cases to iteratively refine and correct generated programs. MAGIS [53] introduces a four-agent system (Manager, Custodian, Developer, and QA) that cooperatively resolves GitHub issues, achieving up to an eightfold increase in resolution rates compared to GPT-4 alone. AgentMesh [30] structures agents into Planner, Coder, Debugger, and Reviewer roles that sequentially transform high-level requirements into tested, validated code, a practical orchestration of developer workflows. Self-Organized Agents [27] propose a dynamically scaled agent group to collaboratively generate and optimize ultra-large codebases. Their framework surpasses single-agent baselines by 5% in pass@1 accuracy (i.e., the , Vol. 1, No. 1, Article . Publication date: October 2026. 6Astekin et al. statistical probability that a single, independent generation attempt by an AI model correctly solves a given problem) on HumanEval through distributed, component-wise code generation. While promising for software engineering tasks, multi-agent LLM systems also introduce new challenges in orchestration overhead, prompt alignment, tool integration, and reproducibility. Moreover, the environmental cost of multi-agent coordination remains largely unexplored. Increased agent interactions and tool calls often translate into higher inference latency and energy usage, raising important concerns around sustainability that existing frameworks do not address explicitly. 2.3 Energy & Sustainability in LLMs The environmental footprint of modern AI has become a central research concern. Early landmark analyses quantified the energy and carbon costs of training large NLP models and called for standardized reporting practices [21,52]. In parallel, the Green AI perspective argued for evaluating models not only by accuracy but also by efficiency metrics such as energy, emissions, and compute, to counter the escalating resource demands of state-of-the-art systems [49]. Subsequent systems- level work showed that lifecycle emissions depend strongly on model size, training/inference efficiency, datacenter efficiency (PUE), and the carbon intensity of the electricity mix, highlighting that location and infrastructure choices can change emissions by orders of magnitude [46]. Beyond training, inference is now a dominant and fast-growing driver of AIâs footprint due to widespread deployment of LLMs. Studies emphasize that prompt length, decoding strategy, and multi-turn/agentic interaction patterns materially increase energy use and latency, raising the need for efficiency-aware evaluation and reporting during inference as well as training [2,10,21, 25,26,46,48]. Tooling has emerged to help practitioners measure and report runtime impacts, e.g., CarbonTracker [3] for training runs and epoch-level estimates, MLCO2/CodeCarbon-style approaches [32] for standardized emissions estimation from energy data and regional grids, and MELODI [26] specialized for measuring energy consumption of LLM inference. A complementary line of work targets algorithmic and systems optimizations to reduce inference cost: low-bit quantization and efficient kernels (e.g., GPTQ [16] and LLM.int8 [13]), context and routing strategies that avoid unnecessary large-model calls (e.g., FrugalGPT [7]), and faster decod- ing methods such as speculative decoding and multi-draft generation that cut token-generation latency and compute [20,33]. Hardware-aware inference optimizationsâincluding fused, I/O-aware attention kernels, paged KV-cache management, and KV-cache compression or evictionâreduce memory traffic and improve inference throughput on GPUs [12,31,67]. However, their energy benefits depend strongly on the workload, software stack, and accelerator architecture [15]. Hard- wareâsoftware co-design for edge inference can further improve energy efficiency, reinforcing the need to jointly tune models, workloads, runtimes, and hardware for sustainable deployment [23]. Finally, sustainability extends beyond carbon and electricity. Resource accounting frameworks increasingly include water footprint and thermal constraints for datacenters, with evidence that large-scale LLM training and inference can incur substantial hidden water usage depending on cooling strategies and geography [35]. Together, these strands motivate rigorous, multi-objective evaluation for accuracy and efficiency (energy,CO 2 , latency), and transparent reporting standards, especially for agentic LLM workflows where longer contexts and multi-turn exchanges can amplify resource consumption. 2.4 Gaps in the Literature While the prior sections document advances in multi-agent LLM systems within software engi- neering and the growing attention to environmental implications of AI, three major gaps remain: , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows7 âąLack of Comprehensive Empirical Evaluations Across Agentic Configurations. Most existing studies primarily evaluate their systems on isolated tasks (e.g., code generation or issue resolution), often comparing single-agent baselines to a single agentic design [18]. However, there is a lack of systematic empirical work that evaluates multiple agentic con- figurations (e.g., non-agentic, single-agent, dual-agent, multi-agent workflows) across a diverse set of software engineering tasks. This makes it difficult to generalize findings across development activities in the software lifecycle. âąMissing Joint Evaluation of Sustainability and Performance Metrics. The growing corpus on Green AI and AIâs environmental impact emphasizes energy andCO 2 metrics alongside accuracy (e.g., [10,21,26,46,48,49,52]). However, no prior work in SE-focused multi-agent LLM systems has simultaneously measured accuracy, inference latency, energy consumption, and estimated carbon emissions for different architectural configurations. This omission hinders our ability to understand the trade-offs between performance and sustainability when deploying agentic systems. âąReevaluating the Need for Agentic Complexity. Recent work such as that by Xia et al. [62] questions whether elaborate agent architectures are always necessary. The authors show that a simpler, non-agentic three-phase approach (localization, repair, patch validation) can outperform complex agents in both performance and cost metrics on SWE-bench Lite benchmarks. This suggests that agentic workflows may not always be the most efficient solution, and emphasizes the need for systematic comparison across configurations with sustainability in mind. By explicitly addressing these gaps (multi-configuration comparison across tasks, integrated performance-sustainability measurement, and validation of agent simplicity), our study fills a critical void in both the SE and AI sustainability literature. 3 Study Design This section presents the experimental setup (see Table 1 for a summary). We begin by outlining the research questions. Subsequently, we introduce the software engineering tasks and the datasets used for their evaluation. We then describe the agentic configurations examined in this study, followed by the implementation details and the models employed. Finally, we present the evaluation metrics, measurements, and the experimental protocol. 3.1 Research Questions We investigate the following Research Questions (RQs) across five software engineering tasks using four datasets, where two of the tasks (log parsing and log analysis) are performed on a shared dataset (HDFS logs). âą RQ1. How do different LLM system configurations perform across software engineering tasks in terms of accuracy? This question aims to assess the effectiveness of each configuration, ranging from non-agentic to complex multi-agent workflows, in achieving accurate task completion across various software engineering scenarios. âą RQ2. How do LLM system configurations differ in energy consumption across software engineer- ing tasks and hardware platforms? This question investigates the sustainability implications of different agent configurations by measuring their energy usage across deployment envi- ronments. âąRQ3. What is the inference latency of different agent configurations across software engineering tasks and hardware platforms? This question explores the responsiveness and real-time , Vol. 1, No. 1, Article . Publication date: October 2026. 8Astekin et al. Table 1. Overview of the empirical study design, including the software engineering tasks and datasets, agentic configurations, LLMs, hardware platforms, prompting strategies, measurement and execution protocols, statistical analyses, and their mapping to the four RQs. The experimental campaign comprises 1,800 runs from the core factorial design and 40 additional GPT-OSS runs on Server-SMU, for 1,840 runs in total. DimensionExperimental Setup Software engineering tasksCode generation, technical debt identification, code vulnerability detection, log parsing, and log analysis. DatasetsHumanEval (code generation), MLCQ (technical debt detection), PrimeVul (code vulnerability detection), and LogHubâs HDFS (log parsing and log analysis). System configurationsNon-agentic single-query models, single-agent setups, dual-agent configurations, and multi-agent workflows. Hardware platformsServer-SMU (8Ă NVIDIA H100), Workstation-SMU (RTX A2000), and Server-STF (2Ă RTX 6000 Ada). Modelsgemma3:4b, codegemma:7b-instruct, qwen3:4b-instruct, qwen2.5-coder:7b-instruct, deepseek-coder:6.7b- instruct, and gpt-oss:20b (Server-SMU only). Prompting strategiesZero-shot and few-shot prompting (3 task-specific examples per prompt). Total configurations5 tasksĂ4 agent configurationsĂ2 prompting strategiesĂ5 modelsĂ3 hardware platformsĂ 3 repetitions = 1800 experimental runs, plus 40 additional runs for gpt-oss:20b (evaluated once on Server-SMU), 1840 in total. Agent frameworkAG2 / AutoGen-style multi-agent orchestration framework. Inference backendOllama. Energy measurement toolCodeCarbon with tracking_mode="process" for process-level energy attribution. Measurement levelsCPU energy, GPU energy, and total pipeline energy (CPU + GPU + memory). Carbon estimationCodeCarbon-based CO 2 estimation using a fixed Canadian carbon-intensity assumption. Latency metricEnd-to-end inference latency measured per task instance. Accuracy metricsTask-specific metrics including pass@k, precision, recall, F1-score, and parsing accuracy. Execution protocolTemperature fixed at 0; Ollama restarted for each run; AG2 caching disabled; each experiment repeated three times where feasible, with GPT-OSS runs executed once due to resource constraints. Statistical testsFriedman tests for multi-group paired comparisons, followed by Wilcoxon signed-rank post-hoc tests and matched-pairs rank-biserial effect sizes. RQ1 designRQ1 compares accuracy across configurations, tasks, models, and hardware. RQ2 designRQ2 compares process-level energy consumption across the same experimental dimensions; CO 2 emissions are derived from energy under a fixed carbon-intensity assumption. RQ3 designRQ3 compares end-to-end inference latency across configurations, LLMs, hardware platforms, and tasks. RQ4 designMulti-objective trade-off analysis comparing accuracy, energy consumption, and latency across configu- rations, models, tasks, and hardware platforms. Reported resultsConfiguration-level means and standard deviations across available repetitions, with aggregate tables averaging over the relevant configuration groups. feasibility of different agent configurations, analyzing how architectural and hardware choices affect end-to-end inference time. âą RQ4. What trade-offs exist between performance (accuracy), energy efficiency, and latency across software engineering tasks and hardware platforms, and are there task-specific patterns that inform the design of sustainable multi-agent LLM systems? This question brings together the individual dimensions of RQ1âRQ3, exploring the multi-objective balance between per- formance, cost, and environmental impact, and deriving actionable design guidelines for practitioners. 3.2 Tasks & Datasets 3.2.1 Code Generation. We used HumanEval, a benchmark introduced by Chen et al. [8] for evaluating functional correctness in code generation with LLMs. The dataset consists of 164 handwritten Python programming problems, each including a function signature, natural language docstring, reference implementation, and several unit tests (on average 7.7 tests per problem) 1 . The problems are deliberately hand-written to avoid contamination from public sources such as GitHub repositories. HumanEval tasks cover language comprehension, reasoning, algorithmic design, and simple mathematics, making it a diverse and challenging benchmark that has been widely adopted for evaluating LLM-based code generation systems [22, 27, 34] 1 https://github.com/openai/human-eval , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows9 Table 2. Composition of the cleaned MLCQ evaluation subset by code-smell class and analysis granularity. Function-level samples include Long Method, Feature Envy, and smell-free functions; class-level samples include Blob, Data Class, and smell-free classes. âNoneâ denotes samples without a target smell, and dashes indicate that a smell category is not applicable at the corresponding granularity. After duplicate removal and ambiguity filtering, the subset contains 385 samples in total. Smell Type Function Level Class Level Total None6767134 Blob-5151 Data Class-6767 Long Method67-67 Feature Envy66-66 Total385 3.2.2 Technical Debt Detection. We used an industry-relevant code smell dataset called MLCQ (Madeyski Lewowski Code Quest) [38]. It contains nearly 15,000 code snippets in Java. Each snippet was independently reviewed by experts who were asked to indicate the presence or absence of a specific smell type. Two of these smell types are function-level (Long Method, Feature Envy). The other two are class-level (Blob, Data Class). Every code snippet that is checked for one of these smell types is labeled with one of the severity levels: None, Minor, Major, Critical. A single code snippet could be checked for multiple smells by the same or different experts. We analyzed recurring code snippets across different samples to consolidate a dataset for our evaluation. For the same code snippet with multiple reviews at either the function or class level: âąIf all reviews label the snippet with severity None, we also assign it the None label, indicating that it is not associated with any smell. âą If multiple smells are reported with severity other than None, we exclude the snippet to avoid ambiguity, since the assigned task is to identify whether any smell was present and, if so, which one. âąIf the severity for one review is None and it is Major or Critical for the other type of smell at the corresponding level, we assign its label for the associated smell. For example, if the review for Long Method indicates severity None, but the review for Feature Envy indicates severity Critical, then we assign the label Feature Envy with severity Critical to that code snippet. âą We discarded entries labeled with Minor severity, as well as those containing code snippets from inaccessible repositories, missing files, or outdated or hidden content. As a result, we obtained a dataset with 6,441 samples without repeated entries (reviews) for any code snippet associated with one of the various types of smells. However, this dataset was still subject to code snippets that are possibly associated with multiple smell types at the same time. Reviewers assessing class-level code smells focused solely on class-level smell types [38]. However, function-level smells can also exist within these classes. To avoid ambiguity and ensure that each code snippet is associated with a single smell type, we manually checked all the class-level code snippets. If a class-level code snippet contained a function-level smell, we excluded it from the dataset. For our evaluation, we selected a reduced sample of 385 code snippets as summarized in Table 2. These samples are randomly shuffled. 3.2.3Code Vulnerability Detection. We used PrimeVul, a benchmark for training and evaluating code language models (LMs) on vulnerability detection tasks [14]. Compared to other benchmarks, PrimeVul is designed to better capture the realistic challenges of detecting subtle vulnerabilities in source code. In this study, we specifically utilized the PrimeVul Pair subset, which presents a challenging benchmark as each vulnerable function is paired with a benign counterpart sharing at , Vol. 1, No. 1, Article . Publication date: October 2026. 10Astekin et al. Table 3. Datasets and evaluation-set sizes for the five software engineering tasks. Sample counts denote HumanEval programming problems, cleaned MLCQ code snippets, PrimeVul Pair functions (193 vulnerable and 193 benign), HDFS-v1 log messages for log parsing, and HDFS-v1 block-ID sessions for log analysis. Although both HDFS tasks contain 385 samples, they use different units of analysis; the log-analysis subset was stratified to preserve the original normal-to-anomalous session distribution. TaskDataset Samples Source Code GenerationHumanEval164[8] Technical DebtMLCQ385[38] Vulnerability DetectionPrimeVul Pair386[14] Log ParsingHDFS-v1385[68] Log AnalysisHDFS-v1385[68] least 80% of their string content. This design emphasizes fine-grained distinctions between secure and insecure implementations, making the task significantly more realistic and difficult. The dataset we utilize contains 193 vulnerable and 193 benign functions, resulting in a total sample size of 386. 3.2.4 Log Parsing and Analysis. For log-related tasks, we used datasets from LogHub, a large collection of system logs that are widely used for log parsing and anomaly detection research [68]. LogHub contains over 77 GB of logs collected from both real-world production systems and controlled lab environments. The collection includes both labeled and unlabeled datasets: labeled logs are annotated for specific tasks such as anomaly detection or issue identification, while unlabeled logs are suitable for general-purpose log analysis. In this study, we chose HDFS due to its widespread adoption in log analysis research. The HDFS dataset contains logs from the Hadoop Distributed File System, a widely used distributed storage system. LogHub provides three HDFS variants (v1, v2, v3). We specifically used HDFS-v1, which was generated on a 203-node cluster running benchmark workloads, with anomalies manually labeled using handcrafted rules. The logs are organized into traces according to block IDs, and each trace is labeled as normal or abnormal. For our experiments, we used 385 log messages for the log parsing task and 385 sessions of log messages for the log analysis task, where each session corresponds to the logs associated with a specific block ID 2 . To ensure that our experimental subset remained representative of the original dataset, we applied a stratified sampling strategy that preserves the original class distribution between normal and abnormal sessions for the log analysis task. In the HDFS dataset, anomalous sessions constitute approximately 3% of the data; therefore, our sampled subset maintains this ratio. While this approach improves representativeness, it also results in a highly imbalanced dataset, which we take into account in our analysis. Rationale of the task and dataset selection: We selected the five software engineering tasks to cover complementary stages of the software lifecycle: code generation represents development-time automation, technical debt detection captures maintainability assessment, vulnerability detection addresses security analysis, while log parsing and log analysis represent operations and runtime system understanding. These tasks differ in reasoning complexity, input/output structure, and evaluation criteria, making them suitable for assessing whether agentic LLM configurations provide consistent benefits across heterogeneous software engineering scenarios. For each task, we selected datasets that are widely used, publicly available, and suitable for reproducible evaluation. HumanEval was chosen for code generation because it provides executable programming problems with unit tests, enabling objective correctness measurement. MLCQ was selected for technical debt detection because it contains expert-reviewed code smell annotations at both function and class levels. PrimeVul was chosen for vulnerability detection because it provides realistic vulnerable and benign code pairs, supporting security-oriented code understanding. For 2 https://github.com/logpai/loghub , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows11 Fig. 1. Overview of the four LLM system configurations evaluated in this study, ordered by increasing agentic complexity: non-agentic (NA), a direct stateless LLM query; single-agent (SA), one autonomous agent; dual- agent (DA), two collaborating task-specific agents; and multi-agent (MA), a coordinated four-agent workflow. log-related tasks, we selected LogHub/HDFS because it offers real-world log data with established templates and anomaly labels, enabling evaluation of both log parsing and log analysis under realistic operational conditions. 3.3 System Configurations To systematically evaluate the impact of agentic collaboration on both task accuracy and energy consumption, we implemented four progressively collaborative configurations for each task, as shown in Figure 1. Table 4 provides an overview of the configurations evaluated in this study. Designing multi-agent LLM systems involves a large design space, including the number of agents, their roles, and their communication and coordination strategies. Consequently, there is no single universally optimal multi-agent architecture for all software engineering tasks. In this study, our objective is not to propose a new task-specific multi-agent design, but rather to evaluate the costâ benefit trade-offs associated with increasing levels of agentic collaboration. Figure 1 therefore illustrates a generic orchestration pattern. The concrete implementations used in our experiments adopt role configurations inspired by patterns previously reported in the literature, allowing us to evaluate representative and validated multi-agent setups rather than claiming task-optimal designs. The set of configurations is explained in detail below: âąNon-Agentic (NA): This configuration involves direct calls to the LLM without maintaining any state or exhibiting agent-like behavior. âąSingle Agent (SA): A single LLM-based agent acts autonomously to complete the task (See the top part of Figure 2). âąDual Agent (DA): This configuration includes two LLM-based agents with task-specific roles. For more structured tasks (log parsing and technical debt detection), a general ActorâCritic setup is used, where the Actor produces an initial output, and the Critic evaluates and finalizes it. For tasks requiring deeper domain reasoning, specialized roles are defined: in code generation, a Programmer who implements the solution and a Refiner who reviews and improves the Programmerâs output to produce the final response; in vulnerability detection, a Code Author who analyzes the code and explains its behavior, and a Security Analyst who reviews the Code Authorâs reasoning and the code itself to make the final vulnerability decision; in log analysis, a Preprocessor parses the log messages in the given session and an Anomaly Detector that identifies whether the session contains anomalies based on the parsed output. âąMulti-Agent (MA): Four agents participate in a coordinated workflow with task-specific roles. The configurations follow established role-based cooperation, critique-and-refinement, and decision-aggregation patterns for LLM agents [37], and are implemented using the conversa- tional orchestration mechanisms provided by AG2/AutoGen [61]. For structured tasks (log parsing and technical debt detection), the workflow consists of a User Proxy that provides the , Vol. 1, No. 1, Article . Publication date: October 2026. 12Astekin et al. Table 4. LLM system configurations evaluated in the study, ordered by increasing agentic complexity. The â# Agentsâ column counts active LLM-based agents, while the concrete DA and MA roles vary by software engineering task as described in Section 3.3. Configuration# Agents Agentic Key Characteristics Non-Agentic (NA)0NoDirect LLM query, no state Single Agent (SA)1YesAutonomous task completion Dual Agent (DA)2YesTask-specific two-agent collaboration Multi-Agent (MA)4YesTask-specific coordinated workflow input, an Actor (Generator) that produces an initial output, a Critic that evaluates it, and a Refiner that incorporates the feedback into the final response. For domain-specific tasks, we define specialized roles according to the analytical steps required by each task. For code generation, inspired by prior role-specialized generation and iterative review workflows [22], a Requirements Analyst interprets the task, a Programmer implements the solution, a Moder- ator reviews its correctness and quality, and a Review Board makes the final assessment. For vulnerability detection, we adopt the four-role mock-court configuration of VulTrial [60]: a Security Researcher identifies potential vulnerabilities, a Code Author responds to the find- ings, a Moderator summarizes the discussion, and a Review Board makes the final decision. For log analysis, we introduce a task-specific decomposition in which a User Proxy provides the input session, a Preprocessor structures the log messages, an Anomaly Detector identifies potential anomalies, and a Critic reviews the findings and makes the final decision. These configurations correspond to distinct collaboration mechanisms [57] and agent design patterns [37], each representing a different level of agentic interaction and orchestration. The NA setup involves no collaboration or state maintenance, aligning with the one-shot model querying [37] pattern. It serves as a non-agentic baseline with minimal reasoning and energy cost, providing a lower bound for performance comparison. The SA configuration corresponds to an agentic variant of the same pattern, where the LLM is encapsulated within an agent abstraction that manages prompts, execution flow, and internal state. Although NA and SA rely on a single LLM to complete the task and are therefore expected to yield comparable task performance, distinguishing them allows us to isolate and quantify the computational and energy overhead introduced by the agent abstraction itself. The DA configuration introduces competitive collaboration [57], associated with the debate- based cooperation and cross-reflection patterns [37]. This setup enhances reasoning consistency and output reliability through adversarial feedback, albeit with increased communication and energy demands. Finally, the MA configuration embodies role-based coopetition [37,57] and voting-based cooperation [37] patterns, representing collective intelligence and division of labor among agents. This design is expected to produce more refined and robust task outcomes while introducing additional coordination overhead and energy consumption. Rationale of the configuration selection: The four configurations were selected to span a representative and interpretable spectrum of agentic complexity, from a direct LLM call to a coordinated four-agent workflow, rather than to exhaustively cover all possible multi-agent designs. This progression allows us to isolate the marginal cost and benefit of each additional level of orchestration, agent abstraction overhead (NAâ>SA), adversarial feedback (SAâ>DA), and collective role-based reasoning (DAâ>MA), while keeping the experimental space tractable. The specific role assignments within DA and MA were informed by agent design patterns reported in the literature [37,60], ensuring that the configurations reflect validated coordination strategies rather than arbitrary design choices. For structured tasks with well-defined input-output formats (log parsing, technical debt detection), generic Actor-Critic and User Proxy-Actor-Critic-Refiner patterns are appropriate because the task does not require domain-specific reasoning beyond what , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows13 Fig. 2. Sequence diagrams of the evaluated LLM system configurations. The non-agentic baseline issues a direct stateless query, whereas the single-agent configuration delegates the task to one autonomous agent. The dual-agent configuration adds a second agent for critique, refinement, or complementary analysis, and the multi-agent configuration coordinates four task-specific roles through a staged workflow that produces the final decision. Agent roles are instantiated separately for each software engineering task. the prompt provides. For tasks involving nuanced code semantics or anomaly interpretation (code , Vol. 1, No. 1, Article . Publication date: October 2026. 14Astekin et al. generation, vulnerability detection, log analysis), task-specific roles were assigned to give each agent a meaningful and distinct analytical function. 3.4 Implementation Details We evaluated the models using Ollama [39], with each model configured using Ollamaâs default context length (4096). To improve reproducibility and reduce output variability, all experiments were conducted with the temperature fixed at 0. Agent workflows were implemented using the AG2 framework (formerly AutoGen) 3 , an open-source platform for building multi-agent systems. For prompt design, we adopted two strategies: zero-shot and few-shot prompting. In zero-shot prompting, the model receives only the task description, contextual information, and constraints, relying entirely on its pretrained knowledge. In few-shot prompting, the prompt additionally includes three manually selected labeled examples designed to capture diverse task-specific patterns (e.g., log event types, code smells, or code segments) to guide model behavior. These examples were not taken from the evaluation datasets to avoid data leakage. All prompt templates can be found in the replication package (https://github.com/merveast/agent-green). 3.5 Models In this study, we utilize five models from three model families (i.e., Gemma family from Google, Qwen family from Alibaba, and DeepSeek-Coder family from DeepSeek AI) as listed below. Additionally, we evaluated gpt-oss:20b exclusively on one of the hardware configurations (the Server-SMU), as its 20B parameter size exceeded the capacity of the other hardware configurations. âągemma3:4b: A compact general-purpose instruction-tuned model from Googleâs Gemma familye [55]. We include this model to represent lightweight, resource-efficient LLMs that can be deployed on constrained hardware while still supporting reasoning and code-related tasks. Gemma 3 models provide a large context window and are designed for efficient local deployment. âą codegemma:7b-instruct: A code-specialized instruction-tuned model from Googleâs CodeGemma family [54]. CodeGemma is designed specifically for programming-related tasks such as code generation, completion, and code understanding. We include it as a representative open- source code-oriented model for evaluating software engineering workflows. âąqwen3:4b-instruct: A compact instruction-tuned model from the Qwen3 family [64]. We include it to evaluate whether a smaller general-purpose model with strong instruction- following and reasoning capabilities can provide competitive results with lower energy and latency costs. âąqwen2.5-coder:7b-instruct: A code-specialized instruction-tuned model from the Qwen2.5- Coder family [24]. It is optimized for code generation, code reasoning, and code repair, making it directly relevant to the software engineering tasks considered in this study. âądeepseek-coder:6.7b-instruct: A code-specialized instruction-tuned model trained on a mixture of code and natural language [17]. We include it as another representative code- oriented model to compare against CodeGemma and Qwen2.5-Coder in terms of accuracy, latency, and energy efficiency. âą gpt-oss:20b (Server-SMU only): A larger open-weight reasoning model from OpenAIâs GPT-OSS family [43]. We include this model to assess how a substantially larger model affects the trade-offs between accuracy, inference latency, and energy consumption. Due to its parameter size and memory requirements, it was evaluated only on the Server-SMU configuration. 3 https://github.com/ag2ai/ag2 , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows15 Table 5. Hardware platforms used for experimental data collection, spanning two server-grade systems and one workstation-class system. The table reports the installed CPU, system memory, GPU configuration, and accelerator memory for each platform. GPU-memory values are given per accelerator, and the multiplicity indicates the number of installed GPUs rather than necessarily the number used concurrently by each experiment. MachineCPUMemory (GB)GPUGPU memory (GB) Server-SMUIntel Xeon Platinum 8480C20488Ă NVIDIA H10080 (HBM3)Ă8 Workstation-SMUIntel Xeon W-2223 8-core @ 3.60 GHz128NVIDIA RTX A200012 Server-STFIntel Xeon w9-3475X 36-core @ 4.8GHz5032Ă NVIDIA RTX 6000 Ada48Ă2 Rationale of the model selection: The selected models cover three complementary dimensions of the experimental design. First, they include both general-purpose instruction-tuned models and code-specialized models, allowing us to compare whether domain-specific code models provide measurable benefits for software engineering tasks. Second, they span different parameter scales, from compact 4B models to a larger 20B model, enabling us to analyze how model size affects accuracy, latency, energy consumption, and CO 2 emissions. Third, the models originate from different model families, which reduces the risk that the findings are specific to a single architecture or training pipeline. This selection, therefore, supports a balanced evaluation of model capability, deployment feasibility, and sustainability across heterogeneous LLM configurations. 3.6 Hardware Table 5 details the hardware platforms used in our data collection. The selected machines differ substantially in CPU architecture, memory capacity, GPU type, and available GPU memory, enabling us to evaluate how deployment environments influence the energy consumption, latency, and accuracy of LLM-based software engineering workflows. The Server-SMU machine represents a high-performance server configuration with substantial memory and multiple high-end GPUs. This setup enables the execution of larger models and more complex agentic workflows that require higher memory capacity and parallel computation. In particular, this machine was used for configurations that could not be executed on the other hardware platforms due to model-size and GPU-memory constraints. The Workstation-SMU machine represents a mid-range workstation environment. It provides a more moderate CPU, memory, and GPU configuration, allowing us to evaluate how LLM-based workflows perform on hardware that is more accessible than large-scale server infrastructure. This setup is useful for studying whether lightweight or moderately complex agentic configurations can be executed efficiently without requiring high-end server resources. The Server-STF machine provides a second server-grade configuration with a different CPU generation and GPU architecture. By including this platform, we can compare the behavior of the same models and workflows across heterogeneous GPU environments and analyze whether energy consumption and latency trends remain consistent across different hardware setups. Rationale of the hardware selection: The hardware platforms were selected to cover a diverse range of realistic deployment settings for LLM-based software engineering tools. This diversity allows us to examine how hardware capabilities affect the feasibility and sustainability of different LLM configurations. The inclusion of both workstation- and server-grade machines enables comparison between accessible local deployment and more powerful infrastructure. Moreover, the variation in GPU memory and computational capacity allows us to study whether larger models and complex multi-agent workflows provide sufficient accuracy gains to justify their higher resource requirements. This hardware selection therefore supports a comprehensive analysis of performanceâsustainability trade-offs across different computing environments. , Vol. 1, No. 1, Article . Publication date: October 2026. 16Astekin et al. Table 6. Energy-measurement configuration used in the experiments. CodeCarbon attributes CPU energy through Intel RAPL, GPU energy through NVIDIA NVML sampled at one-second intervals, and total pipeline energy as the combined CPU, GPU, and memory consumption recorded during each run. All values are reported in kWh usingtracking_mode="process"to estimate the energy attributable to the experimental process rather than the full machine. Measurement Level Data Source / ToolWhat It CapturesReporting Unit CPU EnergyCodeCarbon (Intel RAPL)Power draw of CPU package(s) during executionkWh (aggregated per run) GPU EnergyCodeCarbon (NVML interface)Power draw of GPU(s) at 1-second intervalskWh (aggregated per run) Total Pipeline EnergyCodeCarbon (combined metrics)Sum of CPU, GPU, and memory energy consump- tion over the run kWh 3.7 Energy Measurement We used CodeCarbon [11] as the tool for monitoring energy consumption during all experimental runs. CodeCarbon provides an end-to-end framework for collecting hardware-level power data and attributing it to a given process. It combines a power estimation model for RAM, Running Average Power Limit (RAPL) for CPU, and NVIDIA Management Library (NVML) for GPU. CodeCarbon was selected for its ease of integration and ability to provide reproducible, component-level energy estimates without requiring external sensors. All experiments were run on the hardware setups described previously. Table 6 gives an overview of our energy measurement setup. CodeCarbon also estimates carbon emissions based on the energy consumption and carbon intensity of the energy grid in the geographic location where the experiments are run. We tracked carbon emissions during experiments using the offline mode of CodeCarbon, which is designed for environments without internet access. This mode preserves the same internal computation methodology but requires specifying a country_iso_code to approximate the carbon intensity of the electricity used. Since our computing resources were distributed across multiple geographic locations, we set the country code to CAN (Canada) as a representative average to ensure consistency across platforms. Canadaâs grid carbon intensity, as estimated by CodeCarbon, is lower than the global average and substantially lower than grids powered predominantly by fossil fuels; the CO 2 figures reported here should therefore be interpreted as conservative lower-bound estimates, and comparisons across studies should account for location-specific grid intensity. Since CO 2 emissions in this case are a linear, constant-factor transformation of energy consumption, it adds no information beyond the energy measurements and are left out of the results in Section 4. We set CodeCarbonâs tracking_mode to "process," which estimates the power attributable specif- ically to the tracked process rather than the full-system ("machine") consumption. In this mode, CodeCarbon isolates the incremental resource usage (CPU via RAPL and GPU via pynvml) asso- ciated with the experiment, rather than reporting total node-level power draw. As a result, the reported energy does not correspond to the full systemâs idle or baseline power consumption (e.g., the substantial idle draw of an 8ĂH100 server). Instead, it reflects the estimated share of power usage attributable to the active processes under measurement. Consequently, no explicit subtraction of an idle baseline was performed, since the methodology inherently aims to exclude non-utilized resources from the accounting. While this may underestimate total node-level energy, it provides a consistent basis for comparing experiments. 3.7.1 Measurement Levels. Energy consumption was recorded at three levels: âąCPU-level energy: Captured using CodeCarbonâs integration with Intel RAPL counters to monitor CPU package power. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows17 Table 7. Task-specific effectiveness metrics and corresponding evaluation datasets. Higher values indicate better performance for all reported metrics. TaskDatasetAccuracy Metric(s) Code GenerationHumanEval [8]Pass@k Technical Debt IdentificationMLCQ Dataset [38]Accuracy, Macro F1, Per-Class F1 Code Vulnerability DetectionPrimeVul [14]Accuracy, Precision, Recall, F1-score Log ParsingLogHub [68] Parsing Accuracy (Exact Match), Edit Similarity, Longest Common Subse- quence Similarity Log AnalysisLogHub [68]Accuracy, Precision, Recall, F1-score âąGPU-level energy: Measured through CodeCarbonâs NVIDIA NVML interface to track GPU power draw over time. âąTotal pipeline energy: Combined CPU, GPU, and memory measurements to reflect the overall energy used during model loading, inference, and agentic coordination. By capturing both component-level and total energy use, we were able to analyze the relative contributions of CPU-bound and GPU-bound computation across different configurations. We additionally recorded execution duration from CodeCarbon and used this value as the end-to-end inference latency metric in subsequent analyses. 3.7.2Reporting. For each configuration-task pair, we ran three independent trials under identical conditions. Reported results include the mean energy consumption with standard deviation values to account for run-to-run variability. This ensures robust and reproducible sustainability measurements across all experimental scenarios. 3.8 Latency & Accuracy Measurement In addition to energy metrics, we evaluated each configuration on latency and task-specific accuracy to capture performance and user-perceived responsiveness. Together, these metrics provide a holistic view of the trade-offs between efficiency, quality, and sustainability. 3.8.1 Latency Measurement. Latency was measured as the end-to-end response time for each experimental run, from the moment a prompt was issued to the system until the final output was returned. For multi-agent configurations, latency included: âą Model initialization time (model loading from disk into memory), âąAgent orchestration overhead (communication between agents, planning steps, and tool invocation), âą Model inference time (token generation time for each agent involved). We instrumented the execution pipeline to log start and end timestamps for each phase, allowing us to report both the total end-to-end latency and its internal breakdown (e.g., percentage of time spent in model loading vs. reasoning vs. tool use). Latency was measured in seconds, and results were averaged across three independent runs per configuration to mitigate transient fluctuations. 3.8.2Accuracy Measurement. Accuracy was assessed using task-specific metrics appropriate to each software engineering task. The five tasks produce different output structures (generated code, extracted templates, and class labels). Therefore, we use the term accuracy as an umbrella term for task performance in the results, while reporting the metric that is most appropriate for each task. Table 7 summarizes the tasks, datasets, and corresponding accuracy evaluation metrics used in the experiments. , Vol. 1, No. 1, Article . Publication date: October 2026. 18Astekin et al. Code Generation. For code generation, performance was measured using Pass@k, which esti- mates the probability that at least one of the top-í generated solutions passes all unit tests [8]. In our experiments, we set í= 1 since only a single solution is requested from the LLMs. Log Parsing. Log parsing is evaluated as a structured prediction task, where the goal is to extract the correct log template from each raw log message. We report three complementary metrics: âąParsing Accuracy (Exact Match): Measures the proportion of predicted templates that exactly match the ground-truth template, with no token deviations [19]. âąEdit Similarity (ES): A normalized edit-distance metric ranging from 0 to 1 that quantifies token-level similarity between predicted and ground-truth templates, where higher values indicate fewer edits [5]. ES captures partial correctness when templates are not exact matches. âąLongest Common Subsequence (LCS) Similarity: Measures normalized structural simi- larity based on the longest shared token subsequence between predicted and ground-truth templates, making it less sensitive to local token variations than ES [5]. Classification Tasks. Technical debt identification, code vulnerability detection, and log analysis are treated as classification tasks. For these tasks, performance is described using standard confusion matrix components: True Positives (TP) are correctly identified positive instances, True Negatives (TN) are correctly identified negative instances, False Positives (FP) correspond to false alarms, and False Negatives (FN) represent missed positive instances. Based on these quantities, we compute: âą Accuracy: The proportion of correct predictions out of the total number of predictions. Accuracy= íí +íí íí +íí + íčí + íčí âą Precision: The proportion of correctly identified positive instances out of all predicted positives. Precision= íí íí + íčí âą Recall: The proportion of correctly identified positive instances out of all actual positives. Recall= íí íí + íčí âą F1-Score: The harmonic mean of precision and recall, providing a balanced measure. F1= 2Ă PrecisionĂ Recall Precision+ Recall For technical debt identification, which is formulated as a multi-class code smell classification task with the classes No Smell, Blob, Data Class, Feature Envy, and Long Method, we additionally report: âąMacro F1: Weights all classes equally and is therefore more informative than plain accuracy when comparing overall code smell detection capability across classes. F1 macro = 1 |C| âïž íâC F1 í whereC is the set of technical debt classes. âąPer-Class F1: One-vs-rest F1 scores for each code smell class, reported to identify which technical debt categories are comparatively easier or more challenging for each model to detect. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows19 3.9 Variables In our experiment, we analyzed the energy efficiency, inference performance, and output accuracy of different LLM system configurations across multiple software engineering tasks, LLM models, and hardware platforms. These aspects are influenced by various factors, which can be categorized as independent and dependent variables. 3.9.1Independent Variables (Factors We Manipulate). These are the variables we control or modify to observe their effects on the dependent variables: (1)Agent configuration â The architectural setup of the LLM system, including Non-Agentic (NA), Single-Agent (SA), Dual-Agent (DA), and Multi-Agent (MA) workflows. (2)Software engineering task â The specific task being performed: code generation, technical debt identification, code vulnerability detection, log parsing, or log analysis. (3) LLM model - The underlying large language model, varying across family, size, and special- ization (general-purpose vs. code-oriented models). (4) Hardware platform â The execution environment: Server-SMU, Workstation-SMU, and Server-STF. (5)Prompting strategy â The method of interacting with the LLM (zero-shot or few-shot prompting). 3.9.2Dependent Variables (Measured Outcomes). These are the variables we observe and measure to evaluate the impact of the independent variables: (1)Energy Consumption â Process-level energy consumed during execution, measured in Joules or kWh. (2) Inference latency â End-to-end execution time from task submission to final response generation, including model inference and agent coordination overhead. (3)Output Accuracy â Task-specific correctness metrics including Pass@1, parsing accuracy, edit similarity, LCS similarity, accuracy, precision, recall, F1-score, Macro F1, and per-class F1, depending on the task. By analyzing the impact of independent variables, we aimed to gain insights into how different design choices in LLM-based system configurations, task types, model selection, hardware platforms, and prompting strategies influence the trade-offs between accuracy, efficiency, and sustainability in software engineering applications. 3.10 Experiment Execution To ensure that our measurements of energy consumption, latency, and accuracy were both con- sistent and reproducible, we implemented a strict and repeatable experiment execution protocol. Each experimental trial followed the same initialization, execution, and termination sequence to minimize uncontrolled variability and prevent cross-run interference. Figure 3 illustrates the overall experimental workflow. âą Model Initialization Control: For each experimental run, the Ollama server was explicitly started immediately before execution and terminated immediately afterward. This ensured that each run began with a cold start, preventing the reuse of a model already loaded into memory. This step is essential because keeping a model resident in memory can artificially reduce both loading time and initial energy draw, thus biasing the measurements. By forcing the model to be loaded fresh for every run, we accurately captured the true energy cost and latency associated with model initialization. Additionally, Ollamaâs keep_alive parameter was set toâ1 to keep the model loaded in memory throughout the duration of each run. , Vol. 1, No. 1, Article . Publication date: October 2026. 20Astekin et al. 1. Select condition Task, dataset, agent configuration, model, prompt, hardware 2. Prepare clean run Start Ollama; load model disable AG2 cache reset monitors/timers 3. Execute workflow NA, SA, DA, or MA agents process the same input 4. Record outcomes Latency, tokens, energy, CO 2 , accuracy Measured interval: timestamps and CodeCarbon cover model loading, inference, and orchestration Repeat three independent trials for each configurationâtask pair; report mean and standard deviation. Fig. 3. Overview of experimental workflow showing model initialization, execution, and measurement phases This prevented mid-run model eviction between the sequential LLM calls, avoiding spurious latency and energy overhead from unintended reloads. âąElimination of Agent-Level Caching: Within the AG2 multi-agent framework, all forms of agent-level caching were disabled. This prevented agents from retrieving previously stored intermediate results or responses, ensuring that each prompt and sub-task was processed from scratch. Disabling caching eliminates potential biases in accuracy measurement (since cached answers might artificially inflate success rates) and ensures that computation and communication energy costs reflect actual agent reasoning and tool usage. âąIsolated Measurement Environment: Each run was executed in an isolated environment without concurrent workloads. This minimized background resource contention, such as CPU scheduling delays or GPU utilization spikes, that could otherwise affect latency and energy measurements. System resource monitoring tools were reset prior to each run to prevent accumulation of previous statistics. âąRepetition for Statistical Robustness: To account for run-to-run variability (e.g., minor fluctuations in system performance, hardware temperature, or network conditions), each configurationâtask pair was executed three independent times. The reported results are the arithmetic mean of these repetitions. In addition, standard deviation values were calculated to assess stability and reliability. GPT-OSS configurations were executed with a single repetition only due to substantially higher computational requirements. âąConsistent Input and Execution Parameters: All prompts, datasets, and execution pa- rameters were kept identical across repetitions for a given experimental condition. The temperature was fixed at 0 for all experiments to reduce output variability. Random seeds were fixed where applicable (e.g., in stochastic decoding strategies such as sampling) to reduce nondeterministic effects while still preserving the typical behavior of the LLM. By adhering to this controlled procedure, we ensured that observed differences in energy con- sumption, latency, or accuracy could be confidently attributed to changes in the independent variables (e.g., system configuration, LLM model, hardware platform) rather than to measurement noise or uncontrolled environmental factors. 3.11 Data Analysis Data Preparation. Our raw data includes energy consumption, processing time, and task-specific accuracy metrics per inference operation. The data is recorded in spreadsheet format, organized by software engineering task, with one sheet per task. Each row corresponds to one experimental run and records the configuration parameters (agent design, LLM model, hardware platform, prompting strategy, and repetition round) alongside the measured outcomes: energy consumption in kWh, , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows21 compute time in seconds, and the task-specific accuracy metric. The full experiment produced a total of 1,840 individual runs across all five tasks, resulting from the cross-product of the independent variables with three repetitions per configuration (except for GPT-OSS configurations, which were run once due to resource constraints). Data Post-processing. The first step involves normalizing and cleaning the raw records. Column values for agent design, model name, hardware platform, and prompt type are standardized to canonical forms (e.g., consistent casing, whitespace trimming, and removal of timestamps from model identifiers). The cleaned run-level data is then aggregated into configuration-level summaries by grouping on the five configuration dimensions (task, agent, model, hardware, prompt) and computing the mean and standard deviation of each dependent variable across the repeated runs. This yields 640 unique configuration-level records, each summarizing up to three replicate runs. Visualization. We employed three visualization techniques to represent our analysis: (1)Box plots: To visualize the distribution of energy consumption metrics across different models and agentic configurations. Each box plot depicts the median (horizontal line splitting the main box), interquartile range (IQR, main box), and whiskers extending to 1.5 times the IQR, effectively summarizing central tendency and variability. (2) Scatter plots: To examine relationships between energy consumption and accuracy, with points representing different model configurations. Data points are colored and styled ac- cording to which model and dataset they represent. For certain figures, we plot the fitted regression lines, indicating trend lines. (3)Pareto frontier curves: Connecting Pareto-optimal configurations to visualize the trade-off boundary between energy efficiency and accuracy. 3.12 Replication Package To support independent replication and extension of this study, we provide a replication package at https://github.com/merveast/agent-green. The package contains the source code used to implement the non-agentic, single-agent, dual-agent, and multi-agent configurations; the task-specific agent roles and interaction workflows; and the zero-shot and few-shot prompt templates. It also includes the scripts used to prepare the evaluation subsets from HumanEval, MLCQ, PrimeVul Pair, and HDFS- v1, together with the sampled instance identifiers and preprocessing rules needed to reconstruct the experimental datasets. The package further provides model and Ollama configurations, AG2 workflow definitions, CodeCarbon instrumentation, hardware and software environment specifications, and scripts for executing the experiments and collecting task outputs, latency, and component-level energy measurements. Aggregated per-configuration results are provided in a structured spreadsheet, together with the scripts used to calculate task-specific accuracy metrics, descriptive statistics, statistical tests, effect sizes, and Pareto fronts are included to enable verification of the reported findings. For models or datasets that cannot be redistributed directly because of licensing or size restrictions, the package provides acquisition instructions, version identifiers, and preprocessing scripts rather than copies of the original artifacts. A detailed README documents environment setup, dependency installation, experiment execu- tion, expected directory structure, and result reproduction. Machine-readable configuration files record the model, prompt strategy, agent configuration, task, hardware platform, and repetition associated with each run. Because absolute energy and latency measurements depend on the deploy- ment environment, exact numerical reproduction may require comparable hardware; nevertheless, the supplied artifacts support replication of the experimental procedure and evaluation pipeline on alternative platforms. , Vol. 1, No. 1, Article . Publication date: October 2026. 22Astekin et al. NASADAMA 0 20 40 60 80 100 Accuracy (%) 96.8 94.2 96.0 94.3 Code Generation NASADAMA 0 20 40 60 42.8 42.3 41.1 40.0 Technical Debt Detection NASADAMA 0 20 40 60 49.7 49.2 35.6 55.4 Vulnerability Detection NASADAMA 0 20 40 60 34.5 31.2 27.6 26.1 Log Parsing NASADAMA 0 2 4 6 8 10 7.5 6.2 5.5 6.1 Log Analysis Fig. 4. RQ1 task-specific effectiveness across LLM system configurations. Bars and error bars report the mean±one standard deviation across modelâprompt combinations after first averaging over the available hardware platforms within each combination. Performance is measured using Pass@1 for code generation, Macro F1 for technical debt detection, F1-score for vulnerability detection and log analysis, and exact-match parsing accuracy for log parsing. NA denotes the non-agentic direct-query baseline, SA the single-agent configuration, DA the dual-agent configuration, and MA the four-agent multi-agent configuration. 4 Experiment and Results This section discusses the results of our case studies, addressing each RQ in turn. Each subsection reports the effect of the four main factors (agent configuration, hardware platform, prompt strategy, and LLM model) in the same order, followed where applicable by task-specific details. Scope of cross-hardware aggregates. The GPT-OSS model was evaluated on a single hardware platform (Server-SMU) and without repeated runs due to its 20-billion-parameter footprint. To avoid biasing cross-hardware comparisons, all hardware-balanced aggregates exclude GPT-OSS unless explicitly noted. Model-level aggregates that span all hardware platforms include GPT-OSS only on Server-SMU; we flag the resulting interpretation caveats where they apply. 4.1 Effectiveness of Configurations (RQ1) RQ1 investigates how the LLM system configurations perform across the five software engineering tasks, by reporting task-specific performance metrics for each task: Pass@1 for code generation, Macro F1 for technical debt detection, F1-score for vulnerability detection and log analysis, and parsing accuracy for log parsing. 4.1.1 Effect of agentic configuration. Figure 4 summarizes the average performance of the four configurations. The main pattern is that increasing agentic complexity does not systematically improve accuracy, but the vulnerability-detection results provide an important exception. The non- agentic baseline (NA) obtains the highest mean score for code generation, technical debt detection, log parsing, and log analysis, whereas MA obtains the highest mean score for vulnerability detection. For code generation, all configurations perform strongly, with Pass@1 ranging from 94.2% for SA to 96.8% for NA. This suggests that, for the HumanEval-style tasks considered here, additional agentic orchestration is not necessary to achieve high functional correctness. Technical debt detection shows a similarly flat pattern across configurations, with Macro F1 between 40.0% and 42.8%. The low absolute scores indicate that the task is challenging, but the differences between agentic designs are small. Vulnerability detection is the clearest exception to the overall pattern. MA obtains the highest average F1-score (55.4%), followed by NA (49.7%) and SA (49.2%), while DA remains substantially lower (35.6%). Thus, the multi-agent workflow appears beneficial for this task, but the benefit is specific to MA rather than to agentic complexity in general. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows23 Table 8. RQ1 pairwise Wilcoxon signed-rank comparisons among the non-agentic (NA), single-agent (SA), dual- agent (DA), and four-agent multi-agent (MA) configurations for vulnerability-detection F1-score, following the significant omnibus Friedman test. Repeated runs are first averaged within each configuration, and hardware is then averaged within each taskâagentâmodelâprompt combination, yielding paired hardware-balanced modelâprompt observations. Statistical significance is assessed atí<0.05. Matched-pairs rank-biserial correlation (í) reports effect size and direction: positive values favor Configuration 1, negative values favor Configuration 2, and larger absolute values indicate stronger differences. Configuration 1Configuration 2í -valueSignificantRank-biserialí DAMA0.0122Yes-0.795 DANA0.0122Yes-0.795 DASA0.0161Yes-0.769 MANA0.339No+0.333 MASA0.339No+0.333 NASA0.677No+0.154 For the two log tasks, the results are weaker overall and more variable. Log parsing reaches 34.5% parsing accuracy for NA, followed by 31.2% for SA, 27.6% for DA, and 26.1% for MA. Log analysis has low F1-scores for all configurations, ranging from 5.5% to 7.5%. These results indicate that the log tasks remain difficult for the evaluated models and prompts, and that adding agents does not by itself solve the underlying extraction and classification challenges. For log analysis in particular, the uniformly low F1-scores are primarily attributable to severe class imbalance: anomalous sessions constitute approximately 3% of the HDFS dataset, and our experimental subset preserves this ratio by design to maintain representativeness. The reported F1-scores therefore reflect the difficulty of detecting rare anomalies rather than a general failure of the configurations, and absolute scores for log analysis should not be compared directly with those of the other tasks. Statistical analysis. Table 10 shows which design factors show statistically significant effects across tasks for both energy consumption and task performance, using Friedman and Wilcoxon signed-rank tests. To align the statistical tests with the RQ1 accuracy aggregation, repeated runs are first averaged within each configuration, and hardware is then averaged within each taskâ agentâmodelâprompt combination. Thus, each observation in the accuracy test corresponds to one paired, hardware-balanced modelâprompt cell rather than to an individual repeated run. Under this aggregation, agentic configuration has a statistically significant effect on accuracy only for vulnerability detection (í=0.0162). The effect is not significant for code generation (í=0.830), technical debt detection (í=0.0752), log parsing (í=0.666), or log analysis (í=0.284). By contrast, energy consumption differs significantly by agentic configuration for all tasks. The vulnerability-specific post-hoc tests in Table 8 show that DA differs significantly from MA, NA, and SA on accuracy, while the differences among MA, NA, and SA are not significant under the paired hardware-balanced test. The corresponding energy comparison in Table 9 shows significant differences between DA and MA and between MA and NA for vulnerability detection. Overall, the statistical analysis in Table 10 supports the pattern that additional agents do not provide a general accuracy benefit, but they do systematically affect resource consumption. The choice of model shows significant difference on energy consumption across all tasks, while it only shows significance for the accuracy on code generation, technical debt detection and vulnerability detection. Prompt type is not consistently significant across the tasks for any metric. Hardware shows statistical significance for both accuracy and energy consumption across all tasks, except the accuracy of vulnerability detection and log analysis. 4.1.2Effect of hardware platform. Although hardware is not expected to change task accuracy in the same direct way as model, prompt, or agentic configuration, platform-specific runtime differences , Vol. 1, No. 1, Article . Publication date: October 2026. 24Astekin et al. Table 9. RQ2 pairwise Wilcoxon signed-rank comparisons of total pipeline energy consumption among the non-agentic (NA), single-agent (SA), dual-agent (DA), and four-agent multi-agent (MA) configurations for vulnerability detection, following the significant omnibus Friedman test. Repeated runs are first averaged within each configuration, and hardware is then averaged within each agentâmodelâprompt combination, yielding paired hardware-balanced modelâprompt observations. Statistical significance is assessed atí<0.05. Matched-pairs rank-biserial correlation (í) reports effect size and direction: positive values indicate higher energy consumption for Configuration 1, negative values indicate higher consumption for Configuration 2, and larger absolute values indicate stronger differences. Configuration 1Configuration 2í -valueSignificantRank-biserialí DAMA0.0049Yes-0.872 DANA0.850No+0.077 DASA0.910No+0.051 MANA0.0210Yes+0.744 MASA0.0771No+0.590 NASA0.424No+0.282 Srv-SMUSrv-STFWS-SMU CodeGen (P@1) TechDebt (Macro F1) Vuln (F1) LogParse (Parse Acc.) LogAnal (F1) +1.6-0.7-0.9 +0.3-0.2-0.1 +0.9-1.1+0.2 -0.4+0.4-0.0 +0.3+0.1-0.4 101 Difference from task mean (p) Fig. 5. RQ1 hardware effects on task-specific performance using the hardware-balanced subset of configura- tions. Each cell reports the deviation, in percentage points, of a hardware platformâs mean score from the mean across the three platforms for the corresponding task; positive and negative values indicate performance above and below the task mean, respectively. Srv-SMU, Srv-STF, and WS-SMU denote Server-SMU, Server-STF, and Workstation-SMU. Aggregates include only the agent configurations, models, and prompting strategies available on all three platforms. and potential non-deterministic factors can still affect observed results. To avoid confounding hardware with model availability, Table 11 and Figure 5 restrict the comparison to configurations that are available on all three hardware platforms, meaning that we exclude the GPT-OSS model. Under this balanced comparison, hardware differences are comparatively small. The largest task- level spreads are 2.5 percentage points for code generation and 2.0 percentage points for vulnerability detection, while technical debt detection, log parsing, and log analysis differ by less than one percentage point across hardware. Thus, hardware is not a major driver of RQ1 effectiveness compared with prompt strategy, model choice, and agentic configuration. 4.1.3Effect of prompt strategy. Table 12 shows that prompt strategy has a strongly task-dependent effect. Zero-shot prompting performs best for code generation, technical debt detection, and log analysis, while few-shot prompting performs marginally better for vulnerability detection and , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows25 Table 10. Factors showing paired-test differences across tasks. Multi-level factors use Friedman tests, while two-level prompt comparisons use Wilcoxon signed-rank tests. Agent, model, and prompt tests use hardware- balanced observations; hardware tests use configuration means because hardware cannot be tested after it has been averaged out. Significant differences (í< 0.05) are highlighted in bold. TaskMetricFactorTestí -value Code Generation Pass@1Agent DesignFriedman0.830 Pass@1ModelFriedman0.0018 Pass@1Prompt Type Wilcoxon 8.65Ă 10 â4 Pass@1HardwareFriedman0.0387 EnergyAgent Design Friedman 3.50Ă 10 â5 EnergyModelFriedman0.0077 EnergyPrompt TypeWilcoxon0.345 EnergyHardwareFriedman 3.52Ă 10 â13 Technical Debt Detection Macro F1Agent DesignFriedman0.0752 Macro F1ModelFriedman 4.03Ă 10 â7 Macro F1Prompt TypeWilcoxon0.603 Macro F1HardwareFriedman0.0360 EnergyAgent Design Friedman 1.98Ă 10 â7 EnergyModelFriedman 3.64Ă 10 â6 EnergyPrompt Type Wilcoxon0.0072 EnergyHardwareFriedman 3.77Ă 10 â10 Vulnerability Detection F1-scoreAgent Design Friedman0.0162 F1-scoreModelFriedman0.0255 F1-scorePrompt TypeWilcoxon0.565 F1-scoreHardwareFriedman0.509 EnergyAgent Design Friedman0.0213 EnergyModelFriedman 1.97Ă 10 â5 EnergyPrompt TypeWilcoxon0.0894 EnergyHardwareFriedman 7.92Ă 10 â17 Log Parsing Parsing accuracyAgent DesignFriedman0.666 Parsing accuracyModelFriedman0.0578 Parsing accuracy Prompt Type Wilcoxon 3.58Ă 10 â7 Parsing accuracy HardwareFriedman0.0124 EnergyAgent Design Friedman 5.49Ă 10 â7 EnergyModelFriedman 1.31Ă 10 â4 EnergyPrompt TypeWilcoxon0.264 EnergyHardwareFriedman 4.52Ă 10 â13 Log Analysis F1-scoreAgent DesignFriedman0.284 F1-scoreModelFriedman0.243 F1-scorePrompt TypeWilcoxon0.277 F1-scoreHardwareFriedman0.251 EnergyAgent Design Friedman 7.35Ă 10 â7 EnergyModelFriedman 2.10Ă 10 â4 EnergyPrompt Type Wilcoxon0.0039 EnergyHardwareFriedman 1.01Ă 10 â13 dramatically better for log parsing. The log parsing result is the task where prompt strategy has the largest effect: few-shot prompting increases parsing accuracy from 3.4% to 54.3%. This suggests that, for output-format-sensitive tasks, concrete examples are much more valuable than additional agentic orchestration. By contrast, code generation is better in the zero-shot setting (97.3% versus 92.7%), indicating that examples can sometimes constrain or distract the model rather than help. 4.1.4Effect of LLM model. Model choice also has a larger and more task-specific effect than agentic configuration. As shown in Table 13, no single model is best across all tasks. G3-4B obtains the highest Pass@1 for code generation (99.8%), GPT-OSS obtains the highest Macro F1 for technical debt detection (58.6%), Q3-4B performs best for vulnerability detection (57.8%), and Q2.5C-7B performs best for both log parsing (42.2%) and log analysis (8.5%). These differences suggest that taskâmodel matching is more important than simply increasing the number of agents. The results , Vol. 1, No. 1, Article . Publication date: October 2026. 26Astekin et al. Table 11. RQ1 task-specific effectiveness across hardware platforms on the hardware-balanced subset. Values report mean±standard deviation over the agent configurations, models, and prompting strategies available on all three platforms; GPT-OSS is excluded. TaskServer-SMUServer-STFWorkstation-SMU mean± stdmean± stdmean± std Code Generation (Pass@1)96.4± 4.794.1± 15.993.9± 10.9 Technical Debt Detection (Macro F1) 38.4± 11.738.0± 11.638.1± 11.5 Vulnerability Detection (F1-score)46.5± 17.144.5± 18.845.9± 17.2 Log Parsing (Parsing Accuracy)27.9± 31.2 28.6± 31.928.2± 31.6 Log Analysis (F1-score)6.8± 3.16.6± 3.06.1± 2.5 Table 12. RQ1 effect of prompting strategy on task-specific effectiveness. Values report mean±standard deviation for zero-shot and three-example few-shot prompting, averaged over agent configurations, models, and hardware platforms. TaskZero-shotFew-shot mean± stdmean± std Code Generation (Pass@1)97.3± 6.492.7± 14.0 Technical Debt Detection (Macro F1) 40.2± 12.238.7± 12.4 Vulnerability Detection (F1-score)45.8± 18.0 46.8± 16.7 Log Parsing (Parsing Accuracy)3.4± 6.954.3± 24.1 Log Analysis (F1-score)7.2± 3.25.7± 2.2 Table 13. RQ1 task-specific effectiveness by LLM model. Values report mean±standard deviation across agent configurations, prompting strategies, and available hardware platforms. Performance is measured using Pass@1 for code generation, Macro F1 for technical debt detection, F1-score for vulnerability detection and log analysis, and exact-match parsing accuracy for log parsing. GPT-OSS results include only Server-SMU. TaskCG-7BDS-6.7BG3-4BGPT-OSSQ2.5C-7BQ3-4B mean± stdmean± stdmean± stdmean± stdmean± stdmean± std Code Generation (Pass@1)95.6± 4.788.8± 19.8 99.8± 0.597.9± 1.998.0± 3.091.8± 12.8 Technical Debt Detection (Macro F1)29.7± 2.923.3± 3.252.6± 2.658.6± 3.747.2± 3.837.9± 6.2 Vulnerability Detection (F1-score)35.1± 19.940.4± 18.251.1± 9.256.5± 5.443.9± 19.1 57.8± 9.8 Log Parsing (Parsing Accuracy)24.9± 30.815.1± 28.128.0± 29.138.1± 27.5 42.2± 37.330.9± 26.4 Log Analysis (F1-score)7.0± 3.75.2± 1.35.6± 0.65.1± 1.58.5± 3.46.4± 3.0 also show that the common assumption that larger models perform better than smaller ones is not always true: 4B models provide the strongest local-model accuracy for code generation and vulnerability detection, while Q2.5C-7B leads the two log tasks. Figure 6 complements the model-level table (Table 13) by showing that the effect of agentic configuration is strongly model- and task-dependent. In vulnerability detection, MA performs strongly for several models, while DA remains highly model-dependent: it performs well with Q3-4B but is much weaker for several other models. Task-specific deep dives. The remainder of this subsection looks more closely at two tasks whose accuracy metrics allow a finer-grained view than the others: technical debt detection (per smell class) and log parsing (per similarity metric). 4.1.5Per-class analysis: Technical Debt Detection. For Technical Debt Detection, Macro F1 is used as the main comparison metric because it gives equal weight to each smell class. However, the per-class results reveal important differences that are hidden by the aggregate score. Table 14 shows , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows27 NASADAMA CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B Model_Abbr 96.696.699.689.3 96.790.788.579.4 99.399.999.8100.0 97.995.799.798.5 96.097.299.699.4 94.384.988.999.2 Code Generation NASADAMA 30.330.730.227.7 22.526.723.620.6 52.952.952.352.5 63.258.754.857.6 42.846.450.049.9 45.138.735.732.0 Technical Debt Detection NASADAMA 28.634.214.862.6 49.252.123.037.1 55.853.138.157.6 52.953.455.863.7 49.147.217.661.8 62.455.064.449.6 Vulnerability Detection NASADAMA 37.036.912.912.9 40.817.41.41.0 29.733.225.323.7 33.229.644.944.5 36.637.550.544.2 30.032.630.930.3 Log Parsing NASADAMA 8.97.05.86.4 4.74.75.75.6 5.85.45.65.4 6.94.04.25.2 9.510.75.68.2 9.05.25.95.6 Log Analysis Fig. 6. RQ1 task-specific effectiveness by LLM model and system configuration. Rows denote CodeGemma-7B (CG-7B), DeepSeek-Coder-6.7B (DS-6.7B), Gemma-3-4B (G3-4B), GPT-OSS-20B (GPT-OSS), Qwen2.5-Coder- 7B (Q2.5C-7B), and Qwen3-4B (Q3-4B). Within each task panel, columns denote the non-agentic direct-query baseline (NA), single-agent (SA), dual-agent (DA), and four-agent multi-agent (MA) configurations. Each cell reports the corresponding task-specific performance metric as a percentage. Values are averaged over prompting strategies and the hardware platforms available for the corresponding model; GPT-OSS includes only Server-SMU measurements. Higher values indicate better performance. Table 14. RQ1 technical-debt-specific per-class results: overall F1 score for each smell class, together with the best agent configuration, model, and prompting strategy. NA denotes the non-agentic direct-query baseline, SA the single-agent configuration, DA the dual-agent configuration, and MA the four-agent multi-agent configuration. Values in parentheses report the corresponding per-class F1 score (%). Smell ClassMean F1StdBest AgentBest ModelBest Prompt No Smell55.814.7NA (57.4)G3-4B (69.6)Zero-shot (56.9) Blob30.020.8NA (32.8)GPT-OSS (61.2)Zero-shot (32.2) Data Class40.626.5MA (41.7)GPT-OSS (75.3)Few-shot (41.3) Feature Envy8.17.2DA (10.1)CG-7B (13.1)Zero-shot (10.1) Long Method62.722.1SA (66.1)GPT-OSS (88.7)Few-shot (63.8) that Long Method and No Smell are detected most reliably, with mean F1-scores of 62.7% and 55.8%, respectively. Data Class is more difficult (40.6%), Blob is lower still (30.0%), and Feature Envy is the main failure mode, with a mean F1-score of only 8.1%. The TD-specific breakdowns by agent, model, and prompt strategy are reported in Appendix A. The best agentic configuration varies by smell class: NA is best for No Smell and Blob, MA is best for Data Class, DA is best for Feature Envy, and SA is best for Long Method. However, these differences are small relative to the standard deviations, so they do not support a general claim that more agentic TD detection is better. Model choice is more consequential: GPT-OSS is best for Blob, Data Class, and Long Method, while G3-4B is best for No Smell and CG-7B is best for Feature Envy. Prompting also varies by class, with zero-shot performing best for No Smell, Blob, and Feature Envy, and few-shot performing best for Data Class and Long Method. Overall, the TD task illustrates why aggregate scores should be interpreted carefully: a configuration can look competitive in Macro F1 while still failing almost completely on a specific smell type. 4.1.6Metric sensitivity: Log Parsing. The main RQ1 analysis uses parsing accuracy as the primary log parsing metric because it is the strictest measure: a predicted template only counts as correct if it exactly matches the ground truth. However, this exact-match view can hide partial template , Vol. 1, No. 1, Article . Publication date: October 2026. 28Astekin et al. Table 15. RQ1 log parsing metric sensitivity: exact-match parsing accuracy and partial template similarity metrics (mean±std in %). The first four columns compare agent configurations using the main RQ1 agent aggregation; the final two columns compare prompt strategies using the main RQ1 prompt aggregation. MetricNASADAMAZero-shotFew-shot mean± stdmean± stdmean± stdmean± stdmean± stdmean± std Parsing Accuracy 34.5± 36.631.2± 34.127.6± 28.626.1± 26.63.4± 6.954.3± 24.1 Edit Similarity69.6± 25.464.0± 27.163.3± 28.765.0± 25.546.7± 16.1 80.8± 22.4 LCS Similarity90.4± 9.389.2± 10.090.5± 7.390.6± 7.984.8± 9.794.7± 3.8 quality. Table 15 therefore reports two additional similarity metrics for log parsing: edit similarity and LCS similarity. The additional metrics change the absolute interpretation of log parsing performance, but not the main configuration-level conclusion. Under exact-match parsing accuracy, NA remains the strongest agentic configuration (34.5%), followed by SA (31.2%), DA (27.6%), and MA (26.1%). With edit similarity, the scores are much higher, ranging from 63.3% to 69.6%, and LCS similarity is higher still, around 89â91% for all agentic configurations. This indicates that many incorrect templates are not arbitrary failures: they often preserve substantial token-level or subsequence structure even when they fail exact matching. At the same time, the partial-credit metrics do not reveal a hidden benefit from more complex agentic orchestration. The strongest pattern remains the prompt effect. Few-shot prompting improves all three log parsing metrics, increasing parsing accuracy from 3.4% to 54.3%, edit similarity from 46.7% to 80.8%, and LCS similarity from 84.8% to 94.7%. Thus, the broader conclusion is robust across metrics: for log parsing, concrete output examples matter more than adding agents. Parsing accuracy is therefore retained as the conservative main metric, while edit similarity and LCS similarity help explain how close the failed templates are to the expected structure. RQ1 Conclusion. More complex agentic configurations do not provide a general accuracy benefit across the evaluated software engineering tasks, but MA is a task-specific exception for vulnerability detection. NA or SA is sufficient, and often best, for most tasks, while DA remains weak for vulnerability detection and the log tasks. The strongest effectiveness drivers are task-specific: prompt examples are crucial for log parsing, model choice varies sharply by task, and technical debt detection requires class-level analysis because Feature Envy remains difficult even when aggregate Macro F1 appears moderate. Hardware has a negligible effect on accuracy, and the hardware-balanced statistical analysis supports this descriptive pattern: agentic configuration has a significant accuracy effect only for vulnerability detection. 4.2 Energy Consumption (RQ2) RQ2 investigates how the system configurations differ in energy consumption across tasks and hardware platforms. The tables report mean±standard deviation across the three repetitions of each experiment (except for the GPT-OSS model, where each experiment was only run once). 4.2.1Effect of agentic configuration. Figure 7 compares energy consumption across tasks, hardware platforms, and agentic configurations, while Table 16 summarizes the corresponding hardware-level and overall averages. The dominant pattern is that energy consumption generally increases with agentic complexity. Averaged over all tasks, models, prompts, and hardware platforms, as shown in the Overall row in Table 16, SA consumes 1.20Ăas much energy as NA, DA consumes 3.06Ăas much, and MA consumes 6.36Ăas much. This increase is especially visible for the log tasks: for , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows29 CodeGen TechDebt Vuln LogParse LogAnal NA SA DA MA 0.0660.0170.6230.0430.062 0.0680.0260.6040.0780.055 0.1350.0710.5390.5010.326 0.3730.1200.8701.2090.368 Srv-SMU CodeGen TechDebt Vuln LogParse LogAnal 0.1290.0270.3810.0240.102 0.1480.0520.5360.0370.094 0.2530.1330.5730.8800.655 0.7390.2591.0482.3291.675 Srv-STF CodeGen TechDebt Vuln LogParse LogAnal 0.0870.0110.2390.0100.063 0.0750.0200.4260.0130.067 0.1630.1300.4480.6980.460 0.2460.1970.8241.6250.538 WS-SMU 10 2 10 1 10 0 Energy (kWh, log scale) Fig. 7. RQ2 mean energy consumption across software engineering tasks, agentic configurations, and hard- ware platforms. Rows represent the non-agentic direct-query baseline (NA), single-agent (SA), dual-agent (DA), and four-agent multi-agent (MA) configurations; columns represent code generation (CodeGen), technical debt detection (TechDebt), vulnerability detection (Vuln), log parsing (LogParse), and log analysis (LogAnal). Srv-SMU, Srv-STF, and WS-SMU denote Server-SMU, Server-STF, and Workstation-SMU, respectively. Each cell reports mean total pipeline energy consumption in kWh, aggregated over the available models, prompting strategies, and repetitions for that taskâconfigurationâhardware combination. Color intensity is mapped logarithmically to accommodate the large range of energy values, whereas the cell annotations report the untransformed means. Table 16. RQ2: Average energy consumption (kWh) by hardware platform and agent configuration (mean± std). Hardware rows average the task-level means shown in Figure 7; the Overall row averages over all task, model, prompt, and hardware configurations. Lowest agent mean per row is bold. HardwareNASADAMAAvg. mean± stdmean± stdmean± stdmean± stdmean± std Server-SMU0.162± 0.0060.166± 0.0120.315± 0.0310.588± 0.0900.308± 0.048 Server-STF0.132± 0.0100.173± 0.0070.499± 0.0211.210± 1.7140.504± 0.857 Workstation-SMU 0.082± 0.0140.120± 0.0220.380± 0.0200.686± 0.1140.317± 0.059 Overall0.128± 0.0110.154± 0.0150.392± 0.0250.813± 0.9930.372± 0.497 log parsing, MA reaches 1.209 kWh on Server-SMU, 2.329 kWh on Server-STF, and 1.625 kWh on Workstation-SMU, whereas NA remains below 0.05 kWh on all three platforms. The lowest-energy configuration is usually lightweight. Across the 15 taskâhardware pairs in Figure 7, NA is the lowest-energy option in 11 cases, SA in three cases, and DA in one case; MA is never the lowest-energy configuration. The aggregate values in Table 16 confirm that the overall energy burden is concentrated in the heavier DA and MA configurations rather than being evenly distributed across all setups. In light of the results of RQ1, this is important because heavier agentic workflows rarely provide a corresponding general performance gain. Vulnerability detection is the main exception: MA gives the highest average F1-score for that task, but it also remains substantially more energy-intensive than NA and SA. DA, by contrast, remains less accurate than NA and SA while still increasing energy on most hardware platforms. While the main pattern shows that adding multiple agents typically increases energy substantially, there are exceptions: SA is slightly lower than NA for code generation on Workstation-SMU and for log analysis on both servers, while DA is lowest for vulnerability detection on Server-SMU. These , Vol. 1, No. 1, Article . Publication date: October 2026. 30Astekin et al. Server-SMUServer-STFWorkstation-SMU 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Mean energy per run (kWh) 97% 48% 58% 28% 10% 24% 31% GPUCPURAM Fig. 8. RQ2 mean process-attributed energy per run, decomposed into GPU, CPU, and RAM components for the three hardware platforms using the hardware-balanced subset of configurations (GPT-OSS excluded). Server-SMU is equipped with eight NVIDIA H100 GPUs with 80 GB HBM3 each, Server-STF with two NVIDIA RTX 6000 Ada GPUs with 48 GB each, and Workstation-SMU with one NVIDIA RTX A2000 GPU with 12 GB. Bar height reports mean total pipeline energy in kWh, while segment labels report each componentâs share of the measured total. These percentages represent CodeCarbon energy attribution and should not be interpreted as device-utilization rates. cases may arise when the agent orchestration produces shorter or more focused outputs than the unguided baseline, so that the reduced generation cost offsets the overhead of agent coordination. 4.2.2 Effect of hardware platform. Hardware affects energy consumption, but the effect depends strongly on the agentic configuration, as summarized in Table 16. For lightweight configurations, the differences between platforms are relatively small: NA ranges from 0.082 kWh on Workstation-SMU to 0.162 kWh on Server-SMU, and SA ranges from 0.120 kWh on Workstation-SMU to 0.173 kWh on Server-STF. For heavier configurations, the hardware gap widens. MA averages 0.588 kWh on Server-SMU, 0.686 kWh on Workstation-SMU, and 1.210 kWh on Server-STF. This suggests that hardware should not be treated as a simple constant multiplier. Hardware comparisons are most important when evaluating more complex agentic workflows; for lightweight configurations, architectural and model choices often dominate. The composition of that energy also differs markedly across platforms, as shown in Figure 8. On Server-SMU, energy is almost entirely GPU-bound (97%), with negligible CPU and RAM contributions. On Server-STF and Workstation-SMU, by contrast, CPU and RAM together account for roughly half of the measured energy (52% and 42%, respectively). The relative utilization of compute components is therefore hardware-specific rather than fixed, which reinforces that hardware cannot be modeled as a single constant multiplier on a GPU-only cost. 4.2.3 Effect of prompt strategy. Table 17 shows that prompt strategy has a task-specific energy effect. Few-shot prompting is not uniformly more expensive, despite the longer input prompt. For technical debt detection, few-shot uses less energy than zero-shot on all three hardware platforms, with Few/Zero energy ratios between 0.63 and 0.66. Log parsing shows the same pattern: few-shot consumes 24â41% less energy than zero-shot across hardware platforms. This is notable because few-shot prompting also substantially improves log parsing accuracy in RQ1, making it one of the clearest cases where better performance and lower energy align. The opposite pattern appears for code generation, where few-shot prompting increases energy on all platforms, most strongly on Server-STF and Workstation-SMU. Vulnerability detection is , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows31 Table 17. RQ2 effect of prompting strategy on total pipeline energy consumption by task and hardware platform. Values report mean±standard deviation in kWh per run for zero-shot and three-example few-shot prompting, aggregated over the available models and agent configurations. TaskHardwareZero-shotFew-shot mean± stdmean± std Code Generation Server-SMU0.157± 0.0010.164± 0.002 Server-STF0.149± 0.0030.485± 1.691 Workstation-SMU 0.089± 0.0440.196± 0.047 Technical Debt Detection Server-SMU0.071± 0.007 0.046± 0.007 Server-STF0.141± 0.008 0.094± 0.011 Workstation-SMU0.110± 0.027 0.069± 0.013 Vulnerability Detection Server-SMU0.681± 0.064 0.637± 0.060 Server-STF0.611± 0.0420.658± 0.029 Workstation-SMU 0.464± 0.0400.504± 0.019 Log Parsing Server-SMU0.520± 0.102 0.396± 0.007 Server-STF1.009± 0.053 0.626± 0.006 Workstation-SMU0.738± 0.130 0.435± 0.103 Log Analysis Server-SMU0.188± 0.0490.218± 0.050 Server-STF0.825± 2.118 0.438± 0.039 Workstation-SMU 0.273± 0.0200.291± 0.017 close to neutral, with Few/Zero ratios near 1.0. Log analysis is hardware-dependent: few-shot is lower on Server-STF but slightly higher on Server-SMU and Workstation-SMU. These mixed results indicate that prompt length alone does not determine energy consumption. The generated output length, model behavior, and agent workflow dynamics can dominate the cost of the initial prompt. 4.2.4 Effect of LLM model. Model choice has a large effect on energy consumption, often larger than the hardware differences observed for a fixed agentic configuration. Table 18 shows that the lowest-energy model varies by task and hardware, but two models dominate the low-energy cases: CG-7B is the lowest-energy model in seven of the 15 taskâhardware pairs, and Q3-4B is the lowest in six. G3-4B and Q2.5C-7B are each lowest in one case. DS-6.7B and GPT-OSS are never the lowest-energy models in this comparison. The magnitude of the model effect can be large. For log parsing, Q3-4B is consistently the lowest- energy model, while Q2.5C-7B consumes more than two orders of magnitude more energy than the best model in several taskâhardware pairs. For vulnerability detection, CG-7B is consistently the lowest-energy model across hardware platforms. However, the energy ranking should not be read as a performance recommendation by itself. In RQ1, CG-7B is not the best-performing model for any task-level model average, whereas Q2.5C-7B performs best for the two log tasks and GPT-OSS performs best for technical debt detection. These models can therefore be attractive from an effectiveness perspective even when they are not the lowest-energy choice. Conversely, Q3-4B is often energy-efficient and also performs best for vulnerability detection, making it a stronger candidate for that task than energy alone would suggest. The results also challenge the common assumption that a model with fewer parameters is generally the more energy-efficient option. Figure 9 makes this explicit by plotting mean energy against model size on Server-SMU, the only platform on which all six models were evaluated. Energy does not grow monotonically with parameter count: the two 7B models differ by a factor of 6.4Ă(CG-7B at 0.088 kWh versus Q2.5C-7B at 0.569 kWh), and CG-7B is in fact the lowest-energy model overall, below both 4B models. Only the 20B GPT-OSS model stands clearly apart at the high end, so larger size is associated with higher energy only at the extreme of the range, not within the cluster of small models. These results reinforce that sustainable configuration design cannot be reduced to choosing fewer agents, a specific hardware platform, or the lowest-energy model in isolation. Model selection is a , Vol. 1, No. 1, Article . Publication date: October 2026. 32Astekin et al. Table 18. RQ2 total pipeline energy consumption by task, LLM model, and hardware platform. Values report mean±standard deviation in kWh per run, averaged over agent configurations, prompting strategies, and available repetitions. Dashes indicate unsupported modelâhardware combinations. GPT-OSS was evaluated only once per condition on Server-SMU; therefore, its reported zero standard deviation does not indicate zero variability. TaskHardwareCG-7BDS-6.7BG3-4BGPT-OSSQ2.5C-7BQ3-4B mean± stdmean± stdmean± stdmean± stdmean± stdmean± std CodeGen Server-SMU0.045± 0.0010.230± 0.0010.062± 0.0010.438± 0.0000.061± 0.0000.127± 0.002 Server-STF0.090± 0.0020.449± 0.0020.129± 0.008â± â0.126± 0.0060.792± 2.673 Workstation-SMU0.071± 0.0000.420± 0.002 0.018± 0.005â± â0.100± 0.0000.104± 0.102 TechDebt Server-SMU0.016± 0.0000.210± 0.0010.018± 0.0100.070± 0.0000.020± 0.0000.017± 0.011 Server-STF0.034± 0.0000.443± 0.0130.047± 0.015â± â0.032± 0.001 0.032± 0.007 Workstation-SMU0.027± 0.0000.340± 0.0450.032± 0.014â± â0.021± 0.0000.027± 0.001 Vuln Server-SMU0.135± 0.0100.480± 0.0020.200± 0.1372.487± 0.0000.271± 0.0010.382± 0.021 Server-STF0.297± 0.0021.114± 0.0170.596± 0.075â± â0.498± 0.0030.667± 0.025 Workstation-SMU 0.239± 0.0010.900± 0.0030.334± 0.030â± â0.380± 0.0490.567± 0.040 LogParse Server-SMU0.018± 0.0000.364± 0.1500.109± 0.0350.551± 0.0001.689± 0.050 0.014± 0.011 Server-STF0.039± 0.0000.657± 0.0080.085± 0.014â± â3.283± 0.083 0.025± 0.001 Workstation-SMU0.029± 0.0000.485± 0.0030.027± 0.002â± â2.374± 0.263 0.017± 0.000 LogAnal Server-SMU0.142± 0.0020.294± 0.0030.197± 0.0940.347± 0.0000.148± 0.001 0.089± 0.059 Server-STF0.310± 0.0010.735± 0.0080.437± 0.037â± â0.316± 0.0011.360± 3.349 Workstation-SMU0.242± 0.0020.486± 0.0020.219± 0.025â± â0.253± 0.002 0.209± 0.032 Note. GPT-OSS was evaluated with a single run on Server-SMU only; reported std = 0.000 reflects the absence of repeated trials rather than zero variability. 468101214161820 Model size (billion parameters) 10 1 Mean energy per run (kWh) G3-4B Q3-4B DS-6.7B CG-7B Q2.5C-7B GPT-OSS Fig. 9. RQ2 relationship between model size and mean total pipeline energy consumption on Server-SMU, the only platform on which all six models were evaluated. Each point represents a modelâs mean energy per run, averaged across the five tasks, four agentic configurations, and two prompting strategies; the vertical axis is logarithmic, while the plotted values remain in kWh. Energy does not scale monotonically with parameter count: CG-7B is the lowest-energy model, whereas the similarly sized Q2.5C-7B consumes 6.4Ămore energy on average. GPT-OSS was evaluated once per condition, while the other models used three repetitions. significant energy driver, but the relevant decision criterion is energy for sufficient task performance. The model-level table is therefore best interpreted as identifying candidate low-energy models that should be cross-checked against the RQ1 performance results and the RQ4 trade-off analysis. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows33 Table 19. RQ3 end-to-end latency by task, hardware platform, and LLM system configuration. Values report mean±standard deviation in seconds, aggregated over available models, prompting strategies, and repeti- tions. NA, SA, DA, and MA denote the non-agentic, single-agent, dual-agent, and four-agent configurations, respectively; the lowest mean in each taskâhardware row is shown in bold. TaskHardwareNASADAMA mean± stdmean± stdmean± stdmean± std Code Generation Server-SMU726.7± 15.8712.0± 9.81403.3± 19.93935.5± 36.5 Server-STF955.6± 6.51023.7± 3.21655.8± 55.32220.4± 570.4 Workstation-SMU2672.6± 142.82307.0± 1519.34978.8± 561.37538.7± 2300.5 Technical Debt Detection Server-SMU199.6± 2.1289.5± 9.0713.6± 6.71175.9± 11.1 Server-STF222.2± 2.4367.9± 1.8951.4± 94.51785.8± 72.9 Workstation-SMU 345.8± 10.3629.4± 140.63983.2± 156.56086.4± 1368.3 Vulnerability Detection Server-SMU6996.8± 56.76592.2± 35.15976.9± 277.49790.7± 331.4 Server-STF2669.1± 8.83564.7± 19.23870.3± 78.57450.1± 676.0 Workstation-SMU 7330.6± 220.913053.2± 234.313734.4± 875.924687.7± 4695.7 Log Parsing Server-SMU526.7± 2.6942.7± 243.24922.4± 119.612475.5± 3518.6 Server-STF204.7± 4.1309.1± 1.95467.7± 15.114273.0± 104.0 Workstation-SMU 313.1± 3.5391.1± 3.121670.2± 566.250185.8± 7039.3 Log Analysis Server-SMU680.9± 14.5572.5± 23.33755.4± 363.84069.1± 133.4 Server-STF698.0± 155.2646.1± 70.14497.4± 82.35370.7± 625.8 Workstation-SMU2262.7± 1560.3 2071.5± 5.314104.7± 267.716512.5± 588.0 RQ2 Conclusion. Energy consumption is primarily driven by agentic complexity and model choice, with hardware effects becoming most pronounced for heavier DA and MA workflows. NA is the lowest-energy configuration in most taskâhardware pairs, and MA is never the lowest- energy option. Prompt strategy has a task-specific effect: few-shot prompting reduces energy for technical debt detection and log parsing, but increases it for code generation. When read together with RQ1, the most defensible energy-efficient designs are not simply the lowest-energy configurations, but lightweight, task-aware configurations that retain sufficient performance. 4.3 Inference Latency Comparison (RQ3) RQ3 examines the end-to-end inference latency of the evaluated configurations. Latency includes the complete time from issuing the prompt to receiving the final output, including model inference and any agent orchestration overhead. As in RQ2, the tables report mean±standard deviation across repeated runs for each configuration. 4.3.1 Effect of agentic configuration. Table 19 presents the latency for each taskâhardware pair across the four agentic configurations, while Table 20 summarizes the corresponding hardware- level and overall averages. The overall pattern closely mirrors the energy results in RQ2: latency increases with agentic complexity, but the magnitude is task-dependent. Averaged over all tasks, models, prompts, and hardware platforms, as shown in the Overall row in Table 20, SA is 1.23Ă slower than NA, DA is 3.32Ăslower, and MA is 6.07Ăslower. The most extreme case is log parsing on Workstation-SMU, where MA takes 50 186 seconds (nearly 14 hours) compared with 313 seconds for NAâa 160Ă slowdown. As in the energy analysis, exceptions exist: For code generation and log analysis, SA is sometimes faster than NA (e.g., SA takes 712 s vs. NAâs 727 s for code generation on Server-SMU, and 573 s vs. 681 s for log analysis on the same platform). These exceptions suggest that single-agent orchestration can occasionally streamline generation by producing more focused outputs. , Vol. 1, No. 1, Article . Publication date: October 2026. 34Astekin et al. Table 20. RQ3: Average inference latency (seconds) by hardware platform and agent configuration (mean± std). Hardware rows average the task-level means shown in Figure 10; the Overall row averages over all task, model, prompt, and hardware configurations. Lowest agent mean per row is bold. HardwareNASADAMAAvg. mean± stdmean± stdmean± stdmean± stdmean± std Server-SMU1826.1± 27.21821.8± 110.53354.3± 211.76289.3± 1581.73322.9± 799.9 Server-STF949.9± 69.61182.3± 32.63288.5± 70.96220.0± 487.92910.2± 249.5 Workstation-SMU 2585.0± 707.63690.4± 690.411694.3± 547.521002.2± 3977.79743.0± 2067.6 Overall1789.5± 410.82205.9± 404.15940.0± 341.410865.5± 2487.45200.2± 1288.0 CodeGen TechDebt Vuln LogParse LogAnal NA SA DA MA 7272006997527681 7122896592943573 1403714597749223755 393511769791124764069 Srv-SMU CodeGen TechDebt Vuln LogParse LogAnal 9562222669205698 10243683565309646 1656951387054684497 222017867450142735371 Srv-STF CodeGen TechDebt Vuln LogParse LogAnal 267334673313132263 2307629130533912071 49793983137342167014105 75396086246885018616513 WS-SMU 10 3 10 4 Latency (s, log scale) Fig. 10. RQ3 mean end-to-end latency across software engineering tasks, agentic configurations, and hardware platforms. Rows represent the non-agentic direct-query baseline (NA), single-agent (SA), dual-agent (DA), and four-agent multi-agent (MA) configurations; columns represent code generation (CodeGen), technical debt detection (TechDebt), vulnerability detection (Vuln), log parsing (LogParse), and log analysis (LogAnal). Srv-SMU, Srv-STF, and WS-SMU denote Server-SMU, Server-STF, and Workstation-SMU, respectively. Each cell reports mean latency in seconds, averaged over the available models, prompting strategies, and repetitions for the corresponding taskâconfigurationâhardware combination. Latency covers model loading, inference, and agent orchestration. Color intensity is mapped logarithmically to accommodate the wide range of values, whereas cell annotations report the untransformed arithmetic means. Figure 10 provides a compact taskâhardwareâagent view of the latency landscape, using the same heatmap format as the RQ2 energy analysis. The logarithmic color scale highlights the orders- of-magnitude range: the darkest cells (DA and MA for log parsing) are two orders of magnitude slower than the lightest cells (NA and SA for technical debt detection). 4.3.2Effect of hardware platform. Hardware has a large effect on latency, particularly for heavier configurations, as summarized in Table 20. For NA and SA, Server-STF is generally the fastest platform (e.g., NA averages 950 s on Server-STF vs. 1 826 s on Server-SMU and 2 585 s on Workstation- SMU). For DA and MA, the hardware gap widens substantially: MA averages 6 220 s on Server-STF but 21 002 s on Workstation-SMUâa 3.4Ădifference driven by the workstationâs slower GPU inference throughput for multi-round agent interactions. This suggests that hardware selection is especially important for complex agentic configurations, where latency differences between platforms grow from a factor ofâŒ2â3Ă for NA toâŒ3.4Ă for MA. 4.3.3 Effect of prompt strategy. Table 21 shows that prompt strategy has a task-specific effect on latency, broadly consistent with the energy patterns in RQ2. Few-shot prompting reduces latency for technical debt detection (28â37% faster across hardware) and log parsing (23â41% faster), while , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows35 Table 21. RQ3 effect of prompting strategy on end-to-end latency by task and hardware platform. Values report mean±standard deviation in seconds per run for zero-shot and three-example few-shot prompting, aggregated over the available models and agent configurations. TaskHardwareZero-shotFew-shot mean± stdmean± std Code Generation Server-SMU1736.2± 18.51652.6± 26.4 Server-STF1056.2± 16.31871.5± 404.9 Workstation-SMU 2730.7± 1358.36017.9± 1457.0 Technical Debt Detection Server-SMU707.9± 9.1481.4± 6.7 Server-STF968.2± 3.8695.5± 84.3 Workstation-SMU3383.2± 884.42139.2± 419.7 Vulnerability Detection Server-SMU7630.7± 216.17047.6± 221.2 Server-STF4247.8± 470.74529.3± 100.9 Workstation-SMU 13948.3± 3335.215454.7± 580.2 Log Parsing Server-SMU5343.4± 2494.24090.3± 76.0 Server-STF6270.8± 73.43856.5± 11.6 Workstation-SMU22814.3± 3804.2 13465.8± 3234.8 Log Analysis Server-SMU2124.1± 188.52414.9± 199.8 Server-STF2638.8± 441.92967.3± 135.7 Workstation-SMU 8468.7± 877.59007.0± 809.9 increasing it for code generation on Server-STF and Workstation-SMU. Vulnerability detection is mixed, with few-shot lower on Server-SMU but higher on Server-STF and Workstation-SMU, whereas log analysis is consistently slower under few-shot prompting. These results confirm that, as with energy, prompt length is not the dominant factor; the modelâs generation behavior in response to examples matters more. 4.3.4 Effect of LLM model. Model choice has a large and consistent effect on latency. Table 22 shows that CG-7B is the fastest model for code generation and vulnerability detection on both servers, while Q3-4B is fastest for log parsing on all platforms and for technical debt detection on the two servers. On Workstation-SMU, Q2.5C-7B is the fastest model for technical debt detection. DS-6.7B is often the slowest local model outside log parsing, reaching 23.6Ăthe fastest latency for code generation on Workstation-SMU. The largest outlier is Q2.5C-7B on log parsing: it reaches 73 544 s on Workstation-SMU (âŒ20 hours), 136.5Ăslower than Q3-4B for the same taskâhardware pair, likely due to extensive output generation for that task. These model-level latency differences are generally larger than the hardware differences, rein- forcing the RQ2 finding that model selection is a more impactful configuration lever than hardware choice. The fastest models (CG-7B, Q3-4B) tend to also be among the lowest-energy models in RQ2, because latency and energy are tightly coupled for local inference. Figure 11 visualizes the modelâhardware interaction across tasks: DS-6.7Bâs slowdown on Workstation-SMU and Q2.5C- 7Bâs pathological behavior on log parsing are clearly visible, as is the fact that GPT-OSS could only be evaluated on Server-SMU due to memory requirements (blank cells). 4.3.5 Energyâlatency relationship. A distinctive feature of RQ3 is the near-perfect correlation between energy and latency. Across all hardware-balanced configurations, the Pearson correlation between log-transformed energy and log-transformed latency isí=0.987 (í<10 â100 ), as shown in Figure 12. This strong logâlog relationship implies that configurations which are energy-expensive are also slow, and vice versa. In practical terms, this means that energy savings and latency savings are largely aligned: a configuration change that reduces one will almost always reduce the other. This tight coupling also means that the latency table can serve as a proxy for relative energy comparisons when energy measurements are unavailable. , Vol. 1, No. 1, Article . Publication date: October 2026. 36Astekin et al. Table 22. RQ3 end-to-end latency by task, LLM model, and hardware platform. Values report mean± standard deviation in seconds per run, averaged over agent configurations, prompting strategies, and available repetitions. Dashes indicate unsupported modelâhardware combinations. GPT-OSS was evaluated once per condition on Server-SMU; therefore, its zero standard deviation does not imply zero variability. TaskHardwareCG-7BDS-6.7BG3-4BGPT-OSSQ2.5C-7BQ3-4B mean± stdmean± stdmean± stdmean± stdmean± stdmean± std CodeGen Server-SMU407.3± 10.01952.5± 20.4878.4± 43.14982.7± 0.0574.3± 11.51371.0± 9.5 Server-STF627.3± 5.72760.9± 1.61085.9± 68.5â± â859.9± 15.01985.3± 636.9 Workstation-SMU2179.7± 7.812862.7± 66.7545.6± 159.3â± â3074.8± 12.83208.5± 3144.8 TechDebt Server-SMU183.9± 3.21889.6± 10.6282.5± 10.9824.4± 0.0225.2± 2.9162.4± 8.3 Server-STF318.0± 1.42765.0± 5.4495.7± 121.5â± â298.1± 2.4282.4± 54.8 Workstation-SMU856.8± 0.810446.6± 1486.61002.0± 430.6â± â656.7± 2.2844.0± 18.8 Vuln Server-SMU1162.1± 75.74051.7± 18.53595.1± 251.528162.0± 0.02418.1± 17.14646.0± 411.6 Server-STF2022.0± 6.66888.3± 7.44737.3± 737.6â± â3243.5± 11.55051.6± 187.5 Workstation-SMU 7328.7± 21.227545.9± 95.110259.9± 926.2â± â10948.8± 5127.817424.0± 1220.5 LogParse Server-SMU207.4± 8.03745.3± 3900.61394.1± 306.17039.0± 0.015743.4± 508.1171.8± 25.8 Server-STF352.9± 2.54070.2± 5.6779.3± 106.9â± â19880.0± 48.4235.8± 2.0 Workstation-SMU903.9± 0.814850.8± 95.9862.3± 59.1â± â73544.4± 7894.8538.7± 3.7 LogAnal Server-SMU1214.2± 34.32512.6± 21.93267.4± 430.63932.2± 0.01317.5± 13.61373.1± 37.1 Server-STF2019.6± 4.54547.2± 3.53537.4± 274.9â± â1969.2± 4.91942.0± 677.2 Workstation-SMU7406.1± 62.614895.9± 58.56739.4± 763.6â± â7775.4± 54.76872.4± 1723.7 Note. GPT-OSS was evaluated with a single run on Server-SMU only; reported std = 0.000 reflects the absence of repeated trials rather than zero variability. CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B Srv-SMU Srv-STF WS-SMU 407195287849835741371 627276110868601985 21801286354630753209 CodeGen CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B 1841890283824225162 3182765496298282 857104471002657844 TechDebt CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B 1162405235952816224184646 20226888473732445052 732927546102601094917424 Vuln CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B 20737451394703915743172 353407077919880236 9041485186273544539 LogParse CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B 121425133267393213181373 20204547353719691942 740614896673977756872 LogAnal 10 3 10 4 Latency (s, log scale) Fig. 11. RQ3 mean end-to-end latency by LLM model and hardware platform, shown separately for code gen- eration (CodeGen), technical debt detection (TechDebt), vulnerability detection (Vuln), log parsing (LogParse), and log analysis (LogAnal). Rows denote Server-SMU (Srv-SMU), Server-STF (Srv-STF), and Workstation- SMU (WS-SMU); columns denote CodeGemma-7B (CG-7B), DeepSeek-Coder-6.7B (DS-6.7B), Gemma-3-4B (G3-4B), GPT-OSS-20B (GPT-OSS), Qwen2.5-Coder-7B (Q2.5C-7B), and Qwen3-4B (Q3-4B). Each cell reports mean latency in seconds, averaged over agent configurations, prompting strategies, and available repetitions. Latency includes model loading, inference, and agent orchestration. Color intensity is mapped logarithmically, whereas cell annotations report the untransformed arithmetic means. Blank cells indicate unsupported modelâ hardware combinations; GPT-OSS was evaluated only on Server-SMU because of its memory requirements. RQ3 Conclusion. Latency follows the same pattern as energy: it increases with agentic com- plexity, with MA being 6.07Ă slower than NA on average and up to 160Ă slower for individual taskâhardware pairs. Hardware choice amplifies latency differences for heavier configurations, with Workstation-SMU being 3.4Ăslower than Server-STF for MA. Model choice has an even larger effect in some taskâhardware pairs, with Q2.5C-7B being up to 136.5Ăslower than Q3-4B for log parsing. Energy and latency are strongly correlated (í=0.987 in logâlog space), meaning that optimizing for one effectively optimizes for the other. As with energy, prompt strategy is task-specific: few-shot reduces latency for technical debt detection and log parsing but can increase it for code generation and log analysis. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows37 10 2 10 3 10 4 10 5 Latency (seconds) 10 2 10 1 10 0 10 1 Energy (kWh) r = 0.987 (log-log) NA SA DA MA Fig. 12. RQ3 relationship between total pipeline energy consumption and end-to-end latency across the hardware-balanced configurations. Each point represents one taskâmodelâpromptâagent configuration, with energy measured in kWh and latency in seconds. Point color denotes the non-agentic baseline (NA), single- agent (SA), dual-agent (DA), or four-agent multi-agent (MA) configuration. Both axes use logarithmic scales, and the reported Pearson correlation is computed on the log-transformed values (í=0.987,í<10 â100 ). The strong positive association indicates that configurations with longer execution times generally also consume more energy; it does not by itself establish a causal relationship. 4.4 Trade-Off Analysis and Design Guidelines (RQ4) RQ4 brings together the accuracy, energy, and latency dimensions examined in RQ1âRQ3 to identify configurations that achieve attractive multi-objective trade-offs. Since energy and latency are tightly correlated (í=0.987, as shown in RQ3), the main visual analysis uses the two-dimensional accuracyâenergy projection, while the configuration recommendations still account for latency when selecting a balanced Pareto-optimal setup. 4.4.1 Pareto front analysis. For each taskâhardware pair, we compute the Pareto front over all configuration-level means (modelĂagentĂprompt). In the two-dimensional projection, a configu- ration is Pareto-optimal if no other configuration achieves higher accuracy at equal or lower energy. Figure 13 shows this accuracyâenergy trade-off at the task level, with Pareto-optimal configurations highlighted. The corresponding Table 23 uses the three-dimensional Pareto front (accuracy, energy, and latency) to select one balanced recommendation for each taskâhardware pair. For a configurationíon the three-dimensional Pareto front of a given taskâhardware pair, we compute a balanced score score(í)= 1 3 eí(í)+ 1â eí(í) + 1â e â(í) ,(1) whereeí(í),eí(í), and e â(í)are the minâmax-normalized accuracy, energy, and latency values ofí within the Pareto front of that taskâhardware pair (each scaled to[0,1], with accuracy higher-is- better and energy/latency lower-is-better). The configuration with the highestscore(í)is reported , Vol. 1, No. 1, Article . Publication date: October 2026. 38Astekin et al. 10 2 10 1 10 0 Energy (kWh) 0 20 40 60 80 100 Accuracy (%) Code Generation 10 2 10 1 10 0 Energy (kWh) 20 30 40 50 60 Accuracy (%) Technical Debt Detection 10 1 10 0 Energy (kWh) 0 10 20 30 40 50 60 Accuracy (%) Vulnerability Detection 10 2 10 1 10 0 10 1 Energy (kWh) 0 20 40 60 80 Accuracy (%) Log Parsing 10 2 10 1 10 0 10 1 Energy (kWh) 2 4 6 8 10 12 14 Accuracy (%) Log Analysis Agent NA SA DA MA Fig. 13. RQ4 task-level trade-off between task-specific performance and total pipeline energy consumption. Each point represents one modelâagentâprompt configuration in the two-dimensional performanceâenergy projection; point color denotes the non-agentic direct-query baseline (NA), single-agent (SA), dual-agent (DA), or four-agent multi-agent (MA) configuration. Energy is reported in kWh on a logarithmic x-axis. The y-axis reports Pass@1 for code generation, Macro F1 for technical debt detection, F1-score for vulnerability detection and log analysis, and exact-match parsing accuracy for log parsing. Black circles identify Pareto-optimal configurations, for which no alternative achieves equal or higher performance with equal or lower energy, and dashed lines trace the resulting Pareto fronts. Latency is not represented in this projection but is included in the three-objective analysis reported in Table 23. in Table 23, with ties being broken by raw accuracy. This definition gives the three objectives equal weight on the Pareto front and avoids favoring the metric with the largest absolute range. Across all 15 taskâhardware pairs, the Pareto fronts contain a total of 66 configurations out of the full set of evaluated configurations for these pairs. Of the 66 Pareto-optimal configurations, 36 (55%) use NA, 23 (35%) use SA, 6 (9%) use DA, and only 1 (2%) uses MA. This confirms that lightweight configurations dominate the efficiency frontier. The few DA and MA Pareto points are task-specific: DA appears for log parsing and vulnerability detection, while MA appears once for log analysis. Thus, heavier agentic designs can occasionally survive the frontier when they improve accuracy, but they do not form a general efficient pattern. Table 23 reports the single best balanced Pareto configuration for each taskâhardware pair. The column "Pareto front size" indicates how many Pareto-optimal configurations exist for that specific taskâhardware pair. The selected configuration is chosen from the three-dimensional Pareto front by balancing normalized accuracy, energy, and latency, rather than by accuracy alone. For code generation, SA configurations achieve 100% Pass@1 on all three hardware platforms while keeping energy and latency low, making them the clear balanced Pareto winners. Across the full table, NA is the balanced-best configuration in 11 of 15 taskâhardware pairs; outside code generation, it is selected in 11 of 12 pairs. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows39 Table 23. RQ4 balanced Pareto-optimal recommendation for each taskâhardware pair, selected from the three- dimensional frontier using equally weighted normalized task performance, energy, and latency. Performance uses each taskâs primary effectiveness metric, and â# Pareto Configsâ reports the size of the corresponding frontier. TaskHardwareAgentModelPromptAccuracyEnergy (kWh)Latency (s)# Pareto Configs CodeGen Server-SMUSADS-6.7BZero-shot100.00.0208194.81 Server-STFSACG-7BZero-shot100.00.0448333.24 Workstation-SMUSAG3-4BFew-shot100.00.002063.61 LogAnal Server-SMUNAQ3-4BFew-shot14.90.0249299.25 Server-STFNACG-7BZero-shot14.10.0222201.26 Workstation-SMUNAG3-4BZero-shot4.90.0156494.86 LogParse Server-SMUNADS-6.7BFew-shot81.30.005362.13 Server-STFNADS-6.7BFew-shot82.10.0140110.64 Workstation-SMUNADS-6.7BFew-shot81.30.0086264.25 TechDebt Server-SMUNAQ3-4BZero-shot48.60.003869.47 Server-STFNAQ3-4BZero-shot48.60.009696.52 Workstation-SMUNAQ3-4BZero-shot50.20.0048156.32 Vuln Server-SMUNAG3-4BZero-shot60.90.06131126.27 Server-STFSACG-7BZero-shot46.20.0908699.87 Workstation-SMUNAG3-4BFew-shot51.70.06421982.16 Table 24. RQ4 task-specific comparison of agentic configurations across effectiveness, energy consumption, and end-to-end latency. Each cell reports the best configuration, with its corresponding mean value in parentheses; âBest Overallâ identifies the strongest combined trade-off across the three objectives. TaskBest AccuracyLowest EnergyLowest LatencyBest Overall Code GenerationNA (96.8)NA (0.0924)SA (1307.8)NA Technical Debt DetectionNA (42.8)NA (0.0181)NA (252.3)NA Vulnerability DetectionMA (55.4)NA (0.4271)NA (5748.7)NA Log ParsingNA (34.5)NA (0.0266)NA (359.3)NA Log AnalysisNA (7.5)SA (0.0712)SA (1063.9)SA 4.4.2Task-specific analysis. Table 24 complements the Pareto analysis by identifying the winning agentic configuration for each task across three objectives: highest accuracy, lowest energy, and lowest latency. The pattern is clear: NA is the best or tied-for-best agent in most cells, consistent with the Pareto front distribution reported. However, the exceptions reveal how task characteristics influence the balance between effectiveness and computational cost. For vulnerability detection, MA achieves the highest average F1-score (55.4%), whereas NA has the lowest energy consumption and latency. Although MA improves accuracy by 5.7 percentage points over NA, its substantially higher execution cost means that it appears on the balanced Pareto front for only one of the 15 taskâhardware pairs. This task illustrates that measurable accuracy improvements from additional agents do not necessarily translate into the best overall costâperformance trade-off. For code generation, SA is the balanced Pareto winner across all three hardware platforms, achieving 100% Pass@1 while incurring comparable or occasionally lower latency than NA. Unlike the other tasks, the single-agent configuration preserves task accuracy while incurring little additional execution cost, making SA competitive with the NA configuration for this workload. For technical debt detection, NA is the straightforward winner across all three objectives and all hardware platforms, with no tension between accuracy and cost. This task therefore provides the cleanest empirical support for the guideline to start non-agentic: additional orchestration neither improves accuracy nor reduces cost for this workload. For log parsing, NA is the balanced Pareto winner across all three hardware platforms. This is consistent with the broader pattern, but notable given that log parsing is also the task where prompt strategy has the largest effect: few-shot prompting raises , Vol. 1, No. 1, Article . Publication date: October 2026. 40Astekin et al. parsing accuracy from 3.4% to 54.3%, making it a more impactful lever than agentic configuration for this workload. For log analysis, SA is the most energy- and latency-efficient configuration, whereas NA achieves the highest average F1-score. Given the uniformly low absolute F1-scores for this task (5.5â7.5%), the practical differences between configurations are small, and the choice between NA and SA is unlikely to affect overall system performance materially. We synthesize these per-task patterns into concrete design guidelines in Section 5. RQ4 Conclusion. Lightweight configurations dominate the efficiency frontier: 55% of Pareto- optimal configurations use NA and 35% use SA, while MA accounts for only 2%. MA is therefore not supported as a default design choice once energy and latency are considered alongside accuracy, even though it improves average vulnerability-detection accuracy. The most effective design lever is not agentic complexity but task-specific model and prompt selection. These findings translate into a clear guideline: start with the simplest configuration (NA), move to heavier workflows only when they provide a measured task-specific benefit, match the model to the task, and use prompt examples where the task demands output structure. 5 Discussion This section synthesizes the findings of RQ1âRQ4 across tasks, configurations, models, and hardware platforms, and turns them into actionable design guidelines for sustainable LLM-based developer tools. Finally, we discuss the practical implications for tool builders and researchers. 5.1 Insights across RQs Four patterns emerge consistently across the four research questions. Together they reframe the role of each design lever in agentic LLM systems for software engineering. Agentic configuration is a first-order factor, but its effect is asymmetric. Agentic configuration is one of the largest single drivers of system behaviour in our study, but its effect on accuracy is not monotonic. On accuracy, NA obtains the highest mean score for four of the five tasks, while MA is the strongest configuration for vulnerability detection. Adding agents otherwise leaves accuracy roughly unchanged (code generation, technical debt detection, log analysis) or degrades it (log parsing, and DA for vulnerability detection). On cost, the same axis grows steeply: averaged across tasks, hardware, models, and prompts, MA consumes 6.36Ăas much energy and runs 6.07Ăas long as NA, with worst-case slowdowns of up to 160Ăfor individual taskâhardware pairs. The asymmetry is therefore that accuracy gains from additional agents are rare and task-specific, while the resource cost is systematic. The Pareto analysis in RQ4 makes this concrete, showing that out of 66 Pareto-optimal configurations across all taskâhardware pairs, 36 are NA, 23 are SA, 6 are DA, and only 1 is MA. This does not mean that one should in general not use agents, but that there should be a clear justification for using them. Our findings complement the recent work of Tran and Kiela [56], who show that under matched reasoning-token budgets, single-agent systems consistently match or outperform multi-agent systems on multi-hop reasoning tasks. While their study focuses on reasoning under equal compute budgets and explains this behavior through an information-theoretic perspective, our results reveal a similar pattern across a broader range of software engineering tasks under realistic deployment conditions while explicitly quantifying the associated energy and latency costs. Across five software engineering tasks, additional agents provide only limited, task-specific accuracy improvements while imposing systematic increases in energy consumption and inference latency. The two studies therefore converge on the same practical conclusion from different perspectives: agentic complexity should not be treated as a default design choice but adopted only when there is clear evidence of a task-specific benefit. , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows41 CodeGen TechDebt Vuln LogParse LogAnal CG-7B DS-6.7B G3-4B GPT-OSS Q2.5C-7B Q3-4B 0.960.300.350.250.07 0.890.230.400.150.05 1.000.530.510.280.06 0.980.590.560.380.05 0.980.470.440.420.08 0.920.380.580.310.06 Accuracy CodeGen TechDebt Vuln LogParse LogAnal 0.070.030.220.030.23 0.370.330.830.500.51 0.070.030.380.070.28 0.440.072.490.550.35 0.100.020.382.450.24 0.340.030.540.020.55 Energy 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy (task-specific units) 10 1 10 0 Energy (kWh, log scale) Fig. 14. Cross-task comparison of LLM effectiveness and energy cost. Rows denote CodeGemma-7B (CG-7B), DeepSeek-Coder-6.7B (DS-6.7B), Gemma-3-4B (G3-4B), GPT-OSS-20B (GPT-OSS), Qwen2.5-Coder-7B (Q2.5C- 7B), and Qwen3-4B (Q3-4B); columns denote code generation (CodeGen), technical debt detection (TechDebt), vulnerability detection (Vuln), log parsing (LogParse), and log analysis (LogAnal). The left heatmap reports mean task-specific performance as a proportionâPass@1, Macro F1, F1-score, or parsing accuracy, depending on the taskâand the right heatmap reports mean total pipeline energy per run in kWh. Hardware is first averaged within each modelâprompt combination, after which values are averaged over agent configurations and prompting strategies. The energy color scale is logarithmic, whereas cell annotations report untransformed mean values. GPT-OSS values are based on Server-SMU because that model was not evaluated on the other platforms. The differing performance and energy leaders across tasks show no model is uniformly optimal. Model and prompt are task-specific levers, not universal ones. While agentic configuration has a mostly monotonic cost effect, model and prompt have heterogeneous effects whose direction depends on the task. RQ1 shows that no single model is best across all tasks (Pass@1 is maximized by G3-4B, Macro F1 for technical debt by GPT-OSS, vulnerability F1 by Q3-4B, and the two log metrics by Q2.5C-7B), and that prompt strategy can swing log-parsing accuracy from 3.4% to 54.3%. RQ2 shows that the lowest-energy model also varies by task, and that the energy gap between models is often larger than the gap between hardware platforms for the same configuration. Figure 14 makes the absence of a universal winner visually explicit: the best accuracy model varies by task, with Q2.5C-7B leading both log tasks, while the lowest-energy model also varies, with CG-7B recurring across several tasks. Model and prompt are therefore the levers that actually deliver task-specific performance improvements, but they require per-task tuning rather than a single global default. Hardware mostly amplifies the cost of heavier configurations. Hardware is the only factor that does not have a strong direct effect on accuracy: across the hardware-balanced subset, accuracy varies by less than 3 percentage points between the three platforms. On the cost side, hardware effects are modest for NA and SA but widen substantially for DA and MA: MA on Workstation-SMU is 3.4Ă slower than MA on Server-STF, and the energy gap follows the same direction. Hardware selection therefore becomes a first-order concern primarily as a function of the agentic configuration it is asked to run; it amplifies rather than counteracts the agentic cost. Energy and latency move together; accuracy moves separately. RQ3 shows a near-perfect logâlog correlation between energy and latency (í=0.987), driven by the fact that local inference on the same hardware draws roughly constant power. As a consequence, energy savings and latency savings are largely co-achieved, where a configuration change that reduces one will almost always reduce the other. Accuracy, in contrast, is decoupled from this axisâit depends on whether the , Vol. 1, No. 1, Article . Publication date: October 2026. 42Astekin et al. chosen model and prompt are appropriate for the task, not on how much energy the configuration consumes. 5.2 Design Guidelines for Sustainable Agentic LLM Systems Drawing on the insights above, we offer the following guidelines for practitioners building LLM- based developer tools. Each guideline is based on our RQ findings. G1: Start non-agentic, escalate only with evidence. For all five tasks studied, NA is the best or tied- for-best configuration on at least one of accuracy, energy, or latency, and it dominates the Pareto front (55% of all Pareto-optimal configurations). Adding agents should be treated as a hypothesis to test, and not a default. SA is a reasonable next step when a measurable task-specific benefit is observed; DA and MA should be reserved for tasks, such as vulnerability detection in our results, where there is concrete evidence that multi-round reasoning helps enough to justify the cost. G2: Match the model to the task before tuning the architecture. Model choice has a larger and more task-specific effect than agentic configuration. Table 13 can be used as a starting point: G3-4B for code generation, GPT-OSS or G3-4B for technical debt detection, Q3-4B for vulnerability detection, Q2.5C-7B for log tasks where accuracy is paramount, and CG-7B/Q3-4B when energy efficiency is important. The attractiveness of a model depends on whether it is sufficient for the task, and not on its parameter count alone: smaller is not automatically more efficient (e.g., Q2.5C-7B can consume more than two orders of magnitude more energy than the best model on log parsing). G3: Use few-shot prompting for output-structure-sensitive tasks. Few-shot prompting is the single most impactful lever for log parsing, raising parsing accuracy from 3.4% to 54.3% and reducing energy by 24â41%. It also reduces energy for technical debt detection. For code generation, however, few-shot examples can hurt both accuracy and energy. Few-shot prompting is therefore appropriate when the task is sensitive to output format, and should be tested empirically otherwise. G4: Choose hardware based on workload, not on raw capability. For lightweight (NA/SA) workloads, hardware differences are modest, and choices can be driven by availability or cost. For heavier (DA/MA) workloads, hardware becomes a first-order concern: high-end servers can be 3.4Ăfaster than workstations on MA, and the energy gap widens accordingly. If the workload must be agentic, invest in suitable hardware; otherwise, the savings from a simpler architecture exceed the savings from upgrading hardware. G5: Report energy alongside accuracy. The strong correlation between energy and latency (RQ3) and the dominance of lightweight configurations on the Pareto front (RQ4) only become visible because all four dimensions were measured uniformly. We encourage authors of new agentic systems to publish accuracy together with energy on a fixed reference platform, so that the community can compare designs on a more complete basis than accuracy alone. 5.3 Implications For tool builders. The strongest practical takeaway is that competitive developer tools can be built around lightweight LLM configurations, provided that the model and prompt are matched to the task. Multi-agent orchestration should be added selectively, instrumented with energy or latency reporting, and validated against a non-agentic baseline before being shipped to users. For researchers. Many existing comparisons of agentic systems report only accuracy and only on a single task. Our results show that conclusions drawn from such single-axis evaluations may be misleading: configurations that look attractive on accuracy alone (e.g., MA on vulnerability detection) can be substantially more expensive than lower-cost alternatives, while configurations , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows43 that look attractive on cost alone may be insufficient for tasks where the accuracy gap matters. We advocate for multi-objective, multi-task evaluation as a standard practice for agentic LLM research. For sustainability assessment. The energy footprint of agentic LLM systems is not determined by any single design choice, but by the multiplicative interaction of model, agent configuration, prompt, and hardware. Across our 1,840 experimental runs, the campaign consumed approxi- mately 651.7 kWh of electricity, which CodeCarbon translates to roughly 110.8 kg CO 2 e at the default carbon intensity used in our experiments (mean implied intensity 0.170 kg CO 2 e/kWh). The absolute number is modestâcomparable to a few hundred kilometres of driving an average passenger car [58]âbut it is dominated by a small subset of configurations: the most expensive single configuration alone accounts for about 5.5 kg CO 2 e, while the median configuration is roughly two orders of magnitude lower. This means that emissions reduction in agentic LLM workflows is not a question of small efficiency gains across the board, but more about avoiding the worst configurations. We therefore caution against treating CO 2 as a stand-alone indicator: under a fixed carbon-intensity assumption, it is a linear transformation of energy and adds no analytical insight beyond the energy results, but its translation into emissions depends strongly on when and where a model is run. Sustainability assessments of agentic systems should report energy at a fixed reference platform (so that designs can be compared) together with a transparent CO 2 estimate that makes the carbon-intensity assumption explicit, and they should highlight the long tail of high-cost configurations rather than the mean. 6 Threats to Validity 6.1 Internal Validity Threats to internal validity concern whether the observed effects can be attributed to the experi- mental factors we studied. To counter such threats, we used fixed prompts and datasets, set the temperature to 0, disabled agent-level caching, restarted the Ollama server between runs, and executed repeated trials under the same protocol. Still, some sources of stochasticity may remain, such as low-level nondeterminism due to GPU kernels, model-serving behavior, and hardware state. Our three repeated runs per configuration (except for GPT-OSS) capture some variability, but do not fully represent long-term variation due to system temperature, background services, or hardware aging. Results involving GPT-OSS should therefore be interpreted with additional caution, since this model was only evaluated on one hardware platform and without repeated runs. To reduce the risk that hardware constraints affected the accuracy analysis, we have done balanced hardware comparisons for RQ1 by averaging the numbers across hardware platforms before averaging across experimentation configurations, preventing that the models run on multiple hardware platforms are weighted more than the GPT-OSS model in the reported numbers. 6.2 External Validity External validity concerns the generalizability of our findings. We evaluate five different software engineering tasks, which gives a broader coverage than single-task benchmarks, but it is still only a subset of developer workflows. The results may not generalize to tasks such as code review, test generation, repository-level maintenance, or autonomous development. The generalizability is also restricted by the datasets, since real-life use cases generally involve more complex, noisy data and larger inputs compared to the curated benchmark datasets used to evaluate each configuration in this study. Additionally, the datasets used in this study cover a limited set of programming languages: HumanEval targets Python, and the MLCQ and PrimeVul datasets consist of Java and C/C++ code, respectively. The findings on code generation, technical debt detection, and , Vol. 1, No. 1, Article . Publication date: October 2026. 44Astekin et al. vulnerability detection may therefore not generalize to other languages, particularly those with different syntactic structures, ecosystem conventions, or levels of LLM training coverage. The model and hardware choices also present a threat to external validity, as we have evaluated only small, local LLMs with a maximum of 20 billion parameters. Results may differ for larger models, and particularly frontier models. Publicly hosted frontier models, however, do not allow direct local measurement of energy consumption, which was one of the main aims of this study. Similarly, cloud deployments may behave differently because of batching, virtualization, datacenter power usage effectiveness, cooling infrastructure, and location-specific carbon intensity. Our results are affected by our specific implementation of the agentic configurations and the prompts used for the various tasks. Agentic systems can be designed in a multitude of ways, and small details can create significant changes. For simpler, more structured tasks such as log parsing and technical debt detection, we adopted generic orchestration patterns (ActorâCritic and User ProxyâActorâCriticâRefiner) that are broadly applicable across tasks. For tasks requiring deeper domain reasoning (vulnerability detection, log analysis, and code generation), we defined task-specific agent roles informed by prior literature, allowing us to evaluate representative and validated multi-agent setups rather than purely generic ones. More specialized or extensively tuned multi-agent designs could yield different accuracy results for individual tasks, particularly for vulnerability detection where our MA configuration already shows a meaningful accuracy advantage. Our claim is therefore not that the specific configurations studied are optimal, but that they represent realistic design choices a practitioner would make. The finding that lightweight configurations dominate the Pareto front should be read as evidence about the costâbenefit ratio of typical agentic orchestration, not as a proof that no multi-agent design can ever achieve a better accuracyâenergy trade-off. Future work could explore the multi-agent design space for individual tasks to identify whether more task-specialized orchestration can shift the Pareto frontier. 6.3 Construct Validity Construct validity concerns whether the measurements used in the study capture the concepts we intend to analyze. For task performance, we use the term accuracy as an umbrella term, while the actual metric is task-specific: Pass@1 for code generation, Macro F1 and per-class F1 for technical debt detection, F1-score for vulnerability detection and log analysis, and parsing accuracy for log parsing. This improves the fit between metric and task, but it also means that performance values are not directly interchangeable across tasks. Cross-task averages should therefore be interpreted as summaries of relative behavior across the benchmark suite, not as a single universal measure of software engineering capability. The technical debt task is a particular construct-validity risk because plain accuracy can hide poor performance on minority smell classes. We mitigate this by using Macro F1 as the primary task metric and by reporting per-class F1 scores. However, these metrics still reduce a complex code- quality judgment to a fixed-label classification problem and do not capture severity, maintainability impact, or disagreement among human reviewers. Energy consumption and latency are also affected by specific measurement choices. CodeCarbon estimates process-level energy consumption from CPU, GPU, and memory signals rather than measuring wall-socket energy with external instrumentation. This provides a consistent basis for comparing configurations, but it may result in greater uncertainty, depending on the idle power draw on various hardware platforms. A further construct-validity consideration concerns the post-processing used to extract predic- tions from LLM outputs. Since LLMs generate free-form natural language responses, each task required a task-specific parser or extraction procedure to map the generated text to the expected evaluation format (e.g., labels, code, or structured log templates). To mitigate this issue, our prompts , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows45 explicitly instructed the models to produce responses in predefined structured formats. Neverthe- less, adherence to these formats varied across models, reflecting differences in response style and instruction-following behavior. Deviations from the expected format occurred in some cases, par- ticularly in multi-agent configurations where the final output resulted from multiple intermediate reasoning steps. Although the post-processing rules were applied consistently across all configura- tions for a given task, alternative extraction strategies or more robust output normalization could yield different measured accuracy, particularly when a response deviated from the expected format while still conveying the correct semantic answer. Consequently, the reported accuracy reflects both the underlying model performance and the effectiveness of the response extraction procedure adopted in this study. 7 Conclusion This paper evaluated the effectiveness, energy consumption, and latency of agentic LLM configura- tions across five software engineering tasks, six LLMs, three hardware platforms, and two prompt strategies. The results show that increasing agentic complexity does not provide a general accuracy benefit. The multi-agent configuration improves average accuracy for vulnerability detection, but NA or SA is sufficient, and often strongest, for the other tasks. The sustainability cost of additional agents is much more systematic. Averaged across tasks, models, prompts, and hardware platforms, MA consumes 6.36Ăas much energy and runs 6.07Ăas long as NA, while appearing in only one of the 66 Pareto-optimal configurations. Energy and latency are also tightly coupled (í=0.987 in logâlog space), which means that inefficient configurations are usually both more energy-intensive and slower. Across the full experimental campaign, the runs consumed 651.7 kWh and corresponded to 110.8 kg CO 2 e under the fixed carbon-intensity assumption used in the study. The main practical implication is that sustainable agentic LLM system design should start from a lightweight baseline and escalate only when a task-specific benefit is measured. Model and prompt selection often matter more than adding agents: model choice varies sharply by task, and few-shot prompting is especially important for structured-output tasks such as log parsing. Future evaluations of agentic software engineering systems should therefore report accuracy together with energy and latency, so that claimed improvements can be assessed as full costâbenefit trade-offs rather than accuracy gains in isolation. Beyond evaluation methodology, future work should investigate alternative role assignments and coordination strategies within multi-agent configurations to determine whether more task-specialized designs can shift the Pareto frontier without proportionally increasing energy and latency costs. Future work will also extend the evaluation to repository-scale and tool-augmented software engineering tasks, additional model families, and production deployment environments to assess the generalizability of the observed trade-offs. We will further investigate adaptive orchestration policies that activate additional agents only when task uncertainty or expected accuracy gains justify the energy and latency overhead. Acknowledgments The work has been conducted as part of the ENFIELD project (101120657) and the INTEND project (101135576) funded by the European Commission within the HEU Programme. References [1]AgentGPT. Visited in 2024.AgentGPT: a tool to configure and deploy Autonomous AI agents. https://agentgpt.reworkd.ai/. [2]Negar Alizadeh, Boris Belchev, Nishant Saurabh, Patricia Kelbert, and Fernando Castor. 2025. Language Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy. In 2025 IEEE/ACM 22nd International , Vol. 1, No. 1, Article . Publication date: October 2026. 46Astekin et al. Conference on Mining Software Repositories (MSR). IEEE, 725â736. [3]Lasse F Wolff Anthony, Benjamin Kanding, and Raghavendra Selvan. 2020. Carbontracker: Tracking and predicting the carbon footprint of training deep learning models. arXiv preprint arXiv:2007.03051 (2020). [4]Merve Astekin, Arda Goknil, Sagar Sen, Simeon Tverdal, and Phu Nguyen. 2025. Detecting Technical Debt in Source Code Changes Using Large Language Models. In International Conference on Product-Focused Software Process Improvement. Springer, 334â352. [5] Merve Astekin, Max Hort, and Leon Moonen. 2024. A Comparative Study on Large Language Models for Log Parsing. In Proceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 234â244. [6]AutoGPT. Visited in 2024.AutoGPT: Build, Deploy, and Run AI Agents.https://github.com/Significant- Gravitas/AutoGPT. [7] Lingjiao Chen, Matei Zaharia, and James Zou. 2023. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176 (2023). [8]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. (2021). arXiv:2107.03374 [cs.LG] [9]Yujia Chen. 2025. AutoReview: An LLM-based Multi-Agent System for Security Issue-Oriented Code Review. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 1022â1024. [10]Kuen Sum Cheung, Mayuri Kaul, Gunel Jahangirova, Mohammad Reza Mousavi, and Eric Zie. 2025. Comparative Analysis of Carbon Footprint in Manual vs. LLM-Assisted Code Development. In Proceedings of the 1st International Workshop on Responsible Software Engineering. 13â20. [11] B. Courty et al.2024. mlco2/codecarbon: v2.4.1. https://doi.org/10.5281/zenodo.11171501. Zenodo, [Online]. Available: https://doi.org/10.5281/zenodo.11171501. [12]Tri Dao, Daniel Y Fu, Stefano Ermon, Atri Rudra, and Christopher RĂ©. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. In Advances in neural information processing systems. [13]Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in neural information processing systems 35 (2022), 30318â30332. [14]Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. 2025. Vulnerability Detection with Code Language Models: How Far are We?. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). 1729â1741. https://doi.org/10.1109/ ICSE55347.2025.00038 [15] Jared Fernandez, Clara Na, Vashisth Tiwari, Yonatan Bisk, Sasha Luccioni, and Emma Strubell. 2025. Energy considera- tions of large language model inference and efficiency optimizations. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 32556â32569. [16] Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323 (2022). [17]Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming â The Rise of Code Intelligence. arXiv:2401.14196 [cs.SE] https://arxiv.org/abs/2401.14196 [18]Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead. ACM Transactions on Software Engineering and Methodology 34, 5 (2025), 1â30. [19]Pinjia He, Jieming Zhu, Zibin Zheng, and Michael R. Lyu. 2017. Drain: An Online Log Parsing Approach with Fixed Depth Tree. In 2017 IEEE International Conference on Web Services (ICWS). 33â40. https://doi.org/10.1109/ICWS.2017.13 [20]Zhenyu He, Zexuan Zhong, Tianle Cai, Jason D Lee, and Di He. 2023. Rest: Retrieval-based speculative decoding. arXiv preprint arXiv:2311.08252 (2023). [21]Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research 21, 248 (2020), 1â43. [22]Dong Huang, Jie M Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. 2023. Agentcoder: Multi-agent- based code generation with iterative testing and optimisation. arXiv preprint arXiv:2312.13010 (2023). , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows47 [23]Mingqiang Huang, Ao Shen, Kai Li, Haoxiang Peng, Boyu Li, Yupeng Su, and Hao Yu. 2025. Edgellm: A highly efficient cpu-fpga heterogeneous edge accelerator for large language models. IEEE Transactions on Circuits and Systems I: Regular Papers (2025). [24]Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, Yunlong Feng, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024. Qwen2.5-Coder Technical Report. arXiv:2409.12186 [cs.CL] https://arxiv.org/abs/2409.12186 [25]Erik Johannes Husom, Arda Goknil, Merve Astekin, Lwin Khin Shar, Andre KĂ„ sen, Sagar Sen, Benedikt Andreas Mithassel, and Ahmet Soylu. 2025. Sustainable llm inference for edge ai: Evaluating quantized llms for energy efficiency, output accuracy, and inference latency. ACM Transactions on Internet of Things 6, 4 (2025), 1â35. [26]Erik Johannes Husom, Arda Goknil, Lwin Khin Shar, and Sagar Sen. 2024. The price of prompting: Profiling energy use in large language models inference. arXiv preprint arXiv:2407.16893 (2024). [27] Yoichi Ishibashi and Yoshimasa Nishimura. 2024. Self-organized agents: A llm multi-agent framework toward ultra large-scale code generation and optimization. arXiv preprint arXiv:2404.02183 (2024). [28]Nidhal Jegham, Marwen Abdelatti, Lassad Elmoubarki, and Abdeltawab Hendawi. 2025. How Hungry is AI? Bench- marking Energy, Water, and Carbon Footprint of LLM Inference. arXiv preprint arXiv:2505.09598 (2025). [29] Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515 (2024). [30]Sourena Khanzadeh. 2025. AgentMesh: A Cooperative Multi-Agent Generative AI Framework for Software Development Automation. arXiv preprint arXiv:2507.19902 (2025). [31]Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles. 611â626. [32]Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019). [33]Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning. PMLR, 19274â19286. [34] Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. 2024. Codetree: Agent-guided tree search for code generation with large language models. arXiv preprint arXiv:2411.04329 (2024). [35]Pengfei Li, Jianyi Yang, Mohammad A. Islam, and Shaolei Ren. 2025. Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models. arXiv:2304.03271 [cs.LG] https://arxiv.org/abs/2304.03271 [36]Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth 1, 1 (2024), 9. [37]Yue Liu, Sin Kit Lo, Qinghua Lu, Liming Zhu, Dehai Zhao, Xiwei Xu, Stefan Harrer, and Jon Whittle. 2025. Agent design pattern catalogue: A collection of architectural patterns for foundation model based agents. Journal of Systems and Software 220 (2025), 112278. [38]Lech Madeyski and Tomasz Lewowski. 2020. MLCQ: Industry-Relevant Code Smell Data Set. In Proceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering (Trondheim, Norway) (EASE â20). Association for Computing Machinery, New York, NY, USA, 342â347. https://doi.org/10.1145/3383219.3383264 [39]F.S. Marcondes, A. Gala, R. MagalhĂŁes, F. Perez de Britto, D. DurĂŁes, and P. Novais. 2025. Using ollama. In Natural Language Analytics with Generative Large-Language Models: A Practical Approach with Ollama and Open-Source LLMs. Springer, 23â35. [40]Noble Saji Mathews and Meiyappan Nagappan. 2024. Test-Driven Development for Code Generation. arXiv preprint arXiv:2402.13521 (2024). [41] Ana Nunez, Nafis Tanveer Islam, Sumit Kumar Jha, and Peyman Najafirad. 2024. Autosafecoder: A multi-agent framework for securing llm code generation through static analysis and fuzz testing. arXiv preprint arXiv:2409.10737 (2024). [42] Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. Is self-repair a silver bullet for code generation? arXiv preprint arXiv:2306.09896 (2023). [43] OpenAI, :, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, Kai Chen, Mark Chen, Enoch Cheung, Aidan Clark, Dan Cook, Marat Dukhan, Casey Dvorak, Kevin Fives, Vlad Fomenko, Timur Garipov, Kristian Georgiev, Mia Glaese, Tarun Gogineni, Adam Goucher, Lukas Gross, Katia Gil Guzman, John Hallman, Jackie Hehir, Johannes Heidecke, Alec Helyar, Haitang Hu, Romain Huet, Jacob Huh, Saachi Jain, Zach Johnson, Chris Koch, Irina Kofman, Dominik Kundel, Jason Kwon, Volodymyr Kyrylov, Elaine Ya Le, Guillaume Leclerc, James Park Lennon, Scott Lessans, Mario Lezcano-Casado, Yuanzhi Li, Zhuohan Li, Ji Lin, Jordan Liss, Lily, Liu, Jiancheng Liu, Kevin Lu, Chris Lu, Zoran Martinovic, Lindsay , Vol. 1, No. 1, Article . Publication date: October 2026. 48Astekin et al. McCallum, Josh McGrath, Scott McKinney, Aidan McLaughlin, Song Mei, Steve Mostovoy, Tong Mu, Gideon Myles, Alexander Neitz, Alex Nichol, Jakub Pachocki, Alex Paino, Dana Palmie, Ashley Pantuliano, Giambattista Parascandolo, Jongsoo Park, Leher Pathak, Carolina Paz, Ludovic Peran, Dmitry Pimenov, Michelle Pokrass, Elizabeth Proehl, Huida Qiu, Gaby Raila, Filippo Raso, Hongyu Ren, Kimmy Richardson, David Robinson, Bob Rotsted, Hadi Salman, Suvansh Sanjeev, Max Schwarzer, D. Sculley, Harshit Sikchi, Kendal Simon, Karan Singhal, Yang Song, Dane Stuckey, Zhiqing Sun, Philippe Tillet, Sam Toizer, Foivos Tsimpourlas, Nikhil Vyas, Eric Wallace, Xin Wang, Miles Wang, Olivia Watkins, Kevin Weil, Amy Wendling, Kevin Whinnery, Cedric Whitney, Hannah Wong, Lin Yang, Yu Yang, Michihiro Yasunaga, Kristen Ying, Wojciech Zaremba, Wenting Zhan, Cyril Zhang, Brian Zhang, Eddie Zhang, and Shengjia Zhao. 2025. gpt-oss-120b & gpt-oss-20b Model Card. arXiv:2508.10925 [cs.CL] https://arxiv.org/abs/2508.10925 [44]Ipek Ozkaya. 2023. Application of large language models to software engineering tasks: Opportunities, risks, and implications. IEEE Software 40, 3 (2023), 4â8. [45] Ruwei Pan, Hongyu Zhang, and Chao Liu. 2025. CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation. arXiv preprint arXiv:2501.07811 (2025). [46]David Patterson, Jeffrey M Gilbert, Marco Gruteser, Efren Robles, Krishna Sekar, Yong Wei, and Tenghui Zhu. 2024. Energy and emissions of machine learning on smartphones vs. the cloud. Commun. ACM 67, 2 (2024), 86â97. [47] Mauro PezzĂš, Silvia AbrahĂŁo, Birgit Penzenstadler, Denys Poshyvanyk, Abhik Roychoudhury, and Tao Yue. 2025. A 2030 Roadmap for Software Engineering. ACM Transactions on Software Engineering and Methodology 34, 5 (2025), 1â55. [48]Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. 2023. From words to watts: Benchmarking the energy costs of large language model inference. In IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1â9. [49] Roy Schwartz, Jesse Dodge, Noah A Smith, and Oren Etzioni. 2020. Green ai. Commun. ACM 63, 12 (2020), 54â63. [50] Semantic Kernel. Visited in 2024. Semantic Kernel: a lightweight, open-source development kit for building AI agents. https://learn.microsoft.com/en-us/semantic-kernel/overview/. [51]Raphael Shu, Nilaksh Das, Michelle Yuan, Monica Sunkara, and Yi Zhang. 2024. Towards effective genAI multi-agent collaboration: Design and evaluation for enterprise applications. arXiv preprint arXiv:2412.05449 (2024). [52] Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 (2019). [53] Wei Tao, Yucheng Zhou, Yanlin Wang, Wenqiang Zhang, Hongyu Zhang, and Yu Cheng. 2024. Magis: Llm-based multi-agent framework for github issue resolution. Advances in Neural Information Processing Systems 37 (2024), 51963â51993. [54]CodeGemma Team, Heri Zhao, Jeffrey Hui, Joshua Howland, Nam Nguyen, Siqi Zuo, Andrea Hu, Christopher A. Choquette-Choo, Jingyue Shen, Joe Kelley, Kshitij Bansal, Luke Vilnis, Mateo Wirth, Paul Michel, Peter Choy, Pratik Joshi, Ravin Kumar, Sarmad Hashmi, Shubham Agrawal, Zhitao Gong, Jane Fine, Tris Warkentin, Ale Jakse Hartman, Bin Ni, Kathy Korevec, Kelly Schaefer, and Scott Huffman. 2024. CodeGemma: Open Code Models Based on Gemma. arXiv:2406.11409 [cs.CL] https://arxiv.org/abs/2406.11409 [55]Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre RamĂ©, Morgane RiviĂšre, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, GaĂ«l Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai, Anton Tsitsulin, Robert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, Iain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, Jan-Thorsten Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alex Feng, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, AndrĂĄs György, AndrĂ© Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, Christopher A. Choquette-Choo, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov, Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimczak-PluciĆska, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, Ivan Nardini, Jean Pouget-Abadie, Jetha Chan, Joe Stanton, John Wieting, Jonathan Lai, Jordi Orbay, Joseph Fernandez, Josh Newlan, Ju yeong Ji, Jyotinder Singh, Kat Black, Kathy Yu, Kevin Hui, Kiran Vodrahalli, Klaus Greff, Linhai Qiu, Marcella Valentine, Marina Coelho, Marvin Ritter, Matt Hoffman, Matthew Watson, Mayank Chaturvedi, Michael Moynihan, Min Ma, Nabila Babar, Natasha Noy, Nathan Byrd, Nick Roy, Nikola Momchev, Nilay Chauhan, Noveen Sachdeva, Oskar Bunyan, Pankil Botarda, Paul Caron, Paul Kishan Rubenstein, Phil Culliton, Philipp Schmid, Pier Giuseppe Sessa, Pingmei Xu, Piotr Stanczyk, Pouya Tafti, Rakesh Shivanna, Renjie Wu, Renke Pan, Reza Rokni, Rob Willoughby, Rohith Vallu, Ryan Mullins, Sammy Jerome, Sara Smoot, Sertan Girgin, Shariq Iqbal, Shashir Reddy, , Vol. 1, No. 1, Article . Publication date: October 2026. Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows49 Shruti Sheth, Siim PĂ”der, Sijal Bhatnagar, Sindhu Raghuram Panyam, Sivan Eiger, Susan Zhang, Tianqi Liu, Trevor Yacovone, Tyler Liechty, Uday Kalra, Utku Evci, Vedant Misra, Vincent Roseberry, Vlad Feinberg, Vlad Kolesnikov, Woohyun Han, Woosuk Kwon, Xi Chen, Yinlam Chow, Yuvein Zhu, Zichuan Wei, Zoltan Egyed, Victor Cotruta, Minh Giang, Phoebe Kirk, Anand Rao, Kat Black, Nabila Babar, Jessica Lo, Erica Moreira, Luiz Gustavo Martins, Omar Sanseviero, Lucas Gonzalez, Zach Gleicher, Tris Warkentin, Vahab Mirrokni, Evan Senter, Eli Collins, Joelle Barral, Zoubin Ghahramani, Raia Hadsell, Yossi Matias, D. Sculley, Slav Petrov, Noah Fiedel, Noam Shazeer, Oriol Vinyals, Jeff Dean, Demis Hassabis, Koray Kavukcuoglu, Clement Farabet, Elena Buchatskaya, Jean-Baptiste Alayrac, Rohan Anil, Dmitry, Lepikhin, Sebastian Borgeaud, Olivier Bachem, Armand Joulin, Alek Andreev, Cassidy Hardin, Robert Dadashi, and LĂ©onard Hussenot. 2025. Gemma 3 Technical Report. arXiv:2503.19786 [cs.CL] https://arxiv.org/abs/2503.19786 [56]Dat Tran and Douwe Kiela. 2026. Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets. arXiv preprint arXiv:2604.02460 (2026). [57] K.-T. Tran, D. Dao, M.-D. Nguyen, Q.-V. Pham, B. OâSullivan, and H.D. Nguyen. 2025. Multi-Agent Collaboration Mechanisms: A Survey of LLMs. arXiv:2501.06322 [cs.AI] https://arxiv.org/abs/2501.06322 [58]U.S. Environmental Protection Agency. 2026. Greenhouse Gas Equivalencies Calculator: Calculations and References. https://w.epa.gov/energy/greenhouse-gas-equivalencies-calculator-calculations-and-references Accessed 2026-07- 27. [59] Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, and Ge Yu. 2023. Intervenor: Prompting the coding ability of large language models with the interactive chain of repair. arXiv preprint arXiv:2311.09868 (2023). [60]Ratnadira Widyasari, Martin Weyssow, Ivana Clairine Irsan, Han Wei Ang, Frank Liauw, Eng Lieh Ouh, Lwin Khin Shar, Hong Jin Kang, and David Lo. 2025. Let the trial begin: A mock-court approach to vulnerability detection using llm-based agents. arXiv preprint arXiv:2505.10961 (2025). [61] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Autogen: Enabling next-gen llm applications via multi-agent conversation framework. arXiv preprint arXiv:2308.08155 (2023). [62]Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2025. Demystifying LLM-Based Software Engineering Agents. Proc. ACM Softw. Eng. 2, FSE, Article FSE037 (June 2025), 24 pages. https://doi.org/10.1145/3715754 [63]Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Luoxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, et al.2023. Openagents: An open platform for language agents in the wild. arXiv preprint arXiv:2310.10634 (2023). [64]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 Technical Report. arXiv:2505.09388 [cs.CL] https://arxiv.org/abs/2505.09388 [65] Zhengmin Yu, Yuan Zhang, Ming Wen, Yinan Nie, Wenhui Zhang, and Min Yang. 2025. CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building. Proceedings of the ACM on Software Engineering 2, FSE (2025), 2618â2640. [66] Quanjun Zhang, Chunrong Fang, Yang Xie, Yaxin Zhang, Yun Yang, Weisong Sun, Shengcheng Yu, and Zhenyu Chen. 2023. A survey on large language models for software engineering. arXiv preprint arXiv:2312.15223 (2023). [67]Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher RĂ©, Clark Barrett, et al.2023. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems 36 (2023), 34661â34710. [68] Jieming Zhu, Shilin He, Pinjia He, Jinyang Liu, and Michael R. Lyu. 2023. Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). 355â366. https://doi.org/10.1109/ISSRE59848.2023.00071 , Vol. 1, No. 1, Article . Publication date: October 2026. 50Astekin et al. A Technical Debt Per-Class Breakdowns This appendix reports the per-smell-class F1 breakdowns for the Technical Debt Detection task. The aggregate per-class results are summarized in Table 14 in Section 4.1; here we provide the same decomposition by agentic configuration (Table 25), LLM model (Table 26), and prompt strategy (Table 27). Table 25. RQ1 TD-specific per-class results: F1 by smell class and agent configuration (mean±std, averaged over models, hardware, and prompts). Smell ClassNASADAMA mean± stdmean± stdmean± stdmean± std No Smell57.4± 10.657.3± 13.257.1± 15.051.3± 18.7 Blob32.8± 24.631.3± 21.328.6± 18.727.4± 18.7 Data Class41.1± 26.239.2± 25.940.2± 28.0 41.7± 27.1 Feature Envy4.9± 5.87.5± 6.110.1± 7.19.8± 8.7 Long Method65.0± 22.6 66.1± 19.561.0± 23.358.9± 23.1 Table 26. RQ1 TD-specific per-class results: F1 by smell class and LLM model (mean±std, averaged over agent configurations, hardware, and prompts). Smell ClassCG-7BDS-6.7BG3-4BGPT-OSSQ2.5C-7BQ3-4B mean± stdmean± stdmean± stdmean± stdmean± stdmean± std No Smell63.7± 11.755.8± 2.2 69.6± 6.667.7± 2.147.5± 9.738.3± 14.6 Blob2.7± 3.917.3± 6.853.7± 10.3 61.2± 12.138.5± 7.027.7± 13.2 Data Class2.9± 7.914.3± 6.557.0± 3.875.3± 3.964.1± 9.453.0± 5.3 Feature Envy 13.1± 9.28.0± 4.82.5± 1.70.0± 0.010.7± 6.08.8± 7.4 Long Method66.3± 5.121.3± 7.480.4± 2.488.7± 2.675.5± 5.061.6± 6.8 Table 27. RQ1 TD-specific per-class results: F1 by smell class and prompt strategy (mean±std, averaged over agent configurations, models, and hardware). Smell ClassZero-shotFew-shot mean± stdmean± std No Smell56.9± 13.854.6± 15.6 Blob32.2± 21.727.9± 19.8 Data Class39.9± 27.8 41.3± 25.3 Feature Envy 10.1± 8.36.0± 5.3 Long Method61.7± 22.7 63.8± 21.6 , Vol. 1, No. 1, Article . Publication date: October 2026.