Paper deep dive
FORGE: Multi-Agent Graduated Exploitation and Detection Engineering
Farooq Shaikh
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 7/9/2026, 1:03:54 AM
Summary
FORGE is a multi-agent system that bridges proof-of-concept generation, vulnerability prioritization, and detection rule engineering through a graduated exploitation depth taxonomy (L0–L3). It generates targeted vulnerable applications, assesses exploitation via an LLM-primary oracle, and produces Sigma/Snort detection rules from OpenTelemetry traces, evaluated on 603 CVEs from the CVE-GENIE dataset.
Entities (10)
Relation Signals (10)
FORGE → uses → Graduated Exploitation Taxonomy
confidence 97% · bridges these three silos through graduated exploitation depth... assessed by an LLM-primary oracle on a four-level taxonomy
LLM-Primary Oracle → assesses → Exploit Agent
confidence 96% · The LLM-primary oracle evaluates every tool call against CWE-specific criteria... producing structured verdicts with confidence scores.
FORGE → comprises → Generator Agent
confidence 95% · A Generator Agent creates targeted vulnerable web applications from CVE metadata
FORGE → comprises → Planner Agent
confidence 95% · The Planner Agent, a pure reasoning agent with no tools, produces a ranked attack plan
FORGE → comprises → Exploit Agent
confidence 95% · The Exploit Agent executes the attack plan against the deployed application.
FORGE → comprises → Detector Agent
confidence 95% · Finally, the Detector Agent runs only if exploitation reaches L1 or higher.
FORGE → comprises → Intel Agent
confidence 95% · Five specialized agents (Intel, Generator, Planner, Exploit, and Detector) execute in a fixed pipeline
FORGE → generates → Sigma
confidence 94% · produces Sigma and Snort detection rules grounded in OpenTelemetry exploitation traces.
FORGE → generates → Snort
confidence 94% · produces Sigma and Snort detection rules grounded in OpenTelemetry exploitation traces.
OpenTelemetry → provides → exploitation traces
confidence 93% · grounded in OpenTelemetry exploitation traces... queries OpenTelemetry spans from the exploitation phase
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vulnerability disclosure volumes now far exceed organizational assessment capacity, yet three adjacent research communities (proof-of-concept generation, vulnerability prioritization, and detection rule engineering) operate largely in isolation. Existing automated exploit generation systems report binary pass/fail outcomes, discarding partial progress and producing no signal for the other two communities. This paper presents FORGE, a multi-agent system that bridges these three silos through graduated exploitation depth. Five specialized agents (Intel, Generator, Planner, Exploit, and Detector) execute in a fixed pipeline that (1) generates targeted vulnerable applications from CVE metadata, (2) conducts coached, multi-turn exploitation assessed by an LLM-primary oracle on a four-level taxonomy (L0: no evidence through L3: full compromise), and (3) produces Sigma and Snort detection rules grounded in OpenTelemetry exploitation traces. Graduated depth is the bridging mechanism: deeper exploitation yields richer behavioral traces for detection engineering, while depth data across scoring bands provides ground truth for prioritization validation. A tiered knowledge architecture accumulates intelligence across assessments, transferring build and exploitation experience to subsequent CVEs. Evaluation on 603 CVEs from the CVE-GENIE dataset achieves 67.8% end-to-end L1+ exploitation at USD 1.50 per CVE across eight languages and 187 CWE types. Exploitation rates remain near 68% regardless of EPSS or CVSS band, indicating that pattern-level reachability is orthogonal to metadata-based prioritization. Detection rules from L2+ exploitation achieve significantly higher span-normalized grounding than L1-derived rules (p=0.035), and 93.4% of generated Snort rules produce zero false positives against a synthetic benign corpus.
Tags
Links
- Source: https://arxiv.org/abs/2606.03453v1
- Canonical: https://arxiv.org/abs/2606.03453v1
Trouble viewing inline? Open PDF directly →
Full Text
50,222 characters extracted from source content.
Expand or collapse full text
FORGE: Multi-Agent Graduated Exploitation and Detection Engineering Farooq Shaikh Dynatrace, Vienna, Austria farooq.shaikh@dynatrace.com Abstract. Vulnerability disclosure volumes now far exceed organiza- tional assessment capacity, yet three adjacent research communities (proof- of-concept generation, vulnerability prioritization, and detection rule en- gineering) operate largely in isolation. Existing automated exploit gener- ation systems report binary pass/fail outcomes, discarding partial progress and producing no signal for the other two communities. This paper presents FORGE, a multi-agent system that bridges these three silos through graduated exploitation depth. Five specialized agents (Intel, Generator, Planner, Exploit, and Detector) execute in a fixed pipeline that (1) generates targeted vulnerable applications from CVE metadata, (2) conducts coached, multi-turn exploitation assessed by an LLM-primary oracle on a four-level taxonomy (L0: no evidence through L3: full compromise), and (3) produces Sigma and Snort detection rules grounded in OpenTelemetry exploitation traces. Graduated depth is the bridging mechanism: deeper exploitation yields richer behavioral traces for detection engineering, while depth data across scoring bands provides ground truth for prioritization validation. A tiered knowledge architec- ture accumulates intelligence across assessments, transferring build and exploitation experience to subsequent CVEs. Evaluation on 603 CVEs from the CVE-GENIE dataset achieves 67.8% end-to-end L1+ exploitation at $1.50 per CVE across eight languages and 187 CWE types. Exploitation rates remain near 68% regardless of EPSS or CVSS band, indicating that pattern-level reachability is orthogonal to metadata-based prioritization. Detection rules from L2+ exploitation achieve significantly higher span-normalized grounding than L1-derived rules (p=0.035), and 93.4% of generated Snort rules produce zero false positives against a synthetic benign corpus. Keywords: Vulnerability assessment· Multi-agent systems· Exploit generation· Detection engineering· Risk prioritization 1 Introduction Vulnerability disclosure rates now outpace human analysis by a widening mar- gin: the Common Vulnerabilities and Exposures (CVE) program published over 48,000 entries in 2025 [5], yet security teams continue to rely on metadata-driven triage that has never been validated against actual exploitation attempts [14, arXiv:2606.03453v1 [cs.CR] 2 Jun 2026 2F. Shaikh 25]. Common Vulnerability Scoring System (CVSS) severity scores and Ex- ploit Prediction Scoring System (EPSS) exploitation probabilities, the two most widely deployed prioritization instruments, both operate on surface-level indica- tors rather than ground-truth exploitability data. EPSS uses exploit-code exis- tence as a binary feature but never verifies whether that code achieves exploita- tion [13,14]; EPSS scores are frequently unavailable or low at the time vulnerabil- ities enter the Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerabilities catalog [25]; and CVSS, Stakeholder-Specific Vulner- ability Categorization (SSVC), and EPSS produce conflicting assessments on identical CVEs [16]. Automated exploit generation offers a promising path toward ground-truth exploitability data at scale. Fang et al. [10] demonstrate that GPT-4 agents can exploit 87% of 15 one-day CVEs given their descriptions, and subsequent multi- agent systems (including CVE-GENIE [32], CVE-Bench [34], and PoCGen [28]) scale to hundreds of CVEs using hierarchical agent architectures [35]. However, all such systems report binary outcomes: a CVE either has a working proof-of- concept (PoC) or it does not. This coarse granularity discards partial progress; an attempt that triggers the vulnerability but cannot achieve full exploitation is recorded identically to zero progress. Binary outcomes also produce no signal for two adjacent research communities, since vulnerability prioritization lacks ground-truth depth data to calibrate scoring models and detection engineering lacks behavioral traces from which to derive signatures. A review of the literature across PoC generation, vulnerability prioritization, and detection rule engineering reveals minimal intersection connecting these do- mains. PoC generation systems do not produce prioritization signals. Prioritiza- tion frameworks do not validate predictions against exploitation data. Detection rule generators, whether classical (TaintCheck [24], COVERS [19]) or LLM-based (FALCON [23], RuleGenie [27]), derive signatures from vulnerability descriptions or malware samples, never from dynamic exploitation telemetry. A further limitation constrains all benchmark-driven approaches, namely the dependence on pre-built containers. CVE-Bench curates 40 critical-severity web- application CVEs [34] and LiveCVEBench provides 190 tasks across 14 languages and 153 repositories [21]. These require manual effort per CVE and exhibit high failure rates when configuration drift or port conflicts arise. To address these gaps, FORGE is presented, a multi-agent system for grad- uated vulnerability exploitability assessment with three contributions. – Graduated exploitation taxonomy. A four-level depth taxonomy (L0– L3) replaces binary pass/fail with measurable exploitation depth. An LLM- primary oracle evaluates every tool call against CWE-specific criteria, appli- cation source code, and server-side snapshots, producing structured verdicts with confidence scores. Structural caps enforce conservative bounds for CWE types where full compromise is architecturally impossible. – Three-silo bridge. A single pipeline produces three outputs from each CVE assessment: (1) graduated exploitation depth as a pattern-level reachabil- ity signal for EPSS/CVSS validation, (2) Sigma and Snort detection rules FORGE: Multi-Agent Graduated Exploitation and Detection Engineering3 generated from OpenTelemetry exploitation traces, and (3) per-CWE at- tack knowledge that accumulates across assessments. Graduated depth is the bridging mechanism: L2+ exploitation produces behavioral signals rich enough for high-precision detection rules, while L1+ data provides ground truth for prioritization validation. – Targeted app generation with knowledge transfer. A Generator Agent creates targeted vulnerable web applications from CVE metadata, eliminat- ing benchmark container dependency. Five knowledge stores, a four-tier in- telligence store, generation recipes, auto-distilled cookbook tips, package- specific experience stores, and a per-CWE detection knowledge base, accu- mulate across assessments, with measurable benefits for generation reliability of known CWE types and exploit efficiency for specific vulnerability classes. Evaluation on 603 CVEs sourced from the CVE-GENIE 841-entry dataset [32] reaches L1+ on 409/603 (67.8%) end-to-end at $1.50 per CVE; 424/603 (70.3%) pass generation and deployment, of which 96.5% reach L1+. The system spans eight programming languages (Python, PHP, JavaScript/TypeScript, Go, Ruby, Java, C, and others), 187 unique CWE types, and 440 distinct GitHub reposi- tories. The remainder of this paper is organized as follows. Section 2 surveys re- lated work across the three research silos. Section 3 presents the methodology. Section 4 describes the experimental protocol and results. Section 5 addresses ethical considerations, limitations, and implications. Section 6 concludes. 2 Background and Related Work Automated vulnerability exploitability assessment draws on three distinct re- search communities that, despite addressing adjacent problems, have minimal methodological overlap. 2.1 Automated PoC and Exploit Generation LLM-based vulnerability reproduction has progressed rapidly since 2024, build- ing on the ReAct paradigm [33] that interleaves LLM reasoning with tool-use actions. Fang et al. [10] demonstrate that GPT-4 agents can exploit 87% of 15 one-day CVEs given their descriptions; Deng et al. [6] introduce Pentest- GPT for interactive penetration testing guided by LLMs; Zhu et al. [35] ex- tend this with HPTSA, a hierarchical planning agent that spawns exploitation subagents, achieving 4.3× improvement over single agents on zero-day targets. CVE-GENIE [32], the closest comparable system, reproduces CVEs by cloning the original vulnerable project inside a VM and verifying a CTF-style flag, achieving 51% binary reproduction across 841 CVEs (22 languages) at $2.77 per CVE. CVE-Bench [34] curates 40 critical-severity web-application CVEs and reports 13% best-case success under a hierarchical multi-agent framework in the one-day setting. PoCGen [28] targets npm packages specifically, combining taint-path analysis with LLM generation to achieve 77% success on a curated 4F. Shaikh subset. DrillAgent [18] introduces execution-state-aware proof-of-vulnerability generation. GONDAR [11] employs CWE-specific sink API catalogs with sepa- rate exploration and exploitation agents. Liu et al. [20] provide the first compre- hensive evaluation of 20 LLM agents for web vulnerability reproduction across 16 dimensions. ZeroDayBench [17] evaluates frontier agents on 22 novel zero- day vulnerabilities, reporting that even the best models find and patch only a fraction. All these systems share two fundamental limitations. First, binary evaluation: a CVE either has a working PoC or it does not. Partial progress is not evalu- ated, preventing correlation with severity metrics and losing information useful for detection engineering. Second, original-project reproduction: CVE-GENIE, CVE-Bench, and PoCGen all require access to the original vulnerable codebase. When complex build dependencies or infrastructure requirements resist auto- mated provisioning, reproduction fails entirely; CVE-GENIE reports 110 setup failures, 198 time/cost-limit failures, and 52 other VM or LLM errors out of 841 candidates [32]. A fundamentally different approach is taken in FORGE: a targeted minimal application is generated that imports the actual vulnerable package at its vulnerable version, exercising the real vulnerability in a simplified environment with infrastructure designed to enable exploitation at each depth level (L1–L3). Elder et al. [8] systematize 39 exploitability assessment approaches; Bui et al. [4] survey automated exploit and security test generation more broadly. No surveyed approach employs graduated depth measurement. 2.2 Vulnerability Prioritization Organizations triage thousands of CVEs using two primary scoring systems. EPSS [13] predicts 30-day exploitation probability from metadata features (CVE age, vendor, CWE type, reference count, partner data). EPSSv3 [14] improves prediction accuracy by 82% over v2. CVSS provides severity scores based on vul- nerability characteristics (attack vector, complexity, impact). SSVC [29] offers stakeholder-specific categorization but requires manual assessment of its Ex- ploitation decision point (none/PoC/active). NIST’s LEV framework [22] simi- larly highlights the absence of automated exploitation evidence in prioritization. Neither system is validated against actual exploitation attempts. EPSS uses exploit-code existence as a primary feature but never verifies whether that code actually achieves exploitation (beyond the information provided through partner data). Koscinski et al. [16] demonstrate significant disagreement between CVSS, SSVC, and EPSS on identical CVEs. Parla [25] shows EPSS scores are frequently low or unavailable when high-severity vulnerabilities enter CISA’s Known Ex- ploited Vulnerabilities catalog. Gordeychik [12] confirms this pattern with lon- gitudinal analysis of EPSS at KEV addition time. Al Haddad et al. [1] evaluate LLMs for SSVC decision-point prediction, finding consistent over-prediction and poor agreement with expert labels, reinforcing that metadata-based prioritiza- tion remains unreliable without ground-truth exploitation data. FORGE: Multi-Agent Graduated Exploitation and Detection Engineering5 No automated system feeds, at scale, exploitation results back to calibrate these prioritization models. In FORGE, ground-truth exploitation depth data are produced that can be directly correlated with EPSS and CVSS scores. 2.3 Detection Rule Generation Detection signature generation from exploit execution has a long history. TaintCheck [24] generates network signatures via taint tracking during exploit execution. Brumley et al. [3] produce vulnerability signatures through symbolic execution. Kobayashi et al. [15] represent the most recent classical approach, gen- erating IDS signatures by executing PoC code and exploring execution paths. Between these early systems and the LLM era, detection rule generation from dynamic exploitation traces was largely dormant as a research direction; the community focused instead on signature derivation from static vulnerability descriptions and malware samples. LLM-based detection rule generation has re- vived the area from a different angle. FALCON [23] generates Snort and YARA rules from CTI reports. EvoSIEM [30] uses LLMs to enhance Sigma rules against evasion, motivated by evidence that deployed SIEM rules are frequently bypass- able [31]. Fairbanks and Serra [9] automate TTP extraction and Sigma synthesis from CTI text. All generate rules from descriptions of attacks (vulnerability advi- sories, CTI reports, malware samples), not from observed exploitation behavior. Breach and Attack Simulation (BAS) platforms occupy the adjacent space. CALDERA [2] automates post-compromise attack execution using ATT&CK techniques; Sánchez-Matas et al. [26] combine chaos engineering with CALDERA. These platforms execute attacks but produce no detection content. This gap motivates the present work. No existing system generates detec- tion rules from dynamic exploitation telemetry. FORGE’s Detector Agent re- ceives OpenTelemetry spans covering every HTTP request, command execu- tion, file access, and server response captured during exploitation, and produces Sigma/Snort rules validated against those same spans. Two key differentiators distinguish FORGE from these systems. First, grad- uated depth measurement (L0–L3) replaces binary pass/fail, enabling priori- tization ground truth and exploitation-grounded detection rules. Second, tar- geted environment generation decouples assessment from source-code availabil- ity: CVE-GENIE requires the original project, CVE-Bench uses curated con- tainers, and PoCGen targets npm packages only; FORGE generates minimal applications that import the actual vulnerable package, supporting eight lan- guages. No existing system combines exploitation, prioritization validation, and detection rule generation in a single pipeline with cross-CVE knowledge transfer. 3 Methodology FORGE comprises five specialized agents executing in a fixed pipeline, a tiered oracle for per-turn exploitation assessment, and a knowledge architecture that accumulates intelligence across CVE assessments. Figure 1 illustrates the pipeline topology. 6F. Shaikh CVE-2025-30370 CWE-78 · jupyterlab-git · Python 1 Intel Agent 10 turns · 6 tools NVD, OSV, ExploitDB, patches PackageResolver: registry val- idation ▷ jupyterlab-git 0.51, Tier 2 2 Generator 5 turns · 4 tools · 2 retries Actor-critic loop, AppVerifier Anti-fabrication, CWE recipes ▷ 2 att, Python/pip 3 Deploy Podman · resource limits Dockerfile build → health check Network isolation, port map- ping ▷ python:3.11, :8080✓ 4 Planner 5 turns · pure reasoning CWE-aware attack planning Primary + escalation + fall- back ▷ shell-escape $() → RCE 5 Exploit 20 turns · 5 tools · gated T1–3 recon only, T4+ exploit L0→L1→L2→L3 progression ▷ 4T, L1→L3 via $() 6 Detector 15 turns · if L1+ · OTEL Detection KB (10 CWEs × 4 tiers) Sigma + Snort rule genera- tion ▷ 2 rules, sigma + snort L3 · $0.95 · 4m 54s LLM-Primary Oracle Per-turn: tool req/resp + CWE criteria + app source Caps: XSS,CSRF → L2; coaching hints every turn Intel Store 4 tiers, 90d decay Gen. KB CWE recipes Cookbook Build tips/lang Pkg Exp. Episodic+semantic Det. KB 10 CWEs × 4 tiers Knowledge Architecture Fig. 1. FORGE pipeline architecture with running example (CVE-2025-30370, jupyterlab-git command-injection RCE via $() substitution that bypasses partial shell- escaping; L3 in 4 exploit turns). The LLM-primary oracle evaluates every exploit turn against CWE-specific criteria. Five knowledge stores accumulate intelligence across as- sessments. 3.1 Pipeline Overview The pipeline processes a single CVE through five stages. Each agent operates within a fixed turn budget and restricted toolset, enforcing bounded cost per assessment. The Intel Agent compiles a structured intelligence report containing root cause analysis, attack surface characterization, and technology stack identifica- tion from NVD advisories, GitHub security advisories, patch diffs, and known exploit databases. A PackageResolver then resolves the vulnerable package name, version, and ecosystem from NVD, OSV, and CVE-GENIE metadata. Candidate ver- sions are validated against actual package registries (npm, PyPI, RubyGems, Packagist); if the metadata version does not exist in the registry, the resolver falls back to the closest available version. This step prevents generation failures caused by stale or non-existent version strings in vulnerability databases. The Generator Agent creates a targeted vulnerable web application that embeds the actual vulnerable package at its vulnerable version. Unlike systems that clone the original project (e.g., CVE-GENIE [32]), a minimal application is generated with infrastructure explicitly shaped by the exploitation taxonomy. FORGE: Multi-Agent Graduated Exploitation and Detection Engineering7 The key design decision is level-aware scaffolding: for L1, vulnerable endpoints propagate exceptions to HTTP responses rather than suppressing errors; for L2, real data infrastructure (SQLite databases seeded with user records, sensitive configuration files) enables meaningful data exfiltration evidence; for L3, code execution paths remain unsandboxed and writable directories exist for marker file creation. This ensures the exploit agent encounters a surface that can produce graduated evidence, not just a binary trigger. Input includes the intelligence report, CWE-specific generation recipes, re- solved package information, and package-specific build experience from prior assessments. An actor-critic validation loop builds, deploys, and tests each gen- erated application; on failure, structured feedback drives the next attempt. An AppVerifier performs multi-layer fabrication detection: verifying that declared dependencies are actually imported in source code, scanning for mock-service patterns (e.g., FakeRedis, raw TCP socket servers), and performing runtime port scans inside the container to detect fake service listeners. The Planner Agent, a pure reasoning agent with no tools, produces a ranked attack plan with escalation paths from the intelligence report, appli- cation manifest, and package-specific exploitation experience. The Exploit Agent executes the attack plan against the deployed applica- tion. Tool gating is the critical design choice: the first three LLM calls restrict the agent to reconnaissance-only tools, preventing premature exploitation attempts before the attack surface is understood. Three calls was set empirically to pre- vent exploitation before vulnerability context is processed; all tools are available from LLM call 4. The LLM-primary oracle evaluates after every tool call, and coaching messages are injected when the agent fails to progress. A configurable grace period (3 turns after L3, 5 turns after L2) allows evidence consolidation before forced exit. Finally, the Detector Agent runs only if exploitation reaches L1 or higher. It queries OpenTelemetry spans from the exploitation phase, retrieves reference rules from a curated detection knowledge base, and generates Sigma and Snort rules grounded in the collected exploitation signals. 3.2 Graduated Exploitation Taxonomy Table 1 defines the four exploitation levels. The running maximum never de- creases during a single assessment; once a level is reached, it persists regardless of subsequent turn outcomes. 3.3 LLM-Primary Oracle Central to the graduated assessment is the oracle, which determines exploitation level; agent self-assessment is ignored. For every tool call, the oracle receives: (1) the tool request and response, (2) CWE-specific evidence criteria covering 20 CWE groups with natural-language descriptions for each level, (3) the generated application’s source code, and (4) a server-side snapshot (created files, running processes, log tail from the sandbox). 8F. Shaikh Table 1. Graduated exploitation taxonomy (L0–L3). Level NameEvidence L0 No evidence No exploitation evidence ob- served L1 TriggeredTargeted payload activates the specific vulnerable functionality: Path traversal sequence returns file contents, XSS payload re- flected without encoding L2 ExploitedSignificant attacker-controlled impact: data exfiltration, code execution evidence, arbitrary file access L3 Compromised Full compromise: RCE with out- put, complete data dump, persis- tence, privilege escalation The oracle produces a structured verdict: level (0–3), confidence (0.0–1.0), and reasoning. Structural caps enforce conservative bounds for CWE types where full compromise is architecturally impossible: XSS (CWE-79), path traver- sal (CWE-22), CSRF (CWE-352), and denial-of-service (CWE-400) are capped at L2 maximum. Tool-level caps prevent inflation from sandbox-local opera- tions: read_file is capped at L1 unless the file was created by the exploit; exec_command is capped at L1 unless the command targets the application. This design addresses the reward-hacking failure mode documented in the LLM-as-judge literature [7]: the exploit agent cannot inflate its own score through self-congratulatory narrative because the oracle evaluates tool outputs, not agent text. 3.4 Knowledge Architecture Underpinning the entire pipeline, five knowledge stores accumulate across as- sessments: Four-Tier Intelligence Store. Tier 1 caches raw API responses. Tier 2 stores com- piled per-CVE intelligence reports. Tier 3 accumulates per-CWE attack knowl- edge with 90-day confidence decay, strengthened by confirmations. Tier 4 is an append-only procedural memory of cross-CVE exploitation patterns. Generation Knowledge Base. Per-CWE design recipes that record framework- specific generation patterns with graduated complexity levels. Success rates are updated after each generation attempt. Cookbook Store. Auto-distilled tips extracted by an LLM critic from build errors and successful exploitation patterns. Organized by language and CWE type, with a 20-tip cap per category to prevent unbounded growth. FORGE: Multi-Agent Graduated Exploitation and Detection Engineering9 Package Experience Store. This store accumulates package-specific learnings keyed by package name and ecosystem rather than CWE type. Three memory tiers, episodic (per-CVE history), semantic (LLM-distilled patterns), and pro- cedural (concrete recipes), capture that building and exploiting a given package has consistent patterns regardless of which CWE is involved. In the CVE-GENIE dataset, 536 of 841 CVEs share a package with at least one other CVE, mak- ing cross-CVE knowledge transfer within package families a potential efficiency lever. Detection Knowledge Base. Per-CWE reference rules organized across four de- tection tiers (web, host, network, application). After each assessment, validated Sigma and Snort rules are appended alongside curated exemplars; the Detector Agent for subsequent CVEs of the same CWE retrieves these accumulated rules as generation templates. 4 Evaluation FORGE is evaluated along four dimensions that correspond to the three-silo bridge articulated in Section 1: exploitation capability (§4.2), correlation with prioritization metrics (§4.3), detection rule structural quality (§4.4), and knowl- edge transfer efficiency (§4.5). 4.1 Experimental Setup Dataset. All CVEs are drawn from the 841-entry CVE-GENIE dataset [32]. Af- ter excluding 37 native-only C/C++ memory safety CVEs and 11 lacking CWE tags, an eligible pool of 793 CVEs remains; 603 are selected via stratified di- versity sampling across CWE type, language, and project type. Native memory- corruption CVEs lie outside the targeted-application generation model and are out of scope. The dataset spans 440 GitHub repositories, 187 CWE types, and eight languages. The PackageResolver uses NVD and OSV as primary sources, with CVE-GENIE metadata supplementing resolution when available; perfor- mance on CVEs outside this distribution is not measured. Infrastructure and budget. Experiments execute on a t3.xlarge AWS EC2 in- stance (4 vCPU, 16 GB RAM) with Podman sandboxing. LLM inference uses Claude Sonnet 4.5 (agents) and GPT-5 Mini (oracle critic) via LiteLLM with OpenTelemetry token tracking. Additionally, per-CVE cost is capped at $2.50. Oracle validation. A structured audit is applied at each level: CWE-class evi- dence rules are matched against the exploit agent’s tool-result corpus and or- acle summary, yielding VALID (level reach plus evidence in both), BORDER- LINE (partial), or INVALID. Across 91 stratified assessments at L1, L2, and L3 (seed=42), 92.3% are VALID (95% CI [84.8%, 96.9%]); per-level VALID rates 10F. Shaikh L0L1L2L3 Exploitation Level 0 100 200 300 Number of CVEs 194 81 295 33 (a) Level distribution across 603 CVEs. 0 10 3 10 2 10 1 10 0 EPSS Score L0 L1 L2 L3 Exploitation Level (b) EPSS vs. exploitation level. Fig. 2. Exploitation depth and prioritization metric correlation (RQ1–RQ2). fall between 86.7% and 96.8%. One L3 case is a likely false positive (autho- rization bypass reaching L2 evidence but not full compromise); the remaining six non-VALID assessments are BORDERLINE. The audit script and per-CVE verdicts are released with the artifact. Baseline. CVE-GENIE [32] is the closest comparable system (50.9% binary suc- cess at $2.77/CVE). The two setups occupy different points on the cost-fidelity curve and are not directly comparable: CVE-GENIE reconstructs the original vulnerable project from its source tree, whereas FORGE consumes only CVE metadata and the resolved package coordinates and exploits a minimal down- stream integration that imports the vulnerable package. The binary CTF-style flag check is replaced by a graduated L0–L3 verdict, and Sigma/Snort detection rules are produced alongside exploitation. Per-CVE cost is 46% lower ($1.50 vs. $2.77). A focused head-to-head against PoCGen [28] on its npm subset is left to future work. Overlap with CVE-GENIE. Of the 429 CVEs that CVE-GENIE successfully reproduced, 316 appear in the FORGE 603-CVE subset. On this shared set, FORGE reaches L1+ on 227/316 (71.8%); generation fails on 79 (25.0%), predominantly Dockerfile build errors. On the 287 CVEs where CVE-GENIE failed, FORGE reaches L1+ on 182 (63.4%), recovering CVEs whose original- project builds resisted automated provisioning. FORGE-only successes (30.2% of the 603) and GENIE-only successes (14.8%) have minimal overlap (OR=1.47, p=0.029), indicating that the targeted-app and original-project approaches fail on different CVE subsets. 4.2 Exploitation Depth at Scale Of 603 CVEs attempted, 424 (70.3%) pass generation and deployment, and 409/603 (67.8%) reach L1+ end-to-end (Table 2, Figure 2a). Of the 424 gen- FORGE: Multi-Agent Graduated Exploitation and Detection Engineering11 Table 2. Exploitation depth across 603 CVEs. MetricValue CVEs attempted603 Generation+deployment success 424/603 (70.3%) L1+ rate (end-to-end)409/603 (67.8%) L1+ rate (of generated)409/424 (96.5%) Mean exploitation level (L1+)1.88 L3 (full compromise) rate33/424 (7.8%) Mean cost per CVE$1.50 Mean exploit turns per CVE9.5 Level distribution (of generated): L0 (no evidence)15 (3.5%) L1 (triggered)81 (19.1%) L2 (exploited)295 (69.6%) L3 (compromised)33 (7.8%) erated applications, 96.5% reach L1+, concentrated at L2 (69.6%); 91.9% of exploited CVEs fall at L1–L2, which a binary metric would collapse into “suc- cess”. Bootstrap 95% CI for mean depth across all 603 CVEs (coding generation failures as L0) is [1.20, 1.36]. Generation failure (29.7%) is the primary bottle- neck and varies sharply by language: Rust (74%), TypeScript (45%), and Ruby (39%) fail most often, while PHP (6%), C# (5%), and Java (17%) rarely fail. Figure 3 breaks the same data down by primary CWE: capped CWEs (XSS, CWE-22, CWE-352, CWE-400; marked∗) cluster at L2 by construction, while code-execution and deserialization classes (CWE-94, CWE-502, CWE-78) carry most of the L3 mass. After the L2 caps, 314/424 (74.1%) generated applica- tions are L3-eligible; the conditional L3 rate among eligible is 33/314 (10.5%). Per-CVE cost averages $1.50. 4.3 Correlation with Prioritization Metrics Given that a substantial fraction of CVEs can be exploited at graduated depth, a natural question is whether this depth correlates with existing prioritization metrics. FORGE exploitation depth is correlated with EPSS and CVSS scores across all 603 CVEs (Figure 2b). Neither metric exhibits significant rank corre- lation with exploitation level (Spearman ρ=0.062, p=0.13 for EPSS; ρ=0.016, p=0.71 for CVSS). Of 409 exploited CVEs, 389 (95.1%) carry EPSS below 0.1, although low-EPSS CVEs are not disproportionately exploitable; rather, 94.7% of all dataset CVEs fall below this threshold. The key finding emerges from per-band analysis: CVEs are exploited at∼68% regardless of EPSS band (<0.001: 68.6%, 0.001–0.01: 68.0%, 0.01–0.1: 67.7%, ≥0.1: 62.5%), and an equally flat profile is observed across CVSS severity bands. It is important to note that graduated depth measures a distinct property: how far exploitation proceeds once code-level vulnerability conditions are satisfied. 12F. Shaikh L0L1L2L3 Exploitation Level CWE-79* (n=97) CWE-200 (n=30) CWE-22* (n=26) CWE-400* (n=25) CWE-284 (n=23) CWE-20 (n=18) CWE-89 (n=17) CWE-918 (n=16) CWE-94 (n=13) CWE-1333 (n=13) CWE-770 (n=13) CWE-863 (n=10) 226720 2720530 194770 364600 5292613 4422330 470530 3112560 318854 1538460 3815460 4010500 0 10 20 30 40 50 60 70 80 % of CVEs Fig. 3. Exploitation level distribution for the 12 most frequent primary CWEs (% of CVEs in each row reaching each level). Cell shade is the within-CWE share of the L0–L3 column; rows marked with∗ are structurally capped at L2 by the oracle (§3.3). Table 3. Detection rule grounding stratified by exploitation depth. Stratum Raw Norm. L1 (n=63) 0.456 0.175 L2+ (n=293) 0.336 0.307 p (raw)0.997 (n.s.) p (norm.)0.035 The flat exploitation profile across all scoring bands provides large-scale empiri- cal evidence that metadata-based prioritization scores do not predict code-level exploitation depth. However, the dataset is drawn from a single source (CVE- GENIE) that may be homogeneous in exploitability characteristics; the flat pro- file could partially reflect dataset composition rather than a universal property of CVEs. That no predictive relationship is found suggests graduated depth con- stitutes an independent measurement axis, one that automated assessment can now produce at scale for any CVE with a package-registry entry. 4.4 Detection Rule Structural Quality Graduated depth also determines detection rule quality. For each L1+ CVE, the Detector Agent generates Sigma and Snort rules from OTEL spans collected during exploitation (Table 3). Span grounding is defined as the fraction of rule field values (URLs, commands, file paths) that appear verbatim in collected spans, indicating rules anchored in observed behavior rather than hallucinated. Raw grounding rates are confounded by observation volume: L1 CVEs gen- erate 2.4× more spans (17.0 vs. 7.1) driven by longer exploit traces (16.1 vs. 7.8 turns), mechanically inflating substring matches. The quantity of interest for any comparison where span counts differ is grounded fields per span, which answers FORGE: Multi-Agent Graduated Exploitation and Detection Engineering13 L1L2+ Exploitation Depth 0.0 0.5 1.0 1.5 Grounded fields / span 0.10 0.22 (a) Span-normalized grounding by depth (p=0.035). 0204060 Share of Total Cost (%) Exploit Generator Detector Planner 45.3% 35.7% 14.6% 4.4% (b) Per-agent cost share. Fig. 4. Detection rule quality (RQ3) and cost decomposition (RQ4). the operational question of how much detection-relevant evidence each collected span carries. Under this normalized metric, L2+ rules achieve marginally sig- nificantly higher grounding (0.307 vs. 0.175, p=0.035): each span from deeper exploitation contributes more detection-relevant evidence. Rules are generated for 380 L1+ CVEs (mean 2.0 rules/CVE). Span grounding measures structural quality (whether rule fields are anchored in observed behavior rather than hal- lucinated) and not detection efficacy. As a direct test of specificity, the 392 gen- erated Snort rules are evaluated against a 1,000-request synthetic benign HTTP corpus: 93.4% produce zero matches, with zero-FP rates of 95.6% at L1 (n=68) and 92.9% at L2+ (n=324). The remaining FP-prone rules cluster on CSRF and access-control patterns where benign state-changing POSTs are indistin- guishable from attacks without business-logic context. FP rates against benign traffic measure specificity; true-positive validation and operational deployment assessment require a replay-based SIEM evaluation. 4.5 Knowledge Reuse This dimension asks whether the knowledge architecture leads the LLM to avoid mistakes that earlier CVEs of the same package or CWE already recorded. Gen- erator failure details are parsed from run logs and canonicalized into mistake sig- natures (wrong package versions, forgotten vulnerable imports, unknown build tags, native-compile failures); mean generation attempts is close to one, so the signature grain replaces cost as the instrument. The exploitation store has accu- mulated 1,758 technique entries across 134 CWE types, the experience store cov- ers 80 multi-CVE packages across 8 ecosystems, and the cookbook store covers 12 languages. The 79 packages with two or more CVEs yield 250 prior-signature observations across 277 generator logs, of which 48.0% are avoided on the next CVE for the same package; directus/npm, evmos/v18/go, and trix/npm reach higher levels between 73–83%. At the CWE grain, 2,510 prior exploit dead-end 14F. Shaikh observations across 57 CWEs are avoided 95.5% of the time on the next CVE of the same CWE (95% CI [94.7%, 96.3%]); exploit-side transfer is markedly stronger than build-side, consistent with the CWE-keyed Tier 3 attack store. Both rates are observational; the KB-injection audit confirms stored signatures were present in the agent prompt at time-of-avoidance, but causal attribution re- quires a controlled empty-KB ablation, left to future work (§5.2). Per-CVE cost stays stable at $1.50 across the run with the breakdown shown in Figure 4b, and exploitation rates are maintained across 134 CWE types and eight languages without per-CVE cost escalation. 5 Discussion 5.1 Ethical Considerations Targeted applications generated by the system itself are exploited, and no real- world systems are targeted at any point. All exploitation is performed within isolated Podman containers with resource limits and network isolation, and only publicly disclosed CVEs with assigned identifiers are processed. Nevertheless, functional offensive artifacts (exploit scripts, payload configu- rations, attack logs) are produced alongside defensive detection rules. Although these target purpose-built applications, they constitute working exploits for the demonstrated vulnerability patterns. Dual-use risk is mitigated by three design choices: (1) targeted applications lack production defenses (WAFs, authentica- tion), requiring adaptation for real targets; (2) knowledge stores contain CWE- pattern abstractions, not target-specific chains; (3) detection rules accompany all exploitation results. 5.2 Limitations Application fidelity and scope. The targeted minimal applications preserve the vulnerability pattern but lack production complexity such as authentication, rate limiting, web application firewalls, and multi-tier architectures; depth there- fore reflects pattern-level reachability rather than deployment-specific risk. An L2 result on a FORGE-generated application demonstrates that the vulnerabil- ity class permits data exfiltration under favorable conditions, but says nothing about whether the same attack would succeed against a hardened deployment with input validation and network segmentation. Conversely, an L0 result does not imply the vulnerability is unexploitable in production — it may indicate that the generated application failed to exercise the vulnerable code path, not that the path is unreachable. Results cover 2024–2025 open-source web and li- brary CVEs from a single dataset, and generalization to proprietary software, embedded systems, or multi-component vulnerability classes is not claimed. Oracle accuracy. A structured audit across 91 stratified assessments finds 92.3% VALID overall (95% CI [84.8%, 96.9%]); separate models for agents (Claude Sonnet 4.5) and oracle (GPT-5 Mini) further mitigate feedback loops. FORGE: Multi-Agent Graduated Exploitation and Detection Engineering15 Reproducibility and knowledge attribution. Commercial LLM APIs introduce non-determinism; temperature is set to 0 for all agents except the Generator (0.2). The avoidance figures are observational across the canonical CVE order; a controlled empty-KB ablation is left to future work. 5.3 Implications Ground-truth exploitability. Automated exploitation validates EPSS predictions with behavioral evidence rather than metadata alone. The flat exploitation rate across all EPSS bands implies that metadata-based prioritization and pattern- level reachability measure orthogonal properties. The overlap analysis further indicates that targeted-app and original-project approaches fail on different CVE subsets, suggesting that combining both would maximize coverage. Detection-capability-centric risk. Generating detection rules grounded in ex- ploitation behavior transforms the detection gap from binary (rule exists or not) into a quality-graded metric. The evaluation quantifies this: L2+ exploita- tion produces rules with 75% higher span-normalized grounding than L1-derived rules, indicating that exploitation depth directly determines detection rule qual- ity and enabling organizations to assess whether their detection infrastructure can observe specific exploitation patterns. Assessment economics. At $1.50 per CVE, continuous vulnerability assessment is made economically viable: against the 48,000+ CVEs disclosed annually, full- corpus assessment would cost approximately $72K/year, although most organi- zations do not need to evaluate all CVEs. The knowledge architecture maintains consistent exploitation rates across 134 CWE types without per-CVE cost esca- lation, demonstrating both the increasing capability of LLMs and the efficiencies of the knowledge architecture. 6 Conclusion This paper presented FORGE, a multi-agent system for graduated vulnerabil- ity exploitability assessment that bridges three research communities, namely proof-of-concept generation, vulnerability prioritization, and detection engineer- ing. The key technical contributions are a four-level exploitation taxonomy (L0– L3) evaluated by an LLM-primary oracle with CWE-specific structural caps, targeted vulnerable application generation that eliminates benchmark container dependency, and a tiered knowledge architecture that accumulates intelligence across CVE assessments. Evaluation on 603 CVEs from the CVE-GENIE dataset demonstrates 67.8% end-to-end L1+ exploitation (409/603) at $1.50 per CVE across eight languages and 187 CWE types. Graduated depth reveals that exploitation rates remain near 68% across all EPSS and CVSS bands, indicating that pattern-level reach- ability is orthogonal to metadata-based prioritization. Detection rules from L2+ 16F. Shaikh exploitation achieve higher span-normalized grounding (p=0.035), and within- package signature analysis shows 48% avoidance of previously recorded build- failure modes. Future work can proceed along three dimensions. First, adaptive pipeline routing (skipping the Planner for well-characterized CWE types) could improve throughput and depth. Second, progressively increasing application complexity beyond minimal reproducers would close the fidelity gap with production envi- ronments. Third, longitudinal operation on newly disclosed CVEs would measure knowledge accumulation over months and could feed graduated depth data back to EPSS calibration as a community resource. All code, per-CVE results, oracle audit verdicts, and analysis scripts are released under an Apache 2.0 license at https://github.com/dynatrace-oss/forge and https://github.com/dynatrace-oss/forge-artifacts. Disclosure of Interests. The authors have no competing interests to declare that are relevant to the content of this article. References 1. Al Haddad, O., Ikram, M., Ahmed, E., Lee, Y.: Prompting the priorities: A first look at evaluating LLMs for vulnerability triage and prioritization. arXiv preprint arXiv:2510.18508 (2025), https://arxiv.org/abs/2510.18508 2. Applebaum, A., Miller, D., Strom, B., Korban, C., Wolf, R.: Intelligent, automated red team emulation. In: Proceedings of the 32nd Annual Conference on Computer Security Applications (ACSAC). p. 363–373 (2016). https://doi.org/10.1145/29 91079.2991111 3. Brumley, D., Newsome, J., Song, D., Wang, H., Jha, S.: Towards automatic gen- eration of vulnerability-based signatures. In: Proceedings of the 2006 IEEE Sym- posium on Security and Privacy (S&P). p. 2–16 (2006). https://doi.org/10.1109/ sp.2006.41 4. Bui, Q.C., Iannone, E., Camporese, M., Hinrichs, T., Tony, C., Tóth, L., Palomba, F., Hegedűs, P., Massacci, F., Scandariato, R.: A systematic literature review on automated exploit and security test generation. arXiv preprint arXiv:2502.04953 (2025), https://arxiv.org/abs/2502.04953 5. CVE.org: CVE metrics. https://w.cve.org/About/Metrics (2025), accessed: February 2026 6. Deng, G., Liu, Y., Mayoral-Vilches, V., Liu, P., Li, Y., Xu, Y., Zhang, T., Liu, Y., Pinzger, M., Rass, S.: PentestGPT: Evaluating and harnessing large language models for automated penetration testing. In: Proceedings of the 33rd USENIX Security Symposium. p. 847–864 (2024), https://w.usenix.org/conference/us enixsecurity24/presentation/deng 7. Denison, C., MacDiarmid, M., Barez, F., Duvenaud, D., Kravec, S., Marks, S., Schiefer, N., Soklaski, R., Tamkin, A., Kaplan, J., Shlegeris, B., Bowman, S.R., Perez, E., Hubinger, E.: Sycophancy to subterfuge: Investigating reward-tampering in large language models. arXiv preprint arXiv:2406.10162 (2024), https://arxiv. org/abs/2406.10162 8. Elder, S., Rahman, M.R., Fringer, G., Kapoor, K., Williams, L.: A survey on software vulnerability exploitability assessment. ACM Computing Surveys 56(8), 1–41 (2024). https://doi.org/10.1145/3648610 FORGE: Multi-Agent Graduated Exploitation and Detection Engineering17 9. Fairbanks, J., Serra, E.: Reflective beam search for automated TTP extraction and sigma rule generation from cyber threat intelligence. In: IEEE International Conference on Big Data (BigData). p. 2130–2135 (2025). https://doi.org/10.110 9/bigdata66926.2025.11401712 10. Fang, R., Bindu, R., Gupta, A., Kang, D.: LLM agents can autonomously exploit one-day vulnerabilities. arXiv preprint arXiv:2404.08144 (2024), https://arxiv.or g/abs/2404.08144 11. Fleischer, F., Zhang, C., Jang, J., Cho, J., Xu, M., Kim, T.: Contextualizing sink knowledge for Java vulnerability discovery. arXiv preprint arXiv:2604.01645 (2026), https://arxiv.org/abs/2604.01645 12. Gordeychik, S.: Prediction meets patch queues: Empirical limits of EPSS-only prioritization using CISA KEV additions in 2025. TechRxiv preprint (2026). https://doi.org/10.36227/techrxiv.176857939.95987957/v1 13. Jacobs, J., Romanosky, S., Adjerid, I., Baker, W.: Improving vulnerability reme- diation through better exploit prediction. Journal of Cybersecurity 6(1) (2020). https://doi.org/10.1093/cybsec/tyaa015 14. Jacobs, J., Romanosky, S., Suciu, O., Edwards, B., Sarabi, A.: Enhancing vul- nerability prioritization: Data-driven exploit predictions with community-driven insights. In: IEEE European Symposium on Security and Privacy Workshops (Eu- roS&PW). p. 194–206 (2023). https://doi.org/10.1109/EuroSPW59978.2023.00 027 15. Kobayashi, M., Kanemoto, Y., Kotani, D., Okabe, Y.: Generation of IDS signatures through exhaustive execution path exploration in PoC codes for vulnerabilities. Journal of Information Processing 31, 591–601 (2023). https://doi.org/10.2197/ip sjjip.31.591 16. Koscinski, V., Nelson, M., Okutan, A., Falso, R., Mirakhorli, M.: Conflicting scores, confusing signals: An empirical study of vulnerability scoring systems. In: Proceed- ings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS). p. 1904–1918 (2025). https://doi.org/10.1145/3719027.3765210 17. Lau, N., Sloot, L., Raj, J., Boscardin, G.M., Harris, E., Bowman, D., Brajkovski, M., Chawla, J., Zhao, D.: ZeroDayBench: Evaluating LLM agents on unseen zero- day vulnerabilities for cyberdefense. arXiv preprint arXiv:2603.02297 (2026), https: //arxiv.org/abs/2603.02297 18. Li, H., Che, X., Wang, Y., Liao, X., Xing, L.: Execution-state-aware LLM reasoning for automated proof-of-vulnerability generation. arXiv preprint arXiv:2602.13574 (2026), https://arxiv.org/abs/2602.13574 19. Liang, Z., Sekar, R.: Fast and automated generation of attack signatures: A basis for building self-protecting servers. In: Proceedings of the 12th ACM Conference on Computer and Communications Security (CCS). p. 213–222 (2005). https: //doi.org/10.1145/1102120.1102150 20. Liu, B., Zhao, Y., Xu, G., Wang, H.: LLM agents for automated web vulnerability reproduction: Are we there yet? arXiv preprint arXiv:2510.14700 (2025), https: //arxiv.org/abs/2510.14700 21. Luo, X., Zhang, J., Zhou, S., Huang, R., Xiao, C., Zhu, Q., Ma, Z., Yue, X., Yue, Y., Zeng, W., Che, W.: CVE-Factory: Scaling expert-level agentic tasks for code security vulnerability. arXiv preprint arXiv:2602.03012 (2026), https://arxiv.org/ abs/2602.03012 22. Mell, P., Spring, J.: Likely exploited vulnerabilities. Tech. Rep. NIST.CSWP.41, National Institute of Standards and Technology (2025), https://nvlpubs.nist.gov /nistpubs/CSWP/NIST.CSWP.41.pdf 18F. Shaikh 23. Mitra, S., Bazarov, A., Duclos, M., Mittal, S., Piplai, A., Rahman, M.R., Zieglar, E., Rahimi, S.: FALCON: Autonomous cyber threat intelligence mining with LLMs for IDS rule generation. arXiv preprint arXiv:2508.18684 (2025), https://arxiv.or g/abs/2508.18684 24. Newsome, J., Song, D.: Dynamic taint analysis for automatic detection, anal- ysis, and signature generation of exploits on commodity software. In: Pro- ceedings of the 12th Network and Distributed System Security Symposium (NDSS) (2005), https://w.ndss-symposium.org/ndss2005/dynamic-taint-anal ysis-automatic-detection-analysis-and-signaturegeneration-exploits-commodity/ 25. Parla, R.: Efficacy of EPSS in high severity CVEs found in KEV. arXiv preprint arXiv:2411.02618 (2024), https://arxiv.org/abs/2411.02618 26. Sánchez-Matas, A., Escribano Ruiz, P., Díaz-López, D., Perales Gómez, A.L., Ne- spoli, P., Martínez Pérez, G.: Simulating cyberattacks through a breach attack simulation (BAS) platform empowered by security chaos engineering (SCE). arXiv preprint arXiv:2508.03882 (2025), https://arxiv.org/abs/2508.03882 27. Shukla, A., Gandhi, P.A., Elovici, Y., Shabtai, A.: RuleGenie: SIEM detection rule set optimization. arXiv preprint arXiv:2505.06701 (2025), https://arxiv.org/abs/ 2505.06701 28. Simsek, D., Eghbali, A., Pradel, M.: PoCGen: Generating proof-of-concept exploits for vulnerabilities in npm packages. arXiv preprint arXiv:2506.04962 (2025), https: //arxiv.org/abs/2506.04962 29. Smart, J., Jun, S.Y., et al.: Blueprint: Stakeholder-Specific Vulnerability Categorization guidance. Tech. rep., Sandia National Laboratories (2026), https://research-hub.nlr.gov/en/publications/blueprint-stakeholder-specific-vul nerability-categorization-guida/ 30. Tran, T.T.V., Le, T.B.T., Truong, T.H.H., Thai, H.V., Hien, D.H., Phan, T.D.: EvoSIEM: Detecting and generating SIEM rule evasion behaviors in network sys- tems. In: IEEE RIVF International Conference on Computing and Communication Technologies. p. 498–503 (2025). https://doi.org/10.1109/rivf68649.2025.1136512 9 31. Uetz, R., Herzog, M., Hackländer, L., Schwarz, S., Henze, M.: You cannot escape me: Detecting evasions of SIEM rules in enterprise networks. In: Proceedings of the 33rd USENIX Security Symposium. p. 5179–5196 (2024), https://w.usen ix.org/conference/usenixsecurity24/presentation/uetz 32. Ullah, S., Balasubramanian, P., Guo, W., Burnett, A., Pearce, H., Kruegel, C., Vigna, G., Stringhini, G.: From CVE entries to verifiable exploits: An automated multi-agent framework for reproducing CVEs. arXiv preprint arXiv:2509.01835 (2025), https://arxiv.org/abs/2509.01835 33. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., Cao, Y.: ReAct: Synergizing reasoning and acting in language models. In: Proceedings of the 11th International Conference on Learning Representations (ICLR) (2023), https://ar xiv.org/abs/2210.03629 34. Zhu, Y., Kellermann, A., Bowman, D., Li, P., Gupta, A., Danda, A., Fang, R., Jensen, C., Ihli, E., Benn, J., Geronimo, J., Dhir, A., Rao, S., Yu, K., Stone, T., Kang, D.: CVE-Bench: A benchmark for AI agents’ ability to exploit real-world web application vulnerabilities. In: Proceedings of the 42nd International Conference on Machine Learning (ICML) (2025), https://arxiv.org/abs/2503.17332 35. Zhu, Y., Kellermann, A., Gupta, A., Li, P., Fang, R., Bindu, R., Kang, D.: Teams of LLM agents can exploit zero-day vulnerabilities. arXiv preprint arXiv:2406.01637 (2024), https://arxiv.org/abs/2406.01637