Paper deep dive
SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs
Hanna Kim, Jian Cui, Minkyoo Song, Hwanjo Heo, Seungwon Shin, Kimin Lee, Xiaojing Liao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 9/9/2026, 5:55:58 AM
Summary
The paper introduces SCRIPTIOC-BENCH, a benchmark for evaluating Large Language Models (LLMs) on static Indicators of Compromise (IOC) extraction from script-based malware (JavaScript, PowerShell, VBScript). The benchmark consists of 634 manually verified samples with ground-truth IOCs categorized by recovery level (Direct, Fixed-transform, Program-dependent). The study evaluates various LLMs, finding that while strong models perform well on direct IOCs, performance drops significantly for program-dependent IOCs. The authors also identify failure modes and propose mitigations like deterministic string utilities and task-specific adaptation to improve precision.
Entities (46)
Relation Signals (37)
SCRIPTIOC-BENCH → containssamplesfrom → Hynek Petrak
confidence 95% · and historical JavaScript samples from the Hynek Petrak collection
SCRIPTIOC-BENCH → containssamplesfrom → Filescan.io
confidence 95% · Samples are drawn from two complementary sources: recent scripts collected from Filescan.io
SCRIPTIOC-BENCH → evaluates → LLM
confidence 95% · Using this benchmark, we evaluate a broad range of proprietary and open-weight LLMs
SCRIPTIOC-BENCH → extractsioctypes → Filesystem Artifact
confidence 95% · covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts)
SCRIPTIOC-BENCH → extractsioctypes → Domain
confidence 95% · covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts)
SCRIPTIOC-BENCH → extractsioctypes → URL
confidence 95% · covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts)
SCRIPTIOC-BENCH → extractsioctypes → IP Address
confidence 95% · covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts)
SCRIPTIOC-BENCH → targetslanguages → VBScript
confidence 95% · The benchmark comprises 634 manually verified JavaScript, PowerShell, and VBScript malware samples
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence. However, statically recovering such indicators is challenging, as relevant values may be dispersed or transformed within code. Although large language models (LLMs) have shown promise in security analysis, their ability to recover IOCs from malicious scripts remains underexplored. We present SCRIPTIOC-BENCH, a benchmark for measuring static IOC extraction capability on real-world malicious scripts. The benchmark comprises 634 manually verified JavaScript, PowerShell, and VBScript malware samples covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts). We further stratify ground-truth IOCs by recovery level, distinguishing directly exposed indicators from those requiring decoding or reconstruction. Using this benchmark, we evaluate a broad range of proprietary and open-weight LLMs and show that IOC recovery without execution remains challenging across model scales: the strongest model reaches only 65.4 F1. To characterize how recovery fails, we introduce a false-positive taxonomy and use it to compare the error profiles of the evaluated models. We further study two mitigations on a small open-weight model, deterministic string utilities and task-specific adaptation, finding that they provide complementary recovery gains, raise precision, and shift errors toward sample-grounded mismatches.
Tags
Links
- Source: https://arxiv.org/abs/2609.06149v1
- Canonical: https://arxiv.org/abs/2609.06149v1
Trouble viewing inline? Open PDF directly →
Full Text
104,902 characters extracted from source content.
Expand or collapse full text
SCRIPTIOC-BENCH: A Benchmark for Recognizing Actionable Threat Intelligence from Script-Based Malware using LLMs Hanna Kim 1 Jian Cui 2 Minkyoo Song 1 Hwanjo Heo 3 Seungwon Shin 1 Kimin Lee 1,∗ Xiaojing Liao 2,∗ 1 KAIST 2 University of Illinois Urbana-Champaign 3 ETRI Abstract Script-based malware remains a prevalent attack technique. These scripts often contain indicators of compromise (IOCs) that provide actionable threat intelligence. However, statically recovering such indicators is challenging, as relevant values may be dispersed or transformed within code. Although large language models (LLMs) have shown promise in security analysis, their ability to recover IOCs from malicious scripts remains underexplored. We present SCRIPTIOC-BENCH, a benchmark for mea- suring static IOC extraction capability on real-world mali- cious scripts. The benchmark comprises 634 manually veri- fied JavaScript, PowerShell, and VBScript malware samples covering four IOC types (URLs, domains, IP addresses, and filesystem artifacts). We further stratify ground-truth IOCs by recovery level, distinguishing directly exposed indicators from those requiring decoding or reconstruction. Using this benchmark, we evaluate a broad range of proprietary and open- weight LLMs and show that IOC recovery without execution remains challenging across model scales: the strongest model reaches only 65.4 F1. To characterize how recovery fails, we introduce a false-positive taxonomy and use it to compare the error profiles of the evaluated models. We further study two mitigations on a small open-weight model, deterministic string utilities and task-specific adaptation, finding that they provide complementary recovery gains, raise precision, and shift errors toward sample-grounded mismatches. 1 Introduction Script-based malware is an important and widely used attack technique in real-world intrusions. By abusing built-in com- mands and scripting interpreters, adversaries can flexibly exe- cute malicious logic, retrieve follow-on payloads, and interact with the host environment without relying on custom binaries ∗ Corresponding authors. in the initial stage. Consequently, scripts such as PowerShell and JavaScript often function as primary execution or deliv- ery artifacts in cybersecurity attacks [8, 28, 37]. At the same time, these scripts often expose information about attacker- controlled infrastructure and accessed resources, making them a valuable source of indicators of compromise (IOCs) [39]. IOCs refer to artifacts that indicate malicious activity, includ- ing URLs and domains, that can support detection, triage, and threat hunting. Recovering such indicators is therefore an im- portant goal of malware analysis because it enables defenders to move from the analysis of a single sample to the identi- fication of related infections and the disruption of broader malicious activity [26, 29]. Static IOC recovery. IOC extraction imposes a practical scale constraint. Large analysis platforms report processing roughly 1.2M previously unseen files per day, making scalable automated analysis nontrivial [17]. Given the large volume of script samples encountered in practice, scalable first-line screening is essential. Static analysis has been widely used for this purpose because it enables script artifacts to be exam- ined without execution. Static IOC recovery, however, is more difficult than searching source code for IOC-shaped strings. Malicious scripts routinely split values across literals, encode them through transformations, or reconstruct them through script-specific computation. Recovering such indicators may therefore require reasoning over dispersed program context. This creates a natural opportunity for large language models (LLMs), whose code-understanding capabilities may enable execution-free analysis beyond conventional pattern match- ing. Yet existing LLM-based malware studies largely focus on adjacent tasks such as code deobfuscation or behavioral analysis [5, 7, 11, 12, 61]. The capabilities and limitations of current LLMs for static IOC recovery therefore remain under-explored. Our study. Against this backdrop, we formulate the following research questions: RQ1. How reliably can LLMs recover stat- ically obtainable IOCs from malicious scripts? RQ2. Which forms of recovery remain challenging, from extracting ex- plicit values to sample-specific reconstruction? RQ3. How 1 arXiv:2609.06149v1 [cs.CR] 5 Sep 2026 do script characteristics such as length affect IOC extraction reliability? RQ4. What failure modes emerge across model capabilities, and to what extent can they be mitigated? Addressing these questions requires a benchmark grounded in real-world scripts, so that evaluation reflects the recovery challenges encountered in practice. Yet no existing dataset provides ground truth restricted to statically obtainable IOCs, as reports from malware-analysis platforms often combine static and dynamic evidence without clearly distinguishing their provenance. Moreover, indicator-level annotation should characterize how each IOC can be recovered, since different indicators may require different forms of recovery even within a single script. We therefore present SCRIPTIOC-BENCH, the first bench- mark for measuring static IOC extraction from real-world script-based malware. It contains 634 manually verified JavaScript, PowerShell, and VBScript samples covering four actionable IOC types: URLs, domains, IP addresses, and filesystem artifacts. Ground-truth indicators are retained only when their values are fully determined by the script file alone. We further annotate each IOC by its recovery level: Direct, when the value appears explicitly in the source; Fixed- transform, when it can be recovered using common determin- istic transformations; and Program-dependent, when recovery requires following logic specific to the program. Findings. Using SCRIPTIOC-BENCH, we evaluate open- weight LLMs [16, 33, 36, 44–47] from 8B to 480B parameters alongside frontier proprietary models [15, 40], and identify three main findings. First, strong LLMs recover directly ex- posed IOCs well, but performance drops sharply as recovery requires deeper reconstruction. (RQ1-RQ2; §5.3.1-§5.3.2) Although the leading LLMs substantially outperform surface- based extractors, even the strongest model reaches only 65.4 F1. Frontier models recover up to 91.7% of Direct IOCs but only 31–46% of Program-dependent IOCs, showing that re- covery through sample-specific program logic remains partic- ularly challenging. Second, input complexity can limit reliable IOC recovery. (RQ3; §5.3.3) Longer scripts generally make LLMs more likely to produce incomplete or unusable outputs and less likely to recover indicators. Such extremely long and redundant samples are common in script-based malware, making inflated context length a practical obstacle to reliable LLM-based IOC recovery. Third, smaller models fail differ- ently, and the resulting failure modes are actionable. (RQ4; §5.4-§6) When recovery becomes difficult, smaller models fre- quently fall back to IOC examples from the prompt, whereas frontier models more often produce sample-grounded recon- struction errors. We target these weaknesses in a controlled study on a small open-weight model using deterministic string utilities and task-specific adaptation: tools improve ground- ing and Direct recovery, while adaptation learns recurring reconstruction patterns and improves Fixed-transform recov- ery. Together, the two approaches raise precision from 31% to 48% while substantially reducing unsupported predictions. Contributions. Our main contributions are as follows: • We introduce SCRIPTIOC-BENCH, a benchmark for static IOC extraction from script-based malware, cover- ing JavaScript, PowerShell, and VBScript with four IOC types over 634 manually verified samples. • We systematically evaluate open-weight and frontier LLMs across recovery difficulty and input complexity, revealing a substantial gap between direct extraction and deeper reconstruction, as well as degraded reliability on long inputs. • We identify distinct failure modes across model scales through a fine-grained false-positive taxonomy, and show that tool augmentation and task-specific adaptation shift a small model’s errors toward sample-grounded errors. Artifact and Leaderboard. We release our datasets and code, and provide a leaderboard; a screenshot is included in Ap- pendix A. 2 Background and Related Work 2.1 Script-based Malware and IOC Recovery Script-based Malware. Script-based malware refers to malicious code delivered as interpreted scripts, primarily JavaScript, PowerShell, and VBScript. These scripts com- monly serve as the initial-access or early execution stage in multi-step attacks. By abusing built-in interpreters and trusted system binaries, adversaries can execute commands, down- load payloads, and prepare follow-on activity while reducing their reliance on custom binaries [2]. The operational impact is significant. Campaigns involving script-based loaders such as Emotet, Qakbot, and TrickBot have infected millions of hosts and enabled ransomware payouts in the hundreds of millions of dollars [53–56]. In 2024, 79% of detections were malware-free, reflecting the continued shift toward script- and living-off-the-land–based intrusions [9]. To evade detection, script-based malware is often heavily obfuscated [38] using string encoding, control-flow flattening, and runtime evalua- tion, such as eval or Invoke-Expression [3, 13, 60]. Indicators of Compromise. The primary actionable threat intelligence exposed by these scripts is indicators of com- promise (IOCs): URLs, domains, IP addresses, and filesys- tem artifacts that reveal attacker-controlled infrastructure or host-side traces [26, 41]. Recovering such indicators helps defenders connect a single sample to related infections and disrupt broader campaigns [22, 29]. However, IOCs in ob- fuscated scripts rarely appear as plain literals. Instead, they may be split across strings, encoded, guarded by conditional logic, or reconstructed through runtime-evaluated expressions. As a result, IOC recovery often requires reasoning over dis- persed code context rather than applying surface-level pattern matching alone. 2 IOC Extraction. IOC extraction is commonly approached through dynamic or static analysis. Dynamic analysis exe- cutes a script in an instrumented sandbox to observe runtime artifacts such as resolved URLs, dropped files, and executed commands [4, 25, 27, 52]. Although effective, sandboxing is costly at scale and can be evaded by anti-analysis tech- niques such as virtualization detection and time-delayed ex- ecution [1, 6, 30]. Static analysis provides an execution-free alternative that scales to large corpora and remains applicable when a script refuses to execute. Prior work has used abstract syntax trees (ASTs), lexical features, and AST rewriting for script detection and deobfuscation [3, 20, 28, 62], but reliable IOC extraction from heavily obfuscated scripts remains diffi- cult. LLMs have recently emerged as an execution-free alter- native, given their strong code-reasoning capability. However, existing LLM-based studies mainly target adjacent tasks such as deobfuscation [5, 12] and behavior auditing [11, 19, 61], or address only a single campaign [42]. Static IOC extrac- tion across multi-language scripts and campaigns therefore remains largely under-explored. 2.2 Related Work Script-based Malware Analysis. Early work on script-based malware analysis applies static and dynamic program analysis either to detect that a script is obfuscated or to recover its un- derlying semantics. For PowerShell, Revoke-Obfuscation [3] treats obfuscation as binary classification, training logistic regression over AST-derived statistical features. In an effort to recover the underlying semantics, the literature succes- sively focuses on finer program units: Li et al. [28] emulate individual AST subtrees in isolation; PowerDrive [52] hooks the PowerShell interpreter’s evaluation cmdlets at runtime; and PowerPeeler [27] traces execution at the instruction level. Similarly, for JavaScript, SAFE-Deobs [20] applies static com- piler optimizations such as constant folding and dead-code elimination, and JSimplifier [62] adds dynamic tracing and LLM-based identifier renaming across 44K real samples. A more recent line applies LLMs directly to malicious- script reasoning. Fang et al. [12] systematically evaluate fron- tier LLMs on general code understanding and JavaScript deobfuscation. JsDeObsBench [5] establishes a focused benchmark for evaluating LLMs’ capability in deobfuscating JavaScript, under zero- and few-shot prompting. CyberSOCE- val [11], MalEval [61], and Cama [19] benchmark prompted LLMs on SOC-style threat reasoning, fine-grained malware behavior auditing, and Android function identification, respec- tively. However, all of these efforts target outputs other than direct actionable IOCs (deobfuscated code, behavior sum- maries, etc.), and they also evaluate only vanilla LLMs on a single language family. Threat Intelligence Extraction. Early studies on threat intelligence extraction leverage classical NLP approaches: iACE [29] and TTPDrill [22] extract OpenIOC items and Predicted IOCs URLdomainIP file LLM Evaluator Benchmark Corpus Script Extract IOCs from the script Task description Context handling IOCRecov. level abc.com abc.com/f1 Tmp/i.png Ground-Truth IOCs Analysis direct fixed-transf. program-dep. Overall perf. Perf.by IOCrecov.level Perf. by scriptcomplexity Failure modes Figure 1: Overview of SCRIPTIOC-BENCH. Given a task description and a malicious script, an LLM extracts statically obtainable IOCs, which are evaluated against ground-truth IOCs. We analyze model performance along four dimensions corresponding to our research questions: overall performance (RQ1), IOC recovery level (RQ2), input complexity (RQ3), and failure modes (RQ4). structured TTPs from technical articles via stable grammat- ical relations and information-retrieval techniques. Recent studies focus on LLM-driven pipelines [7, 51]: they construct CTI knowledge graphs from natural-language reports and ingest real-world CTI feeds at scale. Closest to our setting, Patsakis et al. [42] evaluate frontier LLMs on PowerShell IOC extraction over a single Emotet campaign, with the best- performing model reaching 69.6%/88.8% accuracy on URL- s/domains. Our work, however, provides a comprehensive benchmark built from real-world malicious scripts across three script languages and four IOC types. We systemati- cally evaluate model performance across recovery difficulty and conduct a fine-grained analysis of their failure modes. 2.3 Problem Scope This work studies static IOC extraction from script-based malware using LLMs. The analysis is restricted to static rea- soning over the script source. We define an IOC as stati- cally recoverable when its value is fully determined by the script and embedded data, without execution or runtime input. This includes values obtained through decoding, string re- construction, constant propagation, or bounded deterministic computation. Some recoverable IOCs may still require inter- preting program-specific logic beyond fixed transformations. We focus on three of the widely used scripting languages: JavaScript, PowerShell, and VBScript; and four actionable IOC categories: URLs, domains, IP addresses, and filesystem artifacts. 3SCRIPTIOC-BENCH In this section, we present SCRIPTIOC-BENCH, a benchmark for static IOC extraction from script-based malware. Figure 1 presents an overview of our framework. 3 Overview. The framework consists of three main compo- nents: task input, LLM-based IOC extraction, and evalua- tion. SCRIPTIOC-BENCH provides a corpus of labeled ma- licious script samples spanning JavaScript, PowerShell, and VBScript, where each instance pairs a script with ground- truth IOC labels, each further annotated with a recovery level (§3.4). Given a script and task description, the LLM produces IOC predictions, which are scored against the ground truth, with further analysis by recovery level, input complexity, and false-positive profiles. The remainder of this section details these components: the task and its input corpus (§3.1, §3.2), the inference setup (§3.3), and the evaluation framework (§3.4). 3.1 Task Formulation Task Definition. For each input, the model is given a natural- language task description and a malicious script sample. The model is instructed to statically extract IOCs from the script without executing it and is restricted to output a structured set of indicators. Since such indicators are often obfuscated, encoded, or dispersed across the code, successful extraction requires reconstructing them from script-level context. The task can be formally defined as follows: Given an inputx = (d, s), whereddenotes the task description andsdenotes the script sample, the model outputs a set of typed IOCs: ˆ Y = f θ (d, s). Each predicted IOC is represented as a pair(v, t), where vis the extracted IOC value and its IOC type,t ∈ URL, DOMAIN, IP, FILE. 3.2 Benchmark Composition Target Script Languages. We focus on JavaScript, Power- Shell, and VBScript, three widely used scripting languages in real-world malware [21, 48]. These three languages capture distinct but complementary features under different script- execution settings relevant to real-world malicious activity. •JavaScript (JS) is a general-purpose interpreted scripting language that operates across browsers and other runtime environments. In malware, its portability and flexibility make it suitable for lures, lightweight loaders, and staged payload delivery across multiple execution contexts. • PowerShell (PS) is a command-line shell and scripting lan- guage designed for system administration and automation, particularly in Windows environments. Because it is natively available on many Windows systems and provides access to system functionality, it is frequently abused for command ex- ecution, payload staging, and post-compromise activity while blending with legitimate administrative behavior. •VBScript (VBS) is a lightweight Visual Basic–derived script- ing language commonly executed through Windows Script Host. Despite being an old Windows component, it remains relevant in malware because it is still available on modern Windows systems and can be used as a simple loader or drop- per to launch additional tools. Recent malware campaigns have also continued to use VBScript as an initial delivery or execution component [35]. Target IOC Types. In this work, we define an IOC as any ma- liciously relevant artifact recoverable from the script through static analysis, including artifacts embedded as literals and those obtained through static decoding, deobfuscation, or string reconstruction. We focus on four IOC types: URLs, domains, IP addresses, and filesystem artifacts. These categories serve as actionable threat intelligence, capturing different aspects of malicious activity, such as network infrastructure and host-side traces. Furthermore, they usually appear in script-based malware: in payload download commands, command-and-control refer- ences, persistence logic, and file-write operations. For example, a script-based downloader may embed a URL or domain used to retrieve a secondary payload, or a hard- coded IP address used to contact attacker-controlled infras- tructure. It may also specify a file path in a temporary di- rectory or user profile to store the downloaded payload or maintain persistence. Even when such artifacts are split across multiple strings, encoded, or lightly obfuscated, they are of- ten recoverable through purely static analysis, making them suitable targets for extraction. Dataset. The benchmark data contains 634 manually verified malicious script samples across the three target languages. Samples are drawn from two complementary sources: recent scripts collected from Filescan.io [14] (Nov 2025–Mar 2026) covering JS, PS, and VBS; and historical JavaScript samples from the Hynek Petrak collection (2015–2017) [43], which form a separate subset capturing earlier obfuscation styles. Two security experts construct the ground truth by direct static analysis of each script, keeping only statically recoverable indicators. Each ground-truth IOC is further annotated with an IOC recovery level—Direct, Fixed-transform, or Program- dependent—according to whether its value appears verbatim, requires a common deterministic transformation, or must be reconstructed from program-specific logic. Table 1 summa- rizes the composition by source, time period, and language. Detailed sample collection, annotation protocol, and corpus- level analysis are presented in §4. 3.3 LLM Inference Setup LLM serves as the inference component of SCRIPTIOC- BENCH. Given a task description and a malicious script, the model identifies IOCs from the script, including values that may require resolving indirect references or deobfuscation. The model response is parsed into a structured set of predicted IOCs for evaluation. Context handling. Many malicious script samples exceed the context window of typical LLMs (§4.2), and a single-pass 4 prompt would truncate the input and discard security-relevant code. To address this, we use a standard map–reduce-style procedure for such cases. Scripts that fit within the context window are processed directly; longer scripts are split into token-bounded chunks, processed independently, and their intermediate findings are subsequently aggregated to produce the final output. 3.4 Evaluation Framework Metrics. The LLM’s capability of extracting IOCs is eval- uated on per-IOC precision, recall, and F1 score, computed independently for each IOC type; the detailed matching rules are described in §5.2.3. We evaluate models along four diagnostic axes. • Overall performance (RQ1). We report aggregate pre- cision, recall, and F1, and compare against regex-based baselines to assess the overall reliability of LLM-based IOC recovery. • IOC recovery level (RQ2). Different recovery levels place different demands on the model, so we analyze per- formance across Direct, Fixed-transform, and Program- dependent indicators to identify which forms of recovery remain challenging. • Input complexity (RQ3). To assess robustness under in- creasingly demanding inputs, we analyze extraction quality across the input-size and code-redundancy measures. • Failure modes (RQ4). Different failure modes impose dif- ferent verification costs and call for different mitigations, so we characterize false positives using the taxonomy and examine how error types vary across models. 4 Datasets This section details the construction and analysis of the SCRIPTIOC-BENCH dataset. §4.1 describes how malware samples were collected and how their IOC labels were ver- ified under category-specific annotation rules. In §4.2, we detail the analysis of the resulting dataset along two aspects: input complexity and IOC label distribution. 4.1 Dataset Construction Malicious Script Collection. We collect malware samples from two sources, Filescan.io [14] and Hynek Petrak [43], spanning Nov. 2025–Mar. 2026 and 2015–2017, respectively. Specifically, we first collected newly-observed JS, PS, and VBS malware samples from Filescan.io [14], a malware database that stores analyzed samples and assigns quality labels, e.g., confirmed threat. We randomly sampled scripts marked as high risk or confirmed threat that exposed at least one target IOC and were available for download. Table 1: Dataset composition by source, collection period, and script language. SourcePeriodLang. # Samples Dataset Filescan.io Nov. 2025–Feb. 2026JS97 JS f Jan. 2026–Mar. 2026PS75 PS f Jan. 2026–Mar. 2026VBS79 VBS f Hynek Petrak2015–2017JS383 JS h p We additionally incorporated malicious JavaScript sam- ples from the JS Malware Collection released by Hynek Pe- trak [43]. This collection contains real-world JavaScript mal- ware samples from 2015 to 2017. We randomly sampled a subset of this collection to expand the JavaScript portion of the dataset. Details of the dataset are summarized in Table 1. IOC Annotation. Evaluating static IOC extraction requires that every labeled indicator actually be recoverable from the script alone. Since no existing source provides labels curated for static recoverability, by following established annotation process [32, 58], we construct the ground truth through expert manual static analysis. Two security experts manually ana- lyze each script, deobfuscating when necessary, to identify and verify each IOC directly from the source. The ground truth retains only statically obtainable indicators: values fully determined by the script and its embedded data. Static recover- ability does not require cleartext presence: retained indicators may require multi-step deobfuscation, but values that depend on runtime or external state are excluded. The annotation proceeds in three stages. The experts first independently review a random subset of 32 scripts (about 5% of the 634 samples) to identify statically recoverable IOCs and develop initial annotation guidelines. The guidelines spec- ify the annotation scope and category-specific decision rules, including inclusion and exclusion criteria for each IOC type. Using these initial guidelines, they independently annotate an- other random subset of 32 scripts, compare their annotations, resolve disagreements, and refine the guidelines to clarify ambiguous cases. They then annotate the remaining samples under the finalized guidelines to obtain the final ground-truth IOC set. Throughout annotation, IOC reports from four malware analysis platforms (Filescan.io, VirusTotal [57], Triage [18], and Hybrid Analysis [23]) were consulted only as auxiliary references for cross-checking candidate indicators: such re- ports mix statically observable artifacts with dynamic-only results and platform-specific enrichment, and may miss in- dicators when malware evades sandbox execution. Retained artifacts follow the category-specific annotation rules below. For URLs, we annotate both fully specified URLs and URLs without a scheme when a host and path are explic- itly present in the script. URLs hosted on benign shared services (e.g., api.telegram.org) are retained in the URL- level ground truth when they are presumed to be used mali- 5 ciously, such as for payload delivery. Conversely, URLs that are themselves benign, e.g., pointing to legitimate service content rather than attacker-staged resources, are not added; benignness is assessed using VirusTotal [57]. For domains, those belonging to these abused legitimate services are ex- cluded from the domain-level ground truth because they are not attacker-controlled, even when URLs hosted on them are retained at the URL level. For filesystem artifacts, we define an artifact as a file or directory on the host that is the stati- cally identifiable target of a creation, drop, extraction, copy, move, or rename operation in the script. These include full and partial paths, as well as file or directory names whose parent paths cannot be determined. Indicators reconstructed through constant propagation or string concatenation are also included. Environment-variable references such as%TEMP% and%APPDATA%are retained in symbolic form rather than resolved to machine-specific paths. IOC Recovery Levels. As a final labeling step, we assign each ground-truth IOC a recovery level to identify where models succeed and where they struggle in IOC extraction. A model may recover an IOC simply by reading a surface string, by undoing a common encoding, or by reasoning through script-specific logic; aggregate extraction results do not dis- tinguish among these qualitatively different cases. We there- fore assign each IOC one of three levels based on how it can be recovered from the script: Direct, indicators that ap- pear verbatim in the source and are recoverable by surface- form matching; Fixed-transform, indicators hidden behind common, parameter-free transforms that a fixed decoder can invert; and Program-dependent, indicators whose recovery requires following logic specific to the script, such as how a string is assembled or decoded. The assignment uses a fixed decoder that applies a set of common reversible transforms over surface strings or bounded parameters, never executing the script. The transforms fall into two families: reassembly operators, which only reorder characters already present in the source (literal concatenation, split/join, etc.), and transcode operators, which invert a codec (Base64 decoding, character-code arrays, etc.). The details and their selection rationale are described in Appendix J. Recovery levels are assigned to individual IOCs rather than entire scripts, allowing a single script to contain indicators at multiple levels. For example, a script may contain a Direct domain appearing verbatim alongside a Program-dependent URL reconstructed through script-specific logic. Note that Program-dependent denotes indicators that are not recovered by the fixed transform set, rather than indicators that are in- herently unrecoverable through static analysis. 4.2 Dataset Analysis In this section, we characterize the benchmark at two lev- els: sample-level input complexity (§4.2.1) and IOC-level distribution and recovery composition (§4.2.2). JS f VBS f 0.0M 0.6M 1.2M 1.8M 2.4M 3.0M PS f 0.0M 0.2M 0.4M 0.6M 0.8M 1.0M JS hp 0.0M 0.1M 0.1M 0.1M 0.2M 0.2M (a) Code token length JS f VBS f PS f JS hp 0.0 0.2 0.4 0.6 0.8 1.0 (b) Gzip compression ratio Figure 2: Distributions of (a) code token length and (b) gzip compression ratio across the benchmark dataset. 4.2.1 Input Complexity We measure the complexity of the benchmark along two com- plementary dimensions: code token length and gzip compres- sion ratio. Code token length measures the size of each script in model tokens. In the LLM setting, longer scripts are gen- erally more difficult to analyze because they occupy a larger fraction of the available context window, making it harder to aggregate relevant evidence across the full input. Gzip com- pression ratio measures how compressible a script is. Lower ratios indicate more compressible code, which is generally associated with greater repetition, whereas higher ratios in- dicate lower compressibility. We use this measure to capture structural variation that is not reflected by input length alone. Figure 2 shows the distribution of complexity metrics across benchmark subsets. Malware samples exhibit heavy- tailed token-length distributions, withJS f markedly longer than the other subsets. Mean token lengths are 325.6K, 25.4K, and 58.0K forJS f ,PS f , andV BS f , respectively, whileJS h p averages 13.3K tokens. Although shorter on average,PS f andV BS f also retain long upper tails. Such input lengths can exceed the practical capacity of many widely used models and remain demanding even for frontier long-context mod- els 1 . Therefore, scripts of this size can still place meaningful pressure on long-context reasoning and evidence aggregation. The gzip compression ratios span a wide range across all datasets, indicating diversity in structural redundancy among samples. This diversity may pose challenges for LLM-based extraction in two different ways. Low-ratio samples can di- lute genuine IOCs within large volumes of repetitive filler code, potentially degrading the signal-to-noise ratio within the model’s context. High-ratio samples, on the other hand, may conceal IOCs inside high-entropy encoded content (Base64 blobs, XOR strings), which could require the model to rec- 1 Google’s Gemini family has supported 1M-token context windows as standard for long-context use. 6 Table 2: IOC distribution and recovery-level composition across script categories. Columns report prevalence, total and unique IOC counts, average instances per sample, and the share of instances at each recovery level: L0 =Direct, L1 = Fixed-transform, and L2 =Program-dependent. Dataset Type% Pos. Inst. (Uniq.) Avg. Rec. dist. (L0/L1/L2) JS f URL99.0110 (104) 1.1356/3/41 Domain57.763 (46) 0.6544/5/51 IP18.618 (11) 0.19 78/0/22 File6.210 (10) 0.1040/30/30 PS f URL92.0147 (140) 1.96 65/13/22 Domain64.066 (58) 0.88 53/17/30 IP21.316 (14) 0.2156/6/38 File30.746 (44) 0.61 81/14/5 VBS f URL98.7140 (100) 1.7734/16/50 Domain81.086 (52) 1.0934/11/55 IP13.912 (10) 0.15 25/67/8 File26.635 (33) 0.4470/15/15 JS h p URL100.0941 (785) 2.4625/19/56 Domain99.7781 (561) 2.0430/22/48 IP0.52 (2) 0.010/50/50 File81.5417 (264) 1.099/15/76 ognize and reverse encoding schemes — a capability that LLMs are known to perform unreliably under stacked obfus- cation [5]. Together, these results indicate that benchmark difficulty arises from different sources across subsets, including input length and structural irregularity. 4.2.2 IOC Label Distribution Here, we characterize IOC-type prevalence and recovery-level composition across script subsets (Table 2) and examine the differences reflected in these distributions. IOC distribution and behavioral patterns. Across all script categories, URLs and domains are the predominant IOC types, while the relative distribution of IOC labels varies substan- tially by script type, reflecting distinct malware behaviors. The recent JS subset,JS f , has an IOC profile that is sparse and URL-centric, typically involving a single primary endpoint, with direct IP communication appearing only occasionally. The PS subset,PS f , has the broadest URL distribution and is more consistent with multi-stage droppers. These samples often contact multiple URLs, sometimes using several paths on the same host, as they retrieve components such as launch- ers, DLLs, archives, or follow-on scripts from shared infras- tructure. Another notable characteristic is the frequent use of legitimate platforms and APIs, including GitHub, Dropbox, and Telegram, as storage or delivery channels. The VBS subset,V BS f , follows a distinct two-stage struc- ture. Many VBS samples exhibit a regular two-URL pattern, with 42% of samples containing an initial retrieval URL and Table 3: LLM configurations used for evaluation. ModelSize Model IDContext Qwen3-8B [45]8.2B Qwen3-8B32K Ministral3-8B [36]8B Ministral-3-8B-Instruct-2512256K Gemma4-8B [16]8B Gemma-4-E4B-it128K Qwen3-C80B ∗ [46]80B Qwen3-Coder-Next262K Qwen3-C480B ∗ [44] 480B Qwen3-Coder-480B-A35B-Instruct262K Qwen3.6-35B [47]35B Qwen3.6-35B-A3B262K Llama4-S109B [33] 109B Llama-4-Scout-17B-16E-Instruct10M Gemini-Pro [15]/ gemini-2.5-pro1M Gemini-Flash [15]/ gemini-2.5-flash1M GPT-Terra [40]/ gpt-5.6-terra922K Note. ∗ indicates models specialized for code generation. a second-stage payload URL. Pastebin-like services are also frequently used to host obfuscated script content for later exe- cution. Compared with PS, VBS samples less often rely on many endpoints from the same server and more often follow this simpler staged-delivery pattern. Finally, the historical JS subset (JS h p ) is particularly dense in domains. This largely reflects downloader-style scripts that iterate over multiple candidate domains and repeatedly issue HTTP requests until a valid payload source is found. As a result, even relatively compact scripts can generate rich domain annotations by embedding multiple fallback hosts within the same retrieval logic. Recovery-level composition. The recovery-level composi- tion reveals differences that are not apparent from IOC fre- quency alone.PS f is comparatively dominated by directly observable indicators, particularly for file IOCs, whereasJS h p contains the largest share of Program-dependent recovery. This contrast also varies by IOC type: files are largely Di- rect inPS f andV BS f , while URLs and domains more of- ten require reconstruction from program logic. JS h p exhibits the opposite extreme for files, with 76% requiring Program- dependent recovery. Fixed-transform indicators form a rela- tively small share overall, suggesting that many non-direct IOCs cannot be recovered by applying a fixed set of transfor- mations alone. 5 Experiments 5.1 LLM Selection As shown in Table 3, we evaluate ten models spanning a wide range of scales, from 8B open-weight models to 480B and frontier proprietary models, to examine how model scale and family affect IOC extraction performance. We focused mainly on open-weight models because they provide greater transparency, reproducibility, and flexibility for controlled evaluation. Smaller models also allow us to assess whether lightweight LLMs can remain competitive under practical constraints such as limited compute, memory, and latency bud- 7 gets. We included code-specialized LLMs alongside general- purpose models, as they are expected to exhibit strong code understanding capabilities. We also incorporated three frontier proprietary models—Gemini-Pro, Gemini-Flash, and GPT- Terra (collectively referred to as frontier models)—as ref- erence baselines. Note that Claude Opus 4.6 and 4.7 were also tested but excluded because they returnedstop_reason: "refusal"on the evaluation samples. Details of the selected LLM configurations are provided in Table 3. 5.2 Evaluation Setup 5.2.1 Baselines We compare LLMs with three rule-based extraction tools: iocextract[24],URLExtract[31], andMSTICPy[34]. iocextractextracts common indicators such as URLs and IP addresses and additionally supports several obfuscated URL forms, including defanged, URL-encoded, hex-encoded, and Base64-encoded variants.URLExtractis a lightweight URL extractor based on top-level-domain (TLD) matching. MSTICPyis a Python library for InfoSec investigation and threat hunting, and its IoCExtract component provides broader regex-based IOC extraction, including URLs, DNS domains, IP addresses, hashes, and file paths. Together, they provide useful reference points for analyzing the strengths and limi- tations of LLMs in IOC extraction. We apply the same post- processing filters to each tool’s raw output; details are pro- vided in Appendix B. 5.2.2 Inference Setup The extraction prompt instructs the model to emit a three- block structured output:<analysis>containing behavioral reasoning and deobfuscation steps,<deobfuscated_code> containing an analyst-readable code reconstruction, and <answer>containing the final JSON-encoded IOC set with classification verdict. The prompt template is in Appendix D. Long-context handling. For samples exceeding the per- chunk code budget derived from each model’s served con- text length, we apply a map–reduce procedure: each chunk is processed independently to record partial findings (deobfus- cation notes, IOC candidates, unresolved references), and a final reduce call aggregates these into the structured output. Short samples bypass chunking entirely. The model serving parameters, chunking budget formula, and per-model trigger- ing statistics are detailed in Appendix E and Appendix F. 5.2.3 Evaluation Metrics We evaluate extraction performance on four IOC categories: URLs, domains, IP addresses, and filesystem artifacts, report- ing precision, recall, and F1 for each. For URLs, our primary metric uses query-insensitive matching: URLs are canoni- calized toscheme + host + path, ignoring query strings, which avoids penalizing variations in query parameters that do not change the contacted endpoint. 2 Under this canonical- ization, the 2,890 raw IOC instances in Table 2 correspond to 2,580 unique indicators used for evaluation. Domains and IP addresses are scored by exact match after normalization, with excluded legitimate-service domains counted as false posi- tives. Filesystem artifacts are evaluated within the labeling scope defined in §4.1. We also define Response-failure rate as the fraction of outputs that fail to produce a usable structured response, in- cluding unparsable, empty, prematurely terminated, or non- terminating outputs. For precision, recall, and F1, such failures are treated as empty predictions. 5.3 Findings This section analyzes LLM-based IOC extraction from multi- ple perspectives. We begin with overall model performance against regex-based baselines, then analyze how performance varies with the recovery level of the target indicator and with input complexity. 5.3.1 Benchmarking LLMs Frontier models lead, but robust IOC extraction remains challenging. Table 4 shows that robust IOC extraction from malicious scripts remains challenging for current LLMs. 3 GPT-Terra and Gemini-Pro lead and are tied at around 65 F1. Gemini-Flash follows, combining high precision with lower recall and a comparatively high response-failure rate. Among open-weight models, Qwen3.6-35B stands out, nearly matching the much larger Qwen3-C480B despite using far fewer active parameters. Scaling within the Qwen family otherwise yields clear gains, with the largest model achieving substantially higher recall and fewer response failures than the smaller variants. The 8B models remain well behind the larger systems, particularly on the more challenging script categories. Regex baselines remain competitive in recall, while LLMs achieve higher precision and recover beyond surface matching. The regex baselines provide a useful reference for surface extraction. The strongest regex baseline, URLEx- tract, reaches only 22.4 F1, leaving a large gap to the leading LLMs. However, several open-weight models show recall comparable to, or even below, the regex baselines, indicating that even surface-oriented IOC recovery is not consistently re- liable.JS h p shows a different pattern: despite low regex recall, even some 8B-scale models recover substantially more indi- cators. This likely reflects that many samples follow regular downloader-style logic that iterates over candidate domains, 2 We additionally report a stricter full-URL metric that includes the query string in Appendix G. 3 For URL IOCs, the main results use the query-insensitive matching of §5.2.3. The stricter URL matching results are reported separately in Ap- pendix G. 8 Table 4: Main results across models. For each script category and overall, we report the response-failure rate (RFail) and micro-averaged precision (P), recall (R), and F1 over IOC instances. Response failures are counted as empty predictions. Model JS f PS f V BS f JS h p Overall RFailPRF1RFailPRF1RFailPRF1RFailPRF1RFailPRF1 Qwen3-8B8.438.5 45.3 41.6 17.9 44.4 43.7 44.0 12.9 30.4 27.1 28.7 14.4 28.0 22.0 24.7 13.7 31.3 26.5 28.7 Ministral3-8B3.823.7 36.7 28.74.936.2 44.1 39.8 30.0 28.4 27.5 27.95.122.6 30.0 25.88.024.3 31.6 27.5 Gemma4-8B10.0 72.4 40.0 51.55.859.6 12.2 20.20.448.9 7.3 12.86.245.4 12.3 19.36.050.8 13.9 21.8 Qwen3-C80B ∗ 11.3 58.4 46.8 51.98.045.2 54.1 49.3 34.2 44.4 34.3 38.7 36.8 54.9 35.6 43.2 29.2 52.5 38.1 44.2 Qwen3.6-35B6.856.3 49.8 52.8 16.9 63.9 48.7 55.2 17.3 51.5 24.9 33.6 14.4 55.9 41.3 47.6 13.9 56.5 41.0 47.5 Llama4-S109B4.645.5 32.7 38.02.242.9 46.5 44.64.339.5 24.9 30.53.039.1 27.6 32.33.340.1 29.5 34.0 Qwen3-C480B ∗ 3.855.3 53.9 54.50.052.1 57.1 54.50.041.3 41.3 41.30.053.2 47.0 49.90.651.9 47.9 49.8 Gemini-Pro17.5 60.2 55.7 57.97.851.9 58.1 54.84.948.0 59.9 53.31.868.9 69.6 69.25.363.7 66.4 65.0 Gemini-Flash56.1 74.7 33.5 46.3 16.9 46.3 54.7 50.2 48.1 64.1 45.1 53.0 14.1 72.3 60.8 66.0 25.2 67.9 56.4 61.6 GPT-Terra6.275.7 57.2 65.20.054.9 59.2 57.00.053.0 57.0 55.00.073.2 64.2 68.40.968.6 62.4 65.4 iocextract0.04.6 48.6 8.40.013.7 47.3 21.20.08.9 26.9 13.40.011.4 12.9 12.10.08.9 20.8 12.5 URLExtract0.022.0 48.6 30.30.042.1 44.7 43.30.015.2 27.3 19.50.026.2 12.6 17.00.024.9 20.3 22.4 MSTICPy0.01.3 47.6 2.50.040.4 48.0 43.90.022.8 26.2 24.40.05.9 12.6 8.00.04.4 20.5 7.2 making program-dependent recovery relatively tractable de- spite requiring more than literal string matching. Even when recall gains are limited, open-weight LLMs generally achieve higher precision than the regex baselines. This indicates that their advantage lies not only in recovering additional indicators, but also in being more selective and pro- ducing fewer unsupported predictions. Appendix I provides the corresponding per-IOC-type breakdown. Response failures vary across models. Qwen3-8B usually fails before reaching the final answer, falling into repetitive reasoning loops such as repeated decoding explanations, sen- tinel values, or encoded substrings. Ministral3-8B, Gemma4- 8B, Qwen3-C80B, and Gemini-Flash more often stop mid- generation after beginning a plausible analysis or deobfusca- tion. Thus, they have distinct error modes: failure to transition into the final answer, premature termination, or minor recov- erable formatting errors. 4 5.3.2 Effect of IOC Recovery Level We stratify each ground-truth IOC by its recovery level to ex- amine how model performance changes as recovery requires progressively more reconstruction. Table 5 reports recall by recovery level, with Fixed-transform further stratified by the number of recovery operations. Direct IOCs are largely recoverable by stronger mod- els. When indicators appear explicitly in the source, stronger models achieve high recall, with GPT-Terra reaching 91.7% and Gemini-Pro 89.4%. In contrast, smaller models show sub- stantially lower recall even at this simplest level, indicating that the mere presence of an indicator in the input does not guarantee reliable extraction. 4 We also tested larger token budgets, but the failed generations often stopped mid-generation in the same way, suggesting that these failures are Table 5: Micro recall (%) by IOC recovery level. Fixed- transform stratified by recovery-operation count, with single- step cases split into reassembly (R) vs. transcode (T). ModelDirectFixed-transf.Program-dep. 1 (R) 1 (T)2 ≥ 3 # IOC instances (858) (230) ( 85) (156) ( 21)(1,230) Qwen3-8B66.927.816.52.60.01.5 Ministral3-8B78.733.016.53.80.02.8 Qwen3.6-35B80.259.137.616.00.014.7 Qwen3-C480B88.268.363.554.59.513.8 Gemini-Flash85.588.367.153.814.331.0 Gemini-Pro89.488.380.069.223.846.1 GPT-Terra91.790.982.469.919.035.4 Recovery degrades rapidly as reconstruction becomes deeper. For Fixed-transform IOCs, recall declines steadily as recovery requires more operations, and a single transcode (T) is harder than a single reassembly (R). GPT-Terra, for exam- ple, falls from 90.9% (single reassembly) and 82.4% (single transcode) to 69.9% at two operations and 19.0% at three or more. Multi-operation chains are particularly challenging because they require the model to accurately invert multi- ple transformations in sequence. Smaller models start from substantially lower recall and deteriorate even more sharply. Thus, even when the required transformations are determinis- tic and known in principle, reliably composing them remains difficult for current LLMs. Program-dependent recovery remains difficult even for frontier models. When recovery depends on sample-specific program logic, recall drops sharply across all models. Even the frontier models recover only 31–46% of these IOCs, and all other models perform substantially worse. Notably, not simply caused by insufficient output length. 9 Qwen3-8B Ministral3-8B Gemma4-8B Qwen3-C80B Qwen3.6-35B Llama4-S109B Qwen3-C480B Gemini-Pro Gemini-Flash GPT-Terra Response-fail rate 0 50 100 % 1.5K2.8K 11K24K 3.6M token length Qwen3-8B Ministral3-8B Gemma4-8B Qwen3-C80B Qwen3.6-35B Llama4-S109B Qwen3-C480B Gemini-Pro Gemini-Flash GPT-Terra Regex-miss recall 0.240.360.410.490.83 gzip ratio 0 50 100 % Figure 3: Effect of input complexity on LLM-based IOC ex- traction. Columns are binned by token length or gzip compres- sion ratio. The top row reports Response-failure rate, where darker is worse; the bottom row reports Regex-missed Recall, where darker is better. Qwen3-C480B performs competitively on simpler recovery levels but reaches only 13.8% recall at this level. This sug- gests that program-dependent recovery remains a major lim- itation of current models, while also exposing the clearest differences in their ability to reason over program-specific logic. The recovery-level trend remains consistent within each dataset (Appendix Table 12). Frontier models struggle with compositional and iterative value recovery. To better understand where the frontier mod- els succeed and fail, we break down their recall by recovery operation in Appendix J.1. For Fixed-transform, models gen- erally handle many isolated operations well, including literal concatenation, split/join, and some single-step transcodes, but recall drops when reconstruction requires multiple operations (Table 10). The models also exhibit different strengths: the Gemini models are substantially stronger than GPT-Terra on constant propagation, whereas GPT-Terra performs better on Base64-to-UTF-16LE decoding. The Program-dependent results further show that difficulty rises when value recov- ery depends on iterative program execution: loop-constructed values stand out as the hardest mechanism for all three mod- els (Table 11). Taken together, these results suggest that the main bottleneck is composing multiple transformations and tracking values through iterative execution. 5.3.3 Effect of Input Complexity We analyze LLM extraction performance along the two di- mensions introduced in §4.2.1: input size (token length) and code compressibility (gzip compression ratio; a lower ratio in- dicates more redundant content). We use response-failure rate to measure response reliability and Regex-missed Recall to capture recovery capability beyond surface-oriented regex ex- traction, with the latter computed over ground-truth indicators missed by the regex baseline. Longer inputs degrade both recovery capability and re- sponse reliability. Response-failure rates tend to increase with token length, although the pattern varies across mod- els. Regex-missed Recall also generally declines as token length increases, indicating that longer inputs make recov- ery beyond surface matching increasingly difficult. Notably, Qwen3-C480B maintains low response-failure rates across most bins while its Regex-missed Recall still declines with length. Thus, reliably completing a structured response does not necessarily imply robust IOC recovery from long inputs. Code complexity shows a weaker and different effect. Lower gzip-ratio bins show a weak tendency toward higher response-failure rates, suggesting that highly redundant in- puts may reduce output reliability. In contrast, Regex-missed Recall tends to be somewhat higher in higher gzip-ratio bins, although the pattern varies across models. 5.4 Failure Modes Analysis 5.4.1 False Positive Category LLMs can generate plausible-looking IOCs that are not di- rectly supported by the input, and such errors can reduce analyst trust and increase downstream verification cost. There- fore, a fine-grained analysis of false positives is necessary for understanding where LLMs are vulnerable as IOC extractors and what types of mitigation are needed. To support systematic error analysis, we define a taxonomy of false positives, summarized in Table 6. Its top-level split is provenance, based on whether a prediction is traceable to the sample exactly or through a bounded normalization or near-match relation. Traceable errors are further divided into distorted reference IOCs and non-reference source strings. Each category is also associated with its likely operational disposition. A deterministic precedence rule assigns every false positive to exactly one of the ten categories; full rules are given in Appendix H. 5.4.2 Results Figure 4 shows each model’s false-positive composition within each IOC recovery level 5 , revealing clear differences in model error profiles. Smaller models fall back to prompt-echo errors at higher recovery levels. Several smaller open-weight models exhibit a distinctive failure mode: when they fail to recover an IOC from the script, they frequently emit IOC examples copied from the prompt. Prompt echo dominates the false positives of Qwen3-8B and Ministral3-8B, and is disproportionately associated with missed Program-dependent IOCs rather than directly observable ones. This suggests a shortcut-like fall- back: when recovery from the script becomes difficult, smaller 5 Because a false positive has no intrinsic recovery level, we associate each error with the recovery difficulty of the reference IOC it most plausibly replaces or accompanies. 10 Table 6: False-positive taxonomy, grouped by provenance; the assignment precedence is deterministic (Appendix H). Disposition (operational handling if the error survives triage):Filterable,Recoverable,Review-inducing,Misleading.G/Pdenote the reference IOC and prediction. GroupCategoryClassification criterionExampleDisp. Type mismatchThe normalized value equals a reference IOC of a different type. G i p :1.2.3.4P dom :1.2.3.4 Normalization errorMalformed, yet deterministically normalizes to a reference IOC. G:https://a.coP:htps:/a.co GranularityThe right entity at a coarser or finer level. G:a.b.com/x/yP:a.b.com/x Near-copy A distorted copy of a reference IOC: a mis-cut boundary or altered characters. G:24sports.caP:sports.ca Benign shared serviceA benign shared platform in the sample, tied to no reference indicator. P:ifconfig.me/ip Source non-IOCA source string that is neither an indicator nor a benign service. P:fileserver Prompt echoA value copied from the instruction template. P:cdn.example.test Generic placeholder A source-absent reserved/private/example pattern, or a hallucinated benign service. P:192.168.1.1 Ungrounded invalidSyntactically invalid and not traceable to any reference IOC or source. P:maa1 Unattributed A specific, valid-looking value with no trace in source, prompt, priors, or gold. P:downprofits.ru Sample -traceable IOC non- IOC Sample -untraceable Qwen3-8B Ministral3-8B Gemma4-8B Qwen3-C80B Qwen3.6-35B Llama4-S109B Qwen3-C480B Gemini-Pro Gemini-Flash GPT-Terra 0 50 100 % of FPs 10944436118136109103777083 (a) Direct Qwen3-8B Ministral3-8B Gemma4-8B Qwen3-C80B Qwen3.6-35B Llama4-S109B Qwen3-C480B Gemini-Pro Gemini-Flash GPT-Terra 901144346397949331924 (b) Fixed-transform Qwen3-8B Ministral3-8B Gemma4-8B Qwen3-C80B Qwen3.6-35B Llama4-S109B Qwen3-C480B Gemini-Pro Gemini-Flash GPT-Terra 12451716266627585847854701401506 (c) Program-dependent granularitytype mismatchnear-copysource non-IOCnormaliz. err.benign svc. prompt-echogeneric plac.unattributedungr. invalid sample-traceable sample-untrace. Figure 4: Per-model false-positive composition within each IOC recovery level. Numbers above bars indicate total false positives. models often reuse readily available IOC examples from the prompt instead of analyzing the malware sample. Prompt-echo errors decrease with scale within a model family. The same pattern persists in some larger open-weight models but decreases sharply with scale. Within the Qwen3- Coder family, Prompt echo accounts for 53% of false positives from Qwen3-C80B but only about 8% from Qwen3-C480B. The remaining errors of the larger model instead shift toward categories such as Granularity and Unattributed. This within- family comparison suggests that scaling substantially reduces reliance on prompt-derived outputs. Frontier-model errors shift from Prompt echo to recon- struction failures. At the frontier end, false positives are less dominated by prompt-derived artifacts and instead re- flect failures during reconstruction. Many remain grounded in the sample, appearing as Near-copy or Granularity errors in which the predicted IOC is closely related to the underlying evidence but does not match the exact indicator. This pattern is consistent with known limitations of LLMs in character- level string manipulation [50, 59]. As recovery becomes more difficult, Unattributed errors also become more prominent: these predictions are no longer directly traceable to the source, suggesting that reconstruction can drift beyond the available evidence during deobfuscation. Thus, frontier models fail less often by reverting to prompt examples, but still struggle with both precise reconstruction and maintaining grounding throughout recovery. Taken together, recovery level reveals a qualitative capa- bility gap in how models fail: smaller models increasingly lose sample grounding when script-specific reconstruction is required, whereas frontier models tend to remain closer to the malware evidence. Increased model capacity shifts errors in the latter direction; in §6 we show that adaptation and tool augmentation reproduce the same shift in a small model with- 11 out scaling. Operationally, stronger models therefore produce fewer obviously spurious indicators, but their residual errors are more plausible and may require closer analyst verification. 6Improving Small-Model IOC Recovery with Tools and Adaptation The benchmark results expose two prominent weaknesses of 8B-scale models: they perform reasonably on directly ob- servable indicators but struggle once IOC recovery requires reconstruction (§5.3.2), and they lose grounding under diffi- cult cases, as reflected in frequent prompt-echo false positives (§5.4). We therefore examine two targeted approaches: 1) de- terministic string utilities, which offload exact decoding and reconstruction, and 2) task-specific adaptation, which teaches the model the analysis behavior required for IOC extraction. We use SCRIPTIOC-BENCH as a testbed to evaluate these approaches separately and together in a controlled case study on a single open-weight model. 6.1 Study Design We conduct the case study on Qwen3-8B, the strongest 8B- scale model in our evaluation (§5.3.1), and compare four vari- ants under the shared inference setup of §3.3: the base model, the base model with tool access, the task-adapted model, and the task-adapted model with tool access. String-utility tool augmentation. We provide deterministic utilities for common decoding, string reconstruction, and re- versible transformations, including operations such as Base64 decoding, concatenation, split/join, reversal, and XOR. The utilities operate only on strings and integer arrays and do not execute code, access the filesystem, or open network connec- tions, preserving the static-analysis setting. Tool outputs are returned to the model for subsequent reasoning; the full tool set and agent configuration are given in Appendices D and E. Task-specific adaptation. We adapt Qwen3-8B using super- vised fine-tuning followed by Group Relative Policy Opti- mization (GRPO) [49] with a task-specific verifiable reward. Training uses an additional 1,100 JavaScript malware sam- ples from 2015–2016, drawn from the same collection as JS h p but disjoint from the evaluation set, together with benign JavaScript samples. The objective encourages valid structured outputs, correct verdicts, and precise IOC extraction. Impor- tantly, the adaptation contains no supervision for tool use, allowing us to evaluate whether adaptation and tool augmen- tation provide complementary gains. Full training and reward details are provided in Appendix C. • Supervised fine-tuning. For SFT, a strong teacher model generates reasoning-rich responses with intermediate anal- ysis and structured IOC outputs, and the student is trained on the filtered synthetic supervision using the standard language-modeling objective. (a) Direct 0 25 50 75 Recall (%) 66.9 72.6 69.3 73.4 (b) Fixed-transf. 16.7 15.4 21.7 17.9 (c) Program-dep. 1.5 2.1 1.7 2.4 base+tool+adapt+both Figure 5: Recall by IOC recovery level for the variants. base +tool +adapt +both (a) Direct 0 50 100 % of FPs 109209115261 base +tool +adapt +both (b) Fixed-transf. 90608658 base +tool +adapt +both (c) Program-dep. 1245587929516 granularitytype mismatchnear-copysource non-IOCnormaliz. err.benign svc. prompt-echogeneric plac.unattributedungr. invalid Figure 6: False-positive composition for the variants. • Reward-based optimization. Starting from the SFT model, we apply GRPO with a task-specific verifiable reward de- signed for structured IOC extraction. The reward jointly encourages accurate IOC extraction through separate pre- cision and recall terms, rewarding coverage of supported indicators while discouraging unsupported over-prediction. It also includes auxiliary terms for output formatting, ma- licious/benign classification, and consistency between de- clared IOC status and emitted indicators. We also introduce longer and more complex scripts gradually during train- ing for optimization stability. Full reward definitions and training details are provided in Appendix C. 6.2 Results Figures 5 and 6 show how tool access and adaptation af- fect both recovery capability and false-positive behavior; Ap- pendix Figure 9 provides the overall performance across script categories. Tool access improves grounding. String-utility access changes both recall and the character of the model’s errors. Its largest recall gain appears on Direct indicators, increasing from 67% to 73%; in contrast, recall decreases on Fixed- transform and increases only slightly on Program-dependent. Tool use also introduces additional tool interactions and inter- mediate reasoning, increasing generation overhead and caus- ing more parsing failures in some cases, which contributes to the reduced Fixed-transform recall. At the same time, tool access sharply suppresses Prompt echo, which dominates the base model’s false positives. Together, these results suggest that tool access primarily improves grounding and helps the 12 model recover indicators that are directly present in the script. Adaptation helps most with Fixed-transform. Adaptation exhibits a different recovery profile. Its largest gain appears on Fixed-transform indicators, where recall increases from 17% to 22%, while the improvement on Direct indicators is more modest. Within Fixed-transform, adaptation improves single-step reassembly recall from 28% to 38%, while single- step transcode recall changes little (16% to 19%). This sug- gests that adaptation helps with recurring reassembly patterns, while reliable transcoding remains challenging. Adaptation also reduces prompt-echo errors, although less sharply than tool access. Improvements do not extend to Program-dependent re- covery. Despite clear gains in grounding and Fixed-transform recall, Program-dependent recall remains at only 1–3% across all four variants. These approaches therefore improve ground- ing and fixed reconstruction without materially extending recovery through sample-specific program logic. This re- maining bottleneck is not specific to Qwen3-8B; Program- dependent recovery is the hardest part even for the frontier models in our benchmark. Both approaches improve precision, with the combined variant performing best. Appendix Figure 9 shows that both tool access and adaptation substantially improve precision, with the combined model achieving the highest precision, raising it from 31% for the base variant to 48%. These gains indicate that these approaches not only recover more IOCs but also reduce unsupported predictions. Scope. This case study is intended to demonstrate that the failure modes SCRIPTIOC-BENCH surfaces are actionable, rather than to establish generality across models; broader studies, including tool-aware adaptation, are discussed in §7. 7 Discussion Implications for Analyst Workflows. Our results show that current LLMs are not yet reliable enough for standalone IOC extraction. Even the strongest model reaches only 65.4 F1, and recall drops sharply for Program-dependent indicators. Moreover, models exhibit different failure modes, suggesting that their outputs require verification tailored to their error pro- files. At the same time, our controlled study on a small open- weight model shows that deterministic string utilities and task-specific adaptation can improve grounding and precision. These findings motivate a layered static-analysis workflow in which conventional extractors, LLMs, and deterministic utili- ties support initial IOC recovery, while unresolved cases are escalated to specialized program analysis or analyst review. Ground-Truth Scope. In this study, we focus on URLs, do- mains, IP addresses, and filesystem artifacts because they collectively capture both network infrastructure and host-side traces and commonly arise in script-based malware. Other IOC types remain outside the current scope. For example, registry artifacts may require a different representation, as the threat-relevant unit can involve not only a key path but also the associated value name, value data, and registry operation. Including them would therefore require additional annotation and matching rules beyond those considered here. Such arti- fact types remain natural targets for future extensions of the benchmark. Data Contamination. Since the Hynek Petrak collection (2015–2017) is publicly available, pretraining contamination onJS h p is a potential concern. In contrast, the more recent JS f ,PS f , andV BS f datasets were collected after the doc- umented knowledge cutoffs of all evaluated models except GPT-Terra, and therefore cannot overlap with their pretraining data under those cutoffs. The main findings persist across both the historical and recent datasets. GPT-Terra is the only ex- ception for which the training window may partially overlap with the recent data, so contamination cannot be ruled out for this model. Nevertheless, its performance remains far from saturated on the recent subsets, reaching only 65.2, 57.0, and 55.0 F1 onJS f ,PS f , andV BS f , respectively. These results do not eliminate contamination as a possibility for GPT-Terra, but they provide no indication that its performance is trivially explained by memorization. Future Work. Future work can extend SCRIPTIOC-BENCH to additional script languages and IOC types. Our tool-use study can also be expanded to multiple models evaluated with the deterministic utilities, allowing controlled compari- son across model families. These experiments could further motivate tool-aware adaptation, where models learn when to invoke external utilities and how to integrate their outputs into IOC predictions. Finally, we plan to maintain the bench- mark and leaderboard through periodic updates covering new models and emerging malware. 8 Conclusion We presented SCRIPTIOC-BENCH, a benchmark for eval- uating whether LLMs can recover concrete IOCs directly from malicious scripts under static analysis. SCRIPTIOC- BENCH targets artifact-level recovery of URLs, domains, IP addresses, and filesystem artifacts from JavaScript, Power- Shell, and VBScript malware, with manually verified labels curated for static recoverability. Our results show that static IOC extraction remains challenging even for frontier models; they perform well on directly exposed IOCs but struggle with deeper reconstruction. A fine-grained false-positive analysis shows that their error profiles vary across model capabili- ties. Tool augmentation and task-specific adaptation improve grounding and precision, but Program-dependent recovery re- mains a key challenge. Overall, SCRIPTIOC-BENCH provides a systematic testbed for measuring progress toward reliable LLM-assisted IOC extraction. 13 References [1] Amir Afianian, Salman Niksefat, Babak Sadeghiyan, and David Baptiste. Malware dynamic analysis eva- sion techniques: A survey. ACM Computing Surveys, 52(6):126:1–126:28, 2019. [2]Frederick Barr-Smith, Xabier Ugarte-Pedrero, Mariano Graziano, Riccardo Spolaor, and Ivan Martinovic. Sur- vivalism: Systematic analysis of Windows malware living-off-the-land. In 2021 IEEE Symposium on Secu- rity and Privacy (S&P), pages 1557–1574. IEEE, 2021. [3]Daniel Bohannon and Lee Holmes.Revoke- Obfuscation: PowerShell obfuscation detection using science. Black Hat USA 2017 Whitepaper, 2017.https: //w.blackhat.com/docs/us-17/thursday/u s-17-Bohannon-Revoke-Obfuscation-PowerShel l-Obfuscation-Detection-And%20Evasion-Usi ng-Science-wp.pdf. [4]CapacitorSet. box-js: A Tool for Studying JavaScript Malware.https://github.com/CapacitorSet/bo x-js. Accessed 2026-04-27. [5]Guoqiang Chen, Xin Jin, and Zhiqiang Lin.Js- DeObsBench: Measuring and benchmarking LLMs for JavaScript deobfuscation. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communi- cations Security (CCS), pages 36–50. ACM, 2025. [6]Xu Chen, Jon Andersen, Z. Morley Mao, Michael Bai- ley, and Jose Nazario. Towards an understanding of anti-virtualization and anti-debugging behavior in mod- ern malware. In 2008 38th Annual IEEE/IFIP Interna- tional Conference on Dependable Systems and Networks (DSN), pages 177–186. IEEE, 2008. [7] Yutong Cheng, Osama Bajaber, Saimon Amanuel Tsegai, Dawn Song, and Peng Gao. CTINexus: Au- tomatic cyber threat intelligence knowledge graph con- struction using large language models. In 2025 IEEE 10th European Symposium on Security and Privacy (Eu- roS&P), pages 923–938. IEEE, 2025. [8] Marco Cova, Christopher Kruegel, and Giovanni Vigna. Detection and analysis of drive-by-download attacks and malicious JavaScript code. In Proceedings of the 19th International Conference on World Wide Web (W), pages 281–290. ACM, 2010. [9] CrowdStrike. 2025 Global Threat Report.https:// w.crowdstrike.com/global-threat-report/, 2025. Accessed 2026-08-23. [10] Charles Curtsinger, Benjamin Livshits, Benjamin Zorn, and Christian Seifert. ZOZZLE: Fast and precise in- browser JavaScript malware detection. In 20th USENIX Security Symposium (USENIX Security 11). USENIX Association, 2011. [11]Lauren Deason, Adam Bali, Ciprian Bejean, Diana Bolo- can, James Crnkovich, Ioana Croitoru, Krishna Durai, Chase Midler, Calin Miron, David Molnar, Brad Moon, Bruno Ostarcevic, Alberto Peltea, Matt Rosenberg, Catalin Sandu, Arthur Saputkin, Sagar Shah, Daniel Stan, Ernest Szocs, Shengye Wan, Spencer Whitman, Sven Krasser, and Joshua Saxe.CyberSOCEval: Benchmarking LLMs capabilities for malware analy- sis and threat intelligence reasoning. arXiv preprint arXiv:2509.20166, 2025. [12]Chongzhou Fang, Ning Miao, Shaurya Srivastav, Jialin Liu, Ruoyu Zhang, Ruijie Fang, Asmita, Ryan Tsang, Najmeh Nazari, Han Wang, and Houman Homayoun. Large language models for code analysis: Do LLMs really do their job? In 33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024. [13] Aurore Fass, Michael Backes, and Ben Stock.Hi- deNoSeek: Camouflaging malicious JavaScript in be- nign ASTs. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 1899–1913. ACM, 2019. [14]Filescan.io. Filescan.io: Next-Gen Malware Analysis Platform.https://w.filescan.io/. Accessed 2026-08-23. [15]Google. Gemini 2.5: Updates to our family of thinking models.https://developers.googleblog.com/ en/gemini-2-5-thinking-model-updates/ , 2025. Accessed 2026-04-28. [16] Google. Gemma 4: Byte for byte, the most capable open models.https://blog.google/innovation-a nd-ai/technology/developers-tools/gemma-4/ , 2026. Accessed 2026-04-28. [17] Google Cloud Threat Intelligence. Scaling Up Malware Analysis with Gemini 1.5 Flash.https://cloud.goog le.com/blog/topics/threat-intelligence/sc aling-up-malware-analysis-with-gemini , 2024. Accessed 2026-04-28. [18]Hatching Triage. Triage: Malware Analysis Sandbox. https://tria.ge/. Accessed 2026-08-23. [19] Yiling He, Hongyu She, Xingzhi Qian, Xinran Zheng, Zhuo Chen, Zhan Qin, and Lorenzo Cavallaro. On benchmarking code LLMs for Android malware analy- sis. arXiv preprint arXiv:2504.00694, 2025. [20]Adrian Herrera. Optimizing away JavaScript obfusca- tion. In 2020 IEEE 20th International Working Con- ference on Source Code Analysis and Manipulation (SCAM), pages 215–220. IEEE, 2020. 14 [21]HP Wolf Security. HP Wolf Security Threat Insights Report, September 2024.https://threatresearch .ext.hp.com/wp-content/uploads/2024/09/HP_ Wolf_Security_Threat_Insights_Report_Septe mber_2024.pdf, 2024. Accessed 2026-04-28. [22] Ghaith Husari, Ehab Al-Shaer, Mohiuddin Ahmed, Bill Chu, and Xi Niu. TTPDrill: Automatic and accurate extraction of threat actions from unstructured text of CTI sources. In Proceedings of the 33rd Annual Computer Security Applications Conference (ACSAC), pages 103– 115. ACM, 2017. [23]Hybrid Analysis. Hybrid Analysis: Free Automated Malware Analysis Service.https://w.hybrid-a nalysis.com/. Accessed 2026-08-23. [24] InQuest. iocextract.https://github.com/InQuest /iocextract. Accessed 2026-01-28. [25] Sainadh Jamalpur, Yamini Sai Navya, Perla Raja, Gam- pala Tagore, and G Rama Koteswara Rao. Dynamic malware analysis using cuckoo sandbox. In 2018 Sec- ond international conference on inventive communica- tion and computational technologies (ICICCT), pages 1056–1060. IEEE, 2018. [26]Christopher Johnson, Lee Badger, David Waltermire, Julie Snyder, and Clem Skorupka. Guide to cyber threat information sharing. Technical Report NIST Special Publication 800-150, National Institute of Standards and Technology, 2016. [27]Ruijie Li, Chenyang Zhang, Huajun Chai, Lingyun Ying, Haixin Duan, and Jun Tao. PowerPeeler: A precise and general dynamic deobfuscation method for Power- Shell scripts. In Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 4539–4553. ACM, 2024. [28]Zhenyuan Li, Qi Alfred Chen, Chunlin Xiong, Yan Chen, Tiantian Zhu, and Hai Yang. Effective and light-weight deobfuscation and semantic-aware attack detection for PowerShell scripts. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 1831–1847. ACM, 2019. [29]Xiaojing Liao, Kan Yuan, XiaoFeng Wang, Zhou Li, Luyi Xing, and Raheem A. Beyah. Acing the IOC game: Toward automatic discovery and analysis of open-source cyber threat intelligence. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communi- cations Security (CCS), pages 755–766. ACM, 2016. [30]Martina Lindorfer, Clemens Kolbitsch, and Paolo Mi- lani Comparetti. Detecting environment-sensitive mal- ware. In Proceedings of the 14th International Sympo- sium on Recent Advances in Intrusion Detection (RAID), pages 338–357. Springer, 2011. [31]Jan Lipovský. URLExtract.https://github.com/l ipoja/URLExtract. Accessed 2026-01-28. [32]Nora McDonald, Sarita Schoenebeck, and Andrea Forte. Reliability and inter-rater reliability in qualitative re- search: Norms and guidelines for CSCW and HCI prac- tice. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):72:1–72:23, 2019. [33]Meta. The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation.https://ai.meta. com/blog/llama-4-multimodal-intelligence/ , 2025. Accessed 2026-04-28. [34]Microsoft. MSTICPy: MSTIC Jupyter and Python Se- curity Tools.https://msticpy.readthedocs.io /en/v2.14.0/GettingStarted.html. Accessed 2026-01-28. [35]Microsoft. WhatsApp malware campaign delivers VB- Script and MSI backdoors.https://w.microsoft. com/en-us/security/blog/2026/03/31/whatsap p-malware-campaign-delivers-vbs-payloads-m si-backdoors, 2026. Accessed 2026-04-13. [36]Mistral AI. Introducing Mistral 3.https://mistral. ai/news/mistral-3, 2025. Accessed 2026-04-28. [37]MITRE ATT&CK. Command and Scripting Interpreter (T1059).https://attack.mitre.org/technique s/T1059/. Accessed 2026-04-27. [38] MITRE ATT&CK. Obfuscated Files or Information (T1027).https://attack.mitre.org/technique s/T1027/. Accessed 2026-01-28. [39] National Institute of Standards and Technology. Indica- tor of compromise.https://csrc.nist.gov/glos sary/term/indicator_of_compromise , n.d. NIST Computer Security Resource Center Glossary. [40] OpenAI. GPT-5.6: Frontier Intelligence that Scales with Your Ambition.https://openai.com/index/gpt-5 -6/, 2026. Accessed 2026-08-22. [41] Palo Alto Networks. Indicators of Compromise (IoCs). https://w.paloaltonetworks.com/cyberpe dia/indicators-of-compromise-iocs . Accessed 2026-01-28. [42]Constantinos Patsakis, Fran Casino, and Nikolaos Lyk- ousas. Assessing LLMs in malicious code deobfuscation of real-world malware campaigns. Expert Systems with Applications, 256:124912, 2024. 15 [43]Hynek Petrak. javascript-malware-collection: Collec- tion of almost 40.000 JavaScript malware samples. https://github.com/HynekPetrak/javascrip t-malware-collection. Accessed 2026-04-28. [44] Qwen Team. Qwen3-Coder: Agentic Coding in the World.https://qwen.ai/blog?id=qwen3-coder, 2025. Accessed 2026-04-28. [45] Qwen Team. Qwen3: Think Deeper, Act Faster.https: //qwen.ai/blog?id=qwen3 , 2025. Accessed 2026- 04-28. [46] Qwen Team. Qwen3-Coder-Next: Pushing Small Hy- brid Models on Agentic Coding.https://qwen.a i/blog?id=qwen3-coder-next, 2026. Accessed 2026-04-28. [47] Qwen Team.Qwen3.6-35B-A3B: Agentic Coding Power, Now Open to All.https://qwen.ai/blog ?id=qwen3.6-35b-a3b, 2026. Accessed 2026-08-22. [48] Red Canary. Threat Detection Report: Techniques.ht tps://redcanary.com/threat-detection-repor t/techniques/. Accessed 2026-04-28. [49] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [50]Andrew Shin and Kunitake Kaneko. Large language models lack understanding of character composition of words. arXiv preprint arXiv:2405.11357, 2024. [51]Giuseppe Siracusano, Davide Sanvito, Roberto Gon- zalez, Manikantan Srinivasan, Sivakaman Kamatchi, Wataru Takahashi, Masaru Kawakita, Takahiro Kaku- maru, and Roberto Bifulco. Time for aCTIon: Auto- mated analysis of cyber threat intelligence in the wild. arXiv preprint arXiv:2307.10214, 2023. [52]Denis Ugarte, Davide Maiorca, Fabrizio Cara, and Gior- gio Giacinto. PowerDrive: Accurate de-obfuscation and analysis of PowerShell malware. In Proceedings of the 16th International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (DIMVA), pages 240–259. Springer, 2019. [53]U.S. Department of Justice. Emotet Botnet Disrupted in International Cyber Operation.https://w.just ice.gov/archives/opa/pr/emotet-botnet-dis rupted-international-cyber-operation, 2021. Press release, accessed 2026-08-23. [54]U.S. Department of Justice. Multiple Foreign Nation- als Charged in Connection with Trickbot Malware and Conti Ransomware Conspiracies.https://w.ju stice.gov/usao-ndoh/pr/multiple-foreign-n ationals-charged-connection-trickbot-mal ware-and-conti-ransomware , 2023. Press release, accessed 2026-08-23. [55] U.S. Department of Justice. Qakbot Malware Disrupted in International Cyber Takedown.https://w.ju stice.gov/archives/opa/pr/qakbot-malware-d isrupted-international-cyber-takedown , 2023. Press release, accessed 2026-08-23. [56]U.S. Department of State. Reward Offers for Infor- mation on Conti Ransomware-as-a-Service Operators. https://w.state.gov/reward-for-informati on-owners-operators-affiliates-of-the-con ti-ransomware-as-a-service-raas/ , 2022. Re- wards for Justice program, accessed 2026-08-23. [57]VirusTotal. VirusTotal.https://w.virustotal.c om/. Accessed 2026-08-23. [58]Daniel Votipka, Kelsey R. Fulton, James Parker, Matthew Hou, Michelle L. Mazurek, and Michael Hicks. Understanding security mistakes developers make: Qual- itative analysis from Build It, Break It, Fix It. In 29th USENIX Security Symposium (USENIX Security 20), pages 109–126. USENIX Association, 2020. [59]Xilong Wang, Hao Fu, Jindong Wang, and Neil Zhen- qiang Gong. StringLLM: Understanding the string processing capability of large language models. arXiv preprint arXiv:2410.01208, 2024. [60]Wei Xu, Fangfang Zhang, and Sencun Zhu. The power of obfuscation techniques in malicious JavaScript code: A measurement study. In 2012 7th International Con- ference on Malicious and Unwanted Software (MAL- WARE), pages 9–16. IEEE, 2012. [61]Xinran Zheng, Xingzhi Qian, Yiling He, Shuo Yang, and Lorenzo Cavallaro. Is “knowing it’s malicious enough?” evaluating LLMs for fine-grained malware behavior au- diting. arXiv preprint arXiv:2509.14335, 2025. [62]Dongchao Zhou, Lingyun Ying, Huajun Chai, and Dong- bin Wang. From obfuscated to obvious: A comprehen- sive JavaScript deobfuscation tool for security analysis. In Proceedings of the 33rd Annual Network and Dis- tributed System Security Symposium (NDSS). Internet Society, 2026. 16 Figure 7: Screenshot of SCRIPTIOC-BENCH leaderboard for tracking static IOC extraction performance across LLMs. A Leaderboard We will release our SCRIPTIOC-BENCH as a public leader- board (Figure 7) to encourage reproducible evaluation and community participation. The leaderboard provides a shared testbed for evaluating LLM-based static IOC extraction sys- tems on realistic malicious scripts. By comparing open- weight LLMs, frontier models, tool-augmented agents, and conventional rule-based baselines under the same protocol, the leaderboard makes it possible to track progress and expose remaining failure modes. Through broader participation, we hope to accelerate the development of reliable LLM-assisted IOC extraction methods and contribute to practical threat- intelligence automation. B Regex Baseline Details URL candidates are truncated at the first quote, whitespace, or control character, and are discarded if they exceed 500 characters or lack a validhttp,https, orftpscheme and host. Domain candidates must have a 2–6 character alpha- betic TLD; we discard candidates whose TLD is a known file extension, whose labels contain common JavaScript key- words or_0x...obfuscation tokens, or whose labels are single-character, purely numeric, or version-like. For file can- didates, we remove paths shorter than five characters and, for MSTICPy path matches, additionally filter strings containing _0x[0-9a-f]+,function(, orreturn ;to avoid JavaScript fragments being misclassified as Linux paths. C Training Details We adapt Qwen3-8B through a two-stage training procedure. The first stage performs supervised fine-tuning (SFT), and the second stage continues from the supervised fine-tuned check- point using reinforcement learning with verifiable rewards. Both stages train LoRA adapters (rank 128,α=32, dropout 0.05, applied to all attention and MLP projections) over the open-weight Qwen3-8B base model on three NVIDIA H200 GPUs in bfloat16. The merged checkpoint is evaluated with the same vLLM map–reduce pipeline as the other Qwen3- 8B rows (three repetitions, deterministic decoding atT=0, 32,768-token served context, 8,192-token generation budget). SFT. The first stage trains Qwen3-8B on the IOC extraction prompt–completion pairs distilled from Gemini 2.5 Pro. The source data includes JavaScript malware samples from mul- tiple difficulty levels, as well as benign JavaScript samples. After filtering to fit the training context, the SFT dataset con- tains 1,823 prompt–completion pairs. Training uses AdamW (learning rate1× 10 −5 , linear schedule, warmup ratio 0.03) with an effective batch of 48 prompts, capped at 1,000 steps; the checkpoint for the next stage is selected by training loss. Silver IOC curation. Report-derived IOC labels serve as the gold reference setG(x). Such reports may include dy- namically observed indicators and miss IOCs that are stati- cally recoverable from the script, and manually auditing every training sample is impractical at training scale. We therefore augmentG(x)with a silver setS(x)of additional teacher- extracted candidates that pass syntactic validity checks and are not already present inG(x). The combined references G(x)∪ S(x)are used by the IOC reward terms in the subse- quent GRPO stage (§C.1), with lower confidence assigned to silver-only matches. GRPO. The second stage continues from the SFT check- point using the clipped GRPO objective (ε=0.2) with group- relative reward normalization, AdamW (learning rate1× 10 −5 , warmup ratio 0.05), and an effective batch of 288 prompts with 8 rollouts each, generated with vLLM atT=0.7 under a 4,096-token completion cap. Training follows a two- stage length curriculum: 500 steps on prompts of at most 4,096 tokens, then 500 further steps on prompts of at most 16K tokens. C.1 Reward Function This subsection details the GRPO reward introduced above. Unlike single-label classification, IOC extraction requires both a malicious/benign verdict and a typed set of indicators, and outputs are typically only partially correct. The reward is therefore tailored to this task structure, comprising five terms that each address a distinct failure mode of malicious-code analysis: R(y; x) = λ f R fmt (y)+ λ c R cls (y; x)+ λ s R status (y) + λ p R prec ioc (y;G(x),S(x))+ λ r R rec ioc (y;G(x),S(x)), (1) whereG(x)andS(x)denote the gold and silver reference sets defined above (Silver IOC curation). 17 Format and classification.R fmt ∈ 0, 0.5, 1rewards parseable outputs, with full credit when the per-type IOC status block is present.R cls is1if the predicted malicious/be- nign verdict matches the gold label and 0 otherwise. Self-consistency.R status rewards agreement between the de- clared per-type IOC status (present/suspected/absent) and the IOCs actually emitted. A mismatch occurs when the model declaresabsentbut emits IOCs, or declarespresent but emits none. IOC precision and recall. Predictions are matched against goldG(x)and silverS(x)reference sets, with credit weighted by reference reliability:1.0for IOCs inG(x)∩ S(x),0.85for gold-only, and0.40for silver-only. Per-type precision and recall are computed forhost,ip,file,urland averaged with equal weights. To prevent reward hacking, we apply two countermeasures. (i) Precision penalty: if a sample has only false positives, precision becomes negative (down to−0.45), with stronger penalty when no gold IOCs exist (penalizing fabrication) than when gold IOCs are simply missed. (i) Recall dampening: when the model emits both matched and unmatched IOCs, recall is divided by1 + 0.65· EwhereEis the number of false positives, preventing the model from maximizing recall through mass enumeration. Reward weights. We useλ f =0.5,λ c =1.0,λ s =2.0, and λ p =λ r =4.0, emphasizing IOC quality over format and ver- dict alone. C.2 Hyperparameters Table 7 summarizes the reward weights and inter- nal constants used throughout training. Type weights forhost,ip,file,urlare equal;filesubcomponent weights are 0.60/0.25/0.15 for file names/dir roots/subdirs. Table 7: Reward hyperparameters. SymbolValueDescription λ f 0.5Format weight λ c 1.0Classification weight λ s 2.0Self-consistency weight λ p 4.0IOC precision weight λ r 4.0IOC recall weight w both 1.00Gold∩ silver match weight w gold 0.85Gold-only match weight w silver 0.40Silver-only match weight α1.2Per-type FP coefficient in precision β0.45Aggregate FP penalty magnitude γ0.65Recall dampening rate ρ sus 0.10Suspected-status partial recall credit D Pipeline Prompts Note: the boxes below are simplified excerpts. The full prompts (verbatim) are released with the evaluation code. We use a single-pass prompt for short samples and a map/reduce pair for long samples. Each has a base variant and a tool-augmented variant; the latter only appends a TOOL USAGE block to the base system prompt. Single-pass extraction — system prompt Interpret the script, deobfuscate as needed, classify it as malicious or benign, and extract deterministic IoCs. Output three sections: <analysis> reasoning, classification justification <deobfuscated_code> minimal behaviour-focused reconstruction <answer> "classification": "malicious" | "benign", "ioc_status": host, ip, file, url -> P|A|S, "url_groups": [canonical_url, placeholder_values], "host":[...], "ip":[...], "file":[...], "url":[...] Status (P/A/S): present (concrete value recovered — list it), absent (no evi- dence), suspected (strong evidence, no concrete value after attempted decoding). Rules. Recovery before suspected; only values explicitly produced by the code. Map step — system prompt Analyse one chunk of a long script. Record partial observations only; do not classify; do not emit <analysis> / <deobfuscated_code> / <answer>. <chunk_summary> one paragraph <deobfuscation_notes> decodings from this chunk; if a blob (base64/hex/char-codes/reversed) cannot be fully decoded here, paste it verbatim with its suspected encoding. <candidate_iocs> host, ip, file, url (concrete values only) <unresolved> deps on other chunks Reduce step — system prompt Same as the single-pass prompt (same three sections, same schema, same status / recovery rules), plus: Cross-chunk synthesis. When a finding is<unresolved>in one chunk but another supplies the missing key/fragment, complete the decoding from the combined evidence and cite the recovered value. Never invent values the evidence does not support. Appended block — TOOL USAGE (tool variants) Call the appropriate tool rather than guessing; guessed decodings are not accept- able. "TVqQ..." -> base64_decode " 48..", hex -> hex_decode Chr(N)& / fromCharCode -> char_codes_to_string "%2F.." -> url_decode xor + key -> xor_hex StrReverse("...") -> string_reverse "pow"&"ersh" -> concat_strings "p-o-w".Replace -> split_and_join "[.]" -> "." -> string_replace ROT-N -> rot_n Call tools before writing the tagged sections; cite the exact tool output. Mark suspected only after a tool fails or returns non-text. Map variant: list cross-chunk dependencies in<unresolved>instead of guessing. Reduce variant: tools are used only to finish blobs left <unresolved> by the map step. 18 E Model and inference configurations We evaluate three model groups. The 8B-scale open-weight group includes Qwen3-8B, Ministral-3-8B-Instruct-2512, and Gemma-4-E4B-it, served locally with each model’s native context length asmax_model_len. The larger open-weight group includes Qwen3-Coder-Next, Qwen3-Coder-480B- A35B-Instruct, Qwen3.6-35B-A3B, and Llama-4-Scout-17B- 16E-Instruct, served locally or via the HuggingFace Inference Router under long-context configurations (Qwen3.6-35B-A3B at a262K served context, evaluated with three repeti- tions atT = 0). The frontier proprietary baselines are Gemini 2.5 Pro and Gemini 2.5 Flash, accessed through Google’s native SDK withmax_model_len=1,048,576, max_output_tokens=32,768, and automatic thinking- budget configuration. GPT-5.6-terra (gpt-5.6-terra), accessed via its API with a922K served context and evaluated with reasoning enabled; temperature was not configurable under this setting, so we used the API default sampling configuration. Per-model output budgets are sized to fit within the served context window: 8,192 tokens for Qwen3-8B, 4,096 for its agent variants (leaving room for tool-call history within the same served context), 32,768 for Ministral3-8B, 16,384 for Gemma4-8B, and 32,768 for the Gemini models. String-utility tool set. The string utilities exposed to the agent cover three categories of deterministic operations: de- code tools convert encoded representations into plaintext, in- cluding Base64, hexadecimal, URL encoding, and numeric character-code arrays; recover tools reconstruct obfuscated strings from script-level operations such as concatenation, split-and-join patterns, substring replacement, and string re- versal; and transform tools apply reversible string or byte transformations, including ROT-n, XOR, Base64 encoding, and hexadecimal encoding. All utilities are pure functions over strings and integer arrays: they do not execute code, ac- cess the filesystem, or open network connections, keeping the setting strictly static. For agent settings, we wrap the same vLLM-served model with a Qwen-Agent tool-calling wrapper that exposes these utilities and feeds tool outputs back into subsequent model calls. We enforce a per-sample limit of ten model calls. F Map–reduce Long-context Handling Samples exceeding the per-chunk code budget are processed with map–reduce. Each chunk produces partial findings, which are aggregated by a final reduce call into the struc- tured output. Map outputs are truncated to 1,500 tokens, so a sample split intoNchunks contributes approximately N· 1,500+ 1,000 tokens to the reduce prompt. LetL model be the served context length,P prompt ≈ 1,000the prompt overhead, andT out the reserved generation budget. We Table 8: URL extraction performance by matching strictness, micro-averaged over IOC instances. Model Endpoint Endpoint + keys Full URL PRF1PRF1 (∆)PRF1 (∆) Qwen3-8B33.6 27.9 30.5 31.2 26.1 28.4 (−2.1) 23.7 23.5 23.6 (−6.9) Ministral3-8B23.2 33.5 27.4 19.3 28.7 23.1 (−4.4) 13.1 25.7 17.3 (−10.1) Gemma4-8B45.2 12.2 19.2 43.3 11.9 18.7 (−0.5) 39.5 10.7 16.9 (−2.3) Qwen3-C80B ∗ 54.2 38.8 45.2 50.4 36.0 42.0 (−3.2) 44.3 32.4 37.4 (−7.8) Qwen3.6-35B51.6 39.2 44.6 50.5 38.3 43.5 (−1.0) 47.9 34.2 39.9 (−4.7) Llama4-S109B 40.0 34.8 37.2 35.6 32.3 33.9 (−3.3) 31.2 28.8 30.0 (−7.2) Qwen3-C480B ∗ 54.7 48.9 51.6 51.0 45.7 48.2 (−3.4) 49.1 41.2 44.8 (−6.8) Gemini-Pro66.3 63.8 65.1 63.9 61.7 62.8 (−2.2) 60.2 60.0 60.1 (−4.9) GPT-Terra70.5 60.3 65.0 68.8 58.7 63.3 (−1.6) 66.2 53.9 59.4 (−5.6) Gemini-Flash69.7 53.8 60.7 65.4 50.6 57.1 (−3.6) 54.7 47.9 51.1 (−9.7) set the per-chunk code budget to B = max 1024, 0.85· L model −(P prompt + T out )− 2048 . where the margin accounts for tokenizer drift and decoded- length expansion. 6 G URL extraction performance Table 8 reports URL extraction performance under three matching criteria: endpoint-level matching, endpoint-plus- query-key matching, and full-URL matching. Overall, F1 scores decrease as the matching criterion becomes stricter, in- dicating that models more often recover the correct endpoint than the exact query structure or full URL string. H False-positive Taxonomy: Labelling Rules We attribute each false positive (FP) to a single provenance category using a deterministic rule set. Notation.For samplesanddimensiond ∈ url, domain, ip, file, letG s,d and ˆ Y s,d be the canoni- calized reference and prediction sets (canonicalization follows the headline scorer), andF s,d = ˆ Y s,d \ G s,d . Let r,ℓ, andeddenote thedifflibsequence-matcher ratio, the longest common contiguous substring length, and the Levenshtein edit distance. Closest reference. Each FP f is paired with g ⋆ ( f) = arg max g∈G s,d max r( f , g), ℓ( f , g)/ min(| f|,|g|) . (2) Precedence. Predicates are evaluated top to bottom; the first match assigns the label, so every FP receives exactly one. Rules 1–2 handle malformed values, the rest well-formed 6 For 8 of 634 extreme-length samples, all evaluated with Qwen3-8B and split into more than 100 chunks, the accumulated map summaries reach the served context limit at the reduce stage; these cases are counted as response failures. All other models remain within the reduce budget. 19 ones. Category definitions and examples are in Table 6; each rule below states only its deciding predicate. 1.NORMALIZATION ERROR: a bounded deterministic nor- malization (scheme repair, leading-zero octet, case-fold) maps the ill-formed f to a reference IOC. 2.UNGROUNDED INVALID:fis ill-formed and traces to neither gold nor source. 3. BENIGN SHARED SERVICE:host( f)is a known benign service andfrelates to no reference IOC; source-absent benign services fall to rule 9 instead. 4. TYPE MISMATCH: normalizedfequals a reference value of a different IOC type (an IP in thedomainfield); do- main↔pathless-URL coincidences count as GRANU- LARITY. 5. GRANULARITY: the right entity at a different altitude, per type: the host or a path-prefix of a reference URL, or a parent/child of a reference domain; IPs admit exact matches only. 6.NEAR-COPY: a syntactically valid distortion ofg ⋆ : a same-type substring or superstring (a boundary mis- cut) or a fuzzy near-copy differing by a few characters (r( f , g ⋆ )≥ 0.90anded≤ 3, orr≥ 0.80with a long com- mon substring). A cut inside a DNS label is Near-copy; a cut at a path segment is Granularity. 7.PROMPT ECHO:f(or its host) occurs in the instruction template. 8. SOURCE NON-IOC:fis grounded in the source (raw or decoded) but is neither an indicator nor a benign service. 9.GENERIC PLACEHOLDER: a source-absent reserved/pri- vate/example pattern, or a benign service absent from the sample. 10.UNATTRIBUTED: none of the above—a specific, valid- looking value with no trace in gold, source, template, or patterns. The Disposition column of Table 6 gives each category’s likely operational handling if the error survives triage; it is a rough severity ordering rather than a single-property axis (UNATTRIBUTED spans O2–O3, since a plausible fabricated value both costs review time and can misdirect). Threshold sensitivity. The numerical cut-offs in the NEAR- COPY rule (ratios of0.90and0.80, edit distance≤ 3, LCS length≥ 8, normalised LCS≥ 0.6) were tuned on a held- out development split of false positives drawn from a non- evaluation model (Qwen2.5-1.5B-Instruct). Shifting either ratio threshold by±0.05moves at most2%of reported FPs between adjacent IOC-related categories and never across the non-IOC / IOC boundary; the cut-offs are reported for repro- ducibility rather than as theoretically motivated constants. I IOC-type-wise analysis Figure 8 shows substantial variation across script and IOC types. URL and domain extraction is relatively stable for the frontier models, suggesting greater robustness to obfuscation and contextual ambiguity. Open-weight models vary more substantially, with notable drops onJS h p , suggesting that their performance depends more strongly on indicators remaining close to surface form. Filesystem artifacts are the most challenging IOC category overall, with consistently low F1 across models and datasets. Unlike URLs or IP addresses, file indicators are less struc- turally constrained: filenames, paths, extensions, temporary artifacts, payload names, and benign-looking strings may all coexist in the same script. Correct recovery therefore requires contextual interpretation of which strings correspond to actual malicious file activity, rather than surface-pattern matching alone. Much of this difficulty is concentrated in the filename. A file indicator consists of a directory and basename, and ei- ther component may be assembled from split or transformed strings. Directories often follow a small set of familiar con- ventions (e.g.,%TEMP%,%APPDATA%), whereas basenames are more often sample-specific and constructed through concate- nation. Consistent with this distinction, directory-level scoring yields substantially higher F1 than full-path scoring: for GPT- Terra, 81.9 versus 61.4, and for Qwen3.6-35B, 67.8 versus 49.8. This consistent gap indicates that reconstructing the basename is the primary bottleneck in full-path recovery. J IOC Recovery-Level Labeling Each ground-truth IOC is graded by how it must be recovered from the script, using a fixed decoder as a reference model of mechanical recovery. The decoder serves as a released, repro- ducible instrument against which the recovery gap achieved by LLMs can be measured (§5.3.1). Transform set. The decoder covers reversible, program- independent transformations commonly documented in prior work on malicious-script obfuscation and deobfuscation [2, 3, 10, 20, 24, 28, 38, 52, 60]. As summarized in Table 9, Re- assembly operators reconstruct strings from characters al- ready present in the source, while Transcode operators invert reversible encodings or byte transformations. No operator interprets sample-specific control flow or dataflow. We exclude program-dependent computation (loops, arith- metic, and control-flow-driven decoders) because recovering those requires executing attacker-controlled logic rather than inverting a fixed codec; that exclusion is what the Program- dependent boundary encodes. Recovery levels. For each eligible (statically obtainable) IOC, we assign one of three levels: 20 Qwen3-8BMinistral3-8BGemma4-8BQwen3-C80BLlama4-S109BQwen3-C480BQwen3.6-35BGemini-ProGemini-FlashGPT-TerraIOCextractURLextractMSTICPy URL Domain IP File (a) JS f Qwen3-8BMinistral3-8BGemma4-8BQwen3-C80BLlama4-S109BQwen3-C480BQwen3.6-35BGemini-ProGemini-FlashGPT-TerraIOCextractURLextractMSTICPy (b) PS f URL Domain IP File (c) VBS f (d) JS hp 0.00 0.25 0.50 0.75 1.00 F1 Figure 8: F1 score across models, datasets, and IOC types. Each cell shows the model performance for a specific IOC type. Darker colors indicate higher F1. LevelAssigned when DirectThe value appears verbatim in the source Fixed-transformThe value appears in some blob produced by the decoder Program-dependentOtherwise: recovery requires reasoning over the script’s own decode logic For composite indicators, we assign the highest recovery level among their components. The labeling is deterministic and model-independent. Importantly, Program-dependent is a one-sided label: it denotes values not reached by the fixed decoder, not values that are impossible to recover statically. J.1 Where Frontier Models Succeed and Fail We further profile GPT-Terra, Gemini-Pro, and Gemini-Flash on the two harder recovery levels. Fixed-transform mechanisms. Table 10 shows that frontier models are generally strongest on common structural recon- struction, such as split/join and literal concatenation, while decoding-heavy transformations such as Base64 are less reli- able. Performance also varies substantially by operator and model. Program-dependent mechanisms. Table 11 groups Program- dependent IOCs by their dominant construction mechanism, including multi-variable dataflow, execution-mediated as- sembly, loop construction, nested encoding or compression, and arithmetic- or lookup-based construction. Unlike Fixed- transform, these indicators cannot be recovered through a predefined sequence of local transforms; recovery requires following computation specific to the sample. Recovery-Level Micro Recall by Dataset. Table 12 shows recall by recovery level within each dataset for three frontier models. The Direct>Fixed-transform>Program-dependent ordering holds in every dataset individually, so the recovery- level difficulty gradient tracks recovery level rather than dataset, collection period, or language. J.2 Case Study: Qwen3-8B Variants RFailPRF1 (a) JS f 0 20 40 60 RFailPRF1 (b) PS f 0 20 40 60 RFailPRF1 (c) VBS f 0 20 40 60 RFailPRF1 (d) JS hp 0 20 40 basetrain tooltrain + tool Figure 9: Effect of string utilities and task-specific adaptation on IOC extraction (Qwen3-8B variants), by script categories. 21 OperatorWhat it does Reassembly: reorders characters already in the source Literal concatJoins adjacent string literals Const. propagationSingle-assignment substitution ofvar←literal or prior-variable concat; no control flow, loops, or computed values Separator stripRemoves the most frequent repeatedn-gram, n∈ [3, 16] Split/joinRewrites split/join with literal arguments NUL stripRemoves embedded NUL bytes De-interleaveUnweaves interleaved character streams ReversalReverses the character order of a string Transcode: inverts a codec Base64Standard alphabet and a 64-char alphabet de- clared in the source HexHexadecimal decoding Char-code array Decimal char codes: plain, offset, and sign vari- ants PercentPercent-decoding JS escapes , Base64→ UTF-16LE Base64 decode reinterpreted as UTF-16LE Single-byte XOR Brute-forced over Base64/hex-decoded blobs under 4 KB ROT-n∈13, 47 Replace-chainsLiteral replace-chains,≤ 40 pairs Table 9: Transform set applied by the fixed decoder. Op- erators are applied recursively to depth4with hash-based de-duplication, a4MB per-blob cap, and≤ 60matches per operator, yielding candidate decoded blobs per script. Table 10: Per-transform recall (%) for the three frontier mod- els over Fixed-transform IOCs. Transformn GPT-Terra Gem.-Pro Gem.-Flash Reassembly (single) Literal concat11691.493.191.4 Split/join83100.085.583.1 Constant propagation2454.279.291.7 NUL strip7100.071.485.7 Reversal2100.0100.0100.0 Transcode (single) JS escapes6785.185.170.1 Base64→ UTF-16LE785.757.142.9 Base64771.471.471.4 Single-byte XOR20.00.00.0 Stacked (≥ 2 transform operations) Reassembly + transcode 16566.166.151.5 Reassembly≥ 2650.066.733.3 Transcode≥ 2616.70.00.0 Table 11: Program-dependent recall (%) for the three frontier models over Program-dependent indicators, grouped by dom- inant construction mechanism (source-pattern heuristics). Mechanismn GPT-Terra Gem.-Pro Gem.-Flash Multi-variable dataflow36230.949.729.6 Execution-mediated52238.549.838.9 Loop-constructed1509.316.74.0 Nested encoding / compression 14351.744.131.5 Computed-arithmetic / lookup5366.073.637.7 Overall Program-dependent123035.446.131.0 Table 12: Micro recall (%) by recovery level within each dataset. L0 = Direct, L1 = Fixed-transform, L2 = Program- dependent. GPT-TerraGem.-ProGem.-Flash DatasetL0L1L2L0L1L2L0L1L2 JS f 95.433.310.789.844.413.151.922.211.9 PS f 75.658.820.077.550.010.075.047.16.0 V BS f 90.147.831.486.154.339.787.152.26.6 JS h p 96.586.138.993.983.951.696.175.736.9 22