Paper deep dive
Implicit Patterns in LLM-Based Binary Analysis
Qiang Li, XiangRui Zhang, Haining Wang
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/22/2026, 6:13:00 AM
Summary
This paper presents a large-scale, trace-level empirical study of LLM-based agents performing iterative binary vulnerability analysis. By analyzing 99,563 reasoning steps across 521 binaries, the authors identify four recurring token-level implicit patterns—early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritization—that govern how agents organize exploration without explicit control-flow heuristics.
Entities (6)
Relation Signals (4)
LLM-based agents → exhibitspattern → Early pruning
confidence 95% · we identify four dominant patterns: early pruning... that emerge implicitly from reasoning traces.
LLM-based agents → exhibitspattern → Path-dependent lock-in
confidence 95% · we identify four dominant patterns... path-dependent lock-in... that emerge implicitly from reasoning traces.
LLM-based agents → exhibitspattern → Targeted backtracking
confidence 95% · we identify four dominant patterns... targeted backtracking... that emerge implicitly from reasoning traces.
LLM-based agents → exhibitspattern → Knowledge-guided prioritization
confidence 95% · we identify four dominant patterns... knowledge-guided prioritization... that emerge implicitly from reasoning traces.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Binary vulnerability analysis is increasingly performed by LLM-based agents in an iterative, multi-pass manner, with the model as the core decision-maker. However, how such systems organize exploration over hundreds of reasoning steps remains poorly understood, due to limited context windows and implicit token-level behaviors. We present the first large-scale, trace-level study showing that multi-pass LLM reasoning gives rise to structured, token-level implicit patterns. Analyzing 521 binaries with 99,563 reasoning steps, we identify four dominant patterns: early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritization that emerge implicitly from reasoning traces. These token-level implicit patterns serve as an abstraction of LLM reasoning: instead of explicit control-flow or predefined heuristics, exploration is organized through implicit decisions regulating path selection, commitment, and revision. Our analysis shows these patterns form a stable, structured system with distinct temporal roles and measurable characteristics. Our results provide the first systematic characterization of LLM-driven binary analysis and a foundation for more reliable analysis systems.
Tags
Links
- Source: https://arxiv.org/abs/2603.19138v1
- Canonical: https://arxiv.org/abs/2603.19138v1
Trouble viewing inline? Open PDF directly →
Full Text
101,656 characters extracted from source content.
Expand or collapse full text
Implicit Patterns in LLM-Based Binary Analysis Qiang Li Beijing Jiaotong University China liqiang@bjtu.edu.cn XiangRui Zhang Beijing Jiaotong University China Haining Wang Virginia Tech USA hnw@vt.edu Abstract Binary vulnerability analysis is increasingly performed by LLM- based agents in an iterative, multi-pass manner, with the model as the core decision-maker. However, how such systems organize exploration over hundreds of reasoning steps remains poorly under- stood, due to limited context windows and implicit token-level be- haviors. We present the first large-scale, trace-level study showing that multi-pass LLM reasoning gives rise to structured, token-level implicit patterns. Analyzing 521 binaries with 99,563 reasoning steps, we identify four dominant patterns—early pruning, path- dependent lock-in, targeted backtracking, and knowledge-guided prioritization—that emerge implicitly from reasoning traces. These token-level implicit patterns serve as an abstraction of LLM rea- soning: instead of explicit control-flow or predefined heuristics, exploration is organized through implicit decisions regulating path selection, commitment, and revision. Our analysis shows these patterns form a stable, structured system with distinct temporal roles and measurable characteristics. Our results provide the first systematic characterization of LLM-driven binary analysis and a foundation for more reliable analysis systems. Keywords LLM Agents, Binary Vulnerability Analysis, Reasoning Patterns ACM Reference Format: Qiang Li, XiangRui Zhang, and Haining Wang. 2026. Implicit Patterns in LLM-Based Binary Analysis. In . ACM, New York, NY, USA, 18 pages. https: //doi.org/10.1145/n.n 1 Introduction Binary vulnerability analysis [8,9,21,26,38] fundamentally con- cerns identifying and reasoning about execution paths that con- nect attacker-influenced sources to security-critical sinks under severe uncertainty. In practice, such analysis follows two distinct paradigms: one-pass analysis and iterative analysis. In one-pass analysis, static analysis constructs a global program representation once, and reasoning operates over this fixed view [3,11,18]. In con- trast, iterative analysis interleaves reasoning and tool interaction, where each decision is conditioned on prior observations, reflecting how human analysts explore binaries in practice. This distinction is fundamental: iterative analysis transforms vulnerability discovery from a static pipeline into a sequential, decision-driven process. Recent advances in large language models (LLMs) have enabled their application to binary analysis tasks, including disassembly nav- igation, code reasoning, and interactive, tool-driven exploration [5, Artifacts: https://github.com/bjtu-SecurityLab/pattern-tl. Conference’17, Washington, DC, USA 2026. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/10.1145/n.n 6,14,18,20,27]. Crucially, LLMs enable the automation of iterative analysis by acting as the core decision-maker in the analysis loop. At each step, the model invokes analysis tools and decides which paths to pursue, defer, or abandon—decisions that directly deter- mine whether vulnerability-relevant behaviors are observed. As a result, the iterative mode, previously requiring human expertise, becomes automatable. This shift from one-pass to multi-pass rea- soning fundamentally changes how static analysis participates in vulnerability discovery, raising a key question: how do LLM agents organize and sustain exploration under such settings? However, despite growing interest, little is known about how LLM-based systems actually organize binary path exploration. This gap arises because understanding LLM-driven iterative analysis is fundamentally more challenging than analyzing traditional one- pass systems. First, agents operate over long analysis sessions span- ning hundreds of reasoning steps, where limited context windows and attention drift lead to fragmented observations rather than a coherent global view. Second, LLM reasoning operates at the token level: behaviors such as prioritization, pruning, and backtracking are not explicitly encoded as program logic or control structures, but emerge implicitly from sequential tokens. This contrasts with traditional analysis systems, where such behaviors are explicitly defined and enforced. As a result, it remains unclear whether LLM agents exhibit systematic, structured reasoning behaviors, or merely produce ad hoc decisions tied to specific inputs. To address this gap, we show that multi-pass LLM reasoning gives rise to structured, token-level implicit patterns that govern long-horizon binary analysis. We present the first large-scale, trace- level empirical study of how LLM-based systems explore binary execution paths in vulnerability analysis. We analyze reasoning traces from 521 real-world binaries across ARM and MIPS archi- tectures, comprising 99,563 reasoning steps. From these traces, we identify four dominant patterns—early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritiza- tion—that emerge implicitly from token-level sequential reasoning rather than being externally prescribed by prompts or program logic. These token-level implicit patterns serve as an abstraction of LLM reasoning: instead of exhaustive enumeration, explicit control- flow traversal, or predefined heuristics, exploration is organized through implicit, token-level decisions that regulate path selection, commitment, and revision over time. Our analysis reveals three key properties of these patterns. First, they are stable and repeatedly invoked across sessions rather than sporadic behaviors: path lock-in and knowledge-guided prioriti- zation appear in 97.6% of sessions, early pruning in 83.5%, and backtracking in 93.8%. Moreover, they exhibit distinct temporal roles, with lock-in dominating early phases, pruning emerging mid-process, and backtracking concentrated in later stages. Second, the patterns form structured relationships rather than operating arXiv:2603.19138v1 [cs.AI] 19 Mar 2026 Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang independently. In particular, path lock-in and early pruning form a bidirectional loop accounting for 79.4% of pattern transitions, while lock-in and prioritization exhibit complementary dynamics. Third, each pattern corresponds to distinct, quantifiable behavioral characteristics, as reflected in differences in path length, branch- ing behavior, and backtracking dynamics. Together, these results show that LLM-driven binary analysis is governed by structured reasoning mechanisms rather than ad hoc exploration. These findings have implications for both security analysis and LLM-based systems. For binary analysis, they demonstrate that vulnerability-oriented exploration can emerge without explicit control-flow reconstruction, through iterative, semantic-guided reasoning. For system design, they provide concrete behavioral primitives that can inform more controllable and effective LLM- based analysis frameworks. The main contributions of this paper are as follows: •We present the first trace-level empirical study of long-horizon, iterative LLM-driven binary analysis, focusing on how agents organize exploration through multi-pass reasoning rather than one-pass execution. •We introduce an iterative perspective that reframes binary analysis as a sequential decision process, contrasting with the one-pass paradigm in both traditional static analysis and prior LLM-based approaches. •We identify four recurring token-level implicit patterns as an abstraction of LLM reasoning beyond explicit program structures or predefined heuristics. •We show that these patterns have distinct roles, temporal dy- namics, and measurable behavioral characteristics, revealing systematic reasoning mechanisms underlying LLM-based binary exploration. 2 Background & Motivation 2.1 One Time or Multiple Times? Binary vulnerability detection can follow two fundamentally differ- ent execution paradigms, illustrated in Figure 1: single-pass analysis and iterative analysis. The key difference between these paradigms lies in the role of static analysis within the vulnerability detection process. One-pass mode. Traditional static analysis tools—disassemblers, decompilers, CFG builders—run once to produce a program repre- sentation. Subsequent reasoning, whether rule-based, symbolic, or LLM-assisted operates over this artifact. The pipeline is: Binary→ Static analysis→Representation→Slicing→Reasoning . Existing LLM-based binary analysis work follows this structure [3,11,18]: the LLM receives a slice of representation (e.g., disassembly, decom- piled code) and reasons over it; static analysis remains a one-time preprocessing step producing a fixed representation. Iterative mode. In practice, expert analysts rarely follow a strict one-pass workflow. Analysts often refine their understanding of a binary through additional analysis steps during investigation. They repeatedly invoke static analysis: inspect a function, decide what to examine next based on the result, inspect another region, and so on. Reasoning and static analysis are interleaved throughout: Binary → Reasoning Binary Static Analysis Representation (CFG / IR / Decompiled Code) Reasoning (rule engine / LLM) Vulnerability Detection Binary Reasoning State Analysis Action Partial Artifact (function / CFG slice / dataflow) State Update Binary Code Space [█░] explored region One Pass Iterat- ive Pass Figure 1: (Left) One-pass paradigm: static analysis constructs a global program representation and vulnerability reasoning operates over this fixed view. (Right) Iterative paradigm: rea- soning interleaves with repeated static analysis operations, proceeding incrementally through multiple tool invocations. → Static analysis → Observation → Reasoning (loop) Each invocation (e.g., disassembling a function, resolving cross- references) is a static analysis operation; reasoning emerges incre- mentally, guided by the preceding observation at each step. This iterative mode is how experts actually work; it has not been sys- tematically studied in automated analysis. 2.2 LLM-Driven Iterative Loop LLM agents that interact with binary analysis tools through tool invocation form a reasoning–action–observation loop. We focus on agents where the LLM is the sole decision-maker—no exter- nal search algorithm or program orchestrates exploration; control emerges from the token-level reasoning that drives each tool invo- cation. At each step, the agent selects an analysis command (e.g., disassemble a function, resolve cross-references); the tool returns a localized program fragment; the agent incorporates this into its reasoning and selects the next command. This loop is a concrete instantiation of the iterative mode: each action is a static analysis operation, and agents perform hundreds of them in sequence. LLM agents thus automate the iterative workflow that experts perform manually, producing observable traces that record how analysis unfolds over time. Figure 2 illustrates this process. With only a fragment exposed per step, the agent has no global view; it must repeatedly decide what to analyze next, which path to pursue, and which to abandon. The trace—an ordered sequence of reasoning steps and analysis ac- tions—captures these decisions. Such traces exhibit strong temporal dependencies (early decisions shape later exploration) and differ fundamentally from chain-of-thought reasoning [28,29,34,35]: they are grounded in concrete tool invocations and observable pro- gram states, not purely internal language. In this work, we treat these traces as the primary object of study and analyze token-level implicit patterns that emerge when agents conduct static analysis under partial observability and bounded resources. Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Reasoning State Actions Observa tions "Initializing binary metadata" [Step 0] i [Step 1] afl Architecture: ARM32 Entry: 0x1000 Bitness: 32 "Locating entry points functions" [Step 2] is~main Func: 26 functions Symbols: main found Boundaries identified "Inspecting entry function structure" [Step 3] pdf @ entry0 Entry function CFG: → fcn.0000db2c → fcn.0000b208 Stage 1Time "Exploring function boundaries" [Step 5] pdf @ fcn.0000b208 fcn.0000b208 tracing "Expanding to adjacent functions" [Step 6] pdf @ fcn.0000ac9c fcn.0000ac9c tracing "Identifying semantic anchors " [Step 7] iz Strings: 26 found ['cmd', 'system', ...] String references "Tracing cross-refer to sinks" [Step 8] axt @@ sym.doSystemCmd Cross-refs to doSystemCmd: 0xdf54, 0xe234, ... 3 call sites "Inspecting instructions near call site" [Step 9] pd 20 @ 0xdf54 Instructions at 0xdf54: mov r0, #0 bl doSystemCmd Local semantics "Backward inspection for dependencies" [Step 10] pd -30 @ 0xdf54 Backward slice: Dependencies from 0xdf20 Control flow "Resolving additional references" [Step 11] axt @@ reloc.wanid Decompiled code: if (condition) doSystemCmd(...) Abstract CFG ...... ........ .......... Stage 2Stage 3 ......... Figure 2: An example of step-by-step LLM-driven binary analysis, illustrating how reasoning states evolve through repeated interactions with external analysis tools. 3 Problem Formulation LLM-based binary analysis agents interact with executables through long sequences of tool invocations, observations, and intermedi- ate reasoning steps. These interactions form long-horizon traces in which exploration control emerges implicitly from token-level reasoning rather than explicit program logic. This work studies the following problem: •What recurring token-level implicit patterns emerge in long-horizon traces produced by LLM agents? 3.1 Trace Representation We consider an analysis session as a complete interaction between an LLM agent and a target binary, starting from initial reconnais- sance and ending when the agent terminates or reports findings. Each session produces an ordered trace푇=⟨푠 1 ,푠 2 , . . .,푠 푇 ⟩, where 푠 푖 corresponds to a step in the LLM-driven binary analysis process. Each step푠 푖 =(푎 푖 ,표 푖 ,푟 푖 )consists of the issued tool command푎 푖 , the tool output표 푖 , and the reasoning state푟 푖 maintained by the agent up to step푖. This representation captures how decisions evolve over time and depend on earlier observations and reasoning. In practice, traces span hundreds of steps and frequently revisit the same code regions, creating long-range dependencies between early decisions and later actions. 3.2 Token-Level Implicit Behaviors In single-pass binary analysis (Section 2), whether rule-based, sym- bolic, or LLM-assisted, behaviors such as prioritization, pruning, focus, and backtracking are explicit: priority queues, branch cuts, de- liberate path selection, and stack-based backtracking are first-class operations. In LLM agents where the model is the core decision- maker, these behaviors become implicit: the agent operates at the token level with no explicit data structures for active paths or deferred candidates; prioritization, pruning, and backtracking man- ifest in the token stream and tool-invocation sequence. Exploration control is thus not directly observable as program structure, but must be inferred from trace dynamics. Extracting and character- izing these behaviors from traces is essential for understanding LLM-based analysis systems, yet they have not been systematically studied in this form. reported separately Binary Corpus (521 binaries) LLM Agent × Run (521×4 sessions) Selected Session(1 per binary) Reasoning Trace (99,563 steps) Vulnerability Metadata (Appendix C) Primary Analysis Unit generates input filters Figure 3: Dataset overview: the relationship between the bi- nary corpus, analysis sessions, reasoning traces, and output metadata. 3.3 Extracting Token-Level Patterns from Traces Our goal is not to assess whether an individual analysis session succeeds, nor to measure vulnerability coverage. Instead, we aim to characterize the implicit structural mechanisms by which LLM agents organize exploration over time. Specifically, we study re- curring behaviors related to path selection, persistence, revisiting, and backtracking—mechanisms that are not explicitly encoded, but emerge from token-level reasoning dynamics and can be identi- fied from observable trace structure. By identifying these patterns consistently across binaries and sessions, we reveal organizational principles of LLM-driven binary exploration that are not captured by outcome-based metrics. 4 Dataset Our dataset consists of long-horizon reasoning traces collected from LLM-driven binary analysis sessions. Unlike datasets that focus on binary coverage or vulnerability labels, ours captures the temporal structure of reasoning traces for agent behavior analysis. We treat the analysis session as the primary unit of analysis, rather than the binary itself. Each session corresponds to one complete interaction between an LLM agent and a target binary, producing a temporally ordered trace of reasoning–action–observation steps. Figure 3 il- lustrates the hierarchical relationship between the binary corpus, Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang Table 1: Dataset and session overview. ItemCount Total analysis binaries521 ArchitecturesARM: 328; MIPS: 193 OSLinux:519; Android:2 EndiannessLittle Endian: 477; Big Endian: 44 FormatELF32: 521 Reasoning sessions521 Total reasoning steps99,563 Median steps per session160.0 Average steps per session191.1± 154.9 LLM:DeepSeek-V3, GPT-5, Claude 3.5 Sonnet, Gemini 3.0 All sessions521×4 analysis sessions, and reasoning traces. The dataset construction and session format are designed to support reproducible analysis of agent behavior under long-horizon, tool-mediated interaction. 4.1 Binary Corpus The binary corpus serves as a source of diverse analysis contexts, rather than the primary object of study. We construct the binary corpus through a three-stage process. First, we download firmware images from the Karonte dataset [23], extract their filesystems, and collect 30,195 binaries from Linux-based firmware images. Sec- ond, we apply Mango [9], a static analysis tool that identifies bi- naries likely to contain taint-style vulnerabilities (e.g., command injection and path traversal), reducing the corpus to 3,500 security- relevant binaries. Third, we deduplicate binaries by filename and randomly select one binary per unique filename, yielding 521 dis- tinct binaries for analysis. Filename-based deduplication is adopted because our focus is on token-level implicit patterns rather than binary-specific vulnerability coverage. Although binaries sharing the same filename may differ across compilation options or ver- sions, they typically implement similar functionality. This strategy avoids over-representation of popular binaries while preserving program diversity. Table 1 summarizes the corpus and session characteristics. The corpus spans multiple architectures, operating systems, and endi- anness; all binaries are ELF32 format, providing consistency while maintaining diversity in code structure and complexity. These bi- naries contain known vulnerabilities and have been widely used as datasets in vulnerability detection research [2,7,9,23,30], which target taint-style vulnerabilities in firmware binaries. Our goal is not exhaustive vulnerability coverage, but to observe sustained exploration behavior under realistic analytical conditions. 4.2 Session Generation Each binary is analyzed independently by an LLM agent equipped with a fixed tool interface. Agents interact with binaries exclusively through standard program analysis tools (radare2andGhidra, via the r2ghidra plugin) and receive no external guidance or human intervention. For each binary and each LLM, we generate one continuous ses- sion—a temporally ordered trace of reasoning–action–observation steps—from initial reconnaissance to termination. At each step, the trace records: (1) the agent’s reasoning state, (2) the selected tool action, and (3) the resulting observation returned by the tool. The complete session format, initialization prompt, and JSONL record structure are specified in Appendix A. Achieving long-horizon sessions (typically 130–300 steps) under LLM context window constraints requires explicit context manage- ment. To this end, traces include LLM context reset records, each representing a new LLM instance initialized with a compressed sum- mary of prior analysis state. The first context reset record contains the unified initialization prompt, while subsequent records preserve accumulated findings and task state. Context resets allow traces to extend beyond single-context limits while preserving observable reasoning behavior; all tool actions and reasoning states remain directly recorded in the trace, enabling the study of long-horizon dynamics. One reasoning segment within a larger vulnerability analysis session is illustrated in Appendix B.2. We analyze four LLMs (DeepSeek-V3, GPT-5, Claude 3.5 Sonnet, Gemini 3.0), generating four independent sessions per binary. For primary analysis, we select one representative session per binary to avoid inter-session dependency. Session selection prioritizes trace completeness and length and does not filter based on vulnerability outcomes or analysis success, illustrated in Appendix B.1. This setup produces traces where exploration behavior unfolds over time as a dynamic process. 4.3 Trace Statistics Table 1 summarizes the dataset characteristics. Sessions vary sub- stantially in length, with a median of 160 steps and an average of 191 steps per session (standard deviation 154.9), reflecting the varying complexity of binary analysis tasks. The total dataset con- tains 99,563 reasoning steps, providing sufficient scale to identify stable token-level implicit patterns across diverse contexts. The scale and variability of traces provide a basis for identifying sta- ble patterns that are not tied to specific binaries or tasks. Note that the analyzed binaries predominantly contain real-world vul- nerabilities: 38.0% of analysis sessions (198 out of 521) identify at least one CWE-labeled vulnerability finding, with a total of 306 distinct vulnerability instances across the dataset (detailed statistics reported in Appendix C). This ensures that observed behaviors arise from non-trivial security analysis tasks rather than trivial or empty exploration scenarios. 5 Pattern This section presents the four token-level patterns we extract from traces. 5.1 Pattern Taxonomy We identify four recurring token-level patterns that characterize how LLM agents explore binary programs during vulnerability analysis. These patterns capture exploration-regulation behaviors emerging in traces under bounded reasoning capacity. Pattern P4 (Knowledge-Guided Prioritization) corresponds to knowledge- guided prioritization behavior that ranks candidate paths based on prior knowledge and structural cues. Patterns P1–P3 describe ob- servable dynamics in traces: early pruning behavior in early phases, Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Table 2: Summary of token-level reasoning patterns in binary analysis. PatternCore BehaviorPrimary Benefit P1: Early Prun- ing Traceexhibitsearly pruning behavior: can- didate paths discarded early, rarely revisited. Efficient reduction of large search spaces. P2: Path Lock- in Trace exhibits path lock- in behavior: reasoning re- mains focused on a se- lected path. Sustainedcontextual coherence during deep analysis. P3:Targeted Backtracking Trace exhibits targeted backtracking behavior: previouslydeferred paths revisited. Recovery from incom- plete or unproductive analysis paths. P4: Knowledge- Guided Prioriti- zation Traceexhibits knowledge-guided prioritization behavior: paths ranked using prior knowledge. Rapid prioritization of analysis directions. start_stepprune_stepprune_step +span ✂ ✂ ✂ Path A Path B Path C Path D Path E 5 candidate paths2 active path 3 pruned path Figure 4: Pattern 1: pruning behavior in traces—candidate paths discarded early, rarely revisited. path lock-in behavior during sustained analysis, and targeted back- tracking behavior when revisiting deferred candidates. Table 2 summarizes the core behaviors and primary benefits of these patterns. To extract patterns from traces, we adopt a rule- based approach that identifies observable structural features (e.g., revisits, path divergence, and persistence) from action–observation sequences. The patterns are empirical regularities emerging in traces; the rules serve only as observable proxies for detection, not as definitions. These patterns are not designed or programmed; they emerge consistently from token-level reasoning dynamics across sessions. Full rule definitions and implementation details are provided in Appendix D (Algorithms 1–4). 5.2Pattern 1: Early Pruning of Candidate Paths Binary vulnerability analysis often generates multiple candidate execution paths as functions, control-flow branches, and cross- references expand the exploration space. Under bounded reasoning capacity, trace segments frequently exhibit early pruning behavior start_stepspan_stepend_step Alternative Path A Main Path Path Selection Alternative Path B ≈ Contradictory Evidence Locked-in Absorbed (Ignored) (Ignored) attention & ignored attention & ignored Contradictions Figure 5: Pattern 2: lock-in behavior in traces—sustained reasoning within the same context, limited exploration of alternatives. early in the analysis process, narrowing the candidate set. Instead of exploring alternatives uniformly, reasoning in the trace discards paths that appear unlikely to yield exploitable behavior. Once early pruning behavior occurs, subsequent reasoning steps rarely revisit the discarded paths, even when additional context later becomes available. We label this pattern Early Pruning of Candidate Paths. Figure 4 illustrates the typical temporal structure: an initial explo- ration phase generates multiple candidate paths, followed by early pruning behavior that removes several alternatives, after which reasoning continues primarily along the retained path. Detection Operation. Pattern 1 is identified when a trace seg- ment exhibits the following observable sequence: (1) explicit reason- ing about multiple candidate analysis paths, (2) a pruning decision that eliminates one or more alternatives based on diagnostic obser- vations, and (3) continued reasoning that focuses exclusively on the retained path without revisiting discarded alternatives within a bounded observation window. Example Trace Snippet. In thepptp-ondemandbinary session, the LLM agent initially explores several potential call sites: "The function is called from three locations in main. I need to examine each call site to understand the arguments passed, especially param_2 and param_3, to determine if they are tainted from external sources." After several reasoning steps, the agent identifies a promising vul- nerability path: "I have the full assembly of the function. I can see the buffer overflow clearly. At address 0x004011d8, there is ’addiu v0, fp, 0x118’, which references the stack buffer..." From this point onward, the analysis focuses exclusively on the identified path, while previously considered call sites are not revisited. 5.3 Pattern 2: Path-Dependent Analysis Lock-in Binary vulnerability analysis often requires extended reasoning along a single candidate execution path, such as tracing data prop- agation through multiple functions or examining a sequence of control-flow decisions. However, switching analysis context be- tween different candidate paths can disrupt reasoning coherence, particularly in long-horizon analyses. Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang start_stepbacktrack_stepend_step Alternative Path A Main Path Alternative Path B (Ignored) (Ignored) ↩ Materialized Path A Materialized Path B Figure 6: Pattern 3: backtracking behavior in traces—when the active path stalls, reasoning returns to a previously de- ferred candidate. Traces therefore often exhibit path lock-in behavior once an analysis path has been selected in the token stream. Subsequent reasoning steps remain strongly conditioned on the established con- text, repeatedly examining related functions, instructions, and data flows associated with the chosen hypothesis. Even when partial contradictions or ambiguous signals appear, the trace often contin- ues within the same path rather than immediately switching to an alternative. We label this pattern Path-Dependent Analysis Lock-in. Figure 5 illustrates the typical temporal structure: path-selection behavior early is followed by sustained reasoning within the same semantic context, while alternative paths receive little or no further exploration. Detection Operation. Pattern 2 is identified when a trace seg- ment exhibits the following observable sequence: (1) an explicit path-selection decision focusing analysis on a specific function, sink, or execution path, (2) an extended reasoning span in which subsequent actions repeatedly operate within the same semantic context (e.g., examining related functions, addresses, or control-flow structures), and (3) limited expansion of alternative paths despite occasional mentions. Example Trace Snippet. In the3322ipbinary session, the trace exhibits path-selection behavior: "The main function is large and complex. I can see several interesting things: it calls doSystemCmd at multiple locations (0xdf54, 0xe198, 0xe27c, 0xe628). The doSystem- Cmd is a dangerous sink (command injection). I need to identify taint sources that could reach doSystemCmd." Over the next several rea- soning steps, path lock-in behavior emerges: the trace continues examining functions and instructions related to thedoSystemCmd path, repeatedly tracing potential data flows toward this sink. Even when intermediate observations weaken the initial hypothesis, the trace maintains focus on the same analysis path rather than switch- ing to alternative sinks or unrelated functions. 5.4 Pattern 3: Multi-Path Exploration with Targeted Backtracking Binary vulnerability analysis rarely follows a single linear reasoning path. Early hypotheses may later prove incomplete or incorrect, requiring analysts to revisit alternative candidate paths that were previously considered but not fully explored. startdecisionanalysis span Path A Path B Path C High Priority (Medium Priority) (Low Priority) ≈ Analogy: Similar to known vulnerability│ Signal: Strong taint indicator │ Signal: Moderate risk indicators │ No strong signals │ ═ (Detailed Analysis) │ ═ (Moderate Analysis) │ Figure 7: Pattern 4: prioritization behavior in traces—paths ranked early using prior knowledge, leading to uneven allo- cation of effort. Traces often exhibit multiple candidate paths as lightweight hy- potheses within the reasoning context. Only a subset of these paths is expanded at any given time, while others remain deferred but implicitly tracked as potential alternatives. When the active analy- sis path reaches an impasse or when new evidence suggests that another path may be more promising, targeted backtracking behav- ior emerges—the trace returns to one of these previously deferred candidates and continues exploration. We label this pattern Multi- Path Exploration with Targeted Backtracking. Figure 6 illustrates the typical temporal structure: multiple candidate paths are initially identified, one path is developed in depth, and a previously deferred candidate is later revisited and expanded. Detection Operation. Pattern 3 is identified when a trace seg- ment exhibits the following sequence: (1) multiple candidate paths are explicitly mentioned during early exploration, (2) one candidate path is selected for deeper analysis while others remain undevel- oped, and (3) a previously deferred candidate path is later revisited and expanded after a substantial reasoning interval. Example Trace Snippet. In the3322ipbinary session, the trace initially enumerates several potential analysis directions while inspecting program structure and function calls. One path focusing on network-related functions is developed in depth. Later in the trace, targeted backtracking behavior emerges: reasoning returns to a previously mentioned candidate involving thedoSystemCmd sink: "This function handles network connection setup and also calls doSystemCmd with format strings ’ip rule del to %s’ and ’ip rule add to %s table wan %d’...". The trace then expands this deferred candidate path, combining information from earlier exploration with the newly examined function. 5.5 Pattern 4: Knowledge-Guided Prioritization Binary vulnerability analysis often exposes a large number of po- tential execution paths. Fully exploring every candidate path is computationally infeasible, requiring analysts to prioritize which paths deserve deeper investigation. Traces frequently exhibit knowledge-guided prioritization be- havior, leveraging knowledge encoded in the model’s pre-training. Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Rather than waiting for complete information, reasoning in the trace evaluates candidate paths using partial evidence and struc- tural cues, such as known dangerous functions, library calls, data- flow signals, or code patterns that resemble previously observed vulnerabilities. Paths that appear more promising receive deeper exploration in subsequent steps, while others receive limited atten- tion or remain deferred. We label this pattern Knowledge-Guided Prioritization. Figure 7 illustrates the typical temporal structure: multiple candidate paths are identified, early reasoning exhibits knowledge-guided prioritization behavior assigning higher priority to some paths based on prior knowledge and partial signals, and subsequent analysis effort becomes unevenly distributed across these paths. Detection Operation. Pattern 4 is identified when a trace seg- ment exhibits the following conditions: (1) the agent explicitly ranks, selects, or prioritizes among multiple candidate paths, and (2) the decision is justified using references to known vulnerability patterns, risky functions, structural cues, or partial data-flow evi- dence. The prioritization decision and its justification must occur within the same reasoning step or in closely adjacent steps. Example Trace Snippet. In the3322ipbinary session, the trace exhibits knowledge-guided prioritization behavior toward a command execution path: "The main function calls doSystemCmd at multiple locations. The doSystemCmd function is a dangerous sink for command injection. I should identify possible taint sources that could reach this function." Based on the known risk associated with command execution sinks, the trace allocates detailed analysis effort to paths involvingdoSystemCmd, while other functions receive limited exploration. 6 Empirical Analysis This section presents our empirical analysis of the reasoning pat- terns. We are attempting to answer three research questions. RQ1: To what extent are the four patterns prevalent across analysis sessions, and how frequently are they invoked when present? (§ 6.1) RQ2: How do patterns transition and organize into sequential struc- tures during reasoning? (§ 6.2) RQ3: How do different patterns show distinct behavioral characteris- tics as measured by trace-level metrics? (§ 6.3) Specifically, RQ1 evaluates whether patterns are consistently present across sessions; RQ2 examines whether patterns exhibit structured relationships rather than random occurrences; and RQ3 analyzes whether patterns correspond to distinct behavioral characteristics. Pattern Usage vs. Vulnerability Outcomes. While one may ask whether certain patterns lead to higher vulnerability discovery success, our study does not treat pattern usage as a performance predictor. In our setting, pattern usage is primarily driven by binary complexity and reasoning dynamics, whereas vulnerability discov- ery outcomes depend on factors such as vulnerability presence, exploitability, and analysis difficulty. As a result, there is no direct causal relationship between the frequency of specific patterns and vulnerability discovery outcomes. Table 3: Pattern prevalence and density per session. Sessions w/ Pattern Coverage Total Avg / Session Avg / Active Session Max P1435 / 52183.5%3,3396.417.6844 P2509 / 52197.6% 9,654 18.5318.9977 P3489 / 52193.8% 995 1.912.035 P4509 / 52197.6% 14,083 27.0327.71278 6.1 Pattern Prevalence and Density We analyze pattern prevalence (the proportion of sessions contain- ing each pattern) and density (the frequency of pattern use when present). Table 3 summarizes the key statistics. Pattern Prevalence. Table 3 shows large differences in pattern prevalence: P2 and P4 appear in 97.6% of sessions (509 out of 521), appearing in almost all reasoning sessions. P3 appears in 93.8% of sessions (489 out of 521), while P1 appears in 83.5% of sessions (435 out of 521). This variation shows that patterns are not uniformly present across all sessions. P1’s lower prevalence (83.5%) suggests it acts as a filtering mechanism—activated when needed for search control, but not mandatory for successful reasoning. P2 and P4 ap- pear in almost all sessions (97.6%), indicating they are fundamental components of LLM reasoning, appearing in almost every analysis session regardless of binary complexity or LLM model. Pattern Density per Session. We quantify pattern density as the number of pattern instances per session, measuring how often each reasoning behavior occurs during a single reasoning session. Table 3 shows large differences in usage frequency. P4 averages 27.7 instances per active session, far exceeding others. P2 averages 19.0 instances, P1 averages 7.7, while P3 averages only 2.0 instances per active session. These differences suggest that some patterns are used repeatedly (P4, P2), while others (P3) are used selectively and sparingly. The maximum values show another important dis- tinction: P4 shows extreme bursts (max=278), indicating that it can dominate decision-making in certain contexts. In contrast, P3 stays within a small range (max=5), never exceeding a small num- ber of uses. This constraint suggests P3 acts as a limited recovery mechanism rather than a dominant mode of operation. We further examine the overall patterns per session—the total number of pattern instances across all patterns within a session. This session-level view shows the intensity of pattern-driven rea- soning across different analysis contexts. Table 4 shows the distribu- tion of total pattern instances across sessions. Most sessions (36.08%) contain 51-100 total pattern instances, with large differences rang- ing from sessions with very few instances (1-10) to sessions with extremely high activity (201+). This variation reflects differences in binary complexity, analysis depth, and reasoning requirements across different tasks. This observation shows that the LLM agentic system uses a dynamic combination of complementary patterns rather than mutually exclusive modes. We observe similar pattern distributions across different architectures, LLM models, and compi- lation settings, further supporting their status as stable behavioral characteristics. Pattern Roles. The density measurements reveal distinct oper- ational characteristics that define each pattern’s role in reasoning. Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang Table 4: Distribution of total pattern instances per session. Total Pattern Instances per Session# SessionsPercentage 061.15% 1-10417.87% 11-3010419.96% 31-5013525.91% 51-10018836.08% 101-200407.68% 201+71.34% Search ControlPath ExplorationRecoveryDecision Prioritization 0 5 10 15 20 25 30 Activation Level (instances/session) P1 P2 P3 P4 83.5%, 6.4 97.6%, 18.5 93.8%, 1.9 97.6%, 27.0 Figure 8: Pattern roles mapped along two dimensions: reason- ing control (X-axis) and activation level measured by average instances per session (Y-axis). Figure 8 maps patterns along two dimensions: their role in reason- ing control (X-axis) and activation characteristics (Y-axis), illustrat- ing that patterns serve distinct roles rather than being redundant or arbitrary classifications. P1 (Early Pruning) appears in the region with moderate activa- tion (83.5% prevalence, 6.4 avg/session). Its selective coverage, com- bined with moderate frequency, suggests that it acts as a filtering mechanism—activated when needed to control search expansion but not required in every reasoning context. P2 (Path-dependent Lock-in) occupies the region with high activation (97.6% preva- lence, 18.5 avg/session). Appearing in almost all sessions with high frequency, P2 forms the backbone of reasoning, providing the un- derlying framework that organizes path-dependent exploration throughout the session. P3 (Multi-Path Exploration) is positioned in the region with low activation (93.8% prevalence, 1.9 avg/session). Despite appearing in most sessions, it has a very low frequency and stays within a small range (max=5). This combination—high presence but low usage—indicates P3 acts as a limited recovery mechanism: available when needed to correct reasoning failures, but used sparingly and with constraint, never dominating the rea- soning process. P4 (Knowledge-guided Prioritization) dominates the region with the highest activation (97.6% prevalence, 27.0 avg/ses- sion). With the highest frequency and extreme bursts (max=278), P4 dominates fine-grained decision-making, operating persistently throughout reasoning and showing bursts when multiple prioritiza- tion decisions cluster together. As the underlying decision-making mechanism that guides prioritization across reasoning stages, P4 [0.0-0.1][0.1-0.2][0.2-0.3][0.3-0.4][0.4-0.5][0.5-0.6][0.6-0.7][0.7-0.8][0.8-0.9][0.9-1.0] Session Progress Pattern 1 Pattern 2 Pattern 3 Pattern 4 0.090.090.090.100.120.130.110.120.110.06 0.240.100.090.100.080.080.080.090.080.06 0.010.030.040.040.060.050.060.090.150.47 0.080.090.110.100.100.100.090.100.100.13 0.0 0.1 0.2 0.3 0.4 Fraction of Pattern Instances Figure 9: The heatmap showing the distribution of pattern in- stances across 10 phase bins (P1–P10) of normalized session progress. Each row represents one pattern, and each cell’s color intensity encodes the fraction of pattern instances oc- curring in that phase. Rows are independently normalized to highlight phase bias rather than absolute frequency. provides the knowledge-guided foundation for decisions made in P1–P3. Temporal Coverage of Patterns We analyze when patterns occur during reasoning sessions by normalizing each session’s timeline to [0,1] and partitioning it into 10 phase bins. Figure 9 shows the distribution of pattern instances across these phases, with each row normalized independently to highlight temporal bias. The heatmap shows distinct temporal roles: Pattern 2 (Path Lock- in) is early-biased (24.0% in P1), reflecting initial hypothesis com- mitment. Pattern 1 (Early Pruning) concentrates in the mid-phase (peak at P6, 12.9%), as pruning decisions require accumulated con- textual information. Pattern 3 (Multi-Path Exploration) is late-biased (46.5% in P10), consistent with its reactive role in correcting ear- lier decisions. Pattern 4 (Knowledge-guided Prioritization) shows relatively uniform distribution (9–13% across P3–P10), indicating opportunistic activation throughout the session. This phase-specific distribution shows that patterns serve distinct temporal roles rather than appearing randomly throughout reasoning. Takeaway 1: LLM reasoning patterns are stable, repeatedly used components with distinct roles, not sporadic artifacts or configuration-specific behaviors. 6.2 Pattern Sequential Structure and Inter-pattern Relationships Since patterns appear together within sessions, we examine how they organize into sequential structures through transitions and recurring sequences. Pattern Transitions. We analyze how patterns transition from one to another by collapsing consecutive identical patterns into blocks and examining block-level transitions. For example, P2, P2, P2, P1, P1, P4, P4, P2 becomes P2→P1→P4→P2. Table 11 and Figure 10 show clear transition preferences. P2 and P1 form a bidirectional loop (P2→P1: 40.0%; P1→P2: 39.4%), together ac- counting for 79.4% of all pattern switches, indicating they frequently alternate during reasoning. P3 transitions primarily to P4 (10.1%), Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA P1P2 P3P4 0.394 0.400 0.101 0.101 0.008 Figure 10: Pattern transition graph. TransitionCountProp. P2→ P11,9470.400 P1→ P21,9180.394 P3→ P44920.101 P4→ P24670.096 P4→ P1400.008 P3→ P210.000 Figure 11: Pattern transition frequency details. Table 5: Most frequent subsequences across all reasoning sessions. RankPattern SequenceFrequency 1P2→ P11,947 2P1→ P21,918 3P2→ P1→ P21,878 4P1→ P2→ P11,547 5P2→ P1→ P2→ P11,508 6P1→ P2→ P1→ P21,488 7P3→ P4492 8P4→ P2467 9P3→ P4→ P2451 10P4→ P2→ P1400 suggesting recovery actions often lead to prioritization decisions. P4 transitions to P2 (9.6%) and P1 (0.8%), indicating it frequently initiates commitment or pruning sequences. P3 rarely transitions to P2 (0.02%), consistent with its targeted recovery role. Pattern Subsequences. We identify recurring sequences of length 2-4 extracted from block-level sequences to examine longer- range structure. Table 5 shows two major reasoning macro-structures. First, P2 and P1 form alternating loops: P2→P1 (1,947) and P1→P2 (1,918) are the most frequent length-2 sequences, while extended alternations P2→P1→P2 (1,878) and P1→P2→P1 (1,547) dominate length-3 sequences. Length-4 sequences continue this pattern, with P2→P1→P2→P1 (1,508) and P1→P2→P1→P2 (1,488) being most common. This persistent alternation suggests a core reasoning rou- tine that repeatedly switches between path lock-in (P2) and selective pruning (P1). Second, P3 and P4 form recovery-to-prioritization se- quences: P3→P4 (492) and P3→P4→P2 (451) indicate that recovery actions (P3) frequently lead to prioritization decisions (P4), which then transition to path lock-in (P2). These recurring sequences show that patterns combine into predictable macro-structures rather than appearing randomly. Pattern Coordination. We examine whether patterns show systematic relationships in their usage intensity across sessions. We normalize pattern counts per session and analyze correlations between normalized counts. Figure 12 shows the joint distribution of normalized pattern counts: P2 vs P4. P2 and P4 show a strong negative correlation (r = -0.845): when P2 is above its median nor- malized count, P4 averages 0.377; when P2 is below its median, P4 averages 0.566. This indicates that path lock-in (P2) and deci- sion prioritization (P4) function as complementary mechanisms. 0.00.20.40.60.81.0 P2 (normalized count) 0.0 0.2 0.4 0.6 0.8 1.0 P4 (normalized count) r = -0.845 Figure 12: Joint distribution of normalized pattern counts: P2 vs P4. Each point represents a session. The strong negative correlation (r = -0.845) indicates complementary resource allocation: high P2 corresponds to low P4, and vice versa. As the underlying decision-making mechanism, P4’s prioritization decisions inform when paths are committed to (P2), pruned (P1), or reconsidered (P3), creating systematic relationships in pattern usage. P1 shows moderate negative correlations with other patterns (r = -0.216 to -0.246), suggesting it acts as a selective mechanism. P3 shows weak correlations (r = -0.075 to -0.126), consistent with its targeted recovery role that operates independently of the main reasoning flow. Together, these analyses show that patterns orga- nize into structured sequences with clear transition preferences and recurring macro-structures, rather than appearing randomly or independently during reasoning. Takeaway 2: Reasoning patterns operate as a coordinated system with predictable structural relationships, rather than independent or interchangeable behaviors. 6.3 Pattern Behavioral Characteristics We quantify how different reasoning patterns show concrete action structures by measuring behavioral metrics over pattern-aligned trace segments. Importantly, these metrics are computed indepen- dently of the pattern extraction rules, providing an external valida- tion of pattern distinctiveness. For each pattern instance, metrics are computed over the corresponding contiguous trace segment (see Appendix E for computation details). Pattern-conditioned Action Metrics. We measure four met- rics to quantify behavioral differences across patterns: path length (퐿), branching factor (퐵), forward step ratio (퐹), and backtrack count (푅). Table 6 reports mean, median, and standard deviation for each metric across all instances of each pattern. Early Pruning (P1) shows the longest and most variable paths (mean퐿=19.9), with low branching (median퐵=1.0), high for- ward commitment (median퐹=0.947), and minimal backtracking. This indicates sustained linear progression with early elimination of candidate paths. Path Lock-in (P2) is characterized by short, sta- ble paths (median퐿=7.0), the strongest forward commitment (median퐹=1.0), and almost no backtracking. These properties Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang Table 6: Metrics statistics: mean, median, and standard devia- tion are reported for each metric across all pattern instances. PatternMetricMean Median Std P1 (3,339) Path Length (퐿)19.99.031.5 Branching Factor (퐵)1.0351.0000.141 Forward Step Ratio (퐹 )0.8690.9470.226 Backtrack Count (푅)0.1750.0000.534 P2 (9,654) Path Length (퐿)8.97.06.3 Branching Factor (퐵)1.0511.0000.197 Forward Step Ratio (퐹 )0.9441.0000.097 Backtrack Count (푅)0.0390.0000.206 P3 (995) Path Length (퐿)5.33.06.3 Branching Factor (퐵)1.0401.0000.187 Forward Step Ratio (퐹 )0.8601.0000.251 Backtrack Count (푅)1.0001.0000.000 P4 (14,083) Path Length (퐿)6.06.00.6 Branching Factor (퐵)1.0631.0000.211 Forward Step Ratio (퐹 )0.8780.8570.147 Backtrack Count (푅)0.0600.0000.250 Table 7: Exploration vs. Path Selection metrics. Branching Factor (퐵) measures exploration breadth, while Pruning Rate (푃) measures path elimination aggressiveness. PatternExploration Path Selection 퐵: Mean 푃: MeanLevelLevel P11.0350.118LowHigh P21.0510.043MediumLow P31.0400.064MediumMedium P41.0630.078HighMedium reflect deterministic, path-dependent progression where early de- cisions constrain subsequent analysis. Multi-Path Exploration (P3) shows short paths (median퐿=3.0) with consistent backtracking (median푅=1.0), confirming its targeted recovery role. Knowledge- guided Prioritization (P4) shows highly stable path lengths (mean and median퐿=6.0), moderate forward progression, and minimal backtracking. Compared to P2, P4 shows slightly higher branching and lower forward ratios, suggesting flexible prioritization guided by analogical cues. Exploration vs. Path Selection. We analyze how different patterns balance exploration breadth and path selection certainty using branching factor (퐵) and pruning rate (푃). Branching factor measures how many options are considered at decision points, while pruning rate measures how aggressively candidate paths are eliminated. Together, these metrics show how patterns trade off between exploring multiple possibilities versus narrowing down to specific paths. Table 7 shows mean and median values of퐵and푃for each pattern. P1 shows the lowest branching factor (퐵=1.035) and the highest pruning rate (푃=0.118), indicating limited exploration with aggressive path elimination. This reflects early narrowing of the search space. P2 shows a slightly higher branching factor but the lowest pruning rate (푃=0.043). Rather than actively eliminat- ing paths, this pattern reflects a state where the reasoning path has already stabilized, reducing the need for further pruning. Explo- ration is moderate, but path selection emerges implicitly through persistence rather than explicit elimination. P3 shows intermediate characteristics. Both branching factor and pruning rate remain mod- erate, reflecting a balance between maintaining multiple candidate paths and selectively discarding them. This preserves flexibility while enabling controlled rollback when necessary. P4 shows the highest branching factor (퐵=1.063) with a moderate pruning rate. This suggests knowledge-guided decisions consider a broader set of alternatives before selection, leading to higher exploratory breadth while still narrowing down to specific paths. Takeaway 3: Patterns show distinct exploration–path selection characteristics: P1 emphasizes rapid path narrowing, P2 reflects implicit selection via persistence, P3 maintains a balanced state, and P4 favors broader exploration before selection. Pattern-conditioned Tool Usage Topology. Beyond path- level behavior, reasoning patterns also appear in the structure of tool usage. Different patterns correspond to distinct command us- age patterns, reflecting how the LLM organizes its interaction with analysis tools during reasoning. We analyze tool usage topology by measuring six metrics over pattern-aligned trace segments: com- mand diversity (number of distinct command types used), sequence length (total tool invocations), max depth (maximum consecutive repetition of the same command), max fan-out (maximum number of different commands following a given command), cycle presence (whether command transitions form cycles), and transition entropy (randomness of command transitions). To illustrate the internal structure of a reasoning segment, Appendix B presents an anno- tated 26-step tool interaction extracted from a longer session (187 steps) that ultimately identified a command injection vulnerability in a binary file. Table 8 reports these metrics for each pattern. Early Pruning (P1) shows the highest command diversity (5.96) and longest sequences (17.48), consistent with broad exploration requiring multiple tool types. Path Lock-in (P2) shows the highest cycle rate (92.5%), indi- cating repetitive command patterns that reflect stable, committed reasoning trajectories. Multi-Path Exploration (P3) has the lowest diversity (2.68) and lowest transition entropy (0.23), showing deter- ministic tool usage during targeted recovery. Knowledge-guided Prioritization (P4) shows moderate diversity (3.29) with a high cycle rate (82.6%), suggesting structured tool usage patterns that support semantic comparisons. These results show that reasoning patterns correspond to distinct tool usage graph structures. High diversity and long sequences in P1 reflect exploration-phase tool usage; high cycle rates in P2 and P4 indicate repetitive command patterns supporting committed or structured reasoning; low diver- sity and entropy in P3 show deterministic tool selection during recovery. Together, these topological differences provide additional evidence that patterns capture systematic organizational principles in LLM reasoning. Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Table 8: Tool usage topology metrics by pattern type. Diversity Length Depth Fan-out Cycles Entropy Mean Mean MeanMean(%)Mean P15.9617.482.772.6570.90.60 P24.498.172.472.1092.50.51 P32.684.711.761.2251.40.23 P43.294.992.071.5982.60.35 Takeaway 4: Reasoning patterns show distinct tool usage struc- tures, indicating systematic differences in how LLMs organize tool interactions. 7 Discussion and Implications Implicit Control from Token-Level Reasoning. A central find- ing is that exploration control in LLM-based binary analysis is not explicitly programmed, but emerges implicitly from token-level se- quential reasoning. Unlike traditional systems where prioritization, pruning, and backtracking are implemented as explicit data struc- tures and algorithms, LLM agents exhibit these behaviors without maintaining any explicit representation of the search state. The four patterns demonstrate that key control functions—path selection, commitment, revision, and prioritization—can arise directly from the token generation process. Early pruning emerges because LLMs cannot maintain all candidate paths simultaneously; path lock-in occurs because token-level reasoning creates strong contextual dependencies; backtracking reflects recovery from path failures; prioritization leverages pre-trained semantic knowledge to guide decisions. This shifts the perspective of agent design: instead of specifying control logic, systems may rely on and shape the model’s implicit reasoning dynamics. Semantic-Guided vs. Structural Exploration. LLM-driven exploration differs fundamentally from prior methods. Depth-first and breadth-first search require explicit graph structures with de- terministic traversal; taint analysis operates with precise data-flow tracking. In contrast, LLM traversal is semantic-guided: decisions are driven by learned semantic associations rather than structural reachability. When encountering a function call or branch, the LLM interprets semantic meaning and uses pre-trained knowledge to infer relevance to vulnerability discovery, enabling prioritization of semantically promising paths over structurally similar but semanti- cally irrelevant ones. From One-Pass Pipelines to Iterative Analysis. Our results highlight a fundamental shift in how static analysis participates in vulnerability discovery. Traditional approaches follow a one-pass paradigm, where analysis constructs a global representation and reasoning operates over this fixed structure. LLM-based systems operate iteratively: analysis and reasoning are interleaved across hundreds of steps, with decisions continuously revised based on partial observations. The identified patterns reveal how this iter- ative process is structured—early pruning regulates search space growth, path lock-in sustains deep analysis, backtracking enables recovery, and prioritization guides exploration under uncertainty. Patterns as Structural Components, Not Heuristics. The four patterns should not be interpreted as task-specific heuristics or incidental behaviors. Their stability across binaries, distinct tem- poral roles, and structured transition relationships (e.g., the bidirec- tional loop between lock-in and pruning) indicate that they function as fundamental organizing principles for long-horizon reasoning. This perspective elevates patterns from descriptive observations to structural components that enable tractable exploration under bounded reasoning capacity. 8 Threats to Validity Task, System, and Model Dependence. Our analysis is conducted on reasoning traces generated during vulnerability-oriented binary analysis tasks, under a fixed system design and prompting pro- tocol with a specific LLM. Task objectives (e.g., vulnerability dis- covery), system orchestration (e.g., iterative analysis), and model choice may influence the frequency or dominance of certain behav- iors. However, the identified patterns are derived from observable decision-level transitions in long-horizon traces, rather than from task-specific heuristics or architecture-dependent features. We ex- pect these behaviors to generalize to other security analysis tasks that require sequential evidence accumulation and adaptive explo- ration. Behavioral Interpretation and Scope. The identified patterns are inferred from externally observable reasoning traces, without claims about internal model states or cognitive mechanisms. Our analysis adopts a pattern-based perspective, focusing on the token- level implicit patterns that govern commitments, revisions, and path-selection decisions that directly affect analysis outcomes. This work prioritizes systematic pattern identification over exhaustive quantification; while grounded in concrete trace evidence, we do not claim complete coverage of all possible behaviors or configurations. Instead, the goal is to establish a trace-level foundation that enables future benchmarking, measurement, and controlled comparison across models and tools. 9 Related Work A. Symbolic Execution and Static Analysis. Symbolic execution and static analysis techniques address challenges such as path ex- plosion, state pruning, and scalability [8,9,21,25,26,36,38]. These approaches typically operate within a fixed analysis pipeline, where exploration strategies are explicitly defined. Prior work focuses on improving efficiency, coverage, and scalability of these mechanisms. In contrast, our work studies how token-level exploration behav- iors arise without explicit implementation, emerging instead from token-level reasoning in LLM-driven analysis. B. Automated Vulnerability Discovery. Automated vulner- ability discovery systems, including fuzzing and learning-based approaches, explore large search spaces using input mutation, sta- tistical sampling, or heuristic guidance [7,10,16,17,22,24,32,39]. While they incorporate implicit or explicit prioritization strategies, they do not expose or analyze the step-by-step decision processes that govern exploration. Our work instead focuses on character- izing the structure of exploration behavior itself, independent of specific optimization objectives. C. LLM-based Program Analysis and Agentic Systems. LLMs have been applied to program analysis and binary security tasks, including vulnerability detection, code reasoning, and tool-assisted Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang exploration [1,5,6,12–14,18,20,27]. Recent agentic systems fur- ther extend LLM capabilities to multi-step workflows, such as code repair and issue resolution, through tool use and iterative interac- tion [4,15,19,31,33,37]. These systems demonstrate the effective- ness of LLMs in long-horizon tasks, but primarily focus on task performance, prompting strategies, or system design. The internal reasoning process—particularly how exploration decisions evolve over long traces—remains unexamined. In contrast, our work pro- vides a trace-level analysis of LLM-driven exploration, treating long-horizon reasoning behavior as the primary object of study. 10 Conclusion This paper presents the first trace-level study of long-horizon LLM- driven binary analysis, showing that multi-pass reasoning gives rise to structured, token-level implicit patterns that govern explo- ration behavior. Across 521 binaries and 99,563 reasoning steps, we identify four dominant patterns—early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritiza- tion—that emerge directly from token-level sequential reasoning rather than explicit program logic. Our findings suggest a shift in perspective: in LLM-based sys- tems, exploration control is not implemented as explicit algorithms, but arises as an emergent property of token-level reasoning. The identified patterns constitute a minimal set of structural mecha- nisms that organize long-horizon analysis under bounded context and uncertainty, providing an abstraction of how LLM agents regu- late exploration over time. This perspective opens new directions for both system design and analysis. Future work may build con- trollable and reliable LLM-based analysis systems by explicitly supporting these implicit mechanisms, as well as extend trace-level methodologies to study long-horizon reasoning in other domains beyond binary security. References [1] Talor Abramovich, Meet Udeshi, Minghao Shao, Kilian Lieret, Haoran Xi, Kim- berly Milner, Sofija Jancheska, John Yang, Carlos E Jimenez, Farshad Khorrami, Prashanth Krishnamurthy, Brendan Dolan-Gavitt, Muhammad Shafique, Karthik Narasimhan, Ramesh Karri, and Ofir Press. 2025. EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities. In Proceedings of the 42nd International Conference on Machine Learning (ICML). [2] Libo Chen, Yanhao Wang, Quanpu Cai, Yunfan Zhan, Hong Hu, Jiaqi Linghu, Qinsheng Hou, Chao Zhang, Haixin Duan, and Zhi Xue. 2021. Sharing more and checking less: Leveraging common input keywords to detect bugs in embedded systems. In 30th USENIX Security Symposium (USENIX Security 21). 303–319. [3]Xiang Chen, Chengfeng Ye, Anshunkang Zhou, and Charles Zhang. 2025. ClearA- gent: Agentic Binary Analysis for Effective Vulnerability Detection. In Proceedings of the 1st ACM SIGPLAN International Workshop on Language Models and Program- ming Languages (LMPL 2025), co-located with ICFP/SPLASH 2025. ACM, Singapore, 130–137. doi:10.1145/3759425.3763397 [4]Zhaoling Chen, Robert Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor Prasanna, Arman Cohan, and Xingyao Wang. 2025. LocAgent: Graph- Guided LLM Agents for Code Localization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), Vol. 1. Vienna, Austria, 8697–8727. [5]Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis. 423–435. [6]Sidong Feng and Chunyang Chen. 2023. Prompting Is All Your Need: Automated Android Bug Replay with Large Language Models. arXiv preprint arXiv:2306.01987 (2023). [7]Xiaotao Feng, Ruoxi Sun, Xiaogang Zhu, Minhui Xue, Sheng Wen, Dongxi Liu, Surya Nepal, and Yang Xiang. 2021. Snipuzz: Black-box fuzzing of iot firmware via message snippet inference. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security. 337–350. [8]Zicong Gao, Chao Zhang, Hangtian Liu, Wenhou Sun, Zhizhuo Tang, Liehui Jiang, Jianjun Chen, and Yong Xie. 2024. Faster and better: Detecting vulnerabili- ties in linux-based iot firmware with optimized reaching definition analysis. In Proceedings of the 2024 Network and Distributed System Security Symposium, San Diego, CA, USA, Vol. 26. [9]Wil Gibbs, Arvind S Raj, Jayakrishna Menon Vadayath, Hui Jun Tay, Justin Miller, Akshay Ajayan, Zion Leonahenahe Basque, Audrey Dutcher, Fangzhou Dong, Xavier Maso, et al.2024. Operation mango: Scalable discovery ofTaint-Style vulnerabilities in binary firmware services. In 33rd USENIX Security Symposium (USENIX Security 24). 7123–7139. [10]Ivan Gotovchits, Rijnard Van Tonder, and David Brumley. 2018. Saluki: finding taint-style vulnerabilities with static property checking. In Proceedings of the NDSS Workshop on Binary Analysis Research, Vol. 2018. [11]Nasir Hussain, Haohan Chen, Chanh Tran, Philip Huang, Zhuohao Li, Pravir Chugh, William Chen, Ashish Kundu, and Yuan Tian. 2025. VulBinLLM: LLM- powered Vulnerability Detection for Stripped Binaries. arXiv:2505.22010 [cs.SE] https://arxiv.org/abs/2505.22010 [12] Jiangan Ji, Chao Zhang, Shuitao Gan, Lin Jian, Hangtian Liu, Tieming Liu, Lei Zheng, and Zhipeng Jia. [n. d.]. FirmAgent: Leveraging Fuzzing to Assist LLM Agents with IoT Firmware Vulnerability Discovery. ([n. d.]). [13]Caroline Lemieux, Jeevana Priya Inala, Shuvendu K Lahiri, and Siddhartha Sen. 2023. Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models. In 2023 IEEE/ACM 45th International Conference on Soft- ware Engineering (ICSE). IEEE, 919–931. [14]Haonan Li, Yu Hao, Yizhuo Zhai, and Zhiyun Qian. 2023. The Hitchhiker’s Guide to Program Analysis: A Journey with Large Language Models. arXiv preprint arXiv:2308.00245 (2023). [15] Weizhen Li, Jianbo Lin, Zhuosong Jiang, Jingyi Cao, Xinpeng Liu, Jiayu Zhang, Zhenqiang Huang, Qianben Chen, Weichen Sun, Qiexiang Wang, Hongxuan Lu, Tianrui Qin, Chenghao Zhu, Yi Yao, Shuying Fan, Xiaowan Li, Tiannan Wang, Pai Liu, King Zhu, He Zhu, Dingfeng Shi, Piaohong Wang, Yeyi Guan, Xiangru Tang, Minghao Liu, Yuchen Eleanor Jiang, Jian Yang, Jiaheng Liu, Ge Zhang, and Wangchunshu Zhou. 2025. Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RL. arXiv preprint arXiv:2508.13167 (2025). [16]Wenqiang Li, Jiameng Shi, Fengjun Li, Jingqiang Lin, Wei Wang, and Le Guan. 2022.휇AFL: non-intrusive feedback-driven fuzzing for microcontroller firmware. In Proceedings of the 44th International Conference on Software Engineering. 1–12. [17]Fengyu Liu, Yuan Zhang, Tian Chen, Youkun Shi, Guangliang Yang, Zihan Lin, Min Yang, Junyao He, and Qi Li. 2025. Detecting Taint-Style Vulnerabilities in Microservice-Structured Web Applications. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 972–990. [18]Puzhuo Liu, Chengnian Sun, Yaowen Zheng, Xuan Feng, Chuan Qin, Yuncheng Wang, Zhenyang Xu, Zhi Li, Peng Di, Yu Jiang, et al.2025. Llm-powered static binary taint analysis. ACM Transactions on Software Engineering and Methodology 34, 3 (2025), 1–36. [19]Samuel Miserendino, Michele Wang, Tejal Patwardhan, and Johannes Heidecke. 2025. SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? arXiv preprint arXiv:2502.12115 (2025). [20]Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2023. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2339–2356. [21]Sebastian Poeplau and Aurélien Francillon. 2020. Symbolic execution with SymCC: Don’t interpret, compile!. In 29th USENIX Security Symposium (USENIX Security 20). 181–198. [22] Abdullah Qasem, Mourad Debbabi, and Andrei Soeanu. 2024. OctopusTaint: Ad- vanced Data Flow Analysis for Detecting Taint-Based Vulnerabilities in IoT/IIoT Firmware. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 2355–2369. [23]Nilo Redini, Aravind Machiry, Ruoyu Wang, Chad Spensky, Andrea Continella, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2020. Karonte: Detecting insecure multi-binary interactions in embedded firmware. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 1544–1561. [24]Tobias Scharnowski, Nils Bars, Moritz Schloegel, Eric Gustafson, Marius Muench, Giovanni Vigna, Christopher Kruegel, Thorsten Holz, and Ali Abbasi. 2022. Fuz- zware: Using preciseMMIOmodeling for effective firmware fuzzing. In 31st USENIX Security Symposium (USENIX Security 22). 1239–1256. [25]Nick Stephens, John Grosen, Christopher Salls, Andrew Dutcher, Ruoyu Wang, Jacopo Corbetta, Yan Shoshitaishvili, Christopher Kruegel, and Giovanni Vigna. 2016. Driller: Augmenting fuzzing through selective symbolic execution.. In NDSS, Vol. 16. 1–16. [26]Jayakrishna Vadayath, Moritz Eckert, Kyle Zeng, Nicolaas Weideman, Gokulkr- ishna Praveen Menon, Yanick Fratantonio, Davide Balzarotti, Adam Doupé, Tiffany Bao, Ruoyu Wang, et al.2022. Arbiter: Bridging the static and dynamic divide in vulnerability discovery on binary programs. In 31st USENIX Security Symposium (USENIX Security 22). 413–430. [27]Chengpeng Wang, Wuqi Zhang, Zian Su, Xiangzhe Xu, and Xiangyu Zhang. 2024. Sanitizing large language models in bug detection with data-flow. In Findings of Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA the Association for Computational Linguistics: EMNLP 2024. 3790–3805. [28]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023). [29]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837. [30] Yuhao Wu, Jinwen Wang, Yujie Wang, Shixuan Zhai, Zihan Li, Yi He, Kun Sun, Qi Li, and Ning Zhang. 2024. Your firmware has arrived: A study of firmware update vulnerabilities. In 33rd USENIX Security Symposium (USENIX Security 24). 5627–5644. [31] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489 (2024). [32]Haoyu Xiao, Ziqi Wei, Jiarun Dai, Bowen Li, Yuan Zhang, and Min Yang. 2025. HouseFuzz: Service-Aware Grey-Box Fuzzing for Vulnerability Detection in Linux-Based Firmware. In 2025 IEEE Symposium on Security and Privacy (SP). IEEE, 3801–3819. [33]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-Computer In- terfaces Enable Automated Software Engineering. arXiv:2405.15793 [cs.SE] https://arxiv.org/abs/2405.15793 [34]Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 (2023). [35]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022). [36] Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018.QSYM: A practical concolic execution engine tailored for hybrid fuzzing. In 27th USENIX Security Symposium (USENIX Security 18). 745–761. [37]Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. Au- tocoderover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1592–1604. [38]Jiaxu Zhao, Yuekang Li, Yanyan Zou, Zhaohui Liang, Yang Xiao, Yeting Li, Bingwei Peng, Nanyu Zhong, Xinyi Wang, Wei Wang, et al.2024. Leveraging semantic relations in code and data to enhance taint analysis of embedded systems. In 33rd USENIX Security Symposium (USENIX Security 24). 7067–7084. [39]Yaowen Zheng, Ali Davanian, Heng Yin, Chengyu Song, Hongsong Zhu, and Limin Sun. 2019.FIRM-AFL:High-Throughputgreybox fuzzing ofIoT firmware via augmented process emulation. In 28th USENIX Security Symposium (USENIX Security 19). 1099–1114. A Session Generation Details A.1 Initialization Prompt All sessions in our dataset use the same initialization prompt, which establishes the task and analysis requirements. The prompt serves solely as a task specification and does not encode system-specific constraints or implementation details. The unified initialization prompt used across all sessions: 1 Please analyze the binary file comprehensively based on user 2 core requirements. Current file being analyzed is: [binary_name]. 3 User core requirements are: Comprehensively analyze the 4 specified binary file, with the core goal of identifying 5 feasible exploitation chains from untrusted input sources 6 (such as network, environment variables, IPC, etc.) to 7 final dangerous operations. Always focus on precise analysis 8 of current specific tasks, avoiding false negatives and 9 false positives. 10 11 ### Requirements: 12 - Follow the function call protocol strictly 13 - Provide detailed analysis for each finding 14 - Use precise addresses and instruction locations This prompt defines the analysis objective (vulnerability discovery through source-to-sink path identification) but does not prescribe specific reasoning strategies or exploration patterns; these emerge from the LLM’s interaction with the binary under the given task constraints. A.2 Complete Session Format Example Each session is stored as a JSONL file, where each line represents a single record in the trace. The session is a sequence of reasoning- action-observation cycles. Below shows two representative record types: Assistant record (thought and action): This record type captures the agent’s internal reasoning state (thought) and selected tool action (action,command,status). Thethoughtfield contains the reasoning process, whileactionandcommandspecify what tool action to take. 1 2 "role": "assistant", 3 "thought ": "I've seen the main function. It opens /dev/ac0 , uses getopt to parse command line arguments. The taint sources are command line arguments (argv). The program seems to be a utility to set/get AC entries via ioctl to a device. There are functions sym.SetAcEntry and sym.GetAcEntry that likely perform ioctl calls. I need to analyze those functions to see if there are any vulnerabilities like buffer overflows due to strcpy or other unsafe operations .", 4 "action ": "r2", 5 "command ": "pdf @sym.SetAcEntry", 6 "status ": "continue" 7 Tool record (observation): This record type captures the tool ex- ecution result (type,tool,command,result). Theresultfield contains the observation returned by the tool, which is the partial program information that the agent uses to update its reasoning state. 1 2 "role": "tool", 3 "type": "tool_result", 4 "tool": "r2", 5 "command ": "pdf @sym.SetAcEntry", 6 "result ": " ; CALL XREFS from main @ 0x4013c(r) [320: sym.SetAcEntry (int32_t arg1 , int32_t arg2 , ...); | 0x00401540 d8ffbd27 addiu sp , sp , -0x28 | 0x00401544 2400 bfaf sw ra , (var_24h) | 0x004015e0 8080828f lw v0 , -sym.imp.ioctl(gp) | 0 x004015ec 09 f82003 jalr t9 | 0 x0040167c 00000000 nop" 7 Key fields: • role: Identifies the record type (LLM context reset, assistant, tool) • thought: Reasoning state inferred from assistant records • action: Type of action taken (tool_invocation, etc.) • command: Specific tool command issued Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang Table 9: Annotated radare2 command trace (26 steps). Each command corresponds to a localized analysis operation that reveals a specific program property. Step CommandImmediate PurposeContext Gained / Used 0 iInitialize binary metadataArchitecture, bitness, entry points 1 aflRecover function listFunction boundaries and symbols 2 is~mainLocate main-related symbolsCandidate entry function 3 pdf @ entry0Inspect entry functionControl-flow of program start 4 pdf @ fcn.0000db2cExpand to adjacent functionCallee function structure 5 pdf @ fcn.0000b208Continue function-space explorationAlternative execution path 6 pdf @ fcn.0000ac9cScan related functionBroader function context 7 izEnumerate stringsCandidate semantic anchors 8 axt @@ sym.imp.doSystemCmdResolve cross-referencesCall sites invoking system command 9 pd 20 @ 0xdf54Inspect instructions near call siteLocal instruction semantics 10 pd -30 @ 0xdf54Backward inspectionPreceding control dependencies 11 axt @@ reloc.wanidResolve additional data referencesAlternative semantic anchor 12 pdf @ fcn.0000b514Jump to referenced functionNew function-level context 13 is~doSystemCmdValidate symbol relevanceConfirmation of target semantics 14 axt @@ reloc.ifnameExplore related referencesConfiguration-dependent behavior 15 pdg @ fcn.0000b208High-level decompilationAbstract control/data flow 16 pd 10 @ 0xb428Instruction-level refinementConcrete execution details 17 /a mov r0, 0xcPattern searchPotential argument setup 18 pd 5 @ 0xb458Verify matched locationLocal confirmation 19 pd -10 @ 0xb458Context expansionBackward dependency check 20 pd 10 @ 0xe198Inspect another siteComparative behavior 21 pd -20 @ 0xe198BacktrackingControl-flow consistency 22 pd -10 @ 0xe27cFurther backward tracingExecution preconditions 23 axt @@ sym.imp.GetIniFileValueTrace configuration usageExternal input dependency 24 pd 20 @ 0x9d58Final local inspectionConfirmation of hypothesis 25 finishTerminate session— • result: Tool output or observation returned • status: Execution status (executed, failed, etc.) This format enables reconstruction of the complete reasoning- action-observation cycle for each session. The initialization prompt in the first LLM context reset record establishes the session con- text, while subsequent context reset records maintain continuity by preserving accumulated analysis state. This structure supports pat- tern detection and behavioral analysis while enabling long-horizon session continuity. B Annotated Tool-Interaction Trace B.1 Trace Generation Achieving long-horizon analysis sessions requires maintaining rea- soning continuity across extended interactions. Because LLMs op- erate under finite context window constraints, we employ periodic context reset operations during a session. A context reset instanti- ates a new LLM instance with a fresh context window. The initial context includes the task description, available tools, and analysis objectives. Subsequent resets include a compact summary of ac- cumulated analysis state, including key findings and the current exploration focus. These summaries are generated automatically and do not introduce new information beyond what is already present in the trace. Context resets serve only to preserve session continuity and do not alter the agent’s action space or decision logic. All reasoning steps, tool actions, and observations remain explicitly recorded in the trace. Our behavioral analysis operates solely on these observable elements and is agnostic to the reset boundaries. As mentioned, we analyze four LLMs (DeepSeek-V3, GPT-5, Claude 3.5 Sonnet, Gemini 3.0), generating four independent ses- sions per binary. Session selection prioritizes trace completeness and quality. We discard incomplete or failed sessions, including runs that terminate prematurely due to tool errors or fail to progress beyond initial reconnaissance. If exactly one model produces a com- plete trace (reaching a minimum length threshold of 130 steps and terminating normally), we select that session. If multiple models produce complete traces, we select the longest trace. If all traces are incomplete or of similar length, we select the trace with the median length. We do not filter traces based on vulnerability out- comes, coverage, or success, as our analysis focuses on behavioral structure rather than effectiveness alone. B.2 Trace Example This illustrative trace corresponds to a single analysis session con- sisting of 26 radare2 commands, as shown in Table 9. The trace contains 10 distinct command types, with instruction-level dis- assembly commands (pd) dominating the interaction (9/26), fol- lowed by function-level disassembly (pdf, 5/26) and cross-reference queries (axt, 4/26). The trace exhibits multiple abstraction shifts between program-level, function-level, and instruction-level anal- ysis, as well as repeated bidirectional inspection around the same addresses. Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Table 10: Vulnerability findings distribution across analysis sessions. Vulnerabilities per Session Count Percentage 1 vulnerability12563.1% 2 vulnerabilities4522.7% 3 vulnerabilities84.04% 4 vulnerabilities94.5% 5 vulnerabilities52.5% 6 vulnerabilities10.5% 7 vulnerabilities10.5% 8 vulnerabilities10.5% 10 vulnerabilities10.5% 11 vulnerabilities10.5% 15 vulnerabilities10.5% Total sessions198100.0% Trace Logic Summary. This trace follows a repeated zoom- in/zoom-out reasoning structure. The agent first establishes a global structural view (Steps 0–3), then explores the function space (Steps 4–6), introduces semantic anchors via strings and cross-references (Steps 7–8), and iteratively validates hypotheses through localized bidirectional disassembly (Steps 9–11, 16–22). High-level abstrac- tion is intermittently reintroduced via decompilation (Step 15) to realign instruction-level findings with function-level semantics. This interleaving of abstraction levels and repeated backward in- spection exemplifies non-linear reasoning within a single tool. We release full tool interaction traces (130–300 steps per session), with binary identifiers and confirmed vulnerability labels, to support reproducibility and independent inspection. C Vulnerability Findings Statistics This appendix provides detailed statistics on the vulnerability find- ings identified across all analysis sessions in our dataset. All trace analyses operate exclusively on the structure and temporal orga- nization of reasoning-action-observation sequences. Vulnerability characteristics are reported here solely to contextualize the corpus and to demonstrate that observed token-level implicit patterns arise under realistic security analysis conditions. Our dataset contains 521 analysis sessions across 521 distinct binaries. Among these sessions, 198 have vulnerability analysis records, resulting in a total of 306 distinct vulnerability instances across 198 sessions that reported at least one vulnerability. Table 10 summarizes the distribution of vulnerability counts per session among the 198 sessions with vulnerability records. The 125 ses- sions (63.1%) reported exactly one vulnerability. A notable subset of sessions (45, 22.7%) reported multiple vulnerabilities, with 28 sessions reporting three or more vulnerabilities. The maximum number of vulnerabilities found in a single session is 15, all of which are CWE-121 stack-based buffer overflows. Table 11 reports the distribution of Common Weakness Enumer- ation (CWE) types across all 306 vulnerability findings. The most prevalent vulnerability type is CWE-78 (OS Command Injection), accounting for 115 findings (37.6%). This is followed by CWE-120 (Classic Buffer Overflow) with 52 findings (16.7%) and CWE-121 (Stack-based Buffer Overflow) with 37 findings (12.2%). Together, Table 11: CWE type distribution across all vulnerability find- ings (Top 10). Rank CWE TypeCount Percentage 1CWE-78 (OS Command Injection)11537.6% 2 CWE-120 (Classic Buffer Overflow)5216.7% 3CWE-121 (Stack-based Buffer Over- flow) 3712.2% 4CWE-134 (Format String)237.2% 5 CWE-20 (Improper Input Valida- tion) 165.3% 6CWE-22 (Path Traversal)123.8% 7CWE-73 (External Control of File Name or Path) 72.3% 8 CWE-676 (Use of Potentially Dan- gerous Function) 31.1% 9CWE-unknown20.8% 10CWE-123 (Write-what-where Con- dition) 20.8% Top 1026987.9% Other3712.1% Total306100.0% these top three CWE types account for 66.5% of all identified vulner- abilities, reflecting common security issues in embedded firmware binaries. D Pattern Detection Implementation This section documents the concrete implementation details used to operationalize the four reasoning patterns studied in this pa- per. To avoid capturing short-lived or incidental behaviors, pattern detection requires both frequency and duration constraints. Specifi- cally, a candidate pattern must persist across multiple non-adjacent steps and span a minimum number of reasoning turns before being recorded. Once a pattern instance is detected, the corresponding path is retired to prevent duplicate counting. Thresholds are fixed a priori and applied uniformly across all sessions. D.1 Pattern 1 Detection Pattern 1 detection operates over thethoughtfield of each rea- soning step, as this field captures the agent’s internal reasoning process where path selection and pruning decisions are articulated. The detector implements a state-tracking algorithm that processes the trace sequentially. Algorithm Overview. The algorithm 1 maintains a list of pend- ing pattern instances throughout trace processing. For each step, it checks for revisit signals and removes pending instances whose se- mantic content matches the revisit keywords (lines 6–7), enforcing condition (3) of the operational definition. It then detects multi- path discussion signals and creates new pending instances (lines 8–9). Next, it checks for pruning decisions and associates them with the most recent pending instance that has not yet been pruned (lines 10–15), ensuring temporal ordering between multi-path ex- ploration and pruning. After processing all steps, instances are recorded only if both start and prune indices are present and the span after pruning exceeds the threshold of 20 steps (lines 17–19). Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang Algorithm 1: Pattern 1 Detection Algorithm Input: Sequence of reasoning steps푇=⟨푠 1 ,푠 2 , . . .,푠 푛 ⟩ Output: List of Pattern 1 instances findings 1 pending←∅; findings←∅; 2 for each step 푠 푖 ∈ 푇 do 3text← 푠 푖 .thought; 4 if text contains revisit signal then 5keywords← ExtractSemanticEntities(text); 6pending←푝 ∈ pending : no keyword matches 푝.evidence; 7 if text matches multi-path pattern then 8푝 ←start_idx : 푖, evidence_multi : text[: 500], prune_idx : None; 9pending.append(푝); 10 for each 푝 ∈ pending in reverse order do 11if 푝.prune_idx= None and text matches prune pattern then 12푝.prune_idx← 푖; 13푝.evidence_prune← text[: 500]; 14break; 15 for each 푝 ∈ pending do 16if푝.prune_idx≠ Noneand(푛− 푝.prune_idx) ≥20 then 17findings.append(start_step : 푝.start_idx, prune_step : 푝.prune_idx, span_after_prune : 푛− 푝.prune_idx); 18 return findings The key algorithmic components are: (1) State maintenance— the pending list tracks multiple concurrent pattern candidates; (2) Revisit exclusion—semantic entity extraction enables precise filter- ing when agents explicitly return to previously abandoned paths; (3) Temporal ordering enforcement—pruning decisions are associ- ated with the most recent unpruned instance; and (4) Threshold filtering—the 20-step minimum span ensures detected instances reflect sustained behavioral focus rather than transient narrowing behaviors. D.2 Pattern 2 Detection Pattern 2 detection operates over thethoughtfield of each reason- ing step, tracking path selection decisions and their subsequent influence on analysis behavior. The detector implements a state- tracking algorithm that maintains active path candidates and moni- tors continuation, alternative mentions, and contradiction absorp- tion patterns. Algorithm Overview. The algorithm 2 maintains a list of ac- tive path selections throughout trace processing. For each step, it checks for global re-evaluation signals and removes active paths whose semantic content matches the re-evaluation keywords (lines 6–7), excluding episodes that exhibit deliberate strategy revision. It then detects path-selection decisions and creates new active path instances (lines 7–8). For each active path, the algorithm tracks con- tinuation through semantic entity overlap (lines 9–10), monitors alternative mentions within the same context (line 11), and de- tects contradiction absorption patterns (lines 12–13). Instances are Algorithm 2: Pattern 2 Detection Algorithm Input: Sequence of reasoning steps푇=⟨푠 1 ,푠 2 , . . .,푠 푛 ⟩ Output: List of Pattern 2 instances findings 1 paths←∅; findings←∅; 2 for each step 푠 푖 ∈ 푇 do 3text← 푠 푖 .thought; kw← ExtractSemanticEntities(text); 4 if text contains global re-evaluation signal then 5paths←푝 ∈ paths : no keyword matches 푝.kw; 6 if text matches path-selection pattern then 7푝 ←start : 푖, kw : kw, snippet : text[: 500], cnt_cont : 0, cnt_alt : 0, cnt_contrad : 0; paths.append(푝); 8 for each 푝 ∈ paths do 9if kw overlaps with 푝.kw then 10푝.cnt_cont← 푝.cnt_cont+ 1; 푝.last_cont← 푖; 11if text matches alternative-mentioned pattern and keywords overlap then 12푝.cnt_alt← 푝.cnt_alt+ 1; 13if text matches contradiction-absorbed pattern and keywords overlap then 14푝.cnt_contrad← 푝.cnt_contrad+ 1; 푝.last_contrad← 푖; 15 for each 푝 ∈ paths do 16if 푝.cnt_cont≥ 5 and span≥ 10 then 17end← max(푝.last_cont,푝.last_contrad,푝.start); 18findings.append(start_step : 푝.start, end_step : end, span : end− 푝.start, cont_count : 푝.cnt_cont, alt_count : 푝.cnt_alt, contrad_count : 푝.cnt_contrad); paths.remove(푝); 19 return findings recorded when continuation count reaches at least 5 occurrences and the span exceeds 10 steps (lines 19–23). The key algorithmic components are: (1) Semantic entity extrac- tion—function names, memory addresses, and path-related terms are extracted to track continuity; (2) Path continuation tracking— overlapping semantic entities between steps indicate sustained focus; (3) Alternative mention detection—signals of alternative paths mentioned but not explored are tracked; (4) Contradiction absorption detection—linguistic patterns indicating acknowledgment of con- tradictions without triggering global re-evaluation; and (5) Global re-evaluation exclusion—explicit signals of approach switching re- move affected paths. D.3 Pattern 3 Detection Pattern 3 detection operates over thethoughtfield of each reason- ing step, tracking deferred path mentions, development events, and targeted backtracking. The detector implements a state-tracking algorithm that maintains lists of deferred and developed paths, monitoring for backtracking signals that indicate revisitation of previously deferred candidates. Implicit Patterns in LLM-Based Binary AnalysisConference’17, July 2017, Washington, DC, USA Algorithm 3: Pattern 3 Detection Algorithm Input: Sequence of reasoning steps푇=⟨푠 1 ,푠 2 , . . .,푠 푛 ⟩ Output: List of Pattern 3 instances findings 1 deferred←∅; materialized←∅; findings←∅; 2 for each step 푠 푖 ∈ 푇 do 3text← 푠 푖 .thought; kw← ExtractSemanticEntities(text); 4 if text matches multi-candidates pattern then 5푝 ←m_idx : 푖, kw : kw, snippet : text[: 500], mat : False, bt : False; deferred.append(푝); 6 if text matches development pattern then 7for each 푝 ∈ deferred do 8if kw overlaps with 푝.kw then 9푝.mat← True; 푝.mat_idx← 푖; materialized.append(deferred_idx : 푝.m_idx, mat_idx : 푖, kw : kw); break; 10 if text matches backtracking pattern then 11new_sig← CheckNewSignal(text,푇,푖); impasse← CheckImpasse(text,푇,푖); 푝 ← FindMatchingDeferredPath(deferred, kw,푖); 12if 푝≠ None then 13푝.bt← True; 푝.bt_idx← 푖; 푝.bt_snippet← text[: 500]; 14 for each 푝 ∈ deferred do 15 if 푝.bt and(푝.bt_idx− 푝.m_idx) ≥ 1 then 16mat_idx← FindMatBeforeBt(materialized,푝.bt_idx); 17findings.append(deferred_step : 푝.m_idx, mat_step : mat_idx, bt_step : 푝.bt_idx, span : 푝.bt_idx− 푝.m_idx); 18 return findings Algorithm Overview. The algorithm 3 maintains two lists: deferred paths (lightweight mentions) and developed paths (explic- itly developed). For each step, it detects multi-candidate mentions and creates deferred path records (lines 7–8), identifies develop- ment events by matching semantic entities between current text and deferred paths (lines 9–14), and detects targeted backtracking through linguistic patterns indicating explicit return to previously mentioned paths (lines 15–19). Backtracking detection includes checks for new evidence signals or analysis impasses. Instances are recorded when a deferred path is backtracked after at least one intervening step (lines 21–24). The key algorithmic components are: (1) Deferred path tracking— multiple candidate paths are recorded when mentioned without full expansion; (2) Development detection—semantic entity overlap indicates explicit development; (3) Targeted backtracking identifica- tion—linguistic patterns combined with semantic entity matching distinguish explicit revisitation from simple continuation; (4) Con- text signal detection—new evidence or impasse signals provide addi- tional context for revisitation triggers; and (5) Temporal separation enforcement—minimum span requirements ensure backtracking represents revisitation rather than immediate continuation. Algorithm 4: Pattern 4 Detection Algorithm Input: Sequence of reasoning steps푇=⟨푠 1 ,푠 2 , . . .,푠 푛 ⟩ Output: List of Pattern 4 instances findings 1 findings←∅; 2 for each step 푠 푖 ∈ 푇 do 3text← 푠 푖 .thought; kw← ExtractSemanticEntities(text); 4analogy← MatchPattern(text, analogy_pat); 5priority← MatchPattern(text, priority_pat); 6sig_score← MatchPattern(text, sig_score_pat); 7partial← MatchPattern(text, partial_pat); 8justif← MatchPattern(text, justif_pat); 9select← MatchPattern(text, select_pat); 10feat_cnt← CntFeat(analogy, priority, sig_score, 11partial, justif, select); 12core← analogy or sig_score; 13decision← select or priority; 14 if (core and decision) or(feat_cnt≥ 3 and core) then 15justif_snip← ExtractJustificationSnippet(text); 16findings.append(step : 푖, feat_cnt : feat_cnt, justif_snip : justif_snip, kw : kw); 17 return findings D.4 Pattern 4 Detection Pattern 4 detection operates over thethoughtfield of each rea- soning step, identifying path-selection or prioritization decisions and their justifications. The detector implements a feature-based classification algorithm that evaluates multiple linguistic signals to determine whether a step represents knowledge-guided prioritiza- tion. Algorithm Overview. The algorithm 4 evaluates each reasoning step for six linguistic features: analogy signals, priority assignments, signal scoring, partial evidence references, justification statements, and path-selection actions. A Pattern 4 instance is recorded when either (1) a core feature (analogy or signal scoring) co-occurs with a decision feature (path selection or priority assignment), or (2) at least three features are present including a core feature (lines 14–16). This two-tier criterion ensures that detected instances represent reasoned prioritization decisions rather than incidental mentions. The key algorithmic components are: (1) Multi-feature detection— six complementary linguistic patterns capture different aspects of prioritization reasoning; (2) Core feature requirement—analogy or signal scoring must be present; (3) Decision feature requirement— path selection or priority assignment must be present; (4) Feature count threshold—alternative criterion of three or more features with a core feature provides robustness; and (5) Justification extraction— snippets containing decision rationale are extracted for evidence preservation. E Metric Measurement Details All metrics are computed over pattern-aligned reasoning segments extracted from full interaction traces. For each detected pattern Conference’17, July 2017, Washington, DC, USAQiang Li, XiangRui Zhang, and Haining Wang instance, we identify the corresponding contiguous segment of as- sistant steps and compute the following metrics using deterministic, script-based procedures released in the artifact. Path Length (퐿). The total number of assistant steps within the pattern-aligned segment, counted from the segment start to end. This metric directly measures the reasoning horizon associated with each pattern instance. Forward Step Ratio (퐹). The fraction of assistant steps labeled withstatus="continue"among all assistant steps in the segment. This captures the degree of forward progression relative to halting or revision behavior. Branching Factor (퐵). Branching behavior is operationalized by detecting explicit decision points in thethoughtfield using keyword-based matching (e.g., “choose”, “select”, “decide”). At each decision point, we detect references to multiple alternatives using predefined linguistic patterns (e.g., “multiple paths”, “several op- tions”, “various candidates”). The branching factor is estimated as 퐵=1.0+훼·푟, where푟is the fraction of decision points mentioning multiple alternatives and훼=1.5 is a fixed scaling constant. This heuristic provides a consistent approximation of how many alter- natives are considered during reasoning, without assuming explicit control-flow reconstruction. Backtrack Count (푅). Backtracking events are detected by matching explicit revision language in thethoughtfield (e.g., “back- track”, “go back”, “return to”, “revisit”) as well as references to earlier analysis locations. For Pattern 3 instances,푅is set to 1 by defini- tion, as Pattern 3 represents targeted backtracking behavior by construction. Pruning Rate (푃). Pruning is measured at detected decision points. We identify pruning behavior by matching elimination- related expressions (e.g., “skip”, “ignore”, “discard”, “not worth”, “irrelevant”, “focus on”). The pruning rate is defined as 푃= number of decision points with pruning signals total number of decision points This metric quantifies how aggressively candidate paths are elimi- nated during reasoning. E.1 Tool Usage Topology Metrics For each pattern-aligned segment, we extract the sequence of tool invocations from assistant steps and compute the following topol- ogy metrics. Command Diversity. The number of distinct command types appearing in the segment. Command types are extracted by pars- ing theactionfield and retaining only the base command name (parameters removed). Sequence Length. The total number of tool invocations in the segment, measured as the count of assistant steps with non-empty action fields. Max Depth. The maximum length of consecutive repetitions of the same command type within the invocation sequence. This captures repetitive tool usage patterns. Max Fan-out. The maximum number of distinct successor com- mand types following any given command. We construct a directed transition graph over command types and compute fan-out per node. Cycle Presence. A binary indicator denoting whether the com- mand transition graph contains at least one directed cycle, detected via depth-first search. Transition Entropy. For each command푐 푖 , we compute the entropy of its outgoing transition distribution 퐻(푐 푖 )=− ∑︁ 푐 푗 푝(푐 푗 |푐 푖 ) log 2 푝(푐 푗 |푐 푖 ) Overall transition entropy is computed as a weighted average over all commands, weighted by their transition frequencies. Lower val- ues indicate deterministic tool usage, while higher values indicate diversified transitions. All metrics are computed using fixed scripts provided in the artifact repository, enabling full reproducibility across binaries, patterns, and experimental settings.