Paper deep dive
Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization
Ziqiao Kong, Wanxu Xia, Chong Wang, Yi Lu, Pan Li, Shaohua Li, Zong Cao, Yang Liu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/31/2026, 1:41:10 AM
Summary
Knowdit is an agentic, knowledge-driven framework for smart contract vulnerability detection. It constructs an auditing knowledge graph from historical audit reports to link DeFi semantics with vulnerability patterns. A multi-agent system then uses this graph to perform iterative specification generation, harness synthesis, fuzz execution, and reflection, significantly outperforming existing baselines in detecting high- and medium-severity vulnerabilities.
Entities (6)
Relation Signals (4)
Knowdit → constructs → Auditing Knowledge Graph
confidence 95% · Knowdit first constructs an auditing knowledge graph from historical human audit reports
Knowdit → evaluateson → Code4rena
confidence 95% · We evaluate Knowdit on 12 recent Code4rena projects
Auditing Knowledge Graph → links → DeFi semantics
confidence 95% · linking fine-grained DeFi semantics with recurring vulnerability patterns.
InsureDAO → sharesvulnerabilitywith → Salty.IO
confidence 90% · they both stem from the same DeFi semantic pattern, proportional-share token accounting
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Smart contracts govern billions of dollars in decentralized finance (DeFi), yet automated vulnerability detection remains challenging because many vulnerabilities are tightly coupled with project-specific business logic. We observe that recurring vulnerabilities across diverse DeFi business models often share the same underlying economic mechanisms, which we term DeFi semantics, and that capturing these shared abstractions can enable more systematic auditing. Building on this insight, we propose Knowdit, a knowledge-driven, agentic framework for smart contract vulnerability detection. Knowdit first constructs an auditing knowledge graph from historical human audit reports, linking fine-grained DeFi semantics with recurring vulnerability patterns. Given a new project, a multi-agent framework leverages this knowledge through an iterative loop of specification generation, harness synthesis, fuzz execution, and finding reflection, driven by a shared working memory for continuous refinement. We evaluate Knowdit on 12 recent Code4rena projects with 75 ground-truth vulnerabilities. Knowdit detects all 14 high-severity and 77\% of medium-severity vulnerabilities with only 2 false positives, significantly outperforming all baselines. Applied to six real-world projects, Knowdit further discovers 12 high- and 10 medium-severity previously unknown vulnerabilities, proving its outstanding performance.
Tags
Links
- Source: https://arxiv.org/abs/2603.26270v1
- Canonical: https://arxiv.org/abs/2603.26270v1
Trouble viewing inline? Open PDF directly →
Full Text
77,225 characters extracted from source content.
Expand or collapse full text
Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization Ziqiao Kong ziqiao001@e.ntu.edu.sg Nanyang Technological University Singapore, Singapore, Singapore Wanxu Xia ysiel@buaa.edu.cn National Superior College for Engineers, Beihang University Beijing, Beijing, China Chong Wang ∗ chong.wang@ntu.edu.sg Nanyang Technological University Singapore, Singapore, Singapore Yi Lu Movebit Singapore, Singapore y@movebit.xyz Pan Li Bitslab Singapore, Singapore paul@bitslab.xyz Shaohua Li shaohuali@cuhk.edu.hk The Chinese University of Hong Kong Hong Kong, Hong Kong, China Cao Zong z.cao@imperial.ac.uk Imperial Global Singapore Singapore, Singapore, Singapore Yang Liu yangliu@ntu.edu.sg Nanyang Technological University Singapore, Singapore, Singapore Abstract Smart contracts govern billions of dollars in decentralized finance (DeFi), yet automated vulnerability detection remains challeng- ing because many vulnerabilities are tightly coupled with project- specific business logic. We observe that recurring vulnerabilities across diverse DeFi business models often share the same under- lying economic mechanisms, which we term DeFi semantics, and that capturing these shared abstractions can enable more system- atic auditing. Building on this insight, we propose Knowdit, a knowledge-driven, agentic framework for smart contract vulnera- bility detection. Knowdit first constructs an auditing knowledge graph from historical human audit reports, linking fine-grained DeFi semantics with recurring vulnerability patterns. Given a new project, a multi-agent framework leverages this knowledge through an iterative loop of specification generation, harness synthesis, fuzz execution, and finding reflection, driven by a shared working mem- ory for continuous refinement. We evaluate Knowdit on 12 recent Code4rena projects with 75 ground-truth vulnerabilities. Knowdit detects all 14 high-severity and 77% of medium-severity vulnerabilities with only 2 false posi- tives, significantly outperforming all baselines. Applied to six real- world projects, Knowdit further discovers 12 high- and 10 medium- severity previously unknown vulnerabilities, proving its outstand- ing performance. ∗ Corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference acronym ’X, Woodstock, NY © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-X-X/2018/06 https://doi.org/X.X CCS Concepts • Do Not Use This Code→Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper. Keywords Do, Not, Use, This, Code, Put, the, Correct, Terms, for, Your, Paper ACM Reference Format: Ziqiao Kong, Wanxu Xia, Chong Wang, Yi Lu, Pan Li, Shaohua Li, Cao Zong, and Yang Liu. 2018. Knowdit: Agentic Smart Contract Vulnerability Detection with Auditing Knowledge Summarization. In Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym ’X). ACM, New York, NY, USA, 12 pages. https://doi. org/X.X 1 Introduction Smart contracts are stateful programs deployed on blockchains that manage assets worth billions of dollars. Due to the transparency and immutability of blockchain systems, vulnerabilities in smart contracts can lead to significant financial losses [53]. For example, a recent security report 1 by CertiK shows that over $3.35 billion in cryptocurrency was stolen through hacks and scams in 2025 alone. To mitigate these risks, automated analysis techniques, such as static auditing [36] and fuzz testing [12,21,34], have been proposed to detect bugs and security vulnerabilities in smart contracts and thereby safeguard on-chain assets. Recently, large language models (LLMs) have shown promising results across a range of code intelligence and security tasks, in- cluding smart contract vulnerability detection [38]. GPTScan [36] pioneers in matching smart contract code with potential attack- ing scenarios to reason about potential vulnerabilities. Several works [27,30,36,39] leverage LLMs’ ability to understand business 1 https://w.certik.com/blog/hack3d-the-web3-security-report-2025 arXiv:2603.26270v1 [cs.CR] 27 Mar 2026 Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. models and generate invariants to detect vulnerabilities. Smart- Poc [11] andA1[17] rely on LLMs for detailed reasoning and exe- cution to synthesize exploits for vulnerabilities. Despite these advances, automated smart contract auditing is still rather challenging. A key reason is that many vulnerabilities are tightly coupled with specific business scenarios [27,51], mak- ing it difficult to systematically model DeFi security properties and leverage them for comprehensive auditing processes and effec- tive detection oracles. Unlike traditional software vulnerabilities (e.g., use-after-free), smart contract vulnerabilities often emerge within DeFi protocols that implement highly customized business logic [51]. This logic governs asset flows, token accounting, and complex financial operations. As a result, the vast space of possible DeFi scenarios and auditing objectives creates significant challenges for automated detection, particularly due to the gap between ac- curately capturing contract semantics and applying appropriate auditing workflows and oracles. For example, generic fuzzing or- acles like profit generation or reentrancy are often insufficient to cover the wide variety of auditing scenarios. Our key insight is that the high false-negative rate largely stems from the lack of high-level abstractions that capture the shared auditing knowledge underlying diverse DeFi protocols and their potential vulnerabilities. We define the fine-grained functionalities that implement specific economic mechanisms as DeFi seman- tics [44]. In practice, many vulnerabilities originate from common DeFi semantics embedded in protocol business logic, along with the attack scenarios that exploit them. While protocol designs and concrete implementations may differ due to varying business re- quirements, the underlying economic mechanisms represented by these DeFi semantics—and their associated attack surfaces—often recur across protocols. For example, two smart contract vulnerabil- ities discovered in InsureDAO in 2022 and Salty.IO in 2024 illustrate this recurring pattern. Although these vulnerabilities appear in en- tirely different protocols—one focused on insurance underwriting and the other on automated market-making and liquidity man- agement—they both stem from the same DeFi semantic pattern, proportional-share token accounting, and share a common attack scenario known as the first-depositor attack. This observation under- scores the importance of leveraging high-quality, abstract auditing knowledge that captures both shared DeFi semantics and the attack scenarios they enable. By modeling these essential properties, rather than being distracted by superficial differences in implementation, we can enable more effective vulnerability detection. However, effectively utilizing such auditing knowledge presents several challenges. First, there is a lack of systematic understanding and modeling of the essential knowledge elements and the relation- ships among them during smart contract auditing. Second, current automated methods are inadequate for fully extracting and evolv- ing the auditing knowledge from heterogeneous sources. Many existing works [27,28,36,39] either rely on the fixed template or extra manual efforts for such knowledge, preventing the large-scale adoption of the workflows. Third, accurately mapping abstract au- diting knowledge to concrete smart contract implementations and performing vulnerability detection remains challenging for general and scalable automated auditing [27, 44]. To address these challenges, we propose a novel knowledge- driven, agentic method for smart contract vulnerability detection, called Knowdit. Our method consists of two main phases. First, we construct an auditing knowledge graphGfrom historical human reports based on a schema that separates the DeFi Space, which captures business types, Solidity projects, and fine-grained DeFi semantics, and the Vulnerability Space, which encodes vulnerability patterns, auditing findings, and attack types, with links representing historical traceability and potential causal relationships. The graph is built incrementally using an LLM-based pipeline that abstracts, classifies, and deduplicates knowledge. Second, given a new Solidity project, our agentic auditing framework uses a shared Working Memory to track execution, coverage, and feedback. It maps the project to relevant DeFi semantics and vulnerability patterns and iteratively processes each semantic and vulnerability pair through Specification Generation, Harness Synthesis, Fuzz Execution, and Finding Reflection. Execution failures or specification issues are recorded to guide regeneration, while confirmed vulnerabilities are reported and integrated back into the knowledge graph, enabling automated, human-like auditing with continuous refinement and knowledge accumulation. We evaluate Knowdit on a dataset of 12 recent Solidity projects from Code4rena, containing 100 contracts and 75 high- and medium- severity vulnerabilities, using a knowledge graph built from 270 historical projects that captures 475 DeFi semantics, 579 vulner- ability patterns, and 2,096 links between them. On this dataset, Knowdit significantly outperforms existing tools by detecting all 14 high-severity vulnerabilities and 77% of 61 medium-severity vulnerabilities with minimal false positives, demonstrating high precision and recall. The knowledge graph contributes substantially to effectiveness, achieving 88% coverage of ground-truth vulnera- bilities compared to 45% for the best baseline, and an ablation study without the knowledge graph shows clear drops in both coverage and precision. In real-world projects, Knowdit identifies 12 high and 10 medium vulnerabilities, all confirmed and fixed by devel- opers, including severe liquidity-draining issues, while generating concrete proofs of exploitation. Regarding cost, Knowdit incurs higher token usage than baselines but remains far cheaper than manual auditing, and this overhead is justified by its ability to detect critical vulnerabilities that threaten millions of dollars. In summary, this paper makes the following contributions: •We propose a novel knowledge-driven, agentic workflow for smart contract vulnerability detection called Knowdit, which leverages high-level DeFi semantics and vulnerability patterns to capture rich auditing knowledge. •We construct an auditing knowledge graph from 270 historical projects, comprising 475 DeFi semantics across 13 categories and 2,096 links between these semantics and vulnerability pat- terns, enabling the generation of more comprehensive auditing specifications. •Evaluation results show that Knowdit effectively discovers vulnerabilities, identifying 14 known high-severity vulnerabili- ties reported on Code4rena and 12 previously unknown high- severity vulnerabilities in new real-world projects, thereby helping prevent the loss of millions of dollars in user assets. Knowdit : Agentic Smart Contract Vulnerability Detection with Auditing Knowledge SummarizationConference acronym ’X, June 03–05, 2018, Woodstock, NY 2 Motivation We present a motivating example to illustrate the challenges of detecting vulnerabilities in diverse smart contracts and our obser- vations to leverage shared DeFi semantics. Vulnerabilities often recur in diverse business scenarios. Vul- nerabilities in smart contracts are recurring, like the notorious reentrancy vulnerabilities causing millions of dollars in loss [9] and numerous exploits of the forked projects [53], due to the in- herent design of the EVM [29,42] and the prevailing code reuse in the Solidity ecosystem [31,35]. Besides the recurring vulnerabili- ties caused by code reuse, distinct smart contracts implementing different business logic may also contain vulnerabilities sharing the same root cause. To illustrate our observation, we manually study one vulnerability that recurs after 2 years with the same pattern on code4rena [4] from InsureDAO 2 in 2022 and Salty.IO 3 in 2024, respectively, a leading platform for public competitive auditing. In InsureDAO, the first depositor can seed a pool with a tiny amount and then inflate the attribution-to-liquidity ratio, causing later depositors to receive zero or severely undervalued shares and allowing the attacker to drain subsequent deposits. In Salty.IO, the first liquidity provider can likewise manipulate the initial rewards-per-share ratio by adding minimal liquidity and then injecting rewards, which breaks staking-reward accounting and lets the attacker capture rewards that should belong to later participants. These illustrate that it is difficult to directly map busi- ness scenarios to specific vulnerabilities, creating a gap that hinders automatic tools from generating effective, logic-specific auditing workflows and detection oracles. Recurring vulnerabilities often share the same DeFi seman- tics. Figure 1 presents the two vulnerable functions in InsureDAO and Salty.IO, respectively. Despite differences in business scenarios and implementation, both projects implement a proportional-shares accounting mechanism for tokens at a microscopic level, similar to the canonical ERC4626 design [1]. Let the total shares held by the contract be S, the total assets be A, a user deposit be푎, and the shares allocated to the user be푠. The allocation follows the ratio푠= 푎푆 퐴 . We refer to such fine-grained functionality that implements a spe- cific economic mechanism as DeFi semantics. As shown in Figure 1, InsureDAO calculates the expected minted shares푠given a specific asset amount_amount, while Salty.IO calculates the assets푎corre- sponding to a given share inputincreaseShareAmount. Because both projects share the same underlying proportional-shares ac- counting mechanism, they are vulnerable to the same first-depositor attack, where an adversary inflates the ratio 퐴 푆 to drain liquidity from other users [2,7]. By reviewing more similar cases, we observe that vulnerabilities in smart contracts are often associated with par- ticular DeFi semantics that recur across different business scenarios. This observation aligns with prior work [21,27,28,44,51], which shows that understanding high-level DeFi semantics enables the creation of precise invariants or oracles for detecting deep vulner- abilities. Therefore, by constructing an auditing knowledge base bridging low-level code details and high-level DeFi semantics, we can systematically identify potentially related vulnerabilities. 2 https://code4rena.com/reports/2022-01-insure. 3 https://code4rena.com/reports/2024-01-saltyio function addValue( uint _amount ) uint _pool = valueAll(); _attributions = (_amount * totalAttributions) / _pool; attributions[_beneficiary] += _attributions; InsureDAO (2022) function _increaseUserShare( uint increaseShareAmount ) uint virtualRewardsToAdd = Math.ceilDiv( totalRewards * increaseShareAmount, existingTotalShares); user.virtualRewards += uint128(virtualRewardsToAdd); Salty.IO (2024) Figure 1: Simplified root cause of the recurring “first depos- itor” pattern in InsureDAO and Salty.IO. The bold key vari- ables involve the proportional share accounting model. Solidity Projects Auditing Knowledge Schema DeFi Semantics Extraction AuditingRecords Vulnerability Pattern Extraction DeFi Space Vuln Space DeFi Semantics Extraction Knowledge Graph Knowledge Mapper Specification Generator Fuzz Executor Harness Synthesizer Finding Reflector FuzzEngine Working Memory TargetProject ReportedFindings AuditingKnowledgeGraphConstruction Agentic Auditing Framework Figure 2: The overview of our method Knowdit. 3 Methodology We propose a novel automated approach for smart contract auditing that integrates knowledge extracted from human auditing reports into a multi-agent fuzzing workflow. 3.1 Overview Figure 2 provides a high-level overview of our methodology, which consists of two main phases: constructing an auditing knowledge graph from historical human reports, and performing knowledge- driven, agentic auditing of new smart contract projects. We first construct an auditing knowledge graphGto capture essential smart contract auditing knowledge. The graph has a bi- partite structure: the DeFi Space encodes business types, Solidity projects, and fine-grained DeFi semantics, while the Vulnerability Space captures vulnerability patterns, auditing findings, and attack types. The two spaces are connected through historical traceability between projects and reports and potential causal links between DeFi semantics and vulnerability patterns. Knowledge is extracted from historical human auditing records using an LLM-based multi- stage pipeline, which abstracts, classifies, and deduplicates DeFi semantics and vulnerability patterns to incrementally build struc- tured, reusable knowledge. Given a new Solidity project, our auditing framework operates as a multi-agent system with a shared Working Memory that tracks ex- ecution, coverage, and reflective feedback. The Knowledge Mapper first identifies the project’s business types and retrieves associated DeFi semantics and vulnerability patterns, producing multiple se- mantic–vulnerability pairs. Each pair is then sequentially processed through the Specification Generator, Harness Synthesizer, Fuzz Ex- ecutor, and Finding Reflector. Failures in execution or issues in specifications are stored in working memory to guide regeneration of the harness or specification, while confirmed vulnerabilities are reported and ingested back intoG. This iterative, memory-driven loop enables automated, human-like auditing with continuous re- finement and knowledge accumulation. Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. 3.2 Auditing Knowledge Graph We first design a graph schema to represent essential auditing knowledge for smart contracts and then extract this knowledge from historical human auditing records. 3.2.1 Expertise-Inspired Knowledge Schema Design. Security audit- ing for smart contracts is a knowledge-intensive process. To enable comprehensive auditing, we first design a knowledge graph schema inspired by human expertise to model the essential knowledge ele- ments and relationships involved. Given a smart contract project, an experienced auditor typically analyzes its business scenarios and mentally connects them to relevant economic mechanisms and potential attack surfaces. For example, if a project’s core business involves a decentralized exchange (DEX), it is naturally associated with token swapping. In such cases, an auditor would reasonably suspect potential vulnerabilities such as price manipulation attacks, given their prevalence in DEX systems [36,48,51]. Therefore, it is important to establish associations among these key elements to provide the necessary knowledge for effective auditing. The schema of our knowledge graph is illustrated in Figure 3. It is a bipartite graph consisting of a DeFi space, which captures DeFi business scenarios and their associated DeFi semantics, and a Vulnerability space, which captures common vulnerability patterns and related attack risks. These two spaces are connected through historical traceability between Solidity projects and auditing find- ings, as well as potential causal links between DeFi semantics and vulnerability patterns. More specifically, the schema consists of the following types of knowledge elements and relationships: •The DeFi Space includes three types of knowledge elements: Solidity projects, which implement specific business logic in DeFi protocols; business types (e.g., decentralized exchange), which characterize the high-level functionality of DeFi business logic; and DeFi semantics, which represent fine-grained eco- nomic mechanisms (e.g., token swapping) implemented in smart contracts. The relationships among these elements include be- longs to between Solidity projects and business types, contains between Solidity projects and DeFi semantics, and underlies be- tween DeFi semantics and business types. All three relationships are many-to-many. •The Vulnerability Space also includes three types of knowl- edge elements: auditing findings, which are reported by hu- man experts and reveal real-world vulnerabilities; attack types (e.g., Denial-of-Service), which categorize the associated attack risks; and vulnerability patterns, which summarize recurring vulnerabilities in smart contracts (e.g., reentrancy could cause inconsistent states). The relationships among these elements in- clude contributes to between vulnerability patterns and auditing findings, poses between vulnerability patterns and attack types, and involves between auditing findings and attack types. All three relationships are many-to-many. •The connections between the DeFi Space and the Vulnera- bility Space include a one-to-many has relationship between Solidity projects and auditing findings, reflecting their historical traceability, and a many-to-many may introduce relationship between DeFi semantics and vulnerability patterns, capturing the potential causal links between them. Business Type Solidity Project DeFi Semantics Attack Type Auditing Finding Vulnerability Pattern DeFiSpace VulnerabilitySpace may introduce has exposes involves contains contributes to underlies belongs to Figure 3: The schema of our auditing knowledge graph. Uniswap V2 Semantic Definition: Swap token X to token Y Category: Dexes Description: Constant Product... Merges: [] function swap(...) require( reserve0 * reserve1>= k**2 ); Uniswap V3 Semantic Definition: Swap token X to token Y Category: Dexes Description: Constant Product of Virtual Reserves... Merges: [] New Semantic Node Definition: Three tokens swapping Category: Dexes Description: EMA Oracle... Merges: [] Merged Semantic Node Definition: Swap token X to token Y Category: Dexes Description: Constant Product Style Auto Market Maker DEX... Merges: ["Uniswap V2", "Uniswap V3"] Extract Extract Extract Deduplicate & Merge Uniswap V2 Uniswap V3 Curve Tricrypto function swap(...) while (amount > 0) swap_at_tick(...); function tricrypto_swap(...) ema_update(reserve1, reserve2, reserve3); Figure 4: An example for extracting and merging semantics from three DEX projects. The business types and attack types are predefined based on prior works [24,51]. Specifically, the business types include Lend- ing, Dexes, Yield, Services, Derivatives, Yield Aggregator, Real World Assets, Stablecoins, Indexes, Insurance, NFT Marketplace, NFT Lend- ing, and Cross Chain, while the attack types include Access Control, Arithmetic, Block Manipulation, Cryptographic, Denial of Service, Reentrancy, and Storage & Memory. 3.2.2 LLM-based Knowledge Extraction. Based on the designed schema, we propose an LLM-based multi-stage pipeline to extract knowledge elements and relationships from historical auditing re- ports written by human experts. In general, we expect the LLM to mimic human learning by summarizing knowledge into highly abstract, structured representations that serve as reusable experi- ence [22]. We further adopt chain-of-thought [40] prompting and in-context learning with examples [19] to enhance the reasoning reliability of the LLM. Stage I: DeFi Semantics Extraction and Abstraction. This stage constructs the DeFi Space subgraph by traversing and processing a given set of Solidity projects. We denote the resulting subgraph as G 푑푒푓푖 . The detailed steps for processing each project are as follows: •Classifying the project’s business type. The project is clas- sified into one or more business types using the classification prompt template in Figure 5. The LLM evaluates each function or component against predefined type definitions and repre- sentative examples. For instance, a project implementing token swaps following the Uniswap V2 design [7] would be classified as a DEX (decentralized exchange). For each classification, the LLM provides step-by-step reasoning before producing the final verdict. Following previous work [36], a small set of examples for each business type is prepared in the prompt template once to enable in-context learning. For completeness, we chunk and feed the source code, tests, and any documents into the LLM’s context window. • Extracting candidate DeFi semantics. The LLM reviews the project’s source code and documentation, which are split into chunks to fit within the context window. Using the extraction Knowdit : Agentic Smart Contract Vulnerability Detection with Auditing Knowledge SummarizationConference acronym ’X, June 03–05, 2018, Woodstock, NY Classification Prompt Template Task Definition Given the following inputs, classify them into one or more categories below. %CATEGORIES WITH EXAMPLES% Step-by-step Instructions You should first read all the provided %PROJECT SOURCE CODE/- DOCUMENTS/REPORTS%. Then, carefully evaluate each "Func- tion"/"Audit Finding" against each category definition and its exam- ples by ignoring any project-specific details like branding. For each category, you must first output a step-by-step technical reasoning ex- plaining why the input aligns or conflicts with the category. After providing your reasoning, state your final verdict. Figure 5: Chain-of-thought style general prompt for classify- ing project code, documents, or audit reports. Extraction Prompt Template Task Definition Given the following inputs, your task is to summarize them into ab- stract "DeFi Semantics"/"Vulnerability Patterns" of the following categories. %CATEGORIES WITH EXAMPLES% Step-by-step Instructions First, thoroughly review the provided %PROJECT SOURCE CODE/- DOCUMENTS/REPORTS%. For each "Contract"/"Audit Find- ing", abstract its core mechanics and intentions, stripping away implementation-specific details to formulate a generalized model fol- lowing the examples. Next, cross-reference these models against the %PREVIOUS SEMANTICS/VULNERABILITIES PATTERN% to evaluate novelty. Focus strictly on the underlying abstract logic. Cru- cially, before making a final determination, you must first articulate your reasoning: explicitly explain which mechanical aspects overlap with an existing entry, or why the core logic is entirely unprecedented. After providing this comparative analysis, categorize the item as new if it is novel; or, if it overlaps, synthesize and update the prior description to comprehensively cover both variations. Figure 6: Simplified prompt for extracting DeFi semantics or vulnerability patterns from the projects. prompt template in Figure 6, the LLM summarizes each con- tract’s DeFi semantics by abstracting its core mechanics and intentions while removing implementation-specific details. In addition, we ask the LLM to check if any extracted DeFi seman- tics can be merged with previous DeFi semantics. This process captures the fine-grained economic mechanisms embedded in the project. The project is then added toG 푑푒푓푖 , and a belongs to relationship is established between it and the corresponding business type. If a project belongs to multiple business types, multiple belongs to relationships are created accordingly. •Deduplicating and summarizing DeFi semantics. Once the business type(s) are determined, we retrieve the subset of ex- isting DeFi semantics inG 푑푒푓푖 corresponding to the same busi- ness type(s). The LLM compares the newly extracted semantics against this subset. If a candidate overlaps with an existing DeFi semantic, the two are merged and synthesized into a new graph node that captures both variations. The original node inG 푑푒푓푖 is replaced with the merged node, and the current project is linked to it via a contains relationship. Otherwise, the candidate is added toG 푑푒푓푖 as a new node, and contains and underlies rela- tionships are established with the project and its business type(s), respectively, following the schema. This combined deduplica- tion and abstraction step reduces redundancy and efficiently builds a structured, reusable knowledge graph. We present an illustrative example of the workflow in Figure 4 that extracts and merges DeFi semantics from three distinct DEX projects, UniswapV2 [7], UniswapV3 [8], and Curve Tricrypto [15]. For UniswapV2 and UniswapV3, although their implementations are dramatically different, they share the same DeFi semantics Swap token X to token Y, and we summarize their economic model Constant Product into the merged semantic node. However, Curve Tricrypto supports swapping with three tokens, unlike the two- token swapping model of Uniswap families. Therefore, we consider it a distinct DeFi semantic node. This stage begins with an emptyG 푑푒푓푖 . As more projects are processed, the graph is continuously updated, and knowledge about DeFi semantics accumulates over time. Stage I: Vulnerability Pattern Extraction and Summariza- tion. This stage constructs the Vulnerability Space subgraph by processing a set of human auditing reports associated with the analyzed projects. We denote the resulting subgraph asG 푣푢푙푛 . The overall workflow of constructing the subgraph follows the same extracting–classifying–deduplicating paradigm and similar prompt templates as Stage I, as illustrated in Figure 5 and Figure 6, but operates on auditing reports instead of source code and focuses on vulnerability-related knowledge. •Classifying the report’s attack type. Each finding is classi- fied into one or more attack types (e.g., Reentrancy) using the classification prompt with predefined categories and examples. •Extracting candidate vulnerability patterns. Specifically, the LLM abstracts each auditing finding into a vulnerability pattern by capturing its root cause and core logic, rather than implementation details. • Deduplicating and summarizing vulnerability patterns. During deduplication, the extracted patterns are compared against existing nodes inG 푣푢푙푛 within the same attack type(s). Over- lapping patterns are merged and refined into unified represen- tations, while novel patterns are added as new nodes. Corre- spondingly, contributes to relationships are established between auditing findings and vulnerability patterns, and involves rela- tionships are created between findings and attack types, follow- ing the schema definition. This stage also begins with an emptyG 푣푢푙푛 and incrementally accumulates structured knowledge about vulnerability patterns as more auditing reports are processed. Stage I: Causal Linking between DeFi Semantics and Vul- nerability Patterns. Given the two subgraphsG 푑푒푓푖 andG 푣푢푙푛 , we further establish causal links between DeFi semantics and vul- nerability patterns. Specifically, we first construct project–report pairs based on the traceability between Solidity projects and their corresponding auditing reports, which are typically available in public auditing records. For each project–report pair, we collect the set of DeFi semanticsDfrom the project, and the set of vulnerability patterns Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. Linking Prompt Template Task Definition You are given a list of DeFi semantics and vulnerability patterns, and your task is to link the vulnerabilities to related DeFi semantics. %DEFI SEMANTICS% %VULNERABILITY PATTERNS% Step-by-step Instructions For each vulnerability pattern, you should review each given DeFi se- mantic and explain whether the semantic is prone to such vulnerabilities. Once you are sure that the semantic is closely related to the vulnerabil- ity pattern, output your reasoning along with the link. Figure 7: Simplified prompt for linking DeFi semantics and vulnerability patterns from the knowledge graph. V from the associated auditing report. We then prompt the LLM to identify potential links between elements inDandV, as illustrated in Figure 7. A link is established between a DeFi semantic and a vulnerability pattern if the semantic can potentially introduce or contribute to the vulnerability characterized by the pattern. After processing all project–report pairs, we obtain the final auditing knowledge graphG. 3.3 Agentic Auditing Framework Given a new Solidity project, we employ an agentic framework consisting of four LLM-based components and a fuzzing component, which leverages the constructed knowledge graph to perform an iterative fuzzing and verification loop. 3.3.1 Overall Workflow. Our framework operates as a multi-agent system with a shared Working Memory that tracks feedback, cov- erage, and execution information throughout the auditing process. Given a new Solidity project, the workflow proceeds as follows with an empty working memory: the Knowledge Mapper identifies the project’s business types and retrieves the associated DeFi seman- tics and linked vulnerability patterns fromG, producing multiple semantic–vulnerability pairs. Each pair is processed sequentially through the Specification Generator, which concretizes abstract knowledge into an auditing specification including several key states of the attacking scenarios; the Harness Synthesizer, which creates an executable fuzzing harness based on the specification; the Fuzz Executor, which runs the harness and collects coverage and execution data; and the Finding Reflector, which validates violations. Execution failures in the Fuzz Executor are recorded in the working memory and trigger re-synthesis of the harness. Feedback from the Finding Reflector is also stored in memory to guide future scheduling: if a violation stems from a problematic specification, the workflow returns to the Specification Generator to regenerate it; if a valid vulnerability is confirmed, it is reported, added toG, and the workflow proceeds to the next pair. This loop iterates over all pairs, enabling memory-driven, iterative refinement and human-like auditing. 3.3.2 Knowledge Mapper. We first analyze the given project to identify its business types and extract its involved DeFi semantics by prompting the LLM with the templates in Figure 5 and Fig- ure 6, respectively. For each extracted DeFi semantic, we prompt the LLM to identify its matches among the semantics associated with the identified business types in the knowledge graphG. We then retrieve the linked vulnerability patterns and assemble a set of semantic–vulnerability pairs. Next, we will iterate over every semantic-vulnerability pair for the following procedures, and we will prefer the semantics that were less tested, by fetching the line coverage from the Working Memory written by Fuzz Executor. 3.3.3 Specification Generator. We then concretize the mapped ab- stract knowledge of DeFi semantics and associated vulnerability patterns into project-specific, actionable auditing specifications. Definition of Auditing Specification. An auditing specification defines several key states of the attacking scenario by the invariants of the relevant state variables. Each semantic-vulnerability pair is supposed to have a unique auditing specification: •Initial State. The state before executing any function calls. Usu- ally, it refers to the outcome of the contract setup. For example, for a Dexes project to test typical Price Manipulation vulnerabil- ity, the initial state will contain the expected pools and tokens that should be available and funded for testing. •Pre-Vuln State. The state before the attack happens, for exam- ple, the pool is supposed to have some liquidity. •Post-Vuln State. The state after the vulnerability is triggered, for example, the pool is drained, and the price is unreasonably shifted. 3.3.4 Harness Synthesizer. We synthesize Foundry [3] fuzzing har- nesses from the generated specifications. Specifically, we prompt the LLM to encode Initial State to a Foundry setUp function that deploys contracts and initializes accounts, translate Pre-Vuln State and Post-Vuln State invariants into require statements as oracles. In addition, we derive Foundry handlers following the practice from Verite [21] as a thin wrapper of a few external calls to implement the given abstract DeFi semantics. We iterate the fuzzing harness by compiling and feeding back the errors and fixes into the Working Memory. 3.3.5 Fuzz Executor. We run the fuzzing harness for the configured timeout and collect the line coverage into the Working Memory. In case any violation is triggered, we further collect the full execution trace and state changes into the Working Memory. 3.3.6 Finding Reflector. Once Knowdit finds a violation, we first prompt LLM to compare the state changes and trace with the audit specification to see if the violation matches the vulnerability pattern. If the violation fails to match the specification, Knowdit further classifies it into: • Expected Behavior. Many violations do not indicate vulner- abilities because smart contracts are often designed to revert under certain conditions. For example, the common onlyOwner check ensures that only privileged users can execute a func- tion; any abort caused by this check is expected and not a valid vulnerability. •Problematic Specification or Harness. Some violations arise from incomplete or inaccurate fuzzing harness or specifications, such as missing deployment setup or incorrect assumptions about initial states. For instance, failing to initialize contracts may produce false alarms for oracles. Knowdit : Agentic Smart Contract Vulnerability Detection with Auditing Knowledge SummarizationConference acronym ’X, June 03–05, 2018, Woodstock, NY For both cases, Knowdit writes the validation results and rea- sons for the verdict into the Working Memory and regenerates the specification and fuzzing harness. Even if the vulnerability matches the specification, Knowdit performs a further review if the vulnerability is out-of-scope ac- cording to the project README and Code4rena general rules 4 . For example, many projects explicitly state they will not integrate any fee-on-transfer tokens, and thus Knowdit will rule out findings involving such tokens, even though the findings could be valid. Once the finding passes the review, Knowdit will report it as a true finding. 4 Evaluation We conduct extensive experiments to evaluate the effectiveness of Knowdit for smart contract auditing. Specifically, we aim to answer the following research questions: •RQ1 (Dataset Effectiveness): Does Knowdit outperform ex- isting auditing tools in end-to-end evaluations on the dataset? •RQ2 (Knowledge Contribution): How does the knowledge graph in Knowdit contribute to the vulnerability identification? •RQ3 (Real-World Auditing): How effective is Knowdit in discovering previously unknown real-world vulnerabilities? •RQ4 (Cost Analysis): What is the token cost of auditing smart contracts with Knowdit? 4.1 Experimental Setup We present the implementation details of Knowdit, along with the construction of the evaluation dataset and the selection of baselines. 4.1.1 Implementation. Knowdit is written in around 10k lines of Rust code and relies on Foundry [3] as the core fuzzing engine. In our current implementation, GPT-5.1 serves as the primary LLM, handling most components of Knowdit. The only exception is the synthesis and repair of fuzzing harnesses, which is performed using GPT-5-mini. This choice is due to the well-defined nature of harness synthesis, which often requires multiple repair iterations. All model parameters (e.g., temperature) are set to their default values. To construct the knowledge graph, we collected public Solidity audit contests from Code4rena [4], including both source code and corresponding audit reports. Considering that the knowledge cut- off dates of GPT-5.1 and GPT-5-mini are 30/09/2024 and 31/05/2024, respectively, we limit our collection to audit contests between 02/04/2021 and 30/09/2024, reserving newer data for evaluation. The audit reports include manually verified vulnerabilities categorized by severity: High, Medium, and Quality Assurance. Following prior work [41,51], we filter out all Quality Assurance issues, as they typically relate to code style or gas optimization and do not impact contract integrity. This process yields a dataset of 270 projects with 3,904 vulnerabilities for knowledge graph construction. Resulting Knowledge Graph. The graph contains 475 dedupli- cated DeFi semantics merged from 1,429 candidates, 579 vulnerabil- ity patterns derived from 3,904 audit findings, and 2,096 validated links between semantics and patterns. The total token cost for building the graph is 144.1$, averaging 0.53$ per project. 4 https://docs.code4rena.com/bounties/bounty-criteria Table 1: Summary of our evaluation dataset AuditEval. ProjectIDContractsLinesHighMedium Ramses Exchange 4477360502 Kleidi 4559291603 LoopFi 45628827825 SecondSwap 46271823320 Lambo.win 4745967410 Flex Perpetuals 4806164202 Next Generation 484683313 Silo Finance 48514377206 Liquid Ron 486672512 IQ AI 4877147913 THORWallet 494232621 Nudge.xyz 4963126304 Sum-100276291461 4.1.2 Evaluation Dataset. To avoid data leakage, we collect public Solidity audit contests on Code4rena [4] that occurred after the LLM knowledge cutoff date (30/09/2024) and apply the same filtering pro- cedure, removing all Quality Assurance records. This results in our evaluation dataset,AuditEval, summarized in Table 1. The dataset spans six months beyond the model knowledge cutoff, including 12 projects and 100 Solidity smart contracts with 14 high-severity and 61 medium-severity vulnerabilities, which we use to rigorously evaluate Knowdit’s performance. Dataset Difficulty: Our datasetAuditEvalis notably more chal- lenging due to its strict prevention of data leakage and the wide- spread adoption of LLM-based analysis and automated testing. For example, the widely used Web3Bugs dataset [51], constructed from Code4rena contests between April 2021 and June 2022, predates the introduction of automated analysis by Code4rena in around October 2022. By contrast, contests in our datasetAuditEvalhave all undergone at least one round of LLM-driven automated analysis before starting auditing. As a result, Web3Bugs averages 4.1 High vulnerabilities per project, whereasAuditEvalcontains only 1.2 High vulnerabilities per project on average, reflecting the increased difficulty of identifying these remaining high-severity issues. 4.1.3 Baseline Methods. We compare Knowdit against several open-source representative LLM-based baselines from different cat- egories. We exclude tools that rely on overly simplistic or general oracles or that require significant manual effort for end-to-end audit- ing, such as fine-tuning models or manually writing specifications. The selected baselines include: • PromFuzz [27] and PropertyGPT [28], which represent state-of- the-art approaches leveraging LLMs to generate invariants for smart contracts. PromFuzz wraps an unmodified ItyFuzz [34], while PropertyGPT uses symbolic execution to verify the gener- ated invariants. •LLMAudit [41] and GPTScan [36], which serve as representative pure LLM-assisted static auditing baselines. All baselines are run with their default settings. For LLMAudit, we execute both Targeted Analysis (TA) and Broad Analysis (BA) modes, and findings from both modes are counted together. Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. Table 2: Bug-finding results onAuditEval. GT corresponds to the ground truth and is reported as High/Medium. For all tools, we report their findings in format High/Medium/False Positives. We highlight the results that cover all findings. IDGTPropertyGPTPromFuzz LLMAudit GPTScanKnowdit 4470/20/0/00/0/00/1/00/0/00/2/0 4550/3 0/0/10/0/00/2/00/0/00/2/0 4562/50/0/00/0/01/2/00/0/42/3/1 4623/200/6/30/0/00/4/00/0/03/16/1 4744/100/0/00/0/01/0/00/0/04/7/0 4800/2 0/0/00/0/00/1/00/0/00/2/0 4841/3 0/0/00/0/00/0/30/0/01/2/0 4850/60/0/00/0/00/3/00/3/10/5/0 4861/20/0/00/0/01/1/00/0/01/1/0 4871/30/0/00/0/00/1/00/1/01/3/0 4942/10/0/00/0/01/2/00/0/02/1/0 4960/4 0/0/00/0/00/1/00/0/00/3/0 Sum14/610/6/40/0/04/18/30/4/514/47/2 4.1.4 End-to-End Evaluation Environment. We conduct end-to-end audits by running them directly on the projects inAuditEvalwith- out any manual setup, such as deploying contracts or crafting ora- cles. To ensure a consistent environment, we build a shared Docker image containing theAuditEvalprojects, the tools, and common dependencies, including the Solidity compiler, Foundry, and Hard- hat frameworks, so that each project can build successfully. Each tool runs in its own container with dedicated 32 CPU cores and 128 GiB of memory. Similar to other fuzzing-based approaches, Knowdit continuously explores possible findings as long as the token budget permits. For a fair economic comparison, all tools are allowed to consume up to $100 worth of tokens per project. 4.2 RQ1: Effectiveness on Evaluation Dataset For each project inAuditEval, we collect all audit findings pro- duced by Knowdit and the baseline methods and manually verify them against the ground-truth audit reports to determine their va- lidity following the same criteria as defined in Section 3.3.6. We report the number of correctly identified in-scope High and Medium vulnerabilities and the number of false positives. 4.2.1 Results. Table 2 presents the vulnerability detection results onAuditEval. The results show that Knowdit significantly out- performs all baselines in automated smart contract auditing, iden- tifying all 14 High vulnerabilities (100%) and 47 out of 61 Medium vulnerabilities (77.0%), with only 2 false positives (3.2%). In contrast, the baselines—except PromFuzz, which produces no reports—detect substantially fewer vulnerabilities (ranging from 4 to 22) while in- curring more false positives. Moreover, on four projects (ID-447, ID-480,ID-487, andID-494), Knowdit successfully identifies all ground-truth vulnerabilities without any false positives. These re- sults demonstrate that Knowdit achieves both high precision and high recall in automated smart contract auditing. Compared to Static Auditing Methods. GPTScan only discovers 4 Medium vulnerabilities and fails to yield any High vulnerabilities because GPTScan relies on fixed attacking scenario models. L- MAudit performs slightly better because it repeatedly reasons the potential vulnerabilities and self-validates the correctness. How- ever, both static methods report more false positives than Knowdit because they lack the concrete execution to confirm vulnerabilities. Compared to Invariant-based Testing Methods. The two invariant- based testing methods, PromFuzz and PropertyGPT , perform poorly on ourAuditEvaldataset. PromFuzz produces no findings, while PropertyGPT identifies only 6 Medium vulnerabilities with 3 false positives. Our manual analysis reveals that their limitations stem from insufficient modeling of complex auditing scenarios, especially the interactions between multiple contracts, which Knowdit is specifically designed to handle. For example, PropertyGPT relies on a symbolic execution engine that supports only single-contract verification, while PromFuzz initializes contracts with all-zero ad- dresses, causing any external calls to fail immediately. In practice, all projects inAuditEvalinvolve at least two contracts and often include rich inter-contract dependencies and interactions. Further inspection of the ground-truth findings shows that nearly 90% of vulnerabilities span multiple contracts. As a result, although these methods may generate relevant invariants, they struggle to produce valid proofs or exploits without substantial manual intervention. This limitation significantly hinders their scalability for end-to-end smart contract auditing. Moreover, the invariants used by these two methods are limited in scope and fail to capture diverse audit- ing scenarios, leading to low vulnerability detection rates. A more detailed discussion of the knowledge coverage of PropertyGPT and PromFuzz is provided in the analysis of RQ2 (Section 4.3). 4.2.2 Analysis. To further analyze the effectiveness of Knowdit, we examine its false positives and false negatives in detail. False Positives. Knowdit produces two false positives that are not filtered out by the validation process, despite the Knowledge Mapper retrieving relevant DeFi semantics and prior findings. Both cases share a common root cause. First, the Specification Generator fails to fully capture the project’s business logic when relying solely on source code, leading to inaccurate auditing specification and consequently invalid violations during fuzzing. Second, the Finding Reflector fails to correctly attribute these violations to problematic specifications given the available context and knowledge, result- ing in misclassification as valid findings. A potential mitigation is to enrich the knowledge graph with common anti-patterns and counterfactual examples, which could improve both specification generation and reflective reasoning. We leave this to future work. False Negatives. Although Knowdit demonstrates strong effec- tiveness onAuditEval, it still misses several Medium vulnerabilities. By examining both the knowledge graph and the execution traces of the agentic framework, we attribute these false negatives to two main causes: (i) Knowdit sometimes fails to generate a valid fuzzing harness within a reasonable number of attempts due to LLM hallucinations, even when the knowledge graph contains the relevant DeFi semantics and vulnerability patterns, and they are correctly retrieved. (i) Some vulnerabilities are highly specific to project implementation details [51], making it difficult for Knowdit to concretize abstract knowledge distilled from historical experi- ence into precise code-level behaviors (A more detailed analysis of knowledge coverage is provided in RQ2 (Section 4.3)). To mitigate these limitations, a potential direction is to incorporate human-in- the-loop support for lightweight refinement of auditing specifica- tions and fuzzing harnesses and learn more projects to have a more complete knowledge graph. Knowdit : Agentic Smart Contract Vulnerability Detection with Auditing Knowledge SummarizationConference acronym ’X, June 03–05, 2018, Woodstock, NY Table 3: Knowledge coverage results of all tools onAuditEval. GT indicates the High and Medium ground-truth vulnerabili- ties for each project. ID refers to the project ID in Table 1. Cnt and Cov refer to the count and the percentage ratio of the covered vulnerabilities. We highlight the results that cover all findings. IDGTPropertyGPTPromFuzzKnowdit-NKGKnowdit CntCovCntCovCntCovCntCov 4470/20/150%0/150%0/150%0/2100% 4550/30/3 100%0/00%0/00%0/3100% 4562/52/257%1/357%0/00%2/486% 4623/201/1361%0/00%1/735%3/1787% 4744/10 0/321%0/214%1/114%4/886% 4800/20/150%0/00%0/150%0/2100% 4841/30/00%0/00%0/00%1/275% 4850/60/233%0/117%0/00%0/583% 4861/21/167%0/00%1/167%1/167% 4871/3 0/250%0/00%0/125%1/3100% 4942/11/033%0/00%0/00%2/1100% 4960/40/125%0/00%0/00%0/4100% Sum14/615/2945%1/711%3/1220%14/52 88% 4.2.3 Case Study. We showcase a representative High vulnerability discovered by Knowdit in projectID-494. This project implements a two-part economic model: TITN is a bridgeable omnichain fun- gible token, while TGT allows users to convert it into TITN at a time-dependent ratio. Due to the cross-chain nature of TITN, the project falls into the Cross Chain category, enabling Knowdit to retrieve relevant knowledge from the graph, including the DeFi semantic Unified cross-chain swap-and-bridge and the vulnerabil- ity pattern Epoch bond accounting exceeds mintable inflation and silently drops payouts. Although the project does not explicitly involve inflation mech- anisms, Knowdit correctly captures the underlying risk: when bridging or converting tokens, exceeding system capacity may lead to silent loss of user funds. Guided by this knowledge, Knowdit generates an auditing specification that models an attack scenario where transfers exceed the limit, TGT_TO_EXCHANGE, and synthe- sizes a fuzzing harness to trigger and verify the violation. As a result, Knowdit reports a finding titled “MergeTgt.onTokenTransfer allows unlimited TGT deposits without enforcing TGT_TO_EXCHANGE cap”, which closely matches the ground-truth report title “MergeTgt has no handling if TGT_TO_EXCHANGE is exceeded during the exchange period”. This case demonstrates that Knowdit can precisely identify real-world vulnerabilities and generate concrete proofs of exploita- tion. 4.3 RQ2: Contribution of Knowledge Graph We evaluate the contribution of the constructed knowledge graph from two perspectives: its coverage of knowledge for the projects in theAuditEvaldataset and its impact on the effectiveness in discovering vulnerabilities. 4.3.1 Knowledge Coverage. We evaluate knowledge coverage through manual verification. For each project inAuditEval, we collect all auditing specifications generated by Knowdit during the auditing process and examine how many ground-truth vulnerabilities are covered by these specifications. We also analyze the invariants gen- erated by PropertyGPT and PromFuzz using the same methodology. Table 3 summarizes the results. Knowdit achieves full coverage for nearly half of the projects and attains an overall coverage of 88%, without missing any High vulnerabilities. In contrast, the best baseline, PropertyGPT , covers only 45% of vulnerabilities. We at- tribute this advantage to the knowledge graph, which captures a wide range of real-world vulnerability patterns. Limitations of PropertyGPT and PromFuzz. The low cover- age of PromFuzz stems from its design, which feeds only a single function into the LLM context, limiting its ability to reason about cross-function behaviors. PropertyGPT performs better by incorpo- rating high-quality human-written properties alongside full con- tract code. However, compared to Knowdit, it still requires manual effort to incorporate new specifications, and its specifications lack generalizability across projects, as evidenced by our evaluation. In addition, Knowdit establishes links between DeFi semantics and vulnerability patterns based on semantic relationships, rather than relying solely on text similarity as in PropertyGPT . 4.3.2 Ablation Comparison. To evaluate the contribution of the knowledge graph to auditing effectiveness, we implement a vari- ant of Knowdit, denoted as Knowdit-NKG, where no knowl- edge graph is provided to the agentic auditing loop. In this setting, auditing specifications are generated solely based on the LLM’s pre-trained knowledge. Using the same manual evaluation procedure, as presented in Table 3, Knowdit-NKG achieves only 20% knowledge coverage, which is significantly lower than that of Knowdit. In terms of end- to-end effectiveness, Knowdit-NKG successfully produces proofs of exploit for 3 High vulnerabilities and 11 Medium vulnerabilities, but incurs 6 false positives. Further analysis shows that these false positives are primarily caused by hallucinations in the generated auditing specifications. The performance gap between Knowdit and Knowdit-NKG demonstrates that the proposed knowledge graph substantially improves both the coverage and precision of auditing specifications. 4.4 RQ3: Application in Real-world Auditing To understand how Knowdit performs on real-world projects, we apply it to six new projects with the same knowledge graph. Knowdit is capable of finding 12 High vulnerabilities and 10 Medium vulnerabilities with no false positives. All findings have been ac- knowledged and fixed by project developers before deployment. We confirm that Knowdit already helps secure at least 2 million dollars in assets and several million dollars in trades for these projects [5,6]. Case Study. Figure 8 presents the core vulnerable code of one High vulnerability discovered by Knowdit in project X 5 , where retry_bridge is designed to retry a failed token bridging request but blindly trusts sender and amount from the function parameters instead of the saved states. An attacker can easily bypass the check by filling both addresses with their own to forge bridge requests, leading to an unauthorized transfer from the project wallet. Knowdit first classifies the project as Cross Chain due to the bridging behavior in Figure 8 and associates it with the DeFi se- mantic Multi-standard token bridging and the vulnerability pattern Cross-chain executor lacks tracking for failed batches, enabling re- peated replay of stale failing calls. Guided by this knowledge, the 5 Details are anonymized for peer review. Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. function retry_bridge( bytes32 refund_id, address sender, address recipient, uint amount) external if (msg.sender != sender && msg.sender != recipient) revert NotAuthorized(); start_refund(refund_id, wallet, recipient, amount); Figure 8: The vulnerable code in the real-world project X. 15255080 Budget (USD) 447 455 456 462 474 480 484 485 486 487 494 496 0%50%100%100% 100%100%100%100% 20%40%80%100% 74%84%100%100% 73%73%82%100% 0%50%100%100% 100%100%100%100% 60%60%60%100% 50%100%100%100% 50%50%100%100% 67%100%100%100% 67%100%100%100% 0.0 0.2 0.4 0.6 0.8 1.0 Progress Ratio Figure 9: Progress ratio under different token budgets, where each row corresponds to a project inAuditEval. The x-axis shows the budget levels (15$, 25$, 50$, and 80$), and each cell reports the fraction of findings discovered at that budget relative to the total number of findings revealed by Knowdit. LLM focuses on the retry logic for failed requests and generates a fuzzing harness that repeats failed bridge calls. Although the project tracks failed requests, it lacks proper access controls, which allows replay attacks on attacker-controlled addresses. 4.5 RQ4: Cost Analysis The heatmap in Figure 9 illustrates the cost of our agentic auditing for discovering vulnerabilities onAuditEval. Overall, Knowdit reaches saturation across all projects after consuming approxi- mately 80$ worth of tokens, although the budget is set to 100$ per project. Summing the token usage at the saturation point yields a total cost of 440.3$. For comparison, we also measure the total to- ken consumption of the baselines: GPTScan, PromFuzz, LLMAudit, and PropertyGPT consume 0.9$, 9.7$, 65.7$, and 67.8$, respectively. Although Knowdit consumes more tokens than the baselines, it remains significantly more cost-effective than manual auditing given its strong vulnerability detection capability. For example, the projectsID-447,ID-480,ID-487andID-494, on which Knowdit achieves full coverage of High and Medium vulnerabilities (Ta- ble 2), incurred a total of 198,000$ in audit contest rewards, whereas Knowdit requires only around 150$ worth of LLM tokens. More- over, Knowdit is the only approach among the tools that identifies all High vulnerabilities associated with liquidity-draining attacks. Such vulnerabilities can put tens of millions of dollars at risk [6]. Therefore, the cost of Knowdit is well justified by its ability to effectively detect critical vulnerabilities and protect user assets. 5 Discussion We highlight several actionable directions for improving the com- bination of knowledge extraction and LLM-based agents in smart contract auditing. Continuous Knowledge Evolution. An autonomous agent could periodically fetch the latest audit reports or related documentation to incrementally update the knowledge graph, keeping it up-to-date with emerging vulnerabilities and techniques. Integration of Procedural Knowledge. The current knowledge graph primarily captures conceptual and factual knowledge. Future work could incorporate procedural knowledge by distilling auditing skills or strategies from reports to guide specification generation and fuzzing harness synthesis more effectively. Token-Efficient Agent Architecture. Most token costs arise from the inclusion of project context in the repeated iterations of specifi- cation generation and harness synthesis. One potential improve- ment is to deploy a dedicated agent to first analyze a project and build a project-specific knowledge base for semantically code re- trieval [14,43,50]. Subagents could then access this knowledge on demand, reducing redundant token-intensive analysis. Human-in-the-Loop Refinement. While Knowdit demonstrates strong automated capabilities, lightweight human oversight could help resolve corner cases where LLM hallucinations or ambiguous specifications produce false positives or missed vulnerabilities. This hybrid approach could enhance reliability without substantially increasing cost. 6 Threats to Validity Internal Threats. Internal validity threats primarily arise from the randomness of LLM outputs and the subjectivity of human evaluation. For human evaluation, we involve multiple annotators, provide detailed guidelines, and measure inter-annotator agreement to ensure consistency and reduce bias. External Threats. External validity threats mainly stem from the limited exploration of model selection in our study. Our experiments are conducted using GPT-5.1 and GPT-5-mini, so the results may not generalize to other LLMs. Another threat comes from our evaluation dataset, which is collected from Code4rena, and the findings may not generalize to auditing results from other platforms. 7 Related Work LLM smart contract auditing. Recent work on LLM-based smart contract auditing has quickly moved from direct prompting over contract code to workflows grounded in richer audit evidence. Much of the early work[36,46] is purely the static approach and shows that LLMs can reason about ERC rules and business-logic vulnera- bilities directly from contract code, while several later works [16, 25,28,33,39] further ground the model in mined specifications, knowledge graphs, traces, or retrieved human-written properties instead of relying on raw source alone. More recent systems bring stronger feedback into the loop: LLM-SmartAudit [41] organizes the audit process as multi-agent collaboration, PromFuzz [27] uses LLM-generated bug-oriented analyses to guide functional bug de- tection, and SmartPoC [11] turns audit reports into executable and validated PoCs. Knowdit : Agentic Smart Contract Vulnerability Detection with Auditing Knowledge SummarizationConference acronym ’X, June 03–05, 2018, Woodstock, NY Retrieval-augmented reasoning for code and security tasks. Retrieval-augmented generation [23] has become a common way to compensate for the limited context and unstable recall of LLMs. In code intelligence, this trend appears in repository-level retrieval sys- tems such as RepoCoder [50] and Repoformer [43], while GraphRAG [14] suggests that graph-structured retrieval can support broader reason- ing over large corpora. The same idea is now widely used in security tasks, including PropertyGPT [28], Vul-RAG [13], VulInstruct [54], and CodeGuarder [26]. Knowdit at a high level also fetches context from a knowledge graph. However, the key difference is that the content retrieved by Knowdit is always semantic-aware instead of text similarity. Fuzzing and proof-of-concept generation. Fuzzing and PoC generation focus on turning vulnerability hypotheses into exe- cutable evidence. Recent LLM-based fuzzing work has started to generalize this process across domains [10,45,49]. In smart con- tracts, sFuzz [32], Smartian [12], ConFuzzius [37], ItyFuzz [34], VERITE [21], and Belobog [20] show the importance of stateful exploration and domain-specific constraints. PoC-oriented work further synthesizes exploits or validation scripts from known weak- nesses or reports [11,18,47,52]. Knowdit focuses on the workflow of extracting expert audit knowledge and applying it to new smart contract projects, while fuzzing is used to better explore the input space and validate PoC. 8 Conclusion We present Knowdit, a knowledge-driven, agentic framework for smart contract vulnerability detection. By constructing an auditing knowledge graph that links fine-grained DeFi semantics with recur- ring vulnerability patterns, and leveraging it through an iterative multi-agent auditing loop, Knowdit enables systematic, automated auditing and outperforms all baselines on our evaluation dataset. References [1] 2021. ERC-4626: Tokenized Vaults. https://eips.ethereum.org/EIPS/eip-4626. [2]2022. Implement or recommend mitigations for ERC4626 inflation attacks. https: //github.com/OpenZeppelin/openzeppelin-contracts/issues/3706. [3] 2025. Foundry. https://github.com/foundry-rs/foundry. [4] 2026. Code4rena. http://code4rena.com/. [5] 2026. Coingecko. https://w.coingecko.com/. Accessed on March, 2026. [6] 2026. DefiLlama. https://defillama.com/. Accessed on March, 2026. [7] Hayden Adams, Noah Zinsmeister, and Dan Robinson. 2020. Uniswap v2 Core. https://uniswap.org/whitepaper.pdf . [8]Hayden Adams, Noah Zinsmeister, Moody Salem, River Keefer, and Dan Robinson. 2021. Uniswap v3 core. https://app.uniswap.org/whitepaper-v3.pdf . [9]Priyanka Bose, Dipanjan Das, Yanju Chen, Yu Feng, Christopher Kruegel, and Giovanni Vigna. 2022. Sailfish: Vetting smart contract state-inconsistency bugs in seconds. In 2022 IEEE Symposium on Security and Privacy (SP). IEEE, 161–178. [10]Chuyang Chen, Brendan Dolan-Gavitt, and Zhiqiang Lin. 2025.ELFuzz: Efficient Input Generation via LLM-driven Synthesis Over Fuzzer Space. arXiv:2506.10323 [cs.CR] https://arxiv.org/abs/2506.10323 [11]Longfei Chen, Ruibin Yan, Taiyu Wong, Yiyang Chen, and Chao Zhang. 2025. SmartPoC: Generating Executable and Validated PoCs for Smart Contract Bug Reports. arXiv preprint arXiv:2511.12993 (2025). [12]Jaeseung Choi, Doyeon Kim, Soomin Kim, Gustavo Grieco, Alex Groce, and Sang Kil Cha. 2021. Smartian: Enhancing smart contract fuzzing with static and dynamic data-flow analyses. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 227–239. doi:10.1109/ASE51524. 2021.9678888 [13] Xueying Du, Geng Zheng, Kaixin Wang, Yi Zou, Yujia Wang, Wentai Deng, Jiayi Feng, Mingwei Liu, Bihuan Chen, Xin Peng, Tao Ma, and Yiling Lou. 2025. Vul- RAG: Enhancing LLM-based Vulnerability Detection via Knowledge-level RAG. arXiv:2406.11147 [cs.SE] https://arxiv.org/abs/2406.11147 [14]Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130 [cs.CL] https://arxiv.org/abs/2404.16130 [15]Michael Egorov and Curve Finance. 2021. Automatic market-making with dy- namic peg. https://classic.curve.finance/files/crypto-pools-paper.pdf . [16]Jiaqi Gao, Zijian Zhang, Yuqiang Sun, Ye Liu, Chengwei Liu, Han Liu, Yi Li, and Yang Liu. 2026. LogicScan: An LLM-driven Framework for Detecting Business Logic Vulnerabilities in Smart Contracts. arXiv:2602.03271 [cs.CR] https://arxiv. org/abs/2602.03271 [17] Arthur Gervais and Liyi Zhou. 2026. AI Agent Smart Contract Exploit Generation. arXiv:2507.05558 [cs.CR] https://arxiv.org/abs/2507.05558 [18]David Jin, Qian Fu, and Yuekang Li. 2025.Good News for Script Kid- dies? Evaluating Large Language Models for Automated Exploit Generation. arXiv:2505.01065 [cs.CR] https://arxiv.org/abs/2505.01065 [19]Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213. [20]Ziqiao Kong, Wanxu Xia, Zhengwei Li, Yi Lu, Pan Li, Liqun Yang, Yang Liu, Xiapu Luo, and Shaohua Li. 2026. Belobog: Move Language Fuzzing Framework For Real-World Smart Contracts. arXiv:2512.02918 [cs.CR] https://arxiv.org/abs/ 2512.02918 [21]Ziqiao Kong, Cen Zhang, Maoyi Xie, Ming Hu, Yue Xue, Ye Liu, Haijun Wang, and Yang Liu. 2025. Smart Contract Fuzzing Towards Profitable Vulnerabilities. Proceedings of the ACM on Software Engineering 2, FSE (2025), 153–175. [22]Dharshan Kumaran, Demis Hassabis, and James L McClelland. 2016. What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends in cognitive sciences 20, 7 (2016), 512–534. [23]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv:2005.11401 [cs.CL] https://arxiv.org/abs/ 2005.11401 [24] Kaixuan Li, Yue Xue, Sen Chen, Han Liu, Kairan Sun, Ming Hu, Haijun Wang, Yang Liu, and Yixiang Chen. 2024. Static application security testing (sast) tools for smart contracts: How far are we? Proceedings of the ACM on Software Engineering 1, FSE (2024), 1447–1470. [25]Xiaoqi Li, Hailu Kuang, Wenkai Li, Zongwei Li, and Shipeng Ye. 2025. CKG-LLM: LLM-Assisted Detection of Smart Contract Access Control Vulnerabilities Based on Knowledge Graphs. arXiv:2512.06846 [cs.CR] https://arxiv.org/abs/2512.06846 [26]Bo Lin, Shangwen Wang, Yihao Qin, Liqian Chen, and Xiaoguang Mao. 2025. Give LLMs a Security Course: Securing Retrieval-Augmented Code Generation via Knowledge Injection. arXiv:2504.16429 [cs.CR] https://arxiv.org/abs/2504.16429 [27]Xingshuang Lin, Qinge Xie, Binbin Zhao, Yuan Tian, Saman Zonouz, Na Ruan, Jiliang Li, Raheem Beyah, and Shouling Ji. 2025. PROMFUZZ: Leveraging LLM- Driven and Bug-Oriented Composite Analysis for Detecting Functional Bugs in Smart Contracts. arXiv preprint arXiv:2503.23718 (2025). [28] Ye Liu, Yue Xue, Daoyuan Wu, Yuqiang Sun, Yi Li, Miaolei Shi, and Yang Liu. 2024. Propertygpt: Llm-driven formal verification of smart contracts through retrieval-augmented property generation. arXiv preprint arXiv:2405.02580 (2024). [29]Loi Luu, Duc-Hiep Chu, Hrishi Olickel, Prateek Saxena, and Aquinas Hobor. 2016. Making Smart Contracts Smarter. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (Vienna, Austria) (CCS ’16). Association for Computing Machinery, New York, NY, USA, 254–269. doi:10. 1145/2976749.2978309 [30] Wei Ma, Daoyuan Wu, Yuqiang Sun, Tianwen Wang, Shangqing Liu, Jian Zhang, Yue Xue, and Yang Liu. 2025. Combining fine-tuning and llm-based agents for intuitive smart contract auditing with justifications. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). IEEE, 1742–1754. [31]Ran Mo, Haopeng Song, Wei Ding, and Chaochao Wu. 2025. Code Cloning in Solidity Smart Contracts: Prevalence, Evolution, and Impact on Development. IEEE Press, 3060–3071. https://doi.org/10.1109/ICSE55347.2025.00123 [32]Tai D. Nguyen, Long H. Pham, Jun Sun, Yun Lin, and Quang Tran Minh. 2020.sFuzz: An Efficient Adaptive Fuzzer for Solidity Smart Contracts. arXiv:2004.08563 [cs.SE] https://arxiv.org/abs/2004.08563 [33]Chenyang Peng, Haijun Wang, Yin Wu, Hao Wu, Ming Fan, Yitao Zhao, and Ting Liu. 2025. ETrace: Event-Driven Vulnerability Detection in Smart Contracts via LLM-Based Trace Analysis. In Proceedings of the 16th International Conference on Internetware (Internetware 2025). ACM, 533–536. doi:10.1145/3755881.3755934 [34]Chaofan Shou, Shangyin Tan, and Koushik Sen. 2023. Ityfuzz: Snapshot-based fuzzer for smart contract. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 322–333. doi:10.1145/3597926. 3598059 [35]Kairan Sun, Zhengzi Xu, Chengwei Liu, Kaixuan Li, and Yang Liu. 2023. Demys- tifying the composition and code reuse in solidity smart contracts. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 796–807. [36]Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Haijun Wang, Zhengzi Xu, Xiaofei Xie, and Yang Liu. 2024. Gptscan: Detecting logic vulnerabilities in smart contracts by combining gpt with program analysis. In Proceedings of the IEEE/ACM 46th Conference acronym ’X, June 03–05, 2018, Woodstock, NYTrovato et al. international conference on software engineering. 1–13. [37]Christof Ferreira Torres, Antonio Ken Iannillo, Arthur Gervais, and Radu State. 2021. ConFuzzius: A Data Dependency-Aware Hybrid Fuzzer for Smart Contracts. arXiv:2005.12156 [cs.CR] https://arxiv.org/abs/2005.12156 [38]Justin Wang, Andreas Bigger, Xiaohai Xu, Justin W Lin, Andy Applebaum, Tejal Patwardhan, Alpin Yukseloglu, and Olivia Watkins. 2026. EVMbench: Evaluating AI Agents on Smart Contract Security. (2026). [39]Sally Junsong Wang, Kexin Pei, and Junfeng Yang. 2024. Smartinv: Multimodal learning for smart contract invariant inference. In 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2217–2235. [40]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824–24837. [41]Zhiyuan Wei, Jing Sun, Yuqiang Sun, Ye Liu, Daoyuan Wu, Zijian Zhang, Xianhao Zhang, Meng Li, Yang Liu, Chunmiao Li, et al.2025. Advanced smart contract vulnerability detection via llm-powered multi-agent systems. IEEE Transactions on Software Engineering (2025). [42]Dr Gavin Wood. 2025. ETHEREUM: A SECURE DECENTRALISED GENER- ALISED TRANSACTION LEDGER SHANGHAI VERSION. https://ethereum. github.io/yellowpaper/paper.pdf . (2025). [43] Di Wu, Wasi Uddin Ahmad, Dejiao Zhang, Murali Krishna Ramanathan, and Xiaofei Ma. 2024. Repoformer: Selective Retrieval for Repository-Level Code Completion. arXiv:2403.10059 [cs.SE] https://arxiv.org/abs/2403.10059 [44]Siwei Wu, Zhou Yu, Dabao Wang, Yajin Zhou, Lei Wu, Haoyu Wang, and Xingliang Yuan. 2023. Defiranger: Detecting defi price manipulation attacks. IEEE Transactions on Dependable and Secure Computing 21, 4 (2023), 4147–4161. [45]Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Fuzz4All: Universal Fuzzing with Large Language Models. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE ’24). ACM, 1–13. doi:10.1145/3597503.3639121 [46] Shihao Xia, Shuai Shao, Mengting He, Tingting Yu, Linhai Song, and Yiy- ing Zhang. 2024.AuditGPT: Auditing Smart Contracts with ChatGPT. arXiv:2404.04306 [cs.CR] https://arxiv.org/abs/2404.04306 [47] Zeke Xiao, Qin Wang, Yuekang Li, and Shiping Chen. 2025. Prompt to pwn: Automated exploit generation for smart contracts. arXiv preprint arXiv:2508.01371 (2025). [48]Maoyi Xie, Ming Hu, Ziqiao Kong, Cen Zhang, Yebo Feng, Haijun Wang, Yue Xue, Hao Zhang, Ye Liu, and Yang Liu. 2024. Defort: Automatic detection and analysis of price manipulation attacks in defi applications. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 402–414. [49]Hanxiang Xu, Yanjie Zhao, and Haoyu Wang. 2025. Directed Greybox Fuzzing via Large Language Model. arXiv:2505.03425 [cs.CR] https://arxiv.org/abs/2505. 03425 [50] Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. arXiv:2303.12570 [cs.CL] https://arxiv.org/abs/2303.12570 [51]Zhuo Zhang, Brian Zhang, Wen Xu, and Zhiqiang Lin. 2023. Demystifying ex- ploitable bugs in smart contracts. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 615–627. doi:10.1109/ICSE48619.2023.00061 [52]Mengyao Zhao, Kaixuan Li, Lyuye Zhang, Wenjing Dang, Chenggong Ding, Sen Chen, and Zheli Liu. 2025. A Systematic Study on Generating Web Vulnerability Proof-of-Concepts Using Large Language Models. arXiv:2510.10148 [cs.SE] https://arxiv.org/abs/2510.10148 [53]Liyi Zhou, Xihan Xiong, Jens Ernstberger, Stefanos Chaliasos, Zhipeng Wang, Ye Wang, Kaihua Qin, Roger Wattenhofer, Dawn Song, and Arthur Gervais. 2023. Sok: Decentralized finance (defi) attacks. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2444–2461. doi:10.1145/3558535.3559780 [54]Hao Zhu, Jia Li, Cuiyun Gao, Jiaru Qian, Yihong Dong, Huanyu Liu, Lecheng Wang, Ziliang Wang, Xiaolong Hu, and Ge Li. 2025. Specification-Guided Vul- nerability Detection with Large Language Models. arXiv:2511.04014 [cs.SE] https://arxiv.org/abs/2511.04014 Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009