Paper deep dive
CryptoFlowBench: Benchmarking Multi-Hop Reasoning on Blockchains for Cryptocurrency Fund Tracing
Runang He, Tongya Zheng, Jing Cao, Yucheng Ying, Shunyu Liu, Bingde Hu, Yang Gao, Zeyuan Hu, Jianrong Zhang, Mingli Song
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Pseudonymous accounts and complex smart-contract ecosystems substantially hinder the tracing of illicit cryptocurrency fund flows. Existing Graph Neural Networks (GNNs) have achieved promising performance in blockchain fraud detection, yet their static node- or edge-level predictions do not naturally generalize to multi-hop fund tracing. Meanwhile, existing tracing methods mainly rely on predefined heuristics to capture multi-hop relationships, limiting their adaptive reasoning capabilities. To bridge this gap, we introduce CryptoFlowBench, a benchmark constructed from authoritative reports on cryptocurrency-related illicit incidents, with four tasks spanning local to multi-hop fund tracing. These tasks pose substantial challenges to existing learning- and heuristic-based methods, particularly in complex multi-hop scenarios. We therefore propose ChainSleuth, a graph-tool-augmented agent that iteratively reasons, invokes graph tools, and updates its tracing trajectory, enabling adaptive information acquisition and recovery from erroneous paths. Experiments show that no baseline consistently dominates all four tasks, revealing substantial gaps between local fraud detection and multi-hop fund tracing. ChainSleuth achieves notable improvements on multi-hop tasks, demonstrating the potential of agentic graph reasoning for cryptocurrency fund investigation.
Tags
Links
- Source: https://arxiv.org/abs/2610.16449v1
- Canonical: https://arxiv.org/abs/2610.16449v1
Trouble viewing inline? Open PDF directly →
Full Text
48,748 characters extracted from source content.
Expand or collapse full text
CryptoFlowBench: Benchmarking Multi-Hop Reasoning on Blockchains for Cryptocurrency Fund Tracing Runang He Tongya Zheng †thanks: Corresponding Author Jing Cao Yucheng Ying Shunyu Liu Bingde Hu Yang Gao Zeyuan Hu Jianrong Zhang Mingli Song Abstract Pseudonymous accounts and complex smart-contract ecosystems substantially hinder the tracing of illicit cryptocurrency fund flows. Existing Graph Neural Networks (GNNs) have achieved promising performance in blockchain fraud detection, yet their static node- or edge-level predictions do not naturally generalize to multi-hop fund tracing. Meanwhile, existing tracing methods mainly rely on predefined heuristics to capture multi-hop relationships, limiting their adaptive reasoning capabilities. To bridge this gap, we introduce CryptoFlowBench, a benchmark constructed from authoritative reports on cryptocurrency-related illicit incidents, with four tasks spanning local to multi-hop fund tracing. These tasks pose substantial challenges to existing learning- and heuristic-based methods, particularly in complex multi-hop scenarios. We therefore propose ChainSleuth, a graph-tool-augmented agent that iteratively reasons, invokes graph tools, and updates its tracing trajectory, enabling adaptive information acquisition and recovery from erroneous paths. Experiments show that no baseline consistently dominates all four tasks, revealing substantial gaps between local fraud detection and multi-hop fund tracing. ChainSleuth achieves notable improvements on multi-hop tasks, demonstrating the potential of agentic graph reasoning for cryptocurrency fund investigation. 1State Key Laboratory of Blockchain and Data Security, Zhejiang University 2Hangzhou City University 3Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security 4China Mobile Group Zhejiang Co., Ltd 5China Unicom Digital Technology Co., Ltd herunang@zju.edu.cn, doujiang_zheng@163.com, jingcao@hzcu.edu.cn, yingyu@zju.edu.cn, shunyu.liu.cs@gmail.com, tonyhu@zju.edu.cn, roygao@zju.edu.cn, huzeyuan@zj.chinamobile.com, zhangjr67@chinaunicom.cn, brooksong@zju.edu.cn Code — https://github.com/LuoXishuang0712/CryptoFlowBench Introduction Figure 1: Comparison of fund-tracing behaviors on a case-centered transaction graph. Red edges in (a) show the verified suspicious flow. The machine-learning/GNN-based method in (b) cannot recover after selecting an irrelevant branch, shown as gray edges, while the rule-based method in (c) over-expands under a fixed propagation policy. ChainSleuth in (d) retrieves on-chain evidence, revises earlier decisions, and suppresses unsupported expansions. Public blockchains provide transparent and append-only transaction records without directly binding accounts to real-world identities. While supporting digital payments and decentralized finance, this infrastructure can also be exploited to steal, conceal, and launder illicit funds. According to the FBI’s 2025 Internet Crime Report, cryptocurrency was involved in approximately $11 billion of nearly $20 billion in reported losses (Federal Bureau of Investigation 2026). To analyze such illicit activities, on-chain transfers can be represented as large, temporal, and attributed money-flow graphs (MFGs), where nodes correspond to addresses or contracts and directed edges represent asset transfers (Turner et al. 2020; Qi et al. 2023). Within these graphs, some nodes are associated with services such as decentralized exchanges and cross-chain bridges, introducing heterogeneous transaction patterns into the tracing process (Liu et al. 2021; Bartoletti et al. 2021; Cunjak Mataković 2022; Almeida et al. 2023). As shown in Figure 1(a), practical fund tracing aims to recover the suspicious flow from limited case evidence while excluding irrelevant transfers. Existing fund-tracing methods broadly follow two paradigms. Heuristic approaches propagate funds using deterministic taint-allocation rules or graph-based relevance and flow objectives (Möser et al. 2014; Tironsakkul et al. 2019; Wu et al. 2023; Lin et al. 2024). Learning-based approaches use classical classifiers or GNNs to prioritize suspicious transactions and addresses from structural and behavioral features (Weber et al. 2019; Shen et al. 2021; Huang et al. 2024). However, fixed propagation policies and independent local predictions provide limited support for revising earlier decisions, while practical tracing requires evidence-driven multi-hop reasoning and adaptive stopping. Supporting and evaluating such practical tracing requires addressing three key challenges. Challenge 1: Case-grounded evaluation. Large detection benchmarks mainly provide binary address or transaction labels, whereas tracing studies often use independently constructed case graphs with inconsistent scopes, labels, and protocols. Neither setting supports unified evaluation of complete investigations across heterogeneous incidents. Challenge 2: Error accumulation. Machine-learning and GNN-based methods generally make independent local predictions, allowing an early false positive to propagate through a multi-hop trajectory without a mechanism for later revision, causing error accumulated along the propagation path, as illustrated in Figure 1(b). Challenge 3: Adaptive exploration and boundary control. Heuristic-based tracing methods rely on predefined attribution or structural policies (Möser et al. 2014; Tironsakkul et al. 2019; Wu et al. 2023; Lin et al. 2024; Wu et al. 2024b). Such policies may expand through irrelevant but highly connected services, causing common on-chain service nodes to repeatedly divert the tracing process, particularly when no reliable seed address is available, as shown in Figure 1(c). To address Challenge 1 and make the sequential and boundary-control requirements in Challenges 2 and 3 measurable, we construct CryptoFlowBench, a case-grounded benchmark that aligns heterogeneous incident reports with locally verified transactions. Each incident is represented as an evidence graph containing address roles, tracing actions, path stages, hard negatives, evidential boundaries, and source provenance. We deterministically derive four complementary tasks under a unified protocol: transaction existence for factual verification; next-step action for seeded decisions among follow, inspect, stop, and ignore; path completion for multi-hop tracing under accumulated errors; and subgraph filtering for seed localization and tracing in distractor-rich graphs. These tasks separate factual grounding, local decisions, sequential reconstruction, and seedless investigation, while exposing over-expansion and boundary violations. To address Challenges 2 and 3, we propose ChainSleuth, an evidence-grounded, tool-augmented LLM agent. Within a bounded ReAct loop (Yao et al. 2022), ChainSleuth maintains an evolving investigation state, selects explicit tracing actions, and retrieves transaction-level evidence through external tools. New observations can be used to revisit earlier hypotheses, suppress unsupported branches, and stop at evidential boundaries, as illustrated in Figure 1(d). Experiments show that strong local prediction does not necessarily translate into effective end-to-end tracing. Although conventional classifiers remain competitive on single-step decisions, ChainSleuth performs better on Path and Subgraph tasks, improving Path F1 by 70.0%, reducing over-expansion by 26.6%, and improving predicted-seed F1 by 21.4% over the strongest baseline for each metric. Our main contributions are summarized as follows: • CryptoFlowBench: a case-grounded benchmark for Ethereum fund-flow investigation. We align heterogeneous incident reports with locally verified transactions to construct case-centered evidence graphs that summarizing on-chain incidents. Based on 22 incidents across five types, the benchmark derives four complementary tasks under a unified protocol, covering factual verification, seeded local decisions, multi-hop path reconstruction, and seedless suspicious-region discovery. • ChainSleuth: an evidence-grounded, tool-augmented LLM agent. ChainSleuth maintains an evolving investigation state, selects explicit tracing actions, and retrieves auditable on-chain evidence through external tools. It can revisit earlier hypotheses as new evidence is acquired, thereby correcting accumulated errors, suppressing unsupported branches, and controlling tracing boundaries. • A comprehensive, subtask-specific evaluation. We compare ChainSleuth with task-appropriate heuristic-based, machine-learning, GNN, and LLM baselines under matched case splits and tracing budgets. On the core tracing tasks, ChainSleuth improves Path F1 by 70.0%, reduces over-expansion by 26.6%, and improves predicted-seed F1 by 21.4% over the strongest specialized baseline for each metric. Related Work Cryptocurrency Illicit-Activity Detection Existing cryptocurrency illicit-activity detection methods can be broadly grouped into several paradigms (Fouzy et al. 2026). Heuristic-based methods include rule-based approaches using predefined attribution policies and graph-based approaches using transaction topology for candidate prioritization; although interpretable, they are limited by fixed assumptions or scalability on large, densely connected graphs. Machine-learning methods, such as logistic regression, random forests, and XGBoost (Weber et al. 2019; Elmougy and Liu 2023), are generally efficient but depend heavily on handcrafted features and capture relational dependencies only indirectly. GNN-based methods, including GCN (Weber et al. 2019), I2BGNN (Shen et al. 2021), and PEAE-GNN (Huang et al. 2024), model complex account interactions more directly, but their computational cost and limited explainability hinder evidence-sensitive forensic use. General graph anomaly detection methods, such as H2GCN (Zhu et al. 2020), PMP (Zhuo et al. 2024), ConsisGAD (Chen et al. 2024), UNPrompt (Niu et al. 2024), ARC (Liu et al. 2024), and AnomalyGFM (Qiao et al. 2025), are not blockchain-specific but offer mechanisms for handling heterophily, label scarcity, and class imbalance. Their effectiveness has also been demonstrated in blockchain anomaly detection (He et al. 2026), although they remain focused on node-level detection rather than evidence-grounded, multi-hop fund tracing. Cryptocurrency Fund-Flow Tracing Early cryptocurrency tracing relied on deterministic taint-propagation rules, including Poison, Haircut, FIFO, LIFO, and TIHO, to associate downstream Bitcoin outputs with known illicit funds (Möser et al. 2014; Tironsakkul et al. 2019). Although transparent and reproducible, these methods depend on fixed value-allocation assumptions and can produce substantially different tracing results for the same source transaction. Learning- and graph-based approaches subsequently introduced more flexible prioritization. The Elliptic study applied classical classifiers and graph convolutional network (GCN) to illicit Bitcoin transaction detection (Weber et al. 2019), while TRacer modeled tracing in account-based blockchains as relevance-guided search over directed, temporal, weighted, and multi-relational graphs (Wu et al. 2023). DenseFlow and DenseFlow+ further combined suspicious-group discovery with Ethereum asset-flow reconstruction using dense-subgraph and flow-based objectives (Lin et al. 2024; Wu et al. 2024b). These methods advance transaction classification, account prioritization, and path reconstruction, but do not explicitly model evidence-grounded investigative decisions or revision after new observations. LLM Agents for Blockchain Security and Analysis LLM agents extend static language-model generation by interleaving reasoning, tool use, and environmental observations, enabling adaptive multi-step decision making (Yao et al. 2022; Paranjape et al. 2023). Recent studies have applied this agentic paradigm to external environments such as web navigation (Zhou et al. 2024; Lu et al. 2025), graphical user interfaces (Xie et al. 2024; Zhai et al. 2025), and structured graphs (Jin et al. 2024; Ying et al. 2026), where agents iteratively gather evidence and refine their actions. Such capabilities are well suited to on-chain analysis, which requires querying large transaction graphs and verifying intermediate decisions with external evidence. LLM agents have also been explored for smart-contract auditing, vulnerability detection, program analysis, and automated repair (He et al. 2024; Wei et al. 2024). However, existing studies largely focus on contract code rather than evidence-grounded, multi-step fund-flow tracing. Preliminaries Blockchain Ledger An account-based blockchain is an append-only block sequence ℒ=(B0,…,BL)L=(B_0,…,B_L). Within a block window W=[bmin,bmax]W=[b_ ,b_ ], each indexed transfer is represented as xi=(hi,bi,ti,si,di,ai,mi,κi)∈W,x_i=(h_i,b_i,t_i,s_i,d_i,a_i,m_i, _i) _W, where hih_i, bib_i, and tit_i denote the transaction hash, block number, and timestamp; sis_i and did_i denote the source and destination addresses; and aia_i, mim_i, and κi _i denote the asset identifier, amount, and transfer type. The indexed transfer set is W=xi∣bmin≤bi≤bmax.X_W=\x_i b_ ≤ b_i≤ b_ \. Exact transaction-existence claims are made only with respect to the indexed chain, block window, and supported transfer types. Transaction-Graph Construction Raw transfers are organized into a directed address-relation graph whose nodes are addresses and whose relation edges retain the underlying transfer records. For each ordered address pair (u,v)(u,v), define uv=xi∈W∣si=u,di=v.X_uv=\x_i _W s_i=u,\ d_i=v\. When uv≠∅X_uv≠ , euv=Agg(uv)e_uv=Agg(X_uv) preserves transaction identifiers and transfer metadata, with amounts aggregated only among records sharing the same asset identifier and transfer type. The resulting graph is ΦW(ℒ)=W=(VW,EW), _W(L)=G_W=(V_W,E_W), where VW=si,di∣xi∈W,EW=euv∣uv≠∅.V_W=\s_i,d_i x_i _W\, E_W=\e_uv _uv≠ \. Fund-Flow Tracing Given WG_W, public incident evidence xcpubx_c^pub, an optional seed set S0S_0, and exploration budget B, fund-flow tracing returns Trace(W,xcpub,S0,)=(S^,E^,^),Trace(G_W,x_c^pub,S_0;B)=( S, E, P), where S S is the supplied or inferred seed set, E E is the selected tracing subgraph, and P contains directed paths whose edges are contained in E E. For a hidden verified reference graph c∗=(Vc∗,Ec∗)G_c^*=(V_c^*,E_c^*), the objective is to recover relevant edges under B while limiting irrelevant expansion E^∖Ec∗ E E_c^*. The reference graph represents the verified evidence for case c, rather than an exhaustive reconstruction of all illicit activity. CryptoFlowBench Case Primary source Type 01 Ronin U.S. Treasury Credential/ UI compromise 02 Harmony Harmony 03 CoinEx CoinEx 04 HTX HECO Bridge HTX 05 Stake.com Stake.com 06 WazirX WazirX 07 Radiant Capital Radiant Capital 08 PlayDapp PlayDapp 09 Bybit Bybit 10 Wormhole CertiK Protocol exploit 11 Nomad Bridge Nomad 12 BNB Bridge BNB Chain 13 Euler Finance Euler Finance 14 Curve Vyper Pools Vyper 15 KyberSwap Elastic KyberSwap 16 Cetus Protocol Sui Foundation 17 Prisma Finance Prisma Finance 18 Penpie Penpie 19 Dough Finance Dough Finance 20 UwU Lend SlowMist Oracle manipulation 21 Munchables Halborn Backdoor 22 Atomic Wallet Atomic Wallet Wallet compromise Table 1: Incident types and primary sources of the benchmark cases. Figure 2: Overview of the benchmark construction pipeline. Public incident reports are consolidated into structured case summaries, aligned with indexed on-chain transactions, and converted into grounded question-answering tasks with task-specific verification. Where “Trans.” indicates “Transaction”. We propose CryptoFlowBench, a benchmark with four complementary fund-tracing tasks: transaction existence, next-step action, path completion, and subgraph filtering. They evaluate factual verification, local tracing decisions, sequential reasoning under accumulated errors, and seedless subgraph-level tracing, respectively. Figure 2 summarizes the benchmark construction pipeline. Let =cii=1NC=\c_i\_i=1^N denote publicly documented cryptocurrency incidents. We use LLM-driven agents colA_col and sumA_sum for report collection and structured summarization, while graph alignment, task generation, and verification are deterministic. For each incident c, the agents collect a document set cD_c and normalize its claims into a structured summary ScS_c. Subsequent stages construct a report-evidence graph ℋcH_c, align it with a bounded on-chain graph c(k)G_c^(k), and generate task instances cQ_c. According to this pipeline, we collect 22 incidents across five incident types, as shown in Table 1. Authoritative Reports Integration Existing cryptocurrency datasets mainly support binary detection and provide limited case context that can be grounded in on-chain evidence. This mismatch hinders the evaluation of practical tracing systems, especially LLM agents that require incident context and verifiable intermediate evidence. We therefore design an authoritative report collection and integration pipeline. We first collect several on-chain incidents and utilize a web-enabled LLM agent to collect online reports. For each incident c, the collection agent produces c=col(c,θcol,col),D_c=A_col(c; _col,T_col), from public sources, including incident disclosures, technical postmortems, security analyses, and blockchain explorer pages. To further reduce the randomness of the agent, we repeat the collection process for multiple passes, which is noted as cpD_c^p, where p indicates the index of pass. Then, to verify and gather reports, we utilize a summarization agent to review all reports Sc=sum((c1,…,cp),θsum,sum).S_c=A_sum((D_c^1,…,D_c^p); _sum,T_sum). The summarization agent independently verifies and distills the reports, and all sources are classified according to provenance priority. On-chain Case Graph Construction To align reports with on-chain evidence, we construct a case graph that maps report relations into on-chain transactions, ensuring all the evidence is case-grounded. We first deterministically transform the structured summary ScS_c into a case-evidence specification ℋc=(VcH,EcH,ρc,πc),H_c=(V_c^H,E_c^H, _c, _c), where VcHV_c^H contains reported addresses, contracts, protocols, and service entities; EcHE_c^H contains reported transfers and stage-level relations; ρc _c records entity roles and tracing stages; and πc _c preserves source provenance, temporal constraints, transaction identifiers, and verification status. The evidence specification provides candidate anchors and alignment constraints, but does not itself assert that reported relations exist on chain. Then, we ground and extend the report-derived evidence using the local blockchain index, which is constructed by fetching transactions from a blockchain client and indexing according to the addresses. Given the max expand hop k, blockchain index ℐI, case evidence ℋcH_c, case block window WcW_c, and resource bounds B, we construct c(k)=Expandk(ℐ,ℋc,Wc,).G_c^(k)=Expand_k(I,H_c;W_c,B). During the expanding process, we first derive candidate anchors from explicitly reported entities, then retrieve their bounded k-hop neighborhoods from the local index. Absence from the bounded expansion is not treated as evidence of non-existence; exact negative instances are generated only from complete index queries. The resulting representation therefore combines report-derived semantics from ℋcH_c with locally verified transaction evidence from c(k)G_c^(k) through explicit alignment and provenance links. Cryptocurrency Fund Tracing Tasks Existing methods are typically evaluated on isolated objectives, such as node classification or fund-flow reconstruction, and therefore do not expose the full investigation process. To better match practical requirements, CryptoFlowBench defines four complementary tasks: transaction existence, next-step action, path completion, and subgraph filtering. First, given the aligned case evidence ℋcH_c and bounded on-chain graph c(k)G_c^(k), we jointly construct task instances and their oracle annotations through a deterministic, graph-first procedure. Each candidate relation e is assigned an evidence label ye∈y_e and an investigation action ue∈=,,,,u_e =\ follow, inspect, stop, ignore\, where follow denotes continuation along the reference fund flow, inspect marks relevant but insufficient evidence, stop identifies an evidential or operational boundary, and ignore denotes an irrelevant relation. This action space extends binary edge classification into an investigative decision interface, where stop and ignore explicitly represent the evidential boundary of a case. Then we generate four types of tasks for each labeled case graph c(k)G_c^(k): transaction existence (Trans), next-step action (Action), path completion (Path), and subgraph filtering (Subgraph). Trans is a standalone diagnostic that asks whether an exact transaction relation exists under explicit block-window and transfer-type constraints. Action evaluates one seeded local decision over the outgoing relations of a known current node. Path composes the same local decision interface over multiple hops, exposing error accumulation under teacher-forced and free-rollout settings. Subgraph removes the trusted starting node and evaluates seed localization followed by tracing within a distractor-rich subgraph. Let ℛ=Trans,Action,Path,Subgraph.R=\Trans,Action,Path,Subgraph\. The task set for case c is generated as c=⋃r∈ℛGenerater(ℋc,c(k),,,r),Q_c= _r Generate_r (H_c,G_c^(k);Y,U,B_r ), where rB_r specifies task-dependent limits on depth, candidate size, node expansion, and evidence access. Task Action, Path, and Subgraph deliberately share the same edge-decision semantics, while Action fixes the seed and evaluates one local state, Path fixes the seed and evaluates a sequential rollout, and Subgraph additionally requires the seed to be inferred. Every task q∈cq _c is separated into a public record xqpubx_q^pub and a disjoint oracle record yqoracley_q^oracle. The former contains only the task question, candidate set, permitted graph context, tool declarations, budget, and response schema. Answers, gold actions, case-path labels, and evaluation annotations remain oracle-only and will be removed from prompts, retrieval stores, and tool outputs. For a fixed case snapshot and configuration, candidate construction, ordering, oracle generation, and public-context sanitization are deterministic. Appendix.A presents representative benchmark cases to illustrate how incident reports are aligned with on-chain evidence and converted into case-centered graphs with tracing labels, evidential boundaries, and task instances. ChainSleuth Existing tracing methods rely on fixed propagation rules or independently scored local decisions, and therefore provide limited support for revising earlier hypotheses during multi-hop investigation. We propose ChainSleuth, a ReAct-based, retrieval-augmented LLM agent for evidence-grounded on-chain fund tracing (Yao et al. 2022). Through a local blockchain index, blockchain explorers, and optional pretrained detectors, ChainSleuth adaptively retrieves evidence and updates its tracing trajectory. The tool design is detailed in Appendix.B. For each public task state xqpubx_q^pub, ChainSleuth receives the task description, candidate set, response schema, and declarations of the allowed and required tools. During execution, ChainSleuth follows a bounded interaction loop. At iteration j, given the public task input and interaction history HjH_j, the model either issues an allowed tool call aj=πθ(Hj,xqpub),a_j= _θ(H_j,x_q^pub), receives a sanitized observation oj=(aj),o_j=T(a_j), and updates Hj+1=Hj∪aj,oj,H_j+1=H_j∪\a_j,o_j\, or terminates with a structured answer. Execution stops when the answer is produced, the evidence coverage is reached, no further progress is made, or the tool budget is exhausted. For tasks Action, Path, and Subgraph, ChainSleuth retrieves transaction-level evidence for public candidate relations and then predicts an action distribution πθ(u∣st,e),u∈, _θ(u s_t,e), u , for each candidate relation e at state sts_t. Only relations assigned follow with probability above the declared threshold may be expanded. Task Action performs this decision once, Path repeatedly applies it along a bounded trajectory, and Subgraph first estimates a seed distribution ρθ(v∣xqpub) _θ(v x_q^pub) before applying the same edge policy. After evidence collection terminates, a final synthesis step produces the required JSON response with tools disabled. Experiments We evaluate CryptoFlowBench at two levels. Task Trans and Action assess fundamental investigation capabilities, including exact tool invocation, evidence grounding, and local action selection. Because these diagnostic tasks do not capture full multi-hop tracing, their results are reported in Appendix.C. Task Path and Subgraph constitute the core tracing benchmark, evaluating sequential investigation under cumulative error and seedless suspicious-region discovery, respectively. For these tasks, we compare ChainSleuth with task-appropriate heuristic- and learning-based baselines under matched candidate nodes, case splits, and inference budgets. Experimental Settings Table 2: Task distribution by incident. Case Trans Action Path Subgraph Total 01 Ronin 658 11 7 13 689 02 Harmony 1,720 4 1 25 1,750 03 CoinEx 1,216 4 0 21 1,241 04 HTX HECO Bridge 220 14 5 13 252 05 Stake.com 148 10 2 13 173 06 WazirX 706 5 1 25 737 07 Radiant Capital 1,228 7 4 13 1,252 08 PlayDapp 1,688 5 1 25 1,719 09 Bybit 19,584 54 4 25 19,667 10 Wormhole 202 1 0 13 216 11 Nomad Bridge 522 5 1 17 545 12 BNB Bridge 1,594 1 0 13 1,608 13 Euler Finance 1,504 13 8 25 1,550 14 Curve Vyper Pools 440 8 3 15 466 15 KyberSwap Elastic 1,844 3 0 25 1,872 16 Cetus Protocol 662 3 0 13 678 17 Prisma Finance 466 6 3 25 500 18 Penpie 328 5 3 13 349 19 Dough Finance 1,610 5 2 25 1,642 20 UwU Lend 1,620 3 0 13 1,636 21 Munchables 238 7 2 13 260 22 Atomic Wallet 350 14 4 14 382 Total 38,548 188 51 397 39,184 Share (%) 98.4 0.5 0.1 1.0 100.0 Dataset and Splits We construct the benchmark from 22 publicly documented Ethereum incidents using the report collection, on-chain alignment, and deterministic task generation pipeline described above. Table 2 reports their distribution across incidents. To prevent case-specific information leakage, all learning-based methods are trained, validated, and tested using case-disjoint splits. Agent Models We evaluate ChainSleuth using Qwen3.6-27B (Qwen Team 2026), DeepSeek-V4-Flash (DS-V4-Flash), and DeepSeek-V4-Pro (DS-V4-Pro) (DeepSeek-AI 2026) as LLM. Baselines We compare ChainSleuth with three groups of task-appropriate baselines. We group heuristic baselines into predefined rule-based and graph-based tracing methods. Rule-based tracing. We adapt Poison, Haircut, and FIFO (Möser et al. 2014), together with LIFO and TIHO (Tironsakkul et al. 2019), to the Ethereum account-based setting. Taint is propagated separately by asset, and all methods use the same initial taint budget, candidate universe, and rollout limits. Graph-based tracing. We include TRacer (Wu et al. 2023) and adaptations inspired by DenseFlow and DenseFlow+ (Lin et al. 2024; Wu et al. 2024b). Because these methods are not originally designed for the present action interface, we map their graph scores to candidate-edge rankings and follow decisions without access to target-case labels. Learning-based classifiers. We train logistic regression (LR), random forest (RF), multilayer perceptron (MLP), GCN (Weber et al. 2019), and XGBoost (Elmougy and Liu 2023) using public candidate-edge features and case-disjoint training folds. We also utilize several blockchain oriented GNN-based methods, including I2BGNN (Shen et al. 2021), PEAE-GNN (Huang et al. 2024), and TokenScout (Wu et al. 2024a). All GNN-based methods are adapted to edge-level prediction because their original formulations classify transaction nodes rather than candidate address relations. A frozen XGBoost node classifier is shared across all edge methods for predicted-seed evaluation in the Subgraph task, the selection of node classifier will be discussed below. Evaluation Metrics All results use deterministic primary scorers and are macro-averaged across incidents. We report 95% confidence intervals in Appendix D. For the Path task, complete-path rate (Comp.) and Path F1 evaluate trajectory reconstruction, teacher-forced accuracy (TF Acc.) isolates local decision quality without accumulated rollout error, and over-expansion (Over-exp.) measures continuation beyond the verified boundary. For the Subgraph task, Path F1 using predicted seeds (Pred F1) evaluates the complete pipeline of seed identification and suspicious-region expansion, while Path F1 using oracle seeds (Oracle F1) removes seed identification errors and isolates propagation quality; path precision (Path P) and recall (Path R) further characterize the trade-off between noise suppression and suspicious-region coverage. For ChainSleuth, structured-output validity and tool-protocol pass rate are also reported to distinguish tracing performance from execution and protocol failures. Comparison Experiments Table 3: Path results on 51 tasks from 16 cases. ∗ indicates ChainSleuth and + indicates detector tool enabled. Method Comp. ↑ Path F1 ↑ TF Acc. ↑ Over-exp. ↓ Poison 0.3300.330 0.3450.345 0.3960.396 0.6440.644 Haircut 0.3300.330 0.3110.311 0.3960.396 0.6740.674 FIFO 0.4220.422 0.3400.340 0.5600.560 0.6280.628 LIFO 0.3010.301 0.3280.328 0.4760.476 0.6510.651 TIHO 0.3130.313 0.3070.307 0.6040.604 0.6460.646 TRacer 0.0870.087 0.1860.186 0.6430.643 0.1580.158 DenseFlow+ 0.2960.296 0.3150.315 0.5170.517 0.3320.332 LR 0.4260.426 0.4320.432 0.7180.718 0.5900.590 RF 0.3740.374 0.3790.379 0.8130.813 0.6030.603 MLP 0.2530.253 0.2120.212 0.8780.878 0.2700.270 GCN 0.3360.336 0.3000.300 0.7740.774 0.6050.605 I²BGNN 0.3050.305 0.2790.279 0.6230.623 0.4500.450 PEAE-GNN 0.2900.290 0.2670.267 0.5720.572 0.4090.409 TokenScout 0.3070.307 0.2860.286 0.6590.659 0.4140.414 XGB 0.2760.276 0.2790.279 0.854¯ 0.854 0.3250.325 DS-V4-Flash ∗ 0.0920.092 0.2020.202 0.2440.244 0.0970.097 DS-V4-Pro ∗ 0.5690.569 0.5410.541 0.5700.570 0.1850.185 Qwen3.6-27B ∗ 0.665¯ 0.665 0.7340.734 0.8240.824 0.116¯ 0.116 Qwen3.6-27B ∗+ 0.7600.760 0.628¯ 0.628 0.6030.603 0.2210.221 Table 4: Subgraph results on the valid-seed, positive-path subset of 22 tasks from 22 cases. ∗ indicates ChainSleuth and + indicates detector tool enabled. Method Pred F1 ↑ Oracle F1 ↑ Path P ↑ Path R ↑ Poison 0.1990.199 0.1670.167 0.2950.295 0.1800.180 Haircut 0.1990.199 0.1670.167 0.2950.295 0.1800.180 FIFO 0.2190.219 0.2280.228 0.3180.318 0.1900.190 LIFO 0.2170.217 0.1360.136 0.3790.379 0.1640.164 TIHO 0.2370.237 0.2280.228 0.3860.386 0.2020.202 TRacer 0.1090.109 0.1370.137 0.2200.220 0.0750.075 DenseFlow+ 0.0870.087 0.1260.126 0.1670.167 0.0630.063 LR 0.3030.303 0.565¯ 0.565 0.4730.473 0.2830.283 RF 0.309¯ 0.309 0.5880.588 0.5530.553 0.2520.252 MLP 0.1960.196 0.3860.386 0.3940.394 0.1450.145 GCN 0.1990.199 0.4230.423 0.2950.295 0.1800.180 I²BGNN 0.1760.176 0.3500.350 0.2650.265 0.1610.161 PEAE-GNN 0.1660.166 0.3830.383 0.2500.250 0.1530.153 TokenScout 0.1870.187 0.4000.400 0.2800.280 0.1700.170 XGB 0.2180.218 0.3910.391 0.485¯ 0.485 0.1480.148 DS-V4-Flash ∗ 0.0940.094 0.0940.094 0.0810.081 0.1270.127 DS-V4-Pro ∗ 0.2130.213 0.2040.204 0.1660.166 0.381¯ 0.381 Qwen3.6-27B ∗ 0.3750.375 0.3060.306 0.3140.314 0.5680.568 Qwen3.6-27B ∗+ 0.1820.182 0.2710.271 0.2230.223 0.1830.183 Table 5: Execution reliability of ChainSleuth on path completion. All the methods are ChainSleuth and + indicates detector tool enabled. Method Structure-valid rate ↑ Tool-protocol pass rate ↑ DS-V4-Flash 0.3180.318 0.0990.099 DS-V4-Pro 0.7800.780 0.2920.292 Qwen3.6-27B + 0.8020.802 0.7660.766 Qwen3.6-27B 1.0001.000 0.9790.979 Table 3 and 4 show the result of baselines and ChainSleuth on the Path and Subgraph tasks, respectively. ChainSleuth versus Specialized Tracing Methods ChainSleuth shows its clearest advantage on the practical tracing objectives that require multi-step coordination and control of irrelevant expansion. On the Path, Qwen3.6-27B achieves a complete-path rate of 0.665 and a Path F1 of 0.734, substantially outperforming the strongest conventional baselines. It also reduces over-expansion to 0.116, compared with 0.158 for the strongest baseline, indicating that ChainSleuth can revise local hypotheses and terminate irrelevant branches before their errors accumulate along the trajectory. A similar pattern appears on the Subgraph, where Qwen3.6-27B obtains the highest predicted-seed F1 of 0.375 and the highest path recall of 0.568. RF performs better when an oracle seed is supplied, achieving an oracle-seed F1 of 0.588 and a path precision of 0.553, but its predicted-seed F1 decreases to 0.309. These results indicate that specialized classifiers remain strong when the starting point is known, whereas ChainSleuth is more effective in the end-to-end setting that jointly requires seed discovery, evidence acquisition, and subsequent path expansion. Effect of LLM Backends The performance gap between Qwen3.6-27B and the DeepSeek-V4-Flash/Pro backends is closely associated with execution reliability rather than being attributable solely to their underlying reasoning capability. As shown in Table 5, on the Path task, Qwen3.6-27B achieves a structured-valid rate of 1.000 and a tool-protocol pass rate of 0.979, whereas DeepSeek-V4-Flash reaches only 0.318 and 0.099, respectively. DeepSeek-V4-Pro improves structured validity to 0.780 and tool-protocol compliance to 0.292, which is accompanied by substantially better tracing performance. The same ordering is observed on Subgraph, where the predicted-seed F1 rises from 0.094 for DeepSeek-V4-Flash to 0.213 for DeepSeek-V4-Pro, but remains below the 0.375 achieved by Qwen3.6-27B. The consistent relationship between protocol reliability and downstream tracing performance suggests that execution reliability is an important contributor to the observed backend gap. Detector as Tool We also evaluate the performance of ChainSleuth with detector as tool, as show in Table 3 and Table 4. In detail, we utilize LR, RF, and XGBoost as tools, as those methods can perform a decent result in the Action, Path, and Subgraph tasks, comparing in baselines. Adding the detector tool increases Path completeness from 0.665 to 0.760, but decreases Path F1 from 0.734 to 0.628 and increases over-expansion from 0.116 to 0.221. On the Subgraph task, it degrades all reported metrics, suggesting that detector scores may bias the agent toward locally plausible but globally inconsistent expansions. Seed-Detector Sensitivity Table 6: Case-disjoint comparison of candidate seed detectors under different illicit-node ratios in the training set. Train rate Method AP ↑ Hit@1 ↑ Hit@5 ↑ MRR ↑ 1% LR 0.651 0.546 0.955 0.746 RF 0.881 0.955 1.000 0.970 XGB 0.883 0.955 1.000 0.970 5% LR 0.659 0.546 0.955 0.735 RF 0.862 0.909 1.000 0.947 XGB 0.877 0.864 1.000 0.921 10% LR 0.688 0.500 1.000 0.750 RF 0.827 0.864 1.000 0.921 XGB 0.829 0.773 1.000 0.886 To ensure a fair comparison in Subgraph, all baselines share the same case-disjoint seed detector under identical public features and target-case splits. As shown in Table 6, RF and XGBoost achieve comparable ranking performance and both outperform LR, indicating that the selected detector is not an unusually favorable choice. We therefore use XGBoost for predicted-seed evaluation and report oracle-seed results separately to isolate propagation performance. Conclusion We introduce a case-grounded benchmark for Ethereum fund-flow tracing that unifies transaction verification, local decisions, multi-step path completion, and seedless subgraph filtering under strict public–oracle separation. We further develop ChainSleuth that retrieves auditable on-chain evidence and makes structured tracing decisions without requiring the full transaction graph in context. Experiments show that conventional methods remain competitive on single-step decisions, whereas ChainSleuth performs better on sequential and seed-uncertain tracing. These results highlight the potential of evidence-grounded agents for practical and reproducible cryptocurrency investigations. Social Impact. Cryptocurrency-enabled crime causes substantial financial harm, yet practical fund tracing remains difficult due to pseudonymous accounts and complex multi-hop transactions. We provide a new AI perspective by modeling tracing as a case-grounded, sequential investigation problem rather than static illicit-activity detection. CryptoFlowBench supports reproducible evaluation across factual verification, local decisions, path reconstruction, and seedless discovery, while ChainSleuth illustrates how tool-augmented agents can acquire evidence, revise earlier decisions, and limit irrelevant expansion. This work provides a clear foundation for follow-up research and future decision-support tools for cryptocurrency investigators. References Almeida et al. (2023) H. Almeida, P. Pinto, and A. F. Vilas A review on cryptocurrency transaction methods for money laundering. arXiv preprint arXiv:2311.17203. Cited by: Introduction. Bartoletti et al. (2021) M. Bartoletti, S. Lande, A. Loddo, L. Pompianu, and S. Serusi Cryptocurrency scams: analysis and perspectives. Ieee Access 9, p. 148353–148373. Cited by: Introduction. Chen et al. (2024) N. Chen, Z. Liu, B. Hooi, B. He, R. Fathony, J. Hu, and J. Chen Consistency training with learnable data augmentation for graph anomaly detection with limited supervision. In The twelfth international conference on learning representations, Cited by: Cryptocurrency Illicit-Activity Detection. Cunjak Mataković (2022) I. Cunjak Mataković Crypto-assets illicit activities: theoretical approach with empirical review. International e-journal of criminal sciences 17. Cited by: Introduction. DeepSeek-AI (2026) DeepSeek-AI DeepSeek-v4: towards highly efficient million-token context intelligence. Cited by: Agent Models. Elmougy and Liu (2023) Y. Elmougy and L. Liu Demystifying fraudulent transactions and illicit nodes in the bitcoin network for financial forensics. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining, p. 3979–3990. Cited by: Cryptocurrency Illicit-Activity Detection, Baselines. Federal Bureau of Investigation (2026) Federal Bureau of Investigation 2025 internet crime report. Technical report Internet Crime Complaint Center. Note: https://w.ic3.gov/AnnualReport/Reports/2025_IC3Report.pdfAccessed: 2026-07-22 External Links: Link Cited by: Introduction. Fouzy et al. (2026) N. Fouzy, M. Elhadef, H. Zarouk, A. Jawish, and K. Modugu Cryptocurrency money laundering techniques and ai-driven detection approaches. In 2026 International Conference on Integrated Intelligence and Cognitive Engineering (ICIICE), p. 1–6. Cited by: Cryptocurrency Illicit-Activity Detection. He et al. (2026) R. He, T. Zheng, H. Peng, Y. Wan, B. Hu, J. Chen, C. Jin, M. Song, and C. Wang Temporal motif-aware graph test-time adaptation for ood blockchain anomaly detection. arXiv preprint arXiv:2605.29526. Cited by: Cryptocurrency Illicit-Activity Detection. He et al. (2024) Z. He, Z. Li, S. Yang, H. Ye, A. Qiao, X. Zhang, X. Luo, and T. Chen Large language models for blockchain security: a systematic literature review. arXiv preprint arXiv:2403.14280. Cited by: LLM Agents for Blockchain Security and Analysis. Huang et al. (2024) H. Huang, X. Zhang, J. Wang, C. Gao, X. Li, R. Zhu, and Q. Ma PEAE-gnn: phishing detection on ethereum via augmentation ego-graph based on graph neural network. IEEE Transactions on Computational Social Systems 11 (3), p. 4326–4339. Cited by: Introduction, Cryptocurrency Illicit-Activity Detection, Baselines. Jin et al. (2024) B. Jin, C. Xie, J. Zhang, K. K. Roy, Y. Zhang, Z. Li, R. Li, X. Tang, S. Wang, Y. Meng, et al. Graph chain-of-thought: augmenting large language models by reasoning on graphs. In Findings of the Association for Computational Linguistics: ACL 2024, p. 163–184. Cited by: LLM Agents for Blockchain Security and Analysis. Lin et al. (2024) D. Lin, J. Wu, Y. Yu, Q. Fu, Z. Zheng, and C. Yang Denseflow: spotting cryptocurrency money laundering in ethereum transaction graphs. In Proceedings of the ACM Web Conference 2024, p. 4429–4438. Cited by: Introduction, Introduction, Cryptocurrency Fund-Flow Tracing, Baselines. Liu et al. (2021) X. F. Liu, X. Jiang, S. Liu, and C. K. Tse Knowledge discovery in cryptocurrency transactions: a survey. Ieee access 9, p. 37229–37254. Cited by: Introduction. Liu et al. (2024) Y. Liu, S. Li, Y. Zheng, Q. Chen, C. Zhang, and S. Pan Arc: a generalist graph anomaly detector with in-context learning. Advances in Neural Information Processing Systems 37, p. 50772–50804. Cited by: Cryptocurrency Illicit-Activity Detection. Lu et al. (2025) R. Lu, Z. Hou, Z. Wang, et al. Deepdive: advancing deep search agents with knowledge graphs and multi-turn rl. arXiv preprint arXiv:2509.10446. Cited by: LLM Agents for Blockchain Security and Analysis. Möser et al. (2014) M. Möser, R. Böhme, and D. Breuker Towards risk scoring of bitcoin transactions. In International conference on financial cryptography and data security, p. 16–32. Cited by: Introduction, Introduction, Cryptocurrency Fund-Flow Tracing, Baselines. Niu et al. (2024) C. Niu, H. Qiao, C. Chen, L. Chen, and G. Pang Zero-shot generalist graph anomaly detection with unified neighborhood prompts. arXiv preprint arXiv:2410.14886. Cited by: Cryptocurrency Illicit-Activity Detection. Paranjape et al. (2023) B. Paranjape, S. Lundberg, S. Singh, H. Hajishirzi, L. Zettlemoyer, and M. T. Ribeiro Art: automatic multi-step reasoning and tool-use for large language models. arXiv preprint arXiv:2303.09014. Cited by: LLM Agents for Blockchain Security and Analysis. Qi et al. (2023) Y. Qi, J. Wu, H. Xu, and M. Guizani Blockchain data mining with graph learning: a survey. IEEE transactions on pattern analysis and machine intelligence 46 (2), p. 729–748. Cited by: Introduction. Qiao et al. (2025) H. Qiao, C. Niu, L. Chen, and G. Pang Anomalygfm: graph foundation model for zero/few-shot anomaly detection. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 2326–2337. Cited by: Cryptocurrency Illicit-Activity Detection. Qwen Team (2026) Qwen Team Qwen3.6-27B: flagship-level coding in a 27B dense model. External Links: Link Cited by: Agent Models. Shen et al. (2021) J. Shen, J. Zhou, Y. Xie, S. Yu, and Q. Xuan Identity inference on blockchain using graph neural network. In International conference on blockchain and trustworthy systems, p. 3–17. Cited by: Introduction, Cryptocurrency Illicit-Activity Detection, Baselines. Tironsakkul et al. (2019) T. Tironsakkul, M. Maarek, A. Eross, and M. Just Probing the mystery of cryptocurrency theft: an investigation into methods for taint analysis. arXiv preprint arXiv:1906.05754. Cited by: Introduction, Introduction, Cryptocurrency Fund-Flow Tracing, Baselines. Turner et al. (2020) A. B. Turner, S. McCombie, and A. J. Uhlmann Analysis techniques for illicit bitcoin transactions. Frontiers in Computer Science 2, p. 600596. Cited by: Introduction. Weber et al. (2019) M. Weber, G. Domeniconi, J. Chen, D. K. I. Weidele, C. Bellei, T. Robinson, and C. E. Leiserson Anti-money laundering in bitcoin: experimenting with graph convolutional networks for financial forensics. arXiv preprint arXiv:1908.02591. Cited by: Introduction, Cryptocurrency Illicit-Activity Detection, Cryptocurrency Fund-Flow Tracing, Baselines. Wei et al. (2024) Z. Wei, J. Sun, Z. Zhang, X. Zhang, M. Li, and Z. Hou Llm-smartaudit: advanced smart contract vulnerability detection. arXiv preprint arXiv:2410.09381. Cited by: LLM Agents for Blockchain Security and Analysis. Wu et al. (2024a) C. Wu, J. Chen, Z. Zhao, K. He, G. Xu, Y. Wu, H. Wang, H. Li, Y. Liu, and Y. Xiang Tokenscout: early detection of ethereum scam tokens via temporal graph learning. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, p. 956–970. Cited by: Baselines. Wu et al. (2024b) J. Wu, Y. Yu, D. Lin, Z. Chen, and Z. Jiang From shadows to light: uncovering money laundering in ethereum transaction graphs. Available at SSRN 5076472. Cited by: Introduction, Cryptocurrency Fund-Flow Tracing, Baselines. Wu et al. (2023) Z. Wu, J. Liu, J. Wu, Z. Zheng, and T. Chen TRacer: scalable graph-based transaction tracing for account-based blockchain trading systems. IEEE Transactions on Information Forensics and Security 18, p. 2609–2621. Cited by: Introduction, Introduction, Cryptocurrency Fund-Flow Tracing, Baselines. Xie et al. (2024) T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, et al. Osworld: benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems 37, p. 52040–52094. Cited by: LLM Agents for Blockchain Security and Analysis. Yao et al. (2022) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao React: synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629. Cited by: Introduction, LLM Agents for Blockchain Security and Analysis, ChainSleuth. Ying et al. (2026) Y. Ying, W. Jiang, T. Zheng, Y. Wang, S. Liu, K. Chen, and M. Song GraphScout: empowering large language models with intrinsic exploration ability for agentic graph reasoning. arXiv preprint arXiv:2603.01410. Cited by: LLM Agents for Blockchain Security and Analysis. Zhai et al. (2025) Y. Zhai, S. Tao, C. Chen, A. Zou, Z. Chen, Q. Fu, S. Mai, L. Yu, J. Deng, Z. Cao, et al. Agentevolver: towards efficient self-evolving agent system. arXiv preprint arXiv:2511.10395. Cited by: LLM Agents for Blockchain Security and Analysis. Zhou et al. (2024) S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, et al. Webarena: a realistic web environment for building autonomous agents. In International Conference on Learning Representations, Vol. 2024, p. 15585–15606. Cited by: LLM Agents for Blockchain Security and Analysis. Zhu et al. (2020) J. Zhu, Y. Yan, L. Zhao, M. Heimann, L. Akoglu, and D. Koutra Beyond homophily in graph neural networks: current limitations and effective designs. Advances in neural information processing systems 33, p. 7793–7804. Cited by: Cryptocurrency Illicit-Activity Detection. Zhuo et al. (2024) W. Zhuo, Z. Liu, B. Hooi, B. He, G. Tan, R. Fathony, and J. Chen Partitioning message passing for graph fraud detection. arXiv preprint arXiv:2412.00020. Cited by: Cryptocurrency Illicit-Activity Detection.