Paper deep dive
EDATracer: An Agentic Framework for Large-Scale EDA Artifact Analysis
Phat Tieu, Sayanti Jana, Matthew DeLorenzo, Jiawen Wu, Narendran Srinivasan, Srinivas Shakkottai, Jiang Hu, Jeyavijayan Rajendran
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 8/9/2026, 2:16:36 AM
Summary
The paper introduces EDATracer, an agentic framework for analyzing large-scale Electronic Design Automation (EDA) artifacts. It addresses challenges in cross-artifact reasoning by organizing heterogeneous design artifacts (source files, logs, netlists, reports) into a knowledge graph paired with a semantic vector index. The authors curate a dataset of 2,787 open-source chip designs and a 90-question benchmark. EDATracer outperforms commercial agents like Cursor and Claude Code in accuracy while using fewer tokens.
Entities (9)
Relation Signals (6)
EDATracer → uses → Knowledge Graph
confidence 95% · EDATracer organizes design artifacts into a knowledge graph paired with a semantic vector index
EDATracer → uses → Semantic Vector Index
confidence 95% · EDATracer organizes design artifacts into a knowledge graph paired with a semantic vector index
EDATracer → outperforms → Cursor
confidence 90% · EDATracer achieves the best pass@1 accuracy, outperforming Cursor and Claude Code by 6.4 and 7.2 percentage points
EDATracer → outperforms → Claude Code
confidence 90% · EDATracer achieves the best pass@1 accuracy, outperforming Cursor and Claude Code by 6.4 and 7.2 percentage points
Dataset → contains → RISC-V
confidence 85% · Notably, the largest category included RISC-V architectures (23.6%).
EDATracer → uses → Neo4j
confidence 85% · structured Cypher queries (Neo4J 2026) over G
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Modern chip design relies on electronic design automation (EDA) tools that generate large, heterogeneous artifacts, including source files, scripts, logs, netlists, and reports. Analyzing these artifacts is critical for debugging, optimization, and design-flow understanding, but remains difficult because relevant evidence is often distributed across many artifact types and design stages. Although LLM agents show promise for EDA assistance, existing approaches lack public benchmarks for large-scale cross-artifact analysis and often struggle to ground reasoning in tool-generated evidence. We present EDATracer, an agentic framework for evidence-grounded EDA artifact analysis. EDATracer organizes design artifacts into a knowledge graph paired with a semantic vector index, enabling LLM agents to retrieve evidence across source files, logs, netlists, and reports. We curate an 18.9 GB dataset of 2,787 synthesizable open-source chip designs and introduce a 90-question benchmark spanning factual, statistical, and reasoning tasks. Across evaluated agents, EDATracer achieves the best pass@1 accuracy, outperforming Cursor and Claude Code by 6.4 and 7.2 percentage points on average, while using 2.0-3.2x fewer tokens.
Tags
Links
- Source: https://arxiv.org/abs/2608.04032v1
- Canonical: https://arxiv.org/abs/2608.04032v1
PDF not stored locally. Use the link above to view on the source site.
Full Text
89,646 characters extracted from source content.
Expand or collapse full text
EDATracer: An Agentic Framework for Large-Scale EDA Artifact Analysis Phat Tieu, Sayanti Jana, Matthew DeLorenzo, Jiawen Wu, Narendran Srinivasan, Srinivas Shakkottai, Jiang Hu, Jeyavijayan Rajendran Abstract Modern chip design relies on electronic design automation (EDA) tools that generate large, heterogeneous artifacts, including source files, scripts, logs, netlists, and reports. Analyzing these artifacts is critical for debugging, optimization, and design-flow understanding, but remains difficult because relevant evidence is often distributed across many artifact types and design stages. Although LLM agents show promise for EDA assistance, existing approaches lack public benchmarks for large-scale cross-artifact analysis and often struggle to ground reasoning in tool-generated evidence. We present EDATracer, an agentic framework for evidence-grounded EDA artifact analysis. EDATracer organizes design artifacts into a knowledge graph paired with a semantic vector index, enabling LLM agents to retrieve evidence across source files, logs, netlists, and reports. We curate an 18.9 GB dataset of 2,787 synthesizable open-source chip designs and introduce a 90-question benchmark spanning factual, statistical, and reasoning tasks. Across evaluated agents, EDATracer achieves the best pass@1 accuracy, outperforming Cursor and Claude Code by 6.4 and 7.2 percentage points on average, while using 2.0–3.2×2.0--3.2× fewer tokens. Introduction Semiconductor chips have become strategic infrastructure for Artificial Intelligence (AI), data centers, and the broader digital economy (U.S. DOC 2024; SIA 2026). At the core of the chip industry lies electronic design automation (EDA) tools (Siemens EDA 2026). Chip design flows use EDA workflows and tools to automate the design, simulation, verification, synthesis, and physical implementation of modern hardware (Siemens 2026). Effective EDA artifact analysis is essential within the design, optimization, and verification stages (Groeneveld 2002). Modern chip design flows generate large, heterogeneous collections of artifacts, including up to thousands of log files and millions of lines of output, making analysis increasingly costly and complex (Huang et al. 2021; Kanagal 2025; Arneberg 2022). This creates significant engineering and economic burdens: artifact analysis accounts for 47% of chip verification effort, while debugging can delay design schedules in an industry where advanced chip costs have risen from approximately $298M at 7 nm in 2018 to $700M at 2 nm in 2026 (Foster 2022; Palma et al. 2022; PwC 2026). Consequently, EDA artifact analysis remains a practical bottleneck for modern chip design (Chen et al. 2026a; Arneberg 2022). Large language model (LLM)-based agentic frameworks have shown success in assisting with chip design flow (Ben Gu 2026). For example, agentic frameworks have been used for verification assistance, where an LLM helps engineers check whether a chip design behaves as intended (Liu et al. 2024a; Pu et al. 2024); automated debugging, where an LLM helps identify the root cause of design or tool failures (Li et al. 2025); and EDA workflow automation, where an LLM-based agent coordinates multi-step design flow tasks such as invoking tools, interpreting intermediate results, and selecting next actions (Lu et al. 2026). These frameworks typically combine retrieval, external analysis tools, and multi-step reasoning to solve complex technical tasks (Chen et al. 2026b; Lewis et al. 2020; Yao et al. 2023; Shinn et al. 2023). Figure 1: Overview of EDATracer, an agentic framework for EDA artifact analysis. However, EDA artifact analysis poses unique challenges for existing LLM and agentic frameworks due to 3 key factors: C1: Lack of public datasets and benchmarks. Previous EDA datasets target chip design or logic synthesis (Liu et al. 2024a; Yu and Ren 2026). In contrast, EDA artifacts contain sensitive design details, tool configurations, and project-specific debugging information. Therefore, there is no publicly available benchmark that systematically supports cross-artifact EDA analysis. Furthermore, no widely used open benchmark currently exists to evaluate agents’ capabilities to accurately analyze EDA artifacts in large-scale design repositories (Kanagal 2025; Chen et al. 2026a). C2: Complex EDA artifacts. EDA analysis requires reasoning over many designs, where each design may contain numerous heterogeneous and lengthy artifacts. For example, a single design can produce multiple types of artifacts, including source code, documentation, logs, netlists, and reports. Each artifact can exceed 100k100k lines in large designs. This makes naive full-context prompting impractical due to context and token limits (Qi et al. 2023). C3: Grounding failures in cross-artifact reasoning. EDA artifact analysis often requires evidence from multiple files and design stages; e.g., the final failure message appears in the log file, but the actual source of the failure may be hidden in a related source file or in an earlier report. Without a reliable reference of the EDA-tool context to ground LLM reasoning to real evidence, LLMs may hallucinate or rely on shallow keyword matches (Pu et al. 2024). We propose EDATracer (Figure 1), an agentic framework for evidence-grounded analysis of large chip design repositories and their EDA artifacts. To address dataset limitations in scale and EDA-related artifacts (C1), we develop an automated dataset curation methodology to extract, filter, and synthesize chip repositories, yielding 2,787 large-scale designs along with their synthesis logs, netlists, and performance reports. To address the size and heterogeneity of EDA artifacts (C2), we organize the dataset into a knowledge graph representation that captures design hierarchy, artifact provenance, and dependency relationships. To support grounded cross-artifact reasoning (C3), EDATracer uses a retrieval agent to identify relevant evidence and a reasoning agent to synthesize answers from the retrieved context. Across a 90-question benchmark covering factual, statistical, and reasoning tasks, EDATracer outperforms existing commercial agent frameworks while using fewer tokens. In summary, the contributions of this paper are: • We curate an 18.9 GB dataset with 2,787 open-source, large-scale designs and associated EDA artifacts capturing the design hierarchy and cross-artifact dependencies. • We introduce EDATracer, an agent framework that organizes EDA artifacts into a knowledge graph and uses structured retrieval with LLM reasoning for grounded question answering. • We develop a 90-question benchmark for EDA artifact analysis that covers factual, statistical, and reasoning tasks at varying levels of difficulty. • We provide a comprehensive evaluation against state-of-the-art agent frameworks, EDATracer achieves the best overall accuracy, outperforming Cursor and Claude Code pass@1 by an average of 6.4 and 7.2 percentage points, respectively, while also using up to 3.2×3.2× fewer total tokens. Background Electronic design automation (EDA) tools are a suite of software and hardware tools that assist engineers across various stages of the chip design flow, automating complex tasks required to design, verify, and manufacture chips (Groeneveld 2002; Huang et al. 2021). For instance, EDA tools can assist at the initial design specification stage, in which developers specify the intended functionality of a chip with code111In this paper, code and source code for chip design refer to register-transfer level (RTL) hardware description language (HDL) code, such as Verilog and SystemVerilog.. Logic synthesis tools then transform code into low-level representations (gate-level netlists), ensuring the design can be implemented using an existing technology library and providing estimates of power, performance, and chip area (PPA) (Synopsys 2026; Cadence Design Systems 2026). EDA tools also assist at different stages of chip design (e.g., simulation, manufacturing, and fabrication) (Siemens 2026). EDA artifacts. EDA tools produce artifacts essential for developers to analyze in support of their design objectives (Kanagal 2025). These artifacts include the (i) synthesis script, which invokes the synthesis tool and specifies the design to be synthesized, the technology library to use, the constraints to apply, and the synthesis outputs to generate. (i) Synthesis logs, which track the detailed compilation procedure from code to netlists, identifying execution status (e.g., design parsing, elaboration, and technology mapping), errors, and warnings (e.g., syntax issues, missing files/modules, or bad practices) (Arneberg 2022). (i) Netlist files are also generated, detailing the exact logic gates, registers, and interconnections required to manufacture the associated chip design (Synopsys 2026; Cadence Design Systems 2026; Wang et al. 2024b). Additionally, (iv) PPA reports are generated post-synthesis, which calculate the estimated power consumed, the performance (timing), and the physical area of the chip (Huang et al. 2021). These quantities are central optimization targets in recent learning-based EDA systems (Xue et al. 2024; Wang et al. 2025). EDA artifact analysis. Existing LLM-based analysis methods remain largely artifact-local. Log analysis targets software logs for anomaly detection or diagnosis (Qi et al. 2023; Liu et al. 2024b). Chip-oriented LLM studies focus on code generation (Bush et al. 2025) or report-level prediction (Wang et al. 2024a). Another work evaluates multi-document QA benchmarks over evidence distributed across documents (Tang and Yang 2024). However, these tasks lack EDA-specific artifact semantics and design-flow dependencies. To the best of our knowledge, prior work lacks systematic, grounded analysis over linked chip design artifacts. EDATracer addresses this gap by organizing EDA artifacts into a structured cross-artifact representation and enabling grounded multi-step reasoning across the design flow. Figure 2: EDA artifact dataset construction flow. Creating the Dataset A representative dataset for agent-based EDA analysis must pair diverse, synthesizable chip designs with the EDA artifacts generated during the design flow. This pairing is necessary because realistic EDA tasks require the capacity to reason across both designs and tool-generated artifacts, where relevant evidence is often distributed across multiple files. Existing EDA datasets are limited for this setting, as many focus on code generation (Chen et al. 2026b) or single-log analysis (Bush et al. 2025) rather than cross-artifact reasoning. To this end, we construct our EDA artifact dataset with two goals: (i) collecting diverse, full-chip open-source designs (source artifacts), and (i) generating the pairing comprehensive flow artifacts (generated artifacts) using common EDA tools and libraries (Synopsys 2026; Cadence Design Systems 2026). Curating our initial dataset has two stages: the collection stage and the generation stage. Collection Stage The collection stage constructs a corpus D of open-source chip designs (source artifacts) from public GitHub repositories. These repositories are often inconsistently organized, and not synthesis-ready. Our goal is to retain synthesizable chip designs spanning diverse categories, including processors, memories, and controllers. We formulate design collection as a staged filtering process that progressively refines an initial candidate set into a curated corpus. We represent each candidate repository as r=(r,ℳr)r=(S_r,M_r), where rS_r is the repository source-file set and ℳrM_r contains repository metadata, such as the repository name, description, and directory structure. If a repository is accepted into the final corpus, it becomes a design d∈d with source-file set dS_d. Let K denote a set of chip-design-related search keywords. The collection process proceeds in four phases: (i) Discovery, which queries public repositories using K and forms an initial candidate set ℛ0=⋃k∈Search(k)R_0= _k Search(k) after deduplication at the URL-level. (i) Heuristic filtering, which applies rule-based predicates h(r)h(r) over repository files and metadata to remove candidates unlikely to contain usable RTL, yielding ℛ1=r∈ℛ0:h(r)=1R_1=\r _0:h(r)=1\. (i) LLM-based evaluation, which applies a fixed-prompt evaluator ℓ(r) (r) to assess whether each remaining repository corresponds to a meaningful chip design with potentially synthesizable structure, yielding ℛ2=r∈ℛ1:ℓ(r)=1R_2=\r _1: (r)=1\. (iv) Synthesis feasibility, which applies a synthesis check v(r)v(r) and retains only repositories that can be processed by the downstream artifact-generation pipeline: =dr:r∈ℛ2,v(r)=1D=\d_r:r _2,\ v(r)=1\, where drd_r denotes the accepted design from repository r. The accepted designs form a curated corpus for EDA artifact generation. Further implementation details can be found in the Appendix. Generation Stage Algorithm 1 Iterative Repair Loop 1:Top-module candidates T, HDL files V, clock candidates C, max attempts AmaxA_ 2:Synthesized artifacts or terminal failure 3:tried←∅T_tried← 4:a←0a← 0 5:t←top()t (T) 6:c←best()c (C) 7:S←MakeTCL(t,,c)S← MakeTCL(t,V,c) 8:while a<Amaxa<A_ do 9: a←a+1a← a+1 10: tried←tried∪tT_tried _tried∪\t\ 11: (L,e,R)←RunSynthesis(S)(L,e,R)← RunSynthesis(S) 12: if Complete(L)(L) and NonZeroArea(R)(R) then 13: Store log, netlist, and PPA reports 14: return Success 15: end if 16: γ←ClassifyAttempt(L,e,R)γ← ClassifyAttempt(L,e,R) 17: Δ←SelectUpdate(γ,,tried,) ← SelectUpdate(γ,T,T_tried,C) 18: if Δ=NoRepair = NoRepair then 19: return Failure(γ)(γ) 20: end if 21: S←ApplyUpdate(S,Δ)S← ApplyUpdate(S, ) 22: (t,c)←UpdateState(t,c,Δ)(t,c)← UpdateState(t,c, ) 23: LogAttempt(t,γ,Δ)(t,γ, ) 24:end while 25:return Failure(AttemptLimit) The objective of the artifact generation stage is to synthesize each accepted design d∈d and produce a generated EDA artifact tuple d=(Ld,Nd,Pd)A_d=(L_d,N_d,P_d), where LdL_d denotes the synthesis log, NdN_d the gate-level netlist, and PdP_d the set of PPA reports. Here, dS_d denotes the original source-file set associated with design d, while dA_d denotes the artifacts generated by our synthesis flow. Because public repositories typically do not include complete synthesis artifacts or synthesis-ready configurations, we formulate artifact generation as a deterministic, iterative synthesis procedure that maps each design either to a valid artifact tuple dA_d or to a terminal failure state. As shown in Figure 2, artifact generation proceeds through the following six stages for each design d. (i) Tool and library assignment assigns a synthesis configuration θd=(τd,λd) _d=( _d, _d), where τd _d is the synthesis tool and λd _d is the technology library. 222We use Synopsys Design Compiler (Synopsys 2026) as the synthesis tool and three open-source technology libraries: NanGate45 (The OpenROAD Project 2026b), SkyWater130 (Google and SkyWater Technology Foundry 2026), and ASAP7 (The OpenROAD Project 2026a). Library selection will be recorded in the generated artifacts. (i) Design analysis applies a static analyzer to the repository file set dS_d to identify candidate HDL files dV_d, remove likely testbench or verification files, reconstruct the module-instantiation graph HdH_d, rank candidate top modules dT_d, and detect candidate clock signals dC_d when possible. Details regarding HDL file selection and top-module ranking are provided in the Appendix. (i) Top-module selection initializes synthesis with the highest-ranked candidate td∈dt_d _d, which is necessary because many repositories do not explicitly specify a synthesizable top module. (iv) TCL script generation constructs a tool-specific synthesis script SdS_d from (d,td,d,θd)(V_d,t_d,C_d, _d). If a clock signal is detected, the script includes an explicit timing constraint; otherwise, it introduces a virtual clock to enable timing analysis. (v) Iterative error repair repeatedly invokes synthesis and updates the script configuration based on the observed failure category, as detailed in Algorithm 1. Each failed attempt is classified using the synthesis log and report, and at most one deterministic update is applied per iteration: switching to the next top-module candidate, extending the search path, or selecting an alternative clock. The loop terminates when synthesis succeeds, no valid update remains, or the max attempt count is reached. (vi) Artifact extraction stores d=(Ld,Nd,Pd)A_d=(L_d,N_d,P_d) for each successful synthesis run. The final artifact dataset therefore consists of successfully synthesized designs, their original source-file sets dS_d, and their generated artifacts dA_d. For later knowledge-graph construction, we define the complete ingestion set for each design as ℐd=d∪Ld,Nd∪PdI_d=S_d∪\L_d,N_d\∪ P_d. Figure 3: Distribution of file types in the dataset. Dataset Results We obtain the following results. The collection stage first identified 23,115 candidate designs, of which 16,514 passed heuristic filtering, 8,812 passed LLM-based evaluation, and 2,787 synthesizable full-chip designs were accepted into the final EDA artifact dataset. The distribution of chip design sources is included in the Appendix. Notably, the largest category included RISC-V architectures (23.6%). After the generation stage, artifacts (synthesis logs, netlists, and PPA reports) for each of the top-level designs were generated, as shown in Figure 3. In total, we obtained a dataset of 18.91 GB, of which 53% is attributable to EDA-generated artifacts, supplementing the curated chip designs. EDATracer Agentic Framework Figure 4: The full EDATracer agentic framework. The goal of EDATracer is to answer a user question q by identifying and reasoning over relevant evidence distributed across the design ingestion sets ℐdd∈\I_d\_d . This setting creates two challenges: (i) artifacts are too large and heterogeneous for direct full-context prompting (C2), and (i) many questions require evidence that spans multiple files and artifact types (C3). As shown in Figure 4, EDATracer addresses these challenges through offline and online stages. In the offline stage, EDATracer converts each design’s ingestion set into a structured knowledge graph and a graph-associated semantic vector index. In the online stage, a retrieval agent and reasoning agent interact over these representations to produce grounded answers with supporting evidence. Offline Stage For each design d, EDATracer maps the ingestion set ℐdI_d into two complementary representations: an attributed, typed knowledge graph Gd=(Vd,Ed)G_d=(V_d,E_d) and a semantic vector index dZ_d. For the knowledge graph, VdV_d and EdE_d denote the set of graph nodes and directed typed edges, respectively. Each node v∈Vdv∈ V_d represents an EDA-relevant entity, such as a design, artifact file, text chunk, RTL module, netlist component, or structural element. Node-specific information is stored as metadata ϕ(v)φ(v), including artifact type, file path, line range, textual content, extracted metrics, warning or error labels, and synthesis status. Each edge is represented as (u,ρ,v)(u,ρ,v), where u,v∈Vdu,v∈ V_d and ρ denotes the relation type. Edges encode artifact hierarchy, provenance, and structural relations. Also, EDATracer builds a semantic vector index over text-bearing graph nodes: d=(v,zv):v∈Vd,emb⊆VdZ_d=\(v,z_v):v∈ V_d,emb V_d\, where Vd,embV_d,emb contains nodes selected for semantic retrieval, such as text-chunk nodes, and zvz_v is the embedding associated with node v. The embedding vectors are stored in the vector index rather than as ordinary node metadata in ϕ(v)φ(v), while the corresponding graph node preserves source information such as the design identifier, artifact path, and line span. The full offline representation is G=(V,E),V=⋃d∈Vd,E=⋃d∈Ed,=⋃d∈dG=(V,E), V= _d V_d, E= _d E_d, = _d Z_d. Graph ingestion consists of three core operations: (i) Artifact parsing partitions input files ℐdI_d into textual files ℐdtextI_d^text (e.g., logs, reports) and structural files ℐdstructI_d^struct (e.g., RTL, netlists), using domain-specific parsers to extract node metadata ϕ(v)φ(v). (i) Provenance linking adds typed edges to preserve structural hierarchy and lineage across generated EDA artifacts. (i) Semantic indexing splits long text files into overlapping chunks, instantiating chunk nodes in GdG_d and indexing their embeddings in dZ_d. This enables semantic retrieval to locate relevant nodes via the vector index and map them back to their graph context G. The complete procedure is detailed in the supplementary material. Online Stage At online inference time, EDATracer answers a user question q using the knowledge graph G=(V,E)G=(V,E) and graph-associated vector index Z. The online stage maps (q,G,)→(y^,q)(q,G,Z)→( y,E_q), where y y is the generated answer and q=(Vq,Eq,ℬq)E_q=(V_q,E_q,B_q) is the accumulated evidence. Here, Vq⊆V_q V and Eq⊆E_q E denote retrieved graph nodes and edges, while ℬq⊆B_q denotes retrieved vector-index hits. As each vector-index entry has the form (v,zv)(v,z_v), each semantic hit is associated with a graph node whose metadata ϕ(v)φ(v) provides the relevant text span, file path, lines, extracted EDA facts, and context. As shown in Figure 4, EDATracer addresses C2 and C3 through an iterative interaction between a reasoning agent and a retrieval agent. The reasoning agent interprets the user question, determines required information, and generates information requests. The retrieval agent answers these requests using semantic search over Z or structured Cypher queries (Neo4J 2026) over G. The reasoning agent then integrates the returned evidence, and either produces an answer or requests additional evidence from the retrieval agent. Retrieval agent. At iteration i, let ℛ(i)=r1(i),…,rm(i)R^(i)=\r_1^(i),…,r_m^(i)\ denote the information requests generated by the reasoning agent. For each request rj(i)r_j^(i), the retrieval agent can use two retrieval modes. In semantic retrieval, the agent embeds the request and searches the vector index with cosine similarity: ℬsem(rj(i))=TopK(v,zv)∈cos_sim(Embed(rj(i)),zv).B_sem(r_j^(i))=TopK_(v,z_v) cos\_sim ( Embed(r_j^(i)),z_v ). The returned entries identify graph-associated text nodes, such as chunks from logs, reports, documentation, or scripts, which can be resolved through G to recover their design and artifact context. In structured retrieval, the agent generates a Cypher query cj(i)c_j^(i) and executes it over the knowledge graph: (ΔVcyp,ΔEcyp)=ExecCypher(cj(i),G).( V_cyp, E_cyp)= ExecCypher(c_j^(i),G). This retrieves graph evidence directly, such as design nodes, artifact nodes, extracted metrics, structural relations, and provenance links. The semantic and structured results are merged into an evidence update Δ(i)=(ΔV(i),ΔE(i),Δℬ(i)) ^(i)=( V^(i), E^(i), ^(i)), which is added to the accumulated evidence: q(i+1)=q(i)∪Δ(i).E_q^(i+1)=E_q^(i)∪ ^(i). Reasoning agent. The reasoning agent coordinates the online analysis. Given the question q and accumulated evidence q(i)=(Vq(i),Eq(i),ℬq(i))E_q^(i)=(V_q^(i),E_q^(i),B_q^(i)), it either produces an answer or information requests: Reason(q,q(i))→y^,if sufficient evidence,ℛ(i),otherwise. Reason(q,E_q^(i))→ cases y,&if sufficient evidence,\\ R^(i),&otherwise. cases As each vector hit is associated with a graph node, define Vℬ(i)=v:(v,zv)∈ℬq(i).V_B^(i)=\v:(v,z_v) _q^(i)\. The reasoning agent then operates over a prompt representation q(i)=Prompt(Vq(i)∪Vℬ(i),Eq(i),ϕ),C_q^(i)= Prompt (V_q^(i)∪ V_B^(i),\,E_q^(i),\,φ ), which serializes selected nodes, their metadata, and their typed relations into the LLM context. The reasoning agent then assesses whether this context is sufficient to answer the question. If the evidence is insufficient, it generates additional information requests; otherwise, it returns the final grounded answer y y. Implementation Model Tool Factual Statistical Reasoning Overall Avg. pass@1 pass@5 Avg. pass@1 pass@5 Avg. pass@1 pass@5 Avg. pass@1 pass@5 Claude Opus 4.7 Cursor 9.40 86.7 90.0 8.33 66.7 83.3 7.93 74.0 96.7 8.56 75.8 90.0 Claude Code 9.33 90.3 96.7 8.40 69.3 76.7 8.00 85.3 100 8.58 81.7 91.1 EDATracer 9.87 97.3 100 9.57 97.3 100 9.20 88.7 96.7 9.54 94.4 98.9 Claude Sonnet 4.6 Cursor 9.53 92.7 96.7 9.00 82.0 96.7 8.40 82.7 96.7 8.98 85.8 96.7 Claude Code 9.53 95.3 96.7 9.03 93.3 100 7.80 76.7 90.0 8.79 88.4 95.6 EDATracer 9.40 90.7 93.3 9.70 92.0 100 8.30 85.3 93.3 9.13 89.3 95.6 Claude Haiku 4.5 Cursor 9.70 96.7 100 7.67 69.3 96.7 7.10 60.7 76.7 8.16 75.6 91.1 Claude Code 8.70 91.3 96.7 7.40 76.7 86.7 5.80 48.0 73.3 7.30 72.0 85.6 EDATracer 8.83 85.3 93.3 9.53 94.7 100 7.67 60.0 80.0 8.68 80.0 91.1 GPT 5.4 Cursor 9.27 83.3 86.7 7.93 74.0 86.7 7.07 71.3 86.7 8.09 76.2 86.7 EDATracer 9.53 94.0 100 8.53 73.3 86.7 8.87 80.7 96.7 8.98 82.7 94.4 GPT 5.4 mini Cursor 8.40 80.0 93.3 6.93 57.3 70.0 7.10 55.3 83.3 7.48 64.2 82.2 EDATracer 8.07 70.0 93.3 7.30 59.3 70.0 6.83 60.0 90.0 7.40 63.1 84.4 Qwen 3.5-9B EDATracer 8.03 69.3 83.3 8.00 70.0 90.0 6.07 36.0 73.3 7.37 58.9 82.2 Table 1: Accuracy comparison between EDATracer and other commercial AI coding assistants. Avg. scores are reported on a 0–10 scale; pass@1 and pass@5 are percentages. Hardware stack. Experiments use a server with an NVIDIA H200 GPU (143 GB), AMD EPYC 9554 64-core processors, and 1.1 TB RAM. Software stack. We implement Memgraph 3.8.0 (Memgraph 2026a, b) for knowledge graph storage. Neo4j 5.11 to perform vector search (Neo4j 2026). Strands Agents 1.23.0 (Strands Agents 2026) for agent orchestration. We compare EDATracer with two commercial agentic frameworks: Cursor (Cursor 2026) and Claude Code (Anthropic 2026). We test on different LLMs, from proprietary models of varying sizes, including Claude Opus 4.7, Claude Sonnet 4.6, and GPT-5.4, to smaller models, Claude Haiku 4.5, GPT-5.4-mini, and an open-weight model, Qwen 3.5-9B. Evaluation benchmark. We manually construct a benchmark with 90 golden question-answer (QA) pairs from the dataset. It contains three categories: factual, statistical, and reasoning, with 30 questions each. Each category has three difficulty levels: easy, medium, and hard, with 10 questions per level. Details are provided in the Appendix. Evaluation metrics. We use three metrics to evaluate the correctness and efficiency of EDA artifact analysis. (i) Average score. Each generated answer receives a score in [0,10][0,10] based on the golden answer, and scores are averaged within each category. Human evaluators score each answer against the golden answer on a [0,10][0,10] scale, following the detailed rubric in the Appendix. (i) Pass@k (Chen et al. 2021) is computed for k=1k=1 and k=5k=5, averaged over all questions in a category. A question passes if at least one score is 8 or higher. (i) Token usage is computed as the total number of input and output tokens consumed by each agentic framework across five repeated runs of the 90-question benchmark. Results We focus on three research questions (RQs): RQ1: Accuracy of analysis. Does EDATracer outperform commercial agentic frameworks on EDA artifact analysis tasks? RQ2: Efficiency of token usage. How cost-efficient is EDATracer when processing complex, large-scale EDA artifacts? RQ3: Local deployment feasibility. Can EDATracer enable smaller, locally deployed open-weight models to perform EDA artifact analysis effectively? RQ1: Accuracy of Analysis The accuracy results from the evaluations of EDATracer relative to alternative agents are provided in Table 1. Regarding RQ1, EDATracer improves the overall average score over Cursor in four of the five shared LLM settings, with the exception of GPT-5.4-mini. Averaged across these settings, EDATracer improves over Cursor by +0.49 score points, from 8.25 to 8.75. Compared with Claude Code, EDATracer improves the overall average score across all three shared Claude settings, with an average gain of +0.89 score points, from 8.22 to 9.12. EDATracer achieves the best pass@1 accuracy, outperforming Cursor and Claude Code by 6.4 and 7.2 percentage points on average. Overall, EDATracer performs most effective with stronger models. Paired with Opus 4.7, it achieves the highest overall performance, with 94.4% pass@1 and 98.9% pass@5. Compared with other agentic frameworks, one key insight is that the largest gains are in statistical questions. EDATracer achieves the highest average score across the five proprietary LLM settings (8.92/10). In contrast, simple factual questions show smaller gains, and weaker models benefit less. With Haiku 4.5 model, EDATracer improves statistical-question accuracy by 1.86 points over the next-best agent. This reflects EDATracer’s ability to retrieve targeted evidence from multiple artifacts and perform multi-step aggregation while preserving correct units and prefixes. RQ2: Efficiency of Token Usage Figure 5 compares the token usage 333Cursor does not expose detailed token usage through its command-line interface; we obtain its token counts from the aggregate usage statistics reported on the Cursor website. of EDATracer with Cursor and Claude Code. EDATracer consistently uses fewer tokens across all evaluated LLMs, using only 20M–36M tokens compared with 54M–85M for Cursor and 63M–115M for Claude Code. Compared with Cursor, EDATracer reduces token usage by 2.0×–3.0× across five models. Compared with Claude Code, EDATracer reduces token usage by 2.1×–3.2× across the three Claude models. These results show that EDATracer reduces token cost for both general-purpose coding agents and model-specific agentic frameworks. A key insight is that the token reduction is consistent across both Claude and GPT backbones, suggesting that the benefit is derived from EDATracer’s retrieval structure rather than from a specific LLM. These results additionally demonstrate that the structured knowledge graph assists in reducing unnecessary context expansion during retrieval and reasoning. Instead of repeatedly passing large artifact contexts to the model, EDATracer retrieves targeted graph nodes and typed relationships, allowing the agent to assemble focused evidence for each question. Thus, EDATracer achieves competitive accuracy while using substantially fewer tokens, demonstrating that graph-based retrieval can improve the cost efficiency of LLM-based EDA artifact analysis. Figure 5: Token usage comparison between EDATracer and other commercial agentic frameworks. RQ3: Local Deployment Feasibility To assess if EDATracer can support accurate EDA artifact analysis with a locally deployed open-weight model, we analyze the results from Qwen 3.5-9B (see Table 1). Although Qwen does not match the strongest proprietary models overall, its performance approaches that of GPT-5.4-mini with EDATracer and remains relatively strong on factual and statistical questions. These results indicate that EDATracer’s knowledge-graph representation can provide sufficiently grounded evidence for a local model to answer many artifact-level and aggregation-oriented questions without relying on proprietary inference APIs. The main limitation appears in reasoning-heavy questions. Qwen is less successful within a single attempt, although multiple attempts continue to improve overall performance. This pattern suggests that the local model is able to retrieve useful evidence, but has more difficulty than larger commercial models integrating it into a final answer. Related Work Chip design datasets, recently, mainly focus on chip design or logic synthesis tasks, such as RTL generation (Thakur et al. 2024), repair (Liu et al. 2024a), and synthesis optimization (Yu and Ren 2026). These datasets are useful for evaluating code-level design tasks, but do not include the full set of artifacts produced during an EDA flow. EDATracer addresses this gap by building a large-scale dataset of EDA artifacts with linked design-flow artifacts. EDA agentic frameworks, recently, support Verilog generation and repair (Thakur et al. 2024; Liu et al. 2024a; Blocklove et al. 2025), unit-test generation (Nandal et al. 2026), EDA question answering (Pu et al. 2024), root-cause analysis (Qiu et al. 2025), and workflow automation (Lu et al. 2026). These systems show the promise of LLMs and agents for EDA but target localized code tasks, document QA, or tool execution. Meanwhile, graph and KG-based methods support relational reasoning (Deng et al. 2025; Anokhin et al. 2025; DeLong et al. 2025), but EDA graph methods often focus on circuit-level structures (Ma et al. 2020; Lopera et al. 2021; Gautam et al. 2025), while graph-based log analysis is more common in software and security (Payne and Xie 2024; Cotti et al. 2026). EDATracer bridges these directions by combining agentic reasoning with graph-grounded retrieval across heterogeneous EDA artifacts. Conclusion We present EDATracer, an agentic framework for large-scale EDA artifact analysis. We curate an 18.9 GB dataset with 2,787 open-source synthesizable designs and 249K EDA files, including source code, scripts, logs, netlists, reports, and documentation. EDATracer organizes these artifacts into a domain-specific knowledge graph and combines structured retrieval with LLM reasoning for grounded EDA queries. We build a benchmark of 90 EDA-focused questions across factual, statistical, and reasoning tasks. EDATracer achieves the best overall pass@1 accuracy, outperforming Cursor and Claude Code by an average of 6.4% and 7.2%, while using between 2.0-3.2× fewer tokens. Future work. EDATracer currently focuses on synthesis-stage artifacts. Our future work will cover design-flow stages, such as floorplanning, placement, routing, timing closure, and sign-off, which introduce artifacts including placement reports, congestion maps, timing reports, and sign-off summaries. This would enable the knowledge graph to capture more complete design-flow dependencies and support queries that trace timing violations to their sources, including source code, constraints, placement, or routing congestion. Another direction is multimodal EDA analysis: future versions can index and reason about visual artifacts such as layouts, floor plans, waveforms, and schematics using vision-language models and multimodal embeddings. Limitations The current framework contains the following limitations. First, the accuracy of EDATracer depends heavily on the quality of knowledge graph construction. If artifacts are poorly parsed, incorrectly chunked, or missing important graph relationships, downstream retrieval and reasoning quality can degrade. In particular, errors in entity and relation extraction can cause the agent to retrieve incomplete or misleading context. This limitation is especially important for EDA workflows, where a warning message, timing value, or report summary is only useful if it is correctly linked to the corresponding design, tool run, script, module, or generated output. Incorrect or missing graph edges can therefore directly affect the reliability of multi-hop retrieval and reasoning. Second, EDATracer introduces an offline preprocessing cost because artifacts must be collected, parsed, embedded, and inserted into the knowledge graph before retrieval can occur. While this cost enables more structured and traceable analysis, it may be less suitable for very small or rapidly changing projects, where a simple file search suffices. In addition, the current schema and benchmark focus primarily on digital EDA artifacts, including source code, synthesis scripts, tool logs, generated reports, netlists, and PPA data. As a result, the framework may require additional engineering to support broader end-to-end chip design workflows, including analog design, physical design, verification, sign-off, and proprietary industrial flows with vendor-specific artifact formats. Acknowledgment The authors acknowledge the support from the Purdue Center for Secure Microelectronics Ecosystem — CSME#210205. References P. Anokhin, N. Semenov, A. Sorokin, D. Evseev, A. Kravchenko, M. Burtsev, and E. Burnaev (2025) AriGraph: learning knowledge graph world models with episodic memory for LLM agents. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, External Links: Link, Document Cited by: Related Work. Anthropic (2026) Claude Code Overview. Note: Accessed: 2026-05-23 External Links: Link Cited by: Implementation. T. Arneberg (2022) Logfile error analysis: how to make sense of millions of lines of logfiles using CHEW_LOGS (Check Errors and Warning in Logfiles). Association for Computing Machinery (ACM), San Francisco, CA, USA. Note: Presented at the 59th Design Automation Conference (DAC) Cited by: Introduction, Background. Ben Gu (2026) AI Agents Transform EDA with Cadence. LinkedIn. External Links: Link Cited by: Introduction. J. Blocklove, S. Thakur, B. Tan, H. Pearce, S. Garg, and R. Karri (2025) Automatically Improving LLM-based Verilog Generation using EDA Tool Feedback. ACM Transactions on Design Automation of Electronic Systems 30 (6). External Links: Document, Link Cited by: Related Work. S. Bush, M. DeLorenzo, P. Tieu, and J. Rajendran (2025) Free and fair hardware: a pathway to copyright infringement-free verilog generation using llms. In 2025 62nd ACM/IEEE Design Automation Conference (DAC), External Links: Document Cited by: Background, Creating the Dataset. Cadence Design Systems (2026) Genus Synthesis Solution Datasheet. Note: Accessed: 2026-05-05 External Links: Link Cited by: Background, Background, Creating the Dataset. D. Chen, V. Ganesh, W. Li, Y. C. Lin, Y. Liu, S. Mitra, D. Z. Pan, R. Puri, J. Cong, and Y. Sun (2026a) Report for NSF Workshop on AI for Electronic Design Automation. External Links: 2601.14541, Link Cited by: Introduction, Introduction. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021) Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374. Cited by: Implementation. Z. Chen, Y. Pu, H. Han, Y. Nie, J. Qin, Y. Qin, T. Qiu, Z. He, J. Zhai, B. Yu, and K. Zhao (2026b) MAEDA: An LLM-Powered Multi-Agent Evaluation Framework for EDA Tool Documentation QA. In 2026 Design, Automation & Test in Europe Conference (DATE 2026), External Links: Link Cited by: Introduction, Creating the Dataset. L. Cotti, I. Drago, A. Rula, D. Bianchini, and F. Cerutti (2026) OntoLogX: Ontology-Guided Knowledge Graph Extraction From Cybersecurity Logs With Large Language Models. Advanced Intelligent Systems 8 (6), p. e202501381. External Links: Document, Link, https://advanced.onlinelibrary.wiley.com/doi/pdf/10.1002/aisy.202501381 Cited by: Related Work. Cursor (2026) Cursor Documentation. Note: Accessed: 2026-05-23 External Links: Link Cited by: Implementation. L. N. DeLong, R. F. Mir, and J. D. Fleuriot (2025) Neurosymbolic AI for Reasoning Over Knowledge Graphs: A Survey. IEEE Transactions on Neural Networks and Learning Systems 36 (5), p. 7822–7842. External Links: ISSN 2162-2388, Link, Document Cited by: Related Work. J. Deng, C. He, J. Chen, B. Qin, J. Wu, Q. Huang, and Y. Li (2025) Constructing a knowledge graph-driven intelligent data-enabled design system for mold using deep semantic understanding and intelligent decision support. Scientific Reports 15 (1), p. 7322. External Links: Document, Link, ISSN 2045-2322 Cited by: Related Work. H. Foster (2022) The 2022 wilson research group functional verification study. Note: https://blogs.sw.siemens.com/verificationhorizons/2022/10/10/prologue-the-2022-wilson-research-group-functional-verification-study/Accessed: 2026-05-20 Cited by: Introduction. A. Gautam, R. Patton, T. Potok, R. Kannan, J. Aimone, and W. Severa (2025) AI-Powered Knowledge Graphs for Neuromorphic and Energy-Efficient Computing. In Proceedings of the Great Lakes Symposium on VLSI 2025, GLSVLSI ’25, p. 996–1001. External Links: ISBN 9798400714962, Link, Document Cited by: Related Work. Google and SkyWater Technology Foundry (2026) SkyWater SKY130 Open-Source PDK and Standard Cell Resources. Note: Accessed: 2026-05-05 External Links: Link Cited by: footnote 2. P.R. Groeneveld (2002) Physical Design Challenges for Billion Transistor Chips. In Proceedings. IEEE International Conference on Computer Design: VLSI in Computers and Processors, Vol. , p. 78–83. External Links: Document Cited by: Introduction, Background. G. Huang, J. Hu, Y. He, J. Liu, M. Ma, Z. Shen, J. Wu, Y. Xu, H. Zhang, K. Zhong, X. Ning, Y. Ma, H. Yang, B. Yu, H. Yang, and Y. Wang (2021) Machine Learning for Electronic Design Automation: A Survey. ACM Transactions on Design Automation of Electronic Systems (TODAES) 26 (5). External Links: ISSN 1084-4309, Link, Document Cited by: Introduction, Background, Background. R. Kanagal (2025) LLM-Powered EDA Log Analysis for Effective Design Debugging. Master’s Thesis, EECS Department, University of California, Berkeley. External Links: Link Cited by: Introduction, Introduction, Background. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2020) Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA. External Links: ISBN 9781713829546 Cited by: Introduction. J. Li, S. Wong, G. Wan, X. Wang, and J. Yang (2025) EDA-Debugger: An LLM-based Framework for Automated EDA Runtime Issue Resolution. In 2025 26th International Symposium on Quality Electronic Design (ISQED), External Links: Document, Link Cited by: Introduction. S. Liu, W. Fang, Y. Lu, Q. Zhang, H. Zhang, and Z. Xie (2024a) RTLCoder: Outperforming GPT-3.5 in Design RTL Generation with Our Open-Source Dataset and Lightweight Solution. In 2024 IEEE LLM Aided Design Workshop (LAD), External Links: Document, Link Cited by: Introduction, Introduction, Related Work, Related Work. Y. Liu, S. Tao, W. Meng, F. Yao, X. Zhao, and H. Yang (2024b) LogPrompt: Prompt Engineering Towards Zero-Shot and Interpretable Log Analysis. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings, ICSE-Companion ’24, p. 364–365. External Links: ISBN 9798400705021, Link, Document Cited by: Background. D. S. Lopera, L. Servadei, G. N. Kiprit, S. Hazra, R. Wille, and W. Ecker (2021) A survey of graph neural networks for electronic design automation. In 2021 ACM/IEEE 3rd Workshop on Machine Learning for CAD (MLCAD), p. 1–6. External Links: Document Cited by: Related Work. Y. Lu, H. I. Au, J. Zhang, J. Pan, G. Zhou, Y. Wang, J. Sun, A. Li, J. Zhang, H. Li, and Y. Chen (2026) AutoEDA: Enabling EDA Flow Automation through Microservice-Based LLM Agents. External Links: 2508.01012, Link Cited by: Introduction, Related Work. Y. Ma, Z. He, W. Li, L. Zhang, and B. Yu (2020) Understanding Graphs in EDA: From Shallow to Deep Learning. In Proceedings of the 2020 International Symposium on Physical Design, ISPD ’20, p. 119–126. External Links: ISBN 9781450370912, Link, Document Cited by: Related Work. Memgraph (2026a) Memgraph Documentation. Note: Accessed: 2026-05-23 External Links: Link Cited by: Implementation. Memgraph (2026b) Vector Search in Memgraph. Note: Accessed: 2026-05-23 External Links: Link Cited by: Implementation. D. Nandal, R. Revalor, S. Dan, and D. Pal (2026) LAUDE: llm-assisted unit test generation and debugging of hardware designs. External Links: 2601.08856, Link Cited by: Related Work. Neo4J (2026) Neo4j Cypher Query Language. Neo4j, Inc.. External Links: Link Cited by: Online Stage. Neo4j (2026) Neo4j vector index and search. Note: https://neo4j.com/developer/genai-ecosystem/vector-search/Neo4j Developer Guide, covers Neo4j 5.11+; accessed May 24, 2026 Cited by: Implementation. R. Palma, R. Varadarajan, J. Goodrich, T. Lopez, and A. Patil (2022) The growing challenge of semiconductor design leadership. BCG Analysis Report. Cited by: Introduction. L. Payne and M. Xie (2024) Log File Anomaly Detection Using Knowledge Graph Completion. In Proceedings of the 2024 8th International Conference on Deep Learning Technologies, ICDLT ’24, New York, NY, USA, p. 42–48. External Links: ISBN 9798400716867, Link, Document Cited by: Related Work. Y. Pu, Z. He, T. Qiu, H. Wu, and B. Yu (2024) Customized Retrieval Augmented Generation and Benchmarking for EDA Tool Documentation QA. External Links: 2407.15353, Link Cited by: Introduction, Introduction, Related Work. PwC (2026) Semiconductor and Beyond 2026. Note: Accessed: 2026-05-21 External Links: Link Cited by: Introduction. J. Qi, S. Huang, Z. Luan, S. Yang, C. Fung, H. Yang, D. Qian, J. Shang, Z. Xiao, and Z. Wu (2023) LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection. In 2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application, p. 273–280. Cited by: Introduction, Background. S. Qiu, M. Wang, R. Afsharmazayejani, M. M. Shahmiri, B. Tan, and H. Pearce (2025) Towards LLM-based Root Cause Analysis of Hardware Design Failures. In 2025 IEEE International Conference on Omni-layer Intelligent Systems (COINS), Vol. , p. 1–6. External Links: Document Cited by: Related Work. N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: Language Agents with Verbal Reinforcement Learning. In Proceedings of the 37th International Conference on Neural Information Processing Systems (NeurIPS), Cited by: Introduction. SIA (2026) Global annual semiconductor sales increase 25.6% to $791.7 billion in 2025. Note: https://w.semiconductors.org/global-annual-semiconductor-sales-increase-25-6-to-791-7-billion-in-2025/Accessed: 2026-05-20 Cited by: Introduction. Siemens EDA (2026) A New Era of EDA Powered by AI. Siemens EDA. Note: https://resources.sw.siemens.com/en-US/white-paper-a-new-era-of-eda-powered-by-ai/Accessed: 2026-05-20 Cited by: Introduction. Siemens (2026) Electronic Design Automation (EDA). Note: https://w.siemens.com/en-us/technology/electronic-design-automation-eda/Accessed: 2026-05-20 Cited by: Introduction, Background. Strands Agents (2026) Strands Agents SDK Documentation. Note: Accessed: 2026-05-23 External Links: Link Cited by: Implementation. Synopsys (2026) Design Compiler: Timing, Area, Power, and Test Optimization. Note: Accessed: 2026-05-05 External Links: Link Cited by: Background, Background, Creating the Dataset, footnote 2. Y. Tang and Y. Yang (2024) MultiHop-rag: benchmarking retrieval-augmented generation for multi-hop queries. In Proceedings of the Conference on Language Modeling, External Links: Link Cited by: Background. S. Thakur, B. Ahmad, H. Pearce, B. Tan, B. Dolan-Gavitt, R. Karri, and S. Garg (2024) VeriGen: A Large Language Model for Verilog Code Generation. ACM Transactions on Design Automation of Electronic Systems 29 (3). Cited by: Related Work, Related Work. The OpenROAD Project (2026a) ASAP7 Library / Design Kit Resources. Note: Accessed: 2026-05-05 External Links: Link Cited by: footnote 2. The OpenROAD Project (2026b) NanGate45 Library Support in OpenROAD-Flow-Scripts. Note: Accessed: 2026-05-05 External Links: Link Cited by: footnote 2. U.S. DOC (2024) Two Years Later: Funding from CHIPS and Science Act Creating Quality Jobs, Growing Local Economies, and Bringing Semiconductor Manufacturing Back to America. Note: https://w.commerce.gov/news/blog/2024/08/two-years-later-funding-chips-and-science-act-creating-quality-jobs-growing-localAccessed: 2026-05-20 Cited by: Introduction. S. Wang, K. Xu, and Z. Ling (2024a) Deep Learning-Based Chip Power Prediction and Optimization: An Intelligent EDA Approach. International Journal of Innovative Research in Computer Science and Technology 12 (4), p. 77–87. External Links: Document, Link Cited by: Background. Y. Wang, W. Ye, P. Guo, et al. (2025) SymRTLO: enhancing rtl code optimization with llms and neuron-inspired symbolic reasoning. In Advances in Neural Information Processing Systems, Cited by: Background. Z. Wang, J. Wang, Q. Yang, Y. Bai, X. Li, L. Chen, J. Hao, M. Yuan, B. Li, Y. Zhang, and F. Wu (2024b) Towards next-generation logic synthesis: a scalable neural circuit generation framework. In Advances in Neural Information Processing Systems, Cited by: Background. K. Xue, R. Chen, X. Lin, Y. Shi, S. Kai, S. Xu, and C. Qian (2024) Reinforcement learning policy as macro regulator rather than macro placer. In Proceedings of the 38th International Conference on Neural Information Processing Systems, Cited by: Background. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2023) ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), Cited by: Introduction. C. Yu and H. Ren (2026) Autonomous Evolution of EDA Tools: Multi-Agent Self-Evolved ABC. In 63rd ACM/IEEE Design Automation Conference (DAC ’26), External Links: Document, Link Cited by: Introduction, Related Work. Technical Appendices and Supplementary Material EDA Artifact Dataset and Artifact Details This section provides additional context for the EDA artifact dataset introduced in the dataset creation section. It complements the dataset creation process by characterizing what the dataset contains: diversity of collected designs in Figure 6 and artifact types associated with each design in Table 2. Figure 6 represents the distribution of design categories assigned during the design collection stage described in the design collection. The largest category consists of CPU/processor designs, reflecting the prevalence of open-source processor cores, such as RISC-V designs, in public RTL repositories. The remaining designs span across peripherals, RTL/IP blocks, GPU/graphics blocks, AI/ML accelerators, SoC integrations, interconnect & bus fabrics, cryptographic blocks, DSP/filter blocks, memory/cache designs, and miscellaneous building blocks. This distribution enables the dataset for EDA artifact analysis across varied RTL structures, design hierarchies, and synthesis outcomes. Table 2 summarizes the six artifact categories used in the dataset: (i) documentation, (i) source code, (i) synthesis scripts, (iv) synthesis logs, (v) netlists, and (vi) PPA reports. These artifacts capture complementary views of the same design. Documentation describes the design intent; source code defines the RTL implementation; synthesis scripts record tool invocations and setup; synthesis logs capture tool execution; netlists represent the synthesized gate-level implementation; and PPA reports summarize post-synthesis power, performance, and area estimates. These artifacts provide the evidence needed for grounded analysis of EDA artifacts. The EDA artifact dataset is useful for EDATracer because its artifacts are both heterogeneous and connected. They differ in format, including natural-language documentation, HDL source code, TCL scripts, textual logs, gate-level netlists, and tabular reports. Yet, they remain linked by the synthesis process: scripts configure tool execution, logs record what happened during the run, netlists capture the generated implementation, and PPA reports summarize the resulting design quality. This motivates the structured artifact representation described in the framework section, in which EDATracer can reason about related artifacts rather than treating each file as an isolated document. Figure 6: Distribution of chip designs in the dataset. Artifact Type Example Files Role in Chip Design Flow Use in EDA Artifact Analysis Documentation README.md *.md *.txt Describes design intent, supported features, configuration assumptions, integration notes, and expected behavior. Provides human-readable context for interpreting whether the synthesized design matches the intended functionality or configuration. Source code *.v *.sv *.vh *.svh Defines the register-transfer-level (RTL) chip design, including modules, datapaths, control logic, parameters, and structural connectivity. Helps connect tool messages, warnings, and reports back to the relevant modules, signals, parameters, or hierarchy in the original design. Synthesis script synthesis.tcl Specifies how the synthesis tool is invoked, including the top module, target library, constraints, optimization commands, and report generation steps. Provides the run configuration needed to interpret logs, netlists, and PPA reports in the context of the synthesis procedure. Synthesis log synthesis.log Records the synthesis run, including tool setup, loaded libraries, elaboration messages, warnings, errors, optimization steps, runtime, and final status. Serves as the main trace of what happened during synthesis and helps identify failures, unresolved references, missing constraints, and unexpected tool behavior. Netlist netlist.v Represents the synthesized gate-level implementation produced from the RTL using a target technology library. It is typically consumed by downstream physical design, equivalence checking, and gate-level simulation. Allows analysis to compare intended RTL structure with the synthesized implementation, detect unresolved black boxes, and inspect technology-mapped design structure. PPA reports area_report.txt power_report.txt timing_report.txt Summarize power, performance, and area outcomes after synthesis, including cell area, timing slack, critical paths, dynamic power, and leakage power. Provide quantitative evidence for design quality, enabling questions about timing closure, area growth, power behavior, and cross-library or cross-design comparisons. Table 2: Artifact types used in our EDA artifact dataset and their role in the chip design flow. Design Collection and Synthesis Artifact Generation This section expands on the four-phase collection stage described in the design collection. The process transforms public GitHub repositories into the curated design corpus =dr:r∈ℛ2,v(r)=1D=\d_r:r _2,\ v(r)=1\ from which downstream EDA artifacts are generated. Here, drd_r denotes the design entry derived from repository r after synthesis feasibility validation, so D contains validated designs rather than raw repositories. We provide detailed documentation of the filter logic, thresholds, and LLM evaluator configuration to ensure the methodology can be reproduced. Exact counts at each phase may differ across reruns because GitHub repository state (e.g., stars, archive status, and default-branch contents) evolves over time. Table 3: discovery keyword set K. The discovery phase queries the GitHub search API using K, organized into three groups: 2 language filters (language:Verilog, language:SystemVerilog) that scope the search to HDL content; 13 topic keywords (e.g., fpga, rtl, asic, risc-v, soc) targeting design-style descriptors; and 33 design-specific keywords spanning processors, peripheral controllers, accelerators, and cryptographic blocks. The initial candidate set is ℛ0=⋃k∈Search(k)R_0= _k Search(k) after URL-level deduplication. We intentionally made K broad to avoid biasing the dataset toward CPU-heavy repositories. The diverse keywords illustrated in Table 3 reveal the wide range of hardware categories in Figure 6. Table 4: heuristic predicate h(r)h(r). Heuristic filtering removes repositories that are unlikely to contain usable HDL designs without requiring LLM intervention. A repository passes h(r)h(r) if and only if every rule in Table 4 holds: the repository is not archived; its primary language is one of Verilog, SystemVerilog, or VHDL; it has at least 2 stars, with a manually curated set of seed repositories exempt from this rule to retain canonical designs; its total size is at least 10 KB; its most recent push is within ten years; non-trivial forks are retained only if the fork itself has at least five stars, filtering out passive fork copies; and neither the repository name nor description matches a case-insensitive regex flagging academic coursework (ρacademic _academic). The retained set is ℛ1=r∈ℛ0:h(r)=1R_1=\r _0:h(r)=1\. Table 5: LLM evaluator ℓ(r) (r). Each r∈ℛ1r _1 is evaluated by GPT-4.1-nano with temperature T=0.1T=0.1 and response format constrained to JSON, using a fixed prompt template. The evaluator input consists of repository metadata, including name, star count, primary language, and description; a README prefix; and a filtered listing of HDL-relevant files. The accept criteria, reject criteria, and output schema are summarized in Table 5. The acceptance rule is ℓ(r)=1 (r)=1 iff the returned decision is ACCEPT and the returned confidence is at least τconf=0.6 _conf=0.6, yielding ℛ2=r∈ℛ1:ℓ(r)=1R_2=\r _1: (r)=1\. The confidence threshold reduces borderline accepts where the LLM is uncertain about synthesizability. Failed API calls or malformed JSON responses are treated as rejected decisions. Table 6: synthesis feasibility check v(r)v(r). Although ℓ(r) (r) rules out non-design repositories, it does not guarantee that a repository has the structural properties required by the synthesis process. The predicate v(r)v(r) therefore checks for: (i) at least one non-testbench Verilog/SystemVerilog source file; (i) at least one parseable module ... endmodule block recovered by regex-based parsing of comment-stripped source; (i) at least one top-module candidate, defined as a module not instantiated by any other module, with all modules treated as candidates if every module is instantiated; and (iv) a recognizable clock input on the selected top module. Testbench files are excluded using both path components and filename patterns, as listed in Table 6. Clock-port detection matches top-module input ports against one of 15 regexes, with a default clk fallback when no match is found. The retained corpus is =dr:r∈ℛ2,v(r)=1D=\d_r:r _2,\ v(r)=1\. Algorithm 2: static design analysis and top-module ranking. Algorithm 2 formalizes the static analysis that v(r)v(r) relies on and that phase (i) of the generation stage consumes. Given the repository’s HDL file set ℱF, the procedure parses module declarations to extract names, port lists, and instantiations, and constructs the directed instantiation graph G=(ℳ,E)G=(M,E) with an edge mi→mjm_i→ m_j whenever mim_i instantiates mjm_j. The candidate root set ℛR is the set of modules with in-degree zero in G. Each candidate m∈ℛm is scored deterministically by the tuple sm=(¬bm,dm,lines(m),−name(m))s_m=( b_m,\,d_m,\,lines(m),\,-name(m)), where bmb_m flags testbench-like modules, dmd_m is the size of the transitive instantiation footprint, and ties are broken by line count and lexicographic name. The ranked candidate list T is returned in descending order of sms_m and used by the iterative repair loop. Filter funnel. From an initial |ℛ0|=23,115|R_0|=23,115 discovered candidates, heuristic filtering retained |ℛ1|=16,514|R_1|=16,514 (71.4%71.4\%); the LLM evaluator retained |ℛ2|=8,812|R_2|=8,812 (38.1%38.1\% of ℛ0R_0, 53.4%53.4\% of ℛ1R_1); and the synthesis feasibility check retained ||=2,787|D|=2,787 (12.1%12.1\% of ℛ0R_0, 31.6%31.6\% of ℛ2R_2). The funnel illustrates that each gate is non-redundant: h(r)h(r) primarily discards stale, non-HDL, or coursework repositories; ℓ(r) (r) primarily discards semantically borderline content such as software projects, verification-only collections, and EDA tooling that heuristic rules cannot identify; and v(r)v(r) primarily discards repositories whose module graphs are unbuildable, such as designs with no top-module candidate, no clock signal, or unparseable source. The process is layered so that each filter addresses a different failure mode. K controls the breadth of design categories; h(r)h(r) removes unmaintained or non-HDL repositories at near-zero cost; ℓ(r) (r) provides semantic judgment about design intent that rules cannot easily express; and v(r)v(r) enforces the structural preconditions required by the synthesis pipeline. The transitive-depth ranking from Algorithm 2 is also reused within the iterative repair loop: when synthesis of the highest-ranked module fails, the next-ranked candidate is tried before declaring the design unsynthesizable. This deterministic ranking allows the repair loop to explore alternative tops without additional LLM calls and makes the bound min(15, 2||) (15,\,2\,|T|) a meaningful repair budget rather than an unconstrained hyperparameter. Category Search Queries Language (2) language:Verilog, language:SystemVerilog Topic (13) fpga, rtl, asic, risc-v, riscv, processor, soc, ip-core, verilog, systemverilog, hdl, hardware-design, digital-design Keyword (33) synthesizable verilog, AXI, UART verilog, SPI verilog, I2C verilog, PCIe verilog, DDR controller, FIFO verilog, NoC router, cache controller verilog, ethernet verilog, USB verilog, JTAG verilog, DMA controller, interrupt controller, RISC-V core, ARM processor verilog, MIPS processor verilog, GPU verilog, DSP verilog, FPU verilog, crypto verilog AES, crypto verilog SHA, neural network accelerator verilog, CORDIC verilog, CRC verilog, PWM verilog, ADC interface verilog, memory controller verilog, bus arbiter verilog, wishbone, APB verilog Table 3: Keywords K used in Discovery phase. No. Rule Threshold (i) Repository not archived r.archived=Falser.archived=False (i) HDL primary language ∈Verilog,SystemVerilog∈\Verilog,SystemVerilog\ (i) Minimum stars ≥2≥ 2; seed repositories exempt (iv) Minimum repository size ≥10≥ 10 KB (v) Activity recency last push ≤10≤ 10 years (vi) Non-trivial fork if fork: stars ≥5≥ 5 (vii) Not academic coursework regex ρacademic _academic on name ∪ description Table 4: Heuristic rules constituting h(r)h(r). Item Description Model GPT-4.1-nano, temperature T=0.1T=0.1 Input fields Repository name, stars, language, description, README prefix, filtered HDL-relevant file tree Accept criteria Synthesizable RTL, IP cores, processors, SoCs, memory controllers, DSP/FPU blocks, crypto accelerators, communication interfaces, FPGA designs, hardware security modules Reject criteria Coursework, simulation-only testbenches, EDA scripts/tools, trivial designs, unmodified forks, documentation-only repositories, software projects, HLS source, board support packages, verification-only repositories with no DUT Output schema JSON object with decision, confidence, category, and reasoning Acceptance rule ℓ(r)=1 (r)=1 iff decision=ACCEPT and confidence ≥τconf=0.6≥ _conf=0.6 Table 5: LLM evaluation ℓ(r) (r) for design collection. Criterion Configuration Source file presence At least one Verilog/SystemVerilog file (.v, .sv) after testbench exclusion Testbench exclusion Drop files whose path contains any of tb, test, tests, sim, verification, uvm, cocotb, testbench, bench, or whose filename matches *_tb.s?v, tb_*.s?v, *_test.s?v, *_testbench.s?v Parseable modules At least one module … endmodule block recovered by regex-based parsing of comment-stripped source Top-module candidate At least one module not instantiated by any other; candidates ranked by transitive instantiation depth (descending). If all modules are instantiated, every module becomes a candidate Clock-port detection Top-module input port matching one of 15 regexes: clk, clock, i_clk, i_clock, sys_clk, aclk, pclk, hclk, mclk, core_clk, clk_i, wb_clk_i, wbclk, *_clk, clk_* (fallback to default clk if none match) Table 6: Synthesis feasibility check v(r)v(r) used to validate collected repositories. Algorithm 2 describes the static design analysis procedure used to rank candidate top modules. The iterative repair loop then applies deterministic fixes, including top-module changes, search-path extension, and clock selection, until synthesis succeeds, no useful candidates remain, or the attempt threshold is reached. Algorithm 2 Static Design Analysis and Top-Module Ranking 1:Repository files ℱF 2:Ranked top-module candidates T 3:←V← Verilog/SystemVerilog files in ℱF 4:ℳ←ParseModules()M← ParseModules(V) 5: Inputs: name, file, ports, lines, instantiations 6:Construct instantiation graph H=(ℳ,E)H=(M,E) 7: Edge mi→mjm_i→ m_j if mim_i instantiates mjm_j 8:ℛ←m∈ℳ:indegreeG(m)=0R←\m :indegree_G(m)=0\ 9: Candidate root modules 10:for all m∈ℛm do 11: dm←ReachableSize(H,m)d_m← ReachableSize(H,m) 12: bm←IsTestbenchLike(m)b_m← IsTestbenchLike(m) 13: sm←(¬bm,dm,lines(m),−name(m))s_m←( b_m,\ d_m,\ lines(m),\ -name(m)) 14:end for 15:←T← modules in ℛR sorted by sms_m descending 16:return T Knowledge Graph Construction This section explains the knowledge graph construction process. EDATracer represents each design as a typed graph with nodes for designs, artifacts, modules, warnings, reports, metrics, and tool stages. Edges encode artifact membership, chunk containment, generation provenance, module definitions, reported metrics, and message references. Algorithm 3 presents the concrete ingestion procedure for instantiating the offline representation introduced in the offline stage section. For each design d, the algorithm first creates a design-level anchor node and then partitions the ingestion set IdI_d based on the parser to be applied. Textual artifacts, including documentation, logs, reports, and synthesis scripts, are inserted as artifact nodes and then divided into overlapping chunks. Each chunk node vc(k)v_c^(k) stores graph metadata such as its source artifact and span, while its embedding zc(k)z_c^(k) is stored separately in the vector index as (vc(k),zc(k))(v_c^(k),z_c^(k)). This allows semantic retrieval to return compact text evidence while preserving a direct path back to the original design and artifact context in the graph. Structural artifacts, including RTL and netlist files, are handled differently. Rather than being represented only as text chunks, they are parsed into syntax-level subgraphs whose nodes and edges capture the artifact’s internal structure, such as modules, instances, signals, or netlist elements. These structural nodes are attached to their corresponding artifact nodes via containment edges. The final output is therefore a global graph G=(V,E)G=(V,E) and vector index Z, where text evidence can be retrieved through Z and then resolved through G to recover artifact hierarchy, design provenance, and structural context. Algorithm 3 Knowledge Graph Ingestion 1:Design dataset D, where each d has ingestion set ℐdI_d; chunk size W, overlap Ω 2:Attributed typed knowledge graph G=(V,E)G=(V,E) and graph-associated vector index Z 3:G←(V,E)←(∅,∅)G←(V,E)←( , ); ←∅Z← 4:for each design d∈d do 5: Create design node vdv_d with metadata ϕ(vd)φ(v_d) 6: V←V∪vdV← V∪\v_d\ 7: 8: ⊳ Artifact parsing 9: ℐdtext←I_d^text← documents, logs, reports, and scripts in ℐdI_d 10: ℐdstruct←I_d^struct← RTL and netlist files in ℐdI_d 11: 12: ⊳ Semantic indexing 13: for each text artifact a∈ℐdtexta _d^text do 14: Create artifact node vav_a with metadata ϕ(va)φ(v_a) 15: V←V∪vaV← V∪\v_a\ 16: E←E∪(vd,HAS_ARTIFACT,va)E← E∪\(v_d,HAS\_ARTIFACT,v_a)\ 17: na←⌈L(a)−ΩW−Ω⌉n_a← L(a)- W- 18: for n=1n=1 to nan_a do 19: ℓs(n)←1+(n−1)(W−Ω) _s^(n)← 1+(n-1)(W- ) 20: ℓe(n)←min(ℓs(n)+W−1,L(a)) _e^(n)← ( _s^(n)+W-1,L(a)) 21: Create chunk node vc(n)v_c^(n) with metadata ϕ(vc(n))φ(v_c^(n)) 22: zc(n)←Embed(vc(n))z_c^(n)← Embed(v_c^(n)) 23: ←∪(vc(n),zc(n))Z ∪\(v_c^(n),z_c^(n))\ 24: V←V∪vc(n)V← V∪\v_c^(n)\ 25: E←E∪(va,HAS_CHUNK,vc(n))E← E∪\(v_a,HAS\_CHUNK,v_c^(n))\ 26: end for 27: end for 28: 29: ⊳ Structural parsing 30: for each structural artifact s∈ℐdstructs _d^struct do 31: Create artifact node vsv_s with metadata ϕ(vs)φ(v_s) 32: V←V∪vsV← V∪\v_s\ 33: E←E∪(vd,HAS_ARTIFACT,vs)E← E∪\(v_d,HAS\_ARTIFACT,v_s)\ 34: Ts←ParseAST(s)T_s← ParseAST(s) 35: (Vs,Es)←BuildSyntaxGraph(Ts)(V_s,E_s)← BuildSyntaxGraph(T_s) 36: V←V∪VsV← V∪ V_s 37: E←E∪EsE← E∪ E_s 38: E←E∪(vs,CONTAINS,v):v∈VsE← E∪\(v_s,CONTAINS,v):v∈ V_s\ 39: end for 40:end for 41:return G=(V,E),G=(V,E),Z Creation of EDA Evaluation Benchmark This section describes the creation of the EDA artifact analysis QA benchmark, which is used to evaluate EDATracer against commercial agentic frameworks in the results section. The benchmark contains 9090 question-answer (QA) pairs organized along 2 axes: 3 question types (factual, statistical, and reasoning) and 3 difficulty levels (easy, medium, and hard), with 1010 QA pairs in each (type,level)(type,level) cell. Table 7 lists the criteria used to assign each level. The 3 question types evaluate deep analytical capabilities: factual questions test retrieval of explicit information from artifacts; statistical questions test retrieval combined with aggregation or numerical computation; and reasoning questions test grounded cross-artifact reasoning across logs, netlists, source files, reports, and other artifacts. The initial QA drafts were generated using Gemini 3.1 Pro, prompted with samples drawn from the curated EDA artifact dataset D. For each draft, the model generated a candidate question based on one or more artifacts from a specific design, along with a corresponding candidate answer. Using a large generative model instead of manually writing every question enabled a wider range of artifact types and phrasing styles than would have been possible with the same effort budget. However, the generated pairs were treated as initial drafts, with all final benchmark content determined by the authors, as described next. Each candidate QA pair was reviewed and modified by the authors using the actual EDA artifact dataset before being included in the benchmark. The authors performed three functions for each pair: (i) grounding: questions were rewritten to reference specific designs and artifacts in D, and any pair that could not be grounded was discarded; (i) golden-answer verification: answers were manually re-extracted by inspecting the cited artifacts, ensuring numeric values were sourced from PPA reports, log evidence was located in the corresponding synthesis log, and structural claims were verified against the netlist or source code until they matched the referenced artifacts; and (i) difficulty assignment: the authors assigned a (type,level)(type,\,level) label based on the modified question and its supporting evidence requirements, according to the criteria in Table 7, rather than carrying over any label suggested by Gemini 3.1 Pro. This protocol ensures that the final benchmark content and labels are author-verified and that all QA pairs are grounded in checkable artifacts from D. Type Level Criteria Factual Easy Single-file lookup Medium Multi-file look-up Hard Multi-step extraction Statistical Easy Basic arithmetic / comparison Medium Multiple statistical measures Hard Extracted trend / relationships Reasoning Easy Simple interpretation Medium Connecting 2 / 3 observations Hard Deep domain expertise Table 7: Evaluation benchmark question taxonomy. Human Expert Benchmark Evaluation Process To then evaluate the agent-generated results of this benchmark, we utilize manual scoring by human experts (as described in the experiments section). Table 8 provides the manual scoring rubric, detailing the intuition provided to reviewers behind the scoring scale (1-10) when assessing the agent response in relation to the golden solution. To mitigate evaluation bias and ensure fair evaluation, the reviewers are provided the responses from each agent/model configuration in a randomized order for each benchmark question, ensuring blindness to the model and agent origin. LLM Benchmark Evaluation Process To supplement the human expert evaluations, we additionally utilize the LLM-council evaluation method to score the agent responses. Within this process, two judge models (GPT-5.4 and Claude Sonnet-4.6) independently score each answer against the golden answer according to the same evaluation rubric 8. Then, a chairman model (Claude Opus-4.7) is provided with the context of both evaluations and determines the final score. Score Meaning 10 Excellent; fully correct; no errors. 8 Good; mostly correct, with minor gaps or imprecision. 6 Partial; captures part of the answer, but has significant gaps. 4 Weak; touches on the topic but is mostly wrong or missing key elements. 2 Wrong answer, but no hallucination. 0 Wrong answer with hallucination. Table 8: Scoring rubric for human expert evaluation. (a) Factual question-answer comparison of EDATracer v/s Cursor CLI. (b) Statistical question-answer comparison of EDATracer v/s Cursor CLI. (c) Reasoning question-answer comparison of EDATracer v/s Cursor CLI. Figure 7: Question-answer comparison of EDATracer v/s Cursor CLI across factual, statistical, and reasoning questions. Qualitative Examples This section presents 3 QA examples drawn from the QA benchmark (defined in Section Creation of EDA Evaluation Benchmark), comparing EDATracer against Cursor on 1 question from each category: factual, statistical, and reasoning. The examples illustrate 3 distinct failure modes that arise when retrieval is insufficiently grounded in the design-specific artifact structure. Figure 7(a) shows a factual question about the process node used for zipcpu synthesis. The correct answer is that the zipcpu design does not exist in the EDA artifact dataset. EDATracer returns this absence of design, whereas Cursor states that zipcpu used the ASAP7 7 nm process node. This example illustrates a grounding failure. When design-specific evidence is missing, the correct behavior is to say that the answer cannot be determined. EDATracer avoids this failure by first resolving the queried design and grounding the answer based on the retrieved artifacts for that design. Figure 7(b) shows a statistical question comparing timing slack between openhwgroup/cv32e40p and openhwgroup/cva5. The correct answer reports raw slack values of 7.357.35 and 9.399.39, with cva5 having the more positive slack. EDATracer retrieves the relevant timing reports and returns the comparison. Cursor fails to locate the timing report evidence, even though the artifacts are present in the dataset. This example illustrates a retrieval miss: the requested information exists, but the framework fails to surface the correct artifacts for the specified designs. EDATracer addresses this by resolving each design and retrieving the corresponding timing artifacts through the structured artifact representation. Figure 7(c) shows a reasoning question asking whether Design Compiler reported a valid timing path for the Vortex GPU design. The correct answer is nuanced: Design Compiler reports a max timing path with slack MET=8.93MET=8.93, but the report must be interpreted in the context of unresolved references and the synthesized scope. EDATracer captures both parts of the evidence: it identifies the reported timing path and also notes the limitations implied by unresolved design references. Cursor instead treats the unresolved references as sufficient to reject the reported path. This example illustrates a failure of reasoning in which real evidence is retrieved but weighted incorrectly. EDATracer handles the case better because it considers the timing report and synthesis-log evidence together rather than allowing one artifact to override the other. Thus, the three examples demonstrate that the key difference between EDATracer and Cursor is not merely whether relevant text is retrieved. The more challenging issues are whether the retrieved evidence aligns with the correct design, whether missing evidence is acknowledged, and whether multiple artifacts are interpreted in context with one another. In the factual example, we see how hallucination can occur due to missing evidence. The statistical example illustrates a retrieval miss, while the reasoning example reveals misjudged cross-artifact evidence. These are the precise instances where a typed, design-aware artifact representation proves beneficial: it provides the agent with a clear design reference, connects each design to its associated artifacts, and facilitates evidence collection across the different kinds of artifacts. Model Tool Human Expert Average Score LLM Council Score Token Usage Factual Statistical Reasoning Overall Factual Statistical Reasoning Overall Claude-Haiku-4.5 RAG 3.37 2.00 2.37 2.58 4.50 3.27 3.70 3.82 24.9M KG 8.83 9.53 7.67 8.68 8.73 9.43 7.17 8.44 36.4M GPT-5.4-mini RAG 3.20 0.20 1.0 1.47 4.53 3.00 2.60 3.19 14.5M KG 8.07 7.30 6.83 7.40 8.80 7.67 7.33 7.93 19.9M Table 9: Comparison between EDATracer using a RAG vs KG Model Tool Factual Statistical Reasoning Overall Tokens Avg. P@1 P@5 Avg. P@1 P@5 Avg. P@1 P@5 Avg. P@1 P@5 Claude Opus 4.7 Cursor 9.60 96.0 100 9.53 94.0 100 8.40 76.7 96.7 9.18 88.9 98.9 78.4M Claude Code 9.13 74.7 100 9.40 82.0 100 7.80 60.7 100 8.78 72.4 100 62.7M EDATracer 9.83 99.3 100 9.57 97.3 100 8.33 76.0 86.7 9.24 90.9 95.6 30.0M Claude Sonnet 4.6 Cursor 9.20 92.0 100 9.40 90.0 100 8.07 72.0 90.0 8.89 84.7 96.7 67.8M Claude Code 9.43 94.7 96.7 9.33 96.7 100 7.97 77.3 90.0 8.91 89.6 95.6 64.0M EDATracer 9.20 92.0 96.7 9.57 96.0 100 8.03 78.7 90.0 8.93 88.9 95.6 30.0M Claude Haiku 4.5 Cursor 9.60 96.7 100.0 8.47 90.7 100 7.33 61.3 76.7 8.47 77.8 92.2 84.6M Claude Code 9.57 92.0 96.7 9.07 91.3 96.7 7.33 60.7 83.3 8.66 81.3 92.2 115M EDATracer 8.73 86.0 93.3 9.43 94.7 100 7.17 59.3 76.7 8.44 80.0 90.0 36.4M GPT 5.4 Cursor 9.23 94.7 96.7 8.43 86.0 96.7 6.90 64.7 83.3 8.19 81.8 92.2 53.6M EDATracer 8.97 92.0 96.7 9.30 92.7 100 8.90 76.7 9.06 96.7 90.0 97.8 26.2M GPT 5.4 mini Cursor 8.80 88.7 96.7 7.67 83.3 100 7.33 58.7 90.0 7.93 77.8 95.6 59.3M EDATracer 8.00 67.3 86.7 7.97 79.3 86.7 6.77 52.0 83.3 7.58 71.1 85.6 19.9M Qwen 3.5 9B EDATracer 8.03 74.0 90.0 8.37 85.3 100 6.07 36.0 73.3 7.49 65.1 87.8 64.1M Table 10: Accuracy comparison between EDATracer and other agents using LLM-council scoring. Ablation Studies Database Formulation A central design decision in EDATracer is to structure EDA artifacts into a typed knowledge graph (KG) rather than a flat vector store used by standard retrieval-augmented generation (RAG). This ablation isolates the contribution of that choice by comparing the same underlying LLM against two retrieval methods: (i) a RAG database built over the same EDA artifact dataset, and (i) EDATracer’s KG described in the offline stage. The QA evaluation benchmark, scoring rubric, and prompts are held fixed; only the retrieval method changes. Results are reported in Table 9. The RAG baseline chunks every artifact in D using the same chunk size W and overlap Ω as those used by the KG semantic index. It also embeds the chunks using the same embedding model and retrieves the top-K chunks based on semantic similarity. Unlike the KG formulation, this baseline does not include typed nodes, artifact provenance, design hierarchy, or graph neighborhoods. The comparison, therefore, tests whether structured artifact relationships improve EDA artifact analysis beyond flat semantic retrieval. Table 9 shows that the KG formulation substantially improves accuracy across both LLM backbones. With Claude Haiku 4.5, the human expert overall score increases from 2.582.58 with RAG to 8.688.68 with KG, a +6.10+6.10 absolute gain and a 3.36× improvement. The LLM-council overall score increases from 3.823.82 to 8.448.44, a +4.62+4.62 absolute gain and a 2.21× improvement. With GPT-5.4-mini, the human expert overall score increases from 1.471.47 to 7.407.40, a +5.93+5.93 absolute gain and a 5.03× improvement, while the LLM-council overall score increases from 3.193.19 to 7.937.93, a +4.74+4.74 absolute gain and a 2.49× improvement. These gains appear across factual, statistical, and reasoning categories, with especially large improvements on statistical and reasoning questions where the system must connect PPA reports, logs, source files, and generated outputs. The KG formulation uses more tokens than the RAG baseline: 36.436.4M vs. 24.924.9M for Claude Haiku 4.5, an increase of 11.511.5M tokens or 1.46×, and 19.919.9M vs. 14.514.5M for GPT-5.4-mini, an increase of 5.45.4M tokens or 1.37×. This increase is expected because the KG database includes graph-resolved context, such as artifact metadata and neighboring evidence, rather than only the retrieved text chunks. However, the token increase is modest compared with the accuracy gain. Therefore, the results suggest that the added structured context is not solely extra prompt content; it provides useful grounding for cross-artifact analysis. Thus, the above ablation study supports the main design choice behind EDATracer. Flat RAG can retrieve locally relevant text, but it lacks an explicit mechanism to preserve which design, synthesis run, artifact type, or generated output a chunk belongs to. As a result, it can retrieve plausible but poorly scoped evidence. The KG formulation keeps retrieved evidence connected to its design context and provenance, allowing the agent to reason over related artifacts rather than isolated chunks. This advantage is especially important for smaller models, where the retrieval substrate plays a larger role in constraining the answer space. Benchmark Scoring Mechanism Here, we detail how the human expert evaluations of the EDA-evaluation benchmark described previously in the results compare against the LLM-Council evaluations across all agent configurations. Moreover, we provide the full LLM-Council results (average score and pass@1/@5) in Table 10 for the same set of agent responses. Additionally, we depict the net differences in each metric between the two strategies in Table 11. Overall, we find the following similarities between the LLM-Council and human expert scoring. EDATracer resulted in the highest average score in relation to other agents in the three large LLMs (Opus 4,7, Sonnet 4.6, and GPT-5.4) across both evaluation methodologies. However, we find that for smaller models (e.g., Haiku 4.5 and 5.4 mini), Cursor and/or Claude Code averages exceed EDATracer ’s LLM-Council results, in contrast to the human expert scoring for the Haiku 4.5 model. Furthermore, Table 11 demonstrates that LLM-Council frequently results in a net increase in average scores relative to human expert scoring, primarily in Cursor and Claude Code agents, demonstrating a positive score increase between (+0.10-1.36) out of 10 across all underlying LLMs, with the exception of Cursor with Claude Sonnet 4.6. In contrast, LLM-council average scores remain consistent or degrade for EDATracer, with net changes ranging between (-0.3 to +0.18). These trends in average score shifts also impact the downstream pass@1 and pass@5 rates, given the threshold for a “success” remains consistent at 8. For instance, in pass@1, we see EDATracer outperforms Claude Code and Cursor in 4 of 5 LLMs with human expert scoring, and 2 of 5 LLMs with LLM-Council. This difference highlights the impact of the threshold score used to delineate successes and failures. Model Tool Factual Statistical Reasoning Overall Avg. pass@1 pass@5 Avg. pass@1 pass@5 Avg. pass@1 pass@5 Avg. pass@1 pass@5 (/10) (p) (p) (/10) (p) (p) (/10) (p) (p) (/10) (p) (p) Claude Opus 4.7 Cursor 0.20 9.3 10.0 1.20 27.3 16.7 0.47 2.7 0.0 0.62 13.1 8.9 Claude Code 0.20 15.6 3.3 1.00 12.7 23.3 0.20 24.6 0.0 0.20 9.3 8.9 EDATracer 0.04 2.0 0.0 0.0 0.0 0.0 0.87 12.7 10.0 0.30 3.5 3.3 Claude Sonnet 4.6 Cursor 0.33 0.7 3.3 0.40 8.0 3.3 0.33 10.7 6.7 0.09 1.1 0.0 Claude Code 0.10 0.6 0.0 0.30 3.4 0.0 0.17 0.6 0.0 0.12 1.2 0.0 EDATracer 0.20 1.3 3.4 0.13 4.0 0.0 0.27 6.6 3.3 0.20 0.4 0.0 Claude Haiku 4.5 Cursor 0.10 0.0 0.0 0.80 21.4 3.3 0.23 0.6 0.0 0.31 2.2 1.1 Claude Code 0.87 0.7 0.0 1.67 14.6 10.0 1.53 12.7 10.0 1.36 9.3 6.6 EDATracer 0.10 0.7 0.0 0.10 0.0 0.0 0.50 0.7 3.3 0.24 0.0 1.1 GPT 5.4 Cursor 0.04 11.4 10.0 0.50 12.0 10.0 0.17 6.6 3.4 0.10 5.6 5.5 EDATracer 0.56 2.0 3.3 0.77 19.4 13.3 0.03 4.0 0.0 0.08 7.3 3.4 GPT 5.4 mini Cursor 0.40 8.7 3.4 0.74 26.0 30.0 0.23 3.4 6.7 0.45 13.6 13.4 EDATracer 0.07 2.7 6.6 0.67 20.0 16.7 0.06 8.0 6.7 0.18 8.0 1.2 Qwen 3.5 9B EDATracer 0.0 4.7 6.7 0.37 15.3 10.0 0.0 0.0 0.0 0.12 6.2 5.6 Table 11: Absolute divergence between LLM and human expert scoring results, computed as |LLM−humanexpert||LLM-humanexpert|. Pass@1 and pass@5 divergences are in percentage points (p). Darker purple indicates larger divergence. Limitations of Council of LLMs as a Judge When assessing the scoring discrepancies between the evaluation strategies, we find the following notable differences. In statistical questions, we find that the LLM-Council results in higher scores relative to human expert scoring, particularly within the Cursor and Claude-Code agents, with the average score increasing by (+0.3-1.67) across all underlying LLMs. In contrast, the results from EDATracer remained largely consistent with the human expert evaluations, with 3 of the 4 LLM configurations yielding equivalent or degraded scores. We find that this is largely due to the way the units’ correctness is assessed relative to the numerical value itself. For instance, LLM-Council was more inclined to award partial credit for a numerical value, even if the unit was either wrong or missing (e.g., Watts, meters, seconds), since the scalar value itself would still be consistent. In contrast, for unit inaccuracies, human experts were more likely to score harshly, given the incorrect nature of the response, regardless of the numerical similarity. Given these limitations in the LLM-Council scoring approach, we determine that the human expert results are most representative of the agent’s capabilities within the EDA benchmark tasks and are used in the paper’s primary findings.