Paper deep dive
Back to the Future: Rethinking EDA Infrastructure for Agentic Systems in Chip Design Verification
Je Yang, Ivan Lobov, Thomas Karpati
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The unprecedented computational scale of modern artificial intelligence depends on complex, multi-billion-transistor Systems-on-Chip, yet the workflows that verify these chips remain stubbornly manual. Although Large Language Models (LLMs) have made rapid inroads into Electronic Design Automation (EDA), approximately 74.6% of existing studies target static Register-Transfer Level (RTL) code generation, leaving post-simulation verification and interactive waveform debugging largely untouched. We introduce Back-to-the-Future (BTTF), an end-to-end agentic framework that closes this infrastructural gap. BTTF distills massive, unstructured simulation dumps into a normalized relational SQLite database and couples it with a collaborative multi-agent orchestration engine that translates natural-language verification queries into schema-aware SQL while correlating signal anomalies with versioned RTL repositories. Across a 150-query benchmark, BTTF attains 95.33% execution accuracy, charting a practical path toward autonomous EDA verification.
Tags
Links
- Source: https://arxiv.org/abs/2610.06790v1
- Canonical: https://arxiv.org/abs/2610.06790v1
Trouble viewing inline? Open PDF directly â
Full Text
22,902 characters extracted from source content.
Expand or collapse full text
AI for Chip Design Back to the Future: Rethinking EDA Infrastructure for Agentic Systems in Chip Design Verification Je Yang â thanks: Work done while interning at Google. Affiliation: Department of Computer Science Affiliation: Columbia University Affiliation: New York, NY, USA Email: je.yang@cs.columbia.edu Ivan Lobov Affiliation: Google DeepMind Affiliation: London, United Kingdom Email: lobov@google.com Thomas Karpati Affiliation: Google Affiliation: Mountain View, CA, USA Email: tkarpati@google.com Abstract The unprecedented computational scale of modern artificial intelligence depends on complex, multi-billion-transistor Systems-on-Chipâyet the workflows that verify these chips remain stubbornly manual. Although Large Language Models (LLMs) have made rapid inroads into Electronic Design Automation (EDA), approximately 74.6% of existing studies target static Register-Transfer Level (RTL) code generation, leaving post-simulation verification and interactive waveform debugging largely untouched. We introduce Back-to-the-Future (BTTF), an end-to-end agentic framework that closes this infrastructural gap. BTTF distills massive, unstructured simulation dumps into a normalized relational SQLite database and couples it with a collaborative multi-agent orchestration engine that translates natural-language verification queries into schema-aware SQL while correlating signal anomalies with versioned RTL repositories. Across a 150-query benchmark, BTTF attains 95.33% execution accuracy, charting a practical path toward autonomous EDA verification. 1 Introduction Sustaining the computational demands of modern AI beyond the end of Mooreâs law increasingly hinges on highly heterogeneous Systems-on-Chip (SoCs) Yang et al. (2026). As architectural complexity grows, design flows built on Hardware Description Languages (HDLs) become ever more labor-intensive, iterative, and error-prone Hu et al. (2025). Verification alone now consumes over 70% of the development cycle becoming the primary bottleneck of modern chip design Cadence Design Systems (2024), and functional bugs that slip into full-chip regression or post-silicon testing carry exponential financial and schedule penalties. Hence, automating register-transfer level (RTL) verification and analysis is an urgent, high-impact goal for the Electronic Design Automation (EDA) community. The rapid advances of Large Language Models (LLMs) has naturally inspired a wave of AI-assisted EDA research Kang et al. (2025); Ping et al. (2026). This wave, however, is strikingly lopsided. While approximately 74.6% of LLM-for-hardware studies concentrate on front-end RTL code generation and synthesis Abdollahi et al. (2024), post-simulation waveform debugging and dynamic verification remain largely underexplored. Since hardware development demands rigorous testing as much as design synthesis, this neglect leaves a substantial share of the automation opportunity untapped. Even the few LLM-aided verification workflows that do exist operate almost exclusively on static source code or toy-scale simulation logs, whereas interactive post-simulation debugging is a fundamentally harder problem. When a dynamic simulation fails, engineers manually scrub through terabytes of temporal dumps in graphical waveform viewers, tracing illegal transitions back to their origin in the HDL hierarchy. Frontier LLM agents are locked out of this loop by what we term Infrastructural Impedance. Raw, ASCII-based waveform dumps exceed model context windows by orders of magnitude, carry no structured semantic index, and expose no native, low-latency query interface that an autonomous agent could invoke. Here, we propose Back-to-the-Future (BTTF), an end-to-end agentic framework that enables natural-language querying for root-cause analysis and formal verification over complex RTL waveforms. Rather than forcing waveforms into an LLMâs context, BTTF revisits a classical EDA paradigm. It recasts unstructured temporal waveforms as a normalized relational SQLite database, then pairs this schema with a collaborative multi-agent engine that plans, queries, and audits on the engineerâs behalf. On 150 expert-annotated queries, BTTF reaches 95.33% overall accuracy, and its multi-agent design outscores a monolithic single-agent baseline by 17.3% on a five-criteria quality audit. 2 Related Work LLM-Driven Hardware Verification. LLMs have demonstrated remarkable capacity across EDA research, assisting developers with bug localization, code generation, and automated verification Hu et al. (2025); Kang et al. (2025); Ping et al. (2026); Zhao et al. (2025); Mastropaolo et al. (2023); Alvanaki et al. (2026). For complex verification tasks, state-of-the-art systems increasingly adopt multi-agent orchestration flows that split hardware synthesis into specialized roles for generation, testbench simulation, and formal counterexample refinement Ping et al. (2026); Zhao et al. (2025); Pan et al. (2026). Yet these frameworks operate predominantly on static source code, compile-time assertion lists, or toy-scale simulation logs, implicitly assuming an LLM can localize functional errors from static HDL traces or terse compiler messages alone. Key Limitation and Our Observation. The closest attempt to bridge this gap equips a coding agent with an Abstract Syntax Tree (AST)-based waveform tracing tool Ho et al. (2025), but such traces remain tied to small, code-centric examples rather than production dumps. BTTF instead builds on a complementary observation: SQL generation is an emergent capability of LLMs, rooted in their deep semantic mapping from natural language to structured code Gao et al. (2023). Once waveforms are made relational, decades of database machinery (e.g., indexing, aggregation, low-latency filtering) become directly accessible to language agents. 3 BTTF Framework Figure 1: Overview of BTTF framework. As Figure 1 illustrates, BTTF comprises two coupled subsystems on opposite sides of a single relational contract: (1) an offline Waveform Preprocessing Pipeline that compresses bulky simulation dumps into an optimized relational SQLite schema, and (2) an online Agentic Orchestration Engine that translates natural-language verification queries into schema-aware SQL and carries out the verification itself. 3.1 Waveform Preprocessing Pipeline Standard EDA simulators emit proprietary binary waveforms (.fsdb) or ASCII Value Change Dump (.vcd) files; for SoC-scale blocks, a raw VCD text file routinely exceeds 10 GBâfar beyond any prospect of direct context embedding. The BTTF pipeline therefore extracts the target hierarchies and normalizes them into a relational SQLite database (waves.db) built around two primary tables: ⢠signal_metadata captures the static signal declarations (RTL signal name and its variable type, etc.) with id. ⢠signal_changes records the temporal behavior as timestamped transitions as a (time, value) with a foreign key to refer metadata. Separating who a signal is from what it does lets agents reason over structure and time independently, joining the two only when a query demands it. A naĂŻve translation of every value change would simply move the scalability problem from the filesystem into the database. BTTF instead applies coverage-aware pruning during on-the-fly database generation: ⢠Clock pruning. High-frequency clock toggles dominate table size while carrying little debug information; since every row in signal_changes is timestamped, clock transitions can be skipped at insertion time without losing temporal anchoring. ⢠On-the-fly inactive-signal pruning (zero/avg). To cope with duplicate VCD identifiers and inactive nets, the ingestion engine maintains a lightweight hash map (vcd_id_to_db_id) alongside a per-signal transition counter (signal_change_count). This bookkeeping enables two graduated policies: dropping statically silent zero-toggle wires (zero), or additionally dropping nets whose toggle rate falls below the design-wide average (avg). ⢠Batched insertion. Row-by-row insertion incurs severe SQLite transaction overhead. Grouping transitions into batched writes removes this bottleneck at ingestion time. 3.2 LLM-Driven Agent Orchestration On top of the generated waves.db, BTTF conducts verification through a collaborative multi-agent architecture. The main Orchestration Agent decomposes a natural-language query into sub-goals, routes them to specialized agents with own toolsets, and audits the merged answer. ⢠Analysis Agent. Armed with schema-aware multi-query execution tools, it composes SQLite queries over the two tables to handle scope discovery, event inspection, and toggle counting. For complex tasks such as power hotspot detection, it leverages an iterative function-calling loop to execute multi-turn queries and aggregate multi-cycle signal statistics. ⢠Verification Agent. Tying waveform anomalies back to version-controlled source by evaluating SVA rules (ASSERT, ASSERT_STATIC, ASSERT_NEVER, ASSERT_KNOWN, COVER) through the regular expression (REGEX) assignment. ⢠Evaluation Agent. Before any root-cause analysis is returned, this agent audits candidate queries against five criteria: (1) correctnessâfiltering, aggregation, and join fidelity to user intent; (2) completenessâinclusion of all required columns and time conditions; (3) efficiencyâproper indexing and avoidance of redundant joins; (4) schema adherenceâstrict conformance to the two-table schema; and (5) context understandingâcorrect inference of implicit simulation timescales and hierarchy scopes. Under the orchestrator, agents communicate solely through a dynamic session context maintained across turns; together with a self-correction loop on failed queries, this shared-memory discipline lets each agent stay narrowly specialized without losing global coherence. Furthermore, when an assertion rule is violated even after self-correction, the agent autonomously isolates the exact violation timestamp and pinpoints the problematic sub-signals responsible for the failure (see Appendix A.1 for detailed root-cause diagnostics). 4 Results Experimental setup. All agents are served through the Gemini API with Gemini-2.5-Pro as the default model. Our benchmark spans multiple SoC design hierarchies of varying complexity (4.2 GB of VCD, 18,450 nets), from which we extract simulation windows ranging from narrow 50 ns high-resolution glitch-debug windows to 800 ns full-transaction simulation dumps. We curate 150 natural-language verification queries representative of daily engineering workflows, each paired with an expert-annotated golden answer. Preprocessing Performance. Pruning Strategy Footprint Reduc. Latency Coverage Fully Unpruned 1,420 MB 0.00% 410.5 ms 100.00% Clock Removed 480 MB 66.20% 125.2 ms 98.51% Zero-Toggle Pruned 395 MB 72.18% 94.8 ms 100.00% Below-Avg Pruned 185 MB 86.97% 41.6 ms 64.23% Table 1: Trade-off of waveform preprocessing across pruning strategies. Batch 1Batch 10Batch 50Batch 10000100100200200240.5240.5100.2100.285.485.483.283.2Time (s) Figure 2: DB generation time speedup with batch insertion. We first quantify the cost of turning a waveform into an agent-ready database. As Table 1 shows, suppressing clock insertion alone shrinks the footprint by 66.20% and cuts query latency by 3.28Ă3.28Ă while preserving 98.51% of functional toggle coverage. Additionally pruning untoggled signals pushes the reduction to 72.18%, while retaining full coverage over active nets. Aggressive below-average pruning compresses furthest but sacrifices 35.77% of coverage, marking the boundary of safe reduction. Relational schema optimization is thus not an implementation detail but a prerequisite for responsive agent interaction. Figure 2 confirms ingestion-side scalability. Batched insertion via SQL EXECUTEMANY accelerates database generation by up to 2.89Ă2.89Ă over single-row insertion, removing the I/O bottleneck for multi-terabyte waveforms. BTTF Framework Performance. Table 2: Comprehensive taxonomy of verification tasks, example natural language user queries, and proof-of-concept execution accuracy. Domain Task / Query Type Example Natural Language Query Accuracy Debugging Scope/Net Discovery âList all logics in the moduleA.â 100.00% RTL Attribution âFind the RTL file related to logicA.â 87.50% Value Inspection âWhat is the value of logicA at time timeA?â 95.00% Value Change Count âHow many times has logicA changed?â 100.00% Event Timing Check âWhen is net logicA first set to valueA?â 100.00% Analyzing Toggle Coverage âCalculate overall toggle coverage in jobA.db.â 91.67% Power Hotspot âGive me the 100ns window with max signal activities.â 91.67% Verification Static SVA Check âVerify ASSERT_STATIC(...).â 84.62% Dynamic SVA Check âRun rule ASSERT_KNOWN(...).â 90.91% Table 2 summarizes BTTFâs coverage across routine debugging, toggle-based performance analysis, and formal verification. Over the full 150-query taxonomy, BTTF achieves 95.33% overall accuracyânear-perfect on debugging primitives and consistently reliable on the harder analysis and verification tasks. A detailed breakdown of representative verification queries, agent tool invocations, and synthesized output responses across all task categories is provided in Appendix A.2 (Table 3). We further compare a monolithic single-agent design against our collaborative architecture, with an autorater judging outputs on the Evaluation Agentâs five criteria. While a monolithic single agent achieves faster raw response times, it frequently suffers from context clutter, schema hallucination, and degraded causal tracing on multi-hop verification tasks, scoring only 3.35/5.00 on the Evaluation audit. In contrast, BTTF cleanly decouples schema-aware SQL generation from RTL source inspection and evaluation, achieving a 17.3% higher reasoning quality score and near-perfect execution accuracy. This specialized delegation incurs an average 1.96Ă latency overhead, representing a practical and necessary trade-off to ensure robust, hallucination-free verification in production EDA workflows. 5 Discussion and Conclusion Our BTTF argues, and empirically supports, a simple thesis. The binding constraint on agentic waveform debugging is data infrastructure, not model capability. By recasting terabyte-scale waveforms into a compact, vendor-agnostic relational schema, BTTF turns an intractable context problem into a routine text-to-SQL problemâone at which modern LLMs already excelâand its orchestration engine converts that access into grounded, audited root-cause analysis. The 95.33% benchmark accuracy and 17.3% quality gain from agent specialization suggest that autonomous EDA verification will be reached through better interfaces between agents and design data rather than ever-larger models alone. To scale across larger full-chip designs, streaming ingestion, standardized scope-path normalization, and a waveform-to-RTL symbol index are the natural next steps; evaluating larger frontier reasoning models to tackle multi-hop causal tracing over these stores remains an exciting direction for future work. More broadly, the recipe behind BTTFânormalize an opaque EDA artifact into a queryable store, then let specialized agents operate over itâextends naturally beyond waveforms to regression logs, coverage databases, and power traces. We therefore view BTTF less as a point tool than as a template for rebuilding EDA infrastructure to meet agents halfway, and we hope it encourages the community to treat data interfaces as a first-class research object in AI-driven chip design. References [1] J. Yang, G. Wallace, J. Zuckerman, G. Tombesi, and L. P. Carloni (2026) A coherence-aware runtime reconfigurable system on chip for efficient language model inference. IEEE Micro 46 (3), p. 33â42. External Links: Document Cited by: §1. [2] Y. Hu, J. Ye, K. Xu, J. Sun, S. Zhang, X. Jiao, D. Pan, J. Zhou, N. Wang, W. Shan, et al. (2025) Uvllm: an automated universal RTL verification framework using LLMs. In 2025 62nd ACM/IEEE Design Automation Conference (DAC), Cited by: §1, §2. [3] Cadence Design Systems (2024) SoC verification: tools, methodologies, and why it matters. Note: https://w.cadence.com/en_US/home/explore/soc-verification.html Cited by: §1. [4] M. Kang, M. Liu, G. B. Hamad, S. M. Suhaib, and H. Ren (2025) Fveval: understanding language model capabilities in formal verification of digital hardware. In 2025 Design, Automation & Test in Europe Conference (DATE), p. 1â6. Cited by: §1, §2. [5] H. Ping, A. Bhattacharjee, P. Zhang, S. Li, W. Yang, A. Cheng, X. Zhang, J. Thomason, A. Jannesari, N. Ahmed, et al. (2026) Verimoa: a mixture-of-agents framework for spec-to-hdl generation. Proceedings of Machine Learning and Systems 8, p. 1277â1290. Cited by: §1, §2. [6] M. Abdollahi, S. F. Yeganli, M. Baharloo, and A. Baniasadi (2024) Hardware design and verification with large language models: a scoping review, challenges, and open issues. Electronics 14 (1), p. 120. Cited by: §1. [7] Y. Zhao, H. Zhang, H. Huang, Z. Yu, and J. Zhao (2025) Mage: a multi-agent engine for automated RTL code generation. In 2025 62nd ACM/IEEE Design Automation Conference (DAC), Cited by: §2. [8] A. Mastropaolo, L. Pascarella, E. Guglielmi, M. Ciniselli, S. Scalabrino, R. Oliveto, and G. Bavota (2023) On the robustness of code generation techniques: an empirical study on github copilot. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE), p. 2149â2160. Cited by: §2. [9] E. L. Alvanaki, J. Yang, B. Seyoum, and L. P. Carloni (2026) NoTB: oracle-free triage of LLM-generated RTL via cross-model formal consensus. External Links: 2608.21962, Link Cited by: §2. [10] Z. Pan, A. D. PATEL, Y. Shen, Z. Hu, Y. Guan, W. Li, L. Qin, Y. Wang, and Y. Ding (2026) KVflow: efficient prefix caching for accelerating LLM-based multi-agent workflows. Advances in Neural Information Processing Systems 38, p. 126246â126265. Cited by: §2. [11] C. Ho, H. Ren, and B. Khailany (2025) Verilogcoder: autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 300â307. Cited by: §2. [12] D. Gao, H. Wang, Y. Li, X. Sun, Y. Qian, B. Ding, and J. Zhou (2023) Text-to-sql empowered by large language models: a benchmark evaluation. arXiv preprint arXiv:2308.15363. Cited by: §2. Appendix A Appendix: Detailed Query Execution Workflow and Output Analysis A.1 Handling Assertion Failures and Root-Cause Diagnostics In standard simulation logs, assertion checkers typically emit binary pass/fail flags without pinpointing the originating nets. Within BTTF, when an assertion rule (such as ASSERT_NEVER or ASSERT_KNOWN) fails during dynamic checking, the agent automatically extracts the exact violation timestamp tviolationt_violation where the condition evaluates to true or where an invalid X/Z state is detected. It then initiates backward causal tracing by querying the prior values of all antecedent operands in the assertion expression at tâ¤tviolationt⤠t_violation. Consequently, the final diagnostic response explicitly highlights the specific sub-signal responsible for the failure (e.g., âAssertion failed at t=6929â nst=6929 ns due to net req_sel transitioning to unknown state x while clk was highâ), bridging the gap between high-level assertion monitoring and low-level root-cause debugging. Table 3: Detailed taxonomy of verification queries, agent tool invocations, and synthesized output responses from benchmark evaluation. Domain Task / Query Type Natural Language User Query Agent Action with Tool Invocation Generated Output Response Debugging Scope/Net Discovery âList all the nets in moduleA.â Action: find_signal_in_scope(db, "moduleA") Output: ["bank_id", "bank_addr"] (Found 2 struct field signals) Signal Scope Discovery âWhat is the scope name of the signal port_id?â Action: find_scope_by_signal_name(db, "port_id") Output: moduleA_ep_leaf, master, follower_id RTL Attribution âFind the RTL file related to signal signalA.cnt.â Action: find_source_module("soc_top", ".../moduleA/rtl/", "cnt") Output: design/.../moduleA.sv:L142 (logic [7:0] cnt) Value Inspection âWhat is the value of signal ctrl_status at time 7155316?â Action: get_value_of_signal_at_time(db, "ctrl_status", 7155316) Output: Value: 8184 (0x1F8 in hex) at t=7,155.316â nst=7,155.316 ns Value Change Count âWhat is the number of toggles of credit_stall?â Action: get_signal_toggle(db, "credit_stall", 6800000, 8000000) Output: 4 toggles across simulation window [680â ns,800â ns][680 ns,800 ns] Event Timing Check âWhen is the first time the signal req_accept[7] is set to 42?â Action: get_first_time_of_signal_value(db, "req_accept[7]", "42", 181.5us, 184us) Output: First transition detected at t=182,031.57â nst=182,031.57 ns (182031570 ps) Analyzing Toggle Coverage âWhat is the toggle coverage of the scope pmu_events?â Action: get_scope_toggle_coverage(db, "pmu_events") Output: 14.70% toggle coverage (21 active nets / 143 total nets in scope) Power Hotspot âGive me a time range with the maximum signal activity.â Action: Iterative rolling-window query across sliding 100 ns intervals Output: Peak activity window: [7612 ns, 9944 ns] with 27 total signal toggles Verification Static SVA Check âVerify rule ASSERT_STATIC(req_mem_width, ...).â Action: verify_static_rule(rule_name, macro_def, rtl_params) Output: PASSED (64 == (48 + $clog2(65536)) â 64 == 64, True) Dynamic SVA Check âRun the rule âASSERT_KNOWN(req_sel_is_known_A, req_sel, clk, rst).â Action: verify_known_rule(db, signal="req_sel", clk="clk", rst="rst_n") Output: PASSED (Zero X/Z unknown states observed on req_sel at clock edges) A.2 End-to-End Query-to-Response Pipeline Table 3 illustrates concrete examples of how the BTTF framework processes natural language user queries into structured tool executions and engineer-facing diagnostic answers. When an engineer submits a verification question (e.g., âList all the nets in moduleAâ), the Orchestration Agent routes the query to the Analysis Agent. The agent leverages schema context to map the natural language scope into a structured function invocation (find_signal_in_scope) or a relational SQL join across signal_metadata and changes. By executing deterministic queries over the indexed waves.db, the system retrieves exact timestamps, signal values in hexadecimal or decimal format, and transition counts without context window truncation or string hallucination. Furthermore, when queries span both temporal waveform behavior and design intent (e.g., âFind the RTL file related to signalA.cntâ), the agent correlates the resolved hierarchy with the versioned repository to extract exact line numbers and declaration semantics.