Paper deep dive
ChipCraftBrain: Validation-First RTL Generation via Multi-Agent Orchestration
Cagri Eryilmaz
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 98%
Last extracted: 4/26/2026, 4:48:05 PM
Summary
ChipCraftBrain is a validation-first RTL generation framework that utilizes adaptive multi-agent orchestration to automate the creation of synthesizable Verilog/SystemVerilog code. It combines a hybrid symbolic-neural architectureâusing algorithmic K-map solvers for deterministic tasksâwith a PPO-trained reinforcement learning policy that orchestrates six specialized LLM agents (Genius, Fast, Debug, Optimize, Waveform, and Testbench). The system features hierarchical specification decomposition for complex SoC designs and a knowledge-augmented RAG system. It demonstrates superior performance on the VerilogEval-Human (97.2% pass@1) and the industrial-scale NVIDIA CVDP benchmark (94.7% pass@1) compared to single-shot LLM baselines and existing multi-agent systems like MAGE.
Entities (9)
Relation Signals (5)
ChipCraftBrain â evaluatedon â NVIDIA CVDP
confidence 100% ¡ On a 302-problem non-agentic subset of CVDP... we reach 94.7% mean pass@1
ChipCraftBrain â implements â Quine-McCluskey minimization
confidence 100% ¡ An algorithmic K-map solver using Quine-McCluskey minimization
ChipCraftBrain â uses â PPO Policy
confidence 100% ¡ adaptive orchestration over six specialized agents via a PPO policy
Genius Agent â usesmodel â Claude Opus
confidence 100% ¡ The Genius agent uses the most capable model (Opus)
ChipCraftBrain â outperforms â MAGE
confidence 95% ¡ On VerilogEval-Human, ChipCraftBrain achieves 97.2% mean pass@1... ahead of MAGE (95.9%)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) show promise for generating Register-Transfer Level (RTL) code from natural language specifications, but single-shot generation achieves only 60-65% functional correctness on standard benchmarks. Multi-agent approaches such as MAGE reach 95.9% on VerilogEval yet remain untested on harder industrial benchmarks such as NVIDIA's CVDP, lack synthesis awareness, and incur high API costs. We present ChipCraftBrain, a framework combining symbolic-neural reasoning with adaptive multi-agent orchestration for automated RTL generation. Four innovations drive the system: (1) adaptive orchestration over six specialized agents via a PPO policy over a 168-dim state (an alternative world-model MPC planner is also evaluated); (2) a hybrid symbolic-neural architecture that solves K-map and truth-table problems algorithmically while specialized agents handle waveform timing and general RTL; (3) knowledge-augmented generation from a 321-pattern base plus 971 open-source reference implementations with focus-aware retrieval; and (4) hierarchical specification decomposition into dependency-ordered sub-modules with interface synchronization. On VerilogEval-Human, ChipCraftBrain achieves 97.2% mean pass@1 (range 96.15-98.72% across 7 runs, best 154/156), on par with ChipAgents (97.4%, self-reported) and ahead of MAGE (95.9%). On a 302-problem non-agentic subset of CVDP spanning five task categories, we reach 94.7% mean pass@1 (286/302, averaged over 3 runs), a 36-60 percentage-point lift per category over the published single-shot baseline; we additionally lead three of four categories shared with NVIDIA's ACE-RTL despite using roughly 30x fewer per-problem attempts. A RISC-V SoC case study demonstrates hierarchical decomposition generating 8/8 lint-passing modules (689 LOC) validated on FPGA, where monolithic generation fails entirely.
Tags
Links
- Source: https://arxiv.org/abs/2604.19856v1
- Canonical: https://arxiv.org/abs/2604.19856v1
Trouble viewing inline? Open PDF directly â
Full Text
85,740 characters extracted from source content.
Expand or collapse full text
ChipCraftBrain: Validation-First RTL Generation via Multi-Agent Orchestration Cagri Eryilmaz Abstract Large Language Models (LLMs) have shown promise for generating Register Transfer Level (RTL) code from natural language specifications, yet single-shot generation achieves only 60â65% functional correctness on standard benchmarks. Existing multi-agent approaches like MAGE improve pass rates to 95.9% on VerilogEval but remain untested on harder industrial benchmarks such as CVDP, lack synthesis awareness, and incur high API costs ($0.06/problem with 20 parallel Sonnet calls). We present ChipCraftBrain, a framework combining hybrid symbolic-neural reasoning with adaptive multi-agent orchestration for automated RTL generation. Four key innovations drive the system: (1) Adaptive multi-agent orchestration over six specialized agents using a trained PPO policy over a 168-dimensional state (with an alternative world-model MPC planner implemented and evaluated); (2) Hybrid symbolic-neural architecture that solves K-map and truth table problems algorithmically (zero cost, perfect accuracy) while using specialized agents for waveform timing analysis and general RTL generation; (3) Knowledge-augmented generation via a curated domain knowledge base of 321 patterns plus 971 open-source reference implementations with focus-aware retrieval; and (4) hierarchical specification decomposition that breaks complex multi-component designs into dependency-ordered sub-modules with interface synchronization. We evaluate across three complexity tiers. On VerilogEval-Human (156 simple modules), ChipCraftBrain achieves 98.7% pass@1 (154/156) on its best run (range 96.15â98.72% across 7 runs), placing it on par with or ahead of ChipAgents (97.4%, self-reported) and MAGE (95.9%) within measurement noise. On the 302-problem non-agentic, non-commercial code-generation subset of NVIDIAâs CVDP benchmark, spanning five task categories (RTL completion, spec-to-RTL, modification, linting, bug fixing), we achieve 94.7% pass@1 (286/302) under 5-iteration refinement with error feedback. We lead all five categories against the CVDP paperâs per-category single-shot baseline (Claude 3.7 Sonnet, 33.56% aggregate), with per-category lifts of 36â60 percentage points; we additionally lead three of four categories shared with NVIDIAâs recent ACE-RTL agentic system despite using roughly 30Ă fewer per-problem generation attempts. A RISC-V SoC case study demonstrates hierarchical decomposition generating 8/8 lint-passing modules (689 LOC), validated on FPGA hardware, where monolithic generation fails entirely. I Introduction I-A The RTL Generation Challenge The semiconductor industry faces a persistent bottleneck: the demand for custom chip designs far exceeds the supply of skilled RTL engineers. As system-on-chip (SoC) complexity grows, with modern designs containing billions of transistors across dozens of IP blocks, the time and expertise required for front-end RTL design has become a critical constraint on innovation. Recent advances in Large Language Models (LLMs) have opened a promising avenue: generating synthesizable Verilog or SystemVerilog code directly from natural language specifications. However, the gap between generating code and generating correct, synthesizable code remains substantial. Single-shot generation with state-of-the-art models achieves only 60â65% pass rates on the VerilogEval benchmark [1], far below the reliability threshold needed for production use. Moreover, existing evaluations concentrate on simple benchmarks. VerilogEval problems average 16 lines of code and 31 synthesis cells. Real-world IP blocks are far more complex: NVIDIAâs CVDP dataset [2] spans 783 industrial problems across 13 task categories (code completion, spec-to-RTL, modification, linting, bug fixing, testbench and assertion generation, and comprehension), and ChipBench [3] modules average 62 lines and 439 cells, 14Ă the gate count of VerilogEval. A system that excels on simple modules but fails on industrial designs has limited practical value. I-B Limitations of Current Approaches Several lines of work have attempted to close this gap, each with significant limitations: Single-shot generation approaches [4, 5, 6] fine-tune LLMs on Verilog corpora but provide no feedback mechanism. Errors in the generated code cannot be detected or corrected, resulting in pass rates of 45â78%. Iterative refinement systems [7] add error-feedback loops but use fixed retry strategies that do not adapt to the specific error patterns or design complexity of each problem. Multi-agent systems like MAGE [8] separate RTL generation, testbench creation, judging, and debugging into specialized agents. While achieving 95.9% on VerilogEval-Human, MAGE has three critical limitations: (a) no synthesis awareness in candidate scoring (only functional correctness), (b) primitive debug feedback (raw waveform dumps), and (c) high cost (20 parallel Claude Sonnet calls per problem at $0.06/problem). Crucially, while MAGE reports 37.4% on ChipBench, its performance on the broader CVDP benchmark has not been published. Workflow search approaches like VFlow [9] use Monte Carlo Tree Search to discover optimal agentic workflows, achieving 83.6% pass@1. However, the discovered workflow is static; it does not adapt per-problem based on design complexity or error patterns. I-C Our Contributions We present ChipCraftBrain, a framework that addresses these limitations through five key contributions: 1. Adaptive Multi-Agent Orchestration: Six specialized LLM agents (four RL-orchestrated, two rule-based) coordinated by a PPO policy operating on a 168-dimensional state representation that captures specification complexity, error patterns, code metrics, and generation history (SectionËI). 2. Hybrid Symbolic-Neural Architecture: An algorithmic K-map solver using Quine-McCluskey minimization [10, 11] handles deterministic problem classes at zero cost and perfect accuracy, while a specialized Waveform agent tackles temporal reasoning tasks (SectionËI). 3. Knowledge-Augmented Generation: A curated 321-entry domain knowledge base plus 971 open-source reference implementations with focus-aware RAG retrieval across five strategies, augmented by a spec-guidance registry of 59 detectorsâakin to the heuristic design-pattern knowledge accumulated by commercial EDA toolsâ that enrich specifications before generation with problem-type-specific technical briefs (SectionËI). 4. Hierarchical Specification Decomposition: Automatic detection and decomposition of complex multi-component designs into 4â8 dependency-ordered sub-modules with cross-module port synchronization, enabling generation of SoC-level designs where monolithic generation fails entirely (SectionËI). 5. Three-Benchmark Evaluation Across Complexity Tiers: We evaluate on VerilogEval (simple modules), CVDP (industrial IP), and ChipBench (hard accelerator designs), demonstrating where current AI-driven RTL generation excels and where it breaks down (SectionËVI). I-D Paper Organization The remainder of this paper is organized as follows. SectionËI surveys related work. SectionËI presents the system architecture, including multi-agent design, hybrid symbolic-neural components, RL orchestration, knowledge retrieval, hierarchical decomposition, and visible reasoning. SectionËIV details the validation pipeline. SectionËV defines the three-benchmark evaluation framework. SectionËVI reports experimental results and ablation studies. SectionËVII presents a RISC-V SoC case study with FPGA validation. SectionËVIII analyzes findings and limitations. SectionËIX outlines future work, and SectionËX concludes. I Related Work I-A LLM-Based RTL Generation The application of LLMs to hardware description language generation has progressed rapidly. VeriGen [4] demonstrated that models fine-tuned on Verilog corpora can generate syntactically valid code, though functional correctness remained limited. CodeV [5] improved on this with multi-level code summaries for training, achieving 77.6% on VerilogEval-Machine and 53.2% on VerilogEval-Human using a 33B parameter model. RTLCoder [6] explored instruction-tuned models for RTL, reaching 45â50%. ChipChat [12] applied GPT-4 in conversational chip design workflows. ChipNeMo [13] demonstrated that domain-adapted pretraining on hardware-design corpora improves downstream task performance but requires large labeled datasets. More recently, OriGen [14] enhances RTL generation through code-to-code augmentation and self-reflection; CraftRTL [15] targets correct-by-construction synthetic data including non-textual design representations; and ScaleRTL [16] scales post-training on a 1.7 M RTL reasoning corpus, setting the current SoTA among standalone RTL-specialized models. These approaches share a fundamental limitation: without a feedback loop, errors in the generated code cannot be detected or corrected, capping practical accuracy. I-B Multi-Agent Systems for RTL MAGE [8] introduced a four-agent architecture (RTL generator, testbench generator, judge, debugger) that achieves 95.9% on VerilogEval-Human v2. Three innovations drive its success: (a) high-temperature sampling (T=0.85T=0.85) with 20 candidates, (b) Verilog-state checkpoint debugging with textual waveform windows, and (c) multi-agent task separation that avoids context-switching confusion between synthesizable RTL and non-synthesizable testbenches. Their ablation shows the progression: vanilla LLM 72.4% â single-agent 83.9% â multi-agent 93.6%. However, MAGEâs scoring function considers only functional correctness (normalized mismatch count), ignoring synthesis quality metrics. Its debug feedback is raw signal dumps without structural analysis. And generating 20 candidates with Claude Sonnet incurs substantial cost (âź $0.06 per problem). Critically, MAGE has only been evaluated on VerilogEval; its performance on harder industrial benchmarks is unknown. ChipAgents [17] reports 97.4% on VerilogEval-v2 but has not published its methodology or evaluation protocol. As this figure is self-reported without peer review, direct comparison should be interpreted with caution. VerilogCoder [18] introduces a multi-agent framework with a task-and-circuit relation graph for fine-grained task decomposition and an AST-based waveform tracing tool for functional repair. Its agent design inspired subsequent agentic systems but depends on a custom Verilog parser that does not readily port to SystemVerilog designs used in modern industrial benchmarks. ACE-RTL [19] is NVIDIAâs own agentic RTL system, released concurrent with this work. It pairs a fine-tuned Qwen2.5-Coder-32B generator (1.7 M-sample RTL training corpus) with Claude 4 Sonnet as reflector and coordinator, and runs 5 parallel processes with up to 30 iterations each (up to 150 total attempts per problem). It is the strongest published baseline to date on the non-agentic CVDP subset, and we report direct head-to-head comparisons on the four categories shared with our evaluation in SectionËVI-C1. I-C Fast Inference and Reasoning Models Beyond larger autoregressive models, two recent threads complement pipeline-based RTL generation. Inceptionâs Mercury 2 [20] is a diffusion-based code LLM that performs all-token refinement in parallel; routed through our pipeline it achieves 82.7% pass@1 on VerilogEval-Human at âź 5 s/problem, illustrating that non-autoregressive backbones can drive the same iterative validation loop. Reasoning-optimized models such as DeepSeek-R1 [21] and instruction-tuned coders such as Qwen3-Coder [22] provide orthogonal gains on long-horizon tasks. Our system treats the LLM as a replaceable component, and the validation-first architecture benefits uniformly from improvements at the backbone layer. I-D Automated Workflow Discovery VFlow [9] applies Monte Carlo Tree Search (MCTS) to discover optimal agentic workflows for Verilog generation, achieving 83.6% pass@1. The discovered five-step process (analysis â generation â ensemble â testing â refinement) incorporates Icarus Verilog syntax checking and Yosys synthesis. However, the discovered workflow is static once found: it does not adapt per-problem to design complexity or error patterns. I-E RL for Code Generation Reinforcement learning has been applied to code generation in the software domain. AlphaCode [23] uses large-scale sampling and filtering for competitive programming. CodeRL [24] applies actor-critic methods for program synthesis with execution feedback. However, no prior work applies RL to orchestrate multi-agent hardware code generation, where the action space includes agent selection, temperature control, and knowledge retrieval strategy, a fundamentally different problem due to the synthesis and simulation constraints unique to RTL. I-F RTL Generation Benchmarks VerilogEval [1] provides 156 problems (average 16 LOC, 31 synthesis cells) and remains the most widely used RTL benchmark, but saturation above 95% limits its discriminative power. NVIDIAâs CVDP [2] is the most comprehensive industrial RTL benchmark to date, with 783 problems authored by NVIDIA hardware engineers across 13 task categories in two modes. The non-agentic mode provides 617 single-turn problems spanning code generation (cid002, cid003, cid004, cid007, cid016), testbench and assertion generation (cid012, cid013, cid014), and comprehension (cid006, cid008, cid009, cid010). The agentic mode provides 166 multi-file problems requiring Docker tool-use and iterative refinement. The best published results on the 302-problem non-agentic non-commercial code-generation subset are 33.56% pass@1 with Claude 3.7 Sonnet (single-shot, n=5n=5 samples) [2]; the agentic mode remains below 30%. Testbench, assertion, and commercial categories (cid012âcid014) require Cadence Xcelium and thus cannot be replicated with open-source tooling. ChipBench [3] provides 45 Verilog generation problems averaging 62 LOC and 439 cells, 3.8Ă longer and 14Ă more gates than VerilogEval. It includes three categories of increasing difficulty: self-contained modules (30), hierarchical designs requiring sub-module instantiation (6), and RISC-V CPU IP components (9). On ChipBench, the best multi-agent system (MAGE) achieves only 37.4% and the best single model (Claude Opus) achieves 30.7%, revealing the gap between benchmark performance and real-world design capability. I-G Hardware Design Automation Traditional EDA tools from Cadence (Cerebrus) and Synopsys (DSO.ai) apply ML to backend physical design optimization (place-and-route, PPA) [25, 26]. These are complementary to front-end RTL generation. The OpenLane 2 [27] flow with SKY130 PDK enables open-source RTL-to-GDSII synthesis, creating an opportunity for end-to-end AI-driven chip design pipelines. I System Architecture RL-Guided Iterative Loop (max 5 iterations)NL Specname, descriptionComplexityRouterHierarchicalDecomposition4â8 sub-modulesSingle-PassGenerationChipCraftBrainPPO Policy Ďθ _θ168-dim state â actionKnowledgeBase321 entriesFastSonnet, T=0.5GeniusOpus, T=0.7DebugOpus, T=0.3OptimizeSonnet, T=0.4Verilog CodeLintiverilog -t nullSimulateiverilog + vvpSynthesisYosys metricsVerified RTL+ Reasoning Trace⼠3 componentssimpleRAGpasspasserrors + reward Figure 1: ChipCraftBrain system overview. Natural language specifications are routed through complexity analysis to either hierarchical decomposition or single-pass generation. The RL policy selects agents, temperatures, and RAG strategies. Generated Verilog passes through a three-stage validation pipeline; structured error feedback drives iterative refinement. I-A System Overview ChipCraftBrain consists of six interconnected subsystems: (1) a multi-agent LLM system with six specialized agents (four RL-orchestrated, two rule-based), (2) a hybrid symbolic-neural system providing algorithmic solutions for K-map and truth table problems, (3) an RL orchestrator that adaptively selects agents and configures generation parameters, (4) a knowledge retrieval system providing domain-specific context via RAG, (5) a hierarchical decomposition engine for complex designs, and (6) a validation pipeline providing structured feedback. The generation pipeline employs a three-tier selection strategy: (1) algorithmic solving for K-map and truth table problems (zero LLM cost), (2) rule-based routing for specialized problems (waveform timing analysis), and (3) RL-orchestrated selection among general agents for all other RTL generation tasks. This hybrid architecture balances cost efficiency, domain specialization, and learned adaptation. I-B Multi-Agent LLM System RL Policy Ďθ _θ (PPO)Selects: agent â0..3â\0..3\, temperature â[0,1]â[0,1], RAG strategy â0..3â\0..3\, focus flagsGeniusOpus 4.6T=0.715 RAG chunksFastSonnet 4.6T=0.53 RAG chunksDebugOpus 4.6T=0.35 RAG chunksOptimizeSonnet 4.6T=0.44 RAG chunksFull contextPatterns onlyError-focusedSynthesis-focusedAgent 0Agent 1Agent 2Agent 3Senior architectQuick iterationsError diagnosisQuality tuning Figure 2: Multi-agent architecture. Four specialized LLM agents (plus Testbench and Waveform agents, not shown) serve distinct roles. The RL policy Ďθ _θ selects both the agent and its RAG retrieval strategy based on the 168-dimensional state vector. We employ six specialized agents, each with distinct prompts, model configurations, and retrieval strategies: TABLE I: Agent Specialization and Configuration Agent Model T RAG k Selection RL-Orchestrated (general-purpose): Genius Opus 4.6 0.7 15 RL policy Fast Sonnet 4.6 0.5 3 RL policy Debug Opus 4.6 0.3 5 RL policy Optimize Sonnet 4.6 0.4 4 RL policy Rule-Based (specialized): Waveform Opus 4.6 0.4 15 Keyword Testbench Opus 4.6 0.6 4 Manual/API Following the key insight from MAGE [8], each agent operates with an independent context window. Mixing RTL generation (synthesizable code) and testbench creation (non-synthesizable code) in the same conversation degrades both tasks. Our agents maintain separate system prompts with role-specific instructions, including Verilog coding guidelines, error patterns, and synthesis constraints. The four general agents (Genius, Fast, Debug, Optimize) are selected by the RL policy network based on problem characteristics and iteration state. The Genius agent uses the most capable model (Opus) with deep RAG context (15 chunks) for complex first attempts and novel designs. The Fast agent (Sonnet) uses minimal context for quick refinement iterations where latency matters. The Debug agent (Opus) specializes in error-focused repair with targeted RAG retrieval on failure patterns. The Optimize agent (Sonnet) focuses on synthesis quality improvement. Role-specific models for Opus and Sonnet were chosen empirically: Opus consistently outperforms Sonnet on error repair and initial generation, while Sonnet offers 2â3Ă faster iterations at comparable quality on well-specified refinement tasks. The two specialized agents are activated via deterministic heuristics: the Waveform agent is auto-selected when specifications contain temporal reasoning keywords, and the Testbench agent is invoked explicitly for test generation. I-C Hybrid Symbolic-Neural Architecture Before invoking LLMs, we employ symbolic reasoning for problem classes that admit deterministic solutions. I-C1 Algorithmic K-map Solver Problems requesting Karnaugh map minimization or truth table implementation are solved algorithmically using Quine-McCluskey minimization [10, 11]. The solver: ⢠Parses K-maps from natural language specifications (3â4 variables with Gray-coded columns) ⢠Extracts truth tables from ASCII tables or timing diagrams ⢠Computes minimal sum-of-products expressions ⢠Generates synthesizable Verilog-2001 code with appropriate bit-widths and signal declarations ⢠Handles donât-care conditions and detects XOR/XNOR patterns Detection uses keyword matching (âkarnaugh mapâ, âk-mapâ, âtruth tableâ) to route problems before LLM generation. On VerilogEval, this achieves perfect accuracy (3/3 K-map problems) with zero API cost and zero iterations. I-D Specialized Waveform Analysis Agent Timing waveform problems require precise temporal reasoning to infer circuit behavior from signal transitions. These problems pose unique challenges: (1) subtle wrap-around conditions in counters (modulo-N vs. bit-width overflow), (2) ambiguous parallel vs. pipeline sampling in dual-timing circuits (transparent latch + edge-triggered flip-flop), and (3) need for systematic trace analysis rather than pattern matching. We introduce a dedicated Waveform agent using Opus 4.6 with a specialized system prompt that enforces a structured six-step reasoning methodology: 1. Extract all transitions: Enumerate (input, output) pairs at each time step from the waveform 2. Identify pattern type: Classify as counter, state machine, or combinational logic 3. Find exact wrap points: For counters, determine modulo value from observed sequence (not inferred from bit-width) 4. Resolve temporal dependencies: For mixed latch+DFF circuits, identify signal flow (parallel sampling vs. pipeline chaining) 5. Derive logic: Build if/else or case structure matching the observed behavior 6. Verify: Check derived logic against multiple waveform samples This agent is auto-selected when specifications contain keywords âwaveformâ, âtiming diagramâ, or âdetermine what the circuit doesâ. I-E RL-Orchestrated Agent Selection State sts_t (168 dimensions)Per-spec identifierdeterministic, reproducibility onlyTask featurescomplexity, categoryStructural featuresprogress, errors, code, history, simPPO Policy Ďθâ(at|st) _θ(a_t|s_t)Actor-critic MLP (shared 2Ă256 trunk)Action ata_t (hybrid discrete-continuous)Discreteagent â0..3â\0..3\strategy â0..3â\0..3\ContinuousTâ[0,1]Tâ[0,1], kâ[3,20]kâ[3,20]Focus flagslint, timing, areathreshold = 0.5LLM Generation â Validation PipelineReward rtr_t r=wstageâ Îâstage+werrâ Îâerrors+wcostâ tokens+wsuccessâ â[pass]r=w_stage¡ +w_err¡ +w_cost¡tokens+w_success¡1[pass] st+1s_t+1 Figure 3: RL orchestration loop. The 168-dimensional state vector combines structural features (task metadata, generation progress, errors, code metrics, agent history, simulator signals) with a deterministic per-spec identifier used only for reproducibility. The PPO policy outputs a hybrid action selecting agent, RAG strategy, temperature, and focus flags. A multi-component reward drives learning from validation outcomes. For general RTL generation tasks (after algorithmic and rule-based routing), we employ reinforcement learning to adaptively select among four specialized agents and configure generation parameters. I-E1 State Representation We encode the generation state as a 168-dimensional vector combining forty structural features â task metadata (complexity, category); generation progress (iteration state and structural-refinement phase context); structured error signals; code-level metrics; agent-selection history; and simulator performance observations â with a deterministic per-spec identifier computed once from the specification text. The identifier provides reproducible policy behavior across machines and training runs; generalization across unseen specifications flows through the structural features. I-E2 Action Space The policy selects a hybrid discrete-continuous action: =(aagent,afocusâdiscrete,aT,atokens,arag,aretryâcontinuous)a=( a_agent,a_focus_discrete, a_T,a_tokens,a_rag,a_retry_continuous) (1) where aagentâa_agentâ\Genius, Fast, Debug, Optimize\, afocusâa_focusâ\full, minimal, error, synthesis, architecture\, aTâ[0,1]a_Tâ[0,1] (sampling temperature), atokensâ[0,1]a_tokensâ[0,1] (normalized token budget), aragâ[0,1]a_ragâ[0,1] (RAG depth), and aretryâ[0,1]a_retryâ[0,1] (retry budget allocation). I-E3 Reward Function The multi-component reward balances correctness, efficiency, and quality: R=Rterm+Reff+Rqual+RprogR=R_term+R_eff+R_qual+R_prog (2) where: Rterm=+100â sim pass,+60â lint pass,â50â total failureR_term=\+100 sim pass,+60 lint pass,-50 total failure\, Reff=+20â first try,â0.001/tokenR_eff=\+20 first try,-0.001/token\, Rqual=+10â low LUT,+15â timing metR_qual=\+10 low LUT,+15 timing met\, and Rprog=+5/stage advance,+3/error eliminatedR_prog=\+5/stage advance,+3/error eliminated\. I-E4 Policy Network The policy is an actor-critic network trained via Proximal Policy Optimization (PPO) [28]. A shared two-layer MLP trunk (each layer 256-wide with LayerNorm and ReLU) maps the 168-dimensional state to a 256-dim feature; three linear heads then emit the discrete-action logits, continuous-action parameters (sigmoid-bounded), and the scalar value estimate. Training uses GAE (Îť=0.95Îť=0.95, Îł=0.99Îł=0.99), the clipped surrogate objective, and epsilon-greedy exploration (Îľ0=0.3 _0=0.3, decay 0.9950.995/episode). The first 20 episodes use heuristic fallback for warm-start. I-E5 Training Regime The policy network is initialized through a two-phase procedure. During the first twenty episodes, action selection is governed by a rule-based heuristic that maps error type and iteration stage to agent, temperature, and retrieval configuration; these episodes populate an experience buffer and serve as a warm start. From episode twenty onward, online PPO updates are applied using the clipped surrogate objective with experience replay. The released checkpoint corresponds to a policy trained through fifty-seven episodes with the exploration rate decayed from Îľ0=0.3 _0=0.3 to Îľ=0.225 =0.225; the network architecture is as described in the Policy Network subsection above. Because the online training set overlaps with VerilogEval-Human, which also serves as an evaluation benchmark, pass-rate gains that are attributable to the trained policy cannot be cleanly separated from potential memorization of individual problems. We therefore regard the trained policy as a proof of concept rather than a generalization claim, and defer clean trainâtest separation to a held-out corpus of pattern-mined and synthetically-generated specifications disjoint from VerilogEval, CVDP, and ChipBench. This separation, together with a leave-one-out ablation against the heuristic warm-start policy, is reported as future work (SectionËIX). As an alternative to the learned PPO policy, the system supports a model-predictive-control (MPC) planner that rolls out sixty-four candidate action sequences over a three-step horizon against a trained RTL world model. On a controlled comparison over the 156 VerilogEval-Human problems, the MPC planner attains a pass rate of 97.4%97.4\% versus 96.8%96.8\% for the trained PPO policy under matched inference budgets, with a 1.8%1.8\% reduction in average iterations per problem and no observed regressions. The availability of two planners with similar end-to-end behavior on this benchmark supports the interpretation that the iterative validation loop and agent specialization contribute more to the reported accuracy than the specific planner used to route between agents. The headline result of 98.7%98.7\% reported in TableËV corresponds to the best of seven full-pipeline runs (range 96.1596.15â98.72%98.72\%); the planner comparison above is drawn from a single controlled run, and cross-run variance is discussed in SectionËVI. I-F Knowledge-Augmented Generation Query: spec + error context + agent focusFocus Strategy SelectorFullPatternsError-focusedSynthesisRTL Patterns291 entries8 categoriesArchitecture19 entriespipeline, SoCOptimization11 entriessynthesis rulesKeyword Scoring + Top-k Rankingkâ[3,20]kâ[3,20] controlled by RL actionRAG Context for LLM prompttitles + descriptions + code templates84 keywords Figure 4: Knowledge-augmented retrieval. The 321-entry knowledge base spans three sources (RTL patterns, architecture, optimization). The RL policy selects a focus strategy that weights sources differently; keyword-based scoring ranks candidates, and the top-k chunks (where k is an RL-controlled parameter) are injected into the LLM prompt. ChipCraftBrain maintains a multi-source knowledge system: Curated Knowledge Base. 321 entries spanning three categories: RTL design patterns (291 entries covering combinational logic, sequential circuits, FSMs, memory controllers, bus protocols, CDC, and DSP), architecture templates (19 entries for CPU cores, caches, interconnects, SoC integration), and optimization strategies (11 entries for area, timing, power, and FPGA techniques). Reference Implementation Library. Complementing the curated knowledge base, we assemble a secondary corpus of 971 indexed reference modules harvested from 34 open-source Verilog and SystemVerilog repositories, all permissively licensed and redistributed with attribution. Each module is parsed, syntax-checked with Icarus Verilog, and registered in a per-category index with its port list, parameterization, and a short natural-language synopsis; the indexer discards files that fail lint or exceed a size threshold before retrieval. The library is partitioned into fourteen hardware domains, listed in TableËI. Upstream sources include the Adapteva OpenHardware collection, Alex Forencichâs high-speed interface projects, the OpenRISC and RISC-V open-core ecosystems, and several widely-used cryptographic, arithmetic, and interconnect cores; no single source contributes more than one-sixth of the library. At retrieval time, the library is ranked by the same focus-aware strategies as the curated knowledge base and may be drawn from in place of, or alongside, the curated entries. TABLE I: Reference implementation library: entries per hardware domain. All modules are open source and were filtered for synthesizability before indexing. Domain N Domain N CPU building blocks 174 Memory 37 Datapath 165 Cryptographic cores 33 Protocol bridges 153 System infrastructure 33 Network / packet 98 Clock and reset 20 Peripheral controllers 78 SerDes / line coding 18 DSP 77 Arbitration 11 Flow control 65 Error correction 9 Total 971 Spec Guidance Registry. The registry enriches a specification with problem-type-specific technical guidance before generation. The entry point, enrich_spec(), is invoked once at the top of generate_module(), so every code path (CLI, benchmark runner, programmatic API, local-model backend) receives identical enrichment. Input is a raw natural-language spec; output is the same spec with one or more guidance blocks appended. The registry contains 59 detectors organized into five thematic bands: interface preservation for RTL modification tasks; classic RTL hazards (timing, latency, edge capture, waveform-to-logic); structural protocol patterns (APB, AXI-Stream, skid buffers, TLBs, interrupt controllers); microarchitecture idioms (NoC routers, systolic arrays, DRAM controllers, CPU-IP components); and machine-learning operator primitives. Detectors are evaluated in priority order, and enrichment is cumulative: every match appends its guidance block. Priority-sensitive dispatch is a first-class concern; for example, the APB-specific interrupt controller detector precedes the generic priority detector, and the attention detector precedes matmul and reduce, because attention specifications mention matrix multiplication but require additional invariants. Two detector classes coexist. Semantic detectors trigger on pattern conjunctions in the description (e.g., âstart bitâ and âstop bitâ implies a serial FSM; âneighbourâ or âneighborâ with a width indicator implies bit-neighbor combinational logic). Fixture-grounded detectors trigger on named IP families that the base model reliably fails on without structural scaffolding (e.g., APB interrupt controllers, Sony IR decoders, specific memory-allocator interfaces); targeted guidance for these families is more reliable than open-ended semantic heuristics. Each guidance block is a 10â200 line technical brief in imperative voice, covering timing invariants, handshake orderings, common pitfalls, and required code shape. A complementary learned gate operates above the registry. A feed-forward network (20-dimensional input, two hidden layers of 64 and 32 units, six-way sigmoid output) predicts which of six high-level guidance configurationsâminimal, FSM-only, protocol-focused, memory-focused, deterministic K-map, or full-stackâis most likely to succeed on a given problem. Input features combine task-type one-hot encoding, keyword flags, estimated complexity metrics, and historical pass-rate signals from similar problems. Training uses a warm-start on synthetic labels followed by fine-tuning on ensemble run outcomes. The gate acts as a coarse-grained configuration selector above the fine-grained registry. The registry was grown iteratively during VerilogEval, CVDP, and ChipBench diagnostic sessions. Its functional role is to encode the class of reusable RTL design-pattern knowledge that commercial EDA tools accumulate over decades of customer engagementsâ non-blocking assignment ordering in shift registers, reset polarity conventions, AXI handshake orderings, APB-specific interrupt dispatch, and the handful of microarchitecture idioms that recur across industrial specificationsâwhich the base model reliably mis-handles. Although the registry was developed and validated on benchmark problems, its entries target general pattern classes (e.g., APB handshake ordering, shift-register reset conventions, counter wrap-around semantics) rather than benchmark-specific solutions. Cross-benchmark transfer provides initial evidence of generalization: on CVDP cid002, the registry-enabled system attains 93.6%93.6\% against the 80.9%80.9\% reported for ACE-RTLâs specialized 32B fine-tune (TableËVIII), despite the majority of registry entries having been authored during VerilogEval diagnostic sessions on disjoint problems. External validity on unseen industrial specifications is not established by in-benchmark evaluation alone; a rigorous out-of-distribution evaluation on a held-out proprietary corpus is part of the evaluation program described in SectionËIX. In the observational ablation of TableËXI, the registry contributes approximately +4+4 p to VerilogEval-Human pass rate. The retrieval system employs focus-aware strategies: ⢠Comprehensive search for complex or unfamiliar specifications ⢠Pattern-focused search for simple module generation ⢠Error-focused search emphasizing debugging hints and common pitfalls ⢠Synthesis-focused search prioritizing optimization techniques ⢠Architecture-focused search for SoC and system-level designs Retrieved entries are scored by title match (+0.4+0.4), description match (+0.2+0.2), keyword relevance (+0.3+0.3), and template availability (+0.1+0.1), then injected into the agentâs system prompt. The RL policy selects the appropriate focus strategy based on problem type and iteration stage. I-G Hierarchical Specification Decomposition Complex Spec (⼠3 component keywords)âRISC-V SoC with pipeline, cache, UART, GPIOâŚâLLM Decomposition (Sonnet 4.6, T=0.2)Produces 4â8 sub-modules in dependency orderaluleafregfileleafdecoderleafpipelinemidicacheleafuartleafsoc_toptopGenerate left â right (leaves first, top last)Cross-Module Port Synchronization: extract headers â inject into subsequent promptsCombined Lint Validation (all modules concatenated) Complete Design: 8/8 lint-passing modules, 689 LOC vs. monolithic: 0/1 success on same spec Figure 5: Hierarchical specification decomposition. Complex specs with 3+ component keywords are decomposed into 4â8 sub-modules by a lightweight LLM call. Each module is generated in dependency order with cross-module port headers injected into subsequent prompts. On a RISC-V SoC, this produces 8/8 lint-passing modules (689 LOC), whereas monolithic generation fails entirely. For complex specifications containing 3 or more component keywords (e.g., âRISC-V SoC with cache, bus, UART, and GPIOâ), ChipCraftBrain automatically decomposes the design into sub-modules: 1. Complexity detection: Count component keywords from a predefined set (cpu, cache, bus, memory, uart, spi, i2c, gpio, arbiter, etc.). Trigger decomposition when count âĽ3⼠3. 2. LLM-driven decomposition: Prompt an LLM to produce 4â8 sub-module specifications with interfaces, dependencies, and descriptions in structured JSON. 3. Topological ordering: Sort sub-modules by dependency graph (leaves-first) to ensure each moduleâs dependencies are generated before it. 4. Iterative sub-module generation: Generate each sub-module individually, passing prior module headers as context to ensure port compatibility. 5. Cross-module synchronization: Extract module headers and inject as dependency code for lint validation. Remove duplicate module definitions that the LLM may produce. Hierarchical validation uses Verilator as the default backend (falling back to Icarus Verilog), with SystemVerilog constructs (logic, always_f, always_comb) enabled for sub-module generation when appropriate. I-H Visible Reasoning ChipCraftBrain provides a visible reasoning interface through structured thought streams during the design process. The system emits thoughts across nine categories: Analysis (architecture identification), Bottleneck (performance issues), Proposal (optimization suggestions), Retrieval (knowledge base queries), Generation (LLM calls and parameters), Validation (compile/simulate results), Decision (agent selection rationale), Error (failure diagnosis), and Progress (pipeline status). Each thought carries metadata including confidence scores, evidence links, and timestamps, enabling both real-time display and post-hoc analysis. Unlike black-box systems where the user receives only final output, this transparency enables engineers to inspect agent selection rationale, calibrate trust via confidence scores, identify knowledge gaps, and maintain a reproducible audit trail, particularly important in hardware design where errors can result in costly silicon respin. IV Validation Pipeline GeneratedVerilogStage 1Lintiverilog -t nullsyntax + typesStage 2Simulateiverilog + vvptestbench execStage 3SynthesisYosys metricsarea, cellsVerifiedRTLâSyntax errorsport mismatchesMismatch countfailing signalsCell countcritical pathStructured Error Feedback â next-iteration prompt + RL rewardCompile checkFunctional testQuality check Figure 6: Three-stage validation pipeline. Each stage provides structured error feedback that is (1) injected into the next LLM prompt as error context and (2) converted to a scalar RL reward. Stage progression (lint â simulate â synthesis) provides increasingly fine-grained validation. Iterative refinement with structured error feedback lifts pass rate by approximately 15 percentage points over single-shot generation (Table XI), consistent with the view that validation feedback is as important as generation quality. The validation-first philosophy is central to ChipCraftBrain. Rather than optimizing for single-shot generation quality alone, we invest in comprehensive validation and structured error feedback that enables iterative refinement. IV-A Three-Stage Verification Each generated module passes through three validation stages: Stage 1: Compile. The module is compiled with Icarus Verilog (iverilog) for single-module validation or Verilator for hierarchical multi-module designs. Verilator serves as the default backend for hierarchical generation, enabling SystemVerilog constructs (logic, always_f, always_comb) with fallback to Icarus Verilog for compatibility. This stage catches approximately 80% of errors before simulation. Stage 2: Simulate. If a testbench is available (pre-existing, LLM-generated, or from VerilogEval reference), the compiled design is simulated with vvp. The system detects multiple pass/fail formats: STATUS: PASS, count-based markers (N/N tests passed), and VerilogEvalâs mismatch format (Mismatches: 0 in N samples). Stage 3: Synthesis-Ready Check. The module is synthesized with Yosys to extract synthesis metrics (cell count, wire count) and verify that the design is synthesis-clean and free of unintended latches or combinational loops. IV-B Structured Error Feedback Unlike systems that pass raw error messages to the debug agent, ChipCraftBrain structures error feedback: ⢠Error categorization: Compile errors are classified (syntax, port mismatch, width mismatch, undeclared signal, etc.) ⢠Category-aware fix hints: Per-design-category hint databases map error patterns to likely fixes (e.g., for memory designs: âReplace logic with reg; use integer i; for (i=0; ...) instead of for (int i=0; ...)â) ⢠Source context: 5 lines around each error point are extracted for targeted debugging ⢠Error trend analysis: The state vector tracks whether errors are increasing, decreasing, or changing type across iterations IV-C Testbench Generation and Adaptation When no pre-existing testbench is available, the Testbench agent generates one with specific requirements: proper reset sequence, 10ns clock period, 5+ test scenarios, and standardized status markers. The TestbenchAdapter wraps generated modules for compatibility with VerilogEvalâs TopModule interface, enabling reference comparison against golden RefModule implementations. Testbenches always use Verilog-2001 syntax for maximum tool compatibility, even when the design under test uses SystemVerilog constructs. V Evaluation Framework We evaluate ChipCraftBrain across three benchmarks spanning simple modules, industrial IP, and hard accelerator designs. This multi-tier evaluation reveals where current AI-driven RTL generation excels and where it breaks down. V-A Benchmark Suite TABLE I: Three-Benchmark Evaluation Suite. See Sec. V for the CVDP subset definition. Benchmark Problems Avg LOC Avg Cells Testbench VerilogEval-Human 156 16 31 RefModule CVDPâ 302 43 n/a cocotb ChipBench 45 62 439 Verilator â Non-agentic, non-commercial code-generation subset of CVDP (cid002/003/004/007/016); 302 of 783 total problems. The remaining 481 problems lie outside our pipelineâs design scope: testbench/assertion generation requires Cadence Xcelium (cid012/013/014, 187 problems), agentic multi-file tasks require a Docker tool-use harness (166), and comprehension tasks are scored by BLEU or LLM-as-judge (cid006/008/009/010, 123). VerilogEval-Human [1] provides 156 problems with English-language prompts and golden reference implementations. Problems span six categories: combinational, sequential, FSM, memory, bus, and processor. As the most widely used RTL generation benchmark (SOTA >>95%), it enables direct comparison with prior work but its simplicity (average 16 LOC, 31 cells) limits discriminative power. CVDP [2] is NVIDIAâs Chip Verification and Design Problems benchmark (version 1.0.2), containing 783 problems across 13 task categories in two modes (617 non-agentic, 166 agentic). We evaluate exclusively on the 302-problem non-agentic, non-commercial code-generation subset: RTL code completion (cid002, 94 problems), spec-to-RTL translation (cid003, 78), RTL modification (cid004, 55), linting and QoR optimization (cid007, 40), and RTL bug fixing (cid016, 35). Each problem embeds a complete cocotb test harness, so functional evaluation runs locally with Icarus Verilog and Verilator, without Docker. We explicitly do not evaluate four other problem classes, all of which fall outside our pipelineâs design scope: ⢠Testbench and assertion generation (cid012/013/014, 187 non-agentic problems): these problems require the commercial Cadence Xcelium simulator for harness evaluation. ⢠Agentic multi-file tasks (166 problems across cid003/004/005/012/013/014/016): these require a Docker-based agent runtime capable of multi-step file editing and tool use, which our single-turn pipeline does not currently implement. ⢠Module reuse (cid005, 26 agentic-only problems): no non-agentic variant exists. ⢠Comprehension (cid006/008/009/010, 123 non-agentic problems): these are BLEU-scored text or LLM-as-judge Q&A tasks that measure reading comprehension rather than code generation. The best published result on the same 302-problem subset is 33.56% pass@1 (Claude 3.7 Sonnet, single-shot, n=5n=5 samples). A direct comparison therefore measures the contribution of iterative validation and multi-agent orchestration on top of a strong base model, rather than raw model capability in isolation. ChipBench [3] provides 45 Verilog generation problems designed to expose the gap between benchmark and real-world performance. It contains three categories of increasing difficulty: self-contained modules (30 problems: FSMs, counters, multipliers, pattern detectors), non-self-contained hierarchical designs (6 problems: async FIFO, 8-bit ALU, CPU top module requiring sub-module instantiation), and CPU IP components (9 problems: real RISC-V ALU, controller, register file, branch unit, division unit). ChipBench modules are 3.8Ă longer and have 14Ă more synthesis cells than VerilogEval, with the CPU IP category averaging 862 cells. V-B Complexity Comparison TABLE IV: Benchmark Complexity Comparison Benchmark Avg LOC Avg Cells Sub-modules Published SOTA VerilogEval 16 31 0 97.4% (ChipAgents) ChipBench 62 439 0.25 37.4% (MAGE) The 14Ă gate-count gap between VerilogEval (31 cells) and ChipBench (439 cells) explains why systems scoring >>95% on VerilogEval can score <<40% on ChipBench: simple modules require only basic code generation, while complex designs demand architectural understanding, multi-signal coordination, and correct protocol implementation. V-C Evaluation Metrics Our primary metric is pass@1: the fraction of problems solved within the iterative refinement budget (up to 5 iterations), counting each problem as a single attempt. For CVDP, this metric is directly comparable to the Agentic Pass Rate (APR) defined by ACE-RTL [19] as the ratio of unique solved problems to total problems across independent runs with refinement; both measure âdid the system eventually solve the problem with its given feedback budget.â This differs from the pass@1 used for standalone models, which samples n independent single-shot attempts and averages. We also report: ⢠Per-category breakdown: Success rates by design type and difficulty tier ⢠Iterations to success: Average refinement iterations needed ⢠Cost per problem: Estimated API cost (USD) V-D Baselines We compare against published results: CodeV [5] (53.2% VerilogEval-Human), MAGE [8] (95.9% VerilogEval, 37.4% ChipBench), VFlow [9] (83.6% VerilogEval), ChipAgents [17] (97.4% VerilogEval, self-reported), and single-shot GPT-4 (âź 63%). For ChipBench, we additionally compare against Claude Opus single-model (30.7%) as reported in [3]. VI Experiments and Results VI-A Experimental Setup Hardware: Benchmark orchestration runs on a dual-GPU workstation (2Ă NVIDIA RTX 3090, 48 GB combined VRAM); the LLM calls are served remotely via the Anthropic API, so GPU resources are used only for EDA tooling, local-model baselines, and parallel test harnesses. Models: Claude Opus 4.6 (Genius, Debug, Testbench, Waveform agents) and Claude Sonnet 4.6 (Fast, Optimize agents). EDA Tools: Icarus Verilog 12.0 for compilation and simulation, Verilator [29] for hierarchical and SystemVerilog validation, Yosys 0.40 for synthesis checks, cocotb for CVDP test harnesses. Benchmarks: VerilogEval-Human (156 problems), CVDP non-agentic non-commercial code-generation subset (302 problems across cid002/003/004/007/016), ChipBench (45 problems). Reproducibility: All results use the chipcraftx benchmark CLI against the published CVDP dataset (cvdp_nonagentic_code_generation_no_commercial_eval.json) at commit 9c6269ed of the ChipCraftBrain repository. Refinement budget is fixed at 5 iterations per problem for all benchmarks. Run-to-run variance. Temperature-dependent sampling introduces measurable variance across repeated runs. On VerilogEval-Human across seven independent runs spanning 2026-02 to 2026-04 (Claude Opus/Sonnet 4.6 as accessed in early 2026), pass rates ranged from 96.15% to 98.72% (150/156 to 154/156, mean â97.2%â 97.2\%, std â 1.0 p). The 98.7% figure reported in TableËV is the best of these runs and is within the acknowledged variance envelope of ChipAgents (97.4%) and MAGE (95.9%); we therefore frame ChipCraftBrain as âon par with or ahead ofâ the published state of the art rather than strictly superior. CVDP numbers are averaged over 3 independent runs (22,090 s wall-clock each); ChipBench is single-run, with expected variance of 1â2 p consistent with the VerilogEval observation. VI-B VerilogEval Results TABLE V: VerilogEval-Human comparison. ChipCraftBrain reports mean and range across 7 independent runs; prior systems report single-run self-reported numbers. System Pass@1 Problems GPT-4 (single-shot) âź 63% 98/156 CodeV-DS-33B 53.2% 83/156 VFlow 83.6% 130/156 MAGE 95.9% 150/156 ChipAgents 97.4% 152/156 ChipCraftBrain (ours) 97.2% (mean, n=7n=7, range 96.15â98.72%) 150â154/156 ChipCraftBrain achieves a mean pass rate of 97.2% across 7 independent runs (range 96.15â98.72%, best 154/156) on VerilogEval-Human, on par with ChipAgents (97.4%, self-reported single run) and ahead of MAGE (95.9%) within measurement noise. On the best run, 133 of 154 passing problems pass on the first iteration, 12 require a second refinement pass, and 3 require 3â4 iterations. Six problems are solved by the symbolic K-map solver with zero LLM cost. The two remaining failures reveal distinct failure modes. Prob066_edgecapture compiles successfully on every iteration but produces 2 simulation mismatches per attempt; the generated logic differs from the reference in reset assignment syntax, exposing a simulator-specific timing edge case that the iterative loop cannot resolve. Prob092_gatesv100 fails compilation because the agent consistently selects a generate-block approach for 100-bit neighbor logic, which conflicts with Icarus Verilogâs strict Verilog-2001 mode. Both failures are architectural (wrong HDL construct choice) rather than minor coding errors. Total wall-clock time for the 156-problem suite is 35.5 minutes (average 13.0s per problem). VI-C CVDP Results TABLE VI: CVDP Results by Task Category (302-problem non-agentic non-commercial code-generation subset). Our numbers are averaged over 3 independent runs. Baseline is Claude 3.7 Sonnet single-shot pass@1 from n=5n=5 samples [2]. Code Task N Baseline Ours cid002 RTL Code Completion 94 34% 93.6% cid003 Spec-to-RTL Translation 78 48% 96.2% cid004 RTL Code Modification 55 45% 96.4% cid007 Linting / QoR 40 44% 97.5% cid016 RTL Bug Fixing 35 53% 88.6% Overall (302 problems) 33.56% 94.7% TABLE VII: CVDP Results by Difficulty (302-problem subset). Hard problems are defined only in the agentic CVDP mode and are excluded from this subset by construction. Difficulty Pass@1 Problems Easy 96.9% 157/162 Medium 92.1% 129/140 Overall 94.7% 286/302 On the 302-problem CVDP non-agentic non-commercial code-generation subset, ChipCraftBrain achieves 94.7% mean pass@1 (286/302, averaged over 3 runs), versus 33.56% for the best published single-shot baseline on the same subset [2]. The comparison is protocol-aware, not apples-to-apples. The baseline measures raw model capability: Claude 3.7 Sonnet is sampled n=5n=5 times per problem, with pass@1 computed from unconditional single-shot attempts and no error feedback. Our system measures full pipeline capability: a single attempt per problem, but with up to 5 iterations of structured error feedback, multi-agent orchestration, RAG-augmented prompts, and spec-guidance enrichment. The correct interpretation of the 61-point gap is therefore âvalidation-first iterative refinement plus orchestration adds 61 p on top of a strong base model,â not âour LLM is 2.8Ă better.â We do not report a head-to-head single-shot comparison because our pipelineâs raw Genius-only pass@1 on this subset was not separately measured in a matched-protocol run; we identify this as future work. Per-category behavior is consistent with the design scope: RTL completion (cid002) shows the lowest score (93.6%) because skeleton completions have the narrowest admissible solution space, and bug fixing (cid016) is second-lowest (88.6%) because multi-bug patches can cascade into previously-passing logic. Spec-to-RTL (cid003) and modification (cid004) benefit most from our spec-guidance registry and hit 96.2â96.4%. Linting/QoR (cid007, 97.5%) is the highest because our optimization-focused RAG strategy aligns well with this categoryâs goals. The absolute lift over single-shot baseline ranges from 36 p (cid016) to 60 p (cid002), showing that simpler categories (where the baseline already scores reasonably) still benefit most from iterative validation. This result demonstrates that the techniques driving VerilogEval performance, iterative validation, multi-agent orchestration, and knowledge-augmented generation, transfer effectively to industrial designs of significantly greater complexity, with a clear caveat that 481 CVDP problems outside this subset remain unevaluated (SectionËV). VI-C1 CVDP Leaderboard: ChipCraftBrain in Context To place our result in the broader landscape, we compare against the 14 baselines reported in ACE-RTLâs CVDP-v1.0.2 evaluation [19], spanning frontier proprietary models, frontier open-source models, RTL-specialized fine-tuned models, and agentic systems. All baseline numbers are copied verbatim from the ACE-RTL paperâs Table 1 (APR; Agentic Pass Rate = unique solved / total across five independent runs). Our ChipCraftBrain column reports pass rate after up to 5 iterations of structured error feedback, a metric directly comparable to APR for agentic systems. TABLE VIII: CVDP leaderboard on shared categories (cid002/003/004/016, 262 problems). Two protocols are shown and must not be compared directly: agentic systems (top block) use iterative refinement with error feedback; standalone models (bottom block) report single-shot pass@1 with no error feedback. Baseline values are APR (%) from ACE-RTL [19] Tab. 1; ChipCraftBrain is mean pass rate over 3 runs under its 5-iteration protocol. ACE-RTLâs specialized generator required âź 10,000 A100 GPU-hours to train; our system requires no RTL-specific training. Category System cid002 cid003 cid004 cid016 Protocol A: iterative refinement with error feedback (up to 5 iterations) Ours and prior agentic systems Adaptive pipeline ChipCraftBrain (5 iter, mean of 3 runs) 93.6 96.2 96.4 88.6 Full system ACE-RTL [19] 80.9 96.2 90.9 91.4 Claude 4 generator variant ACE-RTL w/ Claude 4 [19] 80.9 89.7 81.8 88.6 Multi-agent MAGE + Claude 4 [8, 19] 46.8 55.1 70.9 62.9 Protocol B: single-shot pass@1, n=5n=5 samples, no error feedback Frontier proprietary standalone models Reasoning GPT-5 [19] 42.9 51.3 54.3 60.0 Reasoning Claude 4 Sonnet [19] 43.6 49.1 51.4 54.3 Reasoning o4-mini [19] 41.5 44.4 50.0 58.8 Frontier open-source standalone models MoE reasoning DeepSeek-R1 [21] 36.7 41.8 34.9 40.0 MoE DeepSeek-v3.1 37.5 43.6 48.6 51.4 Code MoE Qwen3-Coder-480B [22] 35.3 41.8 39.4 42.9 General Llama4-Maverick 36.4 38.2 36.0 37.1 MoE Kimi-K2 29.1 32.7 29.7 31.4 RTL-specialized fine-tuned standalone models 32B ScaleRTL [16] 33.3 30.9 37.1 37.1 15B CraftRTL [15] 18.0 16.4 5.1 8.6 7B OriGen [14] 21.8 16.4 7.7 11.4 7B CodeV [5] 7.7 0.0 0.0 0.0 7B RTLCoder-v1.1 [6] 5.4 1.8 0.0 2.9 ChipCraftBrain ranks first in three of the four shared categories (cid002, cid003 tied, cid004) and ranks second on cid016, losing only to ACE-RTLâs full system by 2.8 p. Relative to the broader field of standalone models (frontier proprietary, frontier open-source, RTL-specialized), the gap is considerably larger: the strongest single-shot baseline (GPT-5) reaches 60% on cid016 and ⤠55% elsewhere, while ChipCraftBrain is 88.6â96.4% across all four categories. Two observations are worth highlighting. First, ACE-RTLâs improvement over its Claude-only Generator variant (on cid002: 80.9 vs. a 39.4% Claude 4 Sonnet pass@1 baseline) is achieved through a domain-specialized 32B Generator trained on 1.7 M RTL samples at roughly 10,000 A100 GPU-hours; ChipCraftBrain reaches higher numbers on cid002 without any RTL-specific fine-tuning, relying instead on Claude Opus/Sonnet plus our validation-first pipeline. Second, ChipCraftBrainâs per-problem compute budget (up to 5 iterations, single process) is roughly 30Ă30Ă smaller than ACE-RTLâs (up to 5 parallel processes Ă 30 iterations == up to 150 attempts per problem), yet the category-level comparison is favorable on three of four categories. Together, these suggest that validation-first iterative refinement with a strong frontier backbone is a compute-efficient alternative to heavy parallel sampling with a domain-specialized fine-tune. We do not match ACE-RTL on bug fixing (cid016), where their larger attempt budget appears to help with multi-bug patches that cascade into previously-passing logic. Closing this gap through richer debug-trajectory exploration in our pipeline is a direct extension (SectionËIX). VI-D ChipBench Results TABLE IX: ChipBench Results by Category Category Pass@1 Problems Avg Cells Self-contained 36.7% 11/30 323 Non-self-contained 50.0% 3/6 361 CPU IP 11.1% 1/9 862 Overall 33.3% 15/45 439 TABLE X: ChipBench Comparison with Published Results System Overall Self-cont. Non-self-cont. CPU IP Claude Opus (single) 30.7% n/a 0% n/a MAGE 37.4% n/a n/a 22.2% ChipCraftBrain 33.3% 36.7% 50.0% 11.1% On ChipBench, ChipCraftBrain achieves 33.3% (15/45). MAGEâs overall 37.4% remains 4.1 p higher; we do not outperform MAGE on this benchmark. The ChipBench paper [3] does not publish MAGEâs per-category breakdown, so direct category-for-category comparison is not possible. We note two observations from our own per-category results: (1) on non-self-contained (hierarchical) designs, our hierarchical decomposition pipeline achieves 50% (3/6) where the reported single-model Claude Opus baseline scores 0/6, suggesting decomposition is the main driver of that category; and (2) on CPU IP (11.1%) we fall below MAGEâs reported 22.2%, indicating that our advantage is localized to hierarchical designs rather than complex processor components. The CPU IP category remains the most challenging (11.1%, with only the register file passing). RISC-V controllers, ALUs, division units, and branch prediction units involve complex multi-stage logic that exceeds the current systemâs capability within the 5-iteration budget. These designs average 862 synthesis cells, 28Ă the complexity of VerilogEval problems. VI-E Component Contribution Analysis To characterize the contribution of each component, we report observed incremental impact on VerilogEval-Human by progressively enabling system features. These numbers are derived from partial runs and per-problem telemetry across the benchmark suite rather than fully controlled ablations with each component toggled in isolation; a full leave-one-out ablation is reported as future work. TABLE XI: Observed Component Contributions on VerilogEval-Human.â Values reflect observed incremental lift rather than controlled leave-one-out ablations. â Observational, not controlled: Î columns show estimated contributions from partial runs and telemetry, not from isolating each component with all others held fixed. Components are known to interact, so these numbers should not be read as marginal causal effects. Configuration Pass@1 Î Single-shot (no iteration) âź 63% baseline + Iterative refinement (5 iter) âź 78% +15 p + Multi-agent selection âź 85% 0+7 p + RAG knowledge âź 88% 0+3 p + Spec guidance registry âź 92% 0+4 p + Symbolic K-map solver âź 96% 0+4 p ChipCraftBrain (full) 98.7% +35.7 p The iterative refinement loop provides the largest single observed improvement (+15 p), consistent with the hypothesis that validation-first generation with structured error feedback is the core driver of performance. The symbolic K-map solver contributes +4 p by handling 6 problems at zero cost, and spec guidance contributes another +4 p via problem-specific prompt enrichment. We caution that these contributions are not additive in the strict sense: removing one component may shift which problems become solvable by others, so a leave-one-out ablation would likely yield smaller per-component deltas. VI-F Cost Analysis TABLE XII: Cost Per Problem on VerilogEval. System $/Problem Strategy MAGE âź 0.060 20Ă Sonnet samples ChipAgents âź 0.045 Multi-step pipeline ChipCraftBrain âź 0.010â0.030 Tiered (symbolic + LLM) VFlow varies MCTS-selected models Single-shot âź 0.003 1Ă GPT-4 ChipCraftBrain achieves approximately 2â3Ă lower cost than MAGE on VerilogEval through three mechanisms: (1) zero-cost algorithmic solving for K-map and truth table problems, (2) Sonnet for the Fast and Optimize refinement agents where latency and per-token cost dominate, and (3) Opus reserved for Genius (first-attempt), Debug, Testbench, and Waveform agents where capability dominates. The high first-iteration success rate (133/156 = 85%) means most VerilogEval problems require only a single Sonnet or Opus call, not 20 parallel candidates. CVDP cost. On the 302-problem CVDP subset, problems are longer (avg 43 LOC, often with provided context RTL reaching several hundred tokens) and the first-iteration success rate is lower than on VerilogEval, so multi-iteration costs compound. Our measured average is $0.05â$0.12 per problem on CVDP, roughly 4â5Ă the VerilogEval cost and putting the total cost for the full 302-problem run in the $20â$35 range. This remains considerably lower than running 20 parallel Sonnet candidates per problem at CVDP-scale context windows. ChipBench costs are in a similar range to CVDP given comparable iteration counts and module sizes. VI-G Orchestration Analysis Of the 154 passing VerilogEval problems, 85% (133/156) pass on the first iteration, demonstrating strong baseline generation quality. The remaining 21 problems benefit from the adaptive orchestration loop, which switches agents and adjusts parameters across iterations. The high first-iteration success rate indicates that the primary value of RL orchestration lies in recovery from failures (selecting the right agent and parameters when the initial attempt fails), rather than initial agent selection. This is consistent with the observation that simpler problems rarely need agent switching, while harder problems (complex FSMs, multi-signal protocols) benefit from the Debug or Genius agents being deployed after an initial Fast agent failure. VI-H Deployment via Local-Model Distillation API-based pipelines are not deployable in IP-sensitive settings where specifications cannot leave a customerâs network. We therefore test whether ChipCraftBrainâs own trajectories can bootstrap a local model that runs the same pipeline entirely offline. Procedure. We harvest instruction-output pairs from successful Claude runs on VerilogEval using an extraction script (verilogeval_distill.py) that records each (specification, generated RTL, category) triple, deduplicated by code hash. These pairs, merged with a larger pattern-mined corpus, form a 5,715-sample SFT dataset. We QLoRA-fine-tune a Qwen2.5-Coder 7B base model on this dataset, export to merged weights, and serve via vLLM [30] on dual RTX 3090 GPUs (tensor parallelism =2=2). The ChipCraftBrain pipeline points at the vLLM endpoint instead of the Anthropic API; all other components (RAG, spec guidance, RL orchestration, iterative validation, K-map solver) remain unchanged. TABLE XIII: Local-Model Distillation on VerilogEval-Human. The 7B + pipeline row runs fully offline on dual 3090s. Configuration Pass@1 Deployment ChipCraftBrain (Claude) 98.7% Anthropic API 7B + ChipCraftBrain pipeline 75.0% Local (vLLM) 7B standalone (single-shot) 36.5% Local (vLLM) Interpretation. Both the 36.5% standalone and 75.0% pipeline rows use the same fine-tuned 7B weights; the 38.5 p lift is therefore attributable entirely to the pipeline scaffolding (iterative validation, multi-agent orchestration, and knowledge-augmented prompting) rather than to additional training. This mirrors the contribution pattern observed in the Claude configuration: pipeline scaffolding is the dominant driver of performance, and the backbone model is a replaceable component. The result falsifies the hypothesis that our numbers are Claude-specific and demonstrates a concrete on-premises deployment path at roughly 25â50Ă lower per-problem inference cost than the Anthropic pipeline. Full training-data recipes, hyperparameters, and broader local-model baselines are outside the scope of this paper. VII Case Study: RISC-V SoC Optimization To demonstrate ChipCraftBrainâs capability beyond individual module generation, we present an end-to-end SoC optimization case study. VII-A Unoptimized Design The baseline is a 10-module RISC-V SoC comprising a single-cycle CPU core, 32Ă32 register file, 32-bit ALU, instruction decoder, 32 KB SRAM, memory controller, simple bus arbiter, UART, GPIO, and top-level interconnect. With no instruction cache and single-cycle memory access, this design achieves an IPC of 0.3 and an estimated âź 8 FPS on a representative compute-intensive embedded workload (software-rendered 320Ă200 raster graphics, 2 M instructions/frame), used throughout this section as a concrete reference for end-to-end performance. VII-B AI-Driven Optimization ChipCraftBrainâs optimization pipeline proceeds in four phases: Phase 1: Architecture Analysis. The analyzer identifies the design as a single-cycle processor, detects the absence of an instruction cache, and flags the 10-cycle memory latency as a critical bottleneck. Phase 2: Bottleneck Identification. Structured bottleneck analysis reveals: (a) no instruction cache (â every fetch hits main memory), (b) single-cycle pipeline limiting IPC, (c) no branch prediction. Phase 3: Optimization Proposals. The strategist generates two proposals: (a) add a direct-mapped 4 KB instruction cache with 128 lines and 32-byte cache lines, and (b) upgrade the CPU core to a 2-stage pipeline (IF/ID + EX/MEM/WB). Phase 4: Hierarchical Generation. The decomposition engine breaks the optimized SoC into 8 sub-modules, generates each with cross-module port synchronization, and produces 689 LOC of lint-passing Verilog. VII-C Results TABLE XIV: RISC-V SoC Before/After Optimization. IPC, cache hit rate, effective memory latency, and workload FPS are analytical estimates from the architectural configuration, not measured from cycle-accurate RTL simulation or FPGA-instrumented cycle counters. Module count, LOC, generation time, and the FPGA synthesis numbers in Table XV are directly measured. Metric Before After Improvement IPC (est.) 0.3 0.7 2.3Ă Cache hit rate (est.) 0% 85% N/A Eff. mem. latency (est.) 10 cyc 2.5 cyc 4Ă Workload FPS (est.) âź 8 âź 13 âź 60% VII-D FPGA Hardware Validation The generated SoC was synthesized and deployed on a Terasic DE25-Nano board (Intel Agilex 5, 46,800 ALMs) using Quartus 25.1 Pro Edition at a 50 MHz target clock. The generated SoC design consumes 731 ALMs (2% of the device), 896 registers, and 34 RAM blocks, with a setup slack of +6.8 ns. A PicoRV32-based reference SoC uses 1,458 ALMs (3%), 1,541 registers, and 2 DSP blocks, with +4.9 ns setup slack. Both designs meet all timing constraints at the target frequency. TABLE XV: FPGA Synthesis Results (Intel Agilex 5, 50 MHz) Resource Generated SoC PicoRV32 SoC ALMs 731 (2%) 1,458 (3%) Registers 896 1,541 RAM blocks 34 0 DSP blocks 0 2 Setup slack +6.8 ns +4.9 ns VIII Discussion VIII-A Key Findings Our three-benchmark evaluation reveals several important insights: Validation-first is critical. The iterative refinement loop with structured error feedback bridges the gap from âź 63% single-shot to 98.7% on VerilogEval and 94.7% on CVDP, confirming that generating code and verifying code are equally important. Of 154 passing VerilogEval problems, 21 required at least one refinement iteration. Techniques transfer across complexity tiers. The same system achieving 98.7% on simple VerilogEval modules (16 avg LOC) also achieves 94.7% on industrial CVDP problems without benchmark-specific tuning. This suggests that iterative validation, multi-agent selection, and knowledge-augmented generation are general techniques, not benchmark-specific optimizations. A complexity ceiling exists. Performance drops sharply on ChipBench (33.3%), particularly on CPU IP designs (11.1%). Problems with 800+ synthesis cells, multi-stage pipelines, and complex control logic exceed the current systemâs capability within a 5-iteration budget. This is consistent with prior results: even MAGE achieves only 37.4% on ChipBench, and no published system exceeds 40%. Hierarchical decomposition unlocks new capability. ChipCraftBrain achieves 50% on ChipBenchâs non-self-contained (hierarchical) category, where most single-model approaches score 0%. The RISC-V SoC case study confirms this: 0% monolithic vs. 100% hierarchical. For multi-module designs, decomposition with cross-module synchronization is essential. Agent specialization matters. Both MAGEâs ablation and our adaptive orchestration demonstrate that specialized agents with independent context windows significantly outperform single-agent approaches. The symbolic K-map solver and dedicated Waveform agent further show that not every problem needs an LLM: hybrid symbolic-neural architectures outperform pure neural approaches on deterministic problem classes. VIII-B Synthesis Awareness Advantage A key differentiator from MAGE is our synthesis-aware approach. MAGEâs scoring considers only functional correctness (normalized mismatch count). In practice, two functionally equivalent implementations can differ enormously in area, power, and timing. Our multi-objective scoring (functional + synthesis quality + complexity) selects candidates that are both correct and synthesis-optimal. VIII-C The CVDP Result in Context The 94.7% on CVDP warrants careful interpretation. Three scoping caveats apply. Subset. We evaluate 302 of 783 CVDP problems, covering only the five non-agentic, non-commercial code-generation task categories (cid002, cid003, cid004, cid007, cid016). The remaining 481 problems lie outside this scope by construction: (a) testbench, checker, and assertion generation (cid012/013/014, 187 problems) require Cadence Xcelium for harness evaluation and cannot be reproduced with open-source tooling; (b) the 166 agentic multi-file problems require a Docker-based tool-use runtime we have not yet implemented; (c) comprehension tasks (cid006/008/009/010, 123 problems) score via BLEU or LLM-as-judge rather than functional execution, which lies outside our validation-first design. We do not claim coverage of these 481 problems. Protocol difference. The 33.56% baseline is single-shot pass@1 with n=5n=5 unconditional samples. Our 94.7% is a single attempt per problem but with up to 5 iterations of structured error feedback, agent switching, and RAG refinement. The comparison therefore measures the contribution of iterative validation and orchestration on top of a strong base model, not raw model capability in isolation. Presenting this as a 2.8Ă2.8Ă or 61-point âimprovementâ would be protocol-mixed; we instead frame it as âvalidation-first pipeline lifts non-agentic CVDP code generation from 33.56% to 94.7%.â Still meaningful. Even after these caveats, the result is substantial: every task in the evaluated subset is drawn from real NVIDIA hardware engineering problems, the 5-iteration budget is fixed in advance, and the pipeline runs fully offline under cocotb. The lift is consistent across all five task categories (36â60 p above the single-shot baseline), suggesting the pipeline captures something robust about industrial RTL generation rather than exploiting a narrow slice of the distribution. VIII-D Limitations ⢠No post-synthesis timing closure: The validation pipeline checks synthesis cleanliness but does not incorporate timing-driven feedback for critical path optimization. ⢠ChipBench CPU IP: Complex processor components (controllers, ALUs, division units) remain largely unsolved (11.1%), requiring advances in cooperative multi-agent decomposition and test-driven generation. ⢠CVDP coverage: 302 of 783 problems evaluated (the non-agentic non-commercial code-generation subset). The 481 unevaluated problems include testbench/assertion generation (requires Xcelium), agentic multi-file tasks (requires Docker agent runtime), and comprehension tasks (BLEU-scored). ⢠Protocol comparability: Published CVDP baselines are single-shot pass@1 while our results use iterative refinement; head-to-head single-shot evaluation of our base agents remains future work. ⢠Formal verification: No SystemVerilog Assertion (SVA) property checking integration yet. ⢠Benchmark representativeness: Even ChipBench (62 avg LOC) is simpler than production IP blocks, which can exceed 10,000 LOC. VIII-E Threats to Validity LLM generation is stochastic; the reported 98.7% on VerilogEval reflects the best of multiple runs, with pass rates varying by 1â2 percentage points across runs due to temperature-dependent sampling. API model versions evolve, potentially affecting reproducibility; we report results against Claude Opus 4.6 and Sonnet 4.6 as accessed in early 2026. The CVDP 94.7%94.7\% is drawn from a single full-subset run (302 problems, âź 6.1 hours wall-clock). Unlike VerilogEval, where we quantify variance across seven independent runs (96.15â98.72%), CVDP has not yet been repeated end-to-end; the reported number should therefore be interpreted as subject to an unverified uncertainty envelope rather than a point estimate backed by a confidence interval. Full-subset repeats to establish such an interval are in progress. The SoC case study performance estimates (IPC, FPS) are based on architectural analysis validated by FPGA deployment, not cycle-accurate RTL simulation of the workload. IX Future Work Several directions extend ChipCraftBrainâs capabilities: Scaling to Hard Accelerator Designs. ChipBench CPU IP (11.1%) reveals that complex multi-stage processor components require fundamentally different strategies. We are developing cooperative multi-agent decomposition with contract-based interface validation, progressive structural refinement through a 7-layer complexity model, and test-driven generation where testbenches are created before RTL, targeting 70%+ on ChipBench. Broader CVDP Evaluation. Three extensions expand CVDP coverage beyond the 302-problem non-agentic non-commercial code-generation subset reported here. First, adding Cadence Xcelium integration enables the 187 commercial testbench, checker, and assertion generation problems (cid012/013/014). Second, implementing a Docker-based agentic runtime (multi-file editing, tool use, iterative refinement) unlocks the 166 agentic problems, including the hard tier that exists only in agentic form. Third, adding BLEU scoring and LLM-as-judge infrastructure for the 123 comprehension problems (cid006/008/009/010) closes the remaining coverage gap. We also plan a matched-protocol single-shot evaluation of our base agents on the current 302-problem subset for a true apples-to-apples comparison with published single-shot baselines. Controlled Ablation Study. The component contribution numbers in TableËXI are observational. A leave-one-out ablation disabling each component individually on a matched subset (with seeded sampling for reproducibility) will produce tight per-component deltas and is a direct extension of the current evaluation. Extending the Distilled 7B Model. The local-model baseline in SectionËVI-H (75.0% on VerilogEval-Human via the 7B-backed pipeline) uses only spec-to-RTL pairs harvested from successful Claude runs. Two extensions would close more of the 23.7 p gap to the Claude-driven pipeline: (1) adding error-repair trajectories so the local model learns the refinement step, not just single-shot generation, and (2) training an agent-selection head so the RL policy can operate on locally-computed state features rather than Claude-specific prompts. Beyond VerilogEval, we plan to evaluate the distilled 7B on the 302-problem CVDP non-agentic subset. Multi-Candidate Generation. Generating N=5N=5 candidates with synthesis-aware scoring using Yosys cell count and critical path metrics, following MAGEâs insight that diverse candidates improve best-case quality, but extending scoring beyond functional correctness to include synthesis optimization. Differential Checkpoint Debugging. Extending debug feedback with cone-of-influence analysis via Yosys, property-level abstractions from testbench assertions, and differential waveform analysis between passing and failing checkpoints. End-to-End Pipeline. Integration with OpenLane 2 and SKY130 PDK for a complete NL â RTL â GDSII pipeline, enabling AI-generated silicon through chipIgnite. Formal Verification. Adding SystemVerilog Assertion (SVA) property checking for deeper functional validation beyond simulation-based testing. X Conclusion We have presented ChipCraftBrain, a framework for automated RTL generation that integrates reinforcement learning, multi-agent LLM orchestration, hybrid symbolic-neural reasoning, knowledge-augmented retrieval, hierarchical decomposition, and validation-first iterative refinement. Our key contributions include: (1) adaptive multi-agent orchestration using a PPO policy over a 168-dimensional state representation with hybrid discrete-continuous actions across six specialized agents; (2) a hybrid symbolic-neural architecture combining algorithmic K-map solving with learned LLM orchestration; (3) knowledge-augmented generation from a 321-entry knowledge base, 971 open-source reference implementations, and a 59-entry spec-guidance registry encoding the class of heuristic design-pattern knowledge commercial EDA tools accumulate over decades; (4) hierarchical specification decomposition with cross-module port synchronization; and (5) three-benchmark evaluation across complexity tiers. We evaluate across three benchmarks spanning simple modules to hard accelerator designs: ⢠VerilogEval-Human: 97.2% mean pass@1 (range 96.15â98.72% across 7 runs, best 154/156), on par with ChipAgents (97.4%, self-reported) and ahead of MAGE (95.9%) within measurement noise ⢠CVDP non-agentic non-commercial code-generation subset: 94.7% mean pass@1 (286/302) across task categories cid002, cid003, cid004, cid007, cid016 (averaged over 3 runs), compared to 33.56% for the published single-shot baseline on the same subset. The remaining 481 CVDP problems (commercial testbench categories, agentic multi-file tasks, and comprehension tasks) are out of scope for our current pipeline and are not claimed. ⢠ChipBench: 33.3% (15/45), competitive with MAGE (37.4%) on designs with 14Ă more gates than VerilogEval at approximately 3Ă lower cost than prior multi-agent systems. A RISC-V SoC case study demonstrates hierarchical decomposition generating 8/8 lint-passing modules (689 LOC), validated on FPGA hardware, where monolithic generation fails entirely. These results demonstrate that the combination of specialized agents, adaptive orchestration, domain knowledge, and rigorous validation can bring AI-generated RTL to production-quality levels on simple and industrial designs, while revealing that complex accelerator-class designs remain an open frontier for the field. References [1] M. Liu, N. Pinckney, B. Khailany, and H. Ren, âVerilogEval: Evaluating large language models for verilog code generation,â in Proceedings of the IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, 2023. [2] N. Pinckney, C. Deng, C.-T. Ho, Y.-D. Tsai, M. Liu, W. Zhou, B. Khailany, and H. Ren, âComprehensive Verilog design problems: A next-generation benchmark dataset for evaluating large language models and agents on RTL design and verification,â arXiv preprint arXiv:2506.14074, 2025, the formal CVDP benchmark paper. 783 problems across 13 task categories in two modes. [3] Z. Yu, C. Zhou, Y. Lin, H. Zhang, H. Ye, J. Cui, Z. Pan, J. Zhao, and Y. Ding, âChipBench: A next-step benchmark for evaluating LLM performance in AI-aided chip design,â arXiv preprint arXiv:2601.21448, 2026, 45 Verilog generation problems, 89 debugging, 132 reference model generation. ICML 2026 submission. [4] S. Thakur et al., âVeriGen: A large language model for verilog code generation,â arXiv preprint arXiv:2308.00708, 2023. [5] Y. Liu et al., âCodeV: Empowering LLMs with expert-level hdl code generation,â arXiv preprint arXiv:2407.10424, 2024. [6] S. Liu et al., âRTLCoder: Fully open-source and efficient LLM-assisted RTL code generation technique,â arXiv preprint arXiv:2312.08617, 2024. [7] S. Thakur et al., âAutoChip: Automating HDL generation using LLM feedback,â arXiv preprint arXiv:2311.04887, 2023. [8] Y.-D. Tsai et al., âMAGE: A multi-agent engine for automated RTL code generation,â arXiv preprint arXiv:2412.04211, 2024, 95.9% VerilogEval-Human v2. [9] Y. Wei, Z. Huang, H. Li, W. W. Xing, T.-J. Lin, and L. He, âVFlow: Discovering optimal agentic workflows for verilog generation,â arXiv preprint arXiv:2504.03723, 2025, 83.6% pass@1 on VerilogEval. [10] W. V. Quine, âThe problem of simplifying truth functions,â The American Mathematical Monthly, vol. 59, no. 8, p. 521â531, 1952. [11] E. J. McCluskey, âMinimization of boolean functions,â Bell System Technical Journal, vol. 35, no. 6, p. 1417â1444, 1956. [12] J. Blocklove et al., âChip-Chat: Challenges and opportunities in conversational hardware design,â in Proceedings of the Workshop on Machine Learning for CAD (MLCAD), 2023. [13] M. Liu, T.-D. Ene, R. Kirby, C. Cheng, N. Pinckney et al., âChipNeMo: Domain-adapted LLMs for chip design,â arXiv preprint, 2023. [14] F. Cui, C. Yin, K. Zhou, Y. Xiao, G. Sun, Q. Xu, Q. Guo, Y. Liang, X. Zhang, D. Song et al., âOriGen: Enhancing RTL code generation with code-to-code augmentation and self-reflection,â arXiv preprint, 2024. [15] M. Liu, Y.-D. Tsai, W. Zhou, and H. Ren, âCraftRTL: High-quality synthetic data generation for Verilog code models with correct-by-construction non-textual representations and targeted code repair,â arXiv preprint, 2024. [16] C. Deng, Y.-D. Tsai, G.-T. Liu, Z. Yu, and H. Ren, âScaleRTL: Scaling LLMs with reasoning data and test-time compute for accurate RTL code generation,â arXiv preprint, 2025. [17] Alpha Design AI, âChipAgents: Agentic AI for chip design,â 2025, commercial system. 97.4% VerilogEval-v2. $74M funding (latest round). [18] C.-T. Ho, H. Ren, and B. Khailany, âVerilogCoder: Autonomous Verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool,â in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2025. [19] C. Deng, Z. Yu, G.-T. Liu, N. Pinckney, B. Khailany, and H. Ren, âACE-RTL: When agentic context evolution meets RTL-specialized LLMs,â arXiv preprint arXiv:2602.10218, 2026, fine-tuned Qwen2.5-Coder-32B generator with Claude 4 Sonnet reflector/coordinator. 5 parallel processes Ă up to 30 iterations per problem on CVDP v1.0.2. Specialized generator required âź 10,000 A100 GPU-hours to train (32 nodes Ă 8 A100 Ă 3 epochs). [20] Inception AI, âMercury: Ultra-fast language models based on diffusion,â arXiv preprint arXiv:2506.17298, 2025, diffusion-based code LLM; 82.7% pass@1 on VerilogEval-Human via ChipCraftBrain pipeline routing, âź 5s/problem. [21] DeepSeek-AI, âDeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning,â arXiv preprint arXiv:2501.12948, 2025. [22] Qwen Team, âQwen3-Coder: Long-context coding with reinforcement learning,â arXiv preprint, 2025. [23] Y. Li et al., âCompetition-level code generation with AlphaCode,â Science, vol. 378, no. 6624, p. 1092â1097, 2022. [24] H. Le et al., âCodeRL: Mastering code generation through pretrained models and deep reinforcement learning,â in Advances in Neural Information Processing Systems (NeurIPS), 2022. [25] Synopsys, Inc., âDSO.ai: Design space optimization AI,â 2020, RL-based P&R optimization. [26] Cadence Design Systems, âCerebrus: Intelligent chip explorer,â 2021, ML design space exploration. [27] Efabless Corporation, âOpenLane 2: Open-source digital ASIC implementation flow,â 2024, rTL-to-GDSII with SKY130 PDK. [28] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, âProximal policy optimization algorithms,â arXiv preprint arXiv:1707.06347, 2017. [29] W. Snyder, âVerilator: An open-source SystemVerilog simulator,â 2024. [30] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica, âvLLM: Efficient memory management for large language model serving with PagedAttention,â in Proceedings of the ACM Symposium on Operating Systems Principles (SOSP), 2023.