Paper deep dive
Coco: An Agentic Copilot for the Hardware--Software Co-Design Lifecycle
Samuel Kushnir, Kavya Sreedhar, Yeshwanth Reddy Pogula, Amir Yazdanbakhsh, Narges Shahidi, Ming Liu, Varun Gohil, Ravi Iyer, Parthasarathy Ranganathan, Christina Delimitrou, Suvinay Subramanian
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/5/2026, 5:52:18 AM
Summary
The paper introduces Coco, an agentic copilot platform designed to accelerate the hardware-software co-design lifecycle for ML accelerators. Coco addresses the challenge of analyzing novel, simulation-heavy data that is absent from LLM pretraining corpora. It employs a four-layer architecture: a relational datastore for normalized simulation sweeps, a library of typed APIs for tools, agents that encode recurring workflows (such as iso-execution analysis), and a UX that captures navigation state as agent context. Early deployment with TPU architects demonstrates a 2x reduction in time-to-simulation and time-to-insight, enabling architects to derive grounded, SQL-traceable insights from complex simulation data.
Entities (10)
Relation Signals (10)
Coco â hascomponent â Datastore
confidence 95% · Coco is built as four layers: (i) a datastore that automatically registers every simulation sweep
Coco â hascomponent â agents
confidence 95% · (iii) agents that encode recurring workflows
Coco â hascomponent â Tools
confidence 95% · (ii) a library of tools with typed APIs
Coco â hascomponent â Platform UX
confidence 95% · (iv) a platform UX whose navigation state doubles as agent context
Coco â isusedby â TPU architects
confidence 95% · Coco is deployed with TPU architects and in daily use for serving-focused analyses
Coco â performsworkflow â Iso-execution analysis
confidence 93% · The flagship is iso-execution analysis, an analysis workflow in which we focus on a given latency target
Coco â achievesimprovement â 2x reduction in time-to-simulation
confidence 90% · We report early deployment experience toward a 2x reduction in time-to-simulation and time-to-insight
Coco â achievesimprovement â 2x reduction in time-to-insight
confidence 90% · We report early deployment experience toward a 2x reduction in time-to-simulation and time-to-insight
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Co-designing ML models and the accelerators that run them is an unusual reasoning task: architects must draw confident, high-stakes conclusions about systems that do not yet exist, and the pace of both model evolution and hardware cadence means the analysis burden grows every quarter. The evidence behind each decision--hundreds of gigabytes of fresh simulation sweeps over novel design points--is by construction absent from any LLM's pretraining corpus, and there is no external literature to retrieve; naive "chat-with-your-data" approaches hallucinate exactly where correctness matters most. We present Coco (Copilot for Codesign), an agentic platform deployed with TPU architects that accelerates the co-design lifecycle of setting up experiments, sweeping simulators, and deriving insights. Coco is built as four layers: (i) a datastore that automatically registers every simulation sweep into a normalized relational schema, so agents ground every number in a SQL query rather than scraping heterogeneous files; (ii) a library of tools with typed APIs that agents compose without human orchestration; (iii) agents that encode recurring analysis workflows--most notably iso-execution analysis, which compares systems at matched execution configurations, including swept-but-dominated points off the Pareto frontier; and (iv) a platform UX whose navigation state doubles as agent context. We report early deployment experience toward a reduction in time-to-simulation and time-to-insight, and argue that co-design is a distinct agentic domain: its data must be retrieved rather than memorized, its workflows are recurring but context-dependent, and expert adoption hinges on UX that balances IDE-style control with interactive exploration.
Tags
Links
- Source: https://arxiv.org/abs/2610.02376v1
- Canonical: https://arxiv.org/abs/2610.02376v1
Trouble viewing inline? Open PDF directly â
Full Text
17,931 characters extracted from source content.
Expand or collapse full text
Coco: An Agentic Copilot for the HardwareâSoftware Co-Design Lifecycle Samuel Kushnir Affiliation: Google DeepMind Kavya Sreedhar Affiliation: Google Yeshwanth Reddy Pogula Affiliation: Google Amir Yazdan Bakhsh Affiliation: Google DeepMind Narges Shahidi Affiliation: Google Ming Liu Affiliation: Google Varun Gohil Affiliation: MIT Ravi Iyer Affiliation: Google Christina Delimitrou Affiliation: MIT Parthasarathy Ranganathan Affiliation: Google Suvinay Subramanian *Equal contribution Affiliation: Google DeepMind Abstract Co-designing ML models and the accelerators that run them is an unusual reasoning task: architects must draw confident, high-stakes conclusions about systems that do not yet exist, and the pace of both model evolution and hardware cadence means the analysis burden grows every quarter. The evidence behind each decisionâhundreds of gigabytes of fresh simulation sweeps over novel design pointsâis by construction absent from any LLMâs pretraining corpus. There is no external literature to retrieve; naive âchat-with-your-dataâ approaches hallucinate and lack reliability. We present Coco (Copilot for Codesign), an agentic platform deployed with TPU architects that accelerates the co-design lifecycle of setting up experiments, sweeping simulators, and deriving insights. Coco is built as four layers: (i) a datastore that automatically registers every simulation sweep into a normalized relational schema, so agents ground every number in a SQL query rather than scraping heterogeneous files; (i) a library of tools with typed APIs that agents compose without human orchestration; (i) agents that encode recurring analysis workflowsâmost notably iso-execution analysis, which compares systems at matched execution configurations, including swept-but-dominated points off the Pareto frontier; and (iv) a platform UX whose navigation state doubles as agent context. We report early deployment experience toward a 2Ă2Ă reduction in time-to-simulation and time-to-insight, and argue that co-design is a distinct agentic domain: its data must be retrieved rather than memorized, its workflows are recurring but context-dependent, and expert adoption hinges on UX that balances IDE-style control with interactive exploration. 1 Introduction Hardwareâsoftware co-design for ML accelerators is a peculiar kind of performance engineering. First, it reasons about systems that do not yet exist: the artifact under study is a proposed chip, interconnect, or packaging option, observable only through simulation and analytical models. Second, its decisions are high-stakes and long-livedâa datacenter accelerator generation commits billions of dollars years before deployment. Third, demand is relentless and increasing: hardware cadence is accelerating, product complexity is growing, and the workload side diversifies just as fastâdense versus mixture-of-experts models (Fedus et al., 2022), proprietary versus open-source families, autoregressive versus diffusion decoding. Concretely, the day-to-day work is dominated by a recurring question shape: a number movedâwhat changed, why, and does it match our intuition? When specifications and model assumptions are refreshed each quarter, the projected advantage of a candidate system over its predecessor can shift by tens of percent; an architect must decompose that shift into spec changes, model changes, and sharding changes before anyone acts on it. Comparative questions have the same texture: why does a dragonfly-class interconnect benefit decode-heavy inference more than a torusâis it only interconnect pressure, or also memory bandwidth? Why is model parallelism rarely chosen on the Pareto frontier? Today, answering any of these places a senior architect on the critical path for hours-to-days of manual sweep archaeology: locating the right simulation runs, decoding configuration encodings, cross-checking simulator quirks, and assembling plots. The experts scale linearly; the questions do not. We built Coco (Copilot for Codesign), an agentic platform that accelerates this lifecycle end to endâsetting up experiments, sweeping and simulating, and analyzing the results into insights (Figure 1, right)âwith a near-term goal of halving time-to-simulation and time-to-insight, and a longer-term goal of letting the underlying LLM itself iterate on and explore new architectural ideas. This paper describes the thesis behind Coco (§2), its four-layer architecture (§3), early deployment experience with TPU (Jouppi et al., 2023) architects (§4), and two end-to-end case studies on public TPU generations (§5). 2 Why co-design needs its own copilot Our thesis is that co-design has unique workflows, context, and artifacts, and that a generic coding or chat assistant serves none of them well. The data cannot be memorized. Every design cycle regenerates hundreds of gigabytes of simulation data over design points that are novel by constructionânew chips, new topologies, new model variants. No pretrained model has seen these numbers, and unlike open-domain retrieval-augmented generation (Lewis et al., 2020) there is no external literature to fall back on: the corpus is the organizationâs own sweeps, specs, and documents, and it changes under the agentâs feet. Grounding must therefore be structural: every quantitative claim should be traceable to a series of queries and tool calls which a human can easily validate. The workflows recur, but the context varies. Analyses repeat in skeleton but every instance differs in the particulars. Almost every workflow consists of running a sweep, extracting Pareto frontiers, comparing systems, and decomposing a delta but the particulars: which use case, which sequence-length regime, which metric, are constantly changing. The productive division of labor is to standardize the skeleton as tools and encoded workflows and let the LLM handle the difference: ârerun this sweep with only the HBM bandwidth changed,â âpull peak memory usage for the Pareto-optimal points of that chart.â The artifacts are collaborative objects. The unit of progress is not a code diff but a Pareto curve, a per-op breakdown, a utilization timeline, a comparison report. Experts need to create, navigate, diff, and share these rapidlyâand the trail of artifacts a user has visited is itself the best statement of their intent. PlatformAgentsToolsDatastoreorchestrate agents with the right UXchat, charts, navigation as contextautomate workflows with apt contexte.g., iso-execution analysisbite-size tasks behind typed APIscomposed by agents, not humanssweeps auto-registered in a relational DBplus specs, hyperparameters, docsaddress user intent âŠby encoding workflows âŠthat leverage tools âŠover a corpus of datasetupexperimentsweep &simulateanalyze &derive insightsco-designlifecycle Figure 1: Cocoâs four-layer stack (left) and the co-design lifecycle it accelerates (right). Each layer is consumed by the one above it; the platformâs navigation state feeds back into agent context. 3 The Coco stack Datastore: sweeps as a database, not files. Every simulation sweepâserving, training, and sampling/thinking workloads, from both analytical and event-driven simulatorsâis automatically registered into a normalized relational schema: test plans, workloads, systems, execution strategies, Pareto frontiers, and per-phase (prefill/decode/combined) serving points carrying latency and throughput metrics down to time-to-first-token. Agents retrieve data by constructing SQL against this schema rather than scraping CSV, proto, or JSON artifacts. In our experience this is the single largest lever against hallucination: the model never âremembersâ a number, it fetches one, and the query itself is an auditable provenance record. The datastore also collates the canonical context around the numbersâsystem specifications, model hyperparameters, ISA and product documentationâso retrieval spans both measurements and their meaning. Tools: bite-size tasks behind typed APIs. On top of the datastore sits a library of tools for well-specified tasks, registered with documentation and made available to LLM agents in the style of modern tool-use systems (Schick et al., 2023; Yao et al., 2023). A representative retrieval chainâfind valid system/model combinations â find valid Pareto curves â fetch a curve â fetch a point â fetch its detailed breakdownâis composed by the agent, interleaved with small amounts of inline code, with no human orchestration or bespoke glue per analysis. Agents: encoded workflows with the right context. Agents automate the recurring workflows: run an analysis across systems and models, synthesize the per-system results into a coherent comparison, and generate a report. The flagship is iso-execution analysis, an analysis workflow in which we focus on a given latency target, and directly compare the two systems performance in-depth at that latency target. This includes diving into op-by-op breakdowns and pinning down exactly which operators and under which use cases the systems differ the most on performance. Architects constantly ask why an under-performing system did not adopt the winning systemâs configuration (âwhy does the torus stop at 16-way expert parallelism when the dragonfly chooses 64-way?â). The answer usually exists in the sweepâthe configuration was evaluated and dominated by a better execution strategy so it doesnât appear on the Pareto curve. Exhuming it by hand means reconstructing equivalent configuration keys across systems and locating which shard of which sweep job produced the point. The agent retrieves the iso-sharding, iso-batch counterpart directly from the datastore and reports why it lost, turning a multi-hour human search into part of every report. Coco is now a multi-agent system, where the orchestrator agent can autonomously launch sub-agents with access to the same tooling. In the iso-execution analysis workflow, sub-agents are used to parallelize the analysis across different latency targets and reviewer agents are then used to check over the integrity of the analysis. The orchestrator agent then synthesizes everything to a single report for the human engineer. Platform: UX as context capture. The platform provides the interaction grammar of architectural analysisâhover a Pareto point for details, click through to its breakdown, open the utilization timeline, diff two breakdowns across systems or execution strategiesâwith an embedded conversational side panel. Critically, the UX is also how agent context gets captured: the chart the user is looking at, the use case and sequence-length regime it encodes, and the trajectory of questions that led here are all attached to the user query automatically, so a user can ask âwhy does this point have poor utilization?â and the agent will have full understanding of the pareto curve and point in question. 4 Early experience Coco is deployed with TPU architects and in daily use for serving-focused analyses; training and further simulator backends are being onboarded. Three observations from early use: Structured retrieval changes trust. Analyses that quote numbers with their generating SQL are treated as grounded evidence by architects, in a way free-form LLM summaries never were. The schema, not the model, carries the burden of being right about data. Iso-execution defaults surface real mechanisms. Comparisons that previously ended at âsystem A wins by X%X\%â now routinely end at which configurations each system chose and what dominated the alternativesâthe level at which architects can connect results to intuition, or flag that a simulator assumption (not the hardware) drove the delta. Context capture is half the product. The single biggest usability gap between a generic chatbot and Coco is not model quality but that Coco knows what the user is looking at. Encoding the navigation trail as agent context eliminated most of the prompt-writing that made earlier assistants feel slower than doing the analysis by hand. Our headline target is a 2Ă2Ă reduction in time-to-simulation and time-to-insight. 5 Case studies Two recent analyses exercise the stack end to end. Both involve public TPU generations: the deployed Ironwood systems and the announced eighth-generation inference chip, TPU 8i (Google, 2026). 5.1 Generation-over-generation analysis: Ironwood to TPU 8i We tasked Coco with a comparative report between a baseline Ironwood (Google Cloud, 2025) system and a (then) next-generation TPU 8i configuration, asking specifically how the increased on-chip SRAM and compute change execution. Coco synthesized the simulation sweeps into a causal, regime-scoped insight rather than a headline ratio: tripling the scratchpad (128 MB to 384 MB) specifically benefits MoE models with fewer than 64 experts under a sub-2-second latency constraint. It also explained why: at those model shapes, the larger SRAM admits a batch-size combination at which expert tensors can be pinned and prefetched to fit the new capacity exactly, taking HBM refetch off the critical pathâand outside that regime (more experts, or looser latency targets) the benefit washes out. Every quoted number in the report links to its generating SQL. 5.2 Network topology what-if: torus versus Boardfly Ironwood pods connect chips in a torus; TPU 8i moves to a hierarchical Boardfly topologyâfully connected four-chip boards, fully connected eight-board groups, 36 groups per pod (Google, 2026). We used Coco for a what-if analysis of the two interconnects under highly sharded MoE serving. Orchestrating the iso-execution tool chain, the agent held sharding and batch size fixed across topologies to isolate the networkâs contribution, showing how Boardfly reshapes the all-to-all dispatch and combine latencies that dominate expert-parallel execution. It then generated side-by-side timeline breakdowns from which the architect could visually verify the bottleneck shifting from network-bound to compute-bound executionâthe level of evidence at which a topology discussion stops being a matter of taste. 6 Conclusion Hardwareâsoftware co-design requires more than generic language models; architects need tools capable of reasoning over fresh simulation data and executing complex, context-dependent workflows. Coco addresses this gap by shifting the burden of data archaeology and routine analysis from the expert to an agentic platform. Rather than relying on model memorization, Coco grounds its insights in a relational datastore and automates recurring tasks like iso-execution comparisons, all within an interface built for active architectural exploration. While prior work has demonstrated the value of machine learning in physical design (Mirhoseini et al., 2021), domain-specific chat assistants (Liu et al., 2023), and early co-design productivity studies (Sreedhar et al., 2025), Coco elevates this assistance directly to the system-architecture level where the highest-stakes decisions are made. By significantly accelerating time-to-insight for architects today, this four-layer stack provides a practical foundation for the autonomous co-designers of tomorrow. References Fedus et al. [2022] William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research, 23(120):1â39, 2022. Google [2026] Google. Our eighth generation TPUs: two chips for the agentic era. https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/eighth-generation-tpu-agentic-era/, 2026. Google Cloud [2025] Google Cloud. Ironwood: The first Google TPU for the age of inference. https://blog.google/products/google-cloud/ironwood-tpu-age-of-inference/, 2025. Jouppi et al. [2023] Norman P Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, et al. TPU v4: An optically reconfigurable supercomputer for machine learning with hardware support for embeddings. In Proceedings of the 50th Annual International Symposium on Computer Architecture, pages 1â14, 2023. Lewis et al. [2020] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich KĂŒttler, Mike Lewis, Wen-tau Yih, Tim RocktĂ€schel, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33:9459â9474, 2020. Liu et al. [2023] Mingjie Liu, Teodor-Dumitru Ene, Robert Kirby, Chris Cheng, Nathaniel Pinckney, Rongjian Liang, Jonah Alben, Himyanshu Anand, Sanmitra Banerjee, Ismet Bayraktaroglu, et al. ChipNeMo: Domain-adapted LLMs for chip design. arXiv preprint arXiv:2311.00176, 2023. Mirhoseini et al. [2021] Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. A graph placement methodology for fast chip design. Nature, 594(7862):207â212, 2021. Schick et al. [2023] Timo Schick, Jane Dwivedi-Yu, Roberto DessĂŹ, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2023. Sreedhar et al. [2025] Kavya Sreedhar, Josh Ogbonda, Pengqi Yin, Narges Shahidi, Kanthi Nagaraj, Zhijie Deng, Rami Cohen, Ton Kalker, Sameer Kumar, Amir Yazdanbakhsh, and Suvinay Subramanian. Leveraging LLMs to improve hardware-software co-design workflow productivity and accessibility. In Workshop on Machine Learning for Computer Architecture and Systems (MLArchSys) at ISCA, 2025. Yao et al. [2023] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023.