Paper deep dive
Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production
Arun Malik
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 7/9/2026, 6:46:39 AM
Summary
Progressive crystallization is a lifecycle methodology that transforms stochastic, high-cost AI agent explorations into deterministic, lower-cost workflows. It introduces a three-stage execution taxonomy (Type 3: agent-orchestrated, Type 2: hybrid, Type 1: deterministic) governed by an evidence-based promotion and demotion mechanism. Evaluated in a production cloud networking AIOps system, the approach increased deterministic execution to 45%, reduced per-incident costs by over 70%, and improved safety and reproducibility over an eight-month period.
Entities (10)
Relation Signals (10)
Progressive Crystallization → defines → Execution-Type Taxonomy
confidence 95% · We define an execution-type taxonomy spanning three points, from fully agent-orchestrated to fully deterministic
Execution-Type Taxonomy → includes → Type 2: Hybrid
confidence 95% · A Type 2 (hybrid) playbook has a fixed step structure in which only specific stages invoke the LLM
Execution-Type Taxonomy → includes → Type 1: Deterministic
confidence 95% · A Type 1 (deterministic) playbook is pre-coded logic that runs with the same guarantees as traditional workflow automation
Execution-Type Taxonomy → includes → Type 3: Agent-orchestrated
confidence 95% · A Type 3 (agent-orchestrated) playbook lets the agent investigate freely within a bounded scope
Progressive Crystallization → reduces → LLM inference cost
confidence 95% · per-incident agent cost fell by more than 70 percent while incident volume doubled
Promotion Mechanism → promotes → Type 2: Hybrid
confidence 92% · After further successful hybrid runs without LLM disagreement, remaining LLM steps are replaced with deterministic equivalents
Promotion Mechanism → promotes → Type 3: Agent-orchestrated
confidence 92% · After repeated successful runs of the template, trace analysis identifies steps where the LLM consistently produces the same classification
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:AI agents deployed for IT operations are typically permanent cost centers because every execution requires full LLM inference, even for previously solved problems. This paper introduces progressive crystallization, a lifecycle that treats agent exploration as a discovery mechanism rather than a permanent execution model. It defines a three-stage execution taxonomy, from fully agent-orchestrated to hybrid to fully deterministic workflows, together with an evidence-based promotion mechanism that converts repeatedly validated agent behaviors into cheaper and more reproducible deterministic workflows, while automatically demoting workflows that regress. Evaluated on a production cloud networking AIOps system processing tens of thousands of incidents per month, the approach increased deterministic execution from 0% to 45% over eight months, reduced per-incident agent costs by more than 70% despite doubling incident volume, and improved safety through greater reproducibility and auditability. The paper also presents the execution taxonomy, promotion and demotion criteria, trace extraction methodology, economic model, safety considerations, and discusses limitations and threats to validity.
Tags
Links
- Source: https://arxiv.org/abs/2607.07052v1
- Canonical: https://arxiv.org/abs/2607.07052v1
Trouble viewing inline? Open PDF directly →
Full Text
16,977 characters extracted from source content.
Expand or collapse full text
Progressive Crystallization: Turning Agent Exploration into Deterministic, Lower-Cost Workflows in Production Arun Malik Microsoft Azure Networking Email: arunma@microsoft.comORCID: 0009-0005-6650-6711 Abstract—AI agents deployed for IT operations are typically permanent cost centers: every execution re-invokes full LLM inference, even for problems the agent has already solved many times. We present progressive crystallization, a lifecycle that treats agent exploration as a discovery mechanism rather than a per- manent execution model. We define an execution-type taxonomy spanning three points, from fully agent-orchestrated (stochastic, expensive) through hybrid to fully deterministic (zero-token, reproducible), and an evidence-based promotion mechanism that converts an agent’s repeatedly validated behavior into progres- sively cheaper and more deterministic workflows, with automatic demotion when a promoted workflow regresses. This makes an agentic platform cheaper, faster, and safer over time without human rewriting of agent-discovered patterns. In a production agentic system for cloud network operations handling tens of thousands of incidents per month, the share of executions served by deterministic workflows rose from zero to 45 percent over eight months, per-incident agent cost fell by more than 70 percent while incident volume doubled, and safety properties improved monotonically across promotions because determinism increases reproducibility and auditability. We describe the taxonomy, the promotion and demotion criteria, the trace-extraction method, an economic model, and the safety argument, and we discuss limitations and threats to validity. Index Terms—agentic AI, AIOps, workflow automation, LLM cost optimization, process mining, deterministic execution, pro- gressive autonomy, safety I. INTRODUCTION Large language model (LLM) agents are increasingly de- ployed for IT operations tasks such as incident triage, root- cause analysis, and automated remediation [1], [2]. An agent observes system state through tool calls, reasons about the problem, and executes actions in a loop. This flexibility is exactly what lets an agent handle novel incidents, but it also makes the agent a permanent cost center. Every execution re- invokes full model inference, so the same recurring incident consumes thousands of tokens each time it recurs, the inves- tigation path is non-deterministic and hard to reproduce, and total cost scales linearly with incident volume. The core waste is that a successful agent investigation is discarded. When an agent resolves an incident through a novel path, that knowledge is lost; the next occurrence of the same failure re-discovers the solution from scratch, at the same token cost and with a possibly different and inferior result. Existing options do not close this gap. Traditional workflow engines cannot handle novel scenarios and require an engineer- ing sprint per new automation. Unconstrained agents never get cheaper. Fine-tuning yields a smaller probabilistic model rather than a deterministic workflow, and recorded-macro or robotic- process-automation approaches capture surface actions, not the reasoning or the data flow, and are brittle to environment change. We argue that agent exploration should be a discovery mechanism, not a permanent execution model. Behavior that an agent discovers and validates through repeated successful execution can be systematically converted into deterministic workflows that require zero LLM tokens to run, while the agent layer remains available for genuinely novel problems. We call this lifecycle progressive crystallization. This paper makes four contributions: • an execution-type taxonomy that places operational workflows on a spectrum from fully agent-orchestrated to fully deterministic (Section I); • a promotion and demotion lifecycle that advances work- flows down the spectrum based on accumulated evidence, and reverts them on regression (Section IV); • an economic model showing that platform inference cost decreases over time even as automation volume grows (Section V); and • a safety-monotonicity argument that each promotion preserves or improves safety, supported by production evidence from a cloud-network agentic system (Sec- tions VI–VIII). I. RELATED WORK LLM agents that reason and act through tools were popu- larized by ReAct [3] and Toolformer [4], and the design space is surveyed in [5], [6]. Applying these agents to operations is the focus of AIOps [7] and its LLM-era successors [1], [2]. Prior work largely treats the agent as the permanent execution engine. Cost-reduction efforts such as FrugalGPT [8] lower per-call cost through model cascades and routing, but the system remains probabilistic and never eliminates inference for solved problems. Our contribution is orthogonal and com- plementary: rather than making each inference cheaper, we remove inference entirely for work that has been proven, by extracting deterministic workflows from execution traces. The extraction step draws on process mining [9], which recovers process models from event logs; here the event logs are agent arXiv:2607.07052v1 [cs.SE] 8 Jul 2026 TABLE I EXECUTION-TYPE TAXONOMY. TypeExecutionDeterm.Token cost Type 3: Agent- orchestrated Sub-agents reason within bounded scope; deterministic checkpoints andHITL gates ∼50%High (10k– 50k) Type2: Hybrid Structured steps; specific stagescall the LLM for interpretation or classification only ∼90%Low(1k– 5k) Type1: Determin- istic Pre-coded logic, conditionals, and typed API calls; no LLM at runtime 100%Zero execution traces and the recovered models are executable playbooks. I. EXECUTION-TYPE TAXONOMY We define three execution types for operational playbooks, forming a spectrum from fully stochastic to fully deterministic (Fig. 1, Table I). A Type 3 (agent-orchestrated) playbook lets the agent inves- tigate freely within a bounded scope, combining autonomous read operations with deterministic checkpoints and human-in- the-loop (HITL) approval for write operations. It is the entry point for any novel incident. A Type 2 (hybrid) playbook has a fixed step structure in which only specific stages in- voke the LLM, and only for interpretation, classification, or summarization; every action is typed and schema-validated, so the LLM decides understanding, not what to do. A Type 1 (deterministic) playbook is pre-coded logic that runs with the same guarantees as traditional workflow automation, at zero token cost and full reproducibility. The three types are not separate systems; they are lifecycle stages of the same discovered behavior. IV. THE CRYSTALLIZATION LIFECYCLE Crystallization advances a playbook from a higher (more stochastic) execution type to a lower one as evidence accumu- lates. Stage 1: discovery. A novel incident with no matching playbook triggers a Type 3 execution. The agent investigates using available tools; read operations run autonomously and write operations pause for approval. The complete execution trace is recorded. TABLE I EVIDENCE-BASED PROMOTION CRITERIA (DEFAULTS; CONFIGURABLE). TransitionRequirements Type 3 → 2 ≥10 successful runs; zero safety violations; ≥90% of runs produce the same action se- quence; all auto-generated acceptance tests pass; no human override in the recent window Type 2 → 1 ≥50 successful hybrid runs; LLM classifica- tion consistency ≥99%; the deterministic rule covers all observed input variation; full re- gression suite passes without the LLM; human review of the deterministic logic Stage 2: capture. On verified successful resolution, the successful path is extracted into a reusable Type 3 template. The extraction algorithm parses the trace into an ordered list of tool calls, detects the branch conditions the agent acted on, infers input and output schemas per step, builds a directed acyclic graph of tool dependencies, parameterizes instance- specific values such as device identifiers and timestamps, and marks human-approval points as explicit gates. Stage 3: promotion to hybrid. After repeated successful runs of the template, trace analysis identifies steps where the LLM consistently produces the same classification (replaced by a deterministic rule) and steps where reasoning varies but the outcome is stable (replaced by a scoped, single-purpose prompt). Acceptance tests are generated automatically from the successful traces, and the candidate Type 2 playbook must pass them. Stage 4: promotion to deterministic. After further suc- cessful hybrid runs without LLM disagreement, remaining LLM steps whose output is drawn from a finite set, or whose decision boundary can be expressed as a rule, are replaced with deterministic equivalents. The final Type 1 playbook needs no runtime tokens and continues to be validated by the Stage 3 acceptance tests. Promotion is gated by the criteria in Table I. Crucially, autonomy is attached to the specific playbook class and action type, based on its evidence, rather than to the capability of the underlying model. A more capable model does not automatically earn more autonomy; a track record does. V. ECONOMIC MODEL Each execution type has a characteristic cost (Fig. 2). A Type 3 run consumes roughly 10,000 to 50,000 tokens and completes in seconds to minutes; a Type 2 run consumes on the order of 1,000 to 5,000 tokens; a Type 1 run consumes none and completes in milliseconds. The key property is not the cost of any single run but the shift in the mix over time. As recurring patterns crystallize, the platform routes each incoming request to the lowest-cost type available for that pattern, so the fraction of Type 1 executions grows and total inference cost falls even as automation volume rises. The agent layer becomes a discovery mechanism whose cost is amortized Fig. 1. The execution-type spectrum and crystallization lifecycle. Workflows are promoted rightward, toward determinism and zero token cost, only on accumulated evidence, and are demoted leftward automatically on failure, drift, or test regression. Fig. 2. Cost per run collapses across execution types. Latency and per-incident dollar cost are annotated above each bar. across all future deterministic executions of what it discovers, rather than a cost incurred on every occurrence. VI. SAFETY MONOTONICITY A natural concern is whether making a workflow cheaper makes it less safe. We argue the opposite: crystallization pre- serves or improves every safety property we track (Table I). Auditability is unchanged, since all three types record full traces. Reproducibility rises monotonically, from roughly 50 percent for Type 3 to 100 percent for Type 1. Blast-radius control moves from runtime HITL gates (Type 3) to schema validation (Type 2) to statically verifiable deterministic logic (Type 1), which is stronger because it can be checked before execution rather than caught during it. Compliance shifts from conditional (depends on a human being present) to built-in. In short, a deterministic playbook is easier to verify, reproduce, and audit than the agent run it was distilled from, so safety and cost improve together rather than trading off. TABLE I SAFETY PROPERTIES ACROSS EXECUTION TYPES. PropertyType 3Type 2Type 1 AuditabilityFull traceFull traceFull trace Reproducibility ∼50% ∼90%100% Blast-radius con- trol HITL gatesSchema validation Deterministic logic ComplianceConditionalBuilt-inBuilt-in VII. DEMOTION AND CONTINUOUS DISCOVERY Crystallization is not one-way. Each promoted playbook is monitored, and a circuit breaker demotes it to a higher execu- tion type on execution failure, safety violation, or acceptance- test regression. In production, a deterministic playbook once broke after a firmware update changed a command’s output format; the deterministic parser failed, the system demoted the playbook to hybrid so the LLM could handle the new format, and after a run of clean executions it was re-promoted. This gives the platform the reliability of deterministic automation for stable patterns and the adaptability of agents for change, without a human deciding when to switch. Genuinely novel incidents always enter as Type 3, so the discovery pipeline never closes. VIII. PRODUCTION EVALUATION We deployed progressive crystallization in a production agentic platform for cloud network operations that resolves incidents across a large managed network and handles tens of thousands of incidents per month [2]. We report three observations. The mix shifts toward determinism. At launch, essentially all executions were Type 3. Over eight months the share of deterministic (Type 1) executions rose from zero to about 45 percent, with roughly 30 percent hybrid and 25 percent agent- orchestrated (Fig. 3). The ratio of Type 1:2:3 executions is a useful platform-maturity metric. Fig. 3.Share of executions by type over eight months in production. Deterministic workflows grow from zero to about 45 percent as patterns crystallize, while agent-orchestrated executions fall to about a quarter. Cost falls as volume rises. Over the same period, per- incident agent cost fell by more than 70 percent while incident volume roughly doubled. This is the central economic claim of the paper realized in production: the platform got cheaper as it did more, because it stopped paying for inference on work it had already learned. Autonomy and quality held. The platform resolves over 90 percent of common incident categories autonomously, mean time to resolution fell from hours to minutes, and the false- positive remediation rate stayed under 5 percent with no customer-visible impact. Crystallization did not trade quality for cost; the deterministic paths are the ones that had already proven reliable as agent runs. IX. DISCUSSION, LIMITATIONS, AND THREATS TO VALIDITY Results come from a single organization and operational domain, so specific thresholds and ratios should be re-derived elsewhere; the lifecycle itself is domain-agnostic. Crystalliza- tion assumes recurring patterns, so its benefit is smaller in environments dominated by genuinely novel, one-off incidents, where most executions will remain Type 3. The economic figures are platform-level observations rather than a controlled comparison, and the eight-month window reflects one maturity trajectory. Automatic promotion depends on the quality of the acceptance tests generated from traces; a pattern that is under-observed can be promoted prematurely, which is why demotion and human review of the final deterministic logic are part of the design. Finally, extraction quality depends on the richness of execution traces; sparse or poorly structured logging limits what can be crystallized, echoing a known dependency in process mining. X. CONCLUSION Agent platforms do not have to be permanent cost centers. By treating agent exploration as discovery and progressively crystallizing proven behavior into deterministic, zero-token workflows, an agentic system can become cheaper, faster, and safer over time without human rewriting. In production, the mix of executions shifted from fully agent-orchestrated toward deterministic, cost fell by more than 70 percent as volume doubled, and safety improved monotonically because determinism buys reproducibility and verifiability. We believe the Type 1:2:3 ratio is a practical maturity metric for any team operating agents at scale, and that the discipline of promoting and demoting workflows on evidence is a general pattern for making autonomy both affordable and trustworthy. REFERENCES [1] L. Zhang, T. Jia, M. Jia, Y. Wu, A. Liu, Y. Yang, Z. Wu, X. Hu, P. S. Yu, and Y. Li, “A survey of AIOps in the era of large language models,” arXiv preprint arXiv:2507.12472, 2025. [2] A. Malik, “Autonomous incident resolution at hyperscale: An agentic AI architecture for network operations,” arXiv preprint arXiv:2606.09122, 2026. [3] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in Inter- national Conference on Learning Representations (ICLR), 2023. [4] T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettle- moyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [5] Z. Xi et al., “The rise and potential of large language model based agents: A survey,” arXiv preprint arXiv:2309.07864, 2023. [6] M. Wooldridge, An Introduction to MultiAgent Systems, 2nd ed.John Wiley & Sons, 2009. [7] P. Notaro, J. Cardoso, and M. Gerndt, “A survey of AIOps methods for failure management,” ACM Transactions on Intelligent Systems and Technology, 2021. [8] L. Chen, M. Zaharia, and J. Zou, “FrugalGPT: How to use large language models while reducing cost and improving performance,” arXiv preprint arXiv:2305.05176, 2023. [9] W. M. P. van der Aalst, Process Mining: Data Science in Action, 2nd ed. Springer, 2016.