Paper deep dive

Automated Synthesis of Cloud Emulators

Archit Bhatnagar, Zhenning Yang, Sarah McClure, Yiming Qiu, Sylvia Ratnasamy, Ang Chen

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 80

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%

Last extracted: 8/26/2026, 4:48:40 AM

Summary

The paper introduces CloudEmu, an automated approach for synthesizing cloud emulators using neurosymbolic code synthesis. It addresses the limitations of manual emulator development (like LocalStack) by combining Large Language Models (LLMs) for documentation understanding with cloud-specific symbolic abstractions (state machines) to ensure precision. CloudEmu constructs emulators from cloud provider documentation (AWS, GCP) and uses the real cloud as an oracle for automated testing and repair, achieving higher coverage and accuracy than existing tools.

Entities (10)

AWS · cloud-provider · 99%GCP · cloud-provider · 99%CloudEmu · software-system · 98%EC2 · cloud-service · 95%LocalStack · software-system · 95%Subnet · cloud-resource · 95%VPC · cloud-resource · 95%State Machine · concept · 92%LLM · technology · 90%Terraform · software-system · 90%

Relation Signals (8)

VPC → contains → Subnet

confidence 95% · A Virtual Private Cloud resource... can contain several Subnet resources

CloudEmu → evaluatedon → AWS

confidence 95% · Our evaluation shows the effectiveness of CloudEmu on major cloud provider (AWS and GCP) services

CloudEmu → evaluatedon → GCP

confidence 95% · Our evaluation shows the effectiveness of CloudEmu on major cloud provider (AWS and GCP) services

CloudEmu → outperforms → LocalStack

confidence 95% · CloudEmu outperforms the existing leading tool LocalStack... in both coverage and accuracy.

CloudEmu → uses → State Machine

confidence 92% · each resource can be modeled as a state machine (SM)... CloudEmu exploits this structure

LocalStack → hascoverageof → EC2

confidence 90% · even on popular services coverage is under 40%... Compute (EC2) ... 32%

Terraform → istypeof → IaC Framework

confidence 90% · Infrastructure-as-Code (IaC) frameworks (e.g., Terraform)

CloudEmu → uses → LLM

confidence 90% · combine LLMs' general strengths in documentation understanding and code generation

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:DevOps programming (e.g., using CLI/API scripts or IaC frameworks) is key to cloud infrastructure management. Unlike traditional programming tasks, DevOps program testing needs provisioning and execution against actual cloud resources, which is often time-consuming, unsafe, and costly. Cloud emulators have gained popularity for easing DevOps program testing; they are generally API-level mocks that can execute DevOps programs in a local environment. Still, building these emulators remains challenging: developers must manually interpret extensive cloud documentation and handcraft logic for each service, API, and their interaction. This does not scale to the complexity of the cloud, which is further a moving target as the services and APIs evolve. CloudEmu is an automated approach that constructs emulators based on cloud documentation via neurosymbolic code synthesis. The key idea is to combine LLMs' general strengths in documentation understanding and code generation with cloud-specific symbolic abstractions that suppress hallucinations and enforce precision at scale, while using the real cloud as an oracle for automated testing, repair, and alignment. Our evaluation shows the effectiveness of CloudEmu on major cloud provider (AWS and GCP) services in both coverage and accuracy. CloudEmu outperforms the existing leading tool LocalStack, which was manually developed by a large team of engineers over a decade.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

76,635 characters extracted from source content.

Expand or collapse full text

Automated Synthesis of Cloud Emulators Archit Bhatnagar Zhenning Yang Sarah McClure‡ Yiming Qiu† Sylvia Ratnasamy‡ Ang Chen Affiliation: University of Michigan †The University of Hong Kong ‡University of California, Berkeley Abstract. DevOps programming (e.g., using CLI/API scripts or IaC frameworks) is key to cloud infrastructure management. Unlike traditional programming tasks, DevOps program testing needs provisioning and execution against actual cloud resources, which is often time-consuming, unsafe, and costly. Cloud emulators have gained popularity for easing DevOps program testing; they are generally API-level mocks that can execute DevOps programs in a local environment. Still, building these emulators remains challenging: developers must manually interpret extensive cloud documentation and handcraft logic for each service, API, and their interaction. This does not scale to the complexity of the cloud, which is further a moving target as the services and APIs evolve. CloudEmu is an automated approach that constructs emulators based on cloud documentation via neurosymbolic code synthesis. The key idea is to combine LLMs’ general strengths in documentation understanding and code generation with cloud-specific symbolic abstractions that suppress hallucinations and enforce precision at scale, while using the real cloud as an oracle for automated testing, repair, and alignment. Our evaluation shows the effectiveness of CloudEmu on major cloud provider (AWS and GCP) services in both coverage and accuracy. CloudEmu outperforms the existing leading tool LocalStack, which was manually developed by a large team of engineers over a decade. 1. Introduction Building and maintaining cloud infrastructure (e.g., virtual machines, gateways) is an essential task. DevOps engineers perform these tasks programmatically—e.g., using CLI commands or Python scripts that directly invoke cloud-level APIs (49, 11) exposed by the provider, or using Infrastructure-as-Code (IaC) frameworks (42, 44, 6) (e.g., Terraform) that compile infrastructure configurations to API calls. However, testing against the cloud is expensive (, 2021), and resource provisioning can be time-consuming. Prices and provisioning time can further increase for resources in high demand (Yang et al., 2023). This complicates DevOps programming and reduces velocity. To enable no-risk, no-cost, and high-velocity cloud development, cloud emulators (33, 38, 46) are quickly gaining traction. Emulators mimic the cloud by exposing identical API interfaces to DevOps programs and simulating their execution in a mock environment, providing a lightweight backend without going through the real cloud. In order to emulate a resource (e.g., VM), emulator developers sift through cloud documentation, identify target APIs, and handcraft the mockup logic based on their understanding of the expected behavior. Interdependent resources (e.g., VM is associated with Subnet and VPC) further need to be emulated in relation to each other. While this is a laudable effort, existing practices of emulator development cannot catch up to the complexity and dynamicity of the cloud ecosystem. For instance, AWS alone provides 240 services (2), and a service can expose up to 200 APIs; Azure and GCP exhibit a similar level of complexity. Market competition means that providers often add new services and upgrade existing ones, making the cloud a moving target (50). Each cloud provider also features a different set of services and APIs, and more players are entering the cloud market. To ensure reliability at scale, tenants often construct multi-cloud deployments, which further propagates the need for DevOps testing across cloud providers. Hence, manual emulator development is a tedious process that needs to be repeated for each provider and turns out to be increasingly difficult. The stark reality is that even the most advanced emulator (33) today only covers 95 out of over 240 AWS services, and only with partial API coverage for these services; and no mature multi-cloud emulators exist to our knowledge. In this paper, we propose a new approach to building cloud emulators. We automatically synthesize emulation logic from cloud documentation, and then align the generated emulator with the live cloud to close any remaining gaps. We observe that publicly available cloud documentation provides a treasure trove of information. No matter how complex a cloud’s services may be, their usage and behavior is often described in painstaking detail by the cloud provider. This is fundamental to the cloud’s business model, as providers need to sufficiently describe their interfaces to the tenants. The incentives are strong for providers to comprehensively document their services and keep the information up to date. While poring over these pages is difficult for human developers, Large Language Models (LLMs) have the potential to digest this information and generate emulation code for the described behavior automatically. If successful, this new kind of emulators will not only achieve higher service coverage, but can also easily adapt to service changes, and generalize across providers without multiplying the engineering effort. This version is an extension of our preliminary attempt at the problem (Bhatnagar et al., 2025). We design three steps to mimic the workflow of a (human) emulator developer: digesting swaths of information from documentation, coding emulation logic for each resource and its interactions, and testing the emulator against the actual cloud to find and close any gaps. While LLMs can help with each step, they can also hallucinate and introduce arbitrary errors. Our key observation is that cloud services follow a highly structured style where each resource can be modeled as a state machine (SM). transitions are triggered by API invocations and may further affect the states and transitions of other resources. CloudEmu exploits this structure throughout all steps to impose precise constraints and provide high assurance—a neurosymbolic workflow that “compiles” text-based documentation to working code, further tested and refined against the actual cloud as the oracle. We call the first step documentation wrangling, where the key challenge stems from the sheer volume of cloud documentation. While LLMs can read/parse voraciously, they can also forget key information after thousands of pages. Retrieval-augmented generation (Lewis et al., 2020; Guu et al., 2020) uses a vector database to help LLMs locate relevant parts of the documentation selectively, but its indexing mechanism is imprecise in nature, using semantic match in the embedding space (e.g., top-k based on the prompt); further, understanding the whole document is particularly important, as cloud documentation exhibits dense cross-references (e.g., VM definitions refer to NIC attachments, documented far apart). CloudEmu develops a resource index, that leverages the service and API boundaries, threading through the documentation based on the structure of cloud services for easier LLM consumption. The previous step indexes and compiles text documentation into a structured representation; and the next code generation step further compiles any remaining textual descriptions into fully symbolic code. We insist that the LLM generates code following the index structure, translating text to code to fill in “holes” in a preprogrammed template that models a hierarchy of state machines (SM). Cloud services are complex but modular, allowing the overall task to be decomposed. This template preprograms key components symbolically. Resources are modeled as state machines (SM), and their hierarchical and dependency relationships constrain how resource states can evolve under API operations. For example, deleting a parent resource may require first reclaiming its children, while creation and update operations must preserve consistency across dependent resources. As a result, the LLM only needs to infer, for each API, the corresponding behavior from a small fragment of API description extracted from the large cloud documentation, within this constrained scaffold. The final step is program alignment which addresses potential gaps that arise from textual ambiguity or under-documentation, testing, and then aligning the behavior of the synthesized emulator to that of the actual cloud. It leverages the structure of the template to perform targeted testing, producing high-coverage traces to exercise different program paths. Alignment means that permissible behaviors should produce the same effects in the emulator and the cloud, and forbidden behaviors should fail in both; ideally, failures should also result in identical error codes and error messages to assist with DevOps debugging. By testing the effect of these traces in both environments, we can detect divergence, track down the source of errors, e.g., to a specific SM implementation, a specific interaction, or even back to the cloud documentation, and perform LLM-assisted program repair. These technical designs also contribute key takeaways for building software systems with the assistance of AI. (1) LLMs are most effective when used selectively to infer semantics where needed, rather than end-to-end system generation. (2) Structured constraints enable generation at scale: symbolic scaffolds that capture architecture and interfaces allow large systems to be synthesized reliably from smaller, composable pieces. (3) LLMs can produce plausible but incorrect outputs, and grounding the resulting systems on external oracles and feedback loops can help with continuous validation and alignment. Our extensive evaluation shows the effectiveness of CloudEmu and validates its high coverage and accuracy over leading, manually-built emulators like LocalStack. 2. Motivation Cloud infrastructure is increasingly managed through code. Whether declarative, IaC/Infrastructure-as-Code configurations (e.g., Terraform (44) and similar tools (6, 42)), or imperative scripts (e.g., Python CLI and SDK programs), these DevOps programs all eventually invoke cloud-level APIs to provision and update resources (e.g., VMs, subnets). DevOps programs are safety-critical software, so they must be thoroughly tested and validated. However, testing against the actual cloud is costly, as API invocations are metered and charged; it also introduces high latency (e.g., resource provisioning delay); buggy programs can further endanger the cloud environment. To enable high velocity DevOps, emulators that mock up cloud APIs have garnered immense popularity; for instance, they respond to CreateVM() calls by adding a mock VM name, state, and location to internal emulator state, but without actually creating actual resources. Emulators are an integral part of the CI/CD pipeline. For instance, a buggy Terraform program may create a VM and its NIC in two different cloud regions; such bugs cannot be detected in the compilation pass (Qiu et al., 2024), but API-level emulation will expose this bug before it hits actual infrastructure. Obviously, this mock cloud infrastructure has its limitations: it cannot host real workloads or faithfully mimic the cloud’s performance characteristics. Nevertheless, DevOps testing in a frictionless environment is a key value add—the leading AWS emulator, LocalStack (33) (with Moto as the backend), has received more than 60k Github stars. 2.1. The Pitfalls of Manual Construction Existing cloud emulators are manually constructed, which is a sisyphean effort, as the emulation logic often struggles to keep pace with the ever-expanding cloud ecosystem. The cloud is simply too vast. AWS alone has over 240 services, with a single service like EC2 exposing upward of 500 APIs. Furthermore, the cloud is a moving target. AWS announces 3000 changes and new features in a year (9), requiring repeated emulator re-engineering and maintenance to keep abreast. This approach leads to both coverage and accuracy limitations. Low coverage: As Table 1 shows, even popular resources like EC2 have incomplete support, and other resources have even lower coverage. For instance, for Network Firewall—a critical service for cloud security policies—LocalStack only implements CreateFirewall() but misses DeleteFirewall(), UpdateFirewallPolicy(), and AssociateSubnets() operations. This forces developers to maintain complex and brittle testing setups that use the emulator for some resources and the real cloud for others. Across the four services shown, the overall coverage is approximately 35%. Further, the penalty of low coverage is combinatorial because an entire DevOps program could fail to execute even if there is one resource/API missing. Low accuracy: Furthermore, subtle behavioral differences between the emulator and the real cloud are common (Mazhar et al., 2025) and can only be revealed by cloud-based testing. For instance, LocalStack allows the DeleteVpc() call to succeed even if the VPC contains an Internet Gateway, while the real AWS would reject this API with a “DependencyViolation” error; calling StartInstances() on a running VM will produce an “IncorrectInstanceState” error in AWS, but LocalStack returns a success code instead. As the root cause, manual implementation is not infallible, which is compounded by the fact that textual documentation can be ambiguous or incomplete (e.g., using “etc.” when describing a set of attributes). Left unaddressed, this would undermine the reliability of the emulator as a testing tool, allowing incorrect code to pass through. Table 1. API coverage of LocalStack (33), the leading cloud emulator; even on popular services coverage is under 40%. Services APIs Emulated Coverage Compute (EC2) 756 244 32% Firewall (WAF) 55 30 54% DNS (Route 53) 68 51 75% CDN (CloudFront) 167 44 26% Overall (subset) 1046 369 ~35% Tremendous effort: LocalStack has accumulated over 800 contributors and 10k+ commits across more than a decade of active development. Scaling emulator development to a complex, stateful system that evolves over time is fundamentally hard. Covering a new API requires an engineer to monitor changes, read its documentation, understand its effect on the underlying resource’s state, implement that logic correctly, and write tests; further, APIs do not exist in isolation—the developer must build a mental model of how one resource/API affects other dependent resources and implement their interactions. For instance, CreateSubnet() must validate that the referenced VPC exists, record the subnet as a child of that VPC, and later prevent DeleteVpc() from succeeding while the subnet remains. Although these behaviors are documented by the provider, the information is scattered across thousands of pages, so capturing everything completely is a daunting task. This labor-intensive process cannot keep pace with a cloud that adds hundreds of APIs per year. 2.2. CloudEmu: The Case for Automated Synthesis Fundamental to tackling this task is the ability to pore over many pages efficiently, and translate textual description to running code. Fortunately, LLMs have demonstrated remarkable capabilities in both text comprehension and code generation, and can operate tirelessly to automate this process. LLM automation also provides other benefits—it can generalize across cloud providers, and capture service/API changes with periodic relearning, without repeated manual reengineering. However, just prompting an LLM to read cloud documentation and generate the emulator code directly performs poorly, exhibiting a wide range of errors—due to context limitations, hallucinations, and the inherent imperfections of documentation. To establish intuition behind CloudEmu’s techniques, we walk through a simplified example with two resources. VPC. A Virtual Private Cloud resource is a virtual deployment environment, and it can contain several Subnet resources; it exposes the following APIs. • CreateVpc(CidrBlock): Creates a VPC with the specified CIDR block. Inputs: CidrBlock (str, required); Returns: VPC VpcId (str), CidrBlock (str) or ErrCode (enum) • DeleteVpc(VpcId): Deletes the specified VPC. The VPC must not have any attached Subnets (link to the subnet page) or other resources. Inputs: VpcId (str, required); Returns: Success or ErrCode (enum) Subnet. A Subnet has several CIDR blocks for a VPC. The CIDR blocks should be valid and within the VPC CIDR range, and they must not conflict with other Subnets in the same VPC. It exposes the following APIs. • CreateSubnet(VpcId): Creates a Subnet resource within a given VPC. Inputs: VpcId (str, required); Returns: ErrCode (enum) • ⋯·s Observation #1: Unique documentation structure. We observe that cloud documentation exhibits distinctive characteristics compared to generic textual data. It has a clear structure and clean pagination based on service and API boundaries. As shown in the illustrative example above, resource descriptions always start with a resource name, followed by a brief textual description, API inputs, outputs, and error codes. The documentation style is also highly consistent across resources, starting with a comprehensive resource catalog in the beginning pages, followed by marked sections on resources such as VPC and Subnet. This well-defined structure affords a unique opportunity to perform symbolic parsing and retrieval to reduce ambiguity and relieve LLM burdens. Instead of RAG-based semantic indexing, each resource is now associated with a feature map, where the key is symbolic (e.g., the resource name) and the value can be textual (e.g., resource description) or symbolic (e.g., API signatures, response formats). Resources also have dependencies—the documentation shows that a Subnet depends on its VPC because CreateSubnet requires a VpcId. These dependencies are symbolic and are represented as edges in this index. Observation #2: State machine models. Building upon the above, we further observe that the documentation structure reflects the inherent nature of cloud resources: they are modular and encapsulate state that evolves only through API interactions, which can be modeled in a state machine abstraction. For instance, a VPC resource contains CidrBlocks and VpcId as state variables, modifiable via the CreateVpc and DeleteVpc APIs; likewise, a Subnet resource has its own state and state-mutating APIs. The essence of the emulator is to build a hierarchy of state machines, each representing a resource that encapsulates a certain state, further interacting with each other through clearly-defined APIs. This SM hierarchy further follows the dependency edges in the resource index—e.g., a VPC resource is a prerequisite of a Subnet. These state machines interact with each other, influencing the state space with APIs as transitions, as shown in Figure 1. As such, we built a generic template that models interacting state machines symbolically. By grounding code generation in a well-defined structure, we reap the benefits of good abstractions (Liskov and Zilles, 1974): encapsulating information within modules to separate concerns, and composing modules for programming at scale. Figure 1. Cloud is a hierarchical state machine allowing inter-SM transitions, resource nodes maintain per-resource state space, with create/destroy API calls to change the state space. Observation #3: Cloud-based alignment. Despite the foundational importance of cloud documentation—and providers’ incentives to keep it up to date—natural language descriptions remain inherently ambiguous and can be incomplete. Like human readers, LLMs are subject to the same range of ambiguities and potential misinterpretations, as they are trained on natural language corpora. For example, while experienced engineers understand that CIDR blocks within the same VPC must not overlap, others may not readily infer that phrases such as “conflicts with each other” in documentation refer specifically to IP address overlap. As a result, emulation logic may fail to enforce such constraints when creating Subnet resources. To address these limitations, validation against the cloud as a ground-truth oracle is necessary. This requires systematic test case generation that explicitly targets potential ambiguities and edge cases in the documentation. Moreover, the testing framework must detect misalignments between the emulator and the real cloud, such as cases where the emulator reports success while the cloud returns errors. 3. CloudEmu Design Figure 2. Overview of CloudEmu. Next, we expand on these observations and design three key techniques for documentation wrangling, emulation code synthesis, and automated alignment, as shown in Fig. 2; exploiting the unique structure of cloud resources & their documents. 3.1. Documentation Wrangling The cloud is vast, and so is its documentation. The sheer volume presents a challenge to LLMs because of their limited context window—the information needed to emulate a resource is scattered across multiple pages, and dependent resources (e.g., VPC and Subnet) may be documented far apart. Hence, a direct prompting approach does not work well; even retrieval augmented generation (RAG), which converts textual documentation into semantic embeddings for similarity-based retrieval, falls short because of imprecise matches. For instance, a RAG query on CreateVPC parameters may accidentally retrieve content from a semantically-similar API, CreateDefaultVPC, yet these two APIs have different input parameters; likewise, a query on VPCs may miss information about dependent resources, such as the state of the Subnet, Security Group, and Load Balancer resources, which are farther apart in the semantic space. The quality of this retrieval directly impacts emulator correctness, since missing information will lead to incorrect state and state-mutating logic. How would a human engineer handle this complexity? We observe that the cloud documentation reflects the modular nature of the resources—each is documented in well-defined boundaries and provides cross-references for precise navigation from one resource to its dependent resources (e.g., hyperlinks from VPC to Subnet). Further, resource documentation uses a combination of textual description (e.g., the function of a resource), and symbolic definitions (e.g., API signatures and response formats). While the former can be fuzzy and require semantic comprehension, which is imprecise in nature, the latter exists in codified forms and does not require “guesswork.” These unique characteristics allow developers to precisely locate and digest needed information, switching between “neural understanding” and “symbolic parsing” across different documentation components. Hence, our first technique is to construct a resource index from raw data in a documentation wrangling phase. Figure 3 shows a partial example of the resource index. The index captures dependencies among resources, and each node includes both structured fields, such as API parameters, and free-form textual descriptions that are later consumed by the LLM. Figure 3. Documentation wrangling builds a cloud resource index. Nodes are resources and edges are (strict or optional) dependencies. Building the index nodes. CloudEmu designs a symbolic parser that decomposes documentation based on service (e.g., EC2), resource (e.g., VPC), and API (e.g., CreateVPC) boundaries. This exploits the structural consistency across documentation pages. Each API has a fixed documentation structure: a textual description about its functionality, followed by marked subsections on input parameters (names, types, required/optional), response formats in JSON, error codes, and sometimes, usage examples. Each resource is an attribute map, with a canonicalized name (e.g., AWS:EC2:VPC) as the primary key. Each resource has a set of attributes that can be indexed hierarchically: e.g., AWS:EC2:VPC:CreateVPC is of the type APIDesc, and its attributes may take symbolic or textual values (e.g., codified API signature vs. functionality description in text). Building the index edges. The second parsing step captures the cross-resource dependencies, e.g., a Subnet lives within a VPC. This is done via a traversal of each resource’s API definitions to mark whether the APIs presuppose the existence of some other parent resource. This also includes optional dependencies, when one resource API references another resource as a non-required parameter, e.g., a Network Interface could be associated with but does not always require an Elastic IP. This resource index is better for LLM consumption than raw documentation. First, by reorganizing textual documentation in an explicit data structure, we can design algorithms to walk this index and guarantee generation completeness. A full traversal of the index systematically produces emulation logic for all resources and APIs. In contrast, directly consuming raw documentation can easily result in omissions of critical information or logic. Second, the index provides a clean separation between neural and symbolic processing. Information already encoded in structured, symbolic form can be deterministically compiled into code, whereas unstructured textual content is delegated to LLM-based translation. This delineation enables more reliable and interpretable code generation. Third, the modular structure of each node supports incremental generation. Because each node encapsulates a self-contained unit of information, the generation procedure can operate locally without needing to reason about unrelated resources. Finally, the index naturally induces a generation order. Emulation logic for parent resources are generated prior to their children, ensuring that each child’s state and logic can readily reference those from the parent resources. 3.2. Code Synthesis Next, CloudEmu synthesizes emulation code to implement the described cloud behavior. Although documentation wrangling significantly improves ease of information digestion, LLM-generated code is never immune from syntactic and semantic errors, e.g., incorrect response fields, missing state variables, and incompatible API interfaces. Our key insight is to constrain LLM synthesis using a well-defined, modular abstraction, as implemented in a symbolic framework. This programmatic scaffolding suppresses classes of LLM errors in an otherwise unfettered generation. We develop this scaffolding as a one-time effort in a resource-agnostic manner, implementing an abstract model of a cloud resource. The LLM generates code segments from the descriptions for a resource’s internals, while the rest is handled symbolically. The state machine abstraction. Abstractions separate concerns and facilitate composition (Liskov, 2009; Denning, 2025). CloudEmu models each resource as a state machine, where transitions are triggered by API invocations and may further affect the state and transitions of other resources. This allows individual modules to be written first and then assembled at scale, which is particularly useful for emulating many interacting resources. For instance, consider the following definition: ⬇ 1 /* An abstract state machine */ 2 SM sm 3 States S; //A collection of state vars 4 Transitions T; //Transitions modify state 5 This is simple yet general enough to model a resource without binding the abstraction to resource-specific implementations, which are filled in later when processing each index node. For instance, when compiling code for the VPC node, CloudEmu instantiates the abstract definition using VpcId and CidrBlocks as the state; it uses CreateVPC and DeleteVPC as the transitions; it will further add dependent Subnet state machines. Specifically, CloudEmu traverses the resource index from the top-level resource (e.g., VPC encloses all other EC2 resources). For each node it visits, CloudEmu compiles its description into a concrete state machine implementation. Hence, compilation proceeds incrementally and modularly for each resource, while temporarily shelving concerns for dependent resources by leaving comments in the code as “compilation hints.” For instance, when compiling for the VPC, the resource index will pinpoint the Subnet as a dependent resource; CloudEmu leaves a hint that DeleteVPC must ensure that Subnet resources are empty. After finishing the traversal, these incrementally generated “modules” are spliced together to form a complete, cloud-wide emulator. Neurosymbolic compilation. For each node, CloudEmu enumerates attributes in its attribute map and switches between neural and symbolic synthesis based on attribute types. State. We first compile from the index node a set of state variables that are needed to emulate that resource. Our state inference algorithm extracts explicit and implicit state variables using symbolic and neural methods, respectively. Explicit state is the union set of variables in a resource that are visible to other resources or DevOps programs, e.g., VpcId and CidrBlocks, thus their corresponding state must be maintained in the emulation. Likewise, Subnet resources depend on their parent VPC, and such dependencies need to be maintained. These externally-visible IO parameters are captured by our compiler symbolically, resulting in a set of state variables for that SM. Further, the implicit state is inferred from the textual description and is typically used to maintain runtime state. Transitions. A resource’s API descriptions are compiled into a set of state machine transitions. The function signatures and response formats for transition APIs are symbolically defined in the index node. For instance, the AWS:EC2:VPC:CreateVPC:IO is typed as an “API interface,” which takes CidrBlock, a string type, as input; likewise, its return values follow a fully specified schema: a VPC object containing VpcId and CidrBlock, serialized in XML. The transition functionality, on the other hand, is described in prose—the AWS:EC2:VPC:CreateVPC:Desc attribute is a text string extracted from cloud documentation. CloudEmu applies a “check-and-fix” loop to compile this textual description into code. At each iteration, CloudEmu invokes neural models to attempt a translation, and then applies symbolic assertions to check for several invariants. For instance, the function body must correctly uphold the IO interface, following the expected data formats; dependent resources (e.g., Subnet) must be empty before their parent resource (e.g., VPC) is deleted; as well as cloud conventions for consistent variable and API naming. The loop finishes when the generated function body upholds all invariants. 3.3. Automated Alignment Despite layers of symbolic constraints, an emulator synthesized from documentation alone can still diverge from the real cloud, because natural language descriptions can be ambiguous or incomplete. For instance, the CIDR overlap constraint described in CreateSubnet is a case in point: the documentation states that the CIDR block must not conflict with other Subnets in the same VPC, but does not specify the exact validation rule. An LLM implementing this constraint may get it partially right, accepting inputs that the real cloud rejects. Likewise, default values from an API response may be missing from documentation, only visible when observing cloud execution. This motivates the third stage—treating the real cloud as an oracle, running test cases against both the emulator and the cloud, and using the behavioral differences to drive repair. This can be framed as a automated program repair problem given a pair of execution traces on the same input program: the cloud’s responses are the ground truth, the emulator is the program under repair, and alignment is the process of iteratively finding differences between the observable behavior of the emulator and the behavior of the oracle, then patching the emulator until its behavior matches the oracle’s. Generating test cases. The complexity in generating test cases lies in the many APIs per resource and the many parameters that can be passed to each API, compounded by the fact that APIs can interleave with each other. For instance, the state space for N=100N=100 APIs with p=5p=5 distinct parameter configurations, and API traces of length k=3k=3, is on the order of (N⋅p)k=125(N· p)^k=125 million. Although state space explosion is a classic problem in program analysis, the cloud further poses additional scalability challenges because API latencies are much higher and more costly than local programs. Hence, the test case selection needs to be wary of limited time and monetary budgets; an exhaustive enumeration does not scale. We use a neurosymbolic approach to test case generation that combines symbolic execution and LLM-based synthesis. Our primary target is the API transition body, which is generated by the LLM. We further isolate each API and perform symbolic execution on its implementation as the basic unit, resulting in a set of path constraints that would exercise each branch in the function. We use these path constraints as coverage targets for test generation alongside the total state touched (based on resource attributes read or assigned). We then invoke an LLM to pick which APIs to test and in which order, while asking the LLM to prioritize realistic invocation orders. Specifically, we ask the LLM to generate CLI or Terraform programs that would trigger the underlying APIs, instead of generating raw, API-level traces, because these DevOps programs reflect real-world API usage. Localizing root-cause from noisy traces. For each test case, we execute the program in both the emulator and the cloud, record and analyze the traces. The goal is to automatically detect where the emulator diverges from the cloud. The challenge arises from noisy traces, especially from the cloud. Many fields, such as server-generated IDs and timestamps, differ across runs and lead to discrepancies without a semantic difference. In addition, cloud APIs have latencies that vary depending on the API, and this results in reordered operations in the cloud side. CloudEmu aligns the two traces by matching the same API call on the same logical resource, such as matching both runs of CreateVpc for the same VPC name. We align the two noisy traces to identify corresponding operations, then localize the earliest root-cause discrepancy while suppressing downstream noise, and finally map the misbehaving API to the exact generated code region for repair. Patching. After localization, the remaining task is to repair the emulator so that its behavior matches the real cloud. At a high level, we classify the surviving API response discrepancies into four categories: MISSING_FIELD, EXTRA_FIELD, VALUE_MISMATCH, and STATUS_MISMATCH. We first perform symbolic passes that attempt to resolve discrepancies whose fixes are directly observable from the reference trace. The most important of these is response seeding: when a field’s correct value is visible in the reference trace. For example, if the reference trace shows that CreateSubnet always returns mapPublicIpOnLaunch: false as a default but the emulator omits the field entirely, response seeding injects the assignment symbolically. A second symbolic pass repairs formatting mismatches, such as when the emulator returns a scalar where the reference returns a list. For example, a CreateSubnet response returning cidrBlockAssociationSet as a plain string rather than a single-element list is a common instance of this kind. Together, these two passes resolve a fraction of discrepancies mechanically and run in seconds, making them worth applying before any model invocation. For discrepancies that symbolic patching cannot resolve, we extract the full method source alongside the structured discrepancy list and use LLM-based repair. This returns a replacement written at the exact line range identified by the AST index. The patcher re-runs the representative case after each attempt and re-localizes surviving findings. If the discrepancy set shrinks, the retry budget extends, rewarding progress; if the patcher makes no progress across retries, we explore patches that cross APIs, such as a DescribeSubnets response that must join state from both the subnet store and the VPC store to populate nested association fields, or a DeleteVpc that requires coordinated cleanup across multiple dependent resource stores. All patches are applied to a separate copy of the generated code, and the patched emulator is re-evaluated in subsequent alignment runs to close the loop. 4. Evaluation Next, we evaluate CloudEmu to answer the following research questions: RQ1: How effective is CloudEmu in emulating the cloud, compared to the leading emulator, LocalStack? RQ2: How effective are the three key designs in improving CloudEmu performance? RQ3: What is the overhead of synthesis and alignment? RQ4: How well does CloudEmu generalize across clouds? 4.1. Methodology Implementation. We implemented CloudEmu in 12.4k lines of code in Python: 1.2k for documentation wrangling, 7k for scaffolding and code synthesis, and 4k for automated alignment. Our main evaluation target is AWS’s EC2, one of the most complex and widely-used services; to demonstrate generalizability, we report results on emulating GCP’s Compute Engine. The synthesized emulator (88k and 65k lines of code for EC2 and GCE, resp.) runs as a local instance, behind a Flask server gateway. DevOps programs, whether imperative CLI scripts or declarative Terraform configurations, can issue API calls into this gateway for emulated execution. Metrics. The key metrics to measure an emulator are coverage and fidelity. Coverage is measured by the number of emulated APIs divided by the total number of APIs in a cloud. Fidelity measures the similarity between an emulator’s response to a test case (one or more API invocations) and the response from the actual cloud. We do not require identical responses, as simulated timestamps, resource IDs, and other fields, do not affect functionality and can be different across actual cloud executions as well. We measure fidelity using three levels. • L0 (Error Class): the emulator and the cloud respond with identical error codes for all APIs involved in a test case, but we do not examine other return values. • L1 (Response Fidelity): Passes L0 (ErrorClass) and also, key response fields (e.g., CidrBlocks) are identical. • L2 (State Consistency): Passes L1 (IO behaviors) and also, the emulated state is equivalent to that of the cloud. We measure fidelity using a suite of test cases (e.g., CLI commands or Terraform configurations). Each test case produces an API trace in the emulator and the cloud, and we consider the emulator to pass a fidelity level (L0-L2) if and only if all API responses and intermediate states pass that level. 4.2. Emulation Effectiveness: Versus AWS Figure 4. LocalStack and CloudEmu scored against real AWS traces (50 CLI + 50 Terraform cases). Figure 5. CLI ablation results (267 cases). Figure 6. Terraform ablation results (63 cases). We start by measuring emulation effectiveness of CloudEmu against the leading baseline, LocalStack, using AWS as the ground truth. For the entire EC2, CloudEmu achieves 100% API coverage across these randomly chosen cases, whereas LocalStack achieves only around 70% case coverage. We further choose the remaining cases (with an additional 20 randomly chosen cases) to align and generate CloudEmu (such that the test set is not used for alignment). For fidelity, as we can see from Figure 4, CloudEmu consistently outperforms LocalStack at all fidelity levels. Generally, Terraform cases are harder to emulate accurately due to the general depth of the number of APIs per case, with state dependencies. Despite this, CloudEmu achieves higher scores than LocalStack for both Response fidelity and State consistency. As expected, Error Class parity is the easiest dimension: both systems match AWS error behavior on CLI cases, but the gap widens on Terraform, where LocalStack accepts invalid configurations and parameters and allows Terraform programs with dependency violations (e.g., deleting a Subnet that still has resources attached). Similarly, for the other two task categories, a primary reason Localstack fails is that it truncates responses with stripped down fields compared to AWS. As an example, on the RevokeSecurityGroupEgress API, LocalStack returns only "return": "true", whereas AWS (and CloudEmu) return the full revokedSecurityGroupRuleSet containing the revoked protocol, port range, and CIDR. CloudEmu is able to leverage alignment to generalize and remove certain classes of errors, which in turn gives gains over a fresh set of test cases as well. Table 2 and 3 cover examples of the kinds of errors and their proportion under CloudEmu. A majority of the errors were due to missing fields in the response (i.e., the backend logic on adding blocks based on input parameters, like IPv6 state, is flawed in certain places, or at times, nested fields aren’t serialized appropriately). 4.3. Ablation Study: Versus LocalStack Table 2. Examples of behavioral corrections introduced by LocalStack-guided alignment in CloudEmu-LS and by AWS-guided alignment in CloudEmu. Mismatch class Example in CloudEmu-LS Alignment Example in CloudEmu Alignment MISSING_FIELD CreateLaunchTemplate: added missing nested response fields that were expected by the LocalStack execution trace. CreateVpc: added missing fields such as CidrBlockAssociationSet, IPv6 state, and owner ID to match AWS responses. STATUS_MISMATCH CreateDhcpOptions: corrected whether the call should return success or an error. DescribeSecurityGroups: corrected the call to return success, matching AWS behavior for a newly created security group. VALUE_MISMATCH CreateSecurityGroup: corrected response values echoed back to the caller to match LocalStack behavior. DescribeKeyPairs: corrected the returned KeyFingerprint to match the fingerprint computed by AWS. EXTRA_FIELD CreateSubnet: removed or reshaped extra response fields so that the returned object matched LocalStack more closely. DescribeInstances: removed internal state fields that appeared in the emulator output but are not present in AWS responses. Table 3. Distribution of patch attempts by mismatch type in the LocalStack- and AWS-guided alignment stages. Mismatch type CloudEmu-LS CloudEmu MISSING_FIELD 16.3% 35.0% STATUS_MISMATCH 13.5% 30.0% VALUE_MISMATCH 32.0% 22.5% EXTRA_FIELD 38.2% 12.5% Next, we evaluate CloudEmu using a larger set of test cases (330 in total, including 267 CLI scripts and 63 Terraform configurations). To save cloud cost (and time to cleanup), we perform this ablation study using LocalStack instead of the actual AWS as the alignment target to evaluate the performance improvements from each technique. We consider several variants of CloudEmu: • Direct to code (D2C): Directly prompting LLMs to read cloud documentation and write the emulator. • Index: Uses our cloud resource index, while leaving resource-level code generation to LLMs. • SM: Further adds symbolic generation based on the state machine abstraction, but without alignment. • Full: All techniques, further aligns the emulator against the oracle, which in this subsection is LocalStack. And we denote this system variant “CloudEmu-LS”. Coverage: Naïve baseline (D2C) reaches a coverage of 66%, failing to cover all APIs. This is because comprehensive coverage fundamentally requires a precise resource index: Even though LLMs can read efficiently, token-based text parsing is not the best way to capture symbolic API definitions, and could lead to missed APIs. The Index, SM and Full variants on the hand achieve perfect coverage. Accuracy: We then measure the improvement in emulation accuracy at each step, and show the results in Figure 6 and 6. Index and SM represent the biggest improvements across both modality of cases, because they address the two key problems with LLM generation: long context and arbitrary errors. Adding the resource index improves CLI Error Class accuracy by 11% and response fidelity by around 15.3%, because it gives the LLM fine-grained, per-resource context rather than asking it to ingest the full documentation at once; the generated handlers are longer, more complete, and correctly populate more response fields. With SM, we found that the biggest accuracy boost comes for Terraform cases with a 15% gain in Error Class accuracy and a big 80% gain in the Response Fidelity. The state-machine scaffold enforces correct API signatures and response schemas before any code is generated, thus avoiding LLM-generated TF handlers returning malformed XML. Moreover, a lot of the dependency violations existing in Localstack programs get prevented since SM naturally follows a dependency graph. For CLI calls, the change is negligible since a lot of the CLI operations aren’t passing any checkable parameters; for the ones passing these parameters most of the problems are because of field initializations and internal logic, not parsing or response creation. Initially, 29.9% APIs for VPC had signature mismatches, and 41.8% APIs had response format errors; after the SM is applied, both reduce significantly, with signature mismatches dropping to 9% and response formats getting completely fixed (with an exception of extra fields for some cases). The alignment phase resulted in a small improvement for error class accuracy and Terraform cases in general, meaning that the documentation is already quite useful to bootstrap the emulator. For the CLI cases, response and state fidelity improved significantly through alignment, because a majority of those errors were due to incomplete state initialization (e.g., default values such as list of existing images at VM startup). Such information is barely covered in the documentation, thereby needs to be properly aligned via an oracle. This shows that alignment is a vital step, as mismatches can occur in purely documentation-based generation. We show two examples where alignment is helpful: i) missed/incomplete validation checks in CreateSubnet, which for reference do not check whether the Subnet being created lies in a different Availability Zone as its parent VPC, since the documentation doesn’t state this directly; and i) incomplete state initialization for fields in the resource state, like for RunInstance with a amiid passed in, real cloud queries through the predefined templates to validate which the emulator has incomplete knowledge about. Similarly a bunch of optional fields like (CarrierIp, NetworkBorderGroup, etc) aren’t initialized correctly for AllocateAddress. Examples of more such errors and their proportion are covered in Table 2 and 3, respectively, under CloudEmu-LS, with the major proportion being extra fields in responses. 4.4. Overhead Analysis Table 4. Normalized cost of CloudEmu code synthesis and automated alignment on AWS EC2. Stage Time (s) Tokens (k) Synthesis 52 / resource 478 / resource CloudEmu-LS Alignment 21 / case 3.4 / case CloudEmu Alignment 126 / case 4.1 / case Next, we measure the overhead of CloudEmu. The resource index takes around 10s to build for EC2, and incurs the least amount of overhead across the three steps. This is because building the index is a deterministic step using symbolic parsers, and although the documentation is long, the processing is efficient, and the parser can extract the pages quickly. As for the SM method, the symbolic framework is written in around 7k lines of code, only a fraction of the entire emulator code. Everything else is generated by the LLM while being in the constraints of this framework. This is a massive amount of workload reduction compared to manually reading and implementing all cloud emulation logic by hand. The entire emulators for AWS and GCP consisted of 88k and 65k lines of code, spanning 90 and 91 resource types, respectively. Per resource, approximately 16 LLM calls are needed to generate the code; smaller resources with fewer APIs took around 10 calls, and larger resources (e.g., instance.py: 3,724 lines) took around 22 calls. We have tried this task using three different LLMs: GPT 5.1, GPT 5.2-mini, and GPT 5.2-codex. We found that weaker or newer models do not result in a visible accuracy difference, because our symbolic method can boost the effectiveness of this generation, so that even smaller models suffice for this task. Though older, smaller models like GPT 4o suffer from limited expressibility and don’t generate entire API logic at once correctly. Overall, as shown in Table 4, the synthesis takes roughly 1.3 hours. Next, we measure the alignment overhead. This is dependent on the alignment target. In Table 4, we show the result on aligning against LocalStack versus the real AWS cloud. As expected, per program the amount of time it takes to align against the oracle LocalStack is 6×6× faster (21 s/case vs. 126 s/case) because it runs locally on device. For reference, in the 100 cases run in 4.2, we found 24 APIs with discrepancies across 48 alignment cases, yielding 41 patch operations. In contrast, for AWS, each alignment run takes longer, as CloudEmu needs to rerun against the cloud, obtain traces, and cleanup for the next run. Figure 7. CDFs of patch size (left) and latency (right) for GPT patching attempts; verified OpenHands escalations are included in the latency panel only. Additionally, alignment cost depends not only on the target oracle but also on the discrepancy types uncovered, shown in Table 3. Different classes of errors demand different repair effort, which directly affects patching overhead. We therefore next break down overhead by error type. Against LocalStack, EXTRA_FIELD and VALUE_MISMATCH dominate, reflecting structural over-generation that LocalStack can detect because it returns those fields itself, whereas against AWS, MISSING_FIELD and STATUS_MISMATCH become dominant, exposing semantic gaps that LocalStack cannot surface because the latter shares the same omissions. Figure 7 shows the CDFs of patching effort for all GPT and coding agent (OpenHands/OH (Wang et al., 2025)) attempts. Half of all patches touch fewer than 84 lines and complete in under 30 seconds, but the distributions have long tails: the top 10% exceed 246 lines and 180 seconds, reflecting cases where correcting existing values or removing fields requires touching interconnected logic. These stalling cases are further escalated to a full coding agent (OH). Beyond the per-discrepancy patching cost, overall alignment efficiency also depends on the strategy used to discover discrepancies. Targeted exploration can uncover alignment gaps with fewer test cases and thus lower total cost. We measure the effectiveness of coverage-guided alignment, targeting emulator branches left uncovered by the benchmark suite, to find alignment gaps more efficiently. We generate 50 CLI and Terraform coverage-guided cases each via a greedy set-cover over static branch targets extracted from the pre-aligned emulator source code. These are synthesized by GPT 5.2-codex, picking branches with low confidence scores (assigned by LLM during code generation) to prioritize cases where the generation was unclear. Comparing this with the strategy of scraping random cases, Figure 8 shows cumulative unique APIs exercised and discrepancies found as the number of cases executed increases (averaged over 200 random orderings for the scraped cases). The coverage-guided strategy helps us uncover much more discrepancies per case than random scraped cases. At a budget of 50 cases each, coverage-guided cases exercise 79 unique EC2 API methods compared to scraped CLI cases(88% more), including 37 APIs absent from the scraped suite entirely. The discrepancy yield is also higher: for Terraform, coverage-guided cases surface 66% discrepant cases compared to 27% for scraped, a 2.5×2.5× improvement. The two strategies expose complementary fault classes: scraped cases find more field-value mismatches (2.2×2.2×) on well-exercised APIs, while coverage-guided cases expose 2.1×2.1× more Error Class mismatches on parameter paths the scraped suite never reaches, surfacing 32 distinct AWS error codes versus 19 for scraped, with 22 unique to the coverage-guided suite. Figure 8. Cumulative unique APIs exercised and discrepancies found for scraped vs. coverage-guided cases 4.5. Translating CloudEmu to GCP CloudEmu transfers to GCP Compute with minimal provider-specific adaptation. The synthesis, patching, and alignment loop remains unchanged; only the template layer must encode provider-specific interface conventions. Adapting the pipeline to GCP. The differences between AWS EC2 and GCP Compute are architectural rather than algorithmic. GCP resources are identified by string names rather than prefixed IDs, all mutating calls return Operation objects rather than direct resource responses, labels replace tags, and error responses use JSON rather than XML. We encoded these differences as GCP-specific templates in the symbolic generator. The OpenHands patching and alignment stages remain unchanged. Table 5. Accuracy on Terraform cases over GCP Compute resource cases CloudEmu-GCP Error-class Resp. Fidelity State Consist. before alignment 0.644 0.571 0.545 after alignment 0.956 0.857 0.734 Results. We evaluate 45 Terraform cases (spanning different numbers of resource blocks and dependencies), across 18 GCP Compute resource types, scraped from official registries and repos. Similar to AWS, we evaluate these cases on three metrics against real GCP. As shown in Table 5, the CloudEmu-GCP after alignment achieves high scores across all metrics. An example of error-class miss is a disk resource gated on a Windows OS license not active on our billing account; GCP returns 403 Permission_Denied while the emulator succeeds; this is orthogonal to resource wiring. Two general classes accounted for the failures at all levels. i) Incorrect resource identifiers: regional backends stored the region as a full resource URL (https://…/regions/us-central1) but GET lookups on the short name, and selfLink fields used the wrong collection name (RegionBackendServices instead of backendServices), causing Terraform’s read to fail. i) Incomplete target validation: forwarding rule insertion rejects some valid VPN configurations as service_attachment and vpn_gateway were not included as valid target types. The one remaining failure, google_compute_network_peering, requires bidirectional state updates not modeled by the per-resource scaffold. These error types get fixed by alignment, and the emulator gains roughly 10-20% on the different metrics. Takeaway. The pipeline generalizes to GCP with only template-layer changes and configurations; synthesis, patching, and alignment stages transfer unchanged. The high scores across the three metrics confirm that the core insight of the paper holds across providers: typed scaffolding and centralized state are sufficient to capture cross-resource wiring, and the alignment loop reliably closes the gap between generated structure and provider-specific behavior. 5. Discussion We discuss the limitations in the design of CloudEmu as well as potential future use cases for this direction. Limitations in performance and failure emulation. Currently, emulators only respond to API invocations but do not simulate the performance characteristics like invocation latency, rate limits, & network latencies. There are applications that may require this level of simulation, e.g., for performance testing or chaos engineering. For these use cases, LocalStack and CloudEmu need to be extended to consider performance or failure profiles in addition to API fidelity. Likewise, simulation of semantic properties, such as concurrency tests or consistency properties, are not handled by existing emulators. Limitations of third-party implementation. Thus far, both LocalStack and CloudEmu take a third-party approach to implementing cloud emulators. This is because cloud providers themselves prioritize building actual cloud features to stay competitive in the market, rather than emulating them. Hence, emulators are primarily developed by third-party developers based on the cloud’s documentation. However, if the cloud providers were to implement emulators themselves, they could take two approaches. One approach is to reuse their existing codebase for the actual cloud service, and then derive a lightweight emulator from the implementation. However, actual implementations are coupled with hardware details and are distributed systems, which may necessitate a high amount of reengineering work to create a local mock version. The other approach they could take is to rely on documentation, whether public API documentation or private design documentation. CloudEmu could help with this latter approach with automated synthesis. Cloud gym: This emulation framework can also act as a playground for learning and testing cloud services for AI agents. There has been a recent line of work on building AI agents for cloud management (Yang et al., 2025a; Chen et al., 2024; Jha et al., 2025; Yang et al., 2025b; Yang et al., 2026), with the goal of eventually automating DevOps engineering. To train such agents, we need a high-fidelity cloud gym, for example, for reinforcement learning, that provides a no-cost, zero-risk environment for generating feedback and learning data. Such a gym could also be used to validate agent behavior before deployment on the real cloud. Documentation engineering: By analyzing the specifications, we can detect potential design flaws and anti-patterns. For instance, a modify() call that requires a long and complex chain of actions updating multiple dependencies across resources may indicate a poorly designed API; or, documentation that consistently leads the AI to generate incorrect logic may be flagged as ambiguous and in need of refinement. This will improve API and documentation engineering (3). The synthesized emulator can also be used as an executable specification than textual documentation. Multi-cloud emulation: Our approach is provider-agnostic and can generalize to any cloud backend. By consuming different cloud providers’ documentation, we can generate a standardized formal model for all of them, and generate a “universal emulator” for testing multi-cloud DevOps programs. Our approach also enables formal, automated comparisons of equivalent services—e.g., whether Azure’s CreateVM() requires the same dependency checks as AWS’s RunInstance() in AWS—and can help improve cross-cloud portability. Quantifying cloud complexity: The resource index comprises a graph of interacting state machines. This provides objective metrics (e.g., number of nodes, edge density) for a quantitative analysis of cloud service complexity. This allows for comparisons, for example, between the complexity of AWS Lambda and Azure Functions, and could assist cloud providers to modularize their resources. Direct generation using LLMs As the frontier LLMs evolve, there have been increasing attempts to build alternatives to the existing emulators ((33)) by one-shot synthesis using coding harnesses. These implementations (e.g., Floci (17)) suffer from severe fidelity issues, rendering them impractical for emulation and requiring lot of manual effort to improve fidelity. 6. Related Work Specification mining. Our work builds on a rich history of techniques for automatically extracting API specifications (Zhong et al., 2009; Nguyen et al., 2014; Motwani and Brun, 2019; Leong and Barbosa, 2021; Liu et al., 2021), especially those that leveraged LLMs to directly translate informal natural language comments and documents into checkable assertions (Endres et al., 2024; Zhai et al., 2020), temporal properties (Liu et al., 2022; Cosler et al., 2023), and other formal specifications (Mandal et al., 2023; Jain et al., 2022; Ma et al., 2024). While we draw heavily from these advances, most of them stop at inferring invariants for existing implementations, rather than generating end-to-end emulation code. Within the domain of spec mining, our work is most related to active automata learning, or “model learning” (Vaandrager, 2017; Tran Van et al., 2024), where an algorithm interactively queries a black-box system to infer a state machine model of its behavior using traces or documentation. The most related work, Hermes (Al Ishtiaq et al., 2024) synthesizes FSMs from network protocol documents to enable security analysis, while other works have focused on automated attack synthesis (Pacheco et al., 2022), automated testing (Li et al., 2025), or detecting bugs in protocol implementations (Fiterau-Brostean et al., 2023; Fiterău-Broştean et al., 2024). To the best of our knowledge, our system is the first to use LLMs to translate API documentation into a complete, executable state machine model, specifically to emulate complex cloud services. Cloud testing. Most cloud testing works focus on finding bugs and vulnerabilities in the cloud service implementation itself, using stateful fuzzing (Atlidakis et al., 2019; Kim et al., 2022; Atlidakis et al., 2020; Wu et al., 2022; Godefroid et al., 2020a; Godefroid et al., 2017) (more so with LLMs (Zheng et al., 2024; Xiang et al., 2025)) or differential testing (Godefroid et al., 2020b). The closest efforts to ours aim to find behavioral gaps between real cloud and emulators (Mazhar et al., 2025), but do not fix them in a principled manner. Our key contribution is to close the loop: we use the discrepancies to refine our learned model itself. We take inspiration from the prior work on protocol reverse engineering (Ye et al., 2021; Luo et al., 2024; Bossert et al., 2014) for efficient automated alignment. 7. Conclusion Cloud DevOps is an safety-critical task, requiring DevOps programs to be thoroughly tested and validated before deployment. Directly testing against the actual cloud incurs cost and high latency, and may introduce unwanted side effects if the tests fail (e.g., risks to existing infrastructure, the need for cleanup and redeployment). Cloud emulators have gained popularity as an alterantive, which run API-level simulations locally in a lightweight manner. However, this task requires constant, error-prone manual efforts, which struggles to catch up with the cloud API surface, which is not only complex, evolving, but also differs across providers. CloudEmu is a new approach, using a neurosymbolic approach to learning the emulation logic from cloud documentation and automatically aligning the emulator to the cloud behavior. We show the effectiveness of CloudEmu against existing emulators, and its generalizability across two major clouds. References Al Ishtiaq et al. (2024) A. Al Ishtiaq, S. S. S. Das, S. M. M. Rashid, A. Ranjbar, K. Tu, T. Wu, Z. Song, W. Wang, M. Akon, R. Zhang, et al. Hermes: unlocking security analysis of cellular network protocols by synthesizing finite state machines from natural language specifications. In 33rd USENIX Security Symposium (USENIX Security 24), p. 4445–4462. Cited by: §6. [2] () Amazon Web Services — aboutamazon.com. Note: https://w.aboutamazon.com/what-we-do/amazon-web-services[Accessed 29-06-2025] Cited by: §1. [3] () API Design Guide | Google Cloud — cloud.google.com. Note: https://cloud.google.com/apis/design[Accessed 09-07-2025] Cited by: §5. Atlidakis et al. (2019) V. Atlidakis, P. Godefroid, and M. Polishchuk Restler: stateful rest api fuzzing. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), p. 748–758. Cited by: §6. Atlidakis et al. (2020) V. Atlidakis, P. Godefroid, and M. Polishchuk Checking security properties of cloud service rest apis. In 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST), p. 387–397. Cited by: §6. [6] AWS CloudFormation. Note: https://aws.amazon.com/cloudformation/ Cited by: §1, §2. Bhatnagar et al. (2025) A. Bhatnagar, Y. Qiu, S. McClure, S. Ratnasamy, and A. Chen A case for learned cloud emulators. HotNets ’25, New York, NY, USA. External Links: ISBN 9798400722806, Link, Document Cited by: §1. Bossert et al. (2014) G. Bossert, F. Guihéry, and G. Hiet Towards automated protocol reverse engineering using semantic information. In Proceedings of the 9th ACM symposium on Information, computer and communications security, p. 51–62. Cited by: §6. [9] Change log of AWS APIs. Note: https://awsapichanges.com/ Cited by: §2.1. Chen et al. (2024) Y. Chen, M. Shetty, G. Somashekar, M. Ma, Y. Simmhan, J. Mace, C. Bansal, R. Wang, and S. Rajmohan AIOpsLab: a holistic framework to evaluate ai agents for enabling autonomous clouds. External Links: Link Cited by: §5. [11] () Cloudcontrolapi — aws.amazon.com. Note: https://aws.amazon.com/cloudcontrolapi/[Accessed 05-07-2025] Cited by: §1. Cosler et al. (2023) M. Cosler, C. Hahn, D. Mendoza, F. Schmitt, and C. Trippel Nl2spec: interactively translating unstructured natural language to temporal logics with large language models. In International Conference on Computer Aided Verification, p. 383–396. Cited by: §6. Denning (2025) P. J. Denning Abstractions. Communications of the ACM 68 (3). External Links: Link Cited by: §3.2. Endres et al. (2024) M. Endres, S. Fakhoury, S. Chakraborty, and S. K. Lahiri Can large language models transform natural language intent into formal method postconditions?. Proceedings of the ACM on Software Engineering 1 (FSE), p. 1889–1912. Cited by: §6. Fiterau-Brostean et al. (2023) P. Fiterau-Brostean, B. Jonsson, K. Sagonas, and F. Tåquist Automata-based automated detection of state machine bugs in protocol implementations.. In NDSS, Cited by: §6. Fiterău-Broştean et al. (2024) P. Fiterău-Broştean, B. Jonsson, K. Sagonas, and F. Tåquist SMBugFinder: an automated framework for testing protocol implementations for state machine bugs. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, p. 1866–1870. Cited by: §6. [17] floci-io/floci: light, fluffy, and always free: the AWS local emulator alternative. Note: https://github.com/floci-io/floci[Accessed 24-08-2026] Cited by: §5. Godefroid et al. (2020a) P. Godefroid, B. Huang, and M. Polishchuk Intelligent rest api data fuzzing. In Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, p. 725–736. Cited by: §6. Godefroid et al. (2020b) P. Godefroid, D. Lehmann, and M. Polishchuk Differential regression testing for rest apis. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, p. 312–323. Cited by: §6. Godefroid et al. (2017) P. Godefroid, H. Peleg, and R. Singh Learn&fuzz: machine learning for input fuzzing. In 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), p. 50–59. Cited by: §6. Guu et al. (2020) K. Guu, K. Lee, Z. Tung, P. Pasupat, and M. Chang Retrieval Augmented Language Model Pre-training. In Proceedings of ICML’20, p. 3929–3938. Cited by: §1. [22] (2021) How can i experiment with cloud (azure, aws, google, etc) without going broke?. Note: https://devops.stackexchange.com/questions/1002/how-can-i-experiment-with-cloud-azure-aws-google-etc-without-going-broke?noredirect=1&lq=1[Accessed 09-07-2025] Cited by: §1. Jain et al. (2022) N. Jain, S. Vaidyanath, A. Iyer, N. Natarajan, S. Parthasarathy, S. Rajamani, and R. Sharma Jigsaw: large language models meet program synthesis. In Proceedings of the 44th International Conference on Software Engineering, p. 1219–1231. Cited by: §6. Jha et al. (2025) S. Jha, R. Arora, Y. Watanabe, T. Yanagawa, Y. Chen, J. Clark, B. Bhavya, M. Verma, H. Kumar, H. Kitahara, N. Zheutlin, S. Takano, D. Pathak, F. George, X. Wu, B. O. Turkkan, G. Vanloo, M. Nidd, T. Dai, O. Chatterjee, P. Gupta, S. Samanta, P. Aggarwal, R. Lee, P. Murali, J. Ahn, D. Kar, A. Rahane, C. Fonseca, A. Paradkar, Y. Deng, P. Moogi, P. Mohapatra, N. Abe, C. Narayanaswami, T. Xu, L. R. Varshney, R. Mahindru, A. Sailer, L. Shwartz, D. Sow, N. C. M. Fuller, and R. Puri ITBench: evaluating ai agents across diverse real-world it automation tasks. External Links: 2502.05352, Link Cited by: §5. Kim et al. (2022) M. Kim, Q. Xin, S. Sinha, and A. Orso Automated test generation for rest apis: no time to rest yet. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, p. 289–301. Cited by: §6. Leong and Barbosa (2021) I. T. Leong and R. Barbosa Generation of oracles using natural language processing. In 2021 28th Asia-Pacific Software Engineering Conference Workshops (APSEC Workshops), p. 25–31. Cited by: §6. Lewis et al. (2020) P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, p. 9459–9474. Cited by: §1. Li et al. (2025) H. Li, Z. Dong, S. Wang, H. Zhang, L. Shen, X. Peng, and D. She Extracting formal specifications from documents using llms for test automation. In 2025 IEEE/ACM 33rd International Conference on Program Comprehension (ICPC), p. 1–12. Cited by: §6. Liskov and Zilles (1974) B. Liskov and S. Zilles Programming with abstract data types. In Proceedings of the ACM SIGPLAN Symposium on Very High Level Languages, New York, NY, USA, p. 50–59. External Links: ISBN 9781450378840, Link, Document Cited by: §2.2. Liskov (2009) B. Liskov The power of abstraction. Note: ACM A.M. Turing Award Lecture Cited by: §3.2. Liu et al. (2022) J. X. Liu, Z. Yang, B. Schornstein, S. Liang, I. Idrees, S. Tellex, and A. Shah Lang2ltl: translating natural language commands to temporal specification with large language models. In Workshop on Language and Robotics at CoRL 2022, Cited by: §6. Liu et al. (2021) M. Liu, X. Peng, A. Marcus, C. Treude, X. Bai, G. Lyu, J. Xie, and X. Zhang Learning-based extraction of first-order logic representations of api directives. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, p. 491–502. Cited by: §6. [33] () LocalStack for AWS — localstack.cloud. Note: https://w.localstack.cloud/localstack-for-aws[Accessed 29-06-2025] Cited by: §1, §1, Table 1, Table 1, §2, §5. Luo et al. (2024) Z. Luo, K. Liang, Y. Zhao, F. Wu, J. Yu, H. Shi, and Y. Jiang DynPRE: protocol reverse engineering via dynamic inference. In Proc. NDSS, p. 1–18. Cited by: §6. Ma et al. (2024) L. Ma, S. Liu, Y. Li, X. Xie, and L. Bu Specgen: automated generation of formal program specifications via large language models. arXiv preprint arXiv:2401.08807. Cited by: §6. Mandal et al. (2023) S. Mandal, A. Chethan, V. Janfaza, S. Mahmud, T. A. Anderson, J. Turek, J. J. Tithi, and A. Muzahid Large language models based automatic synthesis of software specifications. arXiv preprint arXiv:2304.09181. Cited by: §6. Mazhar et al. (2025) A. Mazhar, S. S. Alam, W. X. Zheng, Y. Chen, S. Nath, and T. Xu Fidelity of cloud emulators: the imitation game of testing cloud-based software. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE), p. 614–614. Cited by: §2.1, §6. [38] () Moto: Mock AWS Services; Moto 5.1.7.dev documentation — docs.getmoto.org. Note: https://docs.getmoto.org/en/latest/[Accessed 02-07-2025] Cited by: §1. Motwani and Brun (2019) M. Motwani and Y. Brun Automatically generating precise oracles from structured natural language specifications. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE), Vol. , p. 188–199. External Links: Document Cited by: §6. Nguyen et al. (2014) H. A. Nguyen, R. Dyer, T. N. Nguyen, and H. Rajan Mining preconditions of apis in large-scale code corpus. In Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering, p. 166–177. Cited by: §6. Pacheco et al. (2022) M. L. Pacheco, M. von Hippel, B. Weintraub, D. Goldwasser, and C. Nita-Rotaru Automated attack synthesis by extracting finite state machines from protocol specification documents. In 2022 IEEE Symposium on Security and Privacy (SP), p. 51–68. Cited by: §6. [42] Pulumi: infrastructure as code in any programming language. Note: https://w.pulumi.com/ Cited by: §1, §2. Qiu et al. (2024) Y. Qiu, P. T. J. Kon, R. Beckett, and A. Chen Unearthing Semantic Checks for Cloud Infrastructure-as-Code Programs. In Proceedings of SOSP’24, p. 574–589. Cited by: §2. [44] Terraform by Hashicorp. Note: https://w.terraform.io/ Cited by: §1, §2. Tran Van et al. (2024) A. Tran Van, O. Levillain, and H. Debar Mealy verifier: an automated, exhaustive, and explainable methodology for analyzing state machines in protocol implementations. In Proceedings of the 19th International Conference on Availability, Reliability and Security, p. 1–10. Cited by: §6. [46] () Use Azurite emulator for local Azure Storage development — learn.microsoft.com. Note: https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=visual-studio%2Cblob-storage[Accessed 02-07-2025] Cited by: §1. Vaandrager (2017) F. Vaandrager Model learning. Communications of the ACM 60 (2), p. 86–95. Cited by: §6. Wang et al. (2025) X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y. Shao, N. Muennighoff, Y. Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig OpenHands: an open platform for ai software developers as generalist agents. External Links: 2407.16741, Link Cited by: §4.4. [49] () What is Terraform | Terraform | HashiCorp Developer — developer.hashicorp.com. Note: https://developer.hashicorp.com/terraform/intro[Accessed 05-07-2025] Cited by: §1. [50] () What’s New at AWS – Cloud Innovation & News — aws.amazon.com. Note: https://aws.amazon.com/new/[Accessed 05-07-2025] Cited by: §1. Wu et al. (2022) H. Wu, L. Xu, X. Niu, and C. Nie Combinatorial testing of restful apis. In Proceedings of the 44th International Conference on Software Engineering, p. 426–437. Cited by: §6. Xiang et al. (2025) Y. Xiang, Z. Yang, J. Peng, H. Bauer, P. T. J. Kon, Y. Qiu, and A. Chen Automated Bug Discovery in Cloud Infrastructure-as-Code Updates with LLM Agents. In Proceedings of AIOps, Cited by: §6. Yang et al. (2025a) Z. Yang, A. Bhatnagar, Y. Qiu, T. Miao, P. Tser Jern Kon, Y. Xiao, Y. Huang, M. Casado, and A. Chen Cloud Infrastructure Management in the Age of AI Agents. SIGOPS Operating Systems Review (OSR) 59 (1), p. 1–8. External Links: Link Cited by: §5. Yang et al. (2026) Z. Yang, K. Gruizenga, T. Miao, P. T. J. Kon, H. Guan, and A. Chen Ambig-iac: multi-level disambiguation for interactive cloud infrastructure-as-code synthesis. External Links: 2604.02382, Link Cited by: §5. Yang et al. (2025b) Z. Yang, H. Guan, V. Nicolet, B. Paulsen, J. Dodds, D. Kroening, and A. Chen Automated cloud infrastructure-as-code reconciliation with ai agents. External Links: 2510.20211, Link Cited by: §5. Yang et al. (2023) Z. Yang, Z. Wu, M. Luo, W. Chiang, R. Bhardwaj, W. Kwon, S. Zhuang, F. S. Luan, G. Mittal, S. Shenker, and I. Stoica SkyPilot: an intercloud broker for sky computing. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), Boston, MA, p. 437–455. External Links: ISBN 978-1-939133-33-5, Link Cited by: §1. Ye et al. (2021) Y. Ye, Z. Zhang, F. Wang, X. Zhang, and D. Xu NetPlier: probabilistic network protocol reverse engineering from message traces.. In NDSS, Cited by: §6. Zhai et al. (2020) J. Zhai, Y. Shi, M. Pan, G. Zhou, Y. Liu, C. Fang, S. Ma, L. Tan, and X. Zhang C2S: translating natural language comments to formal program specifications. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2020, New York, NY, USA, p. 25–37. External Links: ISBN 9781450370431, Link, Document Cited by: §6. Zheng et al. (2024) T. Zheng, J. Shao, J. Dai, S. Jiang, X. Chen, and C. Shen RESTLess: enhancing state-of-the-art rest api fuzzing with llms in cloud service computing. IEEE Transactions on Services Computing. Cited by: §6. Zhong et al. (2009) H. Zhong, L. Zhang, T. Xie, and H. Mei Inferring resource specifications from natural language api documentation. In 2009 IEEE/ACM International Conference on Automated Software Engineering, Vol. , p. 307–318. External Links: Document Cited by: §6.

Instant research discovery