Paper deep dive

LLM-Rosetta: A Hub-and-Spoke Intermediate Representation for Cross-Provider LLM API Translation

Peng Ding

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 50

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 98%

Last extracted: 4/14/2026, 1:52:51 AM

Summary

LLM-Rosetta is an open-source translation framework that uses a hub-and-spoke Intermediate Representation (IR) to enable cross-provider LLM API translation. By decoupling provider-specific API formats from a shared semantic core, it reduces the complexity of building multi-provider architectures from O(N^2) to O(N) and supports bidirectional conversion, streaming, and lossless round-trip fidelity.

Entities (6)

Anthropic Messages · api-standard · 100%Argonne National Laboratory · organization · 100%Google GenAI · api-standard · 100%LiteLLM · software-framework · 100%LLM-Rosetta · software-framework · 100%OpenAI Chat Completions · api-standard · 100%

Relation Signals (4)

LLM-Rosetta → deployedat → Argonne National Laboratory

confidence 100% · deployed in production at Argonne National Laboratory

LLM-Rosetta → supports → OpenAI Chat Completions

confidence 100% · We implement converters for four API standards (OpenAI Chat Completions...)

LLM-Rosetta → supports → Anthropic Messages

confidence 100% · We implement converters for four API standards (...Anthropic Messages...)

LLM-Rosetta → supports → Google GenAI

confidence 100% · We implement converters for four API standards (...and Google GenAI)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The rapid proliferation of Large Language Model (LLM) providers--each exposing proprietary API formats--has created a fragmented ecosystem where applications become tightly coupled to individual vendors. Switching or bridging providers requires $O(N^2)$ bilateral adapters, impeding portability and multi-provider architectures. We observe that despite substantial syntactic divergence, the major LLM APIs share a common semantic core: the practical challenge is the combinatorial surface of syntactic variations, not deep semantic incompatibility. Based on this finding, we present LLM-Rosetta, an open-source translation framework built on a hub-and-spoke Intermediate Representation (IR) that captures the shared semantic core--messages, content parts, tool calls, reasoning traces, and generation controls--in a 9-type content model and 10-type stream event schema. A modular Ops-composition converter architecture enables each API standard to be added independently. LLM-Rosetta supports bidirectional conversion (provider-to-IR-to-provider) for both request and response payloads, including chunk-level streaming with stateful context management. We implement converters for four API standards (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google GenAI), covering the vast majority of commercial providers. Empirical evaluation demonstrates lossless round-trip fidelity, correct streaming behavior, and sub-100 microsecond conversion overhead--competitive with LiteLLM's single-pass approach while providing bidirectionality and provider neutrality. LLM-Rosetta passes the Open Responses compliance suite and is deployed in production at Argonne National Laboratory. Code is available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

49,896 characters extracted from source content.

Expand or collapse full text

LLM-ROSETTA: A HUB-AND-SPOKE INTERMEDIATE REPRESENTATION FOR CROSS-PROVIDER LLM API TRANSLATION A PREPRINT Peng Ding University of Chicago dingpeng@uchicago.edu April 13, 2026 ABSTRACT The rapid proliferation of Large Language Model (LLM) providers—each exposing proprietary API formats—has created a fragmented ecosystem where applications become tightly coupled to individual vendors. Switching or bridging providers requiresO(N 2 )bilateral adapters, impeding portability and multi-provider architectures. We observe that despite substantial syntactic divergence, the major LLM APIs share a common semantic core: the practical challenge is the combinato- rial surface of syntactic variations, not deep semantic incompatibility. Based on this finding, we presentLLM-Rosetta, an open-source translation framework built on a hub-and-spoke Interme- diate Representation (IR) that captures the shared semantic core—messages, content parts, tool calls, reasoning traces, and generation controls—in a 9-type content model and 10-type stream event schema. A modular Ops-composition converter architecture enables each API standard to be added independently.LLM-Rosettasupports bidirectional conversion (provider↔IR ↔provider) for both request and response payloads, including chunk-level streaming with stateful context man- agement. We implement converters for four API standards (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google GenAI), covering the vast majority of commercial providers. Empirical evaluation demonstrates lossless round-trip fidelity, correct streaming behav- ior, and sub-100 μs conversion overhead—competitive with LiteLLM’s single-pass approach while providing bidirectionality and provider neutrality.LLM-Rosettapasses the Open Responses com- pliance suite and is deployed in production at Argonne National Laboratory. Code is available at https://github.com/Oaklight/llm-rosetta. Keywords LLM· API Translation· Intermediate Representation· Interoperability· Streaming· Multi-Provider 1 Introduction Large Language Models (LLMs) [Achiam et al., 2023, Gemini Team et al., 2023] are increasingly accessed through cloud-hosted APIs, yet the ecosystem lacks a universal wire format. OpenAI’s Chat Completions [OpenAI, 2024], Anthropic’s Messages [Anthropic, 2024a], Google’s Generative AI [Google, 2024], and the newer OpenAI Responses API [OpenAI, 2025] each define their own schemas for messages, tool calls, streaming, and generation controls. This divergence forces application developers to write provider-specific glue code, and organizations evaluating multiple models face a combinatorial integration burden. TheO(N 2 )problem. GivenNproviders, naïve pairwise translation requires N 2 bilateral adapters. Each adapter must handle request construction, response parsing, streaming event mapping, and tool-call serialization—all of which drift as providers evolve their APIs. In practice, most projects either lock into a single vendor or adopt heavyweight SDK wrappers that hide—but do not solve—the underlying format mismatch. arXiv:2604.09360v1 [cs.SE] 10 Apr 2026 LLM-RosettaA PREPRINT Hub-and-spoke asO(N)solution. The hub-and-spoke pattern—routing all conversions through a single Interme- diate Representation (IR)—is well established in compiler infrastructure [Lattner and Adve, 2004] and data inter- change [Apache Software Foundation, 2016], where it reducesM × Nadapters toM + N. LLM API translation is a structurally simpler domain (dict-to-dict mapping rather than semantic-preserving program transformation), but the same combinatorial argument applies: withNproviders and multiple feature dimensions, pairwise adapters grow quadratically. We apply the hub-and-spoke principle to this domain. Contributions. We present LLM-Rosetta, an open-source framework that introduces: 1.An empirical characterization of API divergence: despite substantial syntactic differences, the four major LLM providers share a common semantic core that can be captured by a 9-type content model and 10-type stream event schema. The practical difficulty lies not in deep semantic gaps but in the combinatorial surface of syntactic variations across providers and feature dimensions (section 3.2). 2.A typed Intermediate Representation (IR) and Ops-composition converter architecture that exploit this observation: since divergence is predominantly syntactic, a provider-neutral IR can faithfully represent all four formats, and each provider adapter can be assembled from four orthogonal operations modules (content, message, tool, config) with effort independent of existing providers (section 3). 3. Bidirectional conversion with streaming support: request and response payloads are translated in both directions (provider↔IR↔provider), and chunk-level streaming is handled through ten typed event kinds with stateful context management (section 4). 4.An empirical evaluation demonstrating lossless round-trip fidelity, correct streaming behavior, and sub- millisecond conversion overhead, validated by 1,364 tests and the Open Responses compliance suite, and deployed in production at Argonne National Laboratory (section 5). LLM-Rosetta currently supports four API standards—OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Google Generative AI—which collectively cover the vast majority of commercial LLM providers, as most adopt one of these wire formats. The code is released under the MIT license athttps://github.com/ Oaklight/llm-rosetta. Paper organization. Section 2 surveys related work. Section 3 presents theIRdesign and converter architecture. Section 4 describes the implementation, including streaming and the gateway proxy. Section 5 evaluates round-trip fidelity, streaming correctness, and performance overhead. Section 6 discusses limitations and future directions. Section 7 concludes. 2 Related Work 2.1 LLM API Ecosystem The landscape of LLM APIs has evolved rapidly since the release of GPT-3 [Brown et al., 2020]. OpenAI’s Chat Completions API [OpenAI, 2024] established an early de facto standard based on role-tagged messages (system, user,assistant), but subsequent providers diverged. Anthropic’s Messages API [Anthropic, 2024a] introduced a separatesystemparameter and block-typed content arrays. Google’s Generative AI API [Google, 2024] adopted acontents/partsschema withuser/modelroles. OpenAI itself introduced the Responses API [OpenAI, 2025], replacing the chat message paradigm with an items-based model where tool calls, reasoning, and text output are sibling items rather than nested content parts. This fragmentation extends to tool calling [Schick et al., 2023, Patil et al., 2023, Qin et al., 2023] (function definitions and invocation formats), streaming (SSE event schemas), multi-modal content [Liu et al., 2024] (image, audio, file encoding), and generation controls (temperature, top-p, reasoning budgets [Wei et al., 2022]). Table 1 summarizes key differences across providers. 2.2 SDK Wrappers and Abstraction Layers Several projects attempt to unify LLM access at different levels of abstraction. Multi-provider frameworks. LangChain [LangChain, Inc., 2024] and Microsoft’s Semantic Kernel [Microsoft, 2024] are the two most widely adopted multi-provider LLM frameworks. Both provide high-level abstractions (chains, agents, planners) that internally dispatch to provider-specific SDK clients. Their multi-provider support operates at the application level: each provider integration is a separate adapter class that maps framework-level abstractions to native 2 LLM-RosettaA PREPRINT Table 1: API format divergence across major LLM providers. AspectOpenAI ChatAnthropicGoogle GenAIOpenAI Responses Message unit messages[] messages[] contents[] input[] items System promptrole in arraytop-level paramrole in array instructions Content modelstring or partsblock array parts[]items with type Tool calls tool_calls[]content block functionCallitem with type Streamingdelta chunksevent typescandidate deltasresponse events Reasoning— thinking thought reasoning item API calls. This design prioritizes developer ergonomics for building LLM applications but does not expose a reusable, format-level translation layer—cross-provider conversion of raw API payloads is not a supported use case. SDK-level proxies. LiteLLM [BerriAI, 2024] provides an OpenAI-compatible proxy that translates requests at the SDK level, mapping all providers into the OpenAI Chat Completions format. While widely adopted, this approach uses a single provider’s schema as the lingua franca, which loses provider-specific features (e.g., Anthropic’s cache control, Google’s grounding metadata) and cannot represent constructs that the target format lacks. AI Gateway [Portkey, 2024] and similar commercial proxies route requests to multiple providers but typically rely on the OpenAI Chat format as the canonical schema, inheriting the same representational limitations. Specification and protocol efforts. The OpenRouter [OpenRouter, 2024] service aggregates providers behind a unified API, and the Open Responses [OpenRouter, 2025] initiative proposes the OpenAI Responses API format as an open standard adopted by multiple inference providers. At a different layer, the Model Context Protocol (MCP) [Anthropic, 2024b] standardizes how LLM applications discover and invoke tools, but does not address the request/response format translation between providers that LLM-Rosetta targets. 2.3 Compiler Intermediate Representations The hub-and-spoke pattern is well established in compiler design. LLVM’s IR [Lattner and Adve, 2004] decouples source languages from target architectures, reducing adapter complexity fromO(M × N)toO(M + N). Apache Arrow [Apache Software Foundation, 2016] applies the same principle to columnar data interchange between analytics systems. Protocol Buffers [Google, 2008] and Apache Thrift [Slee et al., 2007] serve as language-neutral serialization IRs. LLM-Rosettaadapts this hub-and-spoke strategy to the LLM API domain, which is structurally simpler than compiler IR (dict-to-dict mapping rather than semantic-preserving program transformation) but faces the same combinatorial cost: a typedIRcaptures the semantic union of provider formats, while per-provider converters serve as frontends and backends. 2.4 Positioning of LLM-Rosetta Unlike application frameworks (LangChain, Semantic Kernel) that abstract over providers at the application level, LLM-Rosettaoperates at the API format level, enabling raw payload translation without requiring adoption of a specific application framework. Unlike SDK wrappers (LiteLLM) that privilege one provider’s format,LLM-Rosetta defines a neutralIRdesigned from the ground up for cross-provider translation. Unlike gateway proxies that focus on routing,LLM-Rosettaprovides bidirectional, field-level conversion with explicit metadata preservation for lossless round-trips. Unlike specification efforts (Open Responses),LLM-Rosettais a runtime translation engine that can bridge existing, incompatible APIs without requiring providers to change their formats. These distinctions do not imply that LLM-Rosettasupersedes existing tools. LiteLLM’s single-provider lingua franca is pragmatically effective for the common case of forwarding requests to diverse backends, and its ecosystem—100+ providers, built-in rate limiting, caching, observability, and an active community of over 15,000 GitHub stars—makes it the more practical choice for applications that do not require bidirectional or cross-provider translation. LangChain and Semantic Kernel provide application-level abstractions (chains, agents, planners) thatLLM-Rosettadoes not attempt to replicate.LLM-Rosetta is complementary: it can serve as the format-translation engine within such frameworks or gateways, handling the low-level payload conversion that they currently implement ad hoc. Table 2 summarizes the positioning. 3 LLM-RosettaA PREPRINT Table 2: Comparison ofLLM-Rosettawith related approaches. × = not supported; n/a = not applicable (specification, not implementation). FeatureLangChainLiteLLMAI Gw.Open R.LLM-Rosetta Provider-neutral IR×n/a✓ Bidir. format conv.×n/a✓ Lossless round-trip×n/a✓ Streaming support✓ App framework✓× Providers (≥50)✓× 3 Design This section presents the design ofLLM-Rosetta’s Intermediate Representation and converter architecture. We first state the design goals (section 3.1), then describe theIRschema (section 3.2), and finally introduce the Ops-composition pattern that structures each provider converter (section 3.3). 3.1 Design Goals 1. Semantic completeness. TheIRmust be expressive enough to represent any construct found in supported providers—messages, multi-modal content, tool definitions and invocations, reasoning traces, generation controls, and streaming events—without loss of application-relevant information. 2. Provider neutrality. No single provider’s format should be privileged. TheIRis designed from the union of all supported schemas, not as an extension of any one. 3.Bidirectional fidelity. Conversion must work in both directions (provider→IRandIR→provider) so that LLM-Rosettacan serve as both a request translator and a response translator. We define lossless round-trip as follows: letto A andfrom A denote the to-IR and from-IR converters for providerA, and let≡ s denote structural equality (identical JSON trees modulo key ordering and insignificant whitespace). A round-trip is lossless in preserve mode whenfrom A (to A (x)) ≡ s xfor every valid provider payloadx. In strip mode, provider-specific metadata fields (e.g.,cache_control,thought_signature) are intentionally discarded, so the property weakens to semantic equivalence: all application-relevant fields (message roles, content, tool definitions, generation parameters) are preserved, while provider-internal annotations may be dropped. 4. Incremental extensibility. Adding a new provider should require implementing only the provider-specific converter without modifying the IR schema or existing converters. 5.Streaming compatibility. The design must support chunk-level streaming translation, not just batch request/re- sponse conversion. 3.2 Intermediate Representation TheIRis defined as a set of typed data structures organized into eight modules: content parts, messages, tools, generation configuration, requests, responses, stream events, and extension types. Figure 1 provides an overview. 3.2.1 Content Parts Content parts are the atomic units of message content. The IR defines the following part types: • TextPart: Plain text content. • ImagePart: Image data (inline base64 or URL reference) with optional detail level. • AudioPart: Audio data with media type. • FilePart: Arbitrary file attachments. • ToolCallPart: A tool invocation with call ID, tool name, and JSON input. • ToolResultPart: The result of a tool invocation, linked by call ID. • ReasoningPart: Chain-of-thought or “thinking” content, with optional signature for caching. • RefusalPart: Model refusal with reason text. 4 LLM-RosettaA PREPRINT • CitationPart: URL or text citations attached to generated content. Each part carries atypediscriminator and an optionalprovider_metadatafield for round-trip preservation of provider-specific attributes. 3.2.2 Messages Messages are role-tagged containers of content parts: • SystemMessage: System-level instructions (role = system). • UserMessage: User input, including text and images (role = user). • AssistantMessage: Model output, including text, tool calls, and reasoning (role = assistant). • ToolMessage: Tool execution results (role = tool). Each message carries aMessageMetadatarecord with optional fields for message ID, timestamp, streaming state, and a custom dictionary for converter-specific round-trip data. 3.2.3 Tool Definitions A ToolDefinition specifies a callable tool with: • name: Unique identifier. • description: Natural-language description for the model. • parameters: JSON Schema object defining the input shape. • type: Tool category (function or mcp). ToolChoicecontrols tool selection behavior with modesnone,auto,any, andtool(force a specific tool). ToolCallConfig provides additional controls such as disabling parallel tool calls. 3.2.4 Generation Configuration GenerationConfig captures sampling and decoding parameters: temperature, top-p, top-k, max tokens, stop se- quences, frequency/presence penalties, logit biases, seed, and logprobs settings.ReasoningConfigcontrols chain- of-thought behavior (enabled, effort level, budget tokens).StreamConfigandResponseFormatConfighandle streaming and structured output settings. 3.2.5 Request and Response IRRequesthas two required fields—modelandmessages—and optional fields for system instruction, tools, tool choice, generation config, response format, streaming, reasoning, caching, and aprovider_extensionsbag for rare provider-specific parameters that do not warrant first-class IR fields. IRResponse contains an ID, timestamp, model identifier, a list ofChoiceInfo(each wrapping a message and finish reason), and optional usage statistics (UsageInfo with prompt, completion, reasoning, and cache token counts). 3.3 Ops-Composition Architecture Rather than implementing each converter as a monolithic class,LLM-Rosettafactors conversion logic into four orthogonal Ops modules. A monolithic converter intermingles content-level concerns (e.g., base64 image encoding) with request-level concerns (e.g., parameter mapping), causing cross-cutting logic like JSON Schema sanitization to be duplicated across all providers. An alternative flat-parameter approach—mapping all provider fields through a single unified body—conflates semantic differences (what a field means) with mechanical differences (how it is serialized), making the converter brittle when providers share structure but differ in semantics. The domain-factored Ops pattern isolates genuinely orthogonal concerns: 1. ContentOps: Converts individual content parts (text, images, tool calls, reasoning, citations) between provider format and IR. 2.MessageOps: Converts message sequences, handling role mapping, system prompt extraction, and multi-turn conversation structure. Delegates to ContentOps for part-level conversion. 5 LLM-RosettaA PREPRINT 3. ToolOps: Converts tool definitions and tool choice configurations. 4. ConfigOps: Converts generation parameters, reasoning settings, response format, and caching configuration. A base class defines the abstract interface for each Ops module. A concrete converter is assembled by specifying four Ops implementations: Listing 1: Ops-composition pattern. 1 class AnthropicConverter(BaseConverter): 2 content_ops_class = AnthropicContentOps 3 message_ops_class = AnthropicMessageOps 4 tool_ops_class = AnthropicToolOps 5 config_ops_class = AnthropicConfigOps This design provides three benefits: • Separation of concerns: Content-level quirks (e.g., Google’spartsnesting) are isolated from message-level concerns (e.g., Anthropic’s separate system parameter). • Reuse: Providers sharing a sub-format can reuse Ops modules. For example, OpenAI Responses reuses aspects of OpenAI Chat’s content ops. • Testability: Each Ops module can be unit-tested in isolation against known input/output pairs. Figure 2 illustrates the overall architecture, showing how the hub-and-spoke pattern connects four provider converters through the central IR. 4 Implementation LLM-Rosetta is implemented in Python (approximately 23,000 lines of library code, excluding vendored dependencies) and released under the MIT license. This section describes the key implementation aspects: type system (section 4.1), converter pipeline (section 4.2), streaming (section 4.3), provider auto-detection (section 4.4), and the gateway proxy (section 4.5). 4.1 Type System TheIRis implemented using Python’sTypedDictwith discriminated unions. Content parts use atypefield as the discriminator: Listing 2: Discriminated union for content parts (simplified). 1 class TextPart(TypedDict): 2 type: Literal["text"] 3 text: str 4 5 class ToolCallPart(TypedDict): 6 type: Literal["tool_call"] 7 tool_call_id: str 8 tool_name: str 9 tool_input: dict[str , Any] Role-specific content types constrain which parts may appear in each message role (e.g.,ToolCallPartonly in assistant messages), providing static type safety.Runtime validation functions (validate_ir_request, validate_ir_response) enforce structural invariants. 4.2 Converter Pipeline Each converter exposes six primary entry points: • request_to_provider(ir_request)→ provider request dict • request_from_provider(provider_request)→ IRRequest 6 LLM-RosettaA PREPRINT • response_to_provider(ir_response)→ provider response dict • response_from_provider(provider_response)→ IRResponse • stream_response_to_provider(ir_events)→ provider SSE chunks • stream_response_from_provider(chunks)→ IR stream events Two additional convenience methods (messages_to_provider,messages_from_provider) provide direct message- level conversion without full request wrapping. AConversionContextobject threads through the pipeline, accumulating warnings and carrying state between conversion stages. The context supports two metadata modes: •Strip mode (default): Provider-specific metadata is discarded during conversion, producing cleanIRoutput suitable for cross-provider forwarding. •Preserve mode: Provider-specific metadata is retained inprovider_metadatafields, enabling lossless round-trip conversion (A→IR→A). Internally, each entry point orchestrates the four Ops modules. For example,request_from_providerpro- ceeds in stages: (1) ConfigOps extracts generation parameters, (2) ToolOps extracts tool definitions, (3) Mes- sageOps (calling ContentOps per part) converts the conversation, and (4) provider-specific extensions are captured in provider_extensions. 4.3 Streaming Streaming translation is a key design challenge because providers use fundamentally different Server-Sent Events (SSE) [W3C, 2015] schemas. OpenAI Chat emitsdeltachunks within achoices[]array; Anthropic emits typed events (content_block_start,content_block_delta,content_block_stop); Google emitscandidates[] with accumulated parts; and OpenAI Responses emits item-level events. LLM-Rosetta normalizes these into ten IR stream event types: 1. stream_start: Session metadata (response ID, model, timestamp). 2. stream_end: End of stream. 3. content_block_start: Begin a new content block (text, tool call, reasoning). 4. content_block_end: Finish a content block. 5. text_delta: Incremental text fragment. 6. reasoning_delta: Incremental reasoning/thinking fragment. 7. tool_call_start: Begin a tool call (ID, name). 8. tool_call_delta: Incremental tool call arguments (JSON fragment). 9. finish: Generation complete, with finish reason. 10. usage: Token usage statistics. AStreamContextextendsConversionContextwith streaming-specific state: the current block index, a tool-call ID-to-name mapping, accumulated tool-call argument buffers, and deferred payloads for usage and finish events that arrive before the logical end of a content block. This stateful design ensures correct ordering of events even when providers report information out of sequence (e.g., Google emitting finish reason before the final content delta). 4.4 Provider Auto-Detection LLM-Rosettaincludes a heuristic auto-detection module that infers the source provider format from a request body’s structure. The detection examines field presence and types in priority order: 1. Google GenAI: Presence of contents with parts sub-structure. 2. OpenAI Responses: Presence of input or output with typed items. 3.Anthropic vs. OpenAI Chat: Both usemessages; differentiated by Anthropic’s separatesystemparameter, anthropic_version field, or block-typed content arrays. Auto-detection enables the gateway proxy to accept requests in any supported format without explicit provider specification. 7 LLM-RosettaA PREPRINT 4.5 Gateway Proxy TheLLM-Rosettagateway is an HTTP proxy built on Starlette [Encode, 2024] that performs live cross-provider translation. Given a request in format A destined for a provider expecting format B, the gateway: 1. Auto-detects (or receives as configuration) the source format. 2. Converts the request: A→ IR→ B. 3. Forwards to the upstream provider. 4. Converts the response: B→ IR→ A. 5. Returns the translated response to the client. For streaming requests, the gateway performs chunk-level SSE translation: each upstream SSE event is converted to an IRstream event, then re-serialized into the source provider’s SSE format, and forwarded to the client in real time. This ensures that streaming latency overhead is bounded by per-chunk conversion time rather than total response time. The gateway supports configurable provider endpoints and API keys, making it suitable for local development, testing, and production deployment behind a reverse proxy. 5 Evaluation We evaluateLLM-Rosettaalong four dimensions: round-trip fidelity (section 5.2), streaming correctness (section 5.3), cross-provider translation (section 5.4), and conversion performance overhead (section 5.5). 5.1 Evaluation Methodology We organize evaluation around the following research questions: • RQ1: Does round-trip conversion (A→IR→A) preserve all application-relevant fields? • RQ2: Does streaming translation maintain correct event ordering and content integrity? • RQ3: Does cross-provider translation (A→IR→B) preserve semantic content? • RQ4: What is the latency and throughput overhead of conversion? 5.2 Round-Trip Fidelity (RQ1) 5.2.1 Test Design We construct a corpus of representative request and response payloads covering: • Simple text conversations (single and multi-turn). • Multi-modal content (text + images, files). • Tool definitions, tool calls, and tool results. • Reasoning/thinking content with signatures. • Complex generation configurations (temperature, top-p, stop sequences, reasoning budgets). • Edge cases: empty content, refusals, citations, multiple choices. For each payload in provider format A, we perform the round-trip A→ IR→A and compare the output against the original using structural equality (ignoring field ordering and whitespace). 5.2.2 Results Table 3 summarizes the test coverage. The suite contains 987 converter-level unit tests across the four providers, plus 377 additional tests for IR types, base converter logic, auto-detection, and public API surface (1,364 total). In preserve mode, all round-trip tests achieve lossless field-level equality. In strip mode, provider-specific metadata (e.g., Anthropic’scache_control, Google’sthought_signature) is intentionally discarded, but all semantically meaningful fields are preserved. 8 LLM-RosettaA PREPRINT Table 3: Unit test counts per conversion category and provider. All tests pass in both strip and preserve metadata modes. CategoryOpenAI ChatAnthropicGoogleResponses Content parts22313343 Messages38232827 Tool defs/calls25283146 Config/params23264744 Full round-trip31415140 Streaming56705970 Total199229255304 5.2.3 Open Responses Compliance LLM-Rosettapasses all six tests in the official Open Responses compliance test suite [OpenRouter, 2025], covering non-streaming text generation, streaming, tool use, multi-turn conversation, image input, and structured output. 5.3 Streaming Correctness (RQ2) 5.3.1 Test Design We capture real streaming sessions from each provider (using recorded SSE traces) and verify that: • Every upstream event produces the correct sequence of IR events. • Event ordering is maintained (start before deltas, deltas before end). • Tool call arguments are correctly accumulated across delta events. • Usage and finish events are correctly positioned. • Re-serialization into the source format produces valid SSE. 5.3.2 Results The streaming test suite contains 255 test cases (56 OpenAI Chat, 70 Anthropic, 59 Google, 70 Responses; see table 3). All four provider converters pass with 100% event-sequence accuracy. TheStreamContextcorrectly handles provider-specific ordering differences: • Anthropic’s explicit block lifecycle events map directly to IR block events. • OpenAI Chat’s implicit block boundaries (inferred from delta field presence) are correctly detected. • Google’s accumulated-part model (where each chunk contains the full response so far) is correctly differenced to produce incremental deltas. • OpenAI Responses’ item-level events are correctly mapped to block-level IR events. 5.4 Cross-Provider Translation (RQ3) Round-trip fidelity (A→IR→A) is the easier case because both legs of the conversion share the same provider logic. Cross-provider translation (A→IR→B) is the more demanding scenario: it exercises both converters in tandem and exposes semantic gaps between formats. 5.4.1 Test Design We test cross-provider conversion across all six provider pairs (OpenAI Chat↔Anthropic, OpenAI Chat↔Google, OpenAI Chat↔Responses, Anthropic↔Google, Anthropic↔Responses, Google↔Responses) covering: • Simple text conversations (role mapping, content structure). • Multi-modal content (text + inline images). • Tool definitions and tool choice configuration. • Multi-turn conversations with mixed roles. • Bidirectional consistency: A→B→A should recover the semantic content of A. 9 LLM-RosettaA PREPRINT Table 4: Round-trip conversion overhead in microseconds (median over 1,000 iterations). “Req” denotes request round-trip (provider→IR→provider); “Resp” denotes response round-trip. PayloadOpenAI ChatAnthropicGoogleResponses Simple text (req)21242422 Multi-turn (req)71757773 Tool calls (req)44464644 Simple text (resp)30293131 Tool calls (resp)55474857 5.4.2 Results All 10 cross-provider conversion tests pass. Semantic content—message text, roles, tool names, tool parameters, and image data—is preserved across all provider pairs. The following provider-specific adaptations are correctly handled: • Role mapping: Google’s model role is correctly mapped to/from assistant in other providers. •System prompt location: Anthropic’s top-levelsystemparameter is correctly extracted from or merged into the message array used by other providers. • Content structure: OpenAI Chat’s string-or-array content, Anthropic’s block arrays, Google’spartsnesting, and Responses’ typed items are all interconvertible. •Tool definitions: Function schema and tool choice configurations translate correctly despite differing nesting structures. The primary limitation of cross-provider translation is the expected loss of provider-specific features that have no equivalent in the target format. For example, Anthropic’scache_controlannotations are not representable in Google’s format, and Google’sgrounding_metadatahas no Anthropic counterpart. These features are silently dropped during cross-provider conversion (with warnings recorded in theConversionContext), while all semantically shared fields are preserved. 5.5 Performance Overhead (RQ4) 5.5.1 Test Design We measure conversion latency using microbenchmarks on representative payloads of varying complexity (simple text, multi-turn with tools, multi-modal). Each benchmark performs 1,000 iterations of the full round-trip conversion (provider→IR→provider) using Python’stime.perf_counter_ns()for nanosecond-resolution timing. Benchmarks were run on a single core of an Intel Core Ultra 7 155H (32 GB RAM) with CPython 3.10 on Linux 6.19. 5.5.2 Results Table 4 reports round-trip (A→IR→A) conversion latency across five payload types. All conversions complete in under 80 μs at the median, with simple requests taking 21–24 μs and the most complex multi-turn payloads reaching 71–77 μs. At the 95th percentile (P95), latencies remain below 115 μs for all payload types, with P95 values typically 5–15% above the median. These overheads are negligible compared to network round-trip times (typically 50–500 ms) and model inference latency (100 ms–10 s+), representing less than 0.01% of end-to-end request latency in practice. Notably, conversion time scales with message count (multi-turn payloads are∼3×slower than simple text) but is largely uniform across providers, confirming that the Ops-composition architecture does not introduce provider-specific bottlenecks. 5.5.3 Comparison with LiteLLM To contextualizeLLM-Rosetta’s overhead, we benchmark LiteLLM’s one-directionaltransform_request(OpenAI Chat→Anthropic, v1.83) againstLLM-Rosetta’s two-hop cross-provider path (OpenAI Chat→IR→Anthropic) using the same payloads. As shown in table 5,LLM-Rosetta’s two-hop conversion is competitive with LiteLLM’s single-pass approach: for simple text payloads,LLM-Rosettais actually faster (24 vs. 28 μs), while multi-turn and tool-call payloads incur a 1.6–2.2×overhead. This modest gap reflects the additional work of constructing typed IR objects at the intermediate hop. Crucially, both tools operate in the sub-100 μs range, so the absolute difference is negligible relative to network 10 LLM-RosettaA PREPRINT Table 5: Cross-provider conversion latency: LiteLLM (one-directional, OpenAI Chat→Anthropic) vs.LLM-Rosetta (two-hop, OpenAI Chat→IR→Anthropic). Median over 1,000 iterations; LiteLLM v1.83. PayloadLiteLLM (μs)LLM-Rosetta (μs)Ratio Simple text28240.8× Multi-turn34762.2× Tool calls29461.6× round-trip times and model inference latency. The two-hop cost buys bidirectionality, provider neutrality, and lossless round-trip capability—features that LiteLLM does not support. 5.6 Threats to Validity Construct validity.Our fidelity evaluation relies on unit tests authored alongside the converters, which risks circular validation: tests may reflect the implementation’s assumptions rather than an independent specification of correct behavior. We mitigate this in two ways. First, test payloads are constructed from official provider documentation and real API responses, not generated from the converter code. Second,LLM-Rosettapasses all six tests in the independently maintained Open Responses compliance suite [OpenRouter, 2025], providing external validation of the core translation logic. Internal validity. The benchmark payloads (simple text, multi-turn, tool calls) are synthetic but representative. We additionally validated performance on anonymized production payloads from Argo-Proxy (64- and 218-message conversations with 41 tool definitions), confirming that the latency characteristics hold at production scale. All benchmarks use CPython 3.10 on a single hardware configuration; results may differ on other Python implementations or hardware. External validity.LLM-Rosettacurrently supports four API standards. While these cover the dominant LLM API providers by market share, our results do not guarantee that theIRdesign or the Ops-composition pattern will generalize equally well to all future providers. The cross-provider evaluation covers all six bidirectional provider pairs but with a limited number of test cases per pair (10 total); more extensive cross-provider testing would strengthen confidence in translation correctness. 6 Discussion 6.1 Semantic vs. Syntactic Translation LLM-Rosettaperforms semantic translation: it maps between provider formats based on the meaning of fields, not their surface syntax. For example, Anthropic’sthinkingcontent blocks and Google’sthoughtparts both represent chain-of-thought reasoning and are mapped to the sameIR ReasoningPart, despite having different JSON structures. This semantic approach enables cross-provider translation (A→IR→B) in addition to round-trip conversion. However, semantic translation inevitably involves judgment calls about equivalence. When provider formats diverge in expressiveness (e.g., one supportstop_kand another does not), theIRtakes the union of features, and converters for less expressive providers simply ignore unsupported fields with a warning. This design choice prioritizes completeness over strict compatibility. 6.2 Metadata Preservation and Round-Trip Fidelity The dual-mode metadata system (strip vs. preserve) provides flexibility for different use cases. Preserve mode enables lossless A→IR→A round-trips by storing provider-specific attributes inprovider_metadatafields. This is essential for testing and debugging converters, and for scenarios where a request must pass through theIRand return to its original format. Strip mode provides clean, provider-neutralIRoutput suitable for cross-provider forwarding. In this mode, provider- specific metadata is intentionally discarded, which means that A→IR→B→IR→A may not reproduce the original A exactly—but the semantic content is preserved. 11 LLM-RosettaA PREPRINT 6.3 Limitations Coverage.LLM-Rosettacurrently supports four API standards (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Google Generative AI), which cover the majority of commercial LLM providers—most emerging providers (Cohere, Mistral, xAI, DeepSeek, etc.) adopt one of these wire formats. Non-chat modalities (embeddings, fine-tuning, batch processing) are not yet covered. API evolution.LLM APIs evolve rapidly. When a provider adds new fields or changes semantics, the corresponding converter must be updated. The modular Ops design localizes these changes (e.g., a new content type only affects ContentOps), but ongoing maintenance is unavoidable. Semantic gaps. Some provider features have no equivalent in other formats. For example, Anthropic’s prompt caching with explicit cache breakpoints has no counterpart in Google’s API.LLM-Rosettacan preserve such features in provider_metadata for round-trips, but cross-provider translation necessarily drops them. Performance at scale. The translation layer itself adds sub-100 μs overhead per conversion at the median (see section 5.5), which is negligible compared to network and inference latency. This makes the converter library suitable for direct integration into production applications—for example, embedded in an API gateway, an orchestration framework, or a multi-provider SDK. The reference gateway included inLLM-Rosettais one such deployment example; it introduces an additional network hop and serialization cycle, which may matter in latency-sensitive settings. In such cases, applications can invoke the translation layer in-process to avoid the extra hop entirely. 6.4 Deployment Experience LLM-Rosettaserves as the translation layer for ARGO-PROXY [Ding, 2024], an LLM API gateway deployed at Argonne National Laboratory. Argo-Proxy provides researchers on the Argonne network with unified access to multiple LLM providers (OpenAI, Anthropic, Google) through a single OpenAI-compatible endpoint. Its previous architecture (v2.x) relied on hand-written format-mapping code that only covered the OpenAI Chat Completions format, and could not keep pace with the evolving demands of Anthropic’s Messages API and the newer OpenAI Responses API. The third-generation architecture (v3.0, currently in beta after 13 pre-release iterations) replaced this bespoke translation layer withLLM-Rosetta’s converter library, eliminating approximately 2,000 lines of ad hoc mapping code while gaining support for all four API standards and bidirectional streaming. The integration exercisesLLM-Rosetta’s core capabilities in a production setting: request translation (OpenAI Chat → IR→target provider), response translation (provider→ IR→OpenAI Chat), and streaming event normalization across all supported providers. The 13-iteration beta cycle (v3.0.0b1–b13) surfaced several edge cases—notably, inconsistent streaming event ordering across providers and corner cases in tool-call argument accumulation—that led to improvements in the converter library itself. This feedback loop between a production deployment and the library’s test suite provides a form of real-world validation beyond unit tests alone. 6.5 Future Directions Provider coverage. Since most emerging LLM providers adopt one of the four supported API standards (most commonly OpenAI Chat Completions), they can already be served by the existing converters. For providers with minor deviations from a standard format, we plan to support configurable provider adaptors that map provider-specific endpoints, authentication, and field variations onto an existing converter, reducing per-provider effort to configuration rather than code. Conformance testing.LLM-Rosettacurrently passes the Open Responses compliance suite (section 5.2), but no analogous third-party suite exists for the other three provider formats. Developing a comprehensive, independently maintained conformance test corpus—ideally derived from real API traffic—would strengthen validation and help track correctness as both LLM-Rosetta and provider APIs evolve. Schema evolution. As the LLM ecosystem matures, new content types (video, structured data), interaction patterns (multi-agent, agentic workflows), and capabilities (real-time voice, computer use) will requireIRextensions. The provider_extensionsmechanism provides an escape hatch, but frequently used extensions should be promoted to first-class IR fields. 12 LLM-RosettaA PREPRINT 7 Conclusion The central finding of this work is that despite substantial surface-level divergence, the four major LLM API providers share a common semantic core—role-tagged messages, typed content parts, tool definitions with JSON Schema inputs, and incremental streaming events—that can be captured by a compact, provider-neutral IR (9 content-part types, 10 stream event types). The practical challenge is not deep semantic incompatibility but the combinatorial surface of syntactic variations: each provider makes different choices about field naming, nesting depth, content encoding, role vocabulary, and streaming granularity, and these differences multiply across feature dimensions (content, tools, config, streaming). A hub-and-spoke IR is effective precisely because the divergence is syntactic: a shared semantic core makes faithful translation feasible, while the combinatorial cost of pairwise adaptation makes an intermediate representation worthwhile. LLM-Rosettademonstrates this through 1,364 passing tests—including the Open Responses compliance suite—lossless round-trip fidelity in preserve mode, and sub-millisecond conversion overhead. The Ops-composition architecture confines provider-specific complexity to well-bounded modules, and the library’s deployment in Argo-Proxy at Argonne National Laboratory validates its production readiness. The primary open challenge is coverage breadth: the four supported API standards cover most commercial providers, but non-chat modalities (embeddings, fine-tuning, batch processing) are not yet addressed. As the ecosystem contin- ues to grow, we expect the IR’s union-based design to accommodate new providers with incremental effort, while theprovider_extensionsmechanism provides a pragmatic escape hatch for features that resist standardization. LLM-Rosetta is available at https://github.com/Oaklight/llm-rosetta. Acknowledgments Large language models were used to assist with proofreading and language editing. The author takes full responsibility for all content. References Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. OpenAI. OpenAI Chat Completions API, 2024. URLhttps://platform.openai.com/docs/api-reference/ chat. Accessed: 2026-04-10. Anthropic. Anthropic Messages API, 2024a. URLhttps://docs.anthropic.com/en/api/messages. Accessed: 2026-04-10. Google. Google Gemini API, 2024. URL https://ai.google.dev/api. Accessed: 2026-04-10. OpenAI.OpenAI Responses API, 2025.URLhttps://platform.openai.com/docs/api-reference/ responses. Accessed: 2026-04-10. Chris Lattner and Vikram Adve. LLVM: A compilation framework for lifelong program analysis & transformation. In International Symposium on Code Generation and Optimization (CGO), pages 75–86. IEEE, 2004. Apache Software Foundation. Apache Arrow: A cross-language development platform for in-memory analytics, 2016. URL https://arrow.apache.org. Accessed: 2026-04-10. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2023. Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. Gorilla: Large language model connected with massive apis. arXiv preprint arXiv:2305.15334, 2023. 13 LLM-RosettaA PREPRINT Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. ToolLLM: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 36, 2024. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022. LangChain, Inc. LangChain: Build context-aware reasoning applications, 2024. URLhttps://github.com/ langchain-ai/langchain. Accessed: 2026-04-10. Microsoft. Semantic Kernel: Integrate cutting-edge llm technology quickly and easily into your apps, 2024. URL https://github.com/microsoft/semantic-kernel. Accessed: 2026-04-10. BerriAI. LiteLLM: Call all llm apis using the openai format, 2024. URLhttps://github.com/BerriAI/litellm. Accessed: 2026-04-10. Portkey. AI Gateway: A fast ai gateway with integrated guardrails, 2024. URLhttps://github.com/Portkey-AI/ gateway. Accessed: 2026-04-10. OpenRouter. OpenRouter: A Unified Interface for LLMs, 2024. URLhttps://openrouter.ai. Accessed: 2026-04- 10. OpenRouter. Open Responses: An Open Standard for LLM APIs, 2025. URLhttps://openresponses.com. Accessed: 2026-04-10. Anthropic. Model Context Protocol (MCP), 2024b. URLhttps://modelcontextprotocol.io. Accessed: 2026- 04-10. Google. Protocol Buffers: Google’s data interchange format, 2008. URLhttps://protobuf.dev. Accessed: 2026-04-10. Mark Slee, Aditya Agarwal, and Marc Kwiatkowski. Thrift: Scalable cross-language services implementation. Facebook White Paper, 2007. W3C. Server-Sent Events, 2015. URLhttps://html.spec.whatwg.org/multipage/server-sent-events. html. W3C Living Standard. Accessed: 2026-04-10. Encode. Starlette: The little asgi framework that shines, 2024. URLhttps://w.starlette.io. Accessed: 2026-04-10. Peng Ding. Argo-Proxy: An llm api gateway for argonne national laboratory, 2024. URLhttps://github.com/ Oaklight/argo-proxy. Accessed: 2026-04-10. 14 LLM-RosettaA PREPRINT containsconfigures composed of IRStreamEvent (10 Event Types) Terminal FinishEvent finish_reason UsageEvent usage: UsageInfo Deltas TextDelta text: str ReasoningDelta reasoning: str ToolCallStart tool_call_id, tool_name ToolCallDelta arguments_delta: str Lifecycle StreamStart response_id, model StreamEnd ContentBlockStart block_index, block_type ContentBlockEnd block_index IRResponse Response Components ChoiceInfo index: int message: Message finish_reason: FinishReason FinishReason reason: stop | length | tool_calls | content_filter | refusal | error | cancelled UsageInfo prompt_tokens completion_tokens reasoning_tokens total_tokens id: str object: response created: int model: str choices: list[ChoiceInfo] usage: UsageInfo Content Parts Special ReasoningPart reasoning, signature RefusalPart refusal: str CitationPart url_citation | text_citation Tool-Related ToolCallPart tool_call_id, tool_name tool_input: dict ToolResultPart tool_call_id result, is_error Basic TextPart text: str ImagePart image_url | image_data AudioPart audio_data | url FilePart file_url | file_data Message Types SystemMessage role = system content: TextPart[] UserMessage role = user content: UserContentPart[] AssistantMessage role = assistant content: AssistantContentPart[] ToolMessage role = tool content: ToolResultPart[] Configuration GenerationConfig temperature, top_p, top_k max_tokens, stop_sequences seed, n StreamConfig enabled include_usage ReasoningConfig enabled, effort budget_tokens ToolDefinition type, name description, parameters IRRequest model: str messages: list[Message] system_instruction tools tool_choice generation stream reasoning provider_extensions Figure 1: Overview of the IR schema. Arrows indicate containment relationships. 15 LLM-RosettaA PREPRINT Gateway Proxy request_from_provider() request_to_provider() to/from IRto/from IRto/from IRto/from IR Google GenAI Ops Modules ContentOps MessageOps ToolOps ConfigOps GoogleConverter Anthropic Ops Modules ContentOps MessageOps ToolOps ConfigOps AnthropicConverter OpenAI Responses Ops Modules ContentOps MessageOps ToolOps ConfigOps OpenAIResponsesConverter OpenAI Chat Completions Ops Modules ContentOps MessageOps ToolOps ConfigOps OpenAIChatConverter Source Format (Any Provider) Target Format (Any Provider) IR (Intermediate Representation) Figure 2: Hub-and-spoke architecture ofLLM-Rosetta. Each provider converter translates bidirectionally between its native format and the IR. Cross-provider translation composes two converters through the IR hub. 16

Instant research discovery